Affordance imitation: Difference between revisions
Jump to navigation
Jump to search
No edit summary |
m (changed some formatting) |
||
| Line 4: | Line 4: | ||
[[Image:arch.jpg]] (updated 28/07/09) | [[Image:arch.jpg]] (updated 28/07/09) | ||
== BlobSegmentation == | == BlobSegmentation == | ||
| Line 11: | Line 10: | ||
Module that takes a raw RGB image as input and provides a segmented (labeled) image at the output, indicating possible objects or object parts present in the scene. | Module that takes a raw RGB image as input and provides a segmented (labeled) image at the output, indicating possible objects or object parts present in the scene. | ||
Ports: | |||
* /conf | * /conf | ||
* /rawimg:i | * /rawimg:i | ||
| Line 23: | Line 22: | ||
Module that receives a labeled image and the corresponding raw image and creates descriptors for each one of the identified objects. | Module that receives a labeled image and the corresponding raw image and creates descriptors for each one of the identified objects. | ||
Ports: | |||
* /conf | * /conf | ||
* /rawimag:i | * /rawimag:i | ||
| Line 37: | Line 36: | ||
* Behavior to AttentionSelection -> vocabs "on" / "off" | * Behavior to AttentionSelection -> vocabs "on" / "off" | ||
* Behavior to Query -> vocabs "on" / "off" | * Behavior to Query -> vocabs "on" / "off". We should add some kind of context to the on command (imitation or learning being the very basic). | ||
We should add some kind of context to the on command (imitation or learning being the very basic). | |||
* Gaze Control -> Behavior: read the current head state/position | * Gaze Control -> Behavior: read the current head state/position | ||
* Query to Behavior -> "end" / "q" | * Query to Behavior -> "end" / "q" | ||
* Query to Effect Detector. The main objective of this port is to start the tracker at the object of interest. We need to send at least: | * Query to Effect Detector. The main objective of this port is to start the tracker at the object of interest. We need to send at least: | ||
** position (x,y) within the image. 2 doubles. | ** position (x,y) within the image. 2 doubles. | ||
| Line 49: | Line 44: | ||
** color histogram. TBD. | ** color histogram. TBD. | ||
** saturation parameters (max min). 2 int. | ** saturation parameters (max min). 2 int. | ||
** | ** intensity (max min). 2 int. | ||
* Effect Detector to Query | * Effect Detector to Query | ||
Revision as of 15:20, 10 August 2009
Modules
This is the general architecture currently under development
(updated 28/07/09)
BlobSegmentation
Implemented by edisonSegmentation module in the iCub repository. Module that takes a raw RGB image as input and provides a segmented (labeled) image at the output, indicating possible objects or object parts present in the scene.
Ports:
- /conf
- /rawimg:i
- /rawimg:o
- /labelimg:o
- /viewimg:o
BlobDescriptor
Implemented by blobDescriptor module in the iCub repository. Module that receives a labeled image and the corresponding raw image and creates descriptors for each one of the identified objects.
Ports:
- /conf
- /rawimag:i
- /labelimg:i
- /rawimg:o
- /viewimg:o
- /affdescriptor:o
- /trackerinit:o
Ports and communication
The interface between modules is under development. The current version (subject to changes as we refine it) is as follows:
- Behavior to AttentionSelection -> vocabs "on" / "off"
- Behavior to Query -> vocabs "on" / "off". We should add some kind of context to the on command (imitation or learning being the very basic).
- Gaze Control -> Behavior: read the current head state/position
- Query to Behavior -> "end" / "q"
- Query to Effect Detector. The main objective of this port is to start the tracker at the object of interest. We need to send at least:
- position (x,y) within the image. 2 doubles.
- size (h,w). 2 doubles.
- color histogram. TBD.
- saturation parameters (max min). 2 int.
- intensity (max min). 2 int.
- Effect Detector to Query
- Camshiftplus format
- blobDescriptor -> query
- Affordance descriptor. Same format as camshiftplus
- tracker init data. histogram (could be different from affordance) + saturation + intensity
- query -> object segmentation
- vocab message: "do seg"
- object segmentation -> blob descriptor
- labelled image
- raw image