ISRWiki - User contributions [en]

Affordance imitation

2009-11-10T11:30:49Z

Lmontesano: /* QueryCollector */

= Modules =

This is the general architecture currently under development

[[Image:arch.jpg]] (updated 28/07/09)

== BlobSegmentation ==

Implemented by edisonSegmentation module in the iCub repository.
Module that takes a raw RGB image as input and provides a segmented (labeled) image at the output, indicating possible objects or object parts present in the scene.

Ports:
* /conf
* /rawimg:i
* /rawimg:o
* /labelimg:o
* /viewimg:o

== BlobDescriptor ==

Implemented by blobDescriptor module in the iCub repository.
Module that receives a labeled image and the corresponding raw image and creates descriptors for each one of the identified objects.

Ports:
* /conf
* /rawimag:i
* /labelimg:i
* /rawimg:o
* /viewimg:o
* /affdescriptor:o
* /trackerinit:o

== EffectDetector ==

Implemented by effectDetector module in the iCub repository.<br />
Algorithm:
1.wait for initialization signal and parameters on /init
2.read the raw image that was used for the segmentation on /rawsegmimg:i
3.read the current image on /rawcurrimg:i
4.check if the the ROI specified as an initialization parameter is similar in the two images
5.if (similarity<threshold)
6. answer 0 on /init and go back to 1.
7.else
8. answer 1 on /init
9. while(not received another signal on /init)
10. estimate the position of the tracked object
11. write the estimate on /effect:o
12. read a new image on /rawcurrimg:i
13. end
14.end

Ports:
* /init //receives a bottle with (u, v, w, h, h1, h2, ..., hn, vmin, vmax, smin), answers 0 or 1.
* /rawsegmimg:i //raw image that was used for the segmentation
* /rawcurrimg:i
* /effect:o //flow of (u,v) positions of the tracked object
* /error:o //signals if the module is working correctly (0) or if there is some problem (1..)

== QueryCollector ==

Implemented by demoAff module in the iCub repository.
Module that receives inputs from the object descriptor module and the effect descriptor. When activated by the behavior controller, info about the objects is used to select actions that will be executed by the robot. In the absence of interaction activity, it notifies it to the behavior controller that may decide to switch to another behavior.

Ports:

* /demoAffv2/effect
* /demoAffv2/synccamshift
* /demoAffv2/objsdesc
* /demoAffv2/
* /demoAffv2/motioncmd
* /demoAffv2/gazecmd
* /demoAffv2/behavior:i
* /demoAffv2/behavior:o
* /demoAffv2/out
* /demoAffv2/out

= Ports and communication =

The interface between modules is under development. The current version (subject to changes as we refine it) is as follows:

* Behavior to AttentionSelection -> vocabs "on" / "off"
* Behavior to Query -> vocabs "on" / "off". We should add some kind of context to the on command (imitation or learning being the very basic).
* Gaze Control -> Behavior: read the current head state/position
* Query to Behavior -> "end" / "q"
* Query to Effect Detector. The main objective of this port is to start the tracker at the object of interest. We need to send at least:
** position (x,y) within the image. 2 doubles.
** size (h,w). 2 doubles.
** color histogram. TBD.
** saturation parameters (max min). 2 int.
** intensity (max min). 2 int.

* Effect Detector to Query
* Camshiftplus format

* blobDescriptor -> query
** Affordance descriptor. Same format as camshiftplus
** tracker init data. histogram (could be different from affordance) + saturation + intensity

* query -> object segmentation
** vocab message: "do seg"

* object segmentation -> blob descriptor
** labelled image
** raw image

Affordance imitation

2009-11-10T11:24:47Z

Lmontesano: /* Modules */

= Modules =

This is the general architecture currently under development

[[Image:arch.jpg]] (updated 28/07/09)

== BlobSegmentation ==

Implemented by edisonSegmentation module in the iCub repository.
Module that takes a raw RGB image as input and provides a segmented (labeled) image at the output, indicating possible objects or object parts present in the scene.

Ports:
* /conf
* /rawimg:i
* /rawimg:o
* /labelimg:o
* /viewimg:o

== BlobDescriptor ==

Implemented by blobDescriptor module in the iCub repository.
Module that receives a labeled image and the corresponding raw image and creates descriptors for each one of the identified objects.

Ports:
* /conf
* /rawimag:i
* /labelimg:i
* /rawimg:o
* /viewimg:o
* /affdescriptor:o
* /trackerinit:o

== EffectDetector ==

Implemented by effectDetector module in the iCub repository.<br />
Algorithm:
1.wait for initialization signal and parameters on /init
2.read the raw image that was used for the segmentation on /rawsegmimg:i
3.read the current image on /rawcurrimg:i
4.check if the the ROI specified as an initialization parameter is similar in the two images
5.if (similarity<threshold)
6. answer 0 on /init and go back to 1.
7.else
8. answer 1 on /init
9. while(not received another signal on /init)
10. estimate the position of the tracked object
11. write the estimate on /effect:o
12. read a new image on /rawcurrimg:i
13. end
14.end

Ports:
* /init //receives a bottle with (u, v, w, h, h1, h2, ..., hn, vmin, vmax, smin), answers 0 or 1.
* /rawsegmimg:i //raw image that was used for the segmentation
* /rawcurrimg:i
* /effect:o //flow of (u,v) positions of the tracked object
* /error:o //signals if the module is working correctly (0) or if there is some problem (1..)

== QueryCollector ==

Implemented by demoAff module in the iCub repository.
Module that receives inputs from the object descriptor module and the effect descriptor. When activated by the behavior controller, info about the objects is used to select actions that will be executed by the robot. In the absence of interaction activity, it notifies it to the behavior controller that may decide to switch to another behavior.

Ports:
* /affdescriptor:i
* /trackerinit:o

= Ports and communication =

The interface between modules is under development. The current version (subject to changes as we refine it) is as follows:

* Behavior to AttentionSelection -> vocabs "on" / "off"
* Behavior to Query -> vocabs "on" / "off". We should add some kind of context to the on command (imitation or learning being the very basic).
* Gaze Control -> Behavior: read the current head state/position
* Query to Behavior -> "end" / "q"
* Query to Effect Detector. The main objective of this port is to start the tracker at the object of interest. We need to send at least:
** position (x,y) within the image. 2 doubles.
** size (h,w). 2 doubles.
** color histogram. TBD.
** saturation parameters (max min). 2 int.
** intensity (max min). 2 int.

* Effect Detector to Query
* Camshiftplus format

* blobDescriptor -> query
** Affordance descriptor. Same format as camshiftplus
** tracker init data. histogram (could be different from affordance) + saturation + intensity

* query -> object segmentation
** vocab message: "do seg"

* object segmentation -> blob descriptor
** labelled image
** raw image

Affordance imitation

2009-07-29T13:40:09Z

Lmontesano:

= Modules =

This is the general architecture currently under development

[[Image:arch.jpg]] (updated 28/07/09)

= Ports and communication =

The interface between modules is under development. The current version (subject to changes as we refine it) is as follows:

* Behavior to AttentionSelection -> vocabs "on" / "off"
* Behavior to Query -> vocabs "on" / "off"

We should add some kind of context to the on command (imitation or learning being the very basic).

* Gaze Control -> Behavior: read the current head state/position
* Query to Behavior -> "end" / "q"

* Query to Effect Detector. The main objective of this port is to start the tracker at the object of interest. We need to send at least:
** position (x,y) within the image. 2 doubles.
** size (h,w). 2 doubles.
** color histogram. TBD.
** saturation parameters (max min). 2 int.
** intenity (max min). 2 int.

* Effect Detector to Query
* Camshiftplus format

* blobDescriptor -> query
** Affordance descriptor. Same format as camshiftplus
** tracker init data. histogram (could be different from affordance) + saturation + intensity

* query -> object segmentation
** vocab message: "do seg"

* object segmentation -> blob descriptor
** labelled image
** raw image

Affordance imitation

2009-07-29T13:33:59Z

Lmontesano:

= Modules =

This is the general architecture currently under development

[[Image:arch.jpg]] (updated 28/07/09)

= Ports and communication =

The interface between modules is under development. The current version (subject to changes as we refine it) is as follows:

* Behavior to AttentionSelection -> vocabs "on" / "off"
* Behavior to Query -> vocabs "on" / "off"

We should add some kind of context to the on command (imitation or learning being the very basic).

* Query to Behavior -> "end" / "q"

* Query to Effect Detector. The main objective of this port is to start the tracker at the object of interest. We need to send at least:
** position (x,y) within the image. 2 doubles.
** size (h,w). 2 doubles.
** color histogram. TBD.
** saturation parameters (max min). 2 int.
** intenity (max min). 2 int.

* blobDescriptor -> query
** Affordance descriptor. Same format as camshiftplus
** tracker init data. histogram (could be different from affordance) + saturation + intensity

* query -> object segmentation
** vocab message: "do seg"

* object segmentation -> blob descriptor
** labelled image
** raw image

Affordance imitation

2009-07-29T13:26:37Z

Lmontesano:

= Modules =

This is the general architecture currently under development

[[Image:arch.jpg]] (updated 28/07/09)

= Ports and communication =

The interface between modules is under development. The current version (subject to changes as we refine it) is as follows:

* Behavior to AttentionSelection -> vocabs "on" / "off"
* Behavior to Query -> vocabs "on" / "off"

We should add some kind of context to the on command (imitation or learning being the very basic).

* Query to Behavior -> "end" / "q"

* Query to Effect Detector. The main objective of this port is to start the tracker at the object of interest. We need to send at least:

** position (x,y) within the image
** size (h,w)
** color histogrambs
** saturation parameters

*

ok &= port_behavior_out.open("/demoAffv2/behavior:o");

ok &= port_eff.open("/demoAffv2/effect");
ok &= port_sync.open("/demoAffv2/synccamshift");
ok &= port_descriptor.open("/demoAffv2/objsdesc");
ok &= port_askobj.open("/demoAffv2/");
ok &= port_primitives.open("/demoAffv2/motioncmd");
ok &= port_gaze.open("/demoAffv2/gazecmd");
ok &= port_output.open("/demoAffv2/out");
ok &= port_emotions.open("/demoAffv2/out");

File:Arch.jpg

2009-07-29T13:22:24Z

Lmontesano:

Affordance imitation

2009-07-29T13:20:32Z

Lmontesano: New page: = Modules = This is the general architecture currently under development Image:arch.jpg (updated 28/07/09) = Ports and communication = The interface between modules is under devel...

= Modules =

This is the general architecture currently under development

[[Image:arch.jpg]] (updated 28/07/09)

= Ports and communication =

The interface between modules is under development. The current version (subject to changes as we refine it) is as follows:

* Behavior to Query -> vocabs "on" / "off"

We should add some kind of context to the on command (imitation or learning being the very basic).

* Query to Behavior -> "end" / "q"

* Query to Effect Detector

This message starts the tracker at the object of interest. We need to send at least:

** position (x,y) within the image
** size (h,w)
** color histogram
** saturation parameters

ok &= port_behavior_out.open("/demoAffv2/behavior:o");

ok &= port_eff.open("/demoAffv2/effect");
ok &= port_sync.open("/demoAffv2/synccamshift");
ok &= port_descriptor.open("/demoAffv2/objsdesc");
ok &= port_askobj.open("/demoAffv2/");
ok &= port_primitives.open("/demoAffv2/motioncmd");
ok &= port_gaze.open("/demoAffv2/gazecmd");
ok &= port_output.open("/demoAffv2/out");
ok &= port_emotions.open("/demoAffv2/out");

Main Page

2009-07-29T13:08:20Z

Lmontesano: /* Research */

__NOTOC__

<big>'''Welcome to the MediaWiki space of the [http://www.isr.ist.utl.pt Institute for Systems and Robotics].'''</big>

Consult [[Help:Contents|these wiki help pages]] for usage information.

Feel free to edit the following list with your pages hosted here.

== Opportunities ==

=== Current ===

=== Past ===

* [[BII 2009]]
* [[Dissertation/Project proposals]]
* [[BII 2008]]

== Research ==

* [[Vislab | Computer and Robot Vision Lab (VisLab)]]
** demos: [[Chico]], [[Cluster Management in VisLab]], [[Innovation Days 2009]] [[Affordance imitation]]
** hardware: [[Chico3 laptop machine configuration]], [[Cortex]], [[iCubBrain server configuration]], [[pc104]]
* [[Intelligent Systems Lab]]
* [[Lisbon-K Chromosome Dataset]]
* [[International conferences deadlines]]
* [[Journals]]

== Teaching ==

* [[Robotics lab resources]]
* [[Autonomous Systems resources]]

== Events ==

=== Upcoming ===

* [http://vislab.isr.ist.utl.pt/RLSS09 RLSS09: Robot Learning Summer School]

=== Past ===

* [[Ocupação Científica de Jovens nas Férias 2009]]
* [[Lisbon Workshop on New Challenges for Cooperative Robotics]]

== Computing ==

* [[ISR computing resources]]
* [[Macintosh resources]]
* [[Python resources]]

== Misc ==

* [[Administrative checklist]] (internal use only) -- '''check here for all rules concerning purchases, conference trips, paper credits, etc.'''
* [[List of restaurants near IST]]
* [http://wiki.isr.ist.utl.pt Former ISR wiki pages]
* Use this [[SandBox]] for experimentation with MediaWiki