Cortex

Cortex is a computation rack for VisLab humanoid robots. It contains 7 machines:

• 1 server that manages startup, shutdown and the file system of the clients;
• 6 clients (named cortex1...cortex6) that run user processes.

All clients numbered 1 to 5 mount the same file system. Therefore, performing changes in the file system of cortex[1-5] will reflect to all other four clients. The client cortex6 is separate for now, because it runs a 64 bit Linux.

An older version of this article can be found at Cortex/Archive.

Network setup

Connectivity

Cortex machines are connected to Cortex Switch, that links to VisLab switch with a fiber optic connection of 4Gbit/s.

Cortex nodes

Cortex server and clients have the following IPs and domain names:

• Server: 10.10.1.240, server.visnet
• Client 1: 10.10.1.1, cortex1.visnet
• Client 2: 10.10.1.2, cortex2.visnet
• Client 3: 10.10.1.3, cortex3.visnet
• Client 4: 10.10.1.4, cortex4.visnet
• Client 5: 10.10.1.5, cortex5.visnet
• Client 6: 10.10.1.6, cortex6.visnet

For further details, see the Vislab#Network and the VisLab network articles.

Additional setup

Boot procedure

This section is a stub, I'm working on it, Matteo.

The clients boot via the network, using the PXE system. Each machine determines its own identity and asks the server for a kernel image and an initial ram disk. Kernel images and initial ram disks are stored on the server in the /tftpboot/ directory. Kernels can be stock kernels, but the initial ram disk must be created in a way that enables booting from the network. This is not as bad as it sounds. It involves invoking the command mkinitramfs, I'll explain this better as soon as I can.

The server decides which kernel and initramdisk to send to each machine based on the information stored in the two files: /tftpboot/pxelinux.cfg/default (for cortexes 1-5) and /tftpboot/pxelinux.cfg/01-00-19-d1-9e-e9-53 (for cortex6).

The roots and the homes of the users are also stored on the server machine, so they are mounted by each client at boot time. The roots are mounted automatically, the homes are not, I guess. Maybe the homes are mounted by the user root and are not visible for the other users? I am not sure what is happening. Should homes not be mounted when you restart the one of the clients, you only need to issue the command: mount /home.

Server machine

The server has:

• a boot folder for the clients at /tftpboot/pxelinux.cfg. It contains the files:
  • default - default boot file;
  • <mac_address> - specific for a machine with the given mac address.
• startup scripts for each machine at /nfsroot/app

Client machines

The clients have:

• A superuser account (compurack) to administer system-wide settings (configurations, libraries, etc.)
• Normal user accounts. By default, the login script runs the contents of file $HOME/.bash_env, where users can set their environment variables, e.g., export ICUB_ROOT=$HOME/iCub. This works for both interactive shell sessions and non-interactive ones (i.e., commands remotely invoked by yarprun).
• A yarp account to update and install the YARP library. Variable YARP_ROOT is set by default to /home/yarp/yarp2 for all users (in /etc/bash.bashrc) <-- change this policy
• An icub account with sudo privileges (created with sudo adduser icub admin on 2009-06-30) <-- change this policy

System-wide libraries and repositories

YARP

Presently (November 2010), the yarp2 SVN repository is installed under user yarp (with sudo make install), last updated on 13-July-2010. <-- change this policy?

iCub

Presently (November 2010), the iCub SVN repository is installed under user icub (with sudo make install), last updated on 13-July-2010. <-- change this policy?

There was a conflict with iKin, which could not find libipopt.so.0, but it is now fixed thanks to setting the environment variable

 LD_LIBRARY_PATH=/opt/Ipopt-3.5.5-linux-x86_32-gcc4.2.4/lib/

into /home/icub/.bash_env.

One module has been disabled in the CMakeList.txt file, because it was not compiling properly: crawling.

Other libraries, manually installed

Please list here the system-wide libraries and applications that were installed by the superuser, especially the ones that do not have a clean 'make install' procedure but were manually installed into /opt:

• ARToolKit
• Ipopt-3.5.5-linux-x86_32-gcc4.2.4

CMake 2.6 does not come with the version of Ubuntu currently installed, but it is needed by the latest version of yarp, so we installed it via this archive.

• cmake 2.6

Other libraries, installed with Ubuntu packages

These packages were installed with apt-get install

 libncurses5-dev
 libace-dev
 libgsl0-dev
 libgtk2.0-dev libgtkmm-2.4-dev libglademm-2.4-dev
 glew-utils libglew1.5-dev
 libglut-dev

OpenCV:

  THE REPOSITORY IS NOW IN SVN FORM, WE NEED TO UPDATE THIS.
  HINT: use the Ubuntu OpenCV 1 package, see iCubBrain page
  cvs -z3 -d:pserver:anonymous@opencvlibrary.cvs.sourceforge.net:/cvsroot/opencvlibrary co -P opencv
  cd opencv
  ./configure
  make
  make install
  add /usr/local/lib to /etc/ld.so.conf

User repositories

Each user should manage his own yarp2 and iCub repositories. <-- then we shouldn't have done sudo make install here :)

We recommend to set your environment variables in a new file, called ~/.bash_env and containing:

 export YARP_ROOT=~/yarp2
 export YARP_DIR=$YARP_ROOT/build
 export ICUB_ROOT=~/iCub
 export ICUB_DIR=$ICUB_ROOT/main/build
 export PATH=$PATH:$YARP_DIR/bin:$ICUB_DIR/bin

Refer to the RobotCub software article for further instructions.

Other configuration

Subversion

We have set the following parameter in /etc/subversion/config:

 store-passwords = no

This implies that SVN will ask you for your password every time you do a commit. (Don't worry about changing your personal ~/.subversion/config file: the parameter is not actually set there, so the global /etc setting is used.)

Network tuning

  sysctl -w net.core.rmem_max=8388608
  sysctl -w net.core.wmem_max=8388608
  sysctl -w net.core.rmem_default=65536
  sysctl -w net.core.wmem_default=65536
  sysctl -w net.ipv4.tcp_rmem='4096 87380 8388608'
  sysctl -w net.ipv4.tcp_wmem='4096 65536 8388608'
  sysctl -w net.ipv4.tcp_mem='8388608 8388608 8388608'
  sysctl -w net.ipv4.route.flush=1

Prompt ($PS1)

The prompt is set to user@cortex?:pwd$ in /etc/bash.bashrc. With those settings, if you log in to Cortex1, the prompt will be user@cortex1:~$. We chose to do so because sometimes it's convenient to have the number of the Cortex machine you're working on embedded in the prompt. By default, though, this configuration is overridden in the users' ~/.bashrc file, and the prompt is set to user@source regardless of the Cortex machine you log in to.
If you want to inhibit this behaviour in ~/.bashrc and thus have a prompt like user@cortex?:pwd, just comment these lines in your ~/.bashrc:

  # set a fancy prompt (non-color, unless we know we "want" color)
  case "$TERM" in
  xterm-color)
      PS1='${debian_chroot:+($debian_chroot)}\[\033[01;32m\]\u@\h\[\033[00m\]:\[\033[01;34m\]\w\[\033[00m\]\$ '
      ;;
  *)
      PS1='${debian_chroot:+($debian_chroot)}\u@\h:\w\$ '
      ;;
  esac

However, for users created after 2009-05-07, the prompt is already set to user@cortex?:pwd$ by default.

Helper commands

• Check the kernel: uname -m
• Check the file versions: file
• Set bash shell in /etc/passwd
• Check disk space: du –sh /home
• Check per-user processes: ps -U <user>