|
|
| (67 intermediate revisions by 2 users not shown) |
| Line 1: |
Line 1: |
| Cortex is a computation rack for VisLab humanoid robots. It contains 7 machines: | | Cortex is a server used by VisLab for running simulations (not the ones with the iCub robot). |
| * 1 server that manages startup, shutdown and the file system of the clients;
| |
| * 6 clients (named <code>cortex1</code>...<code>cortex6</code>) that run user processes.
| |
| All clients numbered 1 to 5 mount the same file system. Therefore, performing changes in the file system of cortex[1-5] will reflect to all other four clients.
| |
| The client <code>cortex6</code> is separate for now, because it runs a 64 bit Linux.
| |
|
| |
|
| ''An older version of this article can be found at [[Cortex/Archive]].'' | | ''Old information can be consulted at [[Cortex/Archive]].'' |
|
| |
|
| = Network setup = | | = Specifications = |
|
| |
|
| == Connectivity ==
| | As of 2017, there is one machine (cortex1) with these specs: |
| | | * 8 x [http://ark.intel.com/products/65523/Intel-Core-i7-3770K-Processor-(8M-Cache-up-to-3_90-GHz) i7-3770K] @ 3.50GHz processor |
| Cortex machines are connected to Cortex Switch, that links to VisLab switch with a fiber optic connection of 4Gbit/s.
| | * 16GB of memory (<code>sudo dmidecode --type 17</code> to see RAM speed and type) |
| | | * 112GB SSD drive + 1TB HDD drive |
| == Cortex nodes ==
| | * NVidia [http://www.geforce.com/hardware/desktop-gpus/geforce-gtx-670 GeForce GTX 670] graphics card (CUDA) |
| | | * internal ISR IP address: 10.10.1.1 |
| Cortex server and clients have the following IPs and domain names:
| |
| * Server: 10.10.1.240, server.visnet
| |
| * Client 1: 10.10.1.1, cortex1.visnet
| |
| * Client 2: 10.10.1.2, cortex2.visnet
| |
| * Client 3: 10.10.1.3, cortex3.visnet | |
| * Client 4: 10.10.1.4, cortex4.visnet
| |
| * Client 5: 10.10.1.5, cortex5.visnet
| |
| * Client 6: 10.10.1.6, cortex6.visnet
| |
| | |
| ''For further details, see the [[Vislab#Network]] and the [[VisLab network]] articles.''
| |
| | |
| = Additional setup =
| |
| | |
| == Boot procedure ==
| |
| | |
| The clients boot via the network, using the PXE system. Each machine determines its own identity and asks the server for a kernel image and an initial ram disk. Kernel images and initial ram disks are stored on the server in the <code>/tftpboot/</code> directory. Kernels can be stock kernels, but the initial ram disk must be created in a way that enables booting from the network. This is not as bad as it sounds. It involves invoking the command <code>mkinitramfs</code>.
| |
| | |
| The server decides which kernel and initramdisk to send to each machine based on the information stored in the two files: <code>/tftpboot/pxelinux.cfg/default</code>(for cortexes 1-5) and <code>/tftpboot/pxelinux.cfg/01-00-19-d1-9e-e9-53</code> (for cortex6).
| |
| | |
| The roots and the homes of the users are also stored on the server machine, so they are mounted by each client at boot time.
| |
| | |
| === Mounting of root directory ===
| |
| | |
| We are not sure which mechanism mounts the root filesystem, exactly. Here is the relevant line from <code>/etc/fstab</code>:
| |
| # <file system> <mount point> <type> <options> <dump> <pass>
| |
| /dev/nfs / nfs defaults 1 1
| |
| | |
| === Mounting of home directory ===
| |
| | |
| The home directory is mounted via the Upstart system a few seconds after booting. When rebooting the system, it is possible to login while <code>/home</code> is still not mounted. In that case, logout and login in order to load your environment variables correctly.
| |
| | |
| In November 2010, we created a file called <code>/etc/init/mountHome-net.conf</code> containing:
| |
| description "Mount network filesystems"
| |
| start on started networking or runlevel 2
| |
| exec /usr/local/bin/mountHome.sh
| |
| | |
| and <code>/usr/local/bin/mountHome.sh</code> containing:
| |
| MOUNTED=$(mount | grep home)
| |
| while [ -z "$MOUNTED" ]
| |
| do
| |
| su icub -c 'mount /home' &> /var/tmp/mountHomeUpstartOut.txt
| |
| MOUNTED=$(mount | grep home)
| |
| echo $MOUNTED
| |
| sleep 1
| |
| done
| |
| ls /home > /var/tmp/mountHomeUpstartLs.txt
| |
| runlevel > /var/tmp/mountHomeUpstartRunlevel.txt
| |
| | |
| After a successful mount, we should see something like
| |
| $ mount | grep home
| |
| 10.10.1.240:/nfsroot.home on /home type nfs (rw,user=icub,noac,addr=10.10.1.240)
| |
| | |
| If <code>/home</code> is wrongly mounted with the <code>noexec</code> flag, users won't be able to execute binaries located inside it.
| |
| | |
| == Server machine ==
| |
| | |
| The server has:
| |
| * a boot folder for the clients at <code>/tftpboot/pxelinux.cfg</code>. It contains the files:
| |
| ** <code>default</code> - default boot file;
| |
| ** <mac_address> - specific for a machine with the given mac address.
| |
| * startup scripts for each machine at <code>/nfsroot/app</code>
| |
| | |
| == Client machines ==
| |
| | |
| The clients have:
| |
| * A superuser account (<code>compurack</code>) to administer system-wide settings (configurations, libraries, etc.)
| |
| * Normal user accounts. By default, the login script runs the contents of file <code>$HOME/.bash_env</code>, where users can set their environment variables, e.g., <code>export ICUB_ROOT=$HOME/iCub</code>. This works for both interactive shell sessions and non-interactive ones (i.e., commands remotely invoked by <code>yarprun</code>). | |
| * A <code>yarp</code> account to update and install the YARP library. Variable <code>YARP_ROOT</code> is set by default to <code>/home/yarp/yarp2</code> for all users (in <code>/etc/bash.bashrc</code>) <-- change this policy
| |
| * An <code>icub</code> account with sudo privileges (created with <code>sudo adduser icub admin</code> on 2009-06-30) <-- change this policy
| |
| | |
| == System-wide libraries and repositories ==
| |
| | |
| === YARP ===
| |
| | |
| yarp2 is installed for user icub, similarly to [[iCubBrain server configuration]].
| |
| | |
| For now, we don't use system-wide installation (<code>sudo make install</code>). We could use it again after we verify that a user can easily override global settings.
| |
| | |
| === iCub ===
| |
| | |
| iCub is installed for user icub, similarly to [[iCubBrain server configuration]].
| |
| | |
| For now, we don't use system-wide installation (<code>sudo make install</code>). We could use it again after we verify that a user can easily override global settings.
| |
| | |
| === Other libraries, manually installed ===
| |
| | |
| Please list here the system-wide libraries and applications that were installed by the superuser, especially the ones that do not have a clean 'make install' procedure but were manually installed into <code>/opt</code>:
| |
| | |
| * ARToolKit | |
| * Ipopt-3.5.5-linux-x86_32-gcc4.2.4 | |
| | |
| CMake 2.6 does not come with the version of Ubuntu currently installed, but it is needed by the latest version of yarp, so we installed it via this [http://www.cmake.org/files/v2.6/cmake-2.6.4-Linux-i386.sh archive].
| |
| * cmake 2.6
| |
| | |
| === Other libraries, installed with Ubuntu packages ===
| |
| | |
| These packages were installed with <code>apt-get install</code>
| |
| libncurses5-dev
| |
| libace-dev
| |
| libgsl0-dev
| |
| libgtk2.0-dev libgtkmm-2.4-dev libglademm-2.4-dev
| |
| glew-utils libglew1.5-dev
| |
| libglut-dev
| |
| | |
| OpenCV:
| |
| libcv-dev libhighgui-dev libcvaux-dev
| |
| | |
| == User repositories ==
| |
| | |
| Each user should manage his own yarp2 and iCub repositories. <-- ''then we shouldn't have done sudo make install here :)''
| |
| | |
| We recommend to set your environment variables in a new file, called <code>~/.bash_env</code> and containing:
| |
| export YARP_ROOT=~/yarp2
| |
| export YARP_DIR=$YARP_ROOT/build
| |
| export ICUB_ROOT=~/iCub
| |
| export ICUB_DIR=$ICUB_ROOT/main/build
| |
| export PATH=$PATH:$YARP_DIR/bin:$ICUB_DIR/bin
| |
| | |
| Refer to the [[RobotCub software]] article for further instructions.
| |
| | |
| = Other configuration =
| |
| | |
| == Subversion ==
| |
| | |
| We have set the following parameter in <code>/etc/subversion/config</code>:
| |
| store-passwords = no
| |
| This implies that SVN will ask you for your password every time you do a commit. (Don't worry about changing your personal <code>~/.subversion/config</code> file: the parameter is not actually set there, so the global <code>/etc</code> setting is used.)
| |
| | |
| == Network tuning ==
| |
| | |
| sysctl -w net.core.rmem_max=8388608
| |
| sysctl -w net.core.wmem_max=8388608
| |
| sysctl -w net.core.rmem_default=65536
| |
| sysctl -w net.core.wmem_default=65536
| |
| sysctl -w net.ipv4.tcp_rmem='4096 87380 8388608'
| |
| sysctl -w net.ipv4.tcp_wmem='4096 65536 8388608'
| |
| sysctl -w net.ipv4.tcp_mem='8388608 8388608 8388608'
| |
| sysctl -w net.ipv4.route.flush=1
| |
| | |
| == Prompt ($PS1) ==
| |
| | |
| The prompt is set to <code>user@cortex?:pwd$</code> in <code>/etc/bash.bashrc</code>. With those settings, if you log in to Cortex1, the prompt will be <code>user@cortex1:~$</code>.
| |
| We chose to do so because sometimes it's convenient to have the number of the Cortex machine you're working on embedded in the prompt.
| |
| By default, though, this configuration is overridden in the users' <code>~/.bashrc</code> file, and the prompt is set to <code>user@source</code> regardless of the Cortex machine you log in to.<br>
| |
| If you want to inhibit this behaviour in <code>~/.bashrc</code> and thus have a prompt like <code>user@cortex?:pwd</code>, just comment these lines in your <code>~/.bashrc</code>:
| |
| # set a fancy prompt (non-color, unless we know we "want" color)
| |
| case "$TERM" in
| |
| xterm-color)
| |
| PS1='${debian_chroot:+($debian_chroot)}\[\033[01;32m\]\u@\h\[\033[00m\]:\[\033[01;34m\]\w\[\033[00m\]\$ '
| |
| ;;
| |
| *)
| |
| PS1='${debian_chroot:+($debian_chroot)}\u@\h:\w\$ '
| |
| ;;
| |
| esac
| |
| However, for users created after 2009-05-07, the prompt is already set to <code>user@cortex?:pwd$</code> by default.
| |
| | |
| = Helper commands =
| |
| | |
| * Check the kernel: <code>uname -m</code>
| |
| * Check the file versions: <code>file</code>
| |
| * Set bash shell in <code>/etc/passwd</code>
| |
| * Check disk space: <code>du –sh /home</code>
| |
| * Check per-user processes: <code>ps -U <user></code>
| |
|
| |
|
| [[Category:Vislab]] | | [[Category:Vislab]] |