Giovanni Saponaro: cleanup

2013-06-12T10:34:12Z

cleanup

← Older revision		Revision as of 10:34, 12 June 2013
Line 1:		Line 1:
	~~[[Cortex \|~~ ''~~Click here to go back to the main Cortex article'']].~~		''Obsolete page, please delete me.''

	~~This~~ page ~~explains the setup of this cluster in detail.~~

	~~== Network setup ==~~

	~~=== Connectivity ===~~

	~~Cortex machines are connected to Cortex Switch, that links to VisLab switch with a fiber optic connection of 4Gbit/s.~~

	~~=== Cortex nodes ===~~

	~~Cortex server and clients have the following IPs and domain names:~~
	* Server: 10.10.1.240, server.visnet
	* Client 1: 10.10.1.1, cortex1.visnet
	* Client 2: 10.10.1.2, cortex2.visnet
	* Client 3: 10.10.1.3, cortex3.visnet
	* Client 4: 10.10.1.4, cortex4.visnet
	* Client 5: 10.10.1.5, cortex5.visnet
	* Client 6: 10.10.1.6, cortex6.visnet

	~~''For further details~~, ~~see [[VisLab network]]~~.''

	~~== Boot procedure ==~~

	The clients boot via the network, using the PXE system. Each machine determines its own identity and asks the server for a kernel image and an initial ram disk. Kernel images and initial ram disks are stored on the server in the <code>/tftpboot/</code> directory. Kernels can be stock kernels, but the initial ram disk must be created in a way that enables booting from the network. This is not as bad as it sounds. It involves invoking the command <code>mkinitramfs</code>.

	The server decides which kernel and initramdisk to send to each machine based on the information stored in the two files: <code>/tftpboot/pxelinux.cfg/default</code>(for cortexes 1-5) and <code>/tftpboot/pxelinux.cfg/01-00-19-d1-9e-e9-53</code> (for cortex6).

	~~The roots and the homes of the users are also stored on the server machine, so they are mounted by each client at boot time.~~

	~~=== Mounting of root directory ===~~

	~~We are not sure which mechanism mounts the root filesystem, exactly. Here is the relevant line from <code>/etc/fstab</code>:~~
	~~# <file system> <mount point> <type> <options> <dump> <pass>~~
	~~/dev/nfs / nfs defaults 1 1~~

	~~=== Mounting of home directory ===~~

	The home directory is mounted via the Upstart system a few seconds after booting. When rebooting the system, it is possible to login while <code>/home</code> is still not mounted. In that case, log out and log in again, in order for your environment variables to be set correctly.

	~~In November 2010, we created a file called <code>/etc/init/mountHome-net.conf</code> containing:~~
	~~description "Mount network filesystems"~~
	~~start on started networking or runlevel 2~~
	~~exec /usr/local/bin/mountHome.sh~~

	~~and <code>/usr/local/bin/mountHome.sh</code> containing:~~
	~~MOUNTED=$(mount \| grep home)~~
	~~while [ -z "$MOUNTED" ]~~
	do
	~~su icub -c 'mount /home' &> /var/tmp/mountHomeUpstartOut.txt~~
	~~MOUNTED=$(mount \| grep home)~~
	~~echo $MOUNTED~~
	~~sleep 1~~
	~~done~~
	~~ls /home > /var/tmp/mountHomeUpstartLs.txt~~
	~~runlevel > /var/tmp/mountHomeUpstartRunlevel.txt~~

	~~After a successful mount, we should see something like~~
	~~$ mount \| grep home~~
	~~10.10.1.240:/nfsroot.home on /home type nfs (rw,user=icub,addr=10.10.1.240)~~

	~~If <code>/home</code> is wrongly mounted with the <code>noexec</code> flag, users won't be able to execute binaries located inside it.~~

	~~== Server machine ==~~

	~~The server has:~~
	* a boot folder for the clients at <code>/tftpboot/pxelinux.cfg</code>. It contains the files:
	** <code>default</code> - default boot file;
	** <mac_address> - specific for a machine with the given mac address.
	* startup scripts for each machine at <code>/nfsroot/app</code>

	~~== Client machines ==~~

	~~The clients have:~~
	* A superuser account (<code>compurack</code>) to administer system-wide settings (configurations, libraries, etc.)
	* Normal user accounts. By default, the login script runs the contents of file <code>$HOME/.bash_env</code>, where users can set their environment variables, e.g., <code>export ICUB_ROOT=$HOME/iCub</code>. This works for both interactive shell sessions and non-interactive ones (i.e., commands remotely invoked by <code>yarprun</code>).
	* A <code>yarp</code> account to update and install the YARP library. Variable <code>YARP_ROOT</code> is set by default to <code>/home/yarp/yarp2</code> for all users (in <code>/etc/bash.bashrc</code>) <-- change this policy
	* An <code>icub</code> account with sudo privileges (created with <code>sudo adduser icub admin</code> on 2009-06-30) <-- change this policy
	* cortex6's <code>/etc/hosts</code> file can include the following line:
	~~127.0.0.1 cortex6~~

Giovanni Saponaro: initial version

2011-10-11T13:28:39Z

initial version

New page

[[Cortex | ''Click here to go back to the main Cortex article'']].

This page explains the setup of this cluster in detail.

== Network setup ==

=== Connectivity ===

Cortex machines are connected to Cortex Switch, that links to VisLab switch with a fiber optic connection of 4Gbit/s.

=== Cortex nodes ===

Cortex server and clients have the following IPs and domain names:
* Server: 10.10.1.240, server.visnet
* Client 1: 10.10.1.1, cortex1.visnet
* Client 2: 10.10.1.2, cortex2.visnet
* Client 3: 10.10.1.3, cortex3.visnet
* Client 4: 10.10.1.4, cortex4.visnet
* Client 5: 10.10.1.5, cortex5.visnet
* Client 6: 10.10.1.6, cortex6.visnet

''For further details, see [[VisLab network]].''

== Boot procedure ==

The clients boot via the network, using the PXE system. Each machine determines its own identity and asks the server for a kernel image and an initial ram disk. Kernel images and initial ram disks are stored on the server in the <code>/tftpboot/</code> directory. Kernels can be stock kernels, but the initial ram disk must be created in a way that enables booting from the network. This is not as bad as it sounds. It involves invoking the command <code>mkinitramfs</code>.

The server decides which kernel and initramdisk to send to each machine based on the information stored in the two files: <code>/tftpboot/pxelinux.cfg/default</code>(for cortexes 1-5) and <code>/tftpboot/pxelinux.cfg/01-00-19-d1-9e-e9-53</code> (for cortex6).

The roots and the homes of the users are also stored on the server machine, so they are mounted by each client at boot time.

=== Mounting of root directory ===

We are not sure which mechanism mounts the root filesystem, exactly. Here is the relevant line from <code>/etc/fstab</code>:
# <file system> <mount point> <type> <options> <dump> <pass>
/dev/nfs / nfs defaults 1 1

=== Mounting of home directory ===

The home directory is mounted via the Upstart system a few seconds after booting. When rebooting the system, it is possible to login while <code>/home</code> is still not mounted. In that case, log out and log in again, in order for your environment variables to be set correctly.

In November 2010, we created a file called <code>/etc/init/mountHome-net.conf</code> containing:
description "Mount network filesystems"
start on started networking or runlevel 2
exec /usr/local/bin/mountHome.sh

and <code>/usr/local/bin/mountHome.sh</code> containing:
MOUNTED=$(mount | grep home)
while [ -z "$MOUNTED" ]
do
su icub -c 'mount /home' &> /var/tmp/mountHomeUpstartOut.txt
MOUNTED=$(mount | grep home)
echo $MOUNTED
sleep 1
done
ls /home > /var/tmp/mountHomeUpstartLs.txt
runlevel > /var/tmp/mountHomeUpstartRunlevel.txt

After a successful mount, we should see something like
$ mount | grep home
10.10.1.240:/nfsroot.home on /home type nfs (rw,user=icub,addr=10.10.1.240)

If <code>/home</code> is wrongly mounted with the <code>noexec</code> flag, users won't be able to execute binaries located inside it.

== Server machine ==

The server has:
* a boot folder for the clients at <code>/tftpboot/pxelinux.cfg</code>. It contains the files:
** <code>default</code> - default boot file;
** <mac_address> - specific for a machine with the given mac address.
* startup scripts for each machine at <code>/nfsroot/app</code>

== Client machines ==

The clients have:
* A superuser account (<code>compurack</code>) to administer system-wide settings (configurations, libraries, etc.)
* Normal user accounts. By default, the login script runs the contents of file <code>$HOME/.bash_env</code>, where users can set their environment variables, e.g., <code>export ICUB_ROOT=$HOME/iCub</code>. This works for both interactive shell sessions and non-interactive ones (i.e., commands remotely invoked by <code>yarprun</code>).
* A <code>yarp</code> account to update and install the YARP library. Variable <code>YARP_ROOT</code> is set by default to <code>/home/yarp/yarp2</code> for all users (in <code>/etc/bash.bashrc</code>) <-- change this policy
* An <code>icub</code> account with sudo privileges (created with <code>sudo adduser icub admin</code> on 2009-06-30) <-- change this policy
* cortex6's <code>/etc/hosts</code> file can include the following line:
127.0.0.1 cortex6

Cortex setup - Revision history

Giovanni Saponaro: cleanup

Giovanni Saponaro: initial version