Nodeconfiguration:

Two guest nodes configured as follows via libvirt and qemu/kvm. See referenced files.

Prerequesits

Server with SLES10 SP2 and provided ocfs2. Install all at least ocfs2-tools.

Install from the SLES10 comoonics channel (see Channels Faq)

RPMS:

• ocfs2-tools-debuginfo-1.3.9-0.1.el5
• ocfs2-tools-1.3.9-0.1.el5
• ocfs2-tools-devel-1.3.9-0.1.el5
• ocfs2console-1.3.9-0.1.el5
• comoonics-pythonosfix-py-0.1-1
• comoonics-cluster-py-0.1-16
• comoonics-bootimage-extras-ocfs2-0.1-1
• comoonics-bootimage-listfiles-sles-0.1-1
• comoonics-bootimage-listfiles-sles10-0.1-1
• comoonics-cs-py-0.1-56
• comoonics-cdsl-py-0.2-11
• comoonics-bootimage-listfiles-all-0.1-1
• comoonics-bootimage-initscripts-1.3-1.sles10
• comoonics-bootimage-1.3-37

Open Issues:

see the parent bug

Clusterconfig:

Create the directory for the cluster.conf to reside in.:

    osrsles10node1:~ # mkdir /etc/cluster

Redhat Cluster.conf

      <cluster config_version="1" name="osrsles10" type="gfs">
         <clusternodes>
            <clusternode name="osrsles10node1" nodeid="1">
              <com_info>
                      <syslog name="syslog-server"/>
                      <rootvolume name="/dev/hdb" fstype="ocfs2"/>
                      <eth name="eth0" ip="192.168.10.121" mask="255.255.255.0" mac="00:16:3E:12:EC:44"/>
                      <fenceackserver user="root" passwd="test123"/>
              </com_info>
            </clusternode>
            <clusternode name="osrsles10node2" nodeid="2">
              <com_info>
                      <syslog name="syslog server"/>
                      <rootvolume name="/dev/hdb" fstype="ocfs2"/>
                      <eth name="eth0" ip="192.168.10.121" mask="255.255.255.0" mac="00:16:3E:12:EC:45"/>
                      <fenceackserver user="root" passwd="test123"/>
              </com_info>
            </clusternode>
         </clusternodes>
      </cluster>

OCFS2 Cluster.conf

The OCFS2 Cluster.conf will be automatically generated by the initrd from /etc/cluster/cluster.conf. To validate use com-queryclusterconf convert ocfs2 and you'll see the output. It should be somehow like as follows.

If so redirect the output to /etc/ocfs2/cluster.conf in order to setup the filesystem.

          node:
                  ip_port = 7777
                  ip_address = 192.168.10.121
                  number = 1
                  name = osrsles10node1
                  cluster = osrsles10
          node:
                  ip_port = 7777
                  ip_address = 192.168.10.122
                  number = 2
                  name = osrsles10node2
                  cluster = osrsles10
          cluster:
                  node_count = 2
                  name = osrsles10

Start the ocfs2 cluster :

   osrsles10node1:~ # /etc/init.d/o2cb configure
   Configuring the O2CB driver.

   This will configure the on-boot properties of the O2CB driver.
   The following questions will determine whether the driver is loaded on
   boot.  The current values will be shown in brackets ('[]').  Hitting
   <ENTER> without typing an answer will keep that current value.  Ctrl-C
   will abort.

   Load O2CB driver on boot (y/n) [n]: n
   Cluster to start on boot (Enter "none" to clear) [ocfs2]: osrsles10
   Specify heartbeat dead threshold (>=7) [31]: 
   Specify network idle timeout in ms (>=5000) [30000]: 
   Specify network keepalive delay in ms (>=1000) [2000]: 
   Specify network reconnect delay in ms (>=2000) [2000]: 
   Use user-space driven heartbeat? (y/n) [n]: 
   Writing O2CB configuration: OK
   osrsles10node1:~ # /etc/init.d/o2cb online osrsles10
   Loading module "configfs": OK
   Mounting configfs filesystem at /sys/kernel/config: OK
   Loading module "ocfs2_nodemanager": OK
   Loading module "ocfs2_dlm": OK
   Loading module "ocfs2_dlmfs": OK
   Creating directory '/dlm': OK
   Mounting ocfs2_dlmfs filesystem at /dlm: OK
   Starting O2CB cluster osrsles10: OK

Create the shared root filesystem

The shared root filesystem must be a logical volume on a shared storage device. Use mkfs.ocfs2 to create the OCFS2 filesystem:

    mkfs.ocfs2 -Losr /dev/hdd

Mount the new filesystem to '/mnt/newroot':

    mount /dev/hdd /mnt/newroot/  

Copy all data from the local installed RHEL5 root filesystem to the shared root filesystem:

    cp -ax / /mnt/newroot/

Create some directories:

    mkdir /mnt/newroot/proc
    mkdir /mnt/newroot/sys

Create a new cdsl infrastructure on the shared root filesystem:

    chroot /mnt/newroot com-mkcdslinfrastructure

Mount the local cdsl infrastructure:

    mount --bind /mnt/newroot/cluster/cdsl/1/ /mnt/newroot/cdsl.local/

Make /var hostdependent:

    chroot /mnt/newroot com-mkcdsl -a /var 

Make /var/lib shared again:

    chroot /mnt/newroot com-mkcdsl -s /var/lib

Make /etc/HOSTNAME hostdependent:

    chroot /mnt/newroot com-mkcdsl -a /etc/HOSTNAME

Create /etc/mtab link to '/proc/mounts':

    cd /mnt/newroot/etc/
    rm -f mtab
    ln -s /proc/mounts mtab
    cd -

Remove cluster network configuration:

    rm -f /mnt/newroot/etc/sysconfig/network/ifcfg-eth*

Disable kudzu:

    chroot /mnt/newroot/ chkconfig o2cb off
    chroot /mnt/newroot/ chkconfig ocfs2 off

Modify '/mnt/newroot/etc/fstab':

    /dev/hdb             /                    ocfs2      defaults        0 0
    #/dev/disk/by-id/ata-QEMU_HARDDISK_QM00001-part1 swap                 swap       defaults              0 0
    proc                 /proc                proc       defaults              0 0
    sysfs                /sys                 sysfs      noauto                0 0
    debugfs              /sys/kernel/debug    debugfs    noauto                0 0
    usbfs                /proc/bus/usb        usbfs      noauto                0 0
    devpts               /dev/pts             devpts     mode=0620,gid=5       0 0

Create shared bootdevice

Mount shared boot:

    mount /dev/hdc1 /mnt/newroot/boot/

Copy boot:

    cp -a /boot/* /mnt/newroot/boot

create '/mnt/newroot/boot/grub/menu.lst':

     # Modified by YaST2. Last modification on Wed Aug  6 11:27:07 UTC 2008
     default 0
     timeout 8
     ##YaST - generic_mbr
     gfxmenu (hd0,0)/boot/message
     ##YaST - activate
     ###Don't change this comment - YaST2 identifier: Original name: linux###
     title SUSE Linux Enterprise Server 10 SP2
         root (hd0,0)
         kernel /boot/vmlinuz-2.6.16.60-0.21-default com-debug
         initrd /boot/initrd-2.6.16.60-0.21-default
     ###Don't change this comment - YaST2 identifier: Original name: failsafe###
     title Failsafe -- SUSE Linux Enterprise Server 10 SP2
         root (hd0,0)
         kernel /boot/vmlinuz-2.6.16.60-0.21-default com-debug ide=nodma apm=off acpi=off noresume edd=off 3
         initrd /boot/initrd-2.6.16.60-0.21-default

Invoke grub shell:

    grub

Install grub:

         GNU GRUB  version 0.97  (640K lower / 3072K upper memory)
      [ Minimal BASH-like line editing is supported.  For the first word, TAB
        lists possible command completions.  Anywhere else TAB lists the possible
        completions of a device/filename. ]
     grub> device (hd0) /dev/hdc
     grub> root (hd0,0)
      Filesystem type is ext2fs, partition type 0x83
     grub> setup (hd0)
      Checking if "/boot/grub/stage1" exists... yes
      Checking if "/boot/grub/stage2" exists... yes
      Checking if "/boot/grub/e2fs_stage1_5" exists... yes
      Running "embed /boot/grub/e2fs_stage1_5 (hd0)"...  15 sectors are embedded.
     succeeded
      Running "install /boot/grub/stage1 (hd0) (hd0)1+15 p (hd0,0)/boot/grub/stage2 /boot/grub/menu.lst"... succeeded
     Done.

Create Shared Root initrd

Create the shared root initrd into the shared boot filesystem:

    /opt/atix/comoonics-bootimage/mkinitrd -f /mnt/newroot/boot/initrd-$(uname -r) $(uname -r)

Boot Cluster

Have Fun !!