CentOS vs. RHEL

The community Linux distribution CentOS is based on Red Hat Enterprise Linux (RHEL) and aims to be 100% binary compatible with the upstream product. You may download and use it free of charge however you will not get any commercial support from Red Hat.

That makes this distribution ideal to try out the Diskless Shared Root Cluster but if you want to run enterprise applications you most certainly want to use a certified and fully supported platform.

Many manufactures (like Oracle or SAP) even specify their software product to run on a specific versions of selected Linux distribution. RHEL is by far the most successful commercial Linux distribution and best fit for such enterprise applications.

Also note that CentOS does not provide recompiled Cluster Suite RPM packages for all architectures and often lacks the errata packages that are needed for an up to date system.

Recommended Cluster Configuration Method

Red Hat Enterprise Linux comes along with various tools that assist the administrator in the setup of a cluster. Most known tool is the system-config-cluster frontend but with RHEL 5 a new tool was introduced to create whole clusters: conga (consisting of luci and ricci). While those tools suffice to create small cluster setups, it is not recommended that these tools are used to create shared root clusters. The main reason for this recommendation is that there is a special section named "com_info" in which various shared root cluster settings are stored. This section is not recognized by the graphical tools. It may be ignored but there is also the danger that all settings are lost or tainted.

Recommended Partitioning

It makes sense that you predefine volumes of different sizes and RAID levels that lateron you only need to present to the cluster nodes. Also don't use all disk space at once because it is always good to know that there is still some emergency space left. Most storage systems support virtual RAID settings and they will manage the disks for you. You should create separate RAID1 and RAID5 disk groups with dedicated spare disks.

Recommended GFS Tweaks

There are lots of variables to tweak a GFS filesystem. However usually the developers set reasonable defaults that provide high performance. You should mount your GFS volumes with noatime and nodiratime, though. The atime parameter stores the time of the last access. This means that every filesystem access changes their inodes. The clusterwide locking that is needed for this operation may reduce GFS' overall performance. To further reduce this issue it is recommended that you structure your filesystem layout so that files are not stored in single directories.

Recommended Memory Configuration

Memory is a crucial part of every server and it is always a good idea to invest in some extra RAM for best performance. As of 2008 we have experience both with systems that are equipped with a really a lot of memory (e.g. 128-256 GB). These memory requirements come from SAP or huge databases and Java application server. Nodes for "simpler" webservices usually are sufficiently equipped with 4-12GB RAM. On the low end we know that difficulties arise with nodes that have 256-512 MB RAM. So if you run a Shared Root Cluster with low end or virtualized machines you should use at least 512 megabytes.

Recommended Fencing Devices

Fencing prevents data corruption and it must be reliable at all costs.

Manual fencing is only suitable for demo scenarios and must not be used in productive environments.

Resetting the power of a server is a safe procedure and we recommend it over other methods like zoning (disrupting fibre channel communication by blocking ports on FC switch).

Ideally your server hardware already includes some management interface (e.g. Hewlett Packards Integrated Lights Out module) that provides a scriptable reset option.

However you should use redundant fencing devices so you should also have a look at the ethernet power switches from various manufacturers (e.g. APC). They can be hooked to your management network and you may switch off specific outlets from remote.