RHCS – Cluster from Scratch | Part 02

Okay after long time back, anyway I’m here and let’s get started with clustering 😀 We are going to implement two (2) node Active/Passive cluster for provide continues web service to the end users. I’n my scenario I’m  using two virtual servers as a cluster nodes and network attach storage (NAS) as a shared storage for both servers. also there are three (3) virtual networks for public network, private network(cluster heartbeat network) and for the storage network. here all of this virtual servers, network and storage are deployed on a CentOS 6 environment using KVM based hypervisor. anyway all virtual resources work as a actual physical resources.

web

above figure show the initial architecture for our high availability web service deployment, In each virtual server has three (3) network interface cards (NIC) for connect to the public, private and storage networks. In addition to the servers, network attached storage has two(2) ip address. we use both this address to configure multi-path for provide efficient and reliable storage infrastructure to our cluster deployment. here the configuration details of servers and storage.

Server 01
2.6 GHZ 2 vCPU’s with 1 GB RAM
Hostname : web1.hasitha.org
NIC 01 : 192.168.0.11 (Public Network)
NIC 02 : 192.168.1.11 (Private Network)
NIC 03 : 192.168.2.11 (Storage Network)

Server 02
2.6 GHZ 2 vCPU’s with 1 GB RAM
Hostname : web2.hasitha.org
NIC 01 : 192.168.0.12 (Public Network)
NIC 02 : 192.168.1.12 (Private Network)
NIC 03 : 192.168.2.12 (Storage Network)

Network Attach Storage (NAS)
NIC 01 : 192.168.2.1 (Storage Network)
NIC 02 : 192.168.2.2 (Storage Network)
LUN 01 : 10GB (iqn.2013-08-10.storage.hasitha.org:web)

Now we are ready to go and next part is all about configuring servers, wait till the next then, see ya soon.

Storage LUN Online Re-Scan on RHEL6

[A]lways storage maintenance and downtimes are comes together under production environment. but it’s not necessary at all the times because ISCSI(NAS) and FC(SAN) storage’s LUN re-scanning and re-sizing(expanding) is quiet easy under linux(not only Red Hat Enterprise Linux 6) environments. Okay then, assume that we have 10GB LUN on the ISCSI(NAS) storage and it has four paths to the server and also that four paths are muiltipathed. now we want to re-size this 10GB LUN to the 20GB on the storage and perform a online re-scan in the RHEL6 linux server to get that extra 10GB blocks to existing 10GB multipathed volume.
[cc lang=”bash”][[email protected] ~]# multipath -ll
mpatha (1IET 00010001) dm-4 IET,VIRTUAL-DISK
size=10G features=’0′ hwhandler=’0′ wp=rw
|-+- policy=’round-robin 0′ prio=1 status=active
| `- 4:0:0:1 sda 8:0 active ready running
|-+- policy=’round-robin 0′ prio=1 status=enabled
| `- 5:0:0:1 sdb 8:16 active ready running
|-+- policy=’round-robin 0′ prio=1 status=enabled
| `- 3:0:0:1 sdc 8:32 active ready running
`-+- policy=’round-robin 0′ prio=1 status=enabled
`- 2:0:0:1 sdd 8:48 active ready running[/cc]
You can see there are four path to “mpatha” multipathed volume and sda, sdb, adc, sdd block devices are underling to it. so now we can expand the LUN in the ISCSI(NAS) and re-scan the block devices to add that extra expanded blocks to the “mpatha” volume.
[cc lang=”bash”][[email protected] ~]# echo 1 > /sys/block/sda/device/rescan
[[email protected] ~]# echo 1 > /sys/block/sdb/device/rescan
[[email protected] ~]# echo 1 > /sys/block/sdc/device/rescan
[[email protected] ~]# echo 1 > /sys/block/sdd/device/rescan[/cc]
Using above commands, we re-scan the “sda”, “sdb”, “sdc” and “sdd” devices and that mean we are re-sized the all four path for “mpatha” volume. but device-mapper-multipath still using the old device blocks and we have to tell that multipathd to underlying devices to “mpatha” are re-sized, using
[cc lang=”bash”][[email protected] ~]# multipathd -k”resize multipath mpatha”
ok[/cc] or simply reloading the multipathd.
[cc lang=”bash”][[email protected] ~]# service multipathd reload
Reloading multipathd: [ OK ][/cc]
Now we are re-scanned the underling block devices for “mpatha” volume and let “device-mapper-multipath” know that “mpatha” was re-sized. that mean we are almost done with our ISCSI(NAS) storage re-scanning part. let’s check it out.
[cc lang=”bash”][[email protected] ~]# multipath -ll
mpatha (1IET 00010001) dm-4 IET,VIRTUAL-DISK
size=20G features=’0′ hwhandler=’0′ wp=rw
|-+- policy=’round-robin 0′ prio=1 status=active
| `- 4:0:0:1 sda 8:0 active ready running
|-+- policy=’round-robin 0′ prio=1 status=enabled
| `- 5:0:0:1 sdb 8:16 active ready running
|-+- policy=’round-robin 0′ prio=1 status=enabled
| `- 2:0:0:1 sdd 8:48 active ready running
`-+- policy=’round-robin 0′ prio=1 status=enabled
`- 3:0:0:1 sdc 8:32 active ready running[/cc]
Oh yeah baby it’s working now. 😛 see the “multipath -ll” output and now “mpatha” volume is 20GB long.

In addition to ISCSI(NAS), if you are in a FC(SAN) environment, online re-scanning is quiet different but easy as previous we did in ISCSI. We can perform a FC(SAN) LUN online re-scan using the preferred command
[cc lang=”bash”]echo “- – -” > /sys/class/scsi_host/hostX/scan[/cc]
where “hostX” is your HBA(Host Bus Adapter), That’s it.