CRS-0184 error and its resolution

While installing the 11g Clusterware on 64 bit AIX 5L, we  encountered the CRS-0184 error while running the root.sh on the second node.

The error was …

Cluster Synchronization Services is active on all the nodes.
 
Waiting for the Oracle CRSD and EVMD to start
 
Oracle CRS stack installed and running under init(1M)
 
Running vipca(silent) for configuring nodeapps
 
Creating VIP application resource on (2) nodes0:CRS-0184: Cannot communicate with the CRS daemon

 
The crsd.log file showed the following lines ….

2009-08-10 14:18:36.059: [ CRSMAIN][10286] Failed to spawn a thread for UI connection. error=-11
2009-08-10 14:19:12.143: [ CRSMAIN][10286] Failed to spawn a thread for UI connection. error=-11
2009-08-10 14:20:15.037: [ CRSMAIN][10286] Failed to spawn a thread for UI connection. error=-11
2009-08-10 14:21:11.379: [ CRSMAIN][10286] Failed to spawn a thread for UI connection. error=-11

The fix for this error was to increase the Shell Limits for the user ‘root’ as the Clusterware processes are started as root.

The documentation does mention that we need to so the same for the ‘oracle’ user account and fails to mention that we need to do it for the root account as well.

After changing the shell limits for root (on BOTH nodes) the CRS-0184 problem was resolved.

middba1 # ulimit -a
time(seconds) unlimited
file(blocks) unlimited
data(kbytes) unlimited
stack(kbytes) 4194304
memory(kbytes) unlimited
coredump(blocks) 2097151
nofiles(descriptors) unlimited

Note that since the root.sh on the second node will internally launch VIPCA and this part did not run to successful completion, we had to manually run the vipca command as root to get the vip,ons amd gsd resources online.