In sweet memories of my ever loving brother "kutty thambi " ARUN KUMAR
Showing posts with label CLUSTERWARE. Show all posts
Showing posts with label CLUSTERWARE. Show all posts

Thursday, August 26, 2010

[PRKC-1044 : Failed to check remote command execution setup for node

[PRKC-1044 : Failed to check remote command execution setup for node racnode1.localdomain using shells /usr/bin/ssh and /usr/bin/rsh Permission denied.]]

solution:
Configure SSH between two nodes

#

Log in as the software owner (in this example, the oracle user).

note: dont use su - oracle , logout and login as oracle user.
make sure you login as oracle user.

#

To ensure that you are logged in as the Oracle user, and that the user ID matches the expected user ID you have assigned to the Oracle user, enter the commands id. Ensure that Oracle user group and user and the terminal window process group and user IDs are identical.
example: check it on both nodes
[oracle@racnode1 ~]$ id 
uid=500(oracle) gid=500(oinstall) groups=500(oinstall),501(dba)
[oracle@racnode2 ~]$ id 
uid=500(oracle) gid=500(oinstall) groups=500(oinstall),501(dba)
#
While logged in as oracle perform the following on each of the nodes of the RAC:

1. cd $HOME
2. mkdir ~/.ssh
3. chmod 700 ~/.ssh
4. /usr/bin/ssh-keygen -t rsa
5. /usr/bin/ssh-keygen -t dsa


On Node 1:

1. cd $HOME/.ssh
2. cat id_rsa.pub >> authorized_keys
3. cat id_dsa.pub >> authorized_keys
4. Copy the authorized_keys file to the node 2. scp authorized_keys racnode2:/home/oracle/.ssh


On Node 2:

1. cd $HOME/.ssh
2. cat id_rsa.pub >> authorized_keys
3. cat id_dsa.pub >> authorized_keys
4. scp authorized_keys racnode1:/home/oracle/.ssh

On each node, enter the following commands to start the SSH agent, and to load the SSH keys into memory:

$ exec /usr/bin/ssh-agent $SHELL
$ /usr/bin/ssh-add

At the prompt, enter the pass phrase for each key that you generated.

* Now perform a ssh between all the nodes including the node-priv hostnames as well. Check to make sure that ssh is configured well without prompting for the password (on both the nodes):
#
Can you make sure your sshd_config file as following entries uncommented?


X11Forwarding yes

X11DisplayOffset 10

X11UseLocalhost yes


If not then edit and uncomment them. Restart the ssh once you make these changes.

Restart OpenSSH server

Type the following command:

#/etc/init.d/ssh restart


Add the entries to the .bashrc file as oracle user
#

if [ -t 0 ]; then
stty intr ^C
fi

#

Try creating a symbolic link:

ln -s /usr/bin/ssh /usr/local/bin/ssh
ln -s /usr/bin/scp /usr/local/bin/scp

After you set the dsa and rsa keys in to the authorized_keys file.

From NODE 1

ssh racnode1 date
ssh racnode2 date

ssh racnode1.localdomain date
ssh racnode2.localdomain date

ssh racnode1-priv date
ssh racnode2-priv date

ssh racnode1-priv.localdomain date
ssh racnode2-priv.localdomain date

From NODE 2

ssh racnode1 date
ssh racnode2 date

ssh racnode1.localdomain date
ssh racnode2.localdomain date

ssh racnode1-priv date
ssh racnode2-priv date

ssh racnode1-priv.localdomain date
ssh racnode2-priv.localdomain date


Hope, This will resolve your issue.

references:
http://download.oracle.com/docs/cd/B28359_01/install.111/b28263/prelinux.htm#BABJBAEB

Friday, August 20, 2010

Recover Corrupt/Missing OCR and Voting Disk with No Backup

Recover Corrupt/Missing OCR and Voting Disk without Backup

It happens. Not very often, but it can happen. You are faced with a corrupt or missing Oracle Cluster Registry (OCR) and have no backup to recover from.

/u01/crs/oracle/product/10.2.0/crs/log/rac1/alertrac1.log

[client(20186)]CRS-1006:The OCR location /dev/raw/raw1 is inaccessible.
Details in /u01/crs/oracle/product/10.2.0/crs/log/rac1/client/ocrcheck_20186.log.

ocrcheck_20186.log file contents
2010-08-20 12:19:19.796: [ default][4143896256]a_init:7!: Backend init unsuccessful : [22]
2010-08-20 12:19:19.804: [OCRCHECK][4143896256]Failed to initialize OCR context:
[PROC-22: The OCR backend has an invalid format]
2010-08-20 12:19:19.804: [OCRCHECK][4143896256]Failed to initialize ocrchek2
2010-08-20 12:19:19.804: [OCRCHECK][4143896256]Exiting [status=failed].

[root@rac1 bin]# ./ocrcheck
PROT-601: Failed to initialize ocrcheck
[root@rac1 bin]# ./crs_stat -t
CRS-0184: Cannot communicate with the CRS daemon.

[root@rac1 bin]#
[root@rac1 bin]# ./crsctl query css votedisk
OCR initialization failed with invalid format: PROC-22: The OCR backend has an invalid format
[root@rac1 bin]#

[root@rac2 bin]# ./crs_stat -t
CRS-0184: Cannot communicate with the CRS daemon.

verify all the database are disconnect and shutdown

1.Execute rootdelete.sh from All Nodes.

The rootdelete.sh script can be found at $ORA_CRS_HOME/install/rootdelete.sh on all nodes in the cluster

[root@rac1 install]# pwd
/u01/crs/oracle/product/10.2.0/crs/install
[root@rac1 install]# ./rootdelete.sh
Shutting down Oracle Cluster Ready Services (CRS):
OCR initialization failed with invalid format: PROC-22: The OCR backend has an invalid format
Shutdown has begun. The daemons should exit soon.
Checking to see if Oracle CRS stack is down...
Oracle CRS stack is not running.
Oracle CRS stack is down now.
Removing script for Oracle Cluster Ready services
Updating ocr file for downgrade
Cleaning up SCR settings in '/etc/oracle/scls_scr'
[root@rac1 install]#

[root@rac2 install]# pwd
/u01/crs/oracle/product/10.2.0/crs/install
[root@rac2 install]# ./rootdelete.sh
Shutting down Oracle Cluster Ready Services (CRS):
OCR initialization failed with invalid format:
PROC-22: The OCR backend has an invalid format
Shutdown has begun. The daemons should exit soon.
Checking to see if Oracle CRS stack is down...
Oracle CRS stack is not running.
Oracle CRS stack is down now.
Removing script for Oracle Cluster Ready services
Updating ocr file for downgrade
Cleaning up SCR settings in '/etc/oracle/scls_scr'
[root@rac2 install]#

2.Run rootdeinstall.sh from the Primary Node.


[root@rac1 install]# ./rootdeinstall.sh

Removing contents from OCR device
2560+0 records in
2560+0 records out

3.Run root.sh from the Primary Node. (same node as above)

[root@rac1 install]# cd ..
[root@rac1 crs]# ./root.sh
WARNING: directory '/u01/crs/oracle/product/10.2.0' is not owned by root
WARNING: directory '/u01/crs/oracle/product' is not owned by root
WARNING: directory '/u01/crs/oracle' is not owned by root
WARNING: directory '/u01/crs' is not owned by root
WARNING: directory '/u01' is not owned by root
Checking to see if Oracle CRS stack is already configured

Setting the permissions on OCR backup directory
Setting up NS directories
Oracle Cluster Registry configuration upgraded successfully
WARNING: directory '/u01/crs/oracle/product/10.2.0' is not owned by root
WARNING: directory '/u01/crs/oracle/product' is not owned by root
WARNING: directory '/u01/crs/oracle' is not owned by root
WARNING: directory '/u01/crs' is not owned by root
WARNING: directory '/u01' is not owned by root
assigning default hostname rac1 for node 1.
assigning default hostname rac2 for node 2.
Successfully accumulated necessary OCR keys.
Using ports: CSS=49895 CRS=49896 EVMC=49898 and EVMR=49897.
node :
node 1: rac1 rac1-priv rac1
node 2: rac2 rac2-priv rac2
Creating OCR keys for user 'root', privgrp 'root'..
Operation successful.
Now formatting voting device: /dev/raw/raw2
Format of 1 voting devices complete.
Startup will be queued to init within 90 seconds.
Adding daemons to inittab
Expecting the CRS daemons to be up within 600 seconds.
CSS is active on these nodes.
rac1
CSS is inactive on these nodes.
rac2
Local node checking complete.
Run root.sh on remaining nodes to start CRS daemons.
[root@rac1 crs]#
[root@rac2 crs]# pwd
/u01/crs/oracle/product/10.2.0/crs
[root@rac2 crs]# ./root.sh
WARNING: directory '/u01/crs/oracle/product/10.2.0' is not owned by root
WARNING: directory '/u01/crs/oracle/product' is not owned by root
WARNING: directory '/u01/crs/oracle' is not owned by root
WARNING: directory '/u01/crs' is not owned by root
WARNING: directory '/u01' is not owned by root
Checking to see if Oracle CRS stack is already configured

Setting the permissions on OCR backup directory
Setting up NS directories
Oracle Cluster Registry configuration upgraded successfully
WARNING: directory '/u01/crs/oracle/product/10.2.0' is not owned by root
WARNING: directory '/u01/crs/oracle/product' is not owned by root
WARNING: directory '/u01/crs/oracle' is not owned by root
WARNING: directory '/u01/crs' is not owned by root
WARNING: directory '/u01' is not owned by root
clscfg: EXISTING configuration version 3 detected.
clscfg: version 3 is 10G Release 2.
assigning default hostname rac1 for node 1.
assigning default hostname rac2 for node 2.
Successfully accumulated necessary OCR keys.
Using ports: CSS=49895 CRS=49896 EVMC=49898 and EVMR=49897.
node :
node 1: rac1 rac1-priv rac1
node 2: rac2 rac2-priv rac2
clscfg: Arguments check out successfully.

NO KEYS WERE WRITTEN. Supply -force parameter to override.
-force is destructive and will destroy any previous cluster
configuration.
Oracle Cluster Registry for cluster has already been initialized
Startup will be queued to init within 90 seconds.
Adding daemons to inittab
Expecting the CRS daemons to be up within 600 seconds.
CSS is active on these nodes.
rac1
rac2
CSS is active on all nodes.
Waiting for the Oracle CRSD and EVMD to start
Waiting for the Oracle CRSD and EVMD to start
Waiting for the Oracle CRSD and EVMD to start
Waiting for the Oracle CRSD and EVMD to start
Waiting for the Oracle CRSD and EVMD to start
Waiting for the Oracle CRSD and EVMD to start
Waiting for the Oracle CRSD and EVMD to start
Oracle CRS stack installed and running under init(1M)
Running vipca(silent) for configuring nodeapps
The given interface(s), "eth0" is not public. Public interfaces should be used to configure virtual IPs.


Oracle 10.2.0.1 users should note that running root.sh on the last node will fail. Most notably is the silent mode VIPCA configuration failing because of BUG 4437727 in 10.2.0.1.

to workaround these errors.

[root@rac2 bin]# pwd
/u01/crs/oracle/product/10.2.0/crs/bin
[root@rac2 bin]# ./vipca &

When the "VIP Configuration Assistant" appears, this is how I answered the screen prompts:

Welcome: Click Next
Network interfaces: Select only the public interface - eth0
Virtual IPs for cluster nodes:
Node Name: rac1
IP Alias Name: rac1-vip.localdomain
IP Address: 192.168.1.111
Subnet Mask: 255.255.255.0

Node Name: rac2
IP Alias Name: rac2-vip.localdomain
IP Address: 192.168.1.112
Subnet Mask: 255.255.255.0

Summary: Click Finish
Configuration Assistant Progress Dialog: Click OK after configuration is complete.
Configuration Results: Click Exit


4.Configure Server-Side ONS using racgons.

CRS_home/bin/racgons add_config hostname1:port hostname2:port

[root@rac1 crs]# cd bin
[root@rac1 bin]# pwd
/u01/crs/oracle/product/10.2.0/crs/bin
[root@rac1 bin]# ./racgons add_config rac1:6200 rac2:6200
[root@rac1 bin]# ./onsctl ping
Number of onsconfiguration retrieved, numcfg = 2
onscfg[0]
{node = rac1, port = 6200}
Adding remote host rac1:6200
GETHOSTBYNAME(rac1): 2
onscfg[1]
{node = rac2, port = 6200}
Adding remote host rac2:6200
GETHOSTBYNAME(rac2): 2
ons is running ...
[root@rac1 bin]#

5.Configure Network Interfaces for Clusterware.
Log in as the owner of the Oracle Clusterware software which is typically the oracle user account and configure all network interfaces. The first step is to identify the current interfaces and IP addresses using oifcfg iflist.

[root@rac1 bin]# pwd
/u01/crs/oracle/product/10.2.0/crs/bin
[root@rac1 bin]# ./oifcfg iflist
eth0 192.168.1.0
eth1 192.168.2.0
[root@rac1 bin]# ./oifcfg setif -global eth0/192.168.1.0:public
[root@rac1 bin]# ./oifcfg setif -global eth1/192.168.2.0:cluster_interconnect
[root@rac1 bin]# ./oifcfg getif
eth0 192.168.1.0 global public
eth1 192.168.2.0 global cluster_interconnect
[root@rac1 bin]#

6.Add TNS Listener using NETCA.

As the Oracle Clusterware software owner (typically oracle), add a cluster TNS listener configuration to OCR using netca. This may give errors if the listener.ora contains the entries already. If this is the case, move the listener.ora to /tmp from the $ORACLE_HOME/network/admin or from the $TNS_ADMIN directory if the TNS_ADMIN environmental is defined and then run netca. Add all the listeners that were added during the original Oracle Clusterware software installation.

[oracle@rac2 admin]$ mv listener.ora /tmp/listener.ora.original

[oracle@rac1 admin]$ mv listener.ora /tmp/listener.ora.original
[oracle@rac1 admin]$ netca &

Screen Name Response
Select the Type of Oracle
Net Services Configuration
Select Cluster configuration
Select the nodes to configure Select all
Type of Configuration Select Listener configuration.
Listener Configuration
Next 6 Screens
The following screens are now like any other normal listener configuration. You can simply accept the default parameters for the next six screens:
What do you want to do: Add
Listener name: LISTENER
Selected protocols: TCP
Port number: 1521
Configure another listener: No
Listener configuration complete! [ Next ]
You will be returned to this Welcome (Type of Configuration) Screen.
Type of Configuration Select Naming Methods configuration.
Naming Methods Configuration The following screens are:
Selected Naming Methods: Local Naming
Naming Methods configuration complete! [ Next ]
You will be returned to this Welcome (Type of Configuration) Screen.
Type of Configuration Click Finish to exit the NETCA.

7.Add all Resources Back to OCR using srvctl



[root@rac1 bin]# ./crs_stat -t
Name Type Target State Host
------------------------------------------------------------
ora....C1.lsnr application ONLINE ONLINE rac1
ora.rac1.gsd application ONLINE ONLINE rac1
ora.rac1.ons application ONLINE ONLINE rac1
ora.rac1.vip application ONLINE ONLINE rac1
ora....C2.lsnr application ONLINE ONLINE rac2
ora.rac2.gsd application ONLINE ONLINE rac2
ora.rac2.ons application ONLINE ONLINE rac2
ora.rac2.vip application ONLINE ONLINE rac2

As a final step, log in as the Oracle Clusterware software owner (typically oracle) and add all resources back to the OCR using the srvctl command.

Please ensure that these commands are not run as the root user account.

Add ASM INSTANCE(S) to OCR:

srvctl add asm -n -i -o

[oracle@rac1 bin]$ pwd
/u01/crs/oracle/product/10.2.0/crs/bin
[oracle@rac1 bin]$ ./srvctl add asm -i +ASM1 -n rac1 -o /u01/app/oracle/product/10.2.0/db_1
[oracle@rac1 bin]$ ./srvctl add asm -i +ASM2 -n rac2 -o /u01/app/oracle/product/10.2.0/db_1
[oracle@rac1 bin]$





Add DATABASE to OCR:

srvctl add database -d -o

[oracle@rac1 bin]$ ./srvctl add database -d cdbs -o /u01/app/oracle/product/10.2.0/db_1

Add INSTANCE(S) to OCR:

srvctl add instance -d -i -n

[oracle@rac1 bin]$ ./srvctl add instance -d cdbs -i cdbs1 -n rac1
[oracle@rac1 bin]$ ./srvctl add instance -d cdbs -i cdbs2 -n rac2

Add SERVICE(S) to OCR:

srvctl add service -d -s -r -P

where TAF_policy is set to NONE, BASIC, or PRECONNECT

[oracle@rac1 bin]$ ./srvctl add service -d cdbs -s cdbs_srvc -r cdbs1,cdbs2 -P BASIC

After completing the steps above, the OCR should have been successfully recreated. Bring up all of the resources that were added to the OCR and run cluvfy to verify the cluster configuration.

[oracle@rac1 bin]$ ./crs_stat -t
Name Type Target State Host
------------------------------------------------------------
ora....s1.inst application OFFLINE OFFLINE
ora....s2.inst application OFFLINE OFFLINE
ora....bs1.srv application OFFLINE OFFLINE
ora....bs2.srv application OFFLINE OFFLINE
ora....srvc.cs application OFFLINE OFFLINE
ora.cdbs.db application OFFLINE OFFLINE
ora....SM1.asm application OFFLINE OFFLINE
ora....C1.lsnr application ONLINE ONLINE rac1
ora.rac1.gsd application ONLINE ONLINE rac1
ora.rac1.ons application ONLINE ONLINE rac1
ora.rac1.vip application ONLINE ONLINE rac1
ora....SM2.asm application OFFLINE OFFLINE
ora....C2.lsnr application ONLINE ONLINE rac2
ora.rac2.gsd application ONLINE ONLINE rac2
ora.rac2.ons application ONLINE ONLINE rac2
ora.rac2.vip application ONLINE ONLINE rac2

[oracle@rac1 bin]$ srvctl start asm -n rac1
[oracle@rac1 bin]$ srvctl start asm -n rac2
[oracle@rac1 bin]$ srvctl start database -d cdbs
[oracle@rac1 bin]$ srvctl start service -d cdbs
[oracle@rac1 bin]$ ./crs_stat -t
Name Type Target State Host
------------------------------------------------------------
ora....s1.inst application ONLINE ONLINE rac1
ora....s2.inst application ONLINE ONLINE rac2
ora....bs1.srv application ONLINE ONLINE rac1
ora....bs2.srv application ONLINE ONLINE rac2
ora....srvc.cs application ONLINE ONLINE rac1
ora.cdbs.db application ONLINE ONLINE rac2
ora....SM1.asm application ONLINE ONLINE rac1
ora....C1.lsnr application ONLINE ONLINE rac1
ora.rac1.gsd application ONLINE ONLINE rac1
ora.rac1.ons application ONLINE ONLINE rac1
ora.rac1.vip application ONLINE ONLINE rac1
ora....SM2.asm application ONLINE ONLINE rac2
ora....C2.lsnr application ONLINE ONLINE rac2
ora.rac2.gsd application ONLINE ONLINE rac2
ora.rac2.ons application ONLINE ONLINE rac2
ora.rac2.vip application ONLINE ONLINE rac2
[oracle@rac1 bin]$

[oracle@rac1 bin]$ cluvfy stage -post crsinst -n rac1,rac2

Performing post-checks for cluster services setup

Checking node reachability...
Node reachability check passed from node "rac1".


Checking user equivalence...
User equivalence check passed for user "oracle".

Checking Cluster manager integrity...


Checking CSS daemon...
Daemon status check passed for "CSS daemon".

Cluster manager integrity check passed.

Checking cluster integrity...


Cluster integrity check passed


Checking OCR integrity...

Checking the absence of a non-clustered configuration...
All nodes free of non-clustered, local-only configurations.

Uniqueness check for OCR device passed.

Checking the version of OCR...
OCR of correct Version "2" exists.

Checking data integrity of OCR...
Data integrity check for OCR passed.

OCR integrity check passed.

Checking CRS integrity...

Checking daemon liveness...
Liveness check passed for "CRS daemon".

Checking daemon liveness...
Liveness check passed for "CSS daemon".

Checking daemon liveness...
Liveness check passed for "EVM daemon".

Checking CRS health...
CRS health check passed.

CRS integrity check passed.

Checking node application existence...


Checking existence of VIP node application (required)
Check passed.

Checking existence of ONS node application (optional)
Check passed.

Checking existence of GSD node application (optional)
Check passed.


Post-check for cluster services setup was successful.
[oracle@rac1 bin]$




source and reference:
http://www.idevelopment.info/data/Oracle/DBA_tips/Oracle10gRAC/CLUSTER_70.shtml#Recover%20Corrupt/Missing%20OCR

Thursday, August 19, 2010

ORA-00439: feature not enabled: Real Application Clusters

Yesterday, i tried to upgrade my Clusterware with patch file p6810189_10204_Linux-x86.zip

due to some mistake the clusterware corrupted.

so i Reinstall CRS Without Disturbing Installed Oracle RDBMS Home

after successful installation of clusterware and add the existing resources to clusterware.

i started ASM with srvctl command, i got

ORA-00439: feature not enabled: Real Application Clusters

Because the Oracle Home disabled RAC, previously i disabled RAC to access single instance (non-cluster) database.

So, we should relink library to enable RAC on this Oracle Home

$ cd $ORACLE_HOME/rdbms/lib

$ make -f ins_rdbms.mk rac_on

$ make -f ins_rdbms.mk ioracle

[oracle@rac1 bin]$ ./srvctl start asm -n rac1
PRKS-1009 : Failed to start ASM instance "+ASM1" on node "rac1",
[PRKS-1009 : Failed to start ASM instance "+ASM1" on node "rac1", [rac1:ora.rac1.ASM1.asm:
rac1:ora.rac1.ASM1.asm:SQL*Plus: Release 10.2.0.1.0 - Production on Thu Aug 19 13:33:01 2010
rac1:ora.rac1.ASM1.asm:
rac1:ora.rac1.ASM1.asm:Copyright (c) 1982, 2005, Oracle. All rights reserved.
rac1:ora.rac1.ASM1.asm:
rac1:ora.rac1.ASM1.asm:Enter user-name: Connected to an idle instance.
rac1:ora.rac1.ASM1.asm:
rac1:ora.rac1.ASM1.asm:SQL> ORA-00439: feature not enabled: Real Application Clusters
rac1:ora.rac1.ASM1.asm:SQL> Disconnected
rac1:ora.rac1.ASM1.asm:
CRS-0215: Could not start resource 'ora.rac1.ASM1.asm'.]]
[PRKS-1009 : Failed to start ASM instance "+ASM1" on node "rac1", [rac1:ora.rac1.ASM1.asm:
rac1:ora.rac1.ASM1.asm:SQL*Plus: Release 10.2.0.1.0 - Production on Thu Aug 19 13:33:01 2010
rac1:ora.rac1.ASM1.asm:
rac1:ora.rac1.ASM1.asm:Copyright (c) 1982, 2005, Oracle. All rights reserved.
rac1:ora.rac1.ASM1.asm:
rac1:ora.rac1.ASM1.asm:Enter user-name: Connected to an idle instance.
rac1:ora.rac1.ASM1.asm:
rac1:ora.rac1.ASM1.asm:SQL> ORA-00439: feature not enabled: Real Application Clusters
rac1:ora.rac1.ASM1.asm:SQL> Disconnected
rac1:ora.rac1.ASM1.asm:
CRS-0215: Could not start resource 'ora.rac1.ASM1.asm'.]]


solution

[oracle@rac1 bin]$ cd $ORACLE_HOME/rdbms/lib
[oracle@rac1 lib]$ pwd
/u01/app/oracle/product/10.2.0/db_1/rdbms/lib
[oracle@rac1 lib]$ make -f ins_rdbms.mk rac_on
rm -f /u01/app/oracle/product/10.2.0/db_1/lib/libskgxp10.so
cp /u01/app/oracle/product/10.2.0/db_1/lib//libskgxpu.so /u01/app/oracle/product/10.2.0/db_1/lib/libskgxp10.so
cp /u01/app/oracle/product/10.2.0/db_1/lib/libskgxns.so /u01/app/oracle/product/10.2.0/db_1/lib/libskgxn2.so
/usr/bin/ar d /u01/app/oracle/product/10.2.0/db_1/rdbms/lib/libknlopt.a ksnkcs.o/usr/bin/ar cr /u01/app/oracle/product/10.2.0/db_1/rdbms/lib/libknlopt.a /u01/app/oracle/product/10.2.0/db_1/rdbms/lib/kcsm.o
[oracle@rac1 lib]$ make -f ins_rdbms.mk ioracle
chmod 755 /u01/app/oracle/product/10.2.0/db_1/bin

- Linking Oracle
rm -f /u01/app/oracle/product/10.2.0/db_1/rdbms/lib/oracle
gcc -o /u01/app/oracle/product/10.2.0/db_1/rdbms/lib/oracle -L/u01/app/oracle/product/10.2.0/db_1/rdbms/lib/ -L/u01/app/oracle/product/10.2.0/db_1/lib/ -L/u01/app/oracle/product/10.2.0/db_1/lib/stubs/ -L/usr/lib -lirc -Wl,-E `test -f /u01/app/oracle/product/10.2.0/db_1/rdbms/lib/sskgpsmti.o && echo /u01/app/oracle/product/10.2.0/db_1/rdbms/lib/sskgpsmti.o` /u01/app/oracle/product/10.2.0/db_1/rdbms/lib/opimai.o /u01/app/oracle/product/10.2.0/db_1/rdbms/lib/ssoraed.o /u01/app/oracle/product/10.2.0/db_1/rdbms/lib/ttcsoi.o /u01/app/oracle/product/10.2.0/db_1/rdbms/lib/defopt.o -Wl,--whole-archive -lperfsrv10 -Wl,--no-whole-archive /u01/app/oracle/product/10.2.0/db_1/lib/nautab.o /u01/app/oracle/product/10.2.0/db_1/lib/naeet.o /u01/app/oracle/product/10.2.0/db_1/lib/naect.o /u01/app/oracle/product/10.2.0/db_1/lib/naedhs.o /u01/app/oracle/product/10.2.0/db_1/rdbms/lib/config.o -lserver10 -lodm10 -lnnet10 -lskgxp10 -lsnls10 -lnls10 -lcore10 -lsnls10 -lnls10 -lcore10 -lsnls10 -lnls10 -lxml10 -lcore10 -lunls10 -lsnls10 -lnls10 -lcore10 -lnls10 -lhasgen10 -lcore10 -lskgxn2 -locr10 -locrb10 -locrutl10 -lhasgen10 -lcore10 -lskgxn2 -lclient10 -lvsn10 -lcommon10 -lgeneric10 -lknlopt `if /usr/bin/ar tv /u01/app/oracle/product/10.2.0/db_1/rdbms/lib/libknlopt.a | grep xsyeolap.o > /dev/null 2>&1 ; then echo "-loraolap10" ; fi` -lslax10 -lpls10 -lplp10 -lserver10 -lclient10 -lvsn10 -lcommon10 -lgeneric10 -lknlopt -lslax10 -lpls10 -lplp10 -ljox10 -lserver10 -lclsra10 -ldbcfg10 -locijdbcst10 -lwwg `cat /u01/app/oracle/product/10.2.0/db_1/lib/ldflags` -lnsslb10 -lncrypt10 -lnsgr10 -lnzjs10 -ln10 -lnnz10 -lnl10 -lnro10 `cat /u01/app/oracle/product/10.2.0/db_1/lib/ldflags` -lnsslb10 -lncrypt10 -lnsgr10 -lnzjs10 -ln10 -lnnz10 -lnl10 -lmm -lsnls10 -lnls10 -lcore10 -lsnls10 -lnls10 -lcore10 -lsnls10 -lnls10 -lxml10 -lcore10 -lunls10 -lsnls10 -lnls10 -lcore10 -lnls10 `cat /u01/app/oracle/product/10.2.0/db_1/lib/ldflags` -lnsslb10 -lncrypt10 -lnsgr10 -lnzjs10 -ln10 -lnnz10 -lnl10 -lnro10 `cat /u01/app/oracle/product/10.2.0/db_1/lib/ldflags` -lnsslb10 -lncrypt10 -lnsgr10 -lnzjs10 -ln10 -lnnz10 -lnl10 -lsnls10 -lnls10 -lcore10 -lsnls10 -lnls10 -lcore10 -lsnls10 -lnls10 -lxml10 -lcore10 -lunls10 -lsnls10 -lnls10 -lcore10 -lnls10 `if /usr/bin/ar tv /u01/app/oracle/product/10.2.0/db_1/rdbms/lib/libknlopt.a | grep "kxmnsd.o" > /dev/null 2>&1 ; then echo " " ; else echo "-lordsdo10"; fi` -lctxc10 -lctx10 -lzx10 -lgx10 -lctx10 -lzx10 -lgx10 -lordimt10 -lsnls10 -lnls10 -lcore10 -lsnls10 -lnls10 -lcore10 -lsnls10 -lnls10 -lxml10 -lcore10 -lunls10 -lsnls10 -lnls10 -lcore10 -lnls10 -lsnls10 -lunls10 -lsnls10 -lnls10 -lcore10 -lsnls10 -lnls10 -lcore10 -lsnls10 -lnls10 -lxml10 -lcore10 -lunls10 -lsnls10 -lnls10 -lcore10 -lnls10 -laio `cat /u01/app/oracle/product/10.2.0/db_1/lib/sysliblist` -Wl,-rpath,/u01/app/oracle/product/10.2.0/db_1/lib -lm `cat /u01/app/oracle/product/10.2.0/db_1/lib/sysliblist` -ldl -lm -L/u01/app/oracle/product/10.2.0/db_1/lib
/u01/app/oracle/product/10.2.0/db_1/lib//libcore10.a(lcd.o)(.text+0x8e8): In function `lcdprm':
: warning: the `gets' function is dangerous and should not be used.
mv -f /u01/app/oracle/product/10.2.0/db_1/bin/oracle /u01/app/oracle/product/10.2.0/db_1/bin/oracleO
mv /u01/app/oracle/product/10.2.0/db_1/rdbms/lib/oracle /u01/app/oracle/product/10.2.0/db_1/bin/oracle
chmod 6751 /u01/app/oracle/product/10.2.0/db_1/bin/oracle
[oracle@rac1 lib]$

[oracle@rac2 ~]$ cd $ORACLE_HOME/rdbms/lib
[oracle@rac2 lib]$ make -f ins_rdbms.mk rac_on
rm -f /u01/app/oracle/product/10.2.0/db_1/lib/libskgxp10.so
cp /u01/app/oracle/product/10.2.0/db_1/lib//libskgxpu.so /u01/app/oracle/product/10.2.0/db_1/lib/libskgxp10.so
cp /u01/app/oracle/product/10.2.0/db_1/lib/libskgxns.so /u01/app/oracle/product/10.2.0/db_1/lib/libskgxn2.so
/usr/bin/ar d /u01/app/oracle/product/10.2.0/db_1/rdbms/lib/libknlopt.a ksnkcs.o/usr/bin/ar cr /u01/app/oracle/product/10.2.0/db_1/rdbms/lib/libknlopt.a /u01/app/oracle/product/10.2.0/db_1/rdbms/lib/kcsm.o
[oracle@rac2 lib]$ make -f ins_rdbms.mk ioracle
chmod 755 /u01/app/oracle/product/10.2.0/db_1/bin

- Linking Oracle
rm -f /u01/app/oracle/product/10.2.0/db_1/rdbms/lib/oracle
gcc -o /u01/app/oracle/product/10.2.0/db_1/rdbms/lib/oracle -L/u01/app/oracle/product/10.2.0/db_1/rdbms/lib/ -L/u01/app/oracle/product/10.2.0/db_1/lib/ -L/u01/app/oracle/product/10.2.0/db_1/lib/stubs/ -L/usr/lib -lirc -Wl,-E `test -f /u01/app/oracle/product/10.2.0/db_1/rdbms/lib/sskgpsmti.o && echo /u01/app/oracle/product/10.2.0/db_1/rdbms/lib/sskgpsmti.o` /u01/app/oracle/product/10.2.0/db_1/rdbms/lib/opimai.o /u01/app/oracle/product/10.2.0/db_1/rdbms/lib/ssoraed.o /u01/app/oracle/product/10.2.0/db_1/rdbms/lib/ttcsoi.o /u01/app/oracle/product/10.2.0/db_1/rdbms/lib/defopt.o -Wl,--whole-archive -lperfsrv10 -Wl,--no-whole-archive /u01/app/oracle/product/10.2.0/db_1/lib/nautab.o /u01/app/oracle/product/10.2.0/db_1/lib/naeet.o /u01/app/oracle/product/10.2.0/db_1/lib/naect.o /u01/app/oracle/product/10.2.0/db_1/lib/naedhs.o /u01/app/oracle/product/10.2.0/db_1/rdbms/lib/config.o -lserver10 -lodm10 -lnnet10 -lskgxp10 -lsnls10 -lnls10 -lcore10 -lsnls10 -lnls10 -lcore10 -lsnls10 -lnls10 -lxml10 -lcore10 -lunls10 -lsnls10 -lnls10 -lcore10 -lnls10 -lhasgen10 -lcore10 -lskgxn2 -locr10 -locrb10 -locrutl10 -lhasgen10 -lcore10 -lskgxn2 -lclient10 -lvsn10 -lcommon10 -lgeneric10 -lknlopt `if /usr/bin/ar tv /u01/app/oracle/product/10.2.0/db_1/rdbms/lib/libknlopt.a | grep xsyeolap.o > /dev/null 2>&1 ; then echo "-loraolap10" ; fi` -lslax10 -lpls10 -lplp10 -lserver10 -lclient10 -lvsn10 -lcommon10 -lgeneric10 -lknlopt -lslax10 -lpls10 -lplp10 -ljox10 -lserver10 -lclsra10 -ldbcfg10 -locijdbcst10 -lwwg `cat /u01/app/oracle/product/10.2.0/db_1/lib/ldflags` -lnsslb10 -lncrypt10 -lnsgr10 -lnzjs10 -ln10 -lnnz10 -lnl10 -lnro10 `cat /u01/app/oracle/product/10.2.0/db_1/lib/ldflags` -lnsslb10 -lncrypt10 -lnsgr10 -lnzjs10 -ln10 -lnnz10 -lnl10 -lmm -lsnls10 -lnls10 -lcore10 -lsnls10 -lnls10 -lcore10 -lsnls10 -lnls10 -lxml10 -lcore10 -lunls10 -lsnls10 -lnls10 -lcore10 -lnls10 `cat /u01/app/oracle/product/10.2.0/db_1/lib/ldflags` -lnsslb10 -lncrypt10 -lnsgr10 -lnzjs10 -ln10 -lnnz10 -lnl10 -lnro10 `cat /u01/app/oracle/product/10.2.0/db_1/lib/ldflags` -lnsslb10 -lncrypt10 -lnsgr10 -lnzjs10 -ln10 -lnnz10 -lnl10 -lsnls10 -lnls10 -lcore10 -lsnls10 -lnls10 -lcore10 -lsnls10 -lnls10 -lxml10 -lcore10 -lunls10 -lsnls10 -lnls10 -lcore10 -lnls10 `if /usr/bin/ar tv /u01/app/oracle/product/10.2.0/db_1/rdbms/lib/libknlopt.a | grep "kxmnsd.o" > /dev/null 2>&1 ; then echo " " ; else echo "-lordsdo10"; fi` -lctxc10 -lctx10 -lzx10 -lgx10 -lctx10 -lzx10 -lgx10 -lordimt10 -lsnls10 -lnls10 -lcore10 -lsnls10 -lnls10 -lcore10 -lsnls10 -lnls10 -lxml10 -lcore10 -lunls10 -lsnls10 -lnls10 -lcore10 -lnls10 -lsnls10 -lunls10 -lsnls10 -lnls10 -lcore10 -lsnls10 -lnls10 -lcore10 -lsnls10 -lnls10 -lxml10 -lcore10 -lunls10 -lsnls10 -lnls10 -lcore10 -lnls10 -laio `cat /u01/app/oracle/product/10.2.0/db_1/lib/sysliblist` -Wl,-rpath,/u01/app/oracle/product/10.2.0/db_1/lib -lm `cat /u01/app/oracle/product/10.2.0/db_1/lib/sysliblist` -ldl -lm -L/u01/app/oracle/product/10.2.0/db_1/lib
/u01/app/oracle/product/10.2.0/db_1/lib//libcore10.a(lcd.o)(.text+0x8e8): In function `lcdprm':
: warning: the `gets' function is dangerous and should not be used.
mv -f /u01/app/oracle/product/10.2.0/db_1/bin/oracle /u01/app/oracle/product/10.2.0/db_1/bin/oracleO
mv /u01/app/oracle/product/10.2.0/db_1/rdbms/lib/oracle /u01/app/oracle/product/10.2.0/db_1/bin/oracle
chmod 6751 /u01/app/oracle/product/10.2.0/db_1/bin/oracle
[oracle@rac2 lib]$

[oracle@rac1 bin]$ ./srvctl start asm -n rac1

[oracle@rac1 bin]$ ./srvctl start asm -n rac1

[oracle@rac1 bin]$ ./srvctl start asm -n rac2

[oracle@rac1 bin]$ ./srvctl start database -d cdbs

[oracle@rac1 bin]$ ./crs_stat -t
Name Type Target State Host
------------------------------------------------------------
ora....bs1.srv application ONLINE ONLINE rac1
ora....bs2.srv application ONLINE ONLINE rac2
ora....cdbs.cs application ONLINE ONLINE rac2
ora....s1.inst application ONLINE ONLINE rac1
ora....s2.inst application ONLINE ONLINE rac2
ora.cdbs.db application ONLINE ONLINE rac2
ora....SM1.asm application ONLINE ONLINE rac1
ora....C1.lsnr application ONLINE ONLINE rac1
ora.rac1.gsd application ONLINE ONLINE rac1
ora.rac1.ons application ONLINE ONLINE rac1
ora.rac1.vip application ONLINE ONLINE rac1
ora....SM2.asm application ONLINE ONLINE rac2
ora....C2.lsnr application ONLINE ONLINE rac2
ora.rac2.gsd application ONLINE ONLINE rac2
ora.rac2.ons application ONLINE ONLINE rac2
ora.rac2.vip application ONLINE ONLINE rac2
[oracle@rac1 bin]$





source and reference:
http://surachartopun.com/2008/11/ora-00439-feature-not-enabled-real.html

ORA-29702: error occurred in Cluster Group Service operation

# problem

SQL> startup;
ORA-29702: error occurred in Cluster Group Service operation

29702, 00000, "error occurred in Cluster Group Service operation"
// *Cause: An unexpected error occurred while performing a CGS operation.
// *Action: Verify that the LMON process is still active. Also,
// check the Oracle LMON trace files for errors.

# solution

cd $ORACLE_HOME/rdbms/lib
make -f ins_rdbms.mk rac_off
make -f ins_rdbms.mk ioracle


[oracle@rac1 ~]$ . oraenv
ORACLE_SID = [RAC1] ? jay
[oracle@rac1 ~]$ sqlplus '/as sysdba'

SQL*Plus: Release 10.2.0.1.0 - Production on Thu Aug 19 11:51:52 2010

Copyright (c) 1982, 2005, Oracle. All rights reserved.

Connected to an idle instance.

SQL> startup
ORA-29702: error occurred in Cluster Group Service operation


[oracle@rac1 ~]$ cd $ORACLE_HOME/rdbms/lib
[oracle@rac1 lib]$ pwd
/u01/app/oracle/product/10.2.0/db_1/rdbms/lib
[oracle@rac1 lib]$ make -f ins_rdbms.mk rac_off
rm -f /u01/app/oracle/product/10.2.0/db_1/lib/libskgxp10.so
cp /u01/app/oracle/product/10.2.0/db_1/lib//libskgxpd.so /u01/app/oracle/product/10.2.0/db_1/lib/libskgxp10.so
rm -f /u01/app/oracle/product/10.2.0/db_1/lib/libskgxn2.so
cp /u01/app/oracle/product/10.2.0/db_1/lib//libskgxns.so \
/u01/app/oracle/product/10.2.0/db_1/lib/libskgxn2.so
/usr/bin/ar d /u01/app/oracle/product/10.2.0/db_1/rdbms/lib/libknlopt.a kcsm.o
/usr/bin/ar cr /u01/app/oracle/product/10.2.0/db_1/rdbms/lib/libknlopt.a /u01/app/oracle/product/10.2.0/db_1/rdbms/lib/ksnkcs.o
[oracle@rac1 lib]$ make -f ins_rdbms.mk ioracle
chmod 755 /u01/app/oracle/product/10.2.0/db_1/bin

- Linking Oracle
rm -f /u01/app/oracle/product/10.2.0/db_1/rdbms/lib/oracle
gcc -o /u01/app/oracle/product/10.2.0/db_1/rdbms/lib/oracle -L/u01/app/oracle/product/10.2.0/db_1/rdbms/lib/ -L/u01/app/oracle/product/10.2.0/db_1/lib/ -L/u01/app/oracle/product/10.2.0/db_1/lib/stubs/ -L/usr/lib -lirc -Wl,-E `test -f /u01/app/oracle/product/10.2.0/db_1/rdbms/lib/sskgpsmti.o && echo /u01/app/oracle/product/10.2.0/db_1/rdbms/lib/sskgpsmti.o` /u01/app/oracle/product/10.2.0/db_1/rdbms/lib/opimai.o /u01/app/oracle/product/10.2.0/db_1/rdbms/lib/ssoraed.o /u01/app/oracle/product/10.2.0/db_1/rdbms/lib/ttcsoi.o /u01/app/oracle/product/10.2.0/db_1/rdbms/lib/defopt.o -Wl,--whole-archive -lperfsrv10 -Wl,--no-whole-archive /u01/app/oracle/product/10.2.0/db_1/lib/nautab.o /u01/app/oracle/product/10.2.0/db_1/lib/naeet.o /u01/app/oracle/product/10.2.0/db_1/lib/naect.o /u01/app/oracle/product/10.2.0/db_1/lib/naedhs.o /u01/app/oracle/product/10.2.0/db_1/rdbms/lib/config.o -lserver10 -lodm10 -lnnet10 -lskgxp10 -lsnls10 -lnls10 -lcore10 -lsnls10 -lnls10 -lcore10 -lsnls10 -lnls10 -lxml10 -lcore10 -lunls10 -lsnls10 -lnls10 -lcore10 -lnls10 -lhasgen10 -lcore10 -lskgxn2 -locr10 -locrb10 -locrutl10 -lhasgen10 -lcore10 -lskgxn2 -lclient10 -lvsn10 -lcommon10 -lgeneric10 -lknlopt `if /usr/bin/ar tv /u01/app/oracle/product/10.2.0/db_1/rdbms/lib/libknlopt.a | grep xsyeolap.o > /dev/null 2>&1 ; then echo "-loraolap10" ; fi` -lslax10 -lpls10 -lplp10 -lserver10 -lclient10 -lvsn10 -lcommon10 -lgeneric10 -lknlopt -lslax10 -lpls10 -lplp10 -ljox10 -lserver10 -lclsra10 -ldbcfg10 -locijdbcst10 -lwwg `cat /u01/app/oracle/product/10.2.0/db_1/lib/ldflags` -lnsslb10 -lncrypt10 -lnsgr10 -lnzjs10 -ln10 -lnnz10 -lnl10 -lnro10 `cat /u01/app/oracle/product/10.2.0/db_1/lib/ldflags` -lnsslb10 -lncrypt10 -lnsgr10 -lnzjs10 -ln10 -lnnz10 -lnl10 -lmm -lsnls10 -lnls10 -lcore10 -lsnls10 -lnls10 -lcore10 -lsnls10 -lnls10 -lxml10 -lcore10 -lunls10 -lsnls10 -lnls10 -lcore10 -lnls10 `cat /u01/app/oracle/product/10.2.0/db_1/lib/ldflags` -lnsslb10 -lncrypt10 -lnsgr10 -lnzjs10 -ln10 -lnnz10 -lnl10 -lnro10 `cat /u01/app/oracle/product/10.2.0/db_1/lib/ldflags` -lnsslb10 -lncrypt10 -lnsgr10 -lnzjs10 -ln10 -lnnz10 -lnl10 -lsnls10 -lnls10 -lcore10 -lsnls10 -lnls10 -lcore10 -lsnls10 -lnls10 -lxml10 -lcore10 -lunls10 -lsnls10 -lnls10 -lcore10 -lnls10 `if /usr/bin/ar tv /u01/app/oracle/product/10.2.0/db_1/rdbms/lib/libknlopt.a | grep "kxmnsd.o" > /dev/null 2>&1 ; then echo " " ; else echo "-lordsdo10"; fi` -lctxc10 -lctx10 -lzx10 -lgx10 -lctx10 -lzx10 -lgx10 -lordimt10 -lsnls10 -lnls10 -lcore10 -lsnls10 -lnls10 -lcore10 -lsnls10 -lnls10 -lxml10 -lcore10 -lunls10 -lsnls10 -lnls10 -lcore10 -lnls10 -lsnls10 -lunls10 -lsnls10 -lnls10 -lcore10 -lsnls10 -lnls10 -lcore10 -lsnls10 -lnls10 -lxml10 -lcore10 -lunls10 -lsnls10 -lnls10 -lcore10 -lnls10 -laio `cat /u01/app/oracle/product/10.2.0/db_1/lib/sysliblist` -Wl,-rpath,/u01/app/oracle/product/10.2.0/db_1/lib -lm `cat /u01/app/oracle/product/10.2.0/db_1/lib/sysliblist` -ldl -lm -L/u01/app/oracle/product/10.2.0/db_1/lib
/u01/app/oracle/product/10.2.0/db_1/lib//libcore10.a(lcd.o)(.text+0x8e8): In function `lcdprm':
: warning: the `gets' function is dangerous and should not be used.
mv -f /u01/app/oracle/product/10.2.0/db_1/bin/oracle /u01/app/oracle/product/10.2.0/db_1/bin/oracleO
mv /u01/app/oracle/product/10.2.0/db_1/rdbms/lib/oracle /u01/app/oracle/product/10.2.0/db_1/bin/oracle
chmod 6751 /u01/app/oracle/product/10.2.0/db_1/bin/oracle
[oracle@rac1 lib]$


SQL> startup
ORACLE instance started.

Total System Global Area 444596224 bytes
Fixed Size 1219904 bytes
Variable Size 130024128 bytes
Database Buffers 310378496 bytes
Redo Buffers 2973696 bytes
Database mounted.
Database opened.
SQL> select name from v$datafile;

NAME
--------------------------------------------------------------------------------/u01/app/oracle/oradata/jay/system01.dbf
/u01/app/oracle/oradata/jay/undotbs01.dbf
/u01/app/oracle/oradata/jay/sysaux01.dbf
/u01/app/oracle/oradata/jay/users01.dbf

SQL> select name from v$database;

NAME
---------
JAY




source and reference:
http://blog.contractoracle.com/2009/08/ora-29702-error-occurred-in-cluster.html

Monday, May 31, 2010

OUI-35000: Fatal cluster error encountered while adding a node.

PROBLEM:
OU1-35000: Fatal cluster error encountered ().
Correct the problem and try the operation again.

Solution:
Occurs during installation or adding a node in RAC environment.
Make sure the private node name or virtual hostnames are reachable. If they are not reachable, make sure they exist in DNS or /etc/hosts file.

Due to non availability or communication with clusterware. or wrong entry of host name, public IP, private IP or Virtual IP address during installation.
action: stop and start the CRS in all nodes. or correct the host IP Address.

[root@rac1 bin]# ./crs_stat -t
Name           Type           Target    State     Host
------------------------------------------------------------
ora.demo.db    application    ONLINE    UNKNOWN   rac3
ora....o1.inst application    ONLINE    OFFLINE
ora....o2.inst application    ONLINE    OFFLINE
ora....osrv.cs application    ONLINE    UNKNOWN   rac1
ora....mo1.srv application    ONLINE    UNKNOWN   rac3
ora....mo2.srv application    ONLINE    UNKNOWN   rac1
ora....SM2.asm application    ONLINE    UNKNOWN   rac1
ora....C1.lsnr application    ONLINE    UNKNOWN   rac1
ora.rac1.gsd   application    ONLINE    UNKNOWN   rac1
ora.rac1.ons   application    ONLINE    UNKNOWN   rac1
ora.rac1.vip   application    ONLINE    ONLINE    rac1
ora....SM1.asm application    ONLINE    UNKNOWN   rac3
ora....C3.lsnr application    ONLINE    UNKNOWN   rac3
ora.rac3.gsd   application    ONLINE    UNKNOWN   rac3
ora.rac3.ons   application    ONLINE    UNKNOWN   rac3
ora.rac3.vip   application    ONLINE    ONLINE    rac3

[root@rac1 bin]# ./crsctl stop crs
Stopping resources.
Successfully stopped CRS resources
Stopping CSSD.
Shutting down CSS daemon.
Shutdown request successfully issued.

[root@rac3 bin]# ./crsctl stop crs
Stopping resources.
Successfully stopped CRS resources
Stopping CSSD.
Shutting down CSS daemon.
Shutdown request successfully issued.

[root@rac1 bin]# ./crsctl start crs
Attempting to start CRS stack
The CRS stack will be started shortly

[root@rac3 bin]# ./crsctl start crs
Attempting to start CRS stack
The CRS stack will be started shortly

[root@rac3 bin]# ./crs_stat -t
Name           Type           Target    State     Host
------------------------------------------------------------
ora.demo.db    application    ONLINE    ONLINE    rac3
ora....o1.inst application    ONLINE    ONLINE    rac3
ora....o2.inst application    ONLINE    ONLINE    rac1
ora....osrv.cs application    ONLINE    ONLINE    rac1
ora....mo1.srv application    ONLINE    ONLINE    rac3
ora....mo2.srv application    ONLINE    ONLINE    rac1
ora....SM2.asm application    ONLINE    ONLINE    rac1
ora....C1.lsnr application    ONLINE    ONLINE    rac1
ora.rac1.gsd   application    ONLINE    ONLINE    rac1
ora.rac1.ons   application    ONLINE    ONLINE    rac1
ora.rac1.vip   application    ONLINE    ONLINE    rac1
ora....SM1.asm application    ONLINE    ONLINE    rac3
ora....C3.lsnr application    ONLINE    ONLINE    rac3
ora.rac3.gsd   application    ONLINE    ONLINE    rac3
ora.rac3.ons   application    ONLINE    ONLINE    rac3
ora.rac3.vip   application    ONLINE    ONLINE    rac3


Due to entering wrong IP address also OU1-35000 Fatal error occurs:
figure 1: mismatching of ip addresses
correcting the ip address: after correcting the ip address , no fatal cluster error occurs, lets try  as show in below figure.
hope, it will helps you. 
regards,
Rajeshkumar.

Monday, January 4, 2010

Remove the Node from the Cluster in VMware

Now that the instance has been removed (and the ASM instance is applicable), we now need to remove the node from the cluster. This is a manual method performed using scripts that need to be run on the deleted node (if available) to remove the CRS install as well as scripts that should be run on any of the existing nodes (i.e. linux1).

Before proceeding to the steps for removing the node, we need to determine the node name and the CRS-assigned node number for each node stored in the Oracle Cluster Registry. This can be run from any of the existing nodes (linux1 for this example).

$ $ORA_CRS_HOME/bin/olsnodes -n
linux1  1
linux2  2
linux3  3

Now that we have the node name and node number, we can start the steps to remove the node from the cluster. Here are the steps that should be executed from a pre-existing (available) node in the cluster (i.e. linux1):

   1. Run the NETCA utility to remove the network configuration:

      $ DISPLAY=:0.0; export DISPLAY
      $ netca &

      Perform the following steps within the NETCA:

         1. Choose "Cluster Configuration" and click [Next].
         2. Only select the node you are removing and click [Next].
         3. Choose "Listener Configuration" and click [Next].
         4. Choose "Delete" and delete any listeners configured on the node you are removing. Acknowledge the dialog box to delete the listener configuration.

            NOTE: For some reason, I needed to login to linux3 and manually kill the process ID for the listener process.

   2. Run the crs_stat command to verify that all database resources are running on nodes that are going to be kept:

      $ $ORA_CRS_HOME/bin/crs_stat

      For example, verify that the node to be removed is not running any database resources. Look for the record of type:

      NAME=ora..db
      TYPE=application
      TARGET=ONLINE
      STAT=ONLINE on

      Assuming the name of the clustered database is orcl, this is the record that was returned from the crs_stat command on my system:

      NAME=ora.orcl.db
      TYPE=application
      TARGET=ONLINE
      STATE=ONLINE on linux1

      I am safe here since the resource is running on linux1 and not linux3 - the node I want to remove.

      If, however, the database resource was running on linux3, we would need to relocate it to a node that we are going to keep (i.e. linux1) using the following:

      $ $ORA_CRS_HOME/bin/crs_relocate ora..db

   3. From a pre-existing node (i.e. linux1), remove the nodeapps from the node you are removing as the root UNIX user account:

      $ su
      Password: xxxxx

      # srvctl stop nodeapps -n linux3
      CRS-0210: Could not find resource ora.linux3.LISTENER_LINUX3.lsnr.

      # srvctl remove nodeapps -n linux3
      Please confirm that you intend to remove the node-level applications on node linux3 (y/[n]) y
      #

   4. The next step is to update the node list using the updateNodeList option to the OUI as the oracle user. This procedure will remove the node to be deleted from the list of node locations maintained by the OUI by listing only those remaining nodes. The only file that I know of that gets modified is $ORACLE_BASE/oraInventory/ContentsXML/inventory.xml. Here is the command I used for removing linux3 from the list. Notice that the DISPLAY variable needs to be set even though the GUI does not run.

      $ DISPLAY=:0.0; export DISPLAY

      $ $ORACLE_HOME/oui/bin/runInstaller -ignoreSysPrereqs -updateNodeList \
      ORACLE_HOME=/u01/app/oracle/product/10.1.0/db_1 \
      CLUSTER_NODES=linux1,linux2

      Note that the command above will produce the following error which can safely be ignored:

      PRKC-1002 : All the submitted commands did not execute successfully

   5. If the node to be removed is still available and running the CRS stack, the DBA will need to stop the CRS stack and remove the ocr.loc file. These tasks should be performed as the root user account and on the node that is to be removed from the cluster. The nosharedvar option assumes the ocr.loc file is not on a shared file system (which is the case in my example). If the file does exist on a shared file system, then specify sharedvar. From the node to be removed (i.e. linux3) and as the root user, run the following:

      $ su
      Password: xxxx

      # cd $ORA_CRS_HOME/install
      # ./rootdelete.sh remote nosharedvar
      Running Oracle10 root.sh script...
      \nThe following environment variables are set as:
          ORACLE_OWNER= oracle
          ORACLE_HOME=  /u01/app/oracle/product/10.1.0/crs
      Finished running generic part of root.sh script.
      Now product-specific root actions will be performed.
      Shutting down Oracle Cluster Ready Services (CRS):
      /etc/init.d/init.crsd: line 188: 29017 Aborted                 $ORA_CRS_HOME/bin/crsd -2

      Shutting down CRS daemon.
      Shutting down EVM daemon.
      Shutting down CSS daemon.
      Shutdown request successfully issued.
      Checking to see if Oracle CRS stack is down...
      Oracle CRS stack is not running.
      Oracle CRS stack is down now.
      Removing script for Oracle Cluster Ready services
      Removing OCR location file '/etc/oracle/ocr.loc'
      Cleaning up SCR settings in '/etc/oracle/scls_scr/linux3'

   6. Next, using the node name and CRS-assigned node number for the node to be deleted, run the rootdeletenode.sh command as follows. Keep in mind that this command should be run from a pre-existing / available node (i.e. linux1) in the cluster as the root UNIX user account:

      $ su
      Password: xxxx

      # cd $ORA_CRS_HOME/install
      # ./rootdeletenode.sh linux3,3
      Running Oracle10 root.sh script...
      \nThe following environment variables are set as:
          ORACLE_OWNER= oracle
          ORACLE_HOME=  /u01/app/oracle/product/10.1.0/crs
      Finished running generic part of root.sh script.
      Now product-specific root actions will be performed.
      clscfg: EXISTING configuration version 2 detected.
      clscfg: version 2 is 10G Release 1.
      Successfully deleted 13 values from OCR.
      Key SYSTEM.css.interfaces.nodelinux3 marked for deletion is not there. Ignoring.
      Successfully deleted 5 keys from OCR.
      Node deletion operation successful.
      'linux3,3' deleted successfully

      To verify that the node was successfully removed, use the following as either the oracle or root user:

      $ $ORA_CRS_HOME/bin/olsnodes -n
      linux1  1
      linux2  2

   7. Now, switch back to the oracle UNIX user account on the same pre-existing node (linux1) and run the runInstaller command to update the OUI node list, however this time for the CRS installation ($ORA_CRS_HOME). This procedure will remove the node to be deleted from the list of node locations maintained by the OUI by listing only those remaining nodes. The only file that I know of that gets modified is $ORACLE_BASE/oraInventory/ContentsXML/inventory.xml. Here is the command I used for removing linux3 from the list. Notice that the DISPLAY variable needs to be set even though the GUI does not run.

      $ DISPLAY=:0.0; export DISPLAY

      $ $ORACLE_HOME/oui/bin/runInstaller -ignoreSysPrereqs -updateNodeList \
      ORACLE_HOME=/u01/app/oracle/product/10.1.0/crs \
      CLUSTER_NODES=linux1,linux2

      Note that each of the commands above will produce the following error which can safely be ignored:

      PRKC-1002 : All the submitted commands did not execute successfully

      The OUI now contains the valid nodes that are part of the cluster!

   8. Now that the node has been removed from the cluster, the DBA should manually remove all Oracle10g RAC installation files from the deleted node. Obviously, this applies only if the removed node is still accessible and only if the files are not on a shared file system that is still being accessed by other nodes in the cluster!

      From the deleted node (linux3) I performed the following tasks as the root UNIX user account:
         1. Remove ORACLE_HOME and ORA_CRS_HOME:

            # rm -rf /u01/app/oracle/product/10.1.0/db_1
            # rm -rf /u01/app/oracle/product/10.1.0/crs

         2. Remove all init scripts and soft links (for Linux). For a list of init scripts and soft links for other UNIX platforms, see Metalink Note: 269320.1

            # rm -f /etc/init.d/init.cssd
            # rm -f /etc/init.d/init.crs
            # rm -f /etc/init.d/init.crsd
            # rm -f /etc/init.d/init.evmd
            # rm -f /etc/rc2.d/K96init.crs
            # rm -f /etc/rc2.d/S96init.crs
            # rm -f /etc/rc3.d/K96init.crs
            # rm -f /etc/rc3.d/S96init.crs
            # rm -f /etc/rc5.d/K96init.crs
            # rm -f /etc/rc5.d/S96init.crs
            # rm -Rf /etc/oracle/scls_scr

         3. Remove all remaining files:

            # rm -rf /etc/oracle
            # rm -f /etc/oratab
            # rm -f /etc/oraInst.loc
            # rm -rf /etc/ORCLcluster
            # rm -rf /u01/app/oracle/oraInventory
            # rm -rf /u01/app/oracle/product
            # rm -rf /u01/app/oracle/admin
            # rm -f /usr/local/bin/coraenv
            # rm -f /usr/local/bin/dbhome
            # rm -f /usr/local/bin/oraenv

         4. Remove all CRS/EVM entries from the file /etc/inittab:

            h1:35:respawn:/etc/init.d/init.evmd run >/dev/null 2>&1

            h2:35:respawn:/etc/init.d/init.cssd fatal >/dev/null 2>&1

            h3:35:respawn:/etc/init.d/init.crsd run >/dev/null 2>&1


regards,
rajeshkumar g

Saturday, December 26, 2009

CRS-0215: Could not start resource 'ora.rac2.LISTENER_RAC2.lsnr'.

problem:
[root@rac1 bin]# ./srvctl start listener -n rac2
CRS-0215: Could not start resource 'ora.rac2.LISTENER_RAC2.lsnr'.

solution:
check /etc/hosts on both nodes
check /etc/sysconfig/network on both nodes

correct it if necessary if u found any mismatch between both nodes
i didnt mention the default gateway ip address for rac2 machine
i corrected the gateway address for node 2 machine
[oracle@rac2 ~]$ su -
Password:
[root@rac2 ~]# vi /etc/sysconfig/network
[root@rac2 ~]# service network restart
Shutting down interface eth0: [ OK ]
Shutting down interface eth1: [ OK ]
Shutting down loopback interface: [ OK ]
Bringing up loopback interface: [ OK ]
Bringing up interface eth0: [ OK ]
Bringing up interface eth1: [ OK ]
[root@rac2 ~]# cd /u01/crs/oracle/product/11.1.0/crs/bin/
[root@rac2 bin]# ./crsctl start crs
Attempting to start Oracle Clusterware stack
The CRS stack will be started shortly
[root@rac2 bin]# ./crs_stat -t
Name Type Target State Host
------------------------------------------------------------
ora....SM1.asm application ONLINE ONLINE rac1
ora....C1.lsnr application ONLINE ONLINE rac1
ora....C1.lsnr application ONLINE ONLINE rac1
ora.rac1.gsd application ONLINE ONLINE rac1
ora.rac1.ons application ONLINE ONLINE rac1
ora.rac1.vip application ONLINE ONLINE rac1
ora....SM2.asm application ONLINE ONLINE rac2
ora....C2.lsnr application ONLINE OFFLINE
ora....C2.lsnr application ONLINE OFFLINE
ora.rac2.gsd application ONLINE ONLINE rac2
ora.rac2.ons application ONLINE ONLINE rac2
ora.rac2.vip application ONLINE ONLINE rac1
[root@rac2 bin]# ./srvctl start nodeapps -n rac2
[root@rac2 bin]# ./crs_stat -t
Name Type Target State Host
------------------------------------------------------------
ora....SM1.asm application ONLINE ONLINE rac1
ora....C1.lsnr application ONLINE ONLINE rac1
ora....C1.lsnr application ONLINE ONLINE rac1
ora.rac1.gsd application ONLINE ONLINE rac1
ora.rac1.ons application ONLINE ONLINE rac1
ora.rac1.vip application ONLINE ONLINE rac1
ora....SM2.asm application ONLINE ONLINE rac2
ora....C2.lsnr application ONLINE ONLINE rac2
ora....C2.lsnr application ONLINE ONLINE rac2
ora.rac2.gsd application ONLINE ONLINE rac2
ora.rac2.ons application ONLINE ONLINE rac2
ora.rac2.vip application ONLINE ONLINE rac2
[root@rac2 bin]#

problem solved,

if got error,
pls follow the steps mentioned in metalink

How to Configure Virtual IPs for 10g RAC [ID 264847.1]
NOTE:264847.1

regards,
rajeshkumar g

How to ADD/REMOVE/REPLACE/MOVE Oracle Cluster Registry (OCR) and Voting Disk

Purpose
To provide step to add, remove, replace or move an OCR or voting disk in Oracle
Real Application Clusters (RAC) 10g Realese 2 (10.2.0.1 and later)
environments.
SCOPE & APPLICATION
This article is intended for DBAs and Support Engineers who need to modify, or
move OCR and voting disk locations. Customers, who have an existing clustered
environment deployed on a storage array, might want to migrate to a new storage
array, with minimal downtime. Typically, one would simply cp or dd the files
once the new storage has been presented to the hosts. In this case, it is a
little more difficult because,
1. The clusterware has the OCR and voting disks open and is actively using
them. (Both the primary and the mirrors)
2. There is a “cluster api” provided for this function (ocrconfig, and
crsctl), which is a better interface than typical cp and/or dd commands.
It is highly recommended to take a backup of the voting disk, and OCR device
before making any changes.
Examples are included at the end of OCR and voting disk command.
ADD/REMOVE/REPLACE OCR Device
Note: You must be logged in as the root user, because root own the OCR files.
Make sure there is a recent copy of the OCR file before making any changes:
ocrconfig –showbackup
If there is not a recent backup copy of the OCR file, an export can be taken
for the current OCR file. Use the following command to generate a an export of
the online OCR file:
ocrconfig –export -s online
If you need to recover using this file, the following command can be used:
ocrconfig import
1. To add an OCR device:
To add an OCR device, provide the full path including file name.
ocrconfig -replace ocr
To add an OCR mirror device, provide the full path including file name.
ocrconfig -replace ocrmirror
2. To remove an OCR device:
To remove an OCR device:
ocrconfig -replace ocr
To remove an OCR mirror device
ocrconfig -replace ocrmirror
3. To replace or move the location of an OCR device:
To replace the OCR device with , provide the full path including file
name.
ocrconfig -replace ocr
To replace the OCR mirror device with , provide the full path
including file name.
ocrconfig -replace ocrmirror
Example moving OCR file from OCFS to raw devices
The OCR disk must be owned by root, must be in the oinstall group, and must have permissions set
to 640. Provide at least 100 MB disk space for the OCR.
In this example the OCR file are located in the ocfs2 file system:
/ocfs2/ocr1
/ocfs2/ocr2
Create raw device files of at least 100 MB. In this example the new OCR file
will be on the following devices:
/dev/raw/raw1
/dev/raw/raw2
Once the raw devices are created, use the dd command to zero out the device and
make sure no data is written to the raw devices:
dd if=/dev/zero of=/dev/raw/raw1
dd if=/dev/zero of=/dev/raw/raw2
Note: Use UNIX man pages for additional information on the dd command.
Now you are ready to move/replace the OCR file to the new storage location.
Move/Replace the OCR device
ocrconfig -replace ocr /dev/raw/raw1
Add /dev/raw/raw2 as OCR mirror device
ocrconfig -replace ocr /dev/raw/raw2
Example of adding an OCR device file
If you have upgraded your environment from a previous version, where you only
had 1 OCR device file, you can use the following step to add an additional OCR
file.
In this example a second OCR device file is added:
Add /dev/raw/raw2 as OCR mirror device
ocrconfig -replace ocr /dev/raw/raw2
ADD/DELETE/MOVE Voting Disk
Note: Shutdown the Oracle Clusterware (crsctl stop crs as root) on all nodes
before making any modification to the voting disk. Determine the current
voting disk location using:
crsctl query css votedisk
Take a backup of all voting disk:
dd if=voting_disk_name of=backup_file_name
Note: Use UNIX man pages for additional information on the dd command.
The following can be used to restore the voting disk from the backup file
created.
dd if=backup_file_name of=voting_disk_name
1. To add a Voting Disk, provide the full path including file name.:
crsctl add votedisk css -force
2. To delete a Voting Disk, provide the full path including file name.:
crsctl delete votedisk css -force
3. To move a Voting Disk, provide the full path including file name.:
crsctl delete votedisk css –force
crsctl add votedisk css –force
After modifying the voting disk, start the Oracle Clusterware stack on all
nodes
crsctl start crs
Verify the voting disk location using
crsctl query css votedisk
Example moving Voting Disk from OCFS to raw devices
The voting disk is a partition that Oracle Clusterware uses to verify cluster
node membership and status.
The voting disk must be owned by the oracle user, must be in the dba
group, and must have permissions set to 644. Provide at least 20 MB disk
space for the voting disk.
In this example the Voting Disks are located in the ocfs2 file system:
/ocfs2/voting1
/ocfs2/voting2
/ocfs2/voting3
Create raw device files of at least 20 MB. In this example the new voting
disks will be on the following devices:
/dev/raw/raw3
/dev/raw/raw4
/dev/raw/raw5
Once the raw devices are created, use the dd command to zero out the device and
make sure no data is written to the raw devices:
dd if=/dev/zero of=/dev/raw/raw3
dd if=/dev/zero of=/dev/raw/raw4
dd if=/dev/zero of=/dev/raw/raw5
Note: Use UNIX man pages for additional information on the dd command.
Now you are ready to move/replace the voting disks to the new storage location.
To move a Voting Disk to new storage location:
crsctl delete votedisk css /ocfs2/voting1 –force
crsctl add votedisk css /dev/raw/raw3 –force
crsctl delete votedisk css /ocfs2/voting2 –force
crsctl add votedisk css /dev/raw/raw4 –force
crsctl delete votedisk css /ocfs2/voting3 –force
crsctl add votedisk css /dev/raw/raw5 –force
Example of adding Voting Disks
If you have upgraded your environment from a previous version, where you only
had 1 voting disk, you can use the following steps to add additional voting
disk.
In this example 2 additional Voting Disks are added:
crsctl add votedisk css /dev/raw/raw4 –force
crsctl add votedisk css /dev/raw/raw5 –force
After modifying the voting disk, start the Oracle Clusterware stack on all
nodes
crsctl start crs
Verify the voting disk location using
crsctl query css votedisk

Administering OCR in RAC

What is a voting disk?
A voting disk is a file that manages information about node membership.
What are the administrative tasks involved with voting disk?
Following administrative tasks are performed with the voting disk :
1) Backing up voting disks
2) Recovering Voting disks
3) Adding voting disks
4) Deleting voting disks
5) Moving voting disks
How do we backup voting disks?
1) Oracle recommends that you back up your voting disk after the initial cluster creation and after we complete any node addition or deletion procedures.
2) First, as root user, stop Oracle Clusterware (with the crsctl stop crs command) on all nodes. Then, determine the current voting disk by issuing the following command:
crsctl query css votedisk
3) Then, issue the dd or ocopy command to back up a voting disk, as appropriate.
Give the syntax of backing up voting disks:-
On Linux or UNIX systems:
dd if=voting_disk_name of=backup_file_name
where,
voting_disk_name is the name of the active voting disk
backup_file_name is the name of the file to which we want to back up the voting disk contents
On Windows systems, use the ocopy command:
ocopy voting_disk_name backup_file_name
What is the Oracle Recommendation for backing up voting disk?
Oracle recommends us to use the dd command to backup the voting disk with aminimum block size of 4KB.
How do you restore a voting disk?
To restore the backup of your voting disk, issue the dd or ocopy command for Linux and UNIX systems or ocopy for Windows systems respectively.
On Linux or UNIX systems:
dd if=backup_file_name of=voting_disk_name
On Windows systems, use the ocopy command:
ocopy backup_file_name voting_disk_name
where,
backup_file_name is the name of the voting disk backup file
voting_disk_name is the name of the active voting disk
How can we add and remove multiple voting disks?
If we have multiple voting disks, then we can remove the voting disks and add them back into our environment using the following commands, where path is the complete path of the location where the voting disk resides:
crsctl delete css votedisk path
crsctl add css votedisk path
How do we stop Oracle Clusterware?When do we stop it?
Before making any modification to the voting disk, as root user, stop Oracle Clusterware using the crsctl stop crs command on all nodes.
How do we add voting disk?
To add a voting disk, issue the following command as the root user, replacing the path variable with the fully qualified path name for the voting disk we want to add:
crsctl add css votedisk path -force
How do we move voting disks?
To move a voting disk, issue the following commands as the root user, replacing the path variable with the fully qualified path name for the voting disk we want to move:
crsctl delete css votedisk path -force
crsctl add css votedisk path -force
How do we remove voting disks?
To remove a voting disk, issue the following command as the root user, replacing the path variable with the fully qualified path name for the voting disk we want to remove:
crsctl delete css votedisk path -force
What should we do after modifying voting disks?
After modifying the voting disk, restart Oracle Clusterware using the crsctl start crs command on all nodes, and verify the voting disk location using the following command:
crsctl query css votedisk
When can we use -force option?
If our cluster is down, then we can include the -force option to modify the voting disk configuration, without interacting with active Oracle Clusterware daemons. However, using the -force option while any cluster node is active may corrupt our configuration.

OCRCHECK in CLUSTERWARE, RAC

What is an OCRCHECK utility?
An ocrcheck utility is a diagnostic tool used for diagnosing OC(Oracle Cluster Registry) Problems.This is used to verify the Oracle Cluster Registry(OCR) integrity.
What does an ocrcheck display?
The OCRCHECK utility displays the version of the OCR’s block format, total space available and used space, OCRID, and the OCR locations that we have configured.
How does ocrcheck perform integrity check?
OCRCHECK performs a block-by-block checksum operation for all of the blocks in all of the OCRs that we have configured. It also returns an individual status for each file as well as a result for the overall OCR integrity check.
Give a sample output of ocrcheck utility:-
Sample of the OCRCHECK utility output:
Status of Oracle Cluster Registry is as follows :
Version : 2
Total space (kbytes) : 262144
Used space (kbytes) : 16256
Available space (kbytes) : 245888
ID : 1918913332
Device/File Name : /dev/raw/raw1
Device/File integrity check succeeded
Device/File Name : /dev/raw/raw2
Device/File integrity check succeeded
Cluster registry integrity check succeeded

Where does an ocrcheck utility create a log file?
OCRCHECK creates a log file in the directory CRS_home/log/hostname/client.
How can we change the amount of logging?
To change amount of logging, edit the file CRS_home/srvm/admin/ocrlog.ini.


Administering Voting Disks in Oracle Real Application Clusters

The Oracle Clusterware is comprised primarily of two components: the voting disk and the OCR (Oracle Cluster Registry). The voting disk is nothing but a file that contains and manages information of all the node memberships and the OCR is a file that manages the cluster and RAC configuration. Let's take a quick look at administering the voting disks

Administering voting disks

Backup and Recovery:

First, let's look at backing up the voting disks by running the following command:

dd if=voting_disk_name of=backup_file_name
example:
dd if=/dev/sdc1 of=/tmp/votingdisk_bkp
2088387+0 records in
2088387+0 records out
1069254144 bytes (1.1 GB) copied, 889.716 seconds, 1.2 MB/s

This operation needs to be performed on all voting disks. Here, clearly you see that the if (input file) is the source file (replace the voting_disk_name with your voting disk) and the of (output file) is the destination backup file containing all information of the voting disk contents. Type dd –help for more information. Running the command with the names of the files reversed will help you recover your voting disk file(s).

dd if=backup_file_name of=voting_disk_name

note:
before restore, stop the crs with the following command:
[root@rac1 bin]# /etc/init.d/init.crs stop

You can use the ocopy command in Windows environments or use the crsctl commands to copy and administer the files. Also, note that if you have multiple voting disks, which are not abnormal to have, you can use the crsctl command to add and delete the voting disks. For instance:

crsctl delete css votedisk path

Here you delete the disk and the path, which is the complete path of the location of the file, and below you add your new or backup files by doing the following:

crsctl add css votedisk path

This way you can either statically or dynamically add or remove your voting disks in your RAC.

You must, however, note that if your cluster is down, then you can use the -force option

crsctl add css votedisk path -force

to modify the voting disk configuration. This way you don’t end up interfering with other Clusterware daemons. Using it in your active configuration may corrupt your configuration.

regards,

rajeshkumar g


free counters
 
Share/Bookmark