Showing posts with label RAC. Show all posts
Showing posts with label RAC. Show all posts

Thursday, January 3, 2013

Restore Database RAC from multiple Instances

Labels: , , , , ,
This is restore & recovery of RAC database running on 3 nodes.

While restore & recover database we allocate channel to all Instances for Load balance.

Oracle Database version is 11.2.0.3.

Steps to perform restore & recovery.

Restore spfile
Restore controlfile
Bring all the instances in mount state (do not put cluster_database=false)
Allocate channel using multiple instances
Restore database
Recover database
Open the database in resetlogs in one node
Open the database on other nodes 

---------RESTORE SPFILE 

TEST271 | DUMMYGG1 | /export/home/oracle/scripts/rman
> rman target / catalog rman/RMAN@catalogdb

Recovery Manager: Release 11.2.0.3.0 - Production on Thu Jan 3 12:06:05 2013

Copyright (c) 1982, 2011, Oracle and/or its affiliates.  All rights reserved.

connected to target database (not started)
connected to recovery catalog database

RMAN> set dbid=90275615

executing command: SET DBID
database name is "DUMMYGG" and DBID is 90275615

RMAN> list backup of spfile;


List of Backup Sets
===================


BS Key  Type LV Size       Device Type Elapsed Time Completion Time
------- ---- -- ---------- ----------- ------------ ---------------
7639172 Incr 0  80.00K     DISK        00:00:01     02-JAN-13
        BP Key: 7639190   Status: AVAILABLE  Compressed: NO  Tag: TAG20130102T121926
        Piece Name: /ORAGG/backup/DUMMYGG15_12nudf6q_1_1
  SPFILE Included: Modification time: 02-JAN-13
  SPFILE db_unique_name: DUMMYGG

BS Key  Type LV Size       Device Type Elapsed Time Completion Time
------- ---- -- ---------- ----------- ------------ ---------------
7640203 Incr 0  80.00K     DISK        00:00:01     02-JAN-13
        BP Key: 7640224   Status: AVAILABLE  Compressed: NO  Tag: TAG20130102T124658
        Piece Name: /ORAGG/backup/DUMMYGG05_2gnudgqg_1_1
  SPFILE Included: Modification time: 02-JAN-13
  SPFILE db_unique_name: DUMMYGG

RMAN> STARTUP FORCE NOMOUNT;
  
RMAN> restore spfile from '/ORAGG/backup/DUMMYGG05_2gnudgqg_1_1';

Starting restore at 03-JAN-13
using channel ORA_DISK_1

channel ORA_DISK_1: restoring spfile from AUTOBACKUP /ORAGG/backup/DUMMYGG05_2gnudgqg_1_1
channel ORA_DISK_1: SPFILE restore from AUTOBACKUP complete
Finished restore at 03-JAN-13

RMAN> exit

Now copy spfile to all nodes & rename according to it.


------Restore controlfile 

> rman target / catalog rman/RMAN@catalogdb

Recovery Manager: Release 11.2.0.3.0 - Production on Thu Jan 3 12:14:22 2013

Copyright (c) 1982, 2011, Oracle and/or its affiliates.  All rights reserved.

connected to target database: DUMMYGG (not mounted)
connected to recovery catalog database

RMAN> set dbid=90275615;

executing command: SET DBID
database name is "DUMMYGG" and DBID is 90275615

RMAN> restore controlfile;

Starting restore at 03-JAN-13
allocated channel: ORA_DISK_1
channel ORA_DISK_1: SID=156 instance=DUMMYGG1 device type=DISK

channel ORA_DISK_1: restoring control file
channel ORA_DISK_1: copied control file copy
input file name=/u00/app/oracle/diag/control.ctl
output file name=+TESTGG/DUMMYGG/controlfile/current.958.803736875
Finished restore at 03-JAN-13
----------Mount all Instance.

Mount database on All nodes & make sure cluster_database parameter is set to TRUE

----------Restore database. 

rman target / catalog rman/RMAN@catalogdb

run {
sql 'alter system set "_backup_disk_bufcnt"=64 scope=memory';
sql 'alter system set "_backup_disk_bufsz"=1048576 scope=memory';
sql 'alter system set "_backup_file_bufcnt"=64 scope=memory';
sql 'alter system set "_backup_file_bufsz"=1048576 scope=memory';
allocate channel c01 DEVICE TYPE DISK FORMAT '/ORAGG/backup/DUMMYGG01_%U' CONNECT 'sys/SYSTEM@DUMMYGG1';
allocate channel c02 DEVICE TYPE DISK FORMAT '/ORAGG/backup/DUMMYGG02_%U' CONNECT 'sys/SYSTEM@DUMMYGG1';
allocate channel c03 DEVICE TYPE DISK FORMAT '/ORAGG/backup/DUMMYGG03_%U' CONNECT 'sys/SYSTEM@DUMMYGG1';
allocate channel c04 DEVICE TYPE DISK FORMAT '/ORAGG/backup/DUMMYGG04_%U' CONNECT 'sys/SYSTEM@DUMMYGG1';
allocate channel c05 DEVICE TYPE DISK FORMAT '/ORAGG/backup/DUMMYGG05_%U' CONNECT 'sys/SYSTEM@DUMMYGG1';
allocate channel c06 DEVICE TYPE DISK FORMAT '/ORAGG/backup/DUMMYGG06_%U' CONNECT 'sys/SYSTEM@DUMMYGG1';
allocate channel c07 DEVICE TYPE DISK FORMAT '/ORAGG/backup/DUMMYGG07_%U' CONNECT 'sys/SYSTEM@DUMMYGG2';
allocate channel c08 DEVICE TYPE DISK FORMAT '/ORAGG/backup/DUMMYGG08_%U' CONNECT 'sys/SYSTEM@DUMMYGG2';
allocate channel c09 DEVICE TYPE DISK FORMAT '/ORAGG/backup/DUMMYGG09_%U' CONNECT 'sys/SYSTEM@DUMMYGG2';
allocate channel c10 DEVICE TYPE DISK FORMAT '/ORAGG/backup/DUMMYGG10_%U' CONNECT 'sys/SYSTEM@DUMMYGG2';
allocate channel c11 DEVICE TYPE DISK FORMAT '/ORAGG/backup/DUMMYGG11_%U' CONNECT 'sys/SYSTEM@DUMMYGG2';
allocate channel c12 DEVICE TYPE DISK FORMAT '/ORAGG/backup/DUMMYGG12_%U' CONNECT 'sys/SYSTEM@DUMMYGG2';
allocate channel c13 DEVICE TYPE DISK FORMAT '/ORAGG/backup/DUMMYGG13_%U' CONNECT 'sys/SYSTEM@DUMMYGG3';
allocate channel c14 DEVICE TYPE DISK FORMAT '/ORAGG/backup/DUMMYGG14_%U' CONNECT 'sys/SYSTEM@DUMMYGG3';
allocate channel c15 DEVICE TYPE DISK FORMAT '/ORAGG/backup/DUMMYGG15_%U' CONNECT 'sys/SYSTEM@DUMMYGG3';
allocate channel c16 DEVICE TYPE DISK FORMAT '/ORAGG/backup/DUMMYGG16_%U' CONNECT 'sys/SYSTEM@DUMMYGG3';
allocate channel c17 DEVICE TYPE DISK FORMAT '/ORAGG/backup/DUMMYGG17_%U' CONNECT 'sys/SYSTEM@DUMMYGG3';
allocate channel c18 DEVICE TYPE DISK FORMAT '/ORAGG/backup/DUMMYGG18_%U' CONNECT 'sys/SYSTEM@DUMMYGG3';
restore database;
release channel c01;
release channel c02;
release channel c03;
release channel c04;
release channel c05;
release channel c06;
release channel c07;
release channel c08;
release channel c09;
release channel c10;
release channel c11;
release channel c12;
release channel c13;
release channel c14;
release channel c15;
release channel c16;
release channel c17;
release channel c18;
}

---------recover database. 

RMAN> run {
sql 'alter system set "_backup_disk_bufcnt"=64 scope=memory';
sql 'alter system set "_backup_disk_bufsz"=1048576 scope=memory';
sql 'alter system set "_backup_file_bufcnt"=64 scope=memory';
sql 'alter system set "_backup_file_bufsz"=1048576 scope=memory';
allocate channel c01 DEVICE TYPE DISK FORMAT '/ORAGG/backup/' CONNECT 'sys/SYSTEM@DUMMYGG1';
allocate channel c07 DEVICE TYPE DISK FORMAT '/ORAGG/backup/' CONNECT 'sys/SYSTEM@DUMMYGG2';
allocate channel c13 DEVICE TYPE DISK FORMAT '/ORAGG/backup/' CONNECT 'sys/SYSTEM@DUMMYGG3';
set until time "to_date('2013-01-02:13:00:00', 'yyyy-mm-dd:hh24:mi:ss')";
recover database;
release channel c01;
release channel c07;
release channel c13;
}

---------Open database in resetlogs on first instance & rest with normal. 

TEST271 | DUMMYGG1 | /export/home/oracle/scripts/rman/log
sqlplus / as sysdba

SQL*Plus: Release 11.2.0.3.0 Production on Thu Jan 3 12:46:30 2013

Copyright (c) 1982, 2011, Oracle.  All rights reserved.


Connected to:
Oracle Database 11g Enterprise Edition Release 11.2.0.3.0 - 64bit Production
With the Partitioning, Real Application Clusters, Automatic Storage Management, OLAP,
Data Mining and Real Application Testing options

12:47:01 @DUMMYGG1 SQL>alter database open resetlogs;

Database altered.

Elapsed: 00:00:14.39

TEST272 | DUMMYGG2 | /export/home/oracle
sqlplus / as sysdba

SQL*Plus: Release 11.2.0.3.0 Production on Thu Jan 3 12:47:43 2013

Copyright (c) 1982, 2011, Oracle.  All rights reserved.


Connected to:
Oracle Database 11g Enterprise Edition Release 11.2.0.3.0 - 64bit Production
With the Partitioning, Real Application Clusters, Automatic Storage Management, OLAP,
Data Mining and Real Application Testing options

12:47:43 @DUMMYGG2 SQL>alter database open;

Database altered.

Elapsed: 00:00:03.92
12:47:50 @DUMMYGG2 SQL>

---------output of restore 

sql statement: alter system set "_backup_disk_bufcnt"=64 scope=memory

sql statement: alter system set "_backup_disk_bufsz"=1048576 scope=memory

sql statement: alter system set "_backup_file_bufcnt"=64 scope=memory

sql statement: alter system set "_backup_file_bufsz"=1048576 scope=memory

allocated channel: c01
channel c01: SID=222 instance=DUMMYGG1 device type=DISK

allocated channel: c02
channel c02: SID=5 instance=DUMMYGG1 device type=DISK

allocated channel: c03
channel c03: SID=68 instance=DUMMYGG1 device type=DISK

allocated channel: c04
channel c04: SID=99 instance=DUMMYGG1 device type=DISK

allocated channel: c05
channel c05: SID=129 instance=DUMMYGG1 device type=DISK

allocated channel: c06
channel c06: SID=162 instance=DUMMYGG1 device type=DISK

allocated channel: c07
channel c07: SID=67 instance=DUMMYGG2 device type=DISK

allocated channel: c08
channel c08: SID=161 instance=DUMMYGG2 device type=DISK

allocated channel: c09
channel c09: SID=191 instance=DUMMYGG2 device type=DISK

allocated channel: c10
channel c10: SID=222 instance=DUMMYGG2 device type=DISK

allocated channel: c11
channel c11: SID=5 instance=DUMMYGG2 device type=DISK

allocated channel: c12
channel c12: SID=38 instance=DUMMYGG2 device type=DISK

allocated channel: c13
channel c13: SID=134 instance=DUMMYGG3 device type=DISK

allocated channel: c14
channel c14: SID=72 instance=DUMMYGG3 device type=DISK

allocated channel: c15
channel c15: SID=133 instance=DUMMYGG3 device type=DISK

allocated channel: c16
channel c16: SID=194 instance=DUMMYGG3 device type=DISK

allocated channel: c17
channel c17: SID=10 instance=DUMMYGG3 device type=DISK

allocated channel: c18
channel c18: SID=74 instance=DUMMYGG3 device type=DISK

Starting restore at 03-JAN-13

channel c01: starting datafile backup set restore
channel c01: specifying datafile(s) to restore from backup set
channel c01: restoring datafile 00003 to +TESTGG/DUMMYGG/datafile/undotbs1.258.792518685
channel c01: reading from backup piece /ORAGG/backup/DUMMYGG05_22nudgq6_1_1
channel c02: starting datafile backup set restore
channel c02: specifying datafile(s) to restore from backup set
channel c02: restoring datafile 00005 to +TESTGG/DUMMYGG/datafile/undotbs2.264.792518777
channel c02: reading from backup piece /ORAGG/backup/DUMMYGG06_23nudgq6_1_1
channel c03: starting datafile backup set restore
channel c03: specifying datafile(s) to restore from backup set
channel c03: restoring datafile 00006 to +TESTGG/DUMMYGG/datafile/undotbs3.265.792518777
channel c03: reading from backup piece /ORAGG/backup/DUMMYGG07_24nudgq6_1_1
channel c04: starting datafile backup set restore
channel c04: specifying datafile(s) to restore from backup set
channel c04: restoring datafile 00013 to +TESTGG/DUMMYGG/datafile/c2oindx_tbs.974.799865727
channel c04: reading from backup piece /ORAGG/backup/DUMMYGG08_25nudgq8_1_1
channel c05: starting datafile backup set restore
channel c05: specifying datafile(s) to restore from backup set
channel c05: restoring datafile 00014 to +TESTGG/DUMMYGG/datafile/c2odata_tbs.939.799865745
channel c05: reading from backup piece /ORAGG/backup/DUMMYGG09_26nudgqa_1_1
channel c06: starting datafile backup set restore
channel c06: specifying datafile(s) to restore from backup set
channel c06: restoring datafile 00009 to +TESTGG/DUMMYGG/datafile/idm_data.344.793627825
channel c06: reading from backup piece /ORAGG/backup/DUMMYGG10_27nudgqa_1_1
channel c07: starting datafile backup set restore
channel c07: specifying datafile(s) to restore from backup set
channel c07: restoring datafile 00015 to +TESTGG/DUMMYGG/datafile/c2odata_tbs_reorg1.948.799865763
channel c07: reading from backup piece /ORAGG/backup/DUMMYGG14_2bnudgqc_1_1
channel c08: starting datafile backup set restore
channel c08: specifying datafile(s) to restore from backup set
channel c08: restoring datafile 00016 to +TESTGG/DUMMYGG/datafile/mvlog_data.735.799865763
channel c08: reading from backup piece /ORAGG/backup/DUMMYGG15_2cnudgqc_1_1
channel c09: starting datafile backup set restore
channel c09: specifying datafile(s) to restore from backup set
channel c09: restoring datafile 00017 to +TESTGG/DUMMYGG/datafile/c2odata_tbs_reorg0.305.799865765
channel c09: reading from backup piece /ORAGG/backup/DUMMYGG16_2dnudgqe_1_1
channel c10: starting datafile backup set restore
channel c10: specifying datafile(s) to restore from backup set
channel c10: restoring datafile 00004 to +TESTGG/DUMMYGG/datafile/users.259.792518685
channel c10: reading from backup piece /ORAGG/backup/DUMMYGG18_2fnudgqg_1_1
channel c11: starting datafile backup set restore
channel c11: specifying datafile(s) to restore from backup set
channel c11: restoring datafile 00007 to +TESTGG/DUMMYGG/datafile/gg_data.312.793368947
channel c11: reading from backup piece /ORAGG/backup/DUMMYGG11_28nudgqb_1_1
channel c12: starting datafile backup set restore
channel c12: specifying datafile(s) to restore from backup set
channel c12: restoring datafile 00011 to +TESTGG/DUMMYGG/datafile/soa_data.328.795
channel c12: reading from backup piece /ORAGG/backup/DUMMYGG13_2anudgqc_1_1
channel c13: starting datafile backup set restore
channel c13: specifying datafile(s) to restore from backup set
channel c13: restoring datafile 00001 to +TESTGG/DUMMYGG/datafile/system.256.79251
channel c13: reading from backup piece /ORAGG/backup/DUMMYGG12_29nudgqb_1_1
channel c14: starting datafile backup set restore
channel c14: specifying datafile(s) to restore from backup set
channel c14: restoring datafile 00012 to +TESTGG/DUMMYGG/datafile/taps_data.617.79
channel c14: reading from backup piece /ORAGG/backup/DUMMYGG03_20nudgq5_1_1
channel c15: starting datafile backup set restore
channel c15: specifying datafile(s) to restore from backup set
channel c15: restoring datafile 00002 to +TESTGG/DUMMYGG/datafile/sysaux.257.79251
channel c15: reading from backup piece /ORAGG/backup/DUMMYGG04_21nudgq6_1_1
channel c16: starting datafile backup set restore
channel c16: specifying datafile(s) to restore from backup set
channel c16: restoring datafile 00008 to +TESTGG/DUMMYGG/datafile/identity_data.34
channel c16: reading from backup piece /ORAGG/backup/DUMMYGG01_1unudgq5_1_1
channel c17: starting datafile backup set restore
channel c17: specifying datafile(s) to restore from backup set
channel c17: restoring datafile 00010 to +TESTGG/DUMMYGG/datafile/taps_data.329.79
channel c17: reading from backup piece /ORAGG/backup/DUMMYGG02_1vnudgq5_1_1
channel c07: piece handle=/ORAGG/backup/DUMMYGG14_2bnudgqc_1_1 tag=TAG20130102T124658
channel c07: restored backup piece 1
channel c07: restore complete, elapsed time: 00:00:05
channel c08: piece handle=/ORAGG/backup/DUMMYGG15_2cnudgqc_1_1 tag=TAG20130102T124658
channel c08: restored backup piece 1
channel c08: restore complete, elapsed time: 00:00:05
channel c09: piece handle=/ORAGG/backup/DUMMYGG16_2dnudgqe_1_1 tag=TAG20130102T124658
channel c09: restored backup piece 1
channel c09: restore complete, elapsed time: 00:00:05
channel c10: piece handle=/ORAGG/backup/DUMMYGG18_2fnudgqg_1_1 tag=TAG20130102T124658
channel c10: restored backup piece 1
channel c10: restore complete, elapsed time: 00:00:17
channel c11: piece handle=/ORAGG/backup/DUMMYGG11_28nudgqb_1_1 tag=TAG20130102T124658
channel c11: restored backup piece 1
channel c11: restore complete, elapsed time: 00:00:17
channel c12: piece handle=/ORAGG/backup/DUMMYGG13_2anudgqc_1_1 tag=TAG20130102T124658
channel c12: restored backup piece 1
channel c12: restore complete, elapsed time: 00:00:17
channel c13: piece handle=/ORAGG/backup/DUMMYGG12_29nudgqb_1_1 tag=TAG20130102T124658
channel c13: restored backup piece 1
channel c13: restore complete, elapsed time: 00:00:37
channel c06: piece handle=/ORAGG/backup/DUMMYGG10_27nudgqa_1_1 tag=TAG20130102T124658
channel c06: restored backup piece 1
channel c06: restore complete, elapsed time: 00:00:57
channel c01: piece handle=/ORAGG/backup/DUMMYGG05_22nudgq6_1_1 tag=TAG20130102T124658
channel c01: restored backup piece 1
channel c01: restore complete, elapsed time: 00:01:58
channel c02: piece handle=/ORAGG/backup/DUMMYGG06_23nudgq6_1_1 tag=TAG20130102T124658
channel c02: restored backup piece 1
channel c02: restore complete, elapsed time: 00:02:48
channel c03: piece handle=/ORAGG/backup/DUMMYGG07_24nudgq6_1_1 tag=TAG20130102T124658
channel c03: restored backup piece 1
channel c03: restore complete, elapsed time: 00:02:48
channel c04: piece handle=/ORAGG/backup/DUMMYGG08_25nudgq8_1_1 tag=TAG20130102T124658
channel c04: restored backup piece 1
channel c04: restore complete, elapsed time: 00:02:48
channel c05: piece handle=/ORAGG/backup/DUMMYGG09_26nudgqa_1_1 tag=TAG20130102T124658
channel c05: restored backup piece 1
channel c05: restore complete, elapsed time: 00:02:48
channel c15: piece handle=/ORAGG/backup/DUMMYGG04_21nudgq6_1_1 tag=TAG20130102T124658
channel c15: restored backup piece 1
channel c15: restore complete, elapsed time: 00:04:37
channel c14: piece handle=/ORAGG/backup/DUMMYGG03_20nudgq5_1_1 tag=TAG20130102T124658
channel c14: restored backup piece 1
channel c14: restore complete, elapsed time: 00:05:07
channel c17: piece handle=/ORAGG/backup/DUMMYGG02_1vnudgq5_1_1 tag=TAG20130102T124658
channel c17: restored backup piece 1
channel c17: restore complete, elapsed time: 00:10:38
channel c16: piece handle=/ORAGG/backup/DUMMYGG01_1unudgq5_1_1 tag=TAG20130102T124658
channel c16: restored backup piece 1
channel c16: restore complete, elapsed time: 00:10:48
Finished restore at 03-JAN-13

released channel: c01

released channel: c02

released channel: c03

released channel: c04

released channel: c05

released channel: c06

released channel: c07

released channel: c08

released channel: c09

released channel: c10

released channel: c11

released channel: c12

released channel: c13

released channel: c14

released channel: c15

released channel: c16

released channel: c17

released channel: c18

RMAN> quit


Recovery Manager complete.



------output of recover database. 

sql statement: alter system set "_backup_disk_bufcnt"=64 scope=memory

sql statement: alter system set "_backup_disk_bufsz"=1048576 scope=memory

sql statement: alter system set "_backup_file_bufcnt"=64 scope=memory

sql statement: alter system set "_backup_file_bufsz"=1048576 scope=memory

allocated channel: c01
channel c01: SID=5 instance=DUMMYGG1 device type=DISK

allocated channel: c07
channel c07: SID=67 instance=DUMMYGG2 device type=DISK

allocated channel: c13
channel c13: SID=133 instance=DUMMYGG3 device type=DISK

executing command: SET until clause

Starting recover at 03-JAN-13

starting media recovery

archived log for thread 1 with sequence 1697 is already on disk as file +TESTGG/DUMMYGG/archivelog/2013_01_02/thread_1_seq_1697.776.803667609
archived log for thread 2 with sequence 1140 is already on disk as file +TESTGG/DUMMYGG/archivelog/2013_01_02/thread_2_seq_1140.1090.803668029
archived log for thread 3 with sequence 1195 is already on disk as file +TESTGG/DUMMYGG/archivelog/2013_01_02/thread_3_seq_1195.485.803669415
archived log file name=+TESTGG/DUMMYGG/archivelog/2013_01_02/thread_1_seq_1697.776.803667609 thread=1 sequence=1697
archived log file name=+TESTGG/DUMMYGG/archivelog/2013_01_02/thread_2_seq_1140.1090.803668029 thread=2 sequence=1140
archived log file name=+TESTGG/DUMMYGG/archivelog/2013_01_02/thread_3_seq_1195.485.803669415 thread=3 sequence=1195
media recovery complete, elapsed time: 00:00:01
Finished recover at 03-JAN-13

released channel: c01

released channel: c07

released channel: c13

RMAN>
Posted by Jignesh Kankrecha
at 6:07 PM

Monday, April 2, 2012

11GR2 Grid Infrastructure Processes

Labels: ,

New 11GR2 Grid Infrastructure Processes.


COMPONENTPROCESSESOWNER
Cluster Ready Service (CRS) crsdroot
Cluster Synchronization Service (CSS)ocssd,cssdmonitor,cssdagentgrid owner, root, root
Event Manager (EVM)evmd, evmlogger grid owner
Cluster Time Synchronization Service (CTSS) octssdroot
Oracle Notification Service (ONS)ONS,eonsgrid owner
Oracle Agent oraagentgrid owner
Oracle Root Agent orarootagentroot
Grid Naming Service (GNS)gnsdroot
Grid Plug and Play (GPnP)gpnpdgrid owner
Multicast Domain Name service (mDNS)mdnsdgrid owner




Posted by Jignesh Kankrecha
at 7:32 PM

Sunday, April 1, 2012

ssh User equivalence

Labels: , , , , , , , ,


[oracle@RACG1 grid]$ ./runcluvfy.sh stage -pre crsinst -n RACG1,RACG2 -r 11gR2 -fixup -verbose

 Performing pre-checks for cluster services setup

Checking node reachability...


Check: Node reachability from node "RACG1"

  Destination Node                      Reachable?
  ------------------------------------  ------------------------
  RACG1                                 yes
  RACG2                                 yes
Result: Node reachability check passed from node "RACG1"

Checking user equivalence...


Check: User equivalence for user "oracle"

  Node Name                             Comment
  ------------------------------------  ------------------------
  RACG1                                 failed
  RACG2                                 failed
Result: PRVF-4007 : User equivalence check failed for user "oracle"

ERROR:

User equivalence unavailable on all the specified nodes
Verification cannot proceed

 Pre-check for cluster services setup was unsuccessful on all the nodes.

-- TO AVOID 

PRVF-4007 : User equivalence check failed for user "oracle 



Setup ssh User equivalence in 11gR2





In new 11GR2 ssh User equivalence can be setup as below. 
[oracle@RACG1 grid]$ ll
total 40
drwxrwxrwx  9 oracle oinstall 4096 Apr  1 08:11 doc
drwxrwxrwx  4 oracle oinstall 4096 Apr  1 08:11 install
drwxrwxrwx  2 oracle oinstall 4096 Apr  1 08:11 response
drwxrwxrwx  2 oracle oinstall 4096 Apr  1 08:11 rpm
-rwxrwxrwx  1 oracle oinstall 3795 Apr  1 08:11 runcluvfy.sh
-rwxrwxrwx  1 oracle oinstall 3227 Apr  1 08:11 runInstaller
drwxrwxrwx  2 oracle oinstall 4096 Apr  1 09:05 sshsetup
drwxrwxrwx 14 oracle oinstall 4096 Apr  1 08:11 stage
-rwxrwxrwx  1 oracle oinstall 4228 Apr  1 08:11 welcome.html


cd sshsetup
./sshUserSetup.sh -user oracle -hosts NODE1,NODE2 -advanced -exverify -confirm

 Below is method to setup ssh Manually step-by-step. 

ON NODE-1

 TESTP1@:/home/oracle : $mkdir -p ~/.ssh
TESTP1@:/home/oracle : $chmod 700 ~/.ssh
TESTP1@:/home/oracle : $/usr/bin/ssh-keygen -t rsa
Generating public/private rsa key pair.
Enter file in which to save the key (/home/oracle/.ssh/id_rsa):
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /home/oracle/.ssh/id_rsa.
Your public key has been saved in /home/oracle/.ssh/id_rsa.pub.
The key fingerprint is:
4a:3b:fe:ea:03:d3:cf:6f:d3:06:fb:1a:ed:1e:b0:6b oracle@TESTP1.localdomain.com
TESTP1@:/home/oracle : $/usr/bin/ssh-keygen -t dsa
Generating public/private dsa key pair.
Enter file in which to save the key (/home/oracle/.ssh/id_dsa):
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /home/oracle/.ssh/id_dsa.
Your public key has been saved in /home/oracle/.ssh/id_dsa.pub.
The key fingerprint is:
5e:cb:46:cc:d8:f1:01:44:3e:7a:20:eb:ce:5d:d4:15 oracle@TESTP1.localdomain.com

 ON NODE-2

TESTP2@:/home/oracle : $mkdir -p ~/.ssh
TESTP2@:/home/oracle : $chmod 700 ~/.ssh
TESTP2@:/home/oracle : $/usr/bin/ssh-keygen -t rsa
Generating public/private rsa key pair.
Enter file in which to save the key (/home/oracle/.ssh/id_rsa):
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /home/oracle/.ssh/id_rsa.
Your public key has been saved in /home/oracle/.ssh/id_rsa.pub.
The key fingerprint is:
dc:4e:e7:c3:ee:71:84:e7:2e:72:99:3d:b0:0a:2b:f9 oracle@TESTP2.localdomain.com
TESTP2@:/home/oracle : $/usr/bin/ssh-keygen -t dsa
Generating public/private dsa key pair.
Enter file in which to save the key (/home/oracle/.ssh/id_dsa):
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /home/oracle/.ssh/id_dsa.
Your public key has been saved in /home/oracle/.ssh/id_dsa.pub.
The key fingerprint is:
5a:49:d4:e2:b9:5d:e9:f8:f5:bb:ce:7c:4b:f4:dd:6c oracle@TESTP2.localdomain.com

 ON NODE-1

TESTP1@:/home/oracle : $ssh TESTP1 cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys
The authenticity of host 'testp1 (192.168.100.181)' can't be established.
RSA key fingerprint is 27:ac:4b:9a:e3:d2:ae:6d:2b:71:99:8d:b9:c0:b1:a7.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added 'testp1,192.168.100.181' (RSA) to the list of known hosts.
oracle@testp1's password:
TESTP1@:/home/oracle : $ssh TESTP1 cat ~/.ssh/id_dsa.pub >> ~/.ssh/authorized_keys
TESTP1@:/home/oracle : $scp /home/oracle/.ssh/authorized_keys oracle@TESTP2:~/.ssh/
oracle@testp2's password:
authorized_keys                                                                                                          100% 1030     1.0KB/s   00:00

 ON NODE-2

 TESTP2@:/home/oracle/.ssh : $ssh TESTP2 cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys
The authenticity of host 'testp2 (192.168.100.182)' can't be established.
RSA key fingerprint is 27:ac:4b:9a:e3:d2:ae:6d:2b:71:99:8d:b9:c0:b1:a7.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added 'testp2,192.168.100.182' (RSA) to the list of known hosts.
oracle@testp2's password:
TESTP2@:/home/oracle/.ssh : $ssh TESTP2 cat ~/.ssh/id_dsa.pub >> ~/.ssh/authorized_keys
TESTP2@:/home/oracle/.ssh : $scp /home/oracle/.ssh/authorized_keys oracle@TESTP1:~/.ssh/
The authenticity of host 'testp1 (192.168.100.181)' can't be established.
RSA key fingerprint is 27:ac:4b:9a:e3:d2:ae:6d:2b:71:99:8d:b9:c0:b1:a7.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added 'testp1,192.168.100.181' (RSA) to the list of known hosts.
oracle@testp1's password:
authorized_keys                                                                                                          100% 2060     2.0KB/s   00:00

 --Confirm ssh on both Nodes. 

 TESTP1@:/home/oracle : $ssh TESTP2 date
Mon Aug  6 09:20:48 EDT 2012
TESTP1@:/home/oracle : $ssh TESTP1 date
Mon Aug  6 09:20:43 EDT 2012

 TESTP2@:/home/oracle/.ssh : $ssh TESTP2 date
Mon Aug  6 09:20:37 EDT 2012
TESTP2@:/home/oracle/.ssh : $ssh TESTP1 date
Mon Aug  6 09:20:31 EDT 2012



It should Ask for password only first time.  if it ask ,,,Do as below temporary 

------perform below step on BOTH NODE. 

exec /usr/bin/ssh-agent $SHELL
/usr/bin/ssh-add
Identity added: /home/oracle/.ssh/id_rsa (/home/oracle/.ssh/id_rsa)

  • login as root
  • vi /etc/sshd_config 
  • mark no instead of yes at line 
passwordauthentication no 



Posted by Jignesh Kankrecha
at 1:01 PM

Wednesday, February 22, 2012

11gR2 - Restore OCR/ VOTEDISK based on ASM, when loss of CRS Diskgroup.

Labels: , , , , , ,
1. Locate the latest automatic OCR backup

When using a non-shared CRS home, automatic OCR backups can be located on any node of the cluster, consequently all nodes need to be checked for the most recent backup:

$ ls -lrt $CRS_HOME/cdata/rac_cluster1/
-rw------- 1 root root 7331840 Mar 10 18:52 week.ocr
-rw------- 1 root root 7651328 Mar 26 01:33 week_.ocr
-rw------- 1 root root 7651328 Mar 29 01:33 day.ocr
-rw------- 1 root root 7651328 Mar 30 01:33 day_.ocr
-rw------- 1 root root 7651328 Mar 30 01:33 backup02.ocr
-rw------- 1 root root 7651328 Mar 30 05:33 backup01.ocr
-rw------- 1 root root 7651328 Mar 30 09:33 backup00.ocr

2. Make sure the Grid Infrastructure is shutdown on all nodes
Given that the OCR diskgroup is missing, the GI stack will not be functional on any node, however there may still be various daemon processes running.  On each node shutdown the GI stack using the force (-f) option:

# $CRS_HOME/bin/crsctl stop crs -f

3. Start the CRS stack in exclusive mode

On the node that has the most recent OCR backup, log on as root and start CRS in exclusive mode, this mode will allow ASM to start & stay up without the presence of a Voting disk and without the CRS daemon process (crsd.bin) running.

11.2.0.1:

# $CRS_HOME/bin/crsctl start crs -excl
...CRS-2672: Attempting to start 'ora.asm' on 'racnode1'
CRS-2676: Start of 'ora.asm' on 'racnode1' succeeded
CRS-2672: Attempting to start 'ora.crsd' on 'racnode1'
CRS-2676: Start of 'ora.crsd' on 'racnode1' succeeded


11.2.0.2:

# $CRS_HOME/bin/crsctl start crs -excl -nocrs
CRS-4123: Oracle High Availability Services has been started.
...
CRS-2672: Attempting to start 'ora.cluster_interconnect.haip' on 'auw2k3'
CRS-2672: Attempting to start 'ora.ctssd' on 'racnode1'
CRS-2676: Start of 'ora.drivers.acfs' on 'racnode1' succeeded
CRS-2676: Start of 'ora.ctssd' on 'racnode1' succeeded
CRS-2676: Start of 'ora.cluster_interconnect.haip' on 'racnode1' succeeded
CRS-2672: Attempting to start 'ora.asm' on 'racnode1'
CRS-2676: Start of 'ora.asm' on 'racnode1' succeeded

4. Label the CRS disk for ASMLIB use

If using ASMLIB the disk to be used for the CRS disk group needs to stamped first, as user root do:

# /usr/sbin/oracleasm createdisk ASMD40 /dev/sdh1

Writing disk header: done
Instantiating disk: done

5. Create the CRS diskgroup via sqlplus

The disk group can now be (re-)created via sqlplus from the grid user. The compatible.asm attribute must be set to 11.2 in order for the disk group to be used by CRS: 

$ sqlplus / as sysasm
SQL*Plus: Release 11.2.0.1.0 Production on Tue Mar 30 11:47:24 2010
Copyright (c) 1982, 2009, Oracle. All rights reserved.
Connected to:
Oracle Database 11g Enterprise Edition Release 11.2.0.1.0 - Production
With the Real Application Clusters and Automatic Storage Management options

SQL> create diskgroup CRS external redundancy disk 'ORCL:ASMD40' attribute 'COMPATIBLE.ASM' = '11.2';

Diskgroup created.
SQL> exit


6. Restore the latest OCR backup

Now that the CRS disk group is created & mounted the OCR can be restored - must be done as the root user:

# cd $CRS_HOME/cdata/rac_cluster1/
# $CRS_HOME/bin/ocrconfig -restore backup00.ocr

7. Start the CRS daemon on the current node (11.2.0.1 only !)

Now that the OCR has been restored the CRS daemon can be started, this is needed to recreate the Voting file. Skip this step for 11.2.0.2.0.

# $CRS_HOME/bin/crsctl start res ora.crsd -init
CRS-2672: Attempting to start 'ora.crsd' on 'racnode1'
CRS-2676: Start of 'ora.crsd' on 'racnode1' succeeded

8. Recreate the Voting file

The Voting file needs to be initialized in the CRS disk group:

# $CRS_HOME/bin/crsctl replace votedisk +CRS
Successful addition of voting disk 00caa5b9c0f54f3abf5bd2a2609f09a9.
Successfully replaced voting disk group with +CRS.
CRS-4266: Voting file(s) successfully replaced


9. Recreate the SPFILE for ASM (optional)

Prepare a pfile (e.g. /tmp/asm_pfile.ora) with the ASM startup parameters - these may vary from the example below. If in doubt consult the ASM alert log  as the ASM instance startup should list all non-default parameter values. Please note the last startup of ASM (in step 2 via CRS start) will not have used an SPFILE, so a startup prior to the loss of the CRS disk group would need to be located.
*.asm_power_limit=1
*.diagnostic_dest='/u01/app/oragrid'
*.instance_type='asm'
*.large_pool_size=12M
*.remote_login_passwordfile='EXCLUSIVE'
Now the SPFILE can be created using this PFILE:
$ sqlplus / as sysasm
SQL*Plus: Release 11.2.0.1.0 Production on Tue Mar 30 11:52:39 2010
Copyright (c) 1982, 2009, Oracle. All rights reserved.

Connected to:
Oracle Database 11g Enterprise Edition Release 11.2.0.1.0 - Production
With the Real Application Clusters and Automatic Storage Management options

SQL> create spfile='+CRS' from pfile='/tmp/asm_pfile.ora';

File created.
SQL> exit

10. Shutdown CRS 

Since CRS is running in exclusive mode, it needs to be shutdown  to allow CRS to run on all nodes again. Use of the force (-f) option may be required:

# $CRS_HOME/bin/crsctl stop crs -f
CRS-2793: Shutdown of Oracle High Availability Services-managed resources on 'auw2k3' has completed
CRS-4133: Oracle High Availability Services has been stopped.

11. Rescan ASM disks

If using ASMLIB rescan all ASM disks on each node as the root user:

# /usr/sbin/oracleasm scandisks
Reloading disk partitions: done
Cleaning any stale ASM disks...
Scanning system for ASM disks...
Instantiating disk "ASMD40"


12. Start CRS 
As the root user submit the CRS startup on all cluster nodes:

# $CRS_HOME/bin/crsctl start crs
CRS-4123: Oracle High Availability Services has been started.

13. Verify CRS 

To verify that CRS is fully functional again:
# $CRS_HOME/bin/crsctl check cluster -all
**************************************************************
racnode1:
CRS-4537: Cluster Ready Services is online
CRS-4529: Cluster Synchronization Services is online
CRS-4533: Event Manager is online
**************************************************************
racnode2:
CRS-4537: Cluster Ready Services is online
CRS-4529: Cluster Synchronization Services is online
CRS-4533: Event Manager is online
**************************************************************

# $CRS_HOME/bin/crsctl
Posted by Jignesh Kankrecha
at 1:11 PM

Monday, October 17, 2011

Upgrade CRS to 11.2.0.1 Grid Infrastructre. / Upgrade clusterware to Grid Infrastructure

Labels: , , , ,
As we all know that, In 11gR2 CRS and ASM has been consolidate in one binaries and named as GRID Infrastructure.
In-place upgrade is not supported in 11gR2.
Upgrade to GRID can be done in two methods.
1) CRS and ASM separate Upgrade.
2) CRS and ASM in together in single Installation.
Its recommended to Upgrade both together.
1. RUN CLUVFY
RACN1:/u00/grid-> ./runcluvfy.sh stage -pre crsinst -n racn1,racn2 -fixup -ve>
Performing pre-checks for cluster services setup
Checking node reachability...
Check: Node reachability from node "RACN1"
Destination Node Reachable?
------------------------------------ ------------------------
racn1 yes
racn2 yes
Result: Node reachability check passed from node "RACN1"
Checking user equivalence...
Check: User equivalence for user "oracle"
Node Name Comment
------------------------------------ ------------------------
racn2 passed
racn1 passed
Result: User equivalence check passed for user "oracle"
Checking node connectivity...
Checking hosts config file...
Node Name Status Comment
------------ ------------------------ ------------------------
racn2 passed
racn1 passed
Verification of the hosts config file successful
Interface information for node "racn2"
Name IP Address Subnet Gateway Def. Gateway HW Address MTU
------ --------------- --------------- --------------- --------------- ----------------- ------
eth0 192.168.133.152 192.168.133.0 0.0.0.0 192.168.133.1 00:0C:29:BA:2F:B6 1500
eth0 192.168.133.156 192.168.133.0 0.0.0.0 192.168.133.1 00:0C:29:BA:2F:B6 1500
eth1 192.168.131.78 192.168.131.0 0.0.0.0 192.168.133.1 00:0C:29:BA:2F:C0 1500
virbr0 192.168.122.1 192.168.122.0 0.0.0.0 192.168.133.1 00:00:00:00:00:00 1500
Interface information for node "racn1"
Name IP Address Subnet Gateway Def. Gateway HW Address MTU
------ --------------- --------------- --------------- --------------- ----------------- ------
eth0 192.168.133.151 192.168.133.0 0.0.0.0 192.168.133.1 00:0C:29:71:48:34 1500
eth0 192.168.133.155 192.168.133.0 0.0.0.0 192.168.133.1 00:0C:29:71:48:34 1500
eth1 192.168.131.76 192.168.131.0 0.0.0.0 192.168.133.1 00:0C:29:71:48:3E 1500
virbr0 192.168.122.1 192.168.122.0 0.0.0.0 192.168.133.1 00:00:00:00:00:00 1500
Check: Node connectivity of subnet "192.168.133.0"
Source Destination Connected?
------------------------------ ------------------------------ ----------------
racn2:eth0 racn2:eth0 yes
racn2:eth0 racn1:eth0 yes
racn2:eth0 racn1:eth0 yes
racn2:eth0 racn1:eth0 yes
racn2:eth0 racn1:eth0 yes
racn1:eth0 racn1:eth0 yes
Result: Node connectivity passed for subnet "192.168.133.0" with node(s) racn2,racn1
Check: TCP connectivity of subnet "192.168.133.0"
Source Destination Connected?
------------------------------ ------------------------------ ----------------
RACN1:192.168.133.151 racn2:192.168.133.152 passed
RACN1:192.168.133.151 racn2:192.168.133.156 passed
RACN1:192.168.133.151 racn1:192.168.133.155 passed
Result: TCP connectivity check failed for subnet "192.168.133.0"
Check: Node connectivity of subnet "192.168.131.0"
Source Destination Connected?
------------------------------ ------------------------------ ----------------
racn2:eth1 racn1:eth1 yes
Result: Node connectivity passed for subnet "192.168.131.0" with node(s) racn2,racn1
Check: TCP connectivity of subnet "192.168.131.0"
Source Destination Connected?
------------------------------ ------------------------------ ----------------
RACN1:192.168.133.151 racn2:192.168.131.78 passed
RACN1:192.168.133.151 racn1:192.168.131.76 passed
Result: TCP connectivity check passed for subnet "192.168.131.0"
Check: Node connectivity of subnet "192.168.122.0"
Source Destination Connected?
------------------------------ ------------------------------ ----------------
racn2:virbr0 racn1:virbr0 yes
Result: Node connectivity passed for subnet "192.168.122.0" with node(s) racn2,racn1
Check: TCP connectivity of subnet "192.168.122.0"
Source Destination Connected?
------------------------------ ------------------------------ ----------------
RACN1:192.168.133.151 racn2:192.168.122.1 passed
RACN1:192.168.133.151 racn1:192.168.122.1 passed
Result: TCP connectivity check passed for subnet "192.168.122.0"
Interfaces found on subnet "192.168.133.0" that are likely candidates for VIP are:
racn2 eth0:192.168.133.152 eth0:192.168.133.156
racn1 eth0:192.168.133.151 eth0:192.168.133.155
Interfaces found on subnet "192.168.131.0" that are likely candidates for a private interconnect are:
racn2 eth1:192.168.131.78
racn1 eth1:192.168.131.76
Interfaces found on subnet "192.168.122.0" that are likely candidates for a private interconnect are:
racn2 virbr0:192.168.122.1
racn1 virbr0:192.168.122.1
Result: Node connectivity check passed
Check: Total memory
Node Name Available Required Comment
------------ ------------------------ ------------------------ ----------
racn2 947.4MB (970140.0KB) 1.5GB (1572864.0KB) failed
racn1 947.4MB (970140.0KB) 1.5GB (1572864.0KB) failed
Result: Total memory check failed
Check: Available memory
Node Name Available Required Comment
------------ ------------------------ ------------------------ ----------
racn2 557.55MB (570932.0KB) 50MB (51200.0KB) passed
racn1 485.21MB (496860.0KB) 50MB (51200.0KB) passed
Result: Available memory check passed
Check: Swap space
Node Name Available Required Comment
------------ ------------------------ ------------------------ ----------
Result: Swap space check failed
Check: Free disk space for "racn2:/tmp"
Path Node Name Mount point Available Required Comment
---------------- ------------ ------------ ------------ ------------ ------------
/tmp racn2 / 12.39GB 1GB passed
Result: Free disk space check passed for "racn2:/tmp"
Check: Free disk space for "racn1:/tmp"
Path Node Name Mount point Available Required Comment
---------------- ------------ ------------ ------------ ------------ ------------
/tmp racn1 / 11.31GB 1GB passed
Result: Free disk space check passed for "racn1:/tmp"
Check: User existence for "oracle"
Node Name Status Comment
------------ ------------------------ ------------------------
racn2 exists passed
racn1 exists passed
Result: User existence check passed for "oracle"
Check: Group existence for "oinstall"
Node Name Status Comment
------------ ------------------------ ------------------------
racn2 exists passed
racn1 exists passed
Result: Group existence check passed for "oinstall"
Check: Group existence for "dba"
Node Name Status Comment
------------ ------------------------ ------------------------
racn2 exists passed
racn1 exists passed
Result: Group existence check passed for "dba"
Check: Membership of user "oracle" in group "oinstall" [as Primary]
Node Name User Exists Group Exists User in Group Primary Comment
---------------- ------------ ------------ ------------ ------------ ------------
racn2 yes yes yes yes passed
racn1 yes yes yes yes passed
Result: Membership check for user "oracle" in group "oinstall" [as Primary] passed
Check: Membership of user "oracle" in group "dba"
Node Name User Exists Group Exists User in Group Comment
---------------- ------------ ------------ ------------ ----------------
racn2 yes yes yes passed
racn1 yes yes yes passed
Result: Membership check for user "oracle" in group "dba" passed
Check: Run level
Node Name run level Required Comment
------------ ------------------------ ------------------------ ----------
racn2 5 3,5 passed
racn1 5 3,5 passed
Result: Run level check passed
Check: Hard limits for "maximum open file descriptors"
Node Name Type Available Required Comment
---------------- ------------ ------------ ------------ ----------------
racn2 hard 65536 65536 passed
racn1 hard 65536 65536 passed
Result: Hard limits check passed for "maximum open file descriptors"
Check: Soft limits for "maximum open file descriptors"
Node Name Type Available Required Comment
---------------- ------------ ------------ ------------ ----------------
racn2 soft 1024 1024 passed
racn1 soft 1024 1024 passed
Result: Soft limits check passed for "maximum open file descriptors"
Check: Hard limits for "maximum user processes"
Node Name Type Available Required Comment
---------------- ------------ ------------ ------------ ----------------
racn2 hard 16384 16384 passed
racn1 hard 16384 16384 passed
Result: Hard limits check passed for "maximum user processes"
Check: Soft limits for "maximum user processes"
Node Name Type Available Required Comment
---------------- ------------ ------------ ------------ ----------------
racn2 soft 2047 2047 passed
racn1 soft 2047 2047 passed
Result: Soft limits check passed for "maximum user processes"
Check: System architecture
Node Name Available Required Comment
------------ ------------------------ ------------------------ ----------
racn2 i686 x86 passed
racn1 i686 x86 passed
Result: System architecture check passed
Check: Kernel version
Node Name Available Required Comment
------------ ------------------------ ------------------------ ----------
racn2 2.6.18-164.el5PAE 2.6.18 passed
racn1 2.6.18-164.el5PAE 2.6.18 passed
Result: Kernel version check passed
Check: Kernel parameter for "semmsl"
Node Name Configured Required Comment
------------ ------------------------ ------------------------ ----------
racn2 250 250 passed
racn1 250 250 passed
Result: Kernel parameter check passed for "semmsl"
Check: Kernel parameter for "semmns"
Node Name Configured Required Comment
------------ ------------------------ ------------------------ ----------
racn2 32000 32000 passed
racn1 32000 32000 passed
Result: Kernel parameter check passed for "semmns"
Check: Kernel parameter for "semopm"
Node Name Configured Required Comment
------------ ------------------------ ------------------------ ----------
racn2 100 100 passed
racn1 100 100 passed
Result: Kernel parameter check passed for "semopm"
Check: Kernel parameter for "semmni"
Node Name Configured Required Comment
------------ ------------------------ ------------------------ ----------
racn2 128 128 passed
racn1 128 128 passed
Result: Kernel parameter check passed for "semmni"
Check: Kernel parameter for "shmmax"
Node Name Configured Required Comment
------------ ------------------------ ------------------------ ----------
racn2 4294967295 536870912 passed
racn1 4294967295 536870912 passed
Result: Kernel parameter check passed for "shmmax"
Check: Kernel parameter for "shmmni"
Node Name Configured Required Comment
------------ ------------------------ ------------------------ ----------
racn2 4096 4096 passed
racn1 4096 4096 passed
Result: Kernel parameter check passed for "shmmni"
Check: Kernel parameter for "shmall"
Node Name Configured Required Comment
------------ ------------------------ ------------------------ ----------
racn2 1073741824 2097152 passed
racn1 1073741824 2097152 passed
Result: Kernel parameter check passed for "shmall"
Check: Kernel parameter for "file-max"
Node Name Configured Required Comment
------------ ------------------------ ------------------------ ----------
racn2 6553600 6815744 failed
racn1 6553600 6815744 failed
Result: Kernel parameter check failed for "file-max"
Check: Kernel parameter for "ip_local_port_range"
Node Name Configured Required Comment
------------ ------------------------ ------------------------ ----------
racn2 between 1024 & 65000 between 9000 & 65500 failed (ignorable)
racn1 between 1024 & 65000 between 9000 & 65500 failed (ignorable)
Result: Kernel parameter check passed for "ip_local_port_range"
Check: Kernel parameter for "rmem_default"
Node Name Configured Required Comment
------------ ------------------------ ------------------------ ----------
racn2 4194304 262144 passed
racn1 4194304 262144 passed
Result: Kernel parameter check passed for "rmem_default"
Check: Kernel parameter for "rmem_max"
Node Name Configured Required Comment
------------ ------------------------ ------------------------ ----------
racn2 4194304 4194304 passed
racn1 4194304 4194304 passed
Result: Kernel parameter check passed for "rmem_max"
Check: Kernel parameter for "wmem_default"
Node Name Configured Required Comment
------------ ------------------------ ------------------------ ----------
racn2 262144 262144 passed
racn1 262144 262144 passed
Result: Kernel parameter check passed for "wmem_default"
Check: Kernel parameter for "wmem_max"
Node Name Configured Required Comment
------------ ------------------------ ------------------------ ----------
racn2 262144 1048576 failed
racn1 262144 1048576 failed
Result: Kernel parameter check failed for "wmem_max"
Check: Kernel parameter for "aio-max-nr"
Node Name Configured Required Comment
------------ ------------------------ ------------------------ ----------
racn2 3145728 1048576 passed
racn1 3145728 1048576 passed
Result: Kernel parameter check passed for "aio-max-nr"
Check: Package existence for "make-3.81"
Node Name Available Required Comment
------------ ------------------------ ------------------------ ----------
racn2 make-3.81-3.el5 make-3.81 passed
racn1 make-3.81-3.el5 make-3.81 passed
Result: Package existence check passed for "make-3.81"
Check: Package existence for "binutils-2.17.50.0.6"
Node Name Available Required Comment
------------ ------------------------ ------------------------ ----------
racn2 binutils-2.17.50.0.6-12.el5 binutils-2.17.50.0.6 passed
racn1 binutils-2.17.50.0.6-12.el5 binutils-2.17.50.0.6 passed
Result: Package existence check passed for "binutils-2.17.50.0.6"
Check: Package existence for "gcc-4.1.2"
Node Name Available Required Comment
------------ ------------------------ ------------------------ ----------
racn2 gcc-4.1.2-46.el5 gcc-4.1.2 passed
racn1 gcc-4.1.2-46.el5 gcc-4.1.2 passed
Result: Package existence check passed for "gcc-4.1.2"
Check: Package existence for "gcc-c++-4.1.2"
Node Name Available Required Comment
------------ ------------------------ ------------------------ ----------
racn2 gcc-c++-4.1.2-46.el5 gcc-c++-4.1.2 passed
racn1 gcc-c++-4.1.2-46.el5 gcc-c++-4.1.2 passed
Result: Package existence check passed for "gcc-c++-4.1.2"
Check: Package existence for "libgomp-4.1.2"
Node Name Available Required Comment
------------ ------------------------ ------------------------ ----------
racn2 libgomp-4.4.0-6.el5 libgomp-4.1.2 passed
racn1 libgomp-4.4.0-6.el5 libgomp-4.1.2 passed
Result: Package existence check passed for "libgomp-4.1.2"
Check: Package existence for "libaio-0.3.106"
Node Name Available Required Comment
------------ ------------------------ ------------------------ ----------
racn2 libaio-0.3.106-3.2 libaio-0.3.106 passed
racn1 libaio-0.3.106-3.2 libaio-0.3.106 passed
Result: Package existence check passed for "libaio-0.3.106"
Check: Package existence for "glibc-2.5-24"
Node Name Available Required Comment
------------ ------------------------ ------------------------ ----------
racn2 glibc-2.5-42 glibc-2.5-24 passed
racn1 glibc-2.5-42 glibc-2.5-24 passed
Result: Package existence check passed for "glibc-2.5-24"
Check: Package existence for "compat-libstdc++-33-3.2.3"
Node Name Available Required Comment
------------ ------------------------ ------------------------ ----------
racn2 compat-libstdc++-33-3.2.3-61 compat-libstdc++-33-3.2.3 passed
racn1 compat-libstdc++-33-3.2.3-61 compat-libstdc++-33-3.2.3 passed
Result: Package existence check passed for "compat-libstdc++-33-3.2.3"
Check: Package existence for "elfutils-libelf-0.125"
Node Name Available Required Comment
------------ ------------------------ ------------------------ ----------
racn2 elfutils-libelf-0.137-3.el5 elfutils-libelf-0.125 passed
racn1 elfutils-libelf-0.137-3.el5 elfutils-libelf-0.125 passed
Result: Package existence check passed for "elfutils-libelf-0.125"
Check: Package existence for "elfutils-libelf-devel-0.125"
Node Name Available Required Comment
------------ ------------------------ ------------------------ ----------
racn2 elfutils-libelf-devel-0.137-3.el5 elfutils-libelf-devel-0.125 passed
racn1 elfutils-libelf-devel-0.137-3.el5 elfutils-libelf-devel-0.125 passed
Result: Package existence check passed for "elfutils-libelf-devel-0.125"
Check: Package existence for "glibc-common-2.5"
Node Name Available Required Comment
------------ ------------------------ ------------------------ ----------
racn2 glibc-common-2.5-42 glibc-common-2.5 passed
racn1 glibc-common-2.5-42 glibc-common-2.5 passed
Result: Package existence check passed for "glibc-common-2.5"
Check: Package existence for "glibc-devel-2.5"
Node Name Available Required Comment
------------ ------------------------ ------------------------ ----------
racn2 glibc-devel-2.5-42 glibc-devel-2.5 passed
racn1 glibc-devel-2.5-42 glibc-devel-2.5 passed
Result: Package existence check passed for "glibc-devel-2.5"
Check: Package existence for "glibc-headers-2.5"
Node Name Available Required Comment
------------ ------------------------ ------------------------ ----------
racn2 glibc-headers-2.5-42 glibc-headers-2.5 passed
racn1 glibc-headers-2.5-42 glibc-headers-2.5 passed
Result: Package existence check passed for "glibc-headers-2.5"
Check: Package existence for "libaio-devel-0.3.106"
Node Name Available Required Comment
------------ ------------------------ ------------------------ ----------
racn2 libaio-devel-0.3.106-3.2 libaio-devel-0.3.106 passed
racn1 libaio-devel-0.3.106-3.2 libaio-devel-0.3.106 passed
Result: Package existence check passed for "libaio-devel-0.3.106"
Check: Package existence for "libgcc-4.1.2"
Node Name Available Required Comment
------------ ------------------------ ------------------------ ----------
racn2 libgcc-4.1.2-46.el5 libgcc-4.1.2 passed
racn1 libgcc-4.1.2-46.el5 libgcc-4.1.2 passed
Result: Package existence check passed for "libgcc-4.1.2"
Check: Package existence for "libstdc++-4.1.2"
Node Name Available Required Comment
------------ ------------------------ ------------------------ ----------
racn2 libstdc++-4.1.2-46.el5 libstdc++-4.1.2 passed
racn1 libstdc++-4.1.2-46.el5 libstdc++-4.1.2 passed
Result: Package existence check passed for "libstdc++-4.1.2"
Check: Package existence for "libstdc++-devel-4.1.2"
Node Name Available Required Comment
------------ ------------------------ ------------------------ ----------
racn2 libstdc++-devel-4.1.2-46.el5 libstdc++-devel-4.1.2 passed
racn1 libstdc++-devel-4.1.2-46.el5 libstdc++-devel-4.1.2 passed
Result: Package existence check passed for "libstdc++-devel-4.1.2"
Check: Package existence for "sysstat-7.0.2"
Node Name Available Required Comment
------------ ------------------------ ------------------------ ----------
racn2 sysstat-7.0.2-3.el5 sysstat-7.0.2 passed
racn1 sysstat-7.0.2-3.el5 sysstat-7.0.2 passed
Result: Package existence check passed for "sysstat-7.0.2"
Check: Package existence for "unixODBC-2.2.11"
Node Name Available Required Comment
------------ ------------------------ ------------------------ ----------
racn2 unixODBC-2.2.11-7.1 unixODBC-2.2.11 passed
racn1 unixODBC-2.2.11-7.1 unixODBC-2.2.11 passed
Result: Package existence check passed for "unixODBC-2.2.11"
Check: Package existence for "unixODBC-devel-2.2.11"
Node Name Available Required Comment
------------ ------------------------ ------------------------ ----------
racn2 unixODBC-devel-2.2.11-7.1 unixODBC-devel-2.2.11 passed
racn1 unixODBC-devel-2.2.11-7.1 unixODBC-devel-2.2.11 passed
Result: Package existence check passed for "unixODBC-devel-2.2.11"
Check: Package existence for "ksh-20060214"
Node Name Available Required Comment
------------ ------------------------ ------------------------ ----------
racn2 ksh-20080202-14.el5 ksh-20060214 passed
racn1 ksh-20080202-14.el5 ksh-20060214 passed
Result: Package existence check passed for "ksh-20060214"
Checking for multiple users with UID value 0
Result: Check for multiple users with UID value 0 passed
Check: Current group ID
Result: Current group ID check passed
Checking Core file name pattern consistency...
Core file name pattern consistency check passed.
Checking to make sure user "oracle" is not in "root" group
Node Name Status Comment
------------ ------------------------ ------------------------
racn2 does not exist passed
racn1 does not exist passed
Result: User "oracle" is not part of "root" group. Check passed
Check default user file creation mask
Node Name Available Required Comment
------------ ------------------------ ------------------------ ----------
racn2 0022 0022 passed
racn1 0022 0022 passed
Result: Default user file creation mask check passed
Starting Clock synchronization checks using Network Time Protocol(NTP)...
NTP Configuration file check started...
The NTP configuration file "/etc/ntp.conf" is available on all nodes
NTP Configuration file check passed
Checking daemon liveness...
Check: Liveness for "ntpd"
Node Name Running?
------------------------------------ ------------------------
racn2 no
racn1 no
Result: Liveness check failed for "ntpd"
PRVF-5415 : Check to see if NTP daemon is running failed
Result: Clock synchronization check using Network Time Protocol(NTP) failed
Fixup information has been generated for following node(s):
racn1,racn2
Please run the following script on each node as "root" user to execute the fixups:
'/tmp/CVU_11.2.0.1.0_oracle/runfixup.sh'
Pre-check for cluster services setup was unsuccessful on all the nodes.
2. FIX KERNAL PARAMETER ERRORS FROM GENERATED SCRIPT (if any) on BOTH NODES.
[root@RACN1 CVU_11.2.0.1.0_oracle]# ./runfixup.sh
Response file being used is :./fixup.response
Enable file being used is :./fixup.enable
Log file location: ./orarun.log
Setting Kernel Parameters...
fs.file-max = 6553600
fs.file-max = 6815744
net.ipv4.ip_local_port_range = 9000 65500
net.core.wmem_max = 262144
net.core.wmem_max=262144
./orarun.sh: line 728: [: 262144net.core.wmem_max: integer expression expected
The value for wmem_max in response file is not greater than value for wmem_max in /etc/sysctl.conf file. Hence not changing it.
net.core.wmem_max = 1048576
[root@RACN2 CVU_11.2.0.1.0_oracle]# ./runfixup.sh
Response file being used is :./fixup.response
Enable file being used is :./fixup.enable
Log file location: ./orarun.log
Setting Kernel Parameters...
fs.file-max = 6553600
fs.file-max = 6815744
net.ipv4.ip_local_port_range = 9000 65500
net.core.wmem_max = 262144
net.core.wmem_max=262144
./orarun.sh: line 728: [: 262144net.core.wmem_max: integer expression expected
The value for wmem_max in response file is not greater than value for wmem_max in /etc/sysctl.conf file. Hence not changing it.
net.core.wmem_max = 1048576
.3. NTP ERROR FIXED
Move /etc/ntp.conf
[root@RACN1 ~]# mv /etc/ntp.conf /etc/ntp.conf.old
[root@RACN2 ~]# mv /etc/ntp.conf /etc/ntp.conf.old
Starting Clock synchronization checks using Network Time Protocol(NTP)...
NTP Configuration file check started...
Network Time Protocol(NTP) configuration file not found on any of the nodes. Oracle Cluster Time Synchronization Service(CTSS) can be used instead of NTP for time synchronization on the cluster nodes
Result: Clock synchronization check using Network Time Protocol(NTP) passed
CONFIGURE SCAN IP
SET ASM ROLLING PARAMETER
asmca_rolling_upgrade=true
Oracle strongly recommends that you configure the SCAN either on DNS or GNS, with a maximum of three IPs in a round-robin fashion


4. Configure DNS files or /etc/hosts
Editing needed in DNS, since I m not using DNS I will edit my /etc/hosts and Add a new IP.
5. Create GRID_HOME and set environment
export GRID_HOME=/u01/app/grid
To Upgrade ASM alone with GRID , we need to export below parameter.
export asmca_rolling_upgrade=true
CRS don’t need to shutdown as its Rolling Upgrade.
While Running rootupgrade.sh after installation .it will shutdown old (11.1.0.7) and start new CRS (11.2.0.1).
RACN1:/home/oracle-> crs_stat -t
Name Type Target State Host
------------------------------------------------------------
ora....SM1.asm application ONLINE ONLINE racn1
ora....N1.lsnr application ONLINE ONLINE racn1
ora.racn1.gsd application ONLINE ONLINE racn1
ora.racn1.ons application ONLINE ONLINE racn1
ora.racn1.vip application ONLINE ONLINE racn1
ora....SM2.asm application ONLINE ONLINE racn2
ora....N2.lsnr application ONLINE ONLINE racn2
ora.racn2.gsd application ONLINE ONLINE racn2
ora.racn2.ons application ONLINE ONLINE racn2
ora.racn2.vip application ONLINE ONLINE racn2
6.Run Installer of Grid Infrastructure setup and select to upgrade Grid Infrastructure.
7. It will detect if its Running on ASM.


8. Node selection / Choose if you want to upgrade ASM along with CRS.

9.enter SCAN detail .


8. 10.Enter ASM pwd.

10. Select OS group for ASM.

11. Enter Installation Location Detail. ($GRID_HOME)



12.Installation in progress.


13.Run the Rootupgrade.sh on both node.



[root@RACN1 grid]# ./rootupgrade.sh
Running Oracle 11g root.sh script...
The following environment variables are set as:
ORACLE_OWNER= oracle
ORACLE_HOME= /u01/app/grid
Enter the full pathname of the local bin directory: [/usr/local/bin]:
The file "dbhome" already exists in /usr/local/bin. Overwrite it? (y/n)
[n]:
The file "oraenv" already exists in /usr/local/bin. Overwrite it? (y/n)
[n]:
The file "coraenv" already exists in /usr/local/bin. Overwrite it? (y/n)
[n]:
Entries will be added to the /etc/oratab file as needed by
Database Configuration Assistant when a database is created
Finished running generic part of root.sh script.
Now product-specific root actions will be performed.
2011-10-12 21:04:39: Parsing the host name
2011-10-12 21:04:39: Checking for super user privileges
2011-10-12 21:04:39: User has super user privileges
Using configuration parameter file: /u01/app/grid/crs/install/crsconfig_params
Creating trace directory
Cluster Synchronization Services appears healthy
Event Manager appears healthy
Cluster Ready Services appears healthy
Shutting down Oracle Cluster Ready Services (CRS):
Oct 12 21:05:11.420 | INF | daemon shutting down
Stopping resources.
This could take several minutes.
Successfully stopped Oracle Clusterware resources
Stopping Cluster Synchronization Services.
Shutting down the Cluster Synchronization Services daemon.
Shutdown request successfully issued.
Shutdown has begun. The daemons should exit soon.
LOCAL ADD MODE
Creating OCR keys for user 'root', privgrp 'root'..
Operation successful.
root wallet
root wallet cert
root cert export
peer wallet
profile reader wallet
pa wallet
peer wallet keys
pa wallet keys
peer cert request
pa cert request
peer cert
pa cert
peer root cert TP
profile reader root cert TP
pa root cert TP
peer pa cert TP
pa peer cert TP
profile reader pa cert TP
profile reader peer cert TP
peer user cert
pa user cert
Adding daemon to inittab
CRS-4123: Oracle High Availability Services has been started.
ohasd is starting
CRS-2672: Attempting to start 'ora.mdnsd' on 'racn1'
CRS-2676: Start of 'ora.mdnsd' on 'racn1' succeeded
CRS-2672: Attempting to start 'ora.gipcd' on 'racn1'
CRS-2676: Start of 'ora.gipcd' on 'racn1' succeeded
CRS-2672: Attempting to start 'ora.gpnpd' on 'racn1'
CRS-2676: Start of 'ora.gpnpd' on 'racn1' succeeded
CRS-2672: Attempting to start 'ora.cssdmonitor' on 'racn1'
CRS-2676: Start of 'ora.cssdmonitor' on 'racn1' succeeded
CRS-2672: Attempting to start 'ora.cssd' on 'racn1'
CRS-2672: Attempting to start 'ora.diskmon' on 'racn1'
CRS-2676: Start of 'ora.diskmon' on 'racn1' succeeded
CRS-2676: Start of 'ora.cssd' on 'racn1' succeeded
CRS-2672: Attempting to start 'ora.ctssd' on 'racn1'
CRS-2676: Start of 'ora.ctssd' on 'racn1' succeeded
CRS-2672: Attempting to start 'ora.crsd' on 'racn1'
CRS-2676: Start of 'ora.crsd' on 'racn1' succeeded
CRS-2672: Attempting to start 'ora.evmd' on 'racn1'
CRS-2676: Start of 'ora.evmd' on 'racn1' succeeded
clscfg: EXISTING configuration version 4 detected.
clscfg: version 4 is 11 Release 1.
Successfully accumulated necessary OCR keys.
Creating OCR keys for user 'root', privgrp 'root'..
Operation successful.
racn1 2011/10/12 21:11:27 /u01/app/grid/cdata/racn1/backup_20111012_211127.olr
Preparing packages for installation...
cvuqdisk-1.0.7-1
Configure Oracle Grid Infrastructure for a Cluster ... succeeded
Updating inventory properties for clusterware
Starting Oracle Universal Installer...
Checking swap space: must be greater than 500 MB. Actual 1139 MB Passed
The inventory pointer is located at /etc/oraInst.loc
The inventory is located at /u01/app/oraInventory
'UpdateNodeList' was successful.
Starting Oracle Universal Installer...
Checking swap space: must be greater than 500 MB. Actual 1022 MB Passed
The inventory pointer is located at /etc/oraInst.loc
The inventory is located at /u01/app/oraInventory
'UpdateNodeList' was successful.
[root@RACN1 grid]#

14. Now ASM will get upgrade.



15. validate all resources.

Validate All Resources are ONLINE with all new command.
./crsctl stat res -t
GSD will be OFFLINE by default in 11R2.
If any 9i database is cluster then GSD must be ONLINE.
To start GSD as ROOT.
./srvctl enable nodeapps -g
./srvctl start nodeapps
Posted by Jignesh Kankrecha
at 7:32 AM