This is Demo of Installation of 11.2.0.2 Grid Infrastructure / database home & DB creation ,with Role separation on Linux x86_64.
| Description | OS Group Name | OS Users Assigned to this Group | Oracle Privilege | Oracle Group Name |
| Oracle Inventory and Software Owner | oinstall | grid, oracle | ||
| Oracle Automatic Storage Management Group | asmadmin | grid | SYSASM | OSASM |
| ASM Database Administrator Group | asmdba | grid, oracle | SYSDBA for ASM | OSDBA for ASM |
| ASM Operator Group | asmoper | grid | SYSOPER for ASM | OSOPER for ASM |
| Database Administrator | dba | oracle | SYSDBA | OSDBA |
| Database Operator | oper | oracle | SYSOPER | OSOPER |
I have skipped few steps and few pre-requirement during this exercise. ssh user equivalence is already configured.
if ASM disks are not visible during DBCA . you can change permission of $ORACLE_HOME/bin/oracle to oracle user and asmadmin group. Make sure you have nosuid in your /etc/fstab for Installer mount point.
/etc/fstab where i have created two seprate mount points.
/u00 for grid and /u01 for oracle. where i have dont have nosuid for any mountpoints.
PNETN1@:/home/grid :+ASM1 $cat /etc/fstab /dev/VolGroup00/LogVol00 / ext3 defaults 1 1 LABEL=/u00 /u00 ext3 defaults 1 2 LABEL=/u01 /u01 ext3 defaults 1 2 LABEL=/boot /boot ext3 defaults 1 2 tmpfs /dev/shm tmpfs size=1700m 0 0 devpts /dev/pts devpts gid=5,mode=620 0 0 sysfs /sys sysfs defaults 0 0 proc /proc proc defaults 0 0 /dev/VolGroup00/LogVol01 swap swap defaults 0 0
Create group/users/binary home directories as below.
/usr/sbin/groupadd -g 1000 oinstall /usr/sbin/groupadd -g 1020 asmadmin /usr/sbin/groupadd -g 1021 asmdba /usr/sbin/groupadd -g 1030 asmoper /usr/sbin/groupadd -g 1031 dba /usr/sbin/groupadd -g 1032 oper useradd -u 1100 -g oinstall -G asmadmin,asmdba,asmoper grid useradd -u 1101 -g oinstall -G dba,asmdba oracle ---create GI/OH and set ownership/permission. --GI home mkdir -p /u00/app/11.2.0/grid chmod 775 /u00/app/11.2.0/grid chown -R grid:oinstall /u00 chmod -R 775 /u00 --OH home mkdir -p /u01/app/oracle/11.2.0/db_1 chmod 775 /u01/app/oracle/11.2.0/db_1 chown -R oracle:oinstall /u01 chmod -R 775 /u01
Configure ASM DISKS.
Configure ASM driver/ MARK asmdisks/ scandisks
[root@PNETN3 ~]# /etc/init.d/oracleasm configure -i
Configuring the Oracle ASM library driver.
This will configure the on-boot properties of the Oracle ASM library
driver. The following questions will determine whether the driver is
loaded on boot and what permissions it will have. The current values
will be shown in brackets ('[]'). Hitting without typing an
answer will keep that current value. Ctrl-C will abort.
Default user to own the driver interface [grid]: grid
Default group to own the driver interface [asmdba]: asmadmin
Start Oracle ASM library driver on boot (y/n) [y]: y
Scan for Oracle ASM disks on boot (y/n) [y]: y
Writing Oracle ASM library driver configuration: done
Initializing the Oracle ASMLib driver: [ OK ]
Scanning the system for Oracle ASMLib disks: [ OK ]
MARK ASMDISKS ---Only on NODE1.
[root@PNETN1 ~]# /etc/init.d/oracleasm createdisk DISK1 /dev/sdb1 Marking disk "DISK1" as an ASM disk: [ OK ] [root@PNETN1 ~]# /etc/init.d/oracleasm createdisk DISK2 /dev/sdc1 Marking disk "DISK2" as an ASM disk: [ OK ] [root@PNETN1 ~]# /etc/init.d/oracleasm createdisk DISK3 /dev/sdd1 Marking disk "DISK3" as an ASM disk: [ OK ] [root@PNETN1 ~]# /etc/init.d/oracleasm createdisk DISK4 /dev/sde1 Marking disk "DISK4" as an ASM disk: [ OK ] [root@PNETN1 ~]# /etc/init.d/oracleasm createdisk DISK5 /dev/sdf1 Marking disk "DISK5" as an ASM disk: [ OK ] [root@PNETN1 ~]# /etc/init.d/oracleasm createdisk DISK6 /dev/sdg1 Marking disk "DISK6" as an ASM disk: [ OK ] [root@PNETN1 ~]#
SCAN/LIST ASMDISKS-----On each NODE.
[root@PNETN2 ~]# /etc/init.d/oracleasm scandisks Scanning the system for Oracle ASMLib disks: [ OK ] [root@PNETN2 ~]# /etc/init.d/oracleasm listdisks DISK1 DISK2 DISK3 DISK4 DISK5 DISK6 [root@PNETN2 ~]#
SET OWNERSHIP/PERMISSION ON ASMDISKS-----on each NODE.
you can either set permission on /etc/rc.d/rc.local OR /etc/udev/rules.d/99-sd.rules
I have used /etc/rc.d/rc.local which resulted in PRVF-5148 , which can be ignored.
[root@PNETN1 ~]# vi /etc/rc.d/rc.local #!/bin/sh touch /var/lock/subsys/local chown grid:asmadmin /dev/sdb1 chown grid:asmadmin /dev/sdc1 chown grid:asmadmin /dev/sdd1 chown grid:asmadmin /dev/sde1 chown grid:asmadmin /dev/sdf1 chown grid:asmadmin /dev/sdg1 chown grid:asmadmin /dev/sdh1 chown grid:asmadmin /dev/sdi1 chown oracle:oinstall /dev/sdj1 chmod 660 /dev/sdb1 chmod 660 /dev/sdc1 chmod 660 /dev/sdd1 chmod 660 /dev/sde1 chmod 660 /dev/sdf1 chmod 660 /dev/sdg1 chmod 660 /dev/sdh1 chmod 660 /dev/sdi1 chmod 777 /dev/sdj1
*OR
You can also do it as below which will result in PRVF-5149.but cause will be unknown.
[root@PNETN3 rules.d]# vi /etc/udev/rules.d/99-sd.rules KERNEL=="sdb[1]", OWNER="grid", GROUP="asmadmin", MODE="660" KERNEL=="sdc[1]", OWNER="grid", GROUP="asmadmin", MODE="660" KERNEL=="sdd[1]", OWNER="grid", GROUP="asmadmin", MODE="660" KERNEL=="sde[1]", OWNER="grid", GROUP="asmadmin", MODE="660" KERNEL=="sdf[1]", OWNER="grid", GROUP="asmadmin", MODE="660" KERNEL=="sdg[1]", OWNER="grid", GROUP="asmadmin", MODE="660" KERNEL=="sdh[1]", OWNER="grid", GROUP="asmadmin", MODE="660" KERNEL=="sdi[1]", OWNER="grid", GROUP="asmadmin", MODE="660" KERNEL=="sdj[1]", OWNER="oracle", GROUP="oinstall", MODE="777" ---copy to each node. [root@PNETN3 rules.d]# vi /etc/udev/rules.d/99-oracle-asmdevices.rules [root@PNETN3 rules.d]# scp /etc/udev/rules.d/99-oracle-asmdevices.rules root@PNETN1:/etc/udev/rules.d/ The authenticity of host 'pnetn1 (192.168.100.126)' can't be established. RSA key fingerprint is 27:ac:4b:9a:e3:d2:ae:6d:2b:71:99:8d:b9:c0:b1:a7. Are you sure you want to continue connecting (yes/no)? yes Warning: Permanently added 'pnetn1,192.168.100.126' (RSA) to the list of known hosts. root@pnetn1's password: 99-oracle-asmdevices.rules 100% 551 0.5KB/s 00:00 [root@PNETN3 rules.d]# scp /etc/udev/rules.d/99-oracle-asmdevices.rules root@PNETN2:/etc/udev/rules.d/ root@pnetn2's password: 99-oracle-asmdevices.rules 100% 551 0.5KB/s 00:00 [root@PNETN3 rules.d]# ---execute on each node. /sbin/partprobe /dev/sdb1 /sbin/partprobe /dev/sdc1 /sbin/partprobe /dev/sdd1 /sbin/partprobe /dev/sde1 /sbin/partprobe /dev/sdf1 /sbin/partprobe /dev/sdg1 /sbin/partprobe /dev/sdh1 /sbin/partprobe /dev/sdi1 /sbin/partprobe /dev/sdj1 ----execute on each node. /sbin/udevcontrol reload_rules /sbin/start_udev [root@PNETN1 rules.d]# /sbin/udevcontrol reload_rules [root@PNETN1 rules.d]# /sbin/start_udev Starting udev: [ OK ]
CHECK PERMISSION.
[root@PNETN1 rules.d]# ls -lart /dev/sd* brw-r----- 1 root disk 8, 144 Jul 25 07:43 /dev/sdj brw-r----- 1 root disk 8, 128 Jul 25 07:43 /dev/sdi brw-r----- 1 root disk 8, 112 Jul 25 07:43 /dev/sdh brw-r----- 1 root disk 8, 96 Jul 25 07:43 /dev/sdg brw-r----- 1 root disk 8, 80 Jul 25 07:43 /dev/sdf brw-r----- 1 root disk 8, 64 Jul 25 07:43 /dev/sde brw-r----- 1 root disk 8, 48 Jul 25 07:43 /dev/sdd brw-r----- 1 root disk 8, 32 Jul 25 07:43 /dev/sdc brw-r----- 1 root disk 8, 16 Jul 25 07:43 /dev/sdb brw-r----- 1 root disk 8, 5 Jul 25 07:43 /dev/sda5 brw-r----- 1 root disk 8, 4 Jul 25 07:43 /dev/sda4 brw-r----- 1 root disk 8, 0 Jul 25 07:43 /dev/sda brw-r----- 1 root disk 8, 3 Jul 25 07:44 /dev/sda3 brw-r----- 1 root disk 8, 2 Jul 25 07:44 /dev/sda2 brw-r----- 1 root disk 8, 1 Jul 25 07:44 /dev/sda1 brw-rw---- 1 grid asmadmin 8, 17 Jul 25 07:44 /dev/sdb1 brw-rw---- 1 grid asmadmin 8, 33 Jul 25 07:44 /dev/sdc1 brw-rw---- 1 grid asmadmin 8, 49 Jul 25 07:44 /dev/sdd1 brw-rw---- 1 grid asmadmin 8, 65 Jul 25 07:44 /dev/sde1 brw-rw---- 1 grid asmadmin 8, 81 Jul 25 07:44 /dev/sdf1 brw-rw---- 1 grid asmadmin 8, 97 Jul 25 07:44 /dev/sdg1 brw-rw---- 1 grid asmadmin 8, 113 Jul 25 07:44 /dev/sdh1 brw-rw---- 1 grid asmadmin 8, 129 Jul 25 07:44 /dev/sdi1 brwxrwxrwx 1 oracle oinstall 8, 145 Jul 25 07:44 /dev/sdj1
----runcluvfy.sh
PNETN1@:/u01/patch/p10098816_112020_Linux-x86-64_grid : $./runcluvfy.sh stage -pre crsinst -n PNETN1,PNETN2,PNETN3 -verbose Performing pre-checks for cluster services setup Checking node reachability... Check: Node reachability from node "PNETN1" Destination Node Reachable? ------------------------------------ ------------------------ PNETN1 yes PNETN2 yes PNETN3 yes Result: Node reachability check passed from node "PNETN1" Checking user equivalence... Check: User equivalence for user "grid" Node Name Comment ------------------------------------ ------------------------ PNETN3 passed PNETN2 passed PNETN1 passed Result: User equivalence check passed for user "grid" Checking node connectivity... Checking hosts config file... Node Name Status Comment ------------ ------------------------ ------------------------ PNETN3 passed PNETN2 passed PNETN1 passed Verification of the hosts config file successful Interface information for node "PNETN3" Name IP Address Subnet Gateway Def. Gateway HW Address MTU ------ --------------- --------------- --------------- --------------- ----------------- ------ eth0 192.168.100.128 192.168.100.0 0.0.0.0 UNKNOWN 00:0C:29:F5:6F:F0 1500 eth1 192.168.152.128 192.168.152.0 0.0.0.0 UNKNOWN 00:0C:29:F5:6F:FA 1500 Interface information for node "PNETN2" Name IP Address Subnet Gateway Def. Gateway HW Address MTU ------ --------------- --------------- --------------- --------------- ----------------- ------ eth0 192.168.100.127 192.168.100.0 0.0.0.0 UNKNOWN 00:0C:29:BB:4A:C2 1500 eth1 192.168.152.127 192.168.152.0 0.0.0.0 UNKNOWN 00:0C:29:BB:4A:CC 1500 Interface information for node "PNETN1" Name IP Address Subnet Gateway Def. Gateway HW Address MTU ------ --------------- --------------- --------------- --------------- ----------------- ------ eth0 192.168.100.126 192.168.100.0 0.0.0.0 192.168.100.1 00:0C:29:01:0F:8B 1500 eth1 192.168.152.126 192.168.152.0 0.0.0.0 192.168.100.1 00:0C:29:01:0F:95 1500 Check: Node connectivity of subnet "192.168.100.0" Source Destination Connected? ------------------------------ ------------------------------ ---------------- PNETN3[192.168.100.128] PNETN2[192.168.100.127] yes PNETN3[192.168.100.128] PNETN1[192.168.100.126] yes PNETN2[192.168.100.127] PNETN1[192.168.100.126] yes Result: Node connectivity passed for subnet "192.168.100.0" with node(s) PNETN3,PNETN2,PNETN1 Check: TCP connectivity of subnet "192.168.100.0" Source Destination Connected? ------------------------------ ------------------------------ ---------------- PNETN1:192.168.100.126 PNETN3:192.168.100.128 passed PNETN1:192.168.100.126 PNETN2:192.168.100.127 passed Result: TCP connectivity check passed for subnet "192.168.100.0" Check: Node connectivity of subnet "192.168.152.0" Source Destination Connected? ------------------------------ ------------------------------ ---------------- PNETN3[192.168.152.128] PNETN2[192.168.152.127] yes PNETN3[192.168.152.128] PNETN1[192.168.152.126] yes PNETN2[192.168.152.127] PNETN1[192.168.152.126] yes Result: Node connectivity passed for subnet "192.168.152.0" with node(s) PNETN3,PNETN2,PNETN1 Check: TCP connectivity of subnet "192.168.152.0" Source Destination Connected? ------------------------------ ------------------------------ ---------------- PNETN1:192.168.152.126 PNETN3:192.168.152.128 passed PNETN1:192.168.152.126 PNETN2:192.168.152.127 passed Result: TCP connectivity check passed for subnet "192.168.152.0" Interfaces found on subnet "192.168.152.0" that are likely candidates for a private interconnect are: PNETN3 eth1:192.168.152.128 PNETN2 eth1:192.168.152.127 PNETN1 eth1:192.168.152.126 WARNING: Could not find a suitable set of interfaces for VIPs Result: Node connectivity check passed Checking ASMLib configuration. Node Name Comment ------------------------------------ ------------------------ PNETN3 passed PNETN2 passed PNETN1 passed Result: Check for ASMLib configuration passed. Check: Total memory Node Name Available Required Comment ------------ ------------------------ ------------------------ ---------- PNETN3 1.6625GB (1743280.0KB) 1.5GB (1572864.0KB) passed PNETN2 1.6625GB (1743280.0KB) 1.5GB (1572864.0KB) passed PNETN1 1.6625GB (1743280.0KB) 1.5GB (1572864.0KB) passed Result: Total memory check passed Check: Available memory Node Name Available Required Comment ------------ ------------------------ ------------------------ ---------- PNETN3 1.5201GB (1593932.0KB) 50MB (51200.0KB) passed PNETN2 1.5202GB (1594000.0KB) 50MB (51200.0KB) passed PNETN1 1.4477GB (1518024.0KB) 50MB (51200.0KB) passed Result: Available memory check passed Check: Swap space Node Name Available Required Comment ------------ ------------------------ ------------------------ ---------- PNETN3 3.8437GB (4030456.0KB) 2.4938GB (2614920.0KB) passed PNETN2 3.8437GB (4030456.0KB) 2.4938GB (2614920.0KB) passed PNETN1 3.8437GB (4030456.0KB) 2.4938GB (2614920.0KB) passed Result: Swap space check passed Check: Free disk space for "PNETN3:/tmp" Path Node Name Mount point Available Required Comment ---------------- ------------ ------------ ------------ ------------ ------------ /tmp PNETN3 / 2.7852GB 1GB passed Result: Free disk space check passed for "PNETN3:/tmp" Check: Free disk space for "PNETN2:/tmp" Path Node Name Mount point Available Required Comment ---------------- ------------ ------------ ------------ ------------ ------------ /tmp PNETN2 / 2.7852GB 1GB passed Result: Free disk space check passed for "PNETN2:/tmp" Check: Free disk space for "PNETN1:/tmp" Path Node Name Mount point Available Required Comment ---------------- ------------ ------------ ------------ ------------ ------------ /tmp PNETN1 / 2.5402GB 1GB passed Result: Free disk space check passed for "PNETN1:/tmp" Check: User existence for "grid" Node Name Status Comment ------------ ------------------------ ------------------------ PNETN3 exists(1100) passed PNETN2 exists(1100) passed PNETN1 exists(1100) passed Checking for multiple users with UID value 1100 Result: Check for multiple users with UID value 1100 passed Result: User existence check passed for "grid" Check: Group existence for "oinstall" Node Name Status Comment ------------ ------------------------ ------------------------ PNETN3 exists passed PNETN2 exists passed PNETN1 exists passed Result: Group existence check passed for "oinstall" Check: Group existence for "dba" Node Name Status Comment ------------ ------------------------ ------------------------ PNETN3 exists passed PNETN2 exists passed PNETN1 exists passed Result: Group existence check passed for "dba" Check: Membership of user "grid" in group "oinstall" [as Primary] Node Name User Exists Group Exists User in Group Primary Comment ---------------- ------------ ------------ ------------ ------------ ------------ PNETN3 yes yes yes yes passed PNETN2 yes yes yes yes passed PNETN1 yes yes yes yes passed Result: Membership check for user "grid" in group "oinstall" [as Primary] passed Check: Membership of user "grid" in group "dba" Node Name User Exists Group Exists User in Group Comment ---------------- ------------ ------------ ------------ ---------------- PNETN3 yes yes no failed PNETN2 yes yes no failed PNETN1 yes yes no failed Result: Membership check for user "grid" in group "dba" failed Check: Run level Node Name run level Required Comment ------------ ------------------------ ------------------------ ---------- PNETN3 5 3,5 passed PNETN2 5 3,5 passed PNETN1 5 3,5 passed Result: Run level check passed Check: Hard limits for "maximum open file descriptors" Node Name Type Available Required Comment ---------------- ------------ ------------ ------------ ---------------- PNETN3 hard 131072 65536 passed PNETN2 hard 131072 65536 passed PNETN1 hard 131072 65536 passed Result: Hard limits check passed for "maximum open file descriptors" Check: Soft limits for "maximum open file descriptors" Node Name Type Available Required Comment ---------------- ------------ ------------ ------------ ---------------- PNETN3 soft 131072 1024 passed PNETN2 soft 131072 1024 passed PNETN1 soft 131072 1024 passed Result: Soft limits check passed for "maximum open file descriptors" Check: Hard limits for "maximum user processes" Node Name Type Available Required Comment ---------------- ------------ ------------ ------------ ---------------- PNETN3 hard 131072 16384 passed PNETN2 hard 131072 16384 passed PNETN1 hard 131072 16384 passed Result: Hard limits check passed for "maximum user processes" Check: Soft limits for "maximum user processes" Node Name Type Available Required Comment ---------------- ------------ ------------ ------------ ---------------- PNETN3 soft 131072 2047 passed PNETN2 soft 131072 2047 passed PNETN1 soft 131072 2047 passed Result: Soft limits check passed for "maximum user processes" Check: System architecture Node Name Available Required Comment ------------ ------------------------ ------------------------ ---------- PNETN3 x86_64 x86_64 passed PNETN2 x86_64 x86_64 passed PNETN1 x86_64 x86_64 passed Result: System architecture check passed Check: Kernel version Node Name Available Required Comment ------------ ------------------------ ------------------------ ---------- PNETN3 2.6.18-164.el5xen 2.6.18 passed PNETN2 2.6.18-164.el5xen 2.6.18 passed PNETN1 2.6.18-164.el5xen 2.6.18 passed Result: Kernel version check passed Check: Kernel parameter for "semmsl" Node Name Configured Required Comment ------------ ------------------------ ------------------------ ---------- PNETN3 250 250 passed PNETN2 250 250 passed PNETN1 250 250 passed Result: Kernel parameter check passed for "semmsl" Check: Kernel parameter for "semmns" Node Name Configured Required Comment ------------ ------------------------ ------------------------ ---------- PNETN3 32000 32000 passed PNETN2 32000 32000 passed PNETN1 32000 32000 passed Result: Kernel parameter check passed for "semmns" Check: Kernel parameter for "semopm" Node Name Configured Required Comment ------------ ------------------------ ------------------------ ---------- PNETN3 100 100 passed PNETN2 100 100 passed PNETN1 100 100 passed Result: Kernel parameter check passed for "semopm" Check: Kernel parameter for "semmni" Node Name Configured Required Comment ------------ ------------------------ ------------------------ ---------- PNETN3 142 128 passed PNETN2 142 128 passed PNETN1 142 128 passed Result: Kernel parameter check passed for "semmni" Check: Kernel parameter for "shmmax" Node Name Configured Required Comment ------------ ------------------------ ------------------------ ---------- PNETN3 4398046511104 892559360 passed PNETN2 4398046511104 892559360 passed PNETN1 4398046511104 892559360 passed Result: Kernel parameter check passed for "shmmax" Check: Kernel parameter for "shmmni" Node Name Configured Required Comment ------------ ------------------------ ------------------------ ---------- PNETN3 4096 4096 passed PNETN2 4096 4096 passed PNETN1 4096 4096 passed Result: Kernel parameter check passed for "shmmni" Check: Kernel parameter for "shmall" Node Name Configured Required Comment ------------ ------------------------ ------------------------ ---------- PNETN3 1073741824 2097152 passed PNETN2 1073741824 2097152 passed PNETN1 1073741824 2097152 passed Result: Kernel parameter check passed for "shmall" Check: Kernel parameter for "file-max" Node Name Configured Required Comment ------------ ------------------------ ------------------------ ---------- PNETN3 6815744 6815744 passed PNETN2 6815744 6815744 passed PNETN1 6815744 6815744 passed Result: Kernel parameter check passed for "file-max" Check: Kernel parameter for "ip_local_port_range" Node Name Configured Required Comment ------------ ------------------------ ------------------------ ---------- PNETN3 between 9000 & 65500 between 9000 & 65500 passed PNETN2 between 9000 & 65500 between 9000 & 65500 passed PNETN1 between 9000 & 65500 between 9000 & 65500 passed Result: Kernel parameter check passed for "ip_local_port_range" Check: Kernel parameter for "rmem_default" Node Name Configured Required Comment ------------ ------------------------ ------------------------ ---------- PNETN3 262144 262144 passed PNETN2 262144 262144 passed PNETN1 262144 262144 passed Result: Kernel parameter check passed for "rmem_default" Check: Kernel parameter for "rmem_max" Node Name Configured Required Comment ------------ ------------------------ ------------------------ ---------- PNETN3 4194304 4194304 passed PNETN2 4194304 4194304 passed PNETN1 4194304 4194304 passed Result: Kernel parameter check passed for "rmem_max" Check: Kernel parameter for "wmem_default" Node Name Configured Required Comment ------------ ------------------------ ------------------------ ---------- PNETN3 262144 262144 passed PNETN2 262144 262144 passed PNETN1 262144 262144 passed Result: Kernel parameter check passed for "wmem_default" Check: Kernel parameter for "wmem_max" Node Name Configured Required Comment ------------ ------------------------ ------------------------ ---------- PNETN3 1048576 1048576 passed PNETN2 1048576 1048576 passed PNETN1 1048576 1048576 passed Result: Kernel parameter check passed for "wmem_max" Check: Kernel parameter for "aio-max-nr" Node Name Configured Required Comment ------------ ------------------------ ------------------------ ---------- PNETN3 3145728 1048576 passed PNETN2 3145728 1048576 passed PNETN1 3145728 1048576 passed Result: Kernel parameter check passed for "aio-max-nr" Check: Package existence for "make-3.81( x86_64)" Node Name Available Required Comment ------------ ------------------------ ------------------------ ---------- PNETN3 make-3.81-3.el5 make-3.81( x86_64) passed PNETN2 make-3.81-3.el5 make-3.81( x86_64) passed PNETN1 make-3.81-3.el5 make-3.81( x86_64) passed Result: Package existence check passed for "make-3.81( x86_64)" Check: Package existence for "binutils-2.17.50.0.6( x86_64)" Node Name Available Required Comment ------------ ------------------------ ------------------------ ---------- PNETN3 binutils-2.17.50.0.6-12.el5 binutils-2.17.50.0.6( x86_64) passed PNETN2 binutils-2.17.50.0.6-12.el5 binutils-2.17.50.0.6( x86_64) passed PNETN1 binutils-2.17.50.0.6-12.el5 binutils-2.17.50.0.6( x86_64) passed Result: Package existence check passed for "binutils-2.17.50.0.6( x86_64)" Check: Package existence for "gcc-4.1.2 (x86_64)( x86_64)" Node Name Available Required Comment ------------ ------------------------ ------------------------ ---------- PNETN3 gcc-4.1.2-46.el5 (x86_64) gcc-4.1.2 (x86_64)( x86_64) passed PNETN2 gcc-4.1.2-46.el5 (x86_64) gcc-4.1.2 (x86_64)( x86_64) passed PNETN1 gcc-4.1.2-46.el5 (x86_64) gcc-4.1.2 (x86_64)( x86_64) passed Result: Package existence check passed for "gcc-4.1.2 (x86_64)( x86_64)" Check: Package existence for "libaio-0.3.106 (x86_64)( x86_64)" Node Name Available Required Comment ------------ ------------------------ ------------------------ ---------- PNETN3 libaio-0.3.106-3.2 (x86_64) libaio-0.3.106 (x86_64)( x86_64) passed PNETN2 libaio-0.3.106-3.2 (x86_64) libaio-0.3.106 (x86_64)( x86_64) passed PNETN1 libaio-0.3.106-3.2 (x86_64) libaio-0.3.106 (x86_64)( x86_64) passed Result: Package existence check passed for "libaio-0.3.106 (x86_64)( x86_64)" Check: Package existence for "glibc-2.5-24 (x86_64)( x86_64)" Node Name Available Required Comment ------------ ------------------------ ------------------------ ---------- PNETN3 glibc-2.5-42 (x86_64) glibc-2.5-24 (x86_64)( x86_64) passed PNETN2 glibc-2.5-42 (x86_64) glibc-2.5-24 (x86_64)( x86_64) passed PNETN1 glibc-2.5-42 (x86_64) glibc-2.5-24 (x86_64)( x86_64) passed Result: Package existence check passed for "glibc-2.5-24 (x86_64)( x86_64)" Check: Package existence for "compat-libstdc++-33-3.2.3 (x86_64)( x86_64)" Node Name Available Required Comment ------------ ------------------------ ------------------------ ---------- PNETN3 compat-libstdc++-33-3.2.3-61 (x86_64) compat-libstdc++-33-3.2.3 (x86_64)( x86_64) passed PNETN2 compat-libstdc++-33-3.2.3-61 (x86_64) compat-libstdc++-33-3.2.3 (x86_64)( x86_64) passed PNETN1 compat-libstdc++-33-3.2.3-61 (x86_64) compat-libstdc++-33-3.2.3 (x86_64)( x86_64) passed Result: Package existence check passed for "compat-libstdc++-33-3.2.3 (x86_64)( x86_64)" Check: Package existence for "elfutils-libelf-0.125 (x86_64)( x86_64)" Node Name Available Required Comment ------------ ------------------------ ------------------------ ---------- PNETN3 elfutils-libelf-0.137-3.el5 (x86_64) elfutils-libelf-0.125 (x86_64)( x86_64) passed PNETN2 elfutils-libelf-0.137-3.el5 (x86_64) elfutils-libelf-0.125 (x86_64)( x86_64) passed PNETN1 elfutils-libelf-0.137-3.el5 (x86_64) elfutils-libelf-0.125 (x86_64)( x86_64) passed Result: Package existence check passed for "elfutils-libelf-0.125 (x86_64)( x86_64)" Check: Package existence for "elfutils-libelf-devel-0.125( x86_64)" Node Name Available Required Comment ------------ ------------------------ ------------------------ ---------- PNETN3 elfutils-libelf-devel-0.137-3.el5 elfutils-libelf-devel-0.125( x86_64) passed PNETN2 elfutils-libelf-devel-0.137-3.el5 elfutils-libelf-devel-0.125( x86_64) passed PNETN1 elfutils-libelf-devel-0.137-3.el5 elfutils-libelf-devel-0.125( x86_64) passed Result: Package existence check passed for "elfutils-libelf-devel-0.125( x86_64)" Check: Package existence for "glibc-common-2.5( x86_64)" Node Name Available Required Comment ------------ ------------------------ ------------------------ ---------- PNETN3 glibc-common-2.5-42 glibc-common-2.5( x86_64) passed PNETN2 glibc-common-2.5-42 glibc-common-2.5( x86_64) passed PNETN1 glibc-common-2.5-42 glibc-common-2.5( x86_64) passed Result: Package existence check passed for "glibc-common-2.5( x86_64)" Check: Package existence for "glibc-devel-2.5 (x86_64)( x86_64)" Node Name Available Required Comment ------------ ------------------------ ------------------------ ---------- PNETN3 glibc-devel-2.5-42 (x86_64) glibc-devel-2.5 (x86_64)( x86_64) passed PNETN2 glibc-devel-2.5-42 (x86_64) glibc-devel-2.5 (x86_64)( x86_64) passed PNETN1 glibc-devel-2.5-42 (x86_64) glibc-devel-2.5 (x86_64)( x86_64) passed Result: Package existence check passed for "glibc-devel-2.5 (x86_64)( x86_64)" Check: Package existence for "glibc-headers-2.5( x86_64)" Node Name Available Required Comment ------------ ------------------------ ------------------------ ---------- PNETN3 glibc-headers-2.5-42 glibc-headers-2.5( x86_64) passed PNETN2 glibc-headers-2.5-42 glibc-headers-2.5( x86_64) passed PNETN1 glibc-headers-2.5-42 glibc-headers-2.5( x86_64) passed Result: Package existence check passed for "glibc-headers-2.5( x86_64)" Check: Package existence for "gcc-c++-4.1.2 (x86_64)( x86_64)" Node Name Available Required Comment ------------ ------------------------ ------------------------ ---------- PNETN3 gcc-c++-4.1.2-46.el5 (x86_64) gcc-c++-4.1.2 (x86_64)( x86_64) passed PNETN2 gcc-c++-4.1.2-46.el5 (x86_64) gcc-c++-4.1.2 (x86_64)( x86_64) passed PNETN1 gcc-c++-4.1.2-46.el5 (x86_64) gcc-c++-4.1.2 (x86_64)( x86_64) passed Result: Package existence check passed for "gcc-c++-4.1.2 (x86_64)( x86_64)" Check: Package existence for "libaio-devel-0.3.106 (x86_64)( x86_64)" Node Name Available Required Comment ------------ ------------------------ ------------------------ ---------- PNETN3 libaio-devel-0.3.106-3.2 (x86_64) libaio-devel-0.3.106 (x86_64)( x86_64) passed PNETN2 libaio-devel-0.3.106-3.2 (x86_64) libaio-devel-0.3.106 (x86_64)( x86_64) passed PNETN1 libaio-devel-0.3.106-3.2 (x86_64) libaio-devel-0.3.106 (x86_64)( x86_64) passed Result: Package existence check passed for "libaio-devel-0.3.106 (x86_64)( x86_64)" Check: Package existence for "libgcc-4.1.2 (x86_64)( x86_64)" Node Name Available Required Comment ------------ ------------------------ ------------------------ ---------- PNETN3 libgcc-4.1.2-46.el5 (x86_64) libgcc-4.1.2 (x86_64)( x86_64) passed PNETN2 libgcc-4.1.2-46.el5 (x86_64) libgcc-4.1.2 (x86_64)( x86_64) passed PNETN1 libgcc-4.1.2-46.el5 (x86_64) libgcc-4.1.2 (x86_64)( x86_64) passed Result: Package existence check passed for "libgcc-4.1.2 (x86_64)( x86_64)" Check: Package existence for "libstdc++-4.1.2 (x86_64)( x86_64)" Node Name Available Required Comment ------------ ------------------------ ------------------------ ---------- PNETN3 libstdc++-4.1.2-46.el5 (x86_64) libstdc++-4.1.2 (x86_64)( x86_64) passed PNETN2 libstdc++-4.1.2-46.el5 (x86_64) libstdc++-4.1.2 (x86_64)( x86_64) passed PNETN1 libstdc++-4.1.2-46.el5 (x86_64) libstdc++-4.1.2 (x86_64)( x86_64) passed Result: Package existence check passed for "libstdc++-4.1.2 (x86_64)( x86_64)" Check: Package existence for "libstdc++-devel-4.1.2 (x86_64)( x86_64)" Node Name Available Required Comment ------------ ------------------------ ------------------------ ---------- PNETN3 libstdc++-devel-4.1.2-46.el5 (x86_64) libstdc++-devel-4.1.2 (x86_64)( x86_64) passed PNETN2 libstdc++-devel-4.1.2-46.el5 (x86_64) libstdc++-devel-4.1.2 (x86_64)( x86_64) passed PNETN1 libstdc++-devel-4.1.2-46.el5 (x86_64) libstdc++-devel-4.1.2 (x86_64)( x86_64) passed Result: Package existence check passed for "libstdc++-devel-4.1.2 (x86_64)( x86_64)" Check: Package existence for "sysstat-7.0.2( x86_64)" Node Name Available Required Comment ------------ ------------------------ ------------------------ ---------- PNETN3 sysstat-7.0.2-3.el5 sysstat-7.0.2( x86_64) passed PNETN2 sysstat-7.0.2-3.el5 sysstat-7.0.2( x86_64) passed PNETN1 sysstat-7.0.2-3.el5 sysstat-7.0.2( x86_64) passed Result: Package existence check passed for "sysstat-7.0.2( x86_64)" Check: Package existence for "ksh-20060214( x86_64)" Node Name Available Required Comment ------------ ------------------------ ------------------------ ---------- PNETN3 ksh-20080202-14.el5 ksh-20060214( x86_64) passed PNETN2 ksh-20080202-14.el5 ksh-20060214( x86_64) passed PNETN1 ksh-20080202-14.el5 ksh-20060214( x86_64) passed Result: Package existence check passed for "ksh-20060214( x86_64)" Checking for multiple users with UID value 0 Result: Check for multiple users with UID value 0 passed Check: Current group ID Result: Current group ID check passed Starting Clock synchronization checks using Network Time Protocol(NTP)... NTP Configuration file check started... Network Time Protocol(NTP) configuration file not found on any of the nodes. Oracle Cluster Time Synchronization Service(CTSS) can be used instead of NTP for time synchronization on the cluster nodes No NTP Daemons or Services were found to be running Result: Clock synchronization check using Network Time Protocol(NTP) passed Checking Core file name pattern consistency... Core file name pattern consistency check passed. Checking to make sure user "grid" is not in "root" group Node Name Status Comment ------------ ------------------------ ------------------------ PNETN3 does not exist passed PNETN2 does not exist passed PNETN1 does not exist passed Result: User "grid" is not part of "root" group. Check passed Check default user file creation mask Node Name Available Required Comment ------------ ------------------------ ------------------------ ---------- PNETN3 0022 0022 passed PNETN2 0022 0022 passed PNETN1 unknown 0022 failed Result: Default user file creation mask check failed Checking consistency of file "/etc/resolv.conf" across nodes Checking the file "/etc/resolv.conf" to make sure only one of domain and search entries is defined File "/etc/resolv.conf" does not have both domain and search entries defined Checking if domain entry in file "/etc/resolv.conf" is consistent across the nodes... domain entry in file "/etc/resolv.conf" is consistent across nodes Checking if search entry in file "/etc/resolv.conf" is consistent across the nodes... search entry in file "/etc/resolv.conf" is consistent across nodes Checking file "/etc/resolv.conf" to make sure that only one search entry is defined All nodes have one search entry defined in file "/etc/resolv.conf" Checking all nodes to make sure that search entry is "localdomain.com" as found on node "PNETN3" All nodes of the cluster have same value for 'search' Checking DNS response time for an unreachable node Node Name Status ------------------------------------ ------------------------ PNETN3 failed PNETN2 failed PNETN1 failed PRVF-5636 : The DNS response time for an unreachable node exceeded "15000" ms on following nodes: PNETN3,PNETN2,PNETN1 File "/etc/resolv.conf" is not consistent across nodes Check: Time zone consistency Result: Time zone consistency check passed Starting check for Huge Pages Existence ... Check for Huge Pages Existence passed Starting check for Hardware Clock synchronization at shutdown ... Check for Hardware Clock synchronization at shutdown passed Pre-check for cluster services setup was unsuccessful on all the nodes.
===============================
I have skipped DNS response time/ user mask in my pre-requirement.
Check DNS response time using
time nslookup
===============================
---INSTALL GI.
Below error is result you can get if you set disk permission using /etc/rc.d/rc.local
INFO: Default user file creation mask check failed
INFO: Check failed on nodes:
INFO: PNETN1
INFO: Checking cluster integrity...
INFO: Checks did not pass for the following node(s):
INFO: PNETN1
INFO:
WARNING:
INFO: Completed Plugin named: Oracle Cluster Verification Utility
---INSTALL ORACLE
[root@PNETN1 grid]# ./root.sh
Running Oracle 11g root script...
The following environment variables are set as:
ORACLE_OWNER= grid
ORACLE_HOME= /u00/app/11.2.0/grid
Enter the full pathname of the local bin directory: [/usr/local/bin]:
Copying dbhome to /usr/local/bin ...
Copying oraenv to /usr/local/bin ...
Copying coraenv to /usr/local/bin ...
Creating /etc/oratab file...
Entries will be added to the /etc/oratab file as needed by
Database Configuration Assistant when a database is created
Finished running generic part of root script.
Now product-specific root actions will be performed.
Using configuration parameter file: /u00/app/11.2.0/grid/crs/install/crsconfig_params
Creating trace directory
LOCAL ADD MODE
Creating OCR keys for user 'root', privgrp 'root'..
Operation successful.
OLR initialization - successful
root wallet
root wallet cert
root cert export
peer wallet
profile reader wallet
pa wallet
peer wallet keys
pa wallet keys
peer cert request
pa cert request
peer cert
pa cert
peer root cert TP
profile reader root cert TP
pa root cert TP
peer pa cert TP
pa peer cert TP
profile reader pa cert TP
profile reader peer cert TP
peer user cert
pa user cert
Adding daemon to inittab
ACFS-9200: Supported
ACFS-9300: ADVM/ACFS distribution files found.
ACFS-9307: Installing requested ADVM/ACFS software.
ACFS-9308: Loading installed ADVM/ACFS drivers.
ACFS-9321: Creating udev for ADVM/ACFS.
ACFS-9323: Creating module dependencies - this may take some time.
ACFS-9327: Verifying ADVM/ACFS devices.
ACFS-9309: ADVM/ACFS installation correctness verified.
CRS-2672: Attempting to start 'ora.mdnsd' on 'pnetn1'
CRS-2676: Start of 'ora.mdnsd' on 'pnetn1' succeeded
CRS-2672: Attempting to start 'ora.gpnpd' on 'pnetn1'
CRS-2676: Start of 'ora.gpnpd' on 'pnetn1' succeeded
CRS-2672: Attempting to start 'ora.cssdmonitor' on 'pnetn1'
CRS-2672: Attempting to start 'ora.gipcd' on 'pnetn1'
CRS-2676: Start of 'ora.gipcd' on 'pnetn1' succeeded
CRS-2676: Start of 'ora.cssdmonitor' on 'pnetn1' succeeded
CRS-2672: Attempting to start 'ora.cssd' on 'pnetn1'
CRS-2672: Attempting to start 'ora.diskmon' on 'pnetn1'
CRS-2676: Start of 'ora.diskmon' on 'pnetn1' succeeded
CRS-2676: Start of 'ora.cssd' on 'pnetn1' succeeded
ASM created and started successfully.
Disk Group CRS created successfully.
clscfg: -install mode specified
Successfully accumulated necessary OCR keys.
Creating OCR keys for user 'root', privgrp 'root'..
Operation successful.
CRS-4256: Updating the profile
Successful addition of voting disk 3e59c3d836244f36bf834c0c3c9f5484.
Successfully replaced voting disk group with +CRS.
CRS-4256: Updating the profile
CRS-4266: Voting file(s) successfully replaced
## STATE File Universal Id File Name Disk group
-- ----- ----------------- --------- ---------
1. ONLINE 3e59c3d836244f36bf834c0c3c9f5484 (/dev/sdb1) [CRS]
Located 1 voting disk(s).
CRS-2672: Attempting to start 'ora.asm' on 'pnetn1'
CRS-2676: Start of 'ora.asm' on 'pnetn1' succeeded
CRS-2672: Attempting to start 'ora.CRS.dg' on 'pnetn1'
CRS-2676: Start of 'ora.CRS.dg' on 'pnetn1' succeeded
ACFS-9200: Supported
ACFS-9200: Supported
CRS-2672: Attempting to start 'ora.registry.acfs' on 'pnetn1'
CRS-2676: Start of 'ora.registry.acfs' on 'pnetn1' succeeded
Configure Oracle Grid Infrastructure for a Cluster ... succeeded
CHECK ALL RESOURCES.
---------------------------------------
[root@PNETN1 ~]# crsctl stat res -t
--------------------------------------------------------------------------------
NAME TARGET STATE SERVER STATE_DETAILS
--------------------------------------------------------------------------------
Local Resources
--------------------------------------------------------------------------------
ora.CRS.dg
ONLINE ONLINE pnetn1
ONLINE ONLINE pnetn2
ONLINE ONLINE pnetn3
ora.LISTENER.lsnr
ONLINE ONLINE pnetn1
ONLINE ONLINE pnetn2
ONLINE ONLINE pnetn3
ora.asm
ONLINE ONLINE pnetn1
ONLINE ONLINE pnetn2
ONLINE ONLINE pnetn3
ora.gsd
OFFLINE OFFLINE pnetn1
OFFLINE OFFLINE pnetn2
OFFLINE OFFLINE pnetn3
ora.net1.network
ONLINE ONLINE pnetn1
ONLINE ONLINE pnetn2
ONLINE ONLINE pnetn3
ora.ons
ONLINE ONLINE pnetn1
ONLINE ONLINE pnetn2
ONLINE ONLINE pnetn3
ora.registry.acfs
ONLINE ONLINE pnetn1
ONLINE ONLINE pnetn2
ONLINE ONLINE pnetn3
--------------------------------------------------------------------------------
Cluster Resources
--------------------------------------------------------------------------------
ora.LISTENER_SCAN1.lsnr
1 ONLINE ONLINE pnetn3
ora.LISTENER_SCAN2.lsnr
1 ONLINE ONLINE pnetn1
ora.LISTENER_SCAN3.lsnr
1 ONLINE ONLINE pnetn2
ora.cvu
1 ONLINE ONLINE pnetn2
ora.oc4j
1 ONLINE ONLINE pnetn2
ora.pnetn1.vip
1 ONLINE ONLINE pnetn1
ora.pnetn2.vip
1 ONLINE ONLINE pnetn2
ora.pnetn3.vip
1 ONLINE ONLINE pnetn3
ora.scan1.vip
1 ONLINE ONLINE pnetn3
ora.scan2.vip
1 ONLINE ONLINE pnetn1
ora.scan3.vip
1 ONLINE ONLINE pnetn2
[root@PNETN1 ~]#
RUN DATABASE root.sh
[root@PNETN3 db_1]# ./root.sh
Running Oracle 11g root script...
The following environment variables are set as:
ORACLE_OWNER= oracle
ORACLE_HOME= /u01/app/oracle/11.2.0/db_1
Enter the full pathname of the local bin directory: [/usr/local/bin]:
The contents of "dbhome" have not changed. No need to overwrite.
The contents of "oraenv" have not changed. No need to overwrite.
The contents of "coraenv" have not changed. No need to overwrite.
Entries will be added to the /etc/oratab file as needed by
Database Configuration Assistant when a database is created
Finished running generic part of root script.
Now product-specific root actions will be performed.
Finished product-specific root actions.
CREATE DISKGROUP For DB.
PNETN1@:/home/grid : $. oraenv ORACLE_SID = [grid] ? +ASM1 The Oracle base has been set to /u00/app/grid PNETN1@:/home/grid :+ASM1 $sqlplus / as sysasm Oracle Database 11g Enterprise Edition Release 11.2.0.2.0 - 64bit Production SQL> CREATE DISKGROUP EDW_DATA EXTERNAL REDUNDANCY DISK '/dev/sdc1','/dev/sdd1','/dev/sde1'; Diskgroup created. SQL> ----MOUNT DISKGROUP ON OTHER NODES. PNETN2@:/home/grid :+ASM2 $sqlplus / as sysasm SQL> ALTER DISKGROUP EDW_DATA MOUNT; Diskgroup altered.
---------------RUN DBCA... SELECT EDW_DATA DISKGROUP for DB CREATION.
---------------LOGIN AS ORACLE and TEST READ/WRITE.
PNETN1@:/home/oracle :EDWP1 $. oraenv ORACLE_SID = [EDWP1] ? EDWP1 The Oracle base has been set to /u01/app/oracle PNETN1@:/home/oracle :EDWP1 $sqlplus / as sysdba SQL*Plus: Release 11.2.0.2.0 Production on Thu Jul 26 06:47:10 2012 Copyright (c) 1982, 2010, Oracle. All rights reserved. Connected to: Oracle Database 11g Enterprise Edition Release 11.2.0.2.0 - 64bit Production With the Partitioning, Real Application Clusters, Automatic Storage Management, OLAP, Data Mining and Real Application Testing options SQL> CREATE TABLESPACE TEST DATAFILE '+EDW_DATA' size 50m; Tablespace created. SQL> DROP TABLESPACE TEST INCLUDING CONTENTS; Tablespace dropped.
