How to Apply patch on Oracle RAC
Prerequisite:-
>> Check Oracle service status
[oracle@rac-node1 ~]$crs_stat -t
Name Type Target State Host
------------------------------------------------------------
ora....SM1.asm application ONLINE ONLINE dbsr...ode1
ora....E1.lsnr application ONLINE ONLINE dbsr...ode1
ora....de1.gsd application ONLINE ONLINE dbsr...ode1
ora....de1.ons application ONLINE ONLINE dbsr...ode1
ora....de1.vip application ONLINE ONLINE dbsr...ode1
ora....SM2.asm application ONLINE ONLINE dbsr...ode2
ora....E2.lsnr application ONLINE ONLINE dbsr...ode2
ora....de2.gsd application ONLINE ONLINE dbsr...ode2
ora....de2.ons application ONLINE ONLINE dbsr...ode2
ora....de2.vip application ONLINE ONLINE dbsr...ode2
ora.prod.db application ONLINE ONLINE dbsr...ode1
ora....b1.inst application ONLINE ONLINE dbsr...ode1
ora....b2.inst application ONLINE ONLINE dbsr...ode2
[oracle@rac-node1 ~]$
>> Stop database instance one by one
[oracle@rac-node1 ~]$ srvctl stop instance -i prod1 -d prod
[oracle@rac-node1 ~]$ srvctl stop instance -i prod2 -d prod
>> Stop ASM instance on each node one by one
[oracle@rac-node1 ~]$ srvctl stop asm -n rac-node1
[oracle@rac-node1 ~]$ srvctl stop asm -n rac-node2
>> Stop node applications one by one on each node
[oracle@rac-node1 ~]$ srvctl stop nodeapps -n rac-node1
[oracle@rac-node1 ~]$ srvctl stop nodeapps -n rac-node2
>> Check status again
[oracle@rac-node1 ~]$ crs_stat -t
Name Type Target State Host
------------------------------------------------------------
ora....SM1.asm application OFFLINE OFFLINE
ora....E1.lsnr application OFFLINE OFFLINE
ora....de1.gsd application OFFLINE OFFLINE
ora....de1.ons application OFFLINE OFFLINE
ora....de1.vip application OFFLINE OFFLINE
ora....SM2.asm application OFFLINE OFFLINE
ora....E2.lsnr application OFFLINE OFFLINE
ora....de2.gsd application OFFLINE OFFLINE
ora....de2.ons application OFFLINE OFFLINE
ora....de2.vip application OFFLINE OFFLINE
ora.prod.db application OFFLINE OFFLINE
ora....b1.inst application OFFLINE OFFLINE
ora....b2.inst application OFFLINE OFFLINE
[oracle@rac-node1 ~]
>> Download Patch from Metalink.
>> Unzip patch folder after that read readme file.
>> Go to Opatch top.
[oracle@rac-node1 rman]$ cd 9949948
OPatch should be in your path as follows
Set Oracle_Home.
[oracle@rac-node1 9949948]$ export PATH=$PATH:/u01/app/oracle/product/10.2.0 /db_1/OPatch
>> Start apply Opatch
[oracle@rac-node1 9949948]$ opatch apply
Invoking OPatch 10.2.0.4.9
Oracle Interim Patch Installer version 10.2.0.4.9
Copyright (c) 2009, Oracle Corporation. All rights reserved.
Oracle Home : /u01/app/oracle/product/10.2.0/db_1
Central Inventory : /u01/app/oracle/OraInventory
from : /etc/oraInst.loc
OPatch version : 10.2.0.4.9
OUI version : 10.2.0.5.0
OUI location : /u01/app/oracle/product/10.2.0/db_1/oui
Log file location : /u01/app/oracle/product/10.2.0/db_1/cfgtoollogs/opatch/opatch2012-09-20_23-38-29PM.log
Patch history file: /u01/app/oracle/product/10.2.0/db_1/cfgtoollogs/opatch/opatch_history.txt
ApplySession applying interim patch '9949948' to OH '/u01/app/oracle/product/10.2.0/db_1'
Running prerequisite checks...
OPatch detected the node list and the local node from the inventory. OPatch will patch the local system then propagate This node is part of an Oracle Real Application Cluster.
Remote nodes: 'rac-node2'
Local node: 'rac-node1'
Please shutdown Oracle instances running out of this ORACLE_HOME on the local system.
(Oracle Home = '/u01/app/oracle/product/10.2.0/db_1')
Is the local system ready for patching? [y|n]
Y ==> I put yes
User Responded with: Y
Backing up files and inventory (not for auto-rollback) for the Oracle Home
Backing up files affected by the patch '9949948' for restore. This might take a while...
Backing up files affected by the patch '9949948' for rollback. This might take a while...
Patching component oracle.rdbms, 10.2.0.5.0...
Updating archive file "/u01/app/oracle/product/10.2.0/db_1/lib/libserver10.a" with "lib/libserver10.a/ksfd.o"
Running make for target ioracle
ApplySession adding interim patch '9949948' to inventory
Verifying the update...
Inventory check OK: Patch ID 9949948 is registered in Oracle Home inventory with proper meta-data.
Files check OK: Files from Patch ID 9949948 are present in Oracle Home.
The local system has been patched. You can restart Oracle instances on it.
Patching in rolling mode.
The node 'rac-node2' will be patched next.
Please shutdown Oracle instances running out of this ORACLE_HOME on 'rac-node2'.
(Oracle Home = '/u01/app/oracle/product/10.2.0/db_1')
Is the node ready for patching? [y|n]
y ==> I put yes for node2
User Responded with: Y
Updating nodes 'rac-node2'
Apply-related files are:
FP = "/u01/app/oracle/product/10.2.0/db_1/.patch_storage/9949948_Aug_13_2010_06_35_20/rac/copy_files.txt"
DP = "/u01/app/oracle/product/10.2.0/db_1/.patch_storage/9949948_Aug_13_2010_06_35_20/rac/copy_dirs.txt"
MP = "/u01/app/oracle/product/10.2.0/db_1/.patch_storage/9949948_Aug_13_2010_06_35_20/rac/make_cmds.txt"
RC = "/u01/app/oracle/product/10.2.0/db_1/.patch_storage/9949948_Aug_13_2010_06_35_20/rac/remote_cmds.txt"
Instantiating the file "/u01/app/oracle/product/10.2.0/db_1/.patch_storage/9949948_Aug_13_2010_06_35_20/rac/copy_Propagating files to remote nodes...
Instantiating the file "/u01/app/oracle/product/10.2.0/db_1/.patch_storage/9949948_Aug_13_2010_06_35_20/rac/copy_Propagating directories to remote nodes...
Instantiating the file "/u01/app/oracle/product/10.2.0/db_1/.patch_storage/9949948_Aug_13_2010_06_35_20/rac/make_Running command on remote node 'rac-node2':
cd /u01/app/oracle/product/10.2.0/db_1/rdbms/lib; /usr/bin/make -f ins_rdbms.mk ioracle ORACLE_HOME=/u01/app/oracle/The node 'rac-node2' has been patched. You can restart Oracle instances on it.
There were relinks on remote nodes. Remember to check the binary size and timestamp on the nodes 'rac-node2' .
The following make commands were invoked on remote nodes:
'cd /u01/app/oracle/product/10.2.0/db_1/rdbms/lib; /usr/bin/make -f ins_rdbms.mk ioracle ORACLE_HOME=/u01/app/oracle/'
OPatch succeeded.
Start Oracle services on both nodes
[oracle@rac-node1 9949948]$ srvctl start nodeapps -n rac-node1
[oracle@rac-node1 9949948]$ srvctl start nodeapps -n rac-node2
[oracle@rac-node1 9949948]$ srvctl start asm -n rac-node1
[oracle@rac-node1 9949948]$ srvctl start asm -n rac-node2
[oracle@rac-node1 9949948]$ crs_stat -t
Name Type Target State Host
------------------------------------------------------------
ora....SM1.asm application ONLINE ONLINE dbsr...ode1
ora....E1.lsnr application ONLINE ONLINE dbsr...ode1
ora....de1.gsd application ONLINE ONLINE dbsr...ode1
ora....de1.ons application ONLINE ONLINE dbsr...ode1
ora....de1.vip application ONLINE ONLINE dbsr...ode1
ora....SM2.asm application ONLINE ONLINE dbsr...ode2
ora....E2.lsnr application ONLINE ONLINE dbsr...ode2
ora....de2.gsd application ONLINE ONLINE dbsr...ode2
ora....de2.ons application ONLINE ONLINE dbsr...ode2
ora....de2.vip application ONLINE ONLINE dbsr...ode2
ora.prod.db application OFFLINE OFFLINE
ora....b1.inst application OFFLINE OFFLINE
ora....b2.inst application OFFLINE OFFLINE
>> Start database instances on both nodes:-
[oracle@rac-node1 9949948]$ srvctl start instance -i prod1 -d prod
[oracle@rac-node1 9949948]$ srvctl start instance -i prod1 -d prod
[oracle@rac-node1 9949948]$ crs-stat -t
Name Type Target State Host
------------------------------------------------------------
ora....SM1.asm application ONLINE ONLINE dbsr...ode1
ora....E1.lsnr application ONLINE ONLINE dbsr...ode1
ora....de1.gsd application ONLINE ONLINE dbsr...ode1
ora....de1.ons application ONLINE ONLINE dbsr...ode1
ora....de1.vip application ONLINE ONLINE dbsr...ode1
ora....SM2.asm application ONLINE ONLINE dbsr...ode2
ora....E2.lsnr application ONLINE ONLINE dbsr...ode2
ora....de2.gsd application ONLINE ONLINE dbsr...ode2
ora....de2.ons application ONLINE ONLINE dbsr...ode2
ora....de2.vip application ONLINE ONLINE dbsr...ode2
ora.prod.db application ONLINE ONLINE dbsr...ode2
ora....b1.inst application ONLINE ONLINE dbsr...ode1
ora....b2.inst application ONLINE ONLINE dbsr...ode2
[oracle@rac-node1 script]$ sqlplus /nolog
SQL*Plus: Release 10.2.0.5.0 – Production on Thu Sep 20 23:44:01 2012
Copyright (c) 1982, 2010, Oracle. All Rights Reserved.
SQL>conn /as sysdba
Connected.
SQL>
Prerequisite:-
>> Check Oracle service status
[oracle@rac-node1 ~]$crs_stat -t
Name Type Target State Host
------------------------------------------------------------
ora....SM1.asm application ONLINE ONLINE dbsr...ode1
ora....E1.lsnr application ONLINE ONLINE dbsr...ode1
ora....de1.gsd application ONLINE ONLINE dbsr...ode1
ora....de1.ons application ONLINE ONLINE dbsr...ode1
ora....de1.vip application ONLINE ONLINE dbsr...ode1
ora....SM2.asm application ONLINE ONLINE dbsr...ode2
ora....E2.lsnr application ONLINE ONLINE dbsr...ode2
ora....de2.gsd application ONLINE ONLINE dbsr...ode2
ora....de2.ons application ONLINE ONLINE dbsr...ode2
ora....de2.vip application ONLINE ONLINE dbsr...ode2
ora.prod.db application ONLINE ONLINE dbsr...ode1
ora....b1.inst application ONLINE ONLINE dbsr...ode1
ora....b2.inst application ONLINE ONLINE dbsr...ode2
[oracle@rac-node1 ~]$
>> Stop database instance one by one
[oracle@rac-node1 ~]$ srvctl stop instance -i prod1 -d prod
[oracle@rac-node1 ~]$ srvctl stop instance -i prod2 -d prod
>> Stop ASM instance on each node one by one
[oracle@rac-node1 ~]$ srvctl stop asm -n rac-node1
[oracle@rac-node1 ~]$ srvctl stop asm -n rac-node2
>> Stop node applications one by one on each node
[oracle@rac-node1 ~]$ srvctl stop nodeapps -n rac-node1
[oracle@rac-node1 ~]$ srvctl stop nodeapps -n rac-node2
>> Check status again
[oracle@rac-node1 ~]$ crs_stat -t
Name Type Target State Host
------------------------------------------------------------
ora....SM1.asm application OFFLINE OFFLINE
ora....E1.lsnr application OFFLINE OFFLINE
ora....de1.gsd application OFFLINE OFFLINE
ora....de1.ons application OFFLINE OFFLINE
ora....de1.vip application OFFLINE OFFLINE
ora....SM2.asm application OFFLINE OFFLINE
ora....E2.lsnr application OFFLINE OFFLINE
ora....de2.gsd application OFFLINE OFFLINE
ora....de2.ons application OFFLINE OFFLINE
ora....de2.vip application OFFLINE OFFLINE
ora.prod.db application OFFLINE OFFLINE
ora....b1.inst application OFFLINE OFFLINE
ora....b2.inst application OFFLINE OFFLINE
[oracle@rac-node1 ~]
>> Download Patch from Metalink.
>> Unzip patch folder after that read readme file.
>> Go to Opatch top.
[oracle@rac-node1 rman]$ cd 9949948
OPatch should be in your path as follows
Set Oracle_Home.
[oracle@rac-node1 9949948]$ export PATH=$PATH:/u01/app/oracle/product/10.2.0 /db_1/OPatch
>> Start apply Opatch
[oracle@rac-node1 9949948]$ opatch apply
Invoking OPatch 10.2.0.4.9
Oracle Interim Patch Installer version 10.2.0.4.9
Copyright (c) 2009, Oracle Corporation. All rights reserved.
Oracle Home : /u01/app/oracle/product/10.2.0/db_1
Central Inventory : /u01/app/oracle/OraInventory
from : /etc/oraInst.loc
OPatch version : 10.2.0.4.9
OUI version : 10.2.0.5.0
OUI location : /u01/app/oracle/product/10.2.0/db_1/oui
Log file location : /u01/app/oracle/product/10.2.0/db_1/cfgtoollogs/opatch/opatch2012-09-20_23-38-29PM.log
Patch history file: /u01/app/oracle/product/10.2.0/db_1/cfgtoollogs/opatch/opatch_history.txt
ApplySession applying interim patch '9949948' to OH '/u01/app/oracle/product/10.2.0/db_1'
Running prerequisite checks...
OPatch detected the node list and the local node from the inventory. OPatch will patch the local system then propagate This node is part of an Oracle Real Application Cluster.
Remote nodes: 'rac-node2'
Local node: 'rac-node1'
Please shutdown Oracle instances running out of this ORACLE_HOME on the local system.
(Oracle Home = '/u01/app/oracle/product/10.2.0/db_1')
Is the local system ready for patching? [y|n]
Y ==> I put yes
User Responded with: Y
Backing up files and inventory (not for auto-rollback) for the Oracle Home
Backing up files affected by the patch '9949948' for restore. This might take a while...
Backing up files affected by the patch '9949948' for rollback. This might take a while...
Patching component oracle.rdbms, 10.2.0.5.0...
Updating archive file "/u01/app/oracle/product/10.2.0/db_1/lib/libserver10.a" with "lib/libserver10.a/ksfd.o"
Running make for target ioracle
ApplySession adding interim patch '9949948' to inventory
Verifying the update...
Inventory check OK: Patch ID 9949948 is registered in Oracle Home inventory with proper meta-data.
Files check OK: Files from Patch ID 9949948 are present in Oracle Home.
The local system has been patched. You can restart Oracle instances on it.
Patching in rolling mode.
The node 'rac-node2' will be patched next.
Please shutdown Oracle instances running out of this ORACLE_HOME on 'rac-node2'.
(Oracle Home = '/u01/app/oracle/product/10.2.0/db_1')
Is the node ready for patching? [y|n]
y ==> I put yes for node2
User Responded with: Y
Updating nodes 'rac-node2'
Apply-related files are:
FP = "/u01/app/oracle/product/10.2.0/db_1/.patch_storage/9949948_Aug_13_2010_06_35_20/rac/copy_files.txt"
DP = "/u01/app/oracle/product/10.2.0/db_1/.patch_storage/9949948_Aug_13_2010_06_35_20/rac/copy_dirs.txt"
MP = "/u01/app/oracle/product/10.2.0/db_1/.patch_storage/9949948_Aug_13_2010_06_35_20/rac/make_cmds.txt"
RC = "/u01/app/oracle/product/10.2.0/db_1/.patch_storage/9949948_Aug_13_2010_06_35_20/rac/remote_cmds.txt"
Instantiating the file "/u01/app/oracle/product/10.2.0/db_1/.patch_storage/9949948_Aug_13_2010_06_35_20/rac/copy_Propagating files to remote nodes...
Instantiating the file "/u01/app/oracle/product/10.2.0/db_1/.patch_storage/9949948_Aug_13_2010_06_35_20/rac/copy_Propagating directories to remote nodes...
Instantiating the file "/u01/app/oracle/product/10.2.0/db_1/.patch_storage/9949948_Aug_13_2010_06_35_20/rac/make_Running command on remote node 'rac-node2':
cd /u01/app/oracle/product/10.2.0/db_1/rdbms/lib; /usr/bin/make -f ins_rdbms.mk ioracle ORACLE_HOME=/u01/app/oracle/The node 'rac-node2' has been patched. You can restart Oracle instances on it.
There were relinks on remote nodes. Remember to check the binary size and timestamp on the nodes 'rac-node2' .
The following make commands were invoked on remote nodes:
'cd /u01/app/oracle/product/10.2.0/db_1/rdbms/lib; /usr/bin/make -f ins_rdbms.mk ioracle ORACLE_HOME=/u01/app/oracle/'
OPatch succeeded.
Start Oracle services on both nodes
[oracle@rac-node1 9949948]$ srvctl start nodeapps -n rac-node1
[oracle@rac-node1 9949948]$ srvctl start nodeapps -n rac-node2
[oracle@rac-node1 9949948]$ srvctl start asm -n rac-node1
[oracle@rac-node1 9949948]$ srvctl start asm -n rac-node2
[oracle@rac-node1 9949948]$ crs_stat -t
Name Type Target State Host
------------------------------------------------------------
ora....SM1.asm application ONLINE ONLINE dbsr...ode1
ora....E1.lsnr application ONLINE ONLINE dbsr...ode1
ora....de1.gsd application ONLINE ONLINE dbsr...ode1
ora....de1.ons application ONLINE ONLINE dbsr...ode1
ora....de1.vip application ONLINE ONLINE dbsr...ode1
ora....SM2.asm application ONLINE ONLINE dbsr...ode2
ora....E2.lsnr application ONLINE ONLINE dbsr...ode2
ora....de2.gsd application ONLINE ONLINE dbsr...ode2
ora....de2.ons application ONLINE ONLINE dbsr...ode2
ora....de2.vip application ONLINE ONLINE dbsr...ode2
ora.prod.db application OFFLINE OFFLINE
ora....b1.inst application OFFLINE OFFLINE
ora....b2.inst application OFFLINE OFFLINE
>> Start database instances on both nodes:-
[oracle@rac-node1 9949948]$ srvctl start instance -i prod1 -d prod
[oracle@rac-node1 9949948]$ srvctl start instance -i prod1 -d prod
[oracle@rac-node1 9949948]$ crs-stat -t
Name Type Target State Host
------------------------------------------------------------
ora....SM1.asm application ONLINE ONLINE dbsr...ode1
ora....E1.lsnr application ONLINE ONLINE dbsr...ode1
ora....de1.gsd application ONLINE ONLINE dbsr...ode1
ora....de1.ons application ONLINE ONLINE dbsr...ode1
ora....de1.vip application ONLINE ONLINE dbsr...ode1
ora....SM2.asm application ONLINE ONLINE dbsr...ode2
ora....E2.lsnr application ONLINE ONLINE dbsr...ode2
ora....de2.gsd application ONLINE ONLINE dbsr...ode2
ora....de2.ons application ONLINE ONLINE dbsr...ode2
ora....de2.vip application ONLINE ONLINE dbsr...ode2
ora.prod.db application ONLINE ONLINE dbsr...ode2
ora....b1.inst application ONLINE ONLINE dbsr...ode1
ora....b2.inst application ONLINE ONLINE dbsr...ode2
[oracle@rac-node1 script]$ sqlplus /nolog
SQL*Plus: Release 10.2.0.5.0 – Production on Thu Sep 20 23:44:01 2012
Copyright (c) 1982, 2010, Oracle. All Rights Reserved.
SQL>conn /as sysdba
Connected.
SQL>
