11gRAC增加节点操作

以下操作参考oracle官方文档，且经过详细测试

前期环境确认，包括主机资源，内存资源等等

用户创建，组创建，创建基于已经搭建好的集群

ssh互信等等，前期准备工作就不详细叙述

确认资源配置正常（预检）

Verify the hardware and operating system installations with the Cluster Verification

Utility (CVU).

After you configure the hardware and operating systems on the nodes you want to add,

you can run the following commands to verify that the nodes you want to add are

reachable by other nodes in the cluster.

[grid@s2-11g ~]$ cluvfy stage -post hwos -n s1-11g -verbose

Performing post-checks for hardware and operating system setup

对于预检的输出内容进行确认

预检报错：

Checking shared

WARNING:

s1-11g:PRVF-7017 : Package cvuqdisk not installed

s1-11g

No shared storage found

Shared storage check failed on nodes "s1-11g"

在一节点上安装

[root@s1-11g rpm]# rpm -ivh

########################################### [100%]

1:cvuqdisk ########################################### [100%]

重新预检，预检通过

1.开始添加节点grid：

1.1

Ensure that you have successfully installed Oracle Clusterware on at least one node in

your cluster environment. To perform the following procedure, Grid_home must identify

your successfully installed Oracle Clusterware home.

1.2

确认节点信息：

cluvfy stage -pre nodeadd -n s1-11g [-fixup [-fixupdir fixup_dir]] [-verbose]

1.3 To extend the Grid Infrastructure home to the node3, navigate to the

Grid_home/oui/bin directory on node1 and run the script using the following

syntax, where node3 is the name of the node that you are adding and node3-vip is the

VIP name for the node:

1.4添加节点：

If you are using Grid Naming Service (GNS), run the following command:

$ ./ "CLUSTER_NEW_NODES={node3}"

If you are not using GNS, run the following command:

$ /oracle/grid/oui/bin/ "CLUSTER_NEW_NODES={s1-11g}"

"CLUSTER_NEW_VIRTUAL_HOSTNAMES={s1-11g-vip}"

[grid@s2-11g ~]$ /oracle/grid/oui/bin/ "CLUSTER_NEW_NODES={s1-11g}"

"CLUSTER_NEW_VIRTUAL_HOSTNAMES={s1-11g-vip}"

如果预检有报错的话将不会继续，需要修复该错误，本次预检报错：

PRVG-11050 : No matching interfaces。。。。。

检查发现OCR中有网卡eth3的信息：

[grid@s2-11g ~]$ oifcfg getif

eth0 172.16.0.0 global public

eth2 10.10.0.0 global cluster_interconnect

eth3 20.20.0.0 global cluster_interconnect

[grid@s2-11g ~]$ oifcfg delif eth3

[grid@s2-11g ~]$ oifcfg delif -global eth3

[grid@s2-11g ~]$ oifcfg getif

eth0 172.16.0.0 global public

eth2 10.10.0.0 global cluster_interconnect

预检通过，正常

备注：当然，如果预检不通过，或者某些原因我们需要跳过预检环节，我们可以再运行脚本前做export：

export IGNORE_PREADDNODE_CHECKS=Y

开始拷贝介质：

Copying to remote nodes (Wednesday, July 10, 2013 9:10:30 AM CST)

......................................................WARNING:Error while copying

directory /oracle/grid with exclude file list '/tmp/OraInstall2013-07-10_09-10-13AM/' to nodes 's1-11g'. [PRKC-PRCF-2015 : One or more commands

were not executed successfully on one or more nodes : ]

----------------------------------------------------------------------------------

s1-11g:

PRCF-2023 : The following contents are not transferred as they are non-readable.

Directories:

1) /oracle/grid/-

Files:

----------------------------------------------------------------------------------

Refer to '/oracle/oraInventory/logs/addNodeActions2013-07-10_' for

details. You may fix the errors on the required remote nodes. Refer to the install

guide for error recovery.

55% Done.

Home copied to new nodes

报错：

检查'/tmp/OraInstall2013-07-10_09-10-13AM/ 文件，不存在

不过从报错信息上看，应该是/oracle/grid/ 文件夹的权限有问题：

[root@s2-11g grid]# ls -rtl /

total 20

drwxr-x--T 6 root root 4096 Apr 27 14:02 wallets

drwxr-x--- 4 root root 4096 Apr 27 14:02 s2-11g

drwxr-x--- 3 root root 4096 Apr 27 14:02 profiles

-rw-r----- 1 root root 167 Apr 27 14:02

drwxr-x--- 2 root root 4096 Apr 27 14:02 init

文件夹确实存在问题，从文件夹看，是一个备份的文件，可以移除或者对权限进行修改

移除后，删除一节点已经拷贝软件，再次add

其实在add node的过程中，权限是个比较麻烦的东西，特别是里面放置了一些通过root用户创建的文件等

参见文档：

Fail: PRCF-2023 : The following contents are not transferred as they are

non-readable [ID 1526405.1]

拷贝结束：

Instantiating scripts for add node (Wednesday, July 10, 2013 9:20:26 AM CST)

. 1% Done.

Instantiation of add node scripts complete

Copying to remote nodes (Wednesday, July 10, 2013 9:20:30 AM CST)

............................................................................................... 96% Done.

Home copied to new nodes

Saving inventory on nodes (Wednesday, July 10, 2013 9:25:36 AM CST)

. 100% Done.

Save inventory complete

WARNING:

The following configuration scripts need to be executed as the "root" user in each

new cluster node. Each script in the list below is followed by a list of nodes.

/oracle/grid/ #On nodes s1-11g

To execute the configuration scripts:

1. Open a terminal window

2. Log in as "root"

3. Run the scripts in each cluster node

The Cluster Node Addition of /oracle/grid was successful.

Please check '/tmp/' for more details.

新加节点运行脚本：

[root@s1-11g ~]# /oracle/grid/

Performing root user operation for Oracle 11g

The following environment variables are set as:

ORACLE_OWNER= grid

ORACLE_HOME= /oracle/grid

Enter the full pathname of the local bin directory: [/usr/local/bin]:

The contents of "dbhome" have not changed. No need to overwrite.

The contents of "oraenv" have not changed. No need to overwrite.

The contents of "coraenv" have not changed. No need to overwrite.

Creating /etc/

Entries will be added to the /etc/oratab file as needed by

Database Configuration Assistant when a database is created

Finished running generic part of root script.

Now product-specific root actions will be performed.

Using configuration parameter file: /oracle/grid/crs/install/crsconfig_params

Creating trace directory

User ignored Prerequisites during installation

OLR initialization - successful

Adding Clusterware entries to inittab

CRS-4402: The CSS daemon was started in exclusive mode but found an active CSS daemon

on node s2-11g, number 2, and is terminating

An active cluster was found during exclusive startup, restarting to join the cluster

clscfg: EXISTING configuration version 5 detected.

clscfg: version 5 is 11g Release 2.

Successfully accumulated necessary OCR keys.

Creating OCR keys for user 'root', privgrp 'root'..

Operation successful.

Configure Oracle Grid Infrastructure for a Cluster ... succeeded

新节点grid软件增加完毕

增加：

If you have an Oracle RAC or Oracle RAC One Node database configured on the cluster

and you have a local Oracle home, then do the following to extend the Oracle database

home to node3:

Navigate to the Oracle_home/oui/bin directory on node1 and run the script

as the user that installed Oracle RAC using the following syntax:

oracle 用户：

[oracle@s2-11g bin]$ /oracle/app/product/11.2.0/db_1/oui/bin/

"CLUSTER_NEW_NODES={s1-11g}"

Performing pre-checks for node addition

Checking

Node reachability check passed from node "s2-11g"

Checking

User equivalence check passed for user "oracle"

WARNING:

Node "s1-11g" already appears to be part of cluster

Pre-check for node addition was successful.

Starting Oracle

Checking swap space: must be greater than 500 MB. Actual 9757 MB Passed

Oracle Universal Installer, Version 11.2.0.3.0 Production

Copyright (C) 1999, 2011, Oracle. All rights reserved.

Performing tests to see whether nodes s1-11g are available

............................................................... 100% Done.

Instantiating scripts for add node (Wednesday, July 10, 2013 9:46:07 AM CST)

. 1% Done.

Instantiation of add node scripts complete

Copying to remote nodes (Wednesday, July 10, 2013 9:46:23 AM CST)

............................................................................................... 96% Done.

Home copied to new nodes

Saving inventory on nodes (Wednesday, July 10, 2013 9:53:48 AM CST)

. 100% Done.

Save inventory complete

WARNING:

The following configuration scripts need to be executed as the "root" user in each new

cluster node. Each script in the list below is followed by a list of nodes.

/oracle/app/product/11.2.0/db_1/ #On nodes s1-11g

To execute the configuration scripts:

1. Open a terminal window

2. Log in as "root"

3. Run the scripts in each cluster node

The Cluster Node Addition of /oracle/app/product/11.2.0/db_1 was successful.

Please check '/tmp/' for more details.

[oracle@s2-11g bin]$

至此，rdbms软件添加完毕

3.

实例添加：

实例添加有两种方式，可以是静默的手动添加，也可以通过DBCA调用图形界面添加

手动添加涉及到以下几个步骤：

修改参数文件

添加redo，undo

激活thread

启动实例

dbca添加如下：

在运行正常节点上运行dbca：