oracle rac 单个实例不能生成awr报告的问题

同事对rac集群生成性能报告时发现rac集群有一个实例没有生成awr快照,另一个实例快照正常。下面是具体处理步骤。
1号实例没有生成awr快照

SQL> select SNAP_ID,END_INTERVAL_TIME,instance_number from dba_hist_snapshot where instance_number=1;

no rows selected

2号实快照正常

SQL> set long 200   
SQL> set linesize 200
SQL> select * from ( select SNAP_ID,END_INTERVAL_TIME,instance_number from dba_hist_snapshot where instance_number=2 order by SNAP_ID desc) where rownum < =10;

   SNAP_ID END_INTERVAL_TIME                                                           INSTANCE_NUMBER
---------- --------------------------------------------------------------------------- ---------------
     24405 17-AUG-19 07.00.47.595 PM                                                                 2
     24404 17-AUG-19 06.00.42.150 PM                                                                 2
     24403 17-AUG-19 05.00.37.041 PM                                                                 2
     24402 17-AUG-19 04.00.31.774 PM                                                                 2
     24401 17-AUG-19 03.00.26.414 PM                                                                 2
     24400 17-AUG-19 02.00.21.176 PM                                                                 2
     24399 17-AUG-19 01.00.16.316 PM                                                                 2
     24398 17-AUG-19 12.00.10.997 PM                                                                 2
     24397 17-AUG-19 11.00.05.446 AM                                                                 2
     24396 17-AUG-19 10.00.59.801 AM                                                                 2

10 rows selected.

mmon进程与awr快照相关,mmnl与ash相关,如是查看两个实例的mmon与mmnl进程
2号实例

[root@db2 ~]# ps -ef | grep mmon
root     128329 127956  0 18:11 pts/2    00:00:00 grep mmon
oracle   201527      1  0  2018 ?        17:17:11 ora_mmon_RLZY2
[root@db2 ~]# ps -ef | grep mmnl
root     131772 127956  0 18:17 pts/2    00:00:00 grep mmnl
oracle   201531      1  0  2018 ?        1-06:06:24 ora_mmnl_RLZY2

1号实例

[root@db1 ~]# ps -ef | grep mmon
root     239020 238963  0 18:52 pts/2    00:00:00 grep mmon
[root@db1 ~]# ps -ef | grep mmnl
root     239052 238963  0 18:52 pts/2    00:00:00 grep mmnl

可以看到1号实例没有mmon与mmnl进程了。

如是查看1号实例的mmon进程的跟踪文件

[root@db1 trace]# ls -lrt *mmon*.trc
-rw-r----- 1 oracle asmadmin 1351052 Jan 19  2018 RLZY1_mmon_20073.trc
-rw-r----- 1 oracle asmadmin  173031 Jan 22  2018 RLZY1_mmon_49119.trc
[root@db1 trace]# more RLZY1_mmon_49119.trc
Trace file /u01/app/oracle/diag/rdbms/rlzy/RLZY1/trace/RLZY1_mmon_49119.trc
Oracle Database 11g Enterprise Edition Release 11.2.0.4.0 - 64bit Production
With the Partitioning, Real Application Clusters, Automatic Storage Management, OLAP,
Data Mining and Real Application Testing options
ORACLE_HOME = /u01/app/oracle/product/11.2.0/db_1
System name:    Linux
Node name:      db1
Release:        3.8.13-68.3.4.el6uek.x86_64
Version:        #2 SMP Tue Jul 14 15:03:36 PDT 2015
Machine:        x86_64
Instance name: RLZY1
Redo thread mounted by this instance: 1
Oracle process number: 36
Unix process pid: 49119, image: oracle@db1 (MMON)


*** 2018-01-19 13:55:20.030
*** SESSION ID:(1369.1) 2018-01-19 13:55:20.030
*** CLIENT ID:() 2018-01-19 13:55:20.030
*** SERVICE NAME:() 2018-01-19 13:55:20.030
*** MODULE NAME:() 2018-01-19 13:55:20.030
*** ACTION NAME:() 2018-01-19 13:55:20.030
 
minact-scn slave-status: grec-scn:0x0000.00000000 gmin-scn:0x0000.00000000 gcalc-scn:0x0000.00000000

*** 2018-01-19 14:00:20.643
minact-scn master-status: grec-scn:0x0e0e.55ad96ef gmin-scn:0x0e0e.55abf256 gcalc-scn:0x0e0e.55ac2a0a

..........
KEBM: MMON action policy violation. 'Block Cleanout Optim, Undo Segment Scan' viol=1; err=12751
minact-scn master-status: grec-scn:0x0e0e.5f0ebf8c gmin-scn:0x0e0e.5f0eac2e gcalc-scn:0x0e0e.5f0ead91
DDE rules only execution for: ORA 12751

*** 2018-01-22 07:06:04.060
----- START Event Driven Actions Dump ----
---- END Event Driven Actions Dump ----
----- START DDE Actions Dump -----
Executing SYNC actions
Executing ASYNC actions
----- START DDE Action: 'ORA_12751_DUMP' (Sync) -----
Runtime exceeded 300 seconds
Time limit violation detected at:
ksedsts()+465< -kspol_12751_dump()+145<-dbgdaExecuteAction()+1065<-dbgerRunAction()+109<-dbgerRunActions()+4134<-dbgexPhaseII()+1873<-dbgexProcessError()+2680<-dbgeExecuteForError()+88<-dbgePostErrorKGE()+2136<-dbkePostKGE_kgsf()+71<-kge
selv()+276<-ksesecl0()+162
<-ksucin()+147<-kcbzwb()+2727<-kcbgtcr()+31325<-ktucloUsMinScn()+539<-ktucloUsegScan()+992<-ksb_run_managed_action()+384<-ksbcti()+2490<-ksbabs()+1735<-kebm_mmon_main()+209<-ksbrdp()+1045<-opirip()+623<-opidrv()+603<-sou2o()+103<-opimai
_real()+250<-ssthrdmain()+265
<-main()+201<-__libc_start_main()+253Current Wait Stack:
 0: waiting for 'gc buffer busy acquire'
    file#=0x5, block#=0x278, class#=0x49
    wait_id=255378 seq_num=59358 snap_id=1
    wait times: snap=5 min 5 sec, exc=5 min 5 sec, total=5 min 5 sec
    wait times: max=infinite, heur=5 min 5 sec
    wait counts: calls=358 os=358
    in_wait=1 iflags=0x15a2
There is at least one session blocking this session.
  Dumping 1 direct blocker(s):
    inst: 1, sid: 990, ser: 1
  Dumping final blocker:
    inst: 1, sid: 990, ser: 1
Wait State:
  fixed_waits=0 flags=0x22 boundary=(nil)/-1
Session Wait History:
    elapsed time of 0.000061 sec since current wait
 0: waited for 'gc cr block 2-way'
    =0x5, =0x258, =0x47
    wait_id=255377 seq_num=59357 snap_id=1
    wait times: snap=0.000478 sec, exc=0.000478 sec, total=0.000478 sec
    wait times: max=infinite
    wait counts: calls=1 os=1
    occurred after 0.000122 sec of elapsed time
 1: waited for 'gc cr block 2-way'
    =0x5, =0x228, =0x45
    wait_id=255376 seq_num=59356 snap_id=1
    wait times: snap=0.000741 sec, exc=0.000741 sec, total=0.000741 sec
    wait times: max=infinite
    wait counts: calls=1 os=1
    occurred after 0.000120 sec of elapsed time
 2: waited for 'gc cr block 2-way'
    =0x5, =0x138, =0x43
    wait_id=255375 seq_num=59355 snap_id=1
    wait times: snap=0.000528 sec, exc=0.000528 sec, total=0.000528 sec
    wait times: max=infinite
    wait counts: calls=1 os=1
    occurred after 0.000111 sec of elapsed time
 3: waited for 'gc cr block 2-way'
    =0x5, =0xb8, =0x41
    wait_id=255374 seq_num=59354 snap_id=1
    wait times: snap=0.000583 sec, exc=0.000583 sec, total=0.000583 sec
    wait times: max=infinite
    wait counts: calls=1 os=1
    occurred after 0.000139 sec of elapsed time
 4: waited for 'gc cr block 2-way'
    =0x5, =0x110, =0x37
    wait_id=255373 seq_num=59353 snap_id=1
    wait times: snap=0.000541 sec, exc=0.000541 sec, total=0.000541 sec
    wait times: max=infinite
    wait counts: calls=1 os=1
    occurred after 0.000110 sec of elapsed time
 5: waited for 'gc cr block 2-way'
    =0x5, =0x100, =0x35
    wait_id=255372 seq_num=59352 snap_id=1
    wait times: snap=0.000629 sec, exc=0.000629 sec, total=0.000629 sec
    wait times: max=infinite
    wait counts: calls=1 os=1
    occurred after 0.000158 sec of elapsed time
 6: waited for 'gc cr block 2-way'
    =0x5, =0xf0, =0x33
    wait_id=255371 seq_num=59351 snap_id=1
    wait times: snap=0.000617 sec, exc=0.000617 sec, total=0.000617 sec
    wait times: max=infinite
    wait counts: calls=1 os=1
    occurred after 0.000128 sec of elapsed time
 7: waited for 'gc cr block 2-way'
    =0x5, =0xe0, =0x31
    wait_id=255370 seq_num=59350 snap_id=1
    wait times: snap=0.000561 sec, exc=0.000561 sec, total=0.000561 sec
    wait times: max=infinite
    wait counts: calls=1 os=1
    occurred after 0.000124 sec of elapsed time
 8: waited for 'gc cr block 2-way'
    =0x5, =0xd0, =0x2f
    wait_id=255369 seq_num=59349 snap_id=1
    wait times: snap=0.000565 sec, exc=0.000565 sec, total=0.000565 sec
    wait times: max=infinite
    wait counts: calls=1 os=1
    occurred after 0.000128 sec of elapsed time
 9: waited for 'gc cr block 2-way'
    =0x5, =0xc0, =0x2d
    wait_id=255368 seq_num=59348 snap_id=1
    wait times: snap=0.000555 sec, exc=0.000555 sec, total=0.000555 sec
    wait times: max=infinite
    wait counts: calls=1 os=1
    occurred after 0.000125 sec of elapsed time
Sampled Session History of session 1369 serial 1
---------------------------------------------------
The sampled session history is constructed by sampling
the target session every 1 second. The sampling process
captures at each sample if the session is in a non-idle wait,
an idle wait, or not in a wait. If the session is in a
non-idle wait then one interval is shown for all the samples
the session was in the same non-idle wait. If the
session is in an idle wait or not in a wait for
consecutive samples then one interval is shown for all
the consecutive samples. Though we display these consecutive
samples  in a single interval the session may NOT be continuously
idle or not in a wait (the sampling process does not know).
 
The history is displayed in reverse chronological order.
 
sample interval: 1 sec, max history 120 sec
---------------------------------------------------
  [121 samples,                                            07:04:03 - 07:06:03]
    waited for 'gc buffer busy acquire', seq_num: 59358
      p1: 'file#'=0x5
      p2: 'block#'=0x278
      p3: 'class#'=0x49
      time_waited: >= 120 sec (still in wait)
---------------------------------------------------
Sampled Session History Summary:
  longest_non_idle_wait: 'gc buffer busy acquire'
  [121 samples, 07:04:03 - 07:06:03]
      time_waited: >= 120 sec (still in wait)
---------------------------------------------------
----- END DDE Action: 'ORA_12751_DUMP' (SUCCESS, 1 csec) -----
----- END DDE Actions Dump (total 1 csec) -----
KEBM: MMON action policy violation. 'Block Cleanout Optim, Undo Segment Scan' viol=1; err=12751
minact-scn master-status: grec-scn:0x0e0e.5f0ec4ce gmin-scn:0x0e0e.5f0eac2e gcalc-scn:0x0e0e.5f0ead91

*** 2018-01-22 07:11:11.071
DDE rules only execution for: ORA 12751
----- START Event Driven Actions Dump ----
---- END Event Driven Actions Dump ----

ORA12751的错误原因是陈旧的SYS对象统计数据会导致生成次优执行计划,从而使AWR自动刷新从操作的语句运行更长时间和超时。

解决方法就是收集新的SYS对象统计信息,为优化器提供更好的统计信息,并生成更高效的执行计划

SQL> EXEC DBMS_STATS.GATHER_FIXED_OBJECTS_STATS;

PL/SQL procedure successfully completed.

SQL> EXEC DBMS_STATS.GATHER_SCHEMA_STATS ('SYS');

PL/SQL procedure successfully completed.

下面就是重启mmon和mmnl进程

SQL> alter system enable restricted session;

System altered.

SQL> alter system disable restricted session;

System altered.

查看alert日志可以看到mmon和mmnl进程已经重启了

Sat Aug 17 19:18:22 2019
Starting background process MMON
Sat Aug 17 19:18:22 2019
Starting background process MMNL
MMON started with pid=399, OS id=10373 
Sat Aug 17 19:18:22 2019
MMNL started with pid=405, OS id=10375 
ALTER SYSTEM enable restricted session;
Sat Aug 17 19:18:25 2019
Some DDE async actions failed or were cancelled
Sat Aug 17 19:18:25 2019
Sweep [inc][48021]: completed
Sweep [inc][48011]: completed
Sweep [inc][48002]: completed
Sweep [inc][35010]: completed
Sweep [inc][34706]: completed
Sweep [inc][34242]: completed
Sweep [inc][33546]: completed
Sweep [inc][33394]: completed
Sweep [inc2][48021]: completed
Sweep [inc2][48011]: completed
Sweep [inc2][48002]: completed
Sweep [inc2][35010]: completed
Sweep [inc2][34706]: completed
Sweep [inc2][34242]: completed
Sweep [inc2][33546]: completed
Sweep [inc2][33394]: completed
minact-scn: Inst 1 is a slave inc#:30 mmon proc-id:10373 status:0x2
minact-scn status: grec-scn:0x0e0e.61cb2e9c gmin-scn:0x0e0e.5f0eac2e gcalc-scn:0x0e0e.5f0ead91
Sat Aug 17 19:18:29 2019
db_recovery_file_dest_size of 10240 MB is 20.87% used. This is a
user-specified limit on the amount of space that will be used by this
database for recovery-related files, and does not reflect the amount of
space available in the underlying filesystem or ASM diskgroup.
Sat Aug 17 19:18:42 2019
ALTER SYSTEM disable restricted session;

再查看1号实例的mmon与mmnl进程状态

[root@db1 ~]# ps -ef | grep mmnl
oracle    10375      1  0 19:18 ?        00:00:00 ora_mmnl_RLZY1
root      10611 238963  0 19:18 pts/2    00:00:00 grep mmnl

[root@db1 ~]# ps -ef | grep mmon
oracle    10373      1  7 19:18 ?        00:00:02 ora_mmon_RLZY1
root      10630 238963  0 19:18 pts/2    00:00:00 grep mmon
过了两个小时去查看1号实例已经生成了两条快照信息
SQL> set long 200
SQL> set linesize 200
SQL> select * from ( select SNAP_ID,END_INTERVAL_TIME,instance_number from dba_hist_snapshot where instance_number=1 order by SNAP_ID desc) where rownum < =10;

   SNAP_ID END_INTERVAL_TIME                                                           INSTANCE_NUMBER
---------- --------------------------------------------------------------------------- ---------------
     24407 17-AUG-19 09.00.58.595 PM                                                                 1
     24406 17-AUG-19 08.00.40.244 PM                                                                 1

到此问题解决了。

当用户无限制使用表空间配额且表空间有足够空间时在执行DML操作时出现超出表空间的空间限额

朋友生产数据库在向特定的一张表插入数据时报超出表空间data的空间限额如是执行下查看用户所用的表空间配额信息,查看用户所使用表空间的配额发现没有限制,因为max_bytes为 -1

SQL>  select * from dba_ts_quotas where username='data';

TABLESPACE_NAME                USERNAME                            BYTES  MAX_BYTES     BLOCKS MAX_BLOCKS DROPPED
------------------------------ ------------------------------ ---------- ---------- ---------- ---------- -------
data                           data                           8825732464         -1  107735992         -1 NO

 1 rows selected

SQL>
SQL> select
  2    fs.tablespace_name            "Tablespace",
  3    (df.totalspace-fs.freespace)  "Used MB",
  4    fs.freespace                  "Free MB",
  5    df.totalspace                 "Total MB",
  6    round(100*(fs.freespace/df.totalspace)) "Pct. Free"
  7  from
  8    (select tablespace_name,round(sum(bytes)/1024/1024) TotalSpace
  9    from dba_data_files group by  tablespace_name) df,
 10    (select tablespace_name,round(sum(bytes)/1024/1024) FreeSpace
 11    from dba_free_space group by tablespace_name) fs
 12  where df.tablespace_name=fs.tablespace_name;

Tablespace                        Used MB    Free MB   Total MB  Pct. Free
------------------------------ ---------- ---------- ---------- ----------
SYSTEM                               7207       3033      10240         30
TEST                                 6790      34170      40960         83
USERS                                 173      25427      25600         99
UNDOTBS2                              227      24013      24240         99
DATA                               990119     176281    1166400         15
SYSAUX                               3925       1195       5120         23
UNDOTBS1                            12898      28062      40960         69

7 rows selected

查看表lv_data的依赖对象

SQL> select NAME,TYPE from dba_dependencies where REFERENCED_NAME='LV_DATA';

NAME                           TYPE
------------------------------ ------------------
LV_DATA                        VIEW
FC_UPDATE_CORPFUND             PROCEDURE
FC_UPDATE_MY                   PROCEDURE
FC_UPDATE_KY                   PROCEDURE
FC_UPDATE_FACTPAY              PROCEDURE
FC_UPDATE_CALCPAY              PROCEDURE
FC_UPDATE_KY                   PROCEDURE
......
LV_DATA                        SYNONYM
LV_DATA                        VIEW
LV_DATA                        SYNONYM
LV_DATA                        SYNONYM

139 rows selected

查看所有依赖对象的所有者

SQL> select  distinct owner from dba_objects where OBJECT_NAME in(select NAME from dba_dependencies where REFERENCED_NAME='LV_DATA');

OWNER
------------------------------
SY
SY_BK
WEB
CX
DATA
OLD
TEST
XC
CZ
OSY
BACKUP
TJ

12 rows selected

对所有依赖对象所有者授权可以无限制使用表空间

SQL> grant unlimited tablespace to OSY;

Grant succeeded

SQL> grant unlimited tablespace to SBK;

Grant succeeded

SQL> grant unlimited tablespace to WEB;

Grant succeeded

SQL> grant unlimited tablespace to CX;

Grant succeeded

SQL> grant unlimited tablespace to DATA;

Grant succeeded

SQL> grant unlimited tablespace to OLD;

Grant succeeded

SQL> grant unlimited tablespace to TEST;

Grant succeeded

SQL> grant unlimited tablespace to XC;

Grant succeeded

SQL> grant unlimited tablespace to CZ;

Grant succeeded

SQL> grant unlimited tablespace to SY;

Grant succeeded

SQL> grant unlimited tablespace to BACKUP;

Grant succeeded

SQL> grant unlimited tablespace to TJ;

Grant succeeded

再向表lv_data插入数据时恢复正常

Oracle alert.log出现OER 7451 in Load Indicator : Error Code = OSD-04500的问题处理

在对某数据库进行巡检时发现alert.log中出现了以下错误信息

OER 7451 in Load Indicator : Error Code = OSD-04500: 指定了非法选项
O/S-Error: (OS 1) 函数不正确。 !
OER 7451 in Load Indicator : Error Code = OSD-04500: 指定了非法选项
O/S-Error: (OS 1) 函数不正确。 !

我们可以查看错误信息说明

07451, 00000, "slskstat: unable to obtain load information."
// *Cause:  kstat library returned an error. Possible OS failure
// *Action: Check result code in sercose[0] for more information.

对于这个问题MOS上有一入篇文章OER 7451 in Load Indicator : Error Code = OSD-04500: illegal option specified [ID 1060806.1]说是操作系统版本与数据库版本不匹配造成的。

OER 7451 in Load Indicator : Error Code = OSD-04500: illegal option specified [ID 1060806.1]
Applies to:
Oracle Server - Enterprise Edition - Version: 10.2.0.1 to 11.2.0.1 - Release: 10.2 to 11.2
Microsoft Windows (32-bit)
Microsoft Windows x64 (64-bit)
***Checked for relevance on 01-Nov-2011***
Symptoms
The following error repeats over and overinthe database alert log immediately after startup.

OER 7451 inLoad Indicator : Error Code = OSD-04500: illegal option specified
O/S-Error: (OS 1) Incorrectfunction. !

Changes
This is a new installation.

Cause
Installed 32-bit Oracle database software on a 64-bit MS Windows OSwhichis not supported.
Here is excerpt from Note: 740926.1 on this subject.

Note: For the Database software, you can ONLYinstallthe x64 version on MS Windows (x64).
          You can NOTinstallthe 32-bit version Database software on MS Windows (x64).

Solution
Install 32-bit Oracle database software only on 32-bit MS Windows OS.

检查数据库版本,发现数据库是32位

C:\Users\Administrator>sqlplus /nolog

SQL*Plus: Release 10.2.0.4.0 - Production on 星期六 7月 21 18:04:28 2018

Copyright (c) 1982, 2007, Oracle.  All Rights Reserved.


SQL> conn sys/system@abc as sysdba
已连接。
SQL> select * from v$version;

BANNER
----------------------------------------------------------------
Oracle Database 10g Enterprise Edition Release 10.2.0.4.0 - Prod
PL/SQL Release 10.2.0.4.0 - Production
CORE    10.2.0.4.0      Production
TNS for 32-bit Windows: Version 10.2.0.4.0 - Production
NLSRTL Version 10.2.0.4.0 - Production

SQL> exit
从 Oracle Database 10g Enterprise Edition Release 10.2.0.4.0 - Production
With the Partitioning, OLAP, Data Mining and Real Application Testing options 断
开

检查操作系统,发现操作系统是64位
C:\Users\Administrator>systeminfo
主机名: WIN-ROUOJ6ERFO2
OS 名称: Microsoft Windows Server 2008 R2 Enterprise
OS 版本: 6.1.7601 Service Pack 1 Build 7601
OS 制造商: Microsoft Corporation
OS 配置: 独立服务器
OS 构件类型: Multiprocessor Free
注册的所有人: Windows 用户
注册的组织:
产品 ID: 00486-OEM-8400691-20006
初始安装日期: 2016/8/25, 11:37:01
系统启动时间: 2018/7/24, 9:32:07
系统制造商: VMware, Inc.
系统型号: VMware Virtual Platform
系统类型: x64-based PC
处理器: 安装了 1 个处理器。
[01]: Intel64 Family 6 Model 26 Stepping 5 GenuineIntel ~2128
Mhz
BIOS 版本: Phoenix Technologies LTD 6.00, 2014/4/14
Windows 目录: C:\Windows
系统目录: C:\Windows\system32

果然是在64位的系统中安装了32位的Oracle软件,数据库版本与操作系统版需要严格遵守安装官方的support list

Oracle 12C ORA-12545 While Connecting to RAC through SCAN Name

操作环境为Oracle Linux 7.1,Oracle 12.2 RAC数据库,SCAN IP是使用GNS方式创建,在使用SCAN IP登录数据库时出现如下错误

SQL> conn sys/abcd@cs as sysdba
ERROR:
ORA-12545: Connect failed because target host or object does not exist

错误信息直译是目标主机或对象不存在。

查看scan的配置信息状态正常

[grid@cs1 ~]$ srvctl config scan
SCAN name: cs-cluster-scan.cs-cluster.jy.net, Network: 1
Subnet IPv4: 10.10.10.0/255.255.255.0/ens160, dhcp
Subnet IPv6: 
SCAN 1 IPv4 VIP: -/scan1-vip/10.10.10.143
SCAN VIP is enabled.
SCAN VIP is individually enabled on nodes: 
SCAN VIP is individually disabled on nodes: 
SCAN 2 IPv4 VIP: -/scan2-vip/10.10.10.141
SCAN VIP is enabled.
SCAN VIP is individually enabled on nodes: 
SCAN VIP is individually disabled on nodes: 
SCAN 3 IPv4 VIP: -/scan3-vip/10.10.10.142
SCAN VIP is enabled.
SCAN VIP is individually enabled on nodes: 
SCAN VIP is individually disabled on nodes: 

查看scan的监听信息状态正常

[grid@cs1 ~]$ srvctl config scan_listener
SCAN Listener LISTENER_SCAN1 exists. Port: TCP:1521
Registration invited nodes: 
Registration invited subnets: 
SCAN Listener is enabled.
SCAN Listener is individually enabled on nodes: 
SCAN Listener is individually disabled on nodes: 
SCAN Listener LISTENER_SCAN2 exists. Port: TCP:1521
Registration invited nodes: 
Registration invited subnets: 
SCAN Listener is enabled.
SCAN Listener is individually enabled on nodes: 
SCAN Listener is individually disabled on nodes: 
SCAN Listener LISTENER_SCAN3 exists. Port: TCP:1521
Registration invited nodes: 
Registration invited subnets: 
SCAN Listener is enabled.
SCAN Listener is individually enabled on nodes: 
SCAN Listener is individually disabled on nodes: 

[grid@cs2 ~]$ lsnrctl status LISTENER_SCAN1

LSNRCTL for Linux: Version 12.2.0.1.0 - Production on 13-MAR-2018 11:05:13

Copyright (c) 1991, 2016, Oracle.  All rights reserved.

Connecting to (DESCRIPTION=(ADDRESS=(PROTOCOL=IPC)(KEY=LISTENER_SCAN1)))
STATUS of the LISTENER
------------------------
Alias                     LISTENER_SCAN1
Version                   TNSLSNR for Linux: Version 12.2.0.1.0 - Production
Start Date                12-MAR-2018 19:10:06
Uptime                    0 days 15 hr. 55 min. 7 sec
Trace Level               off
Security                  ON: Local OS Authentication
SNMP                      OFF
Listener Parameter File   /u01/app/product/12.2.0/crs/network/admin/listener.ora
Listener Log File         /u01/app/grid/diag/tnslsnr/cs2/listener_scan1/alert/log.xml
Listening Endpoints Summary...
  (DESCRIPTION=(ADDRESS=(PROTOCOL=ipc)(KEY=LISTENER_SCAN1)))
  (DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=10.10.10.143)(PORT=1521)))
Services Summary...
Service "-MGMTDBXDB" has 1 instance(s).
  Instance "-MGMTDB", status READY, has 1 handler(s) for this service...
Service "66fa07fcd41a56f0e053be828a0abc70" has 1 instance(s).
  Instance "-MGMTDB", status READY, has 1 handler(s) for this service...
Service "_mgmtdb" has 1 instance(s).
  Instance "-MGMTDB", status READY, has 1 handler(s) for this service...
Service "cs" has 2 instance(s).
  Instance "cs1", status READY, has 1 handler(s) for this service...
  Instance "cs2", status READY, has 1 handler(s) for this service...
Service "csXDB" has 2 instance(s).
  Instance "cs1", status READY, has 1 handler(s) for this service...
  Instance "cs2", status READY, has 1 handler(s) for this service...
Service "gimr_dscrep_10" has 1 instance(s).
  Instance "-MGMTDB", status READY, has 1 handler(s) for this service...
The command completed successfully

[grid@cs1 ~]$ lsnrctl status LISTENER_SCAN2

LSNRCTL for Linux: Version 12.2.0.1.0 - Production on 13-MAR-2018 11:05:29

Copyright (c) 1991, 2016, Oracle.  All rights reserved.

Connecting to (DESCRIPTION=(ADDRESS=(PROTOCOL=IPC)(KEY=LISTENER_SCAN2)))
STATUS of the LISTENER
------------------------
Alias                     LISTENER_SCAN2
Version                   TNSLSNR for Linux: Version 12.2.0.1.0 - Production
Start Date                12-MAR-2018 19:08:54
Uptime                    0 days 15 hr. 56 min. 35 sec
Trace Level               off
Security                  ON: Local OS Authentication
SNMP                      OFF
Listener Parameter File   /u01/app/product/12.2.0/crs/network/admin/listener.ora
Listener Log File         /u01/app/grid/diag/tnslsnr/cs1/listener_scan2/alert/log.xml
Listening Endpoints Summary...
  (DESCRIPTION=(ADDRESS=(PROTOCOL=ipc)(KEY=LISTENER_SCAN2)))
  (DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=10.10.10.141)(PORT=1521)))
Services Summary...
Service "-MGMTDBXDB" has 1 instance(s).
  Instance "-MGMTDB", status READY, has 1 handler(s) for this service...
Service "66fa07fcd41a56f0e053be828a0abc70" has 1 instance(s).
  Instance "-MGMTDB", status READY, has 1 handler(s) for this service...
Service "_mgmtdb" has 1 instance(s).
  Instance "-MGMTDB", status READY, has 1 handler(s) for this service...
Service "cs" has 2 instance(s).
  Instance "cs1", status READY, has 1 handler(s) for this service...
  Instance "cs2", status READY, has 1 handler(s) for this service...
Service "csXDB" has 2 instance(s).
  Instance "cs1", status READY, has 1 handler(s) for this service...
  Instance "cs2", status READY, has 1 handler(s) for this service...
Service "gimr_dscrep_10" has 1 instance(s).
  Instance "-MGMTDB", status READY, has 1 handler(s) for this service...
The command completed successfully
[grid@cs1 ~]$ lsnrctl status LISTENER_SCAN3

LSNRCTL for Linux: Version 12.2.0.1.0 - Production on 13-MAR-2018 11:05:32

Copyright (c) 1991, 2016, Oracle.  All rights reserved.

Connecting to (DESCRIPTION=(ADDRESS=(PROTOCOL=IPC)(KEY=LISTENER_SCAN3)))
STATUS of the LISTENER
------------------------
Alias                     LISTENER_SCAN3
Version                   TNSLSNR for Linux: Version 12.2.0.1.0 - Production
Start Date                12-MAR-2018 19:08:52
Uptime                    0 days 15 hr. 56 min. 40 sec
Trace Level               off
Security                  ON: Local OS Authentication
SNMP                      OFF
Listener Parameter File   /u01/app/product/12.2.0/crs/network/admin/listener.ora
Listener Log File         /u01/app/grid/diag/tnslsnr/cs1/listener_scan3/alert/log.xml
Listening Endpoints Summary...
  (DESCRIPTION=(ADDRESS=(PROTOCOL=ipc)(KEY=LISTENER_SCAN3)))
  (DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=10.10.10.142)(PORT=1521)))
Services Summary...
Service "-MGMTDBXDB" has 1 instance(s).
  Instance "-MGMTDB", status READY, has 1 handler(s) for this service...
Service "66fa07fcd41a56f0e053be828a0abc70" has 1 instance(s).
  Instance "-MGMTDB", status READY, has 1 handler(s) for this service...
Service "_mgmtdb" has 1 instance(s).
  Instance "-MGMTDB", status READY, has 1 handler(s) for this service...
Service "cs" has 2 instance(s).
  Instance "cs1", status READY, has 1 handler(s) for this service...
  Instance "cs2", status READY, has 1 handler(s) for this service...
Service "csXDB" has 2 instance(s).
  Instance "cs1", status READY, has 1 handler(s) for this service...
  Instance "cs2", status READY, has 1 handler(s) for this service...
Service "gimr_dscrep_10" has 1 instance(s).
  Instance "-MGMTDB", status READY, has 1 handler(s) for this service...
The command completed successfully

在通过SCAN Name来连接RAC数据库时,客户端可以解析所有有完整域名的SCAN Name与VIP Name,但是不能解析没有域名的SCAN Name与VIP Name。通过以下操作可以解决这个问题。

1.在数据库级别使用有完整域名的VIP Name或VIP来设置pfile或spfile文件中的local_listener参数

alter system set local_listener='(DESCRIPTION=(ADDRESS_LIST=(ADDRESS=(PROTOCOL=TCP)(HOST=10.10.10.140)(PORT=1521))))' scope=both sid='cs1';

or
alter system set local_listener='(DESCRIPTION=(ADDRESS_LIST=(ADDRESS=(PROTOCOL=TCP)(HOST=cs1-vip.jy.net)(PORT=1521))))' scope=both sid='cs1';

and
alter system register;

在RAC的每个节点都执行类似上面的操作,但我的环境中local_listener的设置是正确的

SQL> show parameter local_listener

NAME                                 TYPE        VALUE
------------------------------------ ----------- ------------------------------
local_listener                       string      (DESCRIPTION=(ADDRESS_LIST=(AD
                                                 DRESS=(PROTOCOL=TCP)(HOST=10.1
                                                 0.10.140)(PORT=1521))))

2.在客户端的hosts文件中增加tnsnames.ora文件中带完整域名的SCAN Name

[root@cs1 ~]# vi /etc/hosts
....
10.10.10.141 cs-cluster-scan.cs-cluster.jy.net
10.10.10.142 cs-cluster-scan.cs-cluster.jy.net
10.10.10.143 cs-cluster-scan.cs-cluster.jy.net

3.再次登录

[oracle@cs11 ~]$ tnsping cs

TNS Ping Utility for Linux: Version 12.2.0.1.0 - Production on 13-MAR-2018 10:40:15

Copyright (c) 1997, 2016, Oracle.  All rights reserved.

Used parameter files:


Used TNSNAMES adapter to resolve the alias
Attempting to contact (DESCRIPTION = (ADDRESS = (PROTOCOL = TCP)(HOST = cs-cluster-scan.cs-cluster.jy.net)(PORT = 1521)) (CONNECT_DATA = (SERVER = DEDICATED) (SERVICE_NAME = cs)))
OK (0 msec)


[oracle@cs1 admin]$ sqlplus /nolog

SQL*Plus: Release 12.2.0.1.0 Production on Tue Mar 13 13:12:32 2018

Copyright (c) 1982, 2016, Oracle.  All rights reserved.

SQL> conn sys/abcd@cs as sysdba
Connected.

到此问题解决了,可以使用SCAN Name来登录数据库了。

Linux中两块device的minor number相同而造成RAC不能启动的问题

Oracle Linux 7.1在安装Oracle 12.2 Grid软件在执行root.sh脚本时,当第一个节点执行成功后,在第二个节点执行出现如下错误

[root@cs2 /]# ./u01/app/product/12.2.0/crs/root.sh
Performing root user operation.

The following environment variables are set as:
    ORACLE_OWNER= grid
    ORACLE_HOME=  /u01/app/product/12.2.0/crs

Enter the full pathname of the local bin directory: [/usr/local/bin]: 
   Copying dbhome to /usr/local/bin ...
   Copying oraenv to /usr/local/bin ...
   Copying coraenv to /usr/local/bin ...


Creating /etc/oratab file...
Entries will be added to the /etc/oratab file as needed by
Database Configuration Assistant when a database is created
Finished running generic part of root script.
Now product-specific root actions will be performed.
Relinking oracle with rac_on option
Using configuration parameter file: /u01/app/product/12.2.0/crs/crs/install/crsconfig_params
The log of current session can be found at:
  /u01/app/grid/crsdata/cs2/crsconfig/rootcrs_cs2_2018-03-09_04-33-22PM.log
2018/03/09 16:34:23 CLSRSC-594: Executing installation step 1 of 19: 'SetupTFA'.
2018/03/09 16:34:23 CLSRSC-4001: Installing Oracle Trace File Analyzer (TFA) Collector.
2018/03/09 16:34:54 CLSRSC-4002: Successfully installed Oracle Trace File Analyzer (TFA) Collector.
2018/03/09 16:34:54 CLSRSC-594: Executing installation step 2 of 19: 'ValidateEnv'.
2018/03/09 16:35:03 CLSRSC-363: User ignored prerequisites during installation
2018/03/09 16:35:03 CLSRSC-594: Executing installation step 3 of 19: 'CheckFirstNode'.
2018/03/09 16:35:05 CLSRSC-594: Executing installation step 4 of 19: 'GenSiteGUIDs'.
2018/03/09 16:35:06 CLSRSC-594: Executing installation step 5 of 19: 'SaveParamFile'.
2018/03/09 16:35:10 CLSRSC-594: Executing installation step 6 of 19: 'SetupOSD'.
2018/03/09 16:35:12 CLSRSC-594: Executing installation step 7 of 19: 'CheckCRSConfig'.
2018/03/09 16:35:13 CLSRSC-594: Executing installation step 8 of 19: 'SetupLocalGPNP'.
2018/03/09 16:35:16 CLSRSC-594: Executing installation step 9 of 19: 'ConfigOLR'.
2018/03/09 16:35:24 CLSRSC-594: Executing installation step 10 of 19: 'ConfigCHMOS'.
2018/03/09 16:35:24 CLSRSC-594: Executing installation step 11 of 19: 'CreateOHASD'.
2018/03/09 16:35:26 CLSRSC-594: Executing installation step 12 of 19: 'ConfigOHASD'.
2018/03/09 16:35:42 CLSRSC-330: Adding Clusterware entries to file 'oracle-ohasd.service'
2018/03/09 16:36:42 CLSRSC-594: Executing installation step 13 of 19: 'InstallAFD'.
2018/03/09 16:38:08 CLSRSC-594: Executing installation step 14 of 19: 'InstallACFS'.
CRS-2791: Starting shutdown of Oracle High Availability Services-managed resources on 'cs2'
CRS-2793: Shutdown of Oracle High Availability Services-managed resources on 'cs2' has completed
CRS-4133: Oracle High Availability Services has been stopped.
CRS-4123: Oracle High Availability Services has been started.
2018/03/09 16:38:37 CLSRSC-594: Executing installation step 15 of 19: 'InstallKA'.
2018/03/09 16:38:39 CLSRSC-594: Executing installation step 16 of 19: 'InitConfig'.
CRS-2791: Starting shutdown of Oracle High Availability Services-managed resources on 'cs2'
CRS-2793: Shutdown of Oracle High Availability Services-managed resources on 'cs2' has completed
CRS-4133: Oracle High Availability Services has been stopped.
CRS-4123: Oracle High Availability Services has been started.
CRS-2791: Starting shutdown of Oracle High Availability Services-managed resources on 'cs2'
CRS-2673: Attempting to stop 'ora.drivers.acfs' on 'cs2'
CRS-2677: Stop of 'ora.drivers.acfs' on 'cs2' succeeded
CRS-2793: Shutdown of Oracle High Availability Services-managed resources on 'cs2' has completed
CRS-4133: Oracle High Availability Services has been stopped.
2018/03/09 16:39:07 CLSRSC-594: Executing installation step 17 of 19: 'StartCluster'.
CRS-4123: Starting Oracle High Availability Services-managed resources
CRS-2672: Attempting to start 'ora.mdnsd' on 'cs2'
CRS-2672: Attempting to start 'ora.evmd' on 'cs2'
CRS-2676: Start of 'ora.mdnsd' on 'cs2' succeeded
CRS-2676: Start of 'ora.evmd' on 'cs2' succeeded
CRS-2672: Attempting to start 'ora.gpnpd' on 'cs2'
CRS-2676: Start of 'ora.gpnpd' on 'cs2' succeeded
CRS-2672: Attempting to start 'ora.gipcd' on 'cs2'
CRS-2676: Start of 'ora.gipcd' on 'cs2' succeeded
CRS-2672: Attempting to start 'ora.cssdmonitor' on 'cs2'
CRS-2676: Start of 'ora.cssdmonitor' on 'cs2' succeeded
CRS-2672: Attempting to start 'ora.cssd' on 'cs2'
CRS-2672: Attempting to start 'ora.diskmon' on 'cs2'
CRS-2676: Start of 'ora.diskmon' on 'cs2' succeeded
CRS-2883: Resource 'ora.cssdmonitor' failed during Clusterware stack start.
CRS-4406: Oracle High Availability Services synchronous start failed.
CRS-4000: Command Start failed, or completed with errors.
2018/03/09 16:39:46 CLSRSC-378: Failed to get the configured node role for the local node
CRS-4535: Cannot communicate with Cluster Ready Services
CRS-4000: Command Status failed, or completed with errors.
2018/03/09 16:39:48 CLSRSC-117: Failed to start Oracle Clusterware stack
Died at /u01/app/product/12.2.0/crs/crs/install/crsinstall.pm line 1494.
The command '/u01/app/product/12.2.0/crs/perl/bin/perl -I/u01/app/product/12.2.0/crs/perl/lib -I/u01/app/product/12.2.0/crs/crs/install /u01/app/product/12.2.0/crs/crs/install/rootcrs.pl ' 

execution failed

出错了,如是再次执行

[root@cs2 /]# ./u01/app/product/12.2.0/crs/root.sh
Performing root user operation.

The following environment variables are set as:
    ORACLE_OWNER= grid
    ORACLE_HOME=  /u01/app/product/12.2.0/crs

Enter the full pathname of the local bin directory: [/usr/local/bin]: 
The contents of "dbhome" have not changed. No need to overwrite.
The contents of "oraenv" have not changed. No need to overwrite.
The contents of "coraenv" have not changed. No need to overwrite.

Entries will be added to the /etc/oratab file as needed by
Database Configuration Assistant when a database is created
Finished running generic part of root script.
Now product-specific root actions will be performed.
Relinking oracle with rac_on option
Using configuration parameter file: /u01/app/product/12.2.0/crs/crs/install/crsconfig_params
The log of current session can be found at:
  /u01/app/grid/crsdata/cs2/crsconfig/rootcrs_cs2_2018-03-09_04-51-03PM.log
2018/03/09 16:51:07 CLSRSC-594: Executing installation step 1 of 19: 'SetupTFA'.
2018/03/09 16:51:07 CLSRSC-4001: Installing Oracle Trace File Analyzer (TFA) Collector.
2018/03/09 16:51:07 CLSRSC-4002: Successfully installed Oracle Trace File Analyzer (TFA) Collector.
2018/03/09 16:51:08 CLSRSC-594: Executing installation step 2 of 19: 'ValidateEnv'.
2018/03/09 16:51:11 CLSRSC-363: User ignored prerequisites during installation
2018/03/09 16:51:11 CLSRSC-594: Executing installation step 3 of 19: 'CheckFirstNode'.
2018/03/09 16:51:12 CLSRSC-594: Executing installation step 4 of 19: 'GenSiteGUIDs'.
2018/03/09 16:51:12 CLSRSC-594: Executing installation step 5 of 19: 'SaveParamFile'.
2018/03/09 16:51:15 CLSRSC-594: Executing installation step 6 of 19: 'SetupOSD'.
2018/03/09 16:51:17 CLSRSC-594: Executing installation step 7 of 19: 'CheckCRSConfig'.
2018/03/09 16:51:18 CLSRSC-594: Executing installation step 8 of 19: 'SetupLocalGPNP'.
2018/03/09 16:51:21 CLSRSC-594: Executing installation step 9 of 19: 'ConfigOLR'.
2018/03/09 16:51:22 CLSRSC-594: Executing installation step 10 of 19: 'ConfigCHMOS'.
2018/03/09 16:52:09 CLSRSC-594: Executing installation step 11 of 19: 'CreateOHASD'.
2018/03/09 16:52:10 CLSRSC-594: Executing installation step 12 of 19: 'ConfigOHASD'.
2018/03/09 16:52:26 CLSRSC-330: Adding Clusterware entries to file 'oracle-ohasd.service'
2018/03/09 16:52:48 CLSRSC-594: Executing installation step 13 of 19: 'InstallAFD'.
2018/03/09 16:52:51 CLSRSC-594: Executing installation step 14 of 19: 'InstallACFS'.
2018/03/09 16:52:53 CLSRSC-594: Executing installation step 15 of 19: 'InstallKA'.
2018/03/09 16:52:55 CLSRSC-594: Executing installation step 16 of 19: 'InitConfig'.
CRS-2791: Starting shutdown of Oracle High Availability Services-managed resources on 'cs2'
CRS-2673: Attempting to stop 'ora.drivers.acfs' on 'cs2'
CRS-2673: Attempting to stop 'ora.mdnsd' on 'cs2'
CRS-2673: Attempting to stop 'ora.evmd' on 'cs2'
CRS-2673: Attempting to stop 'ora.gpnpd' on 'cs2'
CRS-2677: Stop of 'ora.drivers.acfs' on 'cs2' succeeded
CRS-2677: Stop of 'ora.mdnsd' on 'cs2' succeeded
CRS-2677: Stop of 'ora.gpnpd' on 'cs2' succeeded
CRS-2677: Stop of 'ora.evmd' on 'cs2' succeeded
CRS-2673: Attempting to stop 'ora.driver.afd' on 'cs2'
CRS-2673: Attempting to stop 'ora.gipcd' on 'cs2'
CRS-2677: Stop of 'ora.driver.afd' on 'cs2' succeeded
CRS-2677: Stop of 'ora.gipcd' on 'cs2' succeeded
CRS-2793: Shutdown of Oracle High Availability Services-managed resources on 'cs2' has completed
CRS-4133: Oracle High Availability Services has been stopped.
2018/03/09 16:53:14 CLSRSC-594: Executing installation step 17 of 19: 'StartCluster'.
CRS-4123: Starting Oracle High Availability Services-managed resources
CRS-2672: Attempting to start 'ora.mdnsd' on 'cs2'
CRS-2672: Attempting to start 'ora.evmd' on 'cs2'
CRS-2676: Start of 'ora.mdnsd' on 'cs2' succeeded
CRS-2676: Start of 'ora.evmd' on 'cs2' succeeded
CRS-2672: Attempting to start 'ora.gpnpd' on 'cs2'
CRS-2676: Start of 'ora.gpnpd' on 'cs2' succeeded
CRS-2672: Attempting to start 'ora.gipcd' on 'cs2'
CRS-2676: Start of 'ora.gipcd' on 'cs2' succeeded
CRS-2672: Attempting to start 'ora.cssdmonitor' on 'cs2'
CRS-2676: Start of 'ora.cssdmonitor' on 'cs2' succeeded
CRS-2672: Attempting to start 'ora.cssd' on 'cs2'
CRS-2672: Attempting to start 'ora.diskmon' on 'cs2'
CRS-2676: Start of 'ora.diskmon' on 'cs2' succeeded
CRS-1705: Found 0 configured voting files but 1 voting files are required, terminating to ensure data integrity; details at (:CSSNM00021:) in /u01/app/grid/diag/crs/cs2/crs/trace/ocssd.trc
CRS-2883: Resource 'ora.cssdmonitor' failed during Clusterware stack start.
CRS-4406: Oracle High Availability Services synchronous start failed.
CRS-4000: Command Start failed, or completed with errors.
2018/03/09 16:53:53 CLSRSC-378: Failed to get the configured node role for the local node
CRS-4535: Cannot communicate with Cluster Ready Services
CRS-4000: Command Status failed, or completed with errors.
2018/03/09 16:53:55 CLSRSC-117: Failed to start Oracle Clusterware stack
Died at /u01/app/product/12.2.0/crs/crs/install/crsinstall.pm line 1494.
The command '/u01/app/product/12.2.0/crs/perl/bin/perl -I/u01/app/product/12.2.0/crs/perl/lib -I/u01/app/product/12.2.0/crs/crs/install /u01/app/product/12.2.0/crs/crs/install/rootcrs.pl ' 

execution failed

还是出错了,但错误信息与第一次执行有所不同了,有以下内容

CRS-1705: Found 0 configured voting files but 1 voting files are required, terminating to ensure data integrity; details at (:CSSNM00021:) in /u01/app/grid/diag/crs/cs2/crs/trace/ocssd.trc

如果查看跟踪文件/u01/app/grid/diag/crs/cs2/crs/trace/ocssd.trc

[root@cs2 ~]# more /u01/app/grid/diag/crs/cs2/crs/trace/ocssd.trc
Trace file /u01/app/grid/diag/crs/cs2/crs/trace/ocssd.trc
Oracle Database 12c Clusterware Release 12.2.0.1.0 - Production Copyright 1996, 2016 Oracle. All rights reserved.
 default:2950017088: 1: clskec:has:CLSU:910 4 args[CLSD00302][mod=clsdadr.c][loc=(:CLSD00302:)][msg=clsdAdrInit: Trace file size and number of segments fetched from environemnt variable: 

ORA_DAEMON_TRACE_FILE_OPTIONS filesize=52428800,n
umsegments=10]

    CLSB:2950017088: Argument count (argc) for this daemon is 3
    CLSB:2950017088: Argument 0 is: /u01/app/product/12.2.0/crs/bin/ocssd.bin
    CLSB:2950017088: Argument 1 is: 
    CLSB:2950017088: Argument 2 is: OSD2PORTIF
2018-03-09 16:39:26.799 :    CSSD:2950017088: (TLM) Starting CSS daemon, version 12.2.0.1.0 with uniqueness value 1520584766
2018-03-09 16:39:26.799 :    CSSD:2950017088: clsu_load_ENV_levels: Module = CSSD, LogLevel = 2
2018-03-09 16:39:26.799 :    CSSD:2950017088: clsu_load_ENV_levels: Module = CSSDNMC, LogLevel = 2
2018-03-09 16:39:26.799 :    CSSD:2950017088: clsu_load_ENV_levels: Module = CSSDGMPC, LogLevel = 2
2018-03-09 16:39:26.799 :    CSSD:2950017088: clsu_load_ENV_levels: Module = CSSDGMCC, LogLevel = 2
2018-03-09 16:39:26.799 :    CSSD:2950017088: clsu_load_ENV_levels: Module = CSSDBCMC, LogLevel = 2
2018-03-09 16:39:26.799 :    CSSD:2950017088: clsu_load_ENV_levels: Module = CSSDGNS, LogLevel = 2
2018-03-09 16:39:26.799 :    CSSD:2950017088: clsu_load_ENV_levels: Module = CLSF, LogLevel = 0
2018-03-09 16:39:26.799 :    CSSD:2950017088: clsu_load_ENV_levels: Module = CLSFA, LogLevel = 3
2018-03-09 16:39:26.799 :    CSSD:2950017088: clsu_load_ENV_levels: Module = SKGFD, LogLevel = 0
2018-03-09 16:39:26.799 :    CSSD:2950017088: clsu_load_ENV_levels: Module = GPNP, LogLevel = 1
2018-03-09 16:39:26.799 :    CSSD:2950017088: clsu_load_ENV_levels: Module = CLSINET, LogLevel = 1
2018-03-09 16:39:26.799 :    CSSD:2950017088: clsu_load_ENV_levels: Module = CLSNS, LogLevel = 3
2018-03-09 16:39:26.799 :    CSSD:2950017088: clsu_load_ENV_levels: Module = OCRAPI, LogLevel = 1
2018-03-09 16:39:26.799 :    CSSD:2950017088: clsu_load_ENV_levels: Module = OCRCLI, LogLevel = 1
2018-03-09 16:39:26.799 :    CSSD:2950017088: clsu_load_ENV_levels: Module = CLSCEVT, LogLevel = 0
2018-03-09 16:39:26.803 :    CSSD:2950017088: clssscInitGlobalCTX: not in a container
    CSSD:2950017088: clsugetconf : Configuration type [4]. 
2018-03-09 16:39:26.809 :    CSSD:2950017088: clssscGetParameterOLR: OLR fetch for parameter auth rep (9) failed with rc 21
2018-03-09 16:39:26.809 :    CSSD:2950017088: clssscagStartAgLsnr: Failed to get auth location from OLR, constructing manually 
2018-03-09 16:39:26.809 :    CSSD:2950017088: clssscagStartAgLsnr: auth location '/u01/app/product/12.2.0/crs/auth/css/' 
2018-03-09 16:39:26.809 :    CSSD:2950017088: clssscGPNPInit: PERF_TIME Initializing GPNP
2018-03-09 16:39:26.810 :    GPNP:2950017088: clsgpnp_Init: HMAC hash type in context is SHA-512
2018-03-09 16:39:26.810 :    GPNP:2950017088: clsgpnp_Init: [at clsgpnp0.c:684] '/u01/app/product/12.2.0/crs' in effect as GPnP home base.
2018-03-09 16:39:26.810 :    GPNP:2950017088: clsgpnp_Init: [at clsgpnp0.c:750] GPnP pid=29348, cli= GPNP comp tracelevel=1, depcomp tracelevel=0, tlsrc:init, apitl:0, tstenv:0, devenv:0, 

envopt:0, flags=3
2018-03-09 16:39:26.815 :    GPNP:2950017088: clsgpnpkwf_initwfloc: [at clsgpnpkwf.c:403] Using FS Wallet Location : /u01/app/product/12.2.0/crs/gpnp/cs2/wallets/peer/

2018-03-09 16:39:26.815 :    GPNP:2950017088: clsgpnpkwf_initwfloc: [at clsgpnpkwf.c:415] Wallet readable. Path: /u01/app/product/12.2.0/crs/gpnp/cs2/wallets/peer/

2018-03-09 16:39:26.817 :    CSSD:2782349056: clssscthrdmain: Starting thread clssscAgListener
2018-03-09 16:39:26.817 :    CSSD:2782349056: clsssclsnrsetup: endp 0x6b for ipc://agent_ag_cs2_
2018-03-09 16:39:26.817 :    CSSD:2782349056: clssscagOpenAgentEndp: listening on ipc://agent_ag_cs2_ (0x6b)
2018-03-09 16:39:26.818 :    CSSD:2782349056: clsssclsnrsetup: endp 0x7c for ipc://monitor_ag_cs2_
2018-03-09 16:39:26.818 :    CSSD:2782349056: clssscagOpenAgentEndp: listening on ipc://monitor_ag_cs2_ (0x7c)
2018-03-09 16:39:26.818 :    CSSD:2782349056: clssscUpdateInitState: Set state to 0x00080000, based on prior state of 0x00000000 and requested change of 0x00080000
2018-03-09 16:39:26.839 :    GPNP:2950017088: clsgpnp_profileCallUrlInt: [at clsgpnp.c:2238] get-profile call to url "ipc://GPNPD_cs2" disco "" [f=0 claimed- host: cname: cguid: cli:gpnp 

p:29348 role: seq: ep: auth: diag:[]] 
2018-03-09 16:39:26.839 : default:2950017088: clsvactversion:4: Retrieving Active Version from local storage.
2018-03-09 16:39:26.842 : default:2950017088: clsvactversion:4: Retrieving Active Version from local storage.
2018-03-09 16:39:26.851 :    GPNP:2950017088: clsgpnp_profileCallUrlInt: [at clsgpnp.c:2381] Result: (0) CLSGPNP_OK. Successful get-profile CALL to remote "ipc://GPNPD_cs2" disco ""
2018-03-09 16:39:26.851 :    CSSD:2950017088: clssscGPNPInit: Successfully initialized GPNP
2018-03-09 16:39:26.851 :    CSSD:2950017088: clssscInitGlobalCTX: PERF_TIME starting CLSFA for Flex
2018-03-09 16:39:26.851 :    CSSD:2950017088: clssscInitGlobalCTX: PERF_TIME started CLSFA for Flex
2018-03-09 16:39:26.851 :    CSSD:2950017088: Starting CSS daemon in big-clustered mode with a role of hub
2018-03-09 16:39:26.851 :    CSSD:2950017088: clssscInitGlobalCTX: Environment is production
2018-03-09 16:39:26.851 :    CSSD:2950017088: clssscInitGlobalCTX: Core file size limit extended
2018-03-09 16:39:26.852 :    CSSD:2950017088: clssscInitGlobalCTX: GIPCHA down 0
2018-03-09 16:39:26.855 :    CSSD:2950017088: clssscqueue_init: queue(0xe708a0), max(0)
2018-03-09 16:39:26.855 :    CSSD:2950017088: clssscUpdateInitState: Set state to 0x00080002, based on prior state of 0x00080000 and requested change of 0x00000002
2018-03-09 16:39:26.855 :    CSSD:2950017088: osdctx->majik=OSD2PORTIF 
2018-03-09 16:39:26.855 :    CSSD:2950017088: ipmi_ipv4 = ()
2018-03-09 16:39:26.855 :    CSSD:2950017088: clssscModifyWalletIP: Running as user grid
2018-03-09 16:39:26.855 :    CSSD:2950017088: clssscModifyEnvironment: unable to update wallet for IPMI
2018-03-09 16:39:26.857 :    CSSD:2950017088: clssscWaitOnInitState: Waiting on requested state 0x00001000, current state 0x00080002, timeout 4294967295
2018-03-09 16:39:26.858 :  CLSDMT:2779895552: PID for the Process [29348], connkey CSSD
2018-03-09 16:39:26.858 :  CLSDMT:2779895552: ERROR: Empty pid name for proc CSSD
2018-03-09 16:39:26.859 :    CSSD:2778318592: clssscthrdmain: Starting thread Alarm
2018-03-09 16:39:26.859 :    CSSD:2778318592: clssscAlarmThread: Thread Spawned
2018-03-09 16:39:26.859 :    CSSD:2778318592: clssscUpdateInitState: Set state to 0x00081002, based on prior state of 0x00080002 and requested change of 0x00001000
2018-03-09 16:39:26.859 :    CSSD:2950017088: clssscWaitOnInitState: returning 1, requested state 0x00001000, current state 0x00081002
2018-03-09 16:39:26.859 :    CSSD:2950017088: clssscqueue_init: queue(0xf1c670), max(0)
2018-03-09 16:39:26.860 :    CSSD:2950017088: clssscStartActivityThread: Waiting for INIT
2018-03-09 16:39:26.860 :    CSSD:2950017088: clssscWaitOnInitState: Waiting on requested state 0x00800000, current state 0x00081002, timeout 4294967295
2018-03-09 16:39:26.860 :    CSSD:2776741632: clssscthrdmain: Starting thread clssscActivityThread
2018-03-09 16:39:26.860 :    CSSD:2776741632: clssscUpdateInitState: Set state to 0x00881002, based on prior state of 0x00081002 and requested change of 0x00800000
2018-03-09 16:39:26.860 :    CSSD:2950017088: clssscWaitOnInitState: returning 1, requested state 0x00800000, current state 0x00881002
2018-03-09 16:39:26.860 :    CSSD:2950017088: clssscGetParameterProfile: profile fetch failed for parameter ocrid (4) with return code CLSGPNP_NOT_FOUND(5)
2018-03-09 16:39:26.860 :    CSSD:2950017088: clssscmain: OCRID is 0
2018-03-09 16:39:26.860 :    CSSD:2950017088: clssscmain: Cluster GUID is 89e968f37e66cf79bf4a5b7353e8e859
2018-03-09 16:39:26.860 :    CSSD:2950017088: clssscUpdateInitState: Set state to 0x00881006, based on prior state of 0x00881002 and requested change of 0x00000004
2018-03-09 16:39:26.860 : CSSDBCM:2950017088: clssbcmInitialize: flags(0)
2018-03-09 16:39:26.860 : CSSDBCM:2950017088: clssbcm_SetGlobal: global(0xe69560)
2018-03-09 16:39:26.860 : CSSDBCM:2950017088: clssbcmContext: context pointer(0x7fff90720590), flags(0)
2018-03-09 16:39:26.861 : CSSDBCM:2950017088: clssbcmNewEvent: event(0xe704c0), name(BCCM State), value(1)
2018-03-09 16:39:26.861 : CSSDBCM:2950017088: clssbcmNewEvent: event(0xe664b0), name(BCCM Work State), value(0)
2018-03-09 16:39:26.861 : CSSDBCM:2950017088: clssbcm_NewWorkThread: Spawn status(1)
2018-03-09 16:39:26.862 :    CSSD:2775164672: clssscthrdmain: Starting thread BCCM_Worker
2018-03-09 16:39:26.862 : CSSDBCM:2775164672: clssbcm_worker: Spawned with thread(0xe673f0)
2018-03-09 16:39:26.862 : CSSDBCM:2950017088: clssbcmContext: context(0xe6bb30) created
2018-03-09 16:39:26.862 : CSSDBCM:2950017088: clssbcmContext: context pointer(0x7fff90720590), flags(0)
2018-03-09 16:39:26.862 : CSSDBCM:2950017088: clssbcmNewEvent: event(0xd07650), name(BCCM State), value(1)
2018-03-09 16:39:26.863 : CSSDBCM:2950017088: clssbcmNewEvent: event(0xd07790), name(BCCM Work State), value(0)
2018-03-09 16:39:26.863 : CSSDBCM:2950017088: clssbcm_NewWorkThread: Spawn status(1)
2018-03-09 16:39:26.864 :    CSSD:2773587712: clssscthrdmain: Starting thread BCCM_Worker
2018-03-09 16:39:26.864 : CSSDBCM:2773587712: clssbcm_worker: Spawned with thread(0xd07a90)
2018-03-09 16:39:26.864 : CSSDBCM:2950017088: clssbcmContext: context(0xd0ecd0) created
2018-03-09 16:39:26.864 : CSSDBCM:2950017088: clssbcmSetGIPCTraceLevel: context(0xe6bb30), trace(2)
2018-03-09 16:39:26.864 : CSSDBCM:2950017088: clssbcmSetGIPCTraceLevel: context(0xd0ecd0), trace(2)
2018-03-09 16:39:26.864 :    CSSD:2950017088: clssscmain: PERF_TIME initializing GNS for Flex
2018-03-09 16:39:26.864 : CSSDGNS:2950017088: clssscBQInitialize: Initialized GNS Work queue

2018-03-09 16:39:26.864 :    CSSD:2950017088: clssscWaitOnInitState: Waiting on requested state 0x00040000, current state 0x00881006, timeout 4294967295
2018-03-09 16:39:26.865 :    CSSD:2772010752: clssscthrdmain: Starting thread GNSWork
2018-03-09 16:39:26.865 :    CSSD:2772010752: clssscUpdateInitState: Set state to 0x008c1006, based on prior state of 0x00881006 and requested change of 0x00040000
2018-03-09 16:39:26.866 :   CLSNS:2772010752: clsns_SetTraceLevel:trace level set to 1.
2018-03-09 16:39:26.866 : CSSDGNS:2772010752: clssgnsCrsQuery: Querying CRS for resource type "ora.gns.type".
2018-03-09 16:39:26.867 :    CSSD:2950017088: clssscWaitOnInitState: returning 1, requested state 0x00040000, current state 0x008c1006
2018-03-09 16:39:26.867 :    CSSD:2950017088: clssscmain: PERF_TIME initialized GNS for Flex
2018-03-09 16:39:26.868 :    CSSD:2950017088: clssscWaitOnInitState: Waiting on requested state 0x00000400, current state 0x008c1006, timeout 4294967295
2018-03-09 16:39:26.869 :    CSSD:2770433792: clssscthrdmain: Starting thread BCNMServer
2018-03-09 16:39:26.869 :    CSSD:2770433792: clssbnmsThread: Spawned
2018-03-09 16:39:26.869 :    CSSD:2770433792: clssbnmInitBCNMCommon: Initialialization complete for comm ctx(0xd0fbb8)
2018-03-09 16:39:26.869 :    CSSD:2770433792: clssscUpdateInitState: Set state to 0x008c1406, based on prior state of 0x008c1006 and requested change of 0x00000400
2018-03-09 16:39:26.869 :    CSSD:2770433792: clssscWaitOnInitState: Waiting on requested state 0x00020000, current state 0x008c1406, timeout 4294967295
2018-03-09 16:39:26.871 :    CSSD:2950017088: clssscWaitOnInitState: returning 1, requested state 0x00000400, current state 0x008c1406
2018-03-09 16:39:26.872 :    CSSD:2950017088: clssscWaitOnInitState: Waiting on requested state 0x00000800, current state 0x008c1406, timeout 4294967295
2018-03-09 16:39:26.873 :    CSSD:2768856832: clssscthrdmain: Starting thread BCNMClient
2018-03-09 16:39:26.873 :    CSSD:2768856832: clssbnmcThread: Thread Spawned
2018-03-09 16:39:26.873 :    CSSD:2768856832: clssbnmInitBCNMCommon: Initialialization complete for comm ctx(0xecc348)
2018-03-09 16:39:26.873 :    CSSD:2768856832: clssscUpdateInitState: Set state to 0x008c1c06, based on prior state of 0x008c1406 and requested change of 0x00000800
2018-03-09 16:39:26.874 : CSSDGNS:2772010752: clssgnsCrsQuery: CRS is not ready. Cannot query GNS resource state.
2018-03-09 16:39:26.874 : CLSCEVT:2772010752: clsce_subscribe 0x7f568011b540 filter='^(CRS_RESOURCE_PROFILE_CHANGE|CRS_RESOURCE_STATE_CHANGE),.*?NAME='ora.gns'', flags=0, handler=0x58fa24, 

arg=0xd082c0
2018-03-09 16:39:26.875 :    CSSD:2950017088: clssscWaitOnInitState: returning 1, requested state 0x00000800, current state 0x008c1c06
2018-03-09 16:39:26.876 :    CSSD:2950017088: clssscGetParameterOLR: OLR fetch for parameter node number hint (6) failed with rc 21
2018-03-09 16:39:26.886 :    CSSD:2950017088: clssgmGMPInitCtx: GMP context init done
2018-03-09 16:39:26.886 :    CSSD:2950017088: clssscUpdateInitState: Set state to 0x008c1e06, based on prior state of 0x008c1c06 and requested change of 0x00000200
2018-03-09 16:39:26.887 : CLSCEVT:2772010752: (:CLSCE0022:)clsce_subscribe 0x7f568011b540 successfully subscribed : 0
2018-03-09 16:39:26.888 :    CSSD:2764130048: clssscthrdmain: Starting thread GM Peer Lsnr
2018-03-09 16:39:26.888 :    CSSD:2764130048: clssgmPeerListener: Spawned for node
2018-03-09 16:39:26.888 :    CSSD:2764130048: clssscWaitOnInitState: Waiting on requested state 0x00000001, current state 0x008c1e06, timeout 4294967295
2018-03-09 16:39:26.890 :    CSSD:2762553088: clssscthrdmain: Starting thread GMC Req Thrd
2018-03-09 16:39:26.890 :    CSSD:2762553088: clssgmGMCRequestThread: Starting
2018-03-09 16:39:26.890 :    CSSD:2950017088: clsswtStartWrkthrds: Workerthread GMPLstnrWorkerThread, thrdnum 1 spawn success
2018-03-09 16:39:26.891 :    CSSD:2950017088: clsswtStartWrkthrds: Workerthread GMPLstnrWorkerThread, thrdnum 2 spawn success
2018-03-09 16:39:26.892 :    CSSD:2950017088: clsswtStartWrkthrds: Workerthread GMPLstnrWorkerThread, thrdnum 3 spawn success
2018-03-09 16:39:26.893 :    CSSD:2950017088: clsswtStartWrkthrds: Workerthread GMCReqWorkerThread, thrdnum 1 spawn success
2018-03-09 16:39:26.893 :    CSSD:2950017088: clsswtStartWrkthrds: Workerthread GMCReqWorkerThread, thrdnum 2 spawn success
2018-03-09 16:39:26.894 :    CSSD:2950017088: clsswtStartWrkthrds: Workerthread GMCReqWorkerThread, thrdnum 3 spawn success
2018-03-09 16:39:26.896 :    CSSD:2760976128: clssscthrdmain: Starting thread GMPLstnrWorkerThread
2018-03-09 16:39:26.896 :    CSSD:2760976128: clssgmPeerWorkerThread: thrdname GMPLstnrWorkerThread, num 35, wrkthrdnum 1
2018-03-09 16:39:26.897 :    CSSD:2759399168: clssscthrdmain: Starting thread GMPLstnrWorkerThread
2018-03-09 16:39:26.897 :    CSSD:2759399168: clssgmPeerWorkerThread: thrdname GMPLstnrWorkerThread, num 35, wrkthrdnum 2
2018-03-09 16:39:26.897 :    CSSD:2757822208: clssscthrdmain: Starting thread GMPLstnrWorkerThread
2018-03-09 16:39:26.898 :    CSSD:2757822208: clssgmPeerWorkerThread: thrdname GMPLstnrWorkerThread, num 35, wrkthrdnum 3
2018-03-09 16:39:26.899 :    CSSD:2756245248: clssscthrdmain: Starting thread GMCReqWorkerThread
2018-03-09 16:39:26.899 :    CSSD:2756245248: clssgmpcGMCReqWorkerThread: thrdname GMCReqWorkerThread, num 36, wrkthrdnum 1 spawned
2018-03-09 16:39:26.899 :    CSSD:2754668288: clssscthrdmain: Starting thread GMCReqWorkerThread
2018-03-09 16:39:26.899 :    CSSD:2754668288: clssgmpcGMCReqWorkerThread: thrdname GMCReqWorkerThread, num 36, wrkthrdnum 2 spawned
2018-03-09 16:39:26.900 :    CSSD:2753091328: clssscthrdmain: Starting thread GMCReqWorkerThread
2018-03-09 16:39:26.900 :    CSSD:2753091328: clssgmpcGMCReqWorkerThread: thrdname GMCReqWorkerThread, num 36, wrkthrdnum 3 spawned
2018-03-09 16:39:26.901 :    CSSD:2950017088: clssscGetParameterOLR: OLR fetch for parameter auth rep (9) failed with rc 21
2018-03-09 16:39:26.901 :    CSSD:2950017088: clssgmceventsub_construct: eventsub(0x13e0610), gmc(0x139adf0), name(Event Subscription Cache)
2018-03-09 16:39:26.901 :    CSSD:2950017088: clssgmGMCInitCtx: GMC context init done
2018-03-09 16:39:26.902 :    CSSD:2011944704: clssscthrdmain: Starting thread GMClientListener
2018-03-09 16:39:26.902 :    CSSD:2011944704: clssgmclientlsnr: Spawned
2018-03-09 16:39:26.902 :    CSSD:2011944704: clssscqueue_init: queue(0x7f5658032cd0), max(0)
2018-03-09 16:39:26.902 :    CSSD:2011944704: clssscqueue_init: queue(0x7f5658032fd0), max(0)
2018-03-09 16:39:26.903 :    CSSD:2011944704: clsssclsnrsetup: endp 0x20a for clsc://(ADDRESS=(PROTOCOL=ipc)(KEY=OCSSD_LL_cs2_)(GIPCID=00000000-00000000-29348))
2018-03-09 16:39:26.903 :    CSSD:2011944704: clssgmclientlsnr: listening on clsc://(ADDRESS=(PROTOCOL=ipc)(KEY=OCSSD_LL_cs2_)(GIPCID=00000000-00000000-29348))
2018-03-09 16:39:26.903 :    CSSD:2011944704: clssgmceventsub_construct: eventsub(0x7f565805a340), gmc(0x139adf0), name(Event Subscription Cache)
2018-03-09 16:39:26.903 :    CSSD:2011944704: clssscWaitOnInitState: Waiting on requested state 0x00000001, current state 0x008c1e06, timeout 4294967295
2018-03-09 16:39:26.905 :    CSSD:2010367744: clssscthrdmain: Starting thread GMDeathCheck
2018-03-09 16:39:26.905 :    CSSD:2010367744: clssgmDeathChkThread: Spawned
2018-03-09 16:39:26.907 :    CSSD:2008790784: clssscthrdmain: Starting thread GMPResponse
2018-03-09 16:39:26.907 :    CSSD:2950017088: clssscUpdateInitState: Set state to 0x008c1e46, based on prior state of 0x008c1e06 and requested change of 0x00000040
2018-03-09 16:39:26.908 :    CSSD:2950017088: clssscGetParameterProfile: buffer passed for parameter ASM discovery (3) is too short, required 25, passed 20
2018-03-09 16:39:26.908 :    CSSD:2950017088: clssnmReadDiscoveryProfile: voting file discovery string(/dev/sd*,/dev/asm*,AFD:*)
2018-03-09 16:39:26.908 :    CSSD:2950017088: clssnkInit: NK generic layer initializing.
2018-03-09 16:39:26.909 :    CSSD:2007213824: clssscthrdmain: Starting thread clssnmvDDiscThread
2018-03-09 16:39:26.910 :    GPNP:2007213824: clsgpnp_Init: HMAC hash type in context is SHA-512
2018-03-09 16:39:26.910 :    GPNP:2007213824: clsgpnp_Init: [at clsgpnp0.c:684] '/u01/app/product/12.2.0/crs' in effect as GPnP home base.
2018-03-09 16:39:26.910 :    GPNP:2007213824: clsgpnp_Init: [at clsgpnp0.c:750] GPnP pid=29348, cli=clsuGpnpg GPNP comp tracelevel=1, depcomp tracelevel=0, tlsrc:init, apitl:0, tstenv:0, 

devenv:0, envopt:0, flags=2003
2018-03-09 16:39:26.915 :    GPNP:2007213824: clsgpnpkwf_initwfloc: [at clsgpnpkwf.c:403] Using FS Wallet Location : /u01/app/product/12.2.0/crs/gpnp/cs2/wallets/peer/

2018-03-09 16:39:26.915 :    GPNP:2007213824: clsgpnpkwf_initwfloc: [at clsgpnpkwf.c:415] Wallet readable. Path: /u01/app/product/12.2.0/crs/gpnp/cs2/wallets/peer/

2018-03-09 16:39:26.940 :    CLSF:2007213824: Allocated CLSF context
2018-03-09 16:39:26.940 :    CSSD:2007213824: clssnmvDDiscThread: using discovery string /dev/sd*,/dev/asm*,AFD:* for initial discovery 
2018-03-09 16:39:26.940 :   SKGFD:2007213824: Discovery with str:/dev/sd*,/dev/asm*,AFD:*:

2018-03-09 16:39:26.940 :   SKGFD:2007213824: UFS discovery with :/dev/sd*:

2018-03-09 16:39:26.940 :   SKGFD:2007213824: Execute glob on the string /dev/sd*

2018-03-09 16:39:26.940 :   SKGFD:2007213824: running stat on disk:/dev/sda2
2018-03-09 16:39:26.940 :   SKGFD:2007213824: running stat on disk:/dev/sda1
2018-03-09 16:39:26.940 :   SKGFD:2007213824: running stat on disk:/dev/sdc
2018-03-09 16:39:26.940 :   SKGFD:2007213824: running stat on disk:/dev/sdb
2018-03-09 16:39:26.940 :   SKGFD:2007213824: running stat on disk:/dev/sdf
2018-03-09 16:39:26.940 :   SKGFD:2007213824: running stat on disk:/dev/sde
2018-03-09 16:39:26.940 :   SKGFD:2007213824: running stat on disk:/dev/sdd
2018-03-09 16:39:26.940 :   SKGFD:2007213824: running stat on disk:/dev/sda
2018-03-09 16:39:27.128 :    CSSD:2782349056: clssscagSelect: endpoint(0x85) authenticated with user(root)
2018-03-09 16:39:27.128 :    CSSD:2782349056: clssscagProcessInitialMsg: Handshake successful with agent 1
2018-03-09 16:39:27.128 :    CSSD:2782349056: clssscagProcAgReq: got a successful connection
2018-03-09 16:39:27.128 :    CSSD:2782349056: clssscagProcAgReq: Sending initdata
2018-03-09 16:39:27.128 :    CSSD:2782349056: clssscagProcessInitialMsg: notify agent 1 that it is active
2018-03-09 16:39:27.128 :    CSSD:2782349056: clssscagProcessInitialMsg: connection from agent 1, endp 0x85 - agents joined 0
2018-03-09 16:39:27.718 :    CSSD:2782349056: clssscagSelect: endpoint(0x74) authenticated with user(root)
2018-03-09 16:39:27.718 :    CSSD:2782349056: clssscagProcessInitialMsg: Handshake successful with agent 0
2018-03-09 16:39:27.718 :    CSSD:2782349056: clssscagProcAgReq: got a successful connection
2018-03-09 16:39:27.718 :    CSSD:2782349056: clssscagProcAgReq: Sending initdata
2018-03-09 16:39:27.718 :    CSSD:2782349056: clssscagProcessInitialMsg: connection from agent 0, endp 0x74 - agents joined 0
2018-03-09 16:39:27.868 :    CSSD:2778318592: clsssc_CLSFAInit_CB: System not ready for CLSFA initialization
2018-03-09 16:39:28.868 :    CSSD:2778318592: clsssc_CLSFAInit_CB: System not ready for CLSFA initialization
2018-03-09 16:39:29.868 :    CSSD:2778318592: clsssc_CLSFAInit_CB: System not ready for CLSFA initialization
2018-03-09 16:39:30.689 :   SKGFD:2007213824: Fetching UFS disk :/dev/sda:

2018-03-09 16:39:30.689 :    CLSF:2007213824: Ignoring 0-byte file /dev/sda

2018-03-09 16:39:30.689 :   SKGFD:2007213824: Fetching UFS disk :/dev/sdd:

2018-03-09 16:39:30.689 :    CLSF:2007213824: Ignoring 0-byte file /dev/sdd

2018-03-09 16:39:30.689 :   SKGFD:2007213824: Fetching UFS disk :/dev/sde:

2018-03-09 16:39:30.689 :    CLSF:2007213824: Ignoring 0-byte file /dev/sde

2018-03-09 16:39:30.689 :   SKGFD:2007213824: Fetching UFS disk :/dev/sdf:

2018-03-09 16:39:30.689 :    CLSF:2007213824: Ignoring 0-byte file /dev/sdf

2018-03-09 16:39:30.689 :   SKGFD:2007213824: Fetching UFS disk :/dev/sdb:

2018-03-09 16:39:30.689 :    CLSF:2007213824: Ignoring 0-byte file /dev/sdb

2018-03-09 16:39:30.689 :   SKGFD:2007213824: Fetching UFS disk :/dev/sdc:

2018-03-09 16:39:30.689 :    CLSF:2007213824: Ignoring 0-byte file /dev/sdc

2018-03-09 16:39:30.689 :   SKGFD:2007213824: Fetching UFS disk :/dev/sda1:

2018-03-09 16:39:30.689 :    CLSF:2007213824: Ignoring 0-byte file /dev/sda1

2018-03-09 16:39:30.689 :   SKGFD:2007213824: Fetching UFS disk :/dev/sda2:

2018-03-09 16:39:30.689 :    CLSF:2007213824: Ignoring 0-byte file /dev/sda2

2018-03-09 16:39:30.689 :   SKGFD:2007213824: OSS discovery with :/dev/sd*:

2018-03-09 16:39:30.690 :   SKGFD:2007213824: Discovery with asmlib :ASM:AFD Library - Generic , version 3 (KABI_V3): str :/dev/sd*:

2018-03-09 16:39:30.690 :   SKGFD:2007213824: Discovery advancing to nxt string :/dev/asm*:

2018-03-09 16:39:30.690 :   SKGFD:2007213824: UFS discovery with :/dev/asm*:

2018-03-09 16:39:30.690 :   SKGFD:2007213824: Execute glob on the string /dev/asm*

2018-03-09 16:39:30.690 :   SKGFD:2007213824: running stat on disk:/dev/asm
2018-03-09 16:39:30.690 :   SKGFD:2007213824: running stat on disk:/dev/asmdisk05
2018-03-09 16:39:30.692 :   SKGFD:2007213824: running stat on disk:/dev/asmdisk04
2018-03-09 16:39:30.693 :   SKGFD:2007213824: running stat on disk:/dev/asmdisk03
2018-03-09 16:39:30.707 :   SKGFD:2007213824: running stat on disk:/dev/asmdisk02
2018-03-09 16:39:30.729 :   SKGFD:2007213824: running stat on disk:/dev/asmdisk01
2018-03-09 16:39:30.869 :    CSSD:2778318592: clsssc_CLSFAInit_CB: System not ready for CLSFA initialization
2018-03-09 16:39:31.869 :    CSSD:2778318592: clsssc_CLSFAInit_CB: System not ready for CLSFA initialization
2018-03-09 16:39:32.869 :    CSSD:2778318592: clsssc_CLSFAInit_CB: System not ready for CLSFA initialization
2018-03-09 16:39:33.869 :    CSSD:2778318592: clsssc_CLSFAInit_CB: System not ready for CLSFA initialization
2018-03-09 16:39:34.869 :    CSSD:2778318592: clsssc_CLSFAInit_CB: System not ready for CLSFA initialization
2018-03-09 16:39:35.163 :   SKGFD:2007213824: Fetching UFS disk :/dev/asmdisk01:

2018-03-09 16:39:35.163 :   SKGFD:2007213824: Fetching UFS disk :/dev/asmdisk02:

2018-03-09 16:39:35.163 :   SKGFD:2007213824: Fetching UFS disk :/dev/asmdisk03:

2018-03-09 16:39:35.163 :   SKGFD:2007213824: Fetching UFS disk :/dev/asmdisk04:

2018-03-09 16:39:35.163 :   SKGFD:2007213824: Fetching UFS disk :/dev/asmdisk05:

2018-03-09 16:39:35.163 :   SKGFD:2007213824: Fetching UFS disk :/dev/asm:

2018-03-09 16:39:35.163 :    CLSF:2007213824: Ignoring 0-byte file /dev/asm

2018-03-09 16:39:35.163 :   SKGFD:2007213824: OSS discovery with :/dev/asm*:

2018-03-09 16:39:35.164 :   SKGFD:2007213824: Discovery with asmlib :ASM:AFD Library - Generic , version 3 (KABI_V3): str :/dev/asm*:

2018-03-09 16:39:35.164 :   SKGFD:2007213824: Discovery advancing to nxt string :AFD:*:

2018-03-09 16:39:35.164 :   SKGFD:2007213824: UFS discovery with :AFD:*:

2018-03-09 16:39:35.164 :   SKGFD:2007213824: Execute glob on the string /u01/app/product/12.2.0/crs/dbs/AFD:*

2018-03-09 16:39:35.869 :    CSSD:2778318592: clsssc_CLSFAInit_CB: System not ready for CLSFA initialization
2018-03-09 16:39:36.870 :    CSSD:2778318592: clsssc_CLSFAInit_CB: System not ready for CLSFA initialization
2018-03-09 16:39:37.870 :    CSSD:2778318592: clsssc_CLSFAInit_CB: System not ready for CLSFA initialization
2018-03-09 16:39:38.691 :   SKGFD:2007213824: OSS discovery with :AFD:*:

2018-03-09 16:39:38.692 :   SKGFD:2007213824: Discovery with asmlib :ASM:AFD Library - Generic , version 3 (KABI_V3): str :AFD:*:

2018-03-09 16:39:38.692 :   SKGFD:2007213824: Fetching asmlib disk :AFD:CRS2:

2018-03-09 16:39:38.692 :   SKGFD:2007213824: Handle 0x7f5644337dc0 from lib :UFS:: for disk :/dev/asmdisk01:

2018-03-09 16:39:38.692 :   SKGFD:2007213824: Handle 0x7f5644338d00 from lib :UFS:: for disk :/dev/asmdisk02:

2018-03-09 16:39:38.692 :   SKGFD:2007213824: Handle 0x7f5644339c40 from lib :UFS:: for disk :/dev/asmdisk03:

2018-03-09 16:39:38.693 :   SKGFD:2007213824: Handle 0x7f564433ab80 from lib :UFS:: for disk :/dev/asmdisk04:

2018-03-09 16:39:38.693 :   SKGFD:2007213824: Handle 0x7f564433bc50 from lib :UFS:: for disk :/dev/asmdisk05:

2018-03-09 16:39:38.693 :   SKGFD:2007213824: Handle 0x7f564433cd20 from lib :ASM:AFD Library - Generic , version 3 (KABI_V3): for disk :AFD:CRS2:

2018-03-09 16:39:38.693 :    CLSF:2007213824:  Warning: Voting disk: /dev/asmdisk01 is Hard mounted 

2018-03-09 16:39:38.693 :   SKGFD:2007213824: Lib :UFS:: closing handle 0x7f5644337dc0 for disk :/dev/asmdisk01:

2018-03-09 16:39:38.693 :    CLSF:2007213824:  Warning: Voting disk: /dev/asmdisk02 is Hard mounted 

2018-03-09 16:39:38.693 :   SKGFD:2007213824: Lib :UFS:: closing handle 0x7f5644338d00 for disk :/dev/asmdisk02:

2018-03-09 16:39:38.693 :    CLSF:2007213824: Read header of dev:/dev/asmdisk03:none:
2018-03-09 16:39:38.693 :   SKGFD:2007213824: Lib :UFS:: closing handle 0x7f5644339c40 for disk :/dev/asmdisk03:

2018-03-09 16:39:38.693 :    CLSF:2007213824: Read header of dev:/dev/asmdisk04:none:
2018-03-09 16:39:38.693 :   SKGFD:2007213824: Lib :UFS:: closing handle 0x7f564433ab80 for disk :/dev/asmdisk04:

2018-03-09 16:39:38.702 :    CLSF:2007213824: Read header of dev:/dev/asmdisk05:none:
2018-03-09 16:39:38.702 :   SKGFD:2007213824: Lib :UFS:: closing handle 0x7f564433bc50 for disk :/dev/asmdisk05:

2018-03-09 16:39:38.702 :   SKGFD:2007213824: Lib :ASM:AFD Library - Generic , version 3 (KABI_V3): closing handle 0x7f564433cd20 for disk :AFD:CRS2:

2018-03-09 16:39:38.702 :    CSSD:2007213824: clssnmvDiskVerify: discovered a potential voting file
2018-03-09 16:39:38.702 :   SKGFD:2007213824: Handle 0x7f564432fec0 from lib :UFS:: for disk :/dev/asmdisk01:

2018-03-09 16:39:38.702 :    CLSF:2007213824: Opened hdl:0x7f564432e790 for dev:/dev/asmdisk01:
2018-03-09 16:39:38.702 :    CSSD:2007213824: clssnmvDiskCreate: name /dev/asmdisk01 blocksz 512
2018-03-09 16:39:38.711 :    CSSD:2007213824: clssnmvDiskCreate: siteid during discovery = 00000000000000000000000000000000
2018-03-09 16:39:38.711 :    CSSD:2007213824: clssnmvDiskVerify: Successful discovery for disk /dev/asmdisk01, UID 750a78e1-ae984fcd-bfb4dbf4-4d337a77, SID 00112233-44556677-8899aabb-

ccddeeff, Pending CIN 0:1520582274:0, Committed CIN 0
:1520582274:0
2018-03-09 16:39:38.711 :    CLSF:2007213824: Closing handle:0x7f564432e790
2018-03-09 16:39:38.711 :   SKGFD:2007213824: Lib :UFS:: closing handle 0x7f564432fec0 for disk :/dev/asmdisk01:

2018-03-09 16:39:38.711 :    CSSD:2007213824: clssnmvDiskVerify: discovered a potential voting file
2018-03-09 16:39:38.711 :   SKGFD:2007213824: Handle 0x7f564432fec0 from lib :UFS:: for disk :/dev/asmdisk02:

2018-03-09 16:39:38.711 :    CLSF:2007213824: Opened hdl:0x7f564432e790 for dev:/dev/asmdisk02:
2018-03-09 16:39:38.711 :    CSSD:2007213824: clssnmvDiskCreate: name /dev/asmdisk02 blocksz 512
2018-03-09 16:39:38.712 :    CSSD:2007213824: clssnmvDiskCreate: siteid during discovery = 00000000000000000000000000000000
2018-03-09 16:39:38.712 :    CSSD:2007213824: clssnmFindVF: found VF by vdin in the discovered queue
2018-03-09 16:39:38.712 :    CSSD:2007213824: clssnmFindVF: Duplicate voting file found in the queue of previously discovered disks queued(/dev/asmdisk01|[750a78e1-ae984fcd-bfb4dbf4-

4d337a77]), found(/dev/asmdisk02|[750a78e1-ae984fcd-bfb4dbf4-4d337a77]), is not corrupted
2018-03-09 16:39:38.712 :    CLSF:2007213824: Resolved (/dev/asmdisk01,/dev/asmdisk02) to NONE
2018-03-09 16:39:38.712 :    CSSD:2007213824: clssnmvDiskCreate: Found a duplicate voting file /dev/asmdisk01 in the discovery queue which appears to be the same physical device as the 

newly discovered disk /dev/asmdisk02. Rejecting bot
h these files
2018-03-09 16:39:38.712 : default:2007213824: clssnmvDiskCreate:destroy_vdisk->vdisk:  dump of 0x0x7f56443d3760, len 10616

发现CRS磁盘组中磁盘/dev/asmdisk01中的voting file有一份重复的副本出现在/dev/asmdisk02磁盘中,因此拒绝了这些文件。

CSSD:2007213824: clssnmvDiskCreate: Found a duplicate voting file /dev/asmdisk01 in the discovery queue which appears to be the same physical device as the newly discovered disk 

/dev/asmdisk02. Rejecting both these files

如是在节点cs2上查看这两块磁盘,可以看到 /dev/asmdisk01与/dev/asmdisk02的minor number都显示为32,而cs1上的/dev/asmdisk01的minor number是16,/dev/asmdisk02的minor number为32

[root@cs2 /]# ls -lrt /dev/asm*
brw-rw---- 1 grid asmadmin 8, 32 Mar  9 12:21 /dev/asmdisk01
brw-rw---- 1 grid asmadmin 8, 32 Mar  9 12:23 /dev/asmdisk02
brw-rw---- 1 grid asmadmin 8, 48 Mar  9 12:23 /dev/asmdisk03
brw-rw---- 1 grid asmadmin 8, 64 Mar  9 12:23 /dev/asmdisk04
brw-rw---- 1 grid asmadmin 8, 80 Mar  9 12:23 /dev/asmdisk05

[root@cs1 /]# ls -lrt /dev/asm*
brw-rw---- 1 grid asmadmin 8, 48 Mar  9 12:23 /dev/asmdisk03
brw-rw---- 1 grid asmadmin 8, 64 Mar  9 12:23 /dev/asmdisk04
brw-rw---- 1 grid asmadmin 8, 80 Mar  9 12:23 /dev/asmdisk05
brw-rw---- 1 grid asmadmin 8, 32 Mar  9 15:57 /dev/asmdisk02
brw-rw---- 1 grid asmadmin 8, 16 Mar  9 15:57 /dev/asmdisk01

如果手动删除这些用来创建asm磁盘组的块设备

[root@cs2 /]# /bin/rm /dev/asmdisk01
[root@cs2 /]# /bin/rm /dev/asmdisk02
[root@cs2 /]# /bin/rm /dev/asmdisk023
[root@cs2 /]# /bin/rm /dev/asmdisk03
[root@cs2 /]# /bin/rm /dev/asmdisk04
[root@cs2 /]# /bin/rm /dev/asmdisk05
[root@cs2 /]# ls -lrt /dev/asm*
total 0

重新创建这些块设备

[root@cs2 /]# /sbin/udevadm trigger --type=devices --action=change
[root@cs2 /]# ls -lrt /dev/asm*
brw-rw---- 1 grid asmadmin 8, 16 Mar  9 17:37 /dev/asmdisk01
brw-rw---- 1 grid asmadmin 8, 32 Mar  9 17:37 /dev/asmdisk02
brw-rw---- 1 grid asmadmin 8, 64 Mar  9 17:37 /dev/asmdisk04
brw-rw---- 1 grid asmadmin 8, 80 Mar  9 17:37 /dev/asmdisk05
brw-rw---- 1 grid asmadmin 8, 48 Mar  9 17:37 /dev/asmdisk03

/dev/asm:
total 0

再次重新执行root.sh脚本就可以执行成功

[root@cs2 /]# ./u01/app/product/12.2.0/crs/root.sh
Performing root user operation.

The following environment variables are set as:
    ORACLE_OWNER= grid
    ORACLE_HOME=  /u01/app/product/12.2.0/crs

Enter the full pathname of the local bin directory: [/usr/local/bin]: 
The contents of "dbhome" have not changed. No need to overwrite.
The contents of "oraenv" have not changed. No need to overwrite.
The contents of "coraenv" have not changed. No need to overwrite.

Entries will be added to the /etc/oratab file as needed by
Database Configuration Assistant when a database is created
Finished running generic part of root script.
Now product-specific root actions will be performed.
Relinking oracle with rac_on option
Using configuration parameter file: /u01/app/product/12.2.0/crs/crs/install/crsconfig_params
The log of current session can be found at:
  /u01/app/grid/crsdata/cs2/crsconfig/rootcrs_cs2_2018-03-09_05-37-57PM.log
2018/03/09 17:38:01 CLSRSC-594: Executing installation step 1 of 19: 'SetupTFA'.
2018/03/09 17:38:01 CLSRSC-4001: Installing Oracle Trace File Analyzer (TFA) Collector.
2018/03/09 17:38:01 CLSRSC-4002: Successfully installed Oracle Trace File Analyzer (TFA) Collector.
2018/03/09 17:38:02 CLSRSC-594: Executing installation step 2 of 19: 'ValidateEnv'.
2018/03/09 17:38:05 CLSRSC-363: User ignored prerequisites during installation
2018/03/09 17:38:05 CLSRSC-594: Executing installation step 3 of 19: 'CheckFirstNode'.
2018/03/09 17:38:06 CLSRSC-594: Executing installation step 4 of 19: 'GenSiteGUIDs'.
2018/03/09 17:38:06 CLSRSC-594: Executing installation step 5 of 19: 'SaveParamFile'.
2018/03/09 17:38:09 CLSRSC-594: Executing installation step 6 of 19: 'SetupOSD'.
2018/03/09 17:38:11 CLSRSC-594: Executing installation step 7 of 19: 'CheckCRSConfig'.
2018/03/09 17:38:12 CLSRSC-594: Executing installation step 8 of 19: 'SetupLocalGPNP'.
2018/03/09 17:38:15 CLSRSC-594: Executing installation step 9 of 19: 'ConfigOLR'.
2018/03/09 17:38:16 CLSRSC-594: Executing installation step 10 of 19: 'ConfigCHMOS'.
2018/03/09 17:39:03 CLSRSC-594: Executing installation step 11 of 19: 'CreateOHASD'.
2018/03/09 17:39:04 CLSRSC-594: Executing installation step 12 of 19: 'ConfigOHASD'.
2018/03/09 17:39:06 CLSRSC-594: Executing installation step 13 of 19: 'InstallAFD'.
2018/03/09 17:39:08 CLSRSC-594: Executing installation step 14 of 19: 'InstallACFS'.
2018/03/09 17:39:11 CLSRSC-594: Executing installation step 15 of 19: 'InstallKA'.
2018/03/09 17:39:12 CLSRSC-594: Executing installation step 16 of 19: 'InitConfig'.
CRS-2791: Starting shutdown of Oracle High Availability Services-managed resources on 'cs2'
CRS-2673: Attempting to stop 'ora.driver.afd' on 'cs2'
CRS-2673: Attempting to stop 'ora.gpnpd' on 'cs2'
CRS-2673: Attempting to stop 'ora.drivers.acfs' on 'cs2'
CRS-2673: Attempting to stop 'ora.gipcd' on 'cs2'
CRS-2673: Attempting to stop 'ora.evmd' on 'cs2'
CRS-2673: Attempting to stop 'ora.cssdmonitor' on 'cs2'
CRS-2673: Attempting to stop 'ora.mdnsd' on 'cs2'
CRS-2677: Stop of 'ora.cssdmonitor' on 'cs2' succeeded
CRS-2677: Stop of 'ora.driver.afd' on 'cs2' succeeded
CRS-2677: Stop of 'ora.drivers.acfs' on 'cs2' succeeded
CRS-2677: Stop of 'ora.gpnpd' on 'cs2' succeeded
CRS-2677: Stop of 'ora.evmd' on 'cs2' succeeded
CRS-2677: Stop of 'ora.gipcd' on 'cs2' succeeded
CRS-2677: Stop of 'ora.mdnsd' on 'cs2' succeeded
CRS-2793: Shutdown of Oracle High Availability Services-managed resources on 'cs2' has completed
CRS-4133: Oracle High Availability Services has been stopped.
2018/03/09 17:39:17 CLSRSC-594: Executing installation step 17 of 19: 'StartCluster'.
CRS-4123: Starting Oracle High Availability Services-managed resources
CRS-2672: Attempting to start 'ora.evmd' on 'cs2'
CRS-2672: Attempting to start 'ora.mdnsd' on 'cs2'
CRS-2676: Start of 'ora.mdnsd' on 'cs2' succeeded
CRS-2676: Start of 'ora.evmd' on 'cs2' succeeded
CRS-2672: Attempting to start 'ora.gpnpd' on 'cs2'
CRS-2676: Start of 'ora.gpnpd' on 'cs2' succeeded
CRS-2672: Attempting to start 'ora.gipcd' on 'cs2'
CRS-2676: Start of 'ora.gipcd' on 'cs2' succeeded
CRS-2672: Attempting to start 'ora.cssdmonitor' on 'cs2'
CRS-2676: Start of 'ora.cssdmonitor' on 'cs2' succeeded
CRS-2672: Attempting to start 'ora.cssd' on 'cs2'
CRS-2672: Attempting to start 'ora.diskmon' on 'cs2'
CRS-2676: Start of 'ora.diskmon' on 'cs2' succeeded
CRS-2676: Start of 'ora.cssd' on 'cs2' succeeded
CRS-2672: Attempting to start 'ora.cluster_interconnect.haip' on 'cs2'
CRS-2672: Attempting to start 'ora.ctssd' on 'cs2'
CRS-2676: Start of 'ora.ctssd' on 'cs2' succeeded
CRS-2676: Start of 'ora.cluster_interconnect.haip' on 'cs2' succeeded
CRS-2672: Attempting to start 'ora.asm' on 'cs2'
CRS-2676: Start of 'ora.asm' on 'cs2' succeeded
CRS-2672: Attempting to start 'ora.storage' on 'cs2'
CRS-2676: Start of 'ora.storage' on 'cs2' succeeded
CRS-2672: Attempting to start 'ora.crf' on 'cs2'
CRS-2676: Start of 'ora.crf' on 'cs2' succeeded
CRS-2672: Attempting to start 'ora.crsd' on 'cs2'
CRS-2676: Start of 'ora.crsd' on 'cs2' succeeded
CRS-6017: Processing resource auto-start for servers: cs2
CRS-2673: Attempting to stop 'ora.LISTENER_SCAN1.lsnr' on 'cs1'
CRS-2672: Attempting to start 'ora.net1.network' on 'cs2'
CRS-2672: Attempting to start 'ora.ASMNET1LSNR_ASM.lsnr' on 'cs2'
CRS-2677: Stop of 'ora.LISTENER_SCAN1.lsnr' on 'cs1' succeeded
CRS-2676: Start of 'ora.net1.network' on 'cs2' succeeded
CRS-2673: Attempting to stop 'ora.scan1.vip' on 'cs1'
CRS-2672: Attempting to start 'ora.ons' on 'cs2'
CRS-2677: Stop of 'ora.scan1.vip' on 'cs1' succeeded
CRS-2672: Attempting to start 'ora.scan1.vip' on 'cs2'
CRS-2676: Start of 'ora.scan1.vip' on 'cs2' succeeded
CRS-2672: Attempting to start 'ora.LISTENER_SCAN1.lsnr' on 'cs2'
CRS-2676: Start of 'ora.ons' on 'cs2' succeeded
CRS-2676: Start of 'ora.ASMNET1LSNR_ASM.lsnr' on 'cs2' succeeded
CRS-2672: Attempting to start 'ora.asm' on 'cs2'
CRS-2676: Start of 'ora.LISTENER_SCAN1.lsnr' on 'cs2' succeeded
CRS-2676: Start of 'ora.asm' on 'cs2' succeeded
CRS-6016: Resource auto-start has completed for server cs2
CRS-6024: Completed start of Oracle Cluster Ready Services-managed resources
CRS-4123: Oracle High Availability Services has been started.
2018/03/09 17:42:11 CLSRSC-343: Successfully started Oracle Clusterware stack
2018/03/09 17:42:11 CLSRSC-594: Executing installation step 18 of 19: 'ConfigNode'.
2018/03/09 17:42:24 CLSRSC-594: Executing installation step 19 of 19: 'PostConfig'.
2018/03/09 17:43:16 CLSRSC-325: Configure Oracle Grid Infrastructure for a Cluster ... succeeded

到此这个问题就修复了

Proudly powered by WordPress | Indrajeet by Sus Hill.