Oracle Database 12c Release 2 (12.2) RAC On Oracle Linux 7 Using VMware

安装环境为Oracle Linux 7.1,Oracle版本为12.2.0.1,下面是RAC环境的IP配置

ip地址          主机名                   类型     解析方式 
10.10.10.171  jytest1 .jydba.net      public   DNS或etc/hosts 
10.10.10.172  jytest2 .jydba.net      public   DNS或etc/hosts 
88.88.88.1    jytest1 .jydba.net      private  DNS或etc/hosts 
88.88.88.2    jytest2 .jydba.net      private  DNS或etc/hosts 
10.10.10.175  jytest1 .jydba.net      virtual  DNS或etc/hosts 
10.10.10.176  jytest2 .jydba.net      virtual  DNS或etc/hosts
10.10.10.177  jytest-scan.jydba.net   scan     DNS或etc/hosts
10.10.10.178  jytest-scan.jydba.net   scan     DNS或etc/hosts
10.10.10.179  jytest-scan.jydba.net   scan     DNS或etc/hosts

[root@jytest1 soft]# cat /etc/hosts

127.0.0.1    localhost.jydba.net localhost
10.10.10.171 jytest1.jydba.net jytest1
10.10.10.172 jytest2.jydba.net jytest2

10.10.10.175 jytest1-vip.jydba.net jytest1-vip
10.10.10.176 jytest2-vip.jydba.net jytest2-vip

88.88.88.1   jytest1-priv.jydba.net jytest1-priv
88.88.88.2   jytest2-priv.jydba.net jytest2-priv

10.10.10.177 jytest-scan.jydba.net  jytest-scan
10.10.10.178 jytest-scan.jydba.net  jytest-scan
10.10.10.179 jytest-scan.jydba.net  jytest-scan 
[root@jytest2 ~]# cat /etc/hosts

127.0.0.1    localhost.jydba.net localhost
10.10.10.171 jytest1.jydba.net jytest1
10.10.10.172 jytest2.jydba.net jytest2

10.10.10.175 jytest1-vip.jydba.net jytest1-vip
10.10.10.176 jytest2-vip.jydba.net jytest2-vip

88.88.88.1   jytest1-priv.jydba.net jytest1-priv
88.88.88.2   jytest2-priv.jydba.net jytest2-priv

10.10.10.177 jytest-scan.jydba.net  jytest-scan
10.10.10.178 jytest-scan.jydba.net  jytest-scan
10.10.10.179 jytest-scan.jydba.net  jytest-scan 

开启ftp

[root@jytest1 vsftpd]# service vsftpd start
Redirecting to /bin/systemctl start  vsftpd.service

[root@jytest2 vsftpd]# service vsftpd start
Redirecting to /bin/systemctl start  vsftpd.service

下面这幅图显示的是/etc/vsftpd/ftpusers的内容,我们需要做的是在“root”字样前面添加注释符,通过这种手段来打开root用户对ftp功能的使用。同理,需要编辑的文件还有user_list:

[root@jytest1 ~]# vi ftpusers
# Users that are not allowed to login via ftp
#root
bin
daemon
adm
lp
sync
shutdown
halt
mail
news
uucp
operator
games
nobody

[root@jytest1 ~]## vi user_list
# vsftpd userlist
# If userlist_deny=NO, only allow users in this file
# If userlist_deny=YES (default), never allow users in this file, and
# do not even prompt for a password.
# Note that the default vsftpd pam config also checks /etc/vsftpd/ftpusers
# for users that are denied.
#root
bin
daemon
adm
lp
sync
shutdown
halt
mail
news
uucp
operator
games
nobody

通过编辑这两个文件,我们就可以在windows下以root用户登陆到虚拟机里同一网段的linux系统下

创建用户组

[root@jytest1 ~]# groupadd -g 1006 asmadmin
[root@jytest1 ~]# groupadd -g 1007 asmdba
[root@jytest1 ~]# groupadd -g 1008 asmoper
[root@jytest1 ~]# groupadd -g 1009 dba
[root@jytest1 ~]# groupadd -g 1010 oper
[root@jytest1 ~]# groupadd -g 1011 oinstall
[root@jytest1 ~]# groupadd -g 1012 backupdba
[root@jytest1 ~]# groupadd -g 1013 dgdba
[root@jytest1 ~]# groupadd -g 1014 kmdba
[root@jytest1 ~]# groupadd -g 1015 racdba


[root@jytest2 ~]# groupadd -g 1006 asmadmin
[root@jytest2 ~]# groupadd -g 1007 asmdba
[root@jytest2 ~]# groupadd -g 1008 asmoper
[root@jytest2 ~]# groupadd -g 1009 dba
[root@jytest2 ~]# groupadd -g 1010 oper
[root@jytest2 ~]# groupadd -g 1011 oinstall
[root@jytest2 ~]# groupadd -g 1012 backupdba
[root@jytest2 ~]# groupadd -g 1013 dgdba
[root@jytest2 ~]# groupadd -g 1014 kmdba
[root@jytest2 ~]# groupadd -g 1015 racdba

创建用户

[root@jytest1 ~]#useradd  -u 1001 -g oinstall -G dba,asmdba,backupdba,dgdba,kmdba,racdba,oper oracle
[root@jytest1 ~]#useradd  -u 1002 -g oinstall -G asmadmin,asmdba,asmoper,dba grid

[root@jytest2 ~]#useradd  -u 1001 -g oinstall -G dba,asmdba,backupdba,dgdba,kmdba,racdba,oper oracle
[root@jytest2 ~]#useradd  -u 1002 -g oinstall -G asmadmin,asmdba,asmoper,dba grid


[root@jytest1 /]# passwd grid
Changing password for user grid.
New password: 
BAD PASSWORD: The password is shorter than 8 characters
Retype new password: 
passwd: all authentication tokens updated successfully.
You have new mail in /var/spool/mail/root
[root@jytest1 /]# passwd oracle
Changing password for user oracle.
New password: 
BAD PASSWORD: The password is shorter than 8 characters
Retype new password: 
passwd: all authentication tokens updated successfully.

[root@jytest2 /]# passwd grid
Changing password for user grid.
New password: 
BAD PASSWORD: The password is shorter than 8 characters
Retype new password: 
passwd: all authentication tokens updated successfully.
[root@jytest2 /]# passwd oracle
Changing password for user oracle.
New password: 
BAD PASSWORD: The password is shorter than 8 characters
Retype new password: 
passwd: all authentication tokens updated successfully.

配置ASM所需磁盘,编辑/etc/udev/rules.d/99-my-asmdevices.rules配置文件

[root@jytest1 ~]# vi /etc/udev/rules.d/99-my-asmdevices.rules

KERNEL=="sd*[!0-9]", ENV{DEVTYPE}=="disk", SUBSYSTEM=="block",  PROGRAM=="/usr/lib/udev/scsi_id -g -u -d $devnode",  RESULT=="36000c293af84fb49683bbfd9a0b377ec", RUN+="/bin/sh -c 'mknod /dev/asmdisk01 b  $major $minor; chown grid:asmadmin /dev/asmdisk01; chmod 0660 /dev/asmdisk01'"

KERNEL=="sd*[!0-9]", ENV{DEVTYPE}=="disk", SUBSYSTEM=="block",  PROGRAM=="/usr/lib/udev/scsi_id -g -u -d $devnode",  RESULT=="36000c290196d69c481a2cf191d33868c", RUN+="/bin/sh -c 'mknod /dev/asmdisk02 b  $major $minor; chown grid:oinstall /dev/asmdisk02; chmod 0660 /dev/asmdisk02'"

KERNEL=="sd*[!0-9]", ENV{DEVTYPE}=="disk", SUBSYSTEM=="block",  PROGRAM=="/usr/lib/udev/scsi_id -g -u -d $devnode",  RESULT=="36000c2903ffae4d3865722d108fadc96", RUN+="/bin/sh -c 'mknod /dev/asmdisk03 b  $major $minor; chown grid:oinstall /dev/asmdisk03; chmod 0660 /dev/asmdisk03'"

KERNEL=="sd*[!0-9]", ENV{DEVTYPE}=="disk", SUBSYSTEM=="block",  PROGRAM=="/usr/lib/udev/scsi_id -g -u -d $devnode",  RESULT=="36000c29e1b47900ecad6f1458a3585ed", RUN+="/bin/sh -c 'mknod /dev/asmdisk04 b  $major $minor; chown grid:oinstall /dev/asmdisk04; chmod 0660 /dev/asmdisk04'"

KERNEL=="sd*[!0-9]", ENV{DEVTYPE}=="disk", SUBSYSTEM=="block",  PROGRAM=="/usr/lib/udev/scsi_id -g -u -d $devnode",  RESULT=="36000c29748d41fffbdddafdea6eb64bb", RUN+="/bin/sh -c 'mknod /dev/asmdisk05 b  $major $minor; chown grid:oinstall /dev/asmdisk05; chmod 0660 /dev/asmdisk05'"

KERNEL=="sd*[!0-9]", ENV{DEVTYPE}=="disk", SUBSYSTEM=="block",  PROGRAM=="/usr/lib/udev/scsi_id -g -u -d $devnode",  RESULT=="36000c2985afa13536000f5887b9acb92", RUN+="/bin/sh -c 'mknod /dev/asmdisk06 b  $major $minor; chown grid:oinstall /dev/asmdisk06; chmod 0660 /dev/asmdisk06'"

[root@jytest2 ~]# vi /etc/udev/rules.d/99-my-asmdevices.rules

KERNEL=="sd*[!0-9]", ENV{DEVTYPE}=="disk", SUBSYSTEM=="block",  PROGRAM=="/usr/lib/udev/scsi_id -g -u -d $devnode",  RESULT=="36000c293af84fb49683bbfd9a0b377ec", RUN+="/bin/sh -c 'mknod /dev/asmdisk01 b  $major $minor; chown grid:asmadmin /dev/asmdisk01; chmod 0660 /dev/asmdisk01'"

KERNEL=="sd*[!0-9]", ENV{DEVTYPE}=="disk", SUBSYSTEM=="block",  PROGRAM=="/usr/lib/udev/scsi_id -g -u -d $devnode",  RESULT=="36000c290196d69c481a2cf191d33868c", RUN+="/bin/sh -c 'mknod /dev/asmdisk02 b  $major $minor; chown grid:oinstall /dev/asmdisk02; chmod 0660 /dev/asmdisk02'"

KERNEL=="sd*[!0-9]", ENV{DEVTYPE}=="disk", SUBSYSTEM=="block",  PROGRAM=="/usr/lib/udev/scsi_id -g -u -d $devnode",  RESULT=="36000c2903ffae4d3865722d108fadc96", RUN+="/bin/sh -c 'mknod /dev/asmdisk03 b  $major $minor; chown grid:oinstall /dev/asmdisk03; chmod 0660 /dev/asmdisk03'"

KERNEL=="sd*[!0-9]", ENV{DEVTYPE}=="disk", SUBSYSTEM=="block",  PROGRAM=="/usr/lib/udev/scsi_id -g -u -d $devnode",  RESULT=="36000c29e1b47900ecad6f1458a3585ed", RUN+="/bin/sh -c 'mknod /dev/asmdisk04 b  $major $minor; chown grid:oinstall /dev/asmdisk04; chmod 0660 /dev/asmdisk04'"

KERNEL=="sd*[!0-9]", ENV{DEVTYPE}=="disk", SUBSYSTEM=="block",  PROGRAM=="/usr/lib/udev/scsi_id -g -u -d $devnode",  RESULT=="36000c29748d41fffbdddafdea6eb64bb", RUN+="/bin/sh -c 'mknod /dev/asmdisk05 b  $major $minor; chown grid:oinstall /dev/asmdisk05; chmod 0660 /dev/asmdisk05'"

KERNEL=="sd*[!0-9]", ENV{DEVTYPE}=="disk", SUBSYSTEM=="block",  PROGRAM=="/usr/lib/udev/scsi_id -g -u -d $devnode",  RESULT=="36000c2985afa13536000f5887b9acb92", RUN+="/bin/sh -c 'mknod /dev/asmdisk06 b  $major $minor; chown grid:oinstall /dev/asmdisk06; chmod 0660 /dev/asmdisk06'"

[root@jytest1 /]# /sbin/udevadm trigger --type=devices --action=change

[root@jytest2 /]# /sbin/udevadm trigger --type=devices --action=change

[root@jytest1 ~]# ls -lrt /dev/asm*
brw-rw---- 1 grid oinstall 8, 32 Mar 20 18:14 /dev/asmdisk02
brw-rw---- 1 grid asmadmin 8, 16 Mar 20 18:14 /dev/asmdisk01
brw-rw---- 1 grid oinstall 8, 48 Mar 20 23:01 /dev/asmdisk03
brw-rw---- 1 grid oinstall 8, 64 Mar 20 23:01 /dev/asmdisk04
brw-rw---- 1 grid oinstall 8, 80 Mar 20 23:02 /dev/asmdisk05
brw-rw---- 1 grid oinstall 8, 96 Mar 20 23:03 /dev/asmdisk06 

[root@jytest2 ~]# ls -lrt /dev/asm*
brw-rw---- 1 grid oinstall 8, 96 Mar 21 03:55 /dev/asmdisk06
brw-rw---- 1 grid oinstall 8, 80 Mar 21 03:55 /dev/asmdisk05
brw-rw---- 1 grid oinstall 8, 48 Mar 21 03:55 /dev/asmdisk03
brw-rw---- 1 grid oinstall 8, 64 Mar 21 03:55 /dev/asmdisk04
brw-rw---- 1 grid asmadmin 8, 16 Mar 21 03:55 /dev/asmdisk01
brw-rw---- 1 grid oinstall 8, 32 Mar 21 03:55 /dev/asmdisk02

以root用户创建“Oracle inventory 目录”

[root@jytest1 /]# mkdir -p /u01/app/oraInventory
[root@jytest1 /]# chown -R grid:oinstall /u01/app/oraInventory
[root@jytest1 /]# chmod -R 775 /u01/app/oraInventory

[root@jytest2 /]# mkdir -p /u01/app/oraInventory
[root@jytest2 /]# chown -R grid:oinstall /u01/app/oraInventory
[root@jytest2 /]# chmod -R 775 /u01/app/oraInventory

以root用户创建“Grid Infrastructure BASE 目录”

[root@jytest1 /]# mkdir -p /u01/app/grid
[root@jytest1 /]# chown -R grid:oinstall /u01/app/grid
[root@jytest1 /]# chmod -R 775 /u01/app/grid

[root@jytest2 /]# mkdir -p /u01/app/grid
[root@jytest2 /]# chown -R grid:oinstall /u01/app/grid
[root@jytest2 /]# chmod -R 775 /u01/app/grid

以root用户创建“Grid Infrastructure Home 目录”

[root@jytest1 /]# mkdir -p /u01/app/product/12.2.0/crs/
[root@jytest1 /]# chown -R grid:oinstall /u01/app/product/12.2.0/crs/
[root@jytest1 /]# chmod -R 775 /u01/app/product/12.2.0/crs/

[root@jytest2 /]# mkdir -p /u01/app/product/12.2.0/crs
[root@jytest2 /]# chown -R grid:oinstall /u01/app/product/12.2.0/crs
[root@jytest2 /]# chmod -R 775 /u01/app/product/12.2.0/crs

以root用户创建“Oracle Base 目录”

[root@jytest1 /]# mkdir -p /u01/app/oracle
[root@jytest1 /]# chown -R oracle:oinstall /u01/app/oracle
[root@jytest1 /]# chmod -R 775 /u01/app/oracle

[root@jytest2 /]# mkdir -p /u01/app/oracle
[root@jytest2 /]# chown -R oracle:oinstall /u01/app/oracle
[root@jytest2 /]# chmod -R 775 /u01/app/oracle

以root用户创建“Oracle RDBMS Home 目录”

[root@jytest1 /]# mkdir -p /u01/app/oracle/product/12.2.0/db
[root@jytest1 /]# chown -R oracle:oinstall /u01/app/oracle/product/12.2.0/db
[root@jytest1 /]# chmod -R 775 /u01/app/oracle/product/12.2.0/db

[root@jytest2 /]# mkdir -p /u01/app/oracle/product/12.2.0/db
[root@jytest2 /]# chown -R oracle:oinstall /u01/app/oracle/product/12.2.0/db
[root@jytest2 /]# chmod -R 775 /u01/app/oracle/product/12.2.0/db

创建一个tmp目录

[root@jytest1 /]# mkdir /u01/tmp
[root@jytest1 /]# chmod a+wr /u01/tmp

[root@jytest2 /]# mkdir /u01/tmp
[root@jytest2 /]# chmod a+wr /u01/tmp

设置环境变量

[root@jytest1 ~]# su - grid
[grid@jytest1 ~]$ vi .bash_profile
# .bash_profile

# Get the aliases and functions
if [ -f ~/.bashrc ]; then
        . ~/.bashrc
fi

# User specific environment and startup programs

PATH=$PATH:$HOME/.local/bin:$HOME/bin

export PATH

TEMP=/u01/tmp
TMPDIR=/u01/tmp
export TEMP TMPDIR
export LD_ASSUME_KERNEL=3.8.13
export ORACLE_BASE=/u01/app/grid
export ORACLE_HOME=/u01/app/product/12.2.0/crs/
export ORACLE_SID=+ASM1
export NLS_LANG=AMERICAN_AMERICA.ZHS16GBK
export ORA_NLS33=$ORACLE_HOME/ocommon/nls/admin/data
LD_LIBRARY_PATH=$ORACLE_HOME/lib:/lib:/usr/lib
LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/lib
export LD_LIBRARY_PATH
export PATH=$PATH:$ORACLE_HOME/bin
CLASSPATH=$ORACLE_HOME/JRE:$ORACLE_HOME/jlib:$ORACLE_HOME/rdbms/jlib
CLASSPATH=$CLASSPATH:$ORACLE_HOME/network/jlib
export CLASSPATH
umask=022

[root@jytest2 ~]# su - grid
[grid@jytest2 ~]$ vi .bash_profile
# .bash_profile

# Get the aliases and functions
if [ -f ~/.bashrc ]; then
        . ~/.bashrc
fi

# User specific environment and startup programs

PATH=$PATH:$HOME/.local/bin:$HOME/bin

export PATH

TEMP=/u01/tmp
TMPDIR=/u01/tmp
export TEMP TMPDIR
export LD_ASSUME_KERNEL=3.8.13
export ORACLE_BASE=/u01/app/grid
export ORACLE_HOME=/u01/app/product/12.2.0/crs
export ORACLE_SID=+ASM2
export NLS_LANG=AMERICAN_AMERICA.ZHS16GBK
export ORA_NLS33=$ORACLE_HOME/ocommon/nls/admin/data
LD_LIBRARY_PATH=$ORACLE_HOME/lib:/lib:/usr/lib
LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/lib
export LD_LIBRARY_PATH
export PATH=$PATH:$ORACLE_HOME/bin
CLASSPATH=$ORACLE_HOME/JRE:$ORACLE_HOME/jlib:$ORACLE_HOME/rdbms/jlib
CLASSPATH=$CLASSPATH:$ORACLE_HOME/network/jlib
export CLASSPATH
umask=022

[root@jytest1 ~]# su – oracle
[oracle@jytest1 ~]$ vi .bash_profile
# .bash_profile

# Get the aliases and functions
if [ -f ~/.bashrc ]; then
        . ~/.bashrc
fi

# User specific environment and startup programs

PATH=$PATH:$HOME/.local/bin:$HOME/bin

export PATH

TEMP=/u01/tmp
TMPDIR=/u01/tmp
export TEMP TMPDIR
export LD_ASSUME_KERNEL=3.8.13
export ORACLE_BASE=/u01/app/oracle
export ORACLE_HOME=/u01/app/oracle/product/12.2.0/db
export ORACLE_SID=jytest1
export ORACLE_UNQNAME=jytest
export NLS_LANG=AMERICAN_AMERICA.ZHS16GBK
export ORA_NLS33=$ORACLE_HOME/ocommon/nls/admin/data
LD_LIBRARY_PATH=$ORACLE_HOME/lib:/lib:/usr/lib
LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/lib
export LD_LIBRARY_PATH
export PATH=$PATH:$ORACLE_HOME/bin
CLASSPATH=$ORACLE_HOME/JRE:$ORACLE_HOME/jlib:$ORACLE_HOME/rdbms/jlib
CLASSPATH=$CLASSPATH:$ORACLE_HOME/network/jlib
export CLASSPATH
umask=022

[root@jytest2 ~]# su - oracle
[oracle@jytest2 ~]$ vi .bash_profile
# .bash_profile

# Get the aliases and functions
if [ -f ~/.bashrc ]; then
        . ~/.bashrc
fi

# User specific environment and startup programs

PATH=$PATH:$HOME/.local/bin:$HOME/bin

export PATH
TEMP=/u01/tmp
TMPDIR=/u01/tmp
export TEMP TMPDIR
export LD_ASSUME_KERNEL=3.8.13
export ORACLE_BASE=/u01/app/oracle
export ORACLE_HOME=/u01/app/oracle/product/12.2.0/db
export ORACLE_SID=jytest2
export ORACLE_UNQNAME=jytest
export NLS_LANG=AMERICAN_AMERICA.ZHS16GBK
export ORA_NLS33=$ORACLE_HOME/ocommon/nls/admin/data
LD_LIBRARY_PATH=$ORACLE_HOME/lib:/lib:/usr/lib
LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/lib
export LD_LIBRARY_PATH
export PATH=$PATH:$ORACLE_HOME/bin
CLASSPATH=$ORACLE_HOME/JRE:$ORACLE_HOME/jlib:$ORACLE_HOME/rdbms/jlib
CLASSPATH=$CLASSPATH:$ORACLE_HOME/network/jlib
export CLASSPATH
umask=022

修改内核参数编辑/etc/sysctl.conf文件

[root@jytest1 ~]# vi /etc/sysctl.conf
# System default settings live in /usr/lib/sysctl.d/00-system.conf.
# To override those settings, enter new settings here, or in an /etc/sysctl.d/.conf file
#
# For more information, see sysctl.conf(5) and sysctl.d(5).
fs.aio-max-nr = 1048576
fs.file-max = 6815744
kernel.shmall = 2097152
kernel.shmmax = 4294967295
kernel.shmmni = 4096
kernel.sem = 250 32000 100 128
net.ipv4.ip_local_port_range = 9000 65500
net.core.rmem_default = 262144
net.core.rmem_max = 4194304
net.core.wmem_default = 262144
net.core.wmem_max = 1048576

[root@jytest1 ~]# sysctl -p
fs.aio-max-nr = 1048576
fs.file-max = 6815744
kernel.shmall = 2097152
kernel.shmmax = 4294967295
kernel.shmmni = 4096
kernel.sem = 250 32000 100 128
net.ipv4.ip_local_port_range = 9000 65500
net.core.rmem_default = 262144
net.core.rmem_max = 4194304
net.core.wmem_default = 262144
net.core.wmem_max = 1048576

[root@jytest2 ~]# vi /etc/sysctl.conf
# System default settings live in /usr/lib/sysctl.d/00-system.conf.
# To override those settings, enter new settings here, or in an /etc/sysctl.d/.conf file
#
# For more information, see sysctl.conf(5) and sysctl.d(5).

fs.aio-max-nr = 1048576
fs.file-max = 6815744
kernel.shmall = 2097152
kernel.shmmax = 4294967295
kernel.shmmni = 4096
kernel.sem = 250 32000 100 128
net.ipv4.ip_local_port_range = 9000 65500
net.core.rmem_default = 262144
net.core.rmem_max = 4194304
net.core.wmem_default = 262144
net.core.wmem_max = 1048576
~
[root@jytest2 ~]# sysctl -p
fs.aio-max-nr = 1048576
fs.file-max = 6815744
kernel.shmall = 2097152
kernel.shmmax = 4294967295
kernel.shmmni = 4096
kernel.sem = 250 32000 100 128
net.ipv4.ip_local_port_range = 9000 65500
net.core.rmem_default = 262144
net.core.rmem_max = 4194304
net.core.wmem_default = 262144
net.core.wmem_max = 1048576

修改oarcle参数的shell限制,在所有节点的/etc/security/limits.conf文件中添加以下参数

grid soft nproc 2047
grid hard nproc 16384
grid soft nofile 1024
grid hard nofile 65536
grid soft stack 10240
grid hard stack 32768
grid soft memlock 3145728
grid hard memlock 3145728

oracle soft nproc 2047
oracle hard nproc 16384
oracle soft nofile 1024
oracle hard nofile 65536
oracle soft stack 10240
oracle hard stack 32768
oracle soft memlock 3145728
oracle hard memlock 3145728

修改shell的默认参数文件,在所有节点的/etc/profile文件中添加以下内容:

if [ $USER = "oracle" ]; then
if [ $SHELL = "/bin/ksh" ]; then
ulimit -p 16384
ulimit -n 65536
else
ulimit -u 16384 -n 65536
fi
fi


if [ $USER = "grid" ]; then
if [ $SHELL = "/bin/ksh" ]; then
ulimit -p 16384
ulimit -n 65536
else
ulimit -u 16384 -n 65536
fi
fi

对C shell(csh or tcsh) 在所有节点的/etc/csh.login文件中增加以下代码

if ( $USER == "oracle" ) then
limit maxproc 16384
limit descriptors 65536
endif

if ( $USER == "grid" ) then
limit maxproc 16384
limit descriptors 65536
endif

解压GI安装压缩包:

[grid@jytest1 soft]cd /u01/app/product/12.2.0/crs/
[grid@jytest1 soft]# unzip -p linuxx64_12201_grid_home.zip

配置vnc

[root@jytest1 system]# rpm -qa|grep tigervnc
tigervnc-server-minimal-1.2.80-0.30.20130314svn5065.el7.x86_64
tigervnc-license-1.2.80-0.30.20130314svn5065.el7.noarch
[root@jytest1 system]# cd /
[root@jytest1 /]# rpm -e tigervnc-server-1.2.80-0.30.20130314svn5065.el7.x86_64
[root@jytest1 /]# cd soft
[root@jytest1 soft]# yum -y install tigervnc-server-1.2.80-0.30.20130314svn5065.el7.x86_64.rpm
Loaded plugins: langpacks
Examining tigervnc-server-1.2.80-0.30.20130314svn5065.el7.x86_64.rpm: tigervnc-server-1.2.80-0.30.20130314svn5065.el7.x86_64
Marking tigervnc-server-1.2.80-0.30.20130314svn5065.el7.x86_64.rpm to be installed
Resolving Dependencies
--> Running transaction check
---> Package tigervnc-server.x86_64 0:1.2.80-0.30.20130314svn5065.el7 will be installed
--> Finished Dependency Resolution
http://public-yum.oracle.com/repo/OracleLinux/OL7/UEKR3/x86_64/repodata/repomd.xml: [Errno 14] curl#6 - "Could not resolve host: public-yum.oracle.com; Unknown error"
Trying other mirror.
http://public-yum.oracle.com/repo/OracleLinux/OL7/latest/x86_64/repodata/repomd.xml: [Errno 14] curl#6 - "Could not resolve host: public-yum.oracle.com; Unknown error"
Trying other mirror.

Dependencies Resolved

=======================================================================================================================================================================
 Package                     Arch               Version                                      Repository                                                           Size
=======================================================================================================================================================================
Installing:
 tigervnc-server             x86_64             1.2.80-0.30.20130314svn5065.el7              /tigervnc-server-1.2.80-0.30.20130314svn5065.el7.x86_64             488 k

Transaction Summary
=======================================================================================================================================================================
Install  1 Package

Total size: 488 k
Installed size: 488 k
Downloading packages:
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
Warning: RPMDB altered outside of yum.
  Installing : tigervnc-server-1.2.80-0.30.20130314svn5065.el7.x86_64                                                                                              1/1 
  Verifying  : tigervnc-server-1.2.80-0.30.20130314svn5065.el7.x86_64                                                                                              1/1 

Installed:
  tigervnc-server.x86_64 0:1.2.80-0.30.20130314svn5065.el7                                                                                                             

Complete!

使用模板创建配置文件

[root@jytest1 soft]# cp /lib/systemd/system/vncserver@.service /etc/systemd/system/vncserver_root@:2.service
[root@jytest1 soft]# cd /etc/systemd/system/
[root@jytest1 system]# vi vncserver_root@:2.service
# The vncserver service unit file
#
# Quick HowTo:
# 1. Copy this file to /etc/systemd/system/vncserver@:.service
# 2. Edit  and vncserver parameters appropriately
#   ("runuser -l  -c /usr/bin/vncserver %i -arg1 -arg2")
# 3. Run `systemctl daemon-reload`
# 4. Run `systemctl enable vncserver@:.service`
#
# DO NOT RUN THIS SERVICE if your local area network is
# untrusted!  For a secure way of using VNC, you should
# limit connections to the local host and then tunnel from
# the machine you want to view VNC on (host A) to the machine
# whose VNC output you want to view (host B)
#
# [user@hostA ~]$ ssh -v -C -L 590N:localhost:590M hostB
#
# this will open a connection on port 590N of your hostA to hostB's port 590M
# (in fact, it ssh-connects to hostB and then connects to localhost (on hostB).
# See the ssh man page for details on port forwarding)
#
# You can then point a VNC client on hostA at vncdisplay N of localhost and with
# the help of ssh, you end up seeing what hostB makes available on port 590M
#
# Use "-nolisten tcp" to prevent X connections to your VNC server via TCP.
#
# Use "-localhost" to prevent remote VNC clients connecting except when
# doing so through a secure tunnel.  See the "-via" option in the
# `man vncviewer' manual page.


[Unit]
Description=Remote desktop service (VNC)
After=syslog.target network.target

[Service]
Type=simple --修改为simple
# Clean any existing files in /tmp/.X11-unix environment
ExecStartPre=/bin/sh -c '/usr/bin/vncserver -kill %i > /dev/null 2>&1 || :'
ExecStart=/sbin/runuser -l root -c "/usr/bin/vncserver %i" --修改为用户root
PIDFile=/home/root/.vnc/%H%i.pid --修改为用户root
ExecStop=/bin/sh -c '/usr/bin/vncserver -kill %i > /dev/null 2>&1 || :'

[Install]
"vncserver_root@:2.service" 45L, 1739C written

设置分辨率

[root@jytest1 system]# sed -i 's/i"/i -geometry 800x800"/g' /etc/systemd/system/vncserver_root@:2.service

启用配置文件

[root@jytest1 system]# systemctl enable vncserver_root@:2.service
ln -s '/etc/systemd/system/vncserver_root@:2.service' '/etc/systemd/system/multi-user.target.wants/vncserver_root@:2.service'

启动vncserver服务

[root@jytest1 system]# vncserver

You will require a password to access your desktops.

Password:
Verify:
xauth:  file /root/.Xauthority does not exist

New 'jytest3:1 (root)' desktop is jytest3:1

Creating default startup script /root/.vnc/xstartup
Starting applications specified in /root/.vnc/xstartup
Log file is /root/.vnc/jytest3:1.log

关闭防火墙

[root@jytest1 system]# systemctl stop firewalld
[root@jytest1 system]# systemctl disable firewalld
rm '/etc/systemd/system/dbus-org.fedoraproject.FirewallD1.service'
rm '/etc/systemd/system/basic.target.wants/firewalld.service'


[root@jytest2 system]# systemctl stop firewalld
[root@jytest2 system]# systemctl disable firewalld
rm '/etc/systemd/system/dbus-org.fedoraproject.FirewallD1.service'
rm '/etc/systemd/system/basic.target.wants/firewalld.service'

禁用selinux,修改所有节点的/etc/selinux/config文件,编辑文本中的SELINUX=enforcing 为SELINUX=disabled

[root@jytest1 ~]# vi /etc/selinux/config

# This file controls the state of SELinux on the system.
# SELINUX= can take one of these three values:
#     enforcing - SELinux security policy is enforced.
#     permissive - SELinux prints warnings instead of enforcing.
#     disabled - No SELinux policy is loaded.
SELINUX=disabled
# SELINUXTYPE= can take one of three two values:
#     targeted - Targeted processes are protected,
#     minimum - Modification of targeted policy. Only selected processes are protected. 
#     mls - Multi Level Security protection.
SELINUXTYPE=targeted 
[root@jytest2 ~]# vi /etc/selinux/config

# This file controls the state of SELinux on the system.
# SELINUX= can take one of these three values:
#     enforcing - SELinux security policy is enforced.
#     permissive - SELinux prints warnings instead of enforcing.
#     disabled - No SELinux policy is loaded.
SELINUX=disabled
# SELINUXTYPE= can take one of three two values:
#     targeted - Targeted processes are protected,
#     minimum - Modification of targeted policy. Only selected processes are protected. 
#     mls - Multi Level Security protection.
SELINUXTYPE=targeted 

开启nscd

[root@jytest1 /]# rpm -qa | grep nscd
nscd-2.17-78.0.1.el7.x86_64
[root@jytest1 /]# chkconfig nscd on
Note: Forwarding request to 'systemctl enable nscd.service'.
ln -s '/usr/lib/systemd/system/nscd.service' '/etc/systemd/system/multi-user.target.wants/nscd.service'
ln -s '/usr/lib/systemd/system/nscd.socket' '/etc/systemd/system/sockets.target.wants/nscd.socket'
[root@jytest1 /]# chkconfig --level 35 nscd on
Note: Forwarding request to 'systemctl enable nscd.service'.
[root@jytest1 /]# service nscd start
Redirecting to /bin/systemctl start  nscd.service

[root@jytest2 /]# rpm -qa | grep nscd
nscd-2.17-78.0.1.el7.x86_64
[root@jytest2 /]# chkconfig nscd on
Note: Forwarding request to 'systemctl enable nscd.service'.
ln -s '/usr/lib/systemd/system/nscd.service' '/etc/systemd/system/multi-user.target.wants/nscd.service'
ln -s '/usr/lib/systemd/system/nscd.socket' '/etc/systemd/system/sockets.target.wants/nscd.socket'
[root@jytest2 /]# chkconfig --level 35 nscd on
Note: Forwarding request to 'systemctl enable nscd.service'.
[root@jytest2 /]# service nscd start
Redirecting to /bin/systemctl start  nscd.service

修改时间同步方式

[root@jytest1 /]# vi /etc/ntp.conf
# For more information about this file, see the man pages
# ntp.conf(5), ntp_acc(5), ntp_auth(5), ntp_clock(5), ntp_misc(5), ntp_mon(5).

driftfile /var/lib/ntp/drift

# Permit time synchronization with our time source, but do not
# permit the source to query or modify the service on this system.
restrict default nomodify notrap nopeer noquery

# Permit all access over the loopback interface.  This could
# be tightened as well, but to do so would effect some of
# the administrative functions.
restrict 127.0.0.1
restrict ::1

# Hosts on local network are less restricted.
#restrict 192.168.1.0 mask 255.255.255.0 nomodify notrap

# Use public servers from the pool.ntp.org project.
# Please consider joining the pool (http://www.pool.ntp.org/join.html).
#server 0.rhel.pool.ntp.org iburst
#server 1.rhel.pool.ntp.org iburst
#server 2.rhel.pool.ntp.org iburst
#server 3.rhel.pool.ntp.org iburst
server 10.138.130.170 --同步服务器

#server 127.127.1.0
#fudge 127.127.1.0 stratum 10

#broadcast 192.168.1.255 autokey        # broadcast server
#broadcastclient                        # broadcast client
#broadcast 224.0.1.1 autokey            # multicast server
#multicastclient 224.0.1.1              # multicast client
#manycastserver 239.255.254.254         # manycast server
#manycastclient 239.255.254.254 autokey # manycast client

# Enable public key cryptography.
#crypto

includefile /etc/ntp/crypto/pw

# Key file containing the keys and key identifiers used when operating
# with symmetric key cryptography.
keys /etc/ntp/keys
"/etc/ntp.conf" 63L, 2118C written


[root@jytest2 /]# vi /etc/ntp.conf
# For more information about this file, see the man pages
# ntp.conf(5), ntp_acc(5), ntp_auth(5), ntp_clock(5), ntp_misc(5), ntp_mon(5).

driftfile /var/lib/ntp/drift

# Permit time synchronization with our time source, but do not
# permit the source to query or modify the service on this system.
restrict default nomodify notrap nopeer noquery

# Permit all access over the loopback interface.  This could
# be tightened as well, but to do so would effect some of
# the administrative functions.
restrict 127.0.0.1
restrict ::1

# Hosts on local network are less restricted.
#restrict 192.168.1.0 mask 255.255.255.0 nomodify notrap

# Use public servers from the pool.ntp.org project.
# Please consider joining the pool (http://www.pool.ntp.org/join.html).
#server 0.rhel.pool.ntp.org iburst
#server 1.rhel.pool.ntp.org iburst
#server 2.rhel.pool.ntp.org iburst
#server 3.rhel.pool.ntp.org iburst
server 10.138.130.170 --同步服务器

#server 127.127.1.0
#fudge 127.127.1.0 stratum 10

#broadcast 192.168.1.255 autokey        # broadcast server
#broadcastclient                        # broadcast client
#broadcast 224.0.1.1 autokey            # multicast server
#multicastclient 224.0.1.1              # multicast client
#manycastserver 239.255.254.254         # manycast server
#manycastclient 239.255.254.254 autokey # manycast client

# Enable public key cryptography.
#crypto

includefile /etc/ntp/crypto/pw

# Key file containing the keys and key identifiers used when operating
# with symmetric key cryptography.
keys /etc/ntp/keys
"/etc/ntp.conf" 63L, 2118C written


[root@jytest1 /]# vi /etc/sysconfig/ntpd
# Command line options for ntpd
OPTIONS="-x -u ntp:ntp -p /var/run/ntpd.pid -g"

[root@jytest2 /]# vi /etc/sysconfig/ntpd
# Command line options for ntpd
OPTIONS="-x -u ntp:ntp -p /var/run/ntpd.pid -g"


[root@jytest1 /]# chkconfig ntpd on
Note: Forwarding request to 'systemctl enable ntpd.service'.

[root@jytest2 /]# chkconfig ntpd on
Note: Forwarding request to 'systemctl enable ntpd.service'.

[root@jytest1 /]# service ntpd restart
Redirecting to /bin/systemctl restart  ntpd.service

[root@jytest2 /]# service ntpd restart
Redirecting to /bin/systemctl restart  ntpd.service

修改nsswitch.conf,nsswitch.conf 保存的是域名检索顺序。在一些系统配置中,NIS(Network Information System)可能会引起SCAN 域名解析异常。为了避免这种问题的发生,对nsswitch.conf 做如下调整。

[root@jytest1 /]# vi /etc/nsswitch.conf
#
# /etc/nsswitch.conf
#
# An example Name Service Switch config file. This file should be
# sorted with the most-used services at the beginning.
#
# The entry '[NOTFOUND=return]' means that the search for an
# entry should stop if the search in the previous entry turned
# up nothing. Note that if the search failed due to some other reason
# (like no NIS server responding) then the search continues with the
# next entry.
#
# Valid entries include:
#
#       nisplus                 Use NIS+ (NIS version 3)
#       nis                     Use NIS (NIS version 2), also called YP
#       dns                     Use DNS (Domain Name Service)
#       files                   Use the local files
#       db                      Use the local database (.db) files
#       compat                  Use NIS on compat mode
#       hesiod                  Use Hesiod for user lookups
#       [NOTFOUND=return]       Stop searching if not found so far
#

# To use db, put the "db" in front of "files" for entries you want to be
# looked up first in the databases
#
# Example:
#passwd:    db files nisplus nis
#shadow:    db files nisplus nis
#group:     db files nisplus nis

passwd:     files sss
shadow:     files sss
group:      files sss
#initgroups: files

#hosts:     db files nisplus nis dns
hosts:      files dns nis --增加nis参数

# Example - obey only what nisplus tells us...
#services:   nisplus [NOTFOUND=return] files
#networks:   nisplus [NOTFOUND=return] files
#protocols:  nisplus [NOTFOUND=return] files
#rpc:        nisplus [NOTFOUND=return] files
#ethers:     nisplus [NOTFOUND=return] files
#netmasks:   nisplus [NOTFOUND=return] files

bootparams: nisplus [NOTFOUND=return] files

ethers:     files
netmasks:   files
networks:   files
protocols:  files
rpc:        files
services:   files sss

netgroup:   files sss

publickey:  nisplus

automount:  files sss
aliases:    files nisplus


[root@jytest2 /]# vi /etc/nsswitch.conf
#
# /etc/nsswitch.conf
#
# An example Name Service Switch config file. This file should be
# sorted with the most-used services at the beginning.
#
# The entry '[NOTFOUND=return]' means that the search for an
# entry should stop if the search in the previous entry turned
# up nothing. Note that if the search failed due to some other reason
# (like no NIS server responding) then the search continues with the
# next entry.
#
# Valid entries include:
#
#       nisplus                 Use NIS+ (NIS version 3)
#       nis                     Use NIS (NIS version 2), also called YP
#       dns                     Use DNS (Domain Name Service)
#       files                   Use the local files
#       db                      Use the local database (.db) files
#       compat                  Use NIS on compat mode
#       hesiod                  Use Hesiod for user lookups
#       [NOTFOUND=return]       Stop searching if not found so far
#

# To use db, put the "db" in front of "files" for entries you want to be
# looked up first in the databases
#
# Example:
#passwd:    db files nisplus nis
#shadow:    db files nisplus nis
#group:     db files nisplus nis

passwd:     files sss
shadow:     files sss
group:      files sss
#initgroups: files

#hosts:     db files nisplus nis dns
hosts:      files dns nis --增加nis参数


# Example - obey only what nisplus tells us...
#services:   nisplus [NOTFOUND=return] files
#networks:   nisplus [NOTFOUND=return] files
#protocols:  nisplus [NOTFOUND=return] files
#rpc:        nisplus [NOTFOUND=return] files
#ethers:     nisplus [NOTFOUND=return] files
#netmasks:   nisplus [NOTFOUND=return] files

bootparams: nisplus [NOTFOUND=return] files

ethers:     files
netmasks:   files
networks:   files
protocols:  files
rpc:        files
services:   files sss

netgroup:   files sss

publickey:  nisplus

automount:  files sss
aliases:    files nisplus

配置NOZEROCONF

[root@jytest1 /]# vi /etc/sysconfig/network
# Created by anaconda
# Recommended value for NOZEROCONF
NOZEROCONF=yes

[root@jytest2 /]# vi /etc/sysconfig/network
# Created by anaconda
# Recommended value for NOZEROCONF
NOZEROCONF=yes

用vnc远程登录执行安装

[root@jytest1 Desktop]# xhost +
access control disabled, clients can connect from any host
[root@jytest1 Desktop]# xdpyinfo
name of display:    :3
version number:    11.0
vendor string:    The X.Org Foundation
vendor release number:    11500000
X.Org version: 1.15.0
maximum request size:  16777212 bytes
motion buffer size:  256
bitmap unit, bit order, padding:    32, LSBFirst, 32
image byte order:    LSBFirst
number of supported pixmap formats:    6
...省略...只需要记录name of display:的值
[root@jytest1 Desktop]# su - grid
Last login: Wed Mar 15 19:01:47 CST 2017 on pts/1
[grid@jytest1 ~]$ export DISPLAY=:3
[grid@jytest1 ~]$ cd $ORACLE_HOME
[grid@jytest1 crs]$ ./gridSetup.sh

以root用户分别在两个节点上执行以下脚本,先在主节点执行。

root@jytest1 /]# ./u01/app/oraInventory/orainstRoot.sh
Changing permissions of /u01/app/oraInventory.
Adding read,write permissions for group.
Removing read,write,execute permissions for world.

Changing groupname of /u01/app/oraInventory to oinstall.
The execution of the script is complete.

[root@jytest2 /]# ./u01/app/oraInventory/orainstRoot.sh
Changing permissions of /u01/app/oraInventory.
Adding read,write permissions for group.
Removing read,write,execute permissions for world.

Changing groupname of /u01/app/oraInventory to oinstall.
The execution of the script is complete.

[root@jytest1 /]# ./u01/app/product/12.2.0/crs/root.sh
Performing root user operation.

The following environment variables are set as:
ORACLE_OWNER= grid
ORACLE_HOME= /u01/app/product/12.2.0/crs

Enter the full pathname of the local bin directory: [/usr/local/bin]:
Copying dbhome to /usr/local/bin ...
Copying oraenv to /usr/local/bin ...
Copying coraenv to /usr/local/bin ...

Creating /etc/oratab file...
Entries will be added to the /etc/oratab file as needed by
Database Configuration Assistant when a database is created
Finished running generic part of root script.
Now product-specific root actions will be performed.
Relinking oracle with rac_on option
Using configuration parameter file: /u01/app/product/12.2.0/crs/crs/install/crsconfig_params
The log of current session can be found at:
/u01/app/grid/crsdata/jytest1/crsconfig/rootcrs_jytest1_2017-03-20_06-07-00PM.log
2017/03/20 18:07:22 CLSRSC-594: Executing installation step 1 of 19: 'SetupTFA'.
2017/03/20 18:07:22 CLSRSC-4001: Installing Oracle Trace File Analyzer (TFA) Collector.
2017/03/20 18:08:15 CLSRSC-4002: Successfully installed Oracle Trace File Analyzer (TFA) Collector.
2017/03/20 18:08:15 CLSRSC-594: Executing installation step 2 of 19: 'ValidateEnv'.
2017/03/20 18:08:21 CLSRSC-363: User ignored prerequisites during installation
2017/03/20 18:08:22 CLSRSC-594: Executing installation step 3 of 19: 'CheckFirstNode'.
2017/03/20 18:08:24 CLSRSC-594: Executing installation step 4 of 19: 'GenSiteGUIDs'.
2017/03/20 18:08:26 CLSRSC-594: Executing installation step 5 of 19: 'SaveParamFile'.
2017/03/20 18:08:36 CLSRSC-594: Executing installation step 6 of 19: 'SetupOSD'.
2017/03/20 18:08:38 CLSRSC-594: Executing installation step 7 of 19: 'CheckCRSConfig'.
2017/03/20 18:08:38 CLSRSC-594: Executing installation step 8 of 19: 'SetupLocalGPNP'.
2017/03/20 18:09:12 CLSRSC-594: Executing installation step 9 of 19: 'ConfigOLR'.
2017/03/20 18:09:26 CLSRSC-594: Executing installation step 10 of 19: 'ConfigCHMOS'.
2017/03/20 18:09:26 CLSRSC-594: Executing installation step 11 of 19: 'CreateOHASD'.
2017/03/20 18:09:34 CLSRSC-594: Executing installation step 12 of 19: 'ConfigOHASD'.
2017/03/20 18:09:49 CLSRSC-330: Adding Clusterware entries to file 'oracle-ohasd.service'
2017/03/20 18:10:46 CLSRSC-594: Executing installation step 13 of 19: 'InstallAFD'.
2017/03/20 18:11:55 CLSRSC-594: Executing installation step 14 of 19: 'InstallACFS'.
CRS-2791: Starting shutdown of Oracle High Availability Services-managed resources on 'jytest1'
CRS-2793: Shutdown of Oracle High Availability Services-managed resources on 'jytest1' has completed
CRS-4133: Oracle High Availability Services has been stopped.
CRS-4123: Oracle High Availability Services has been started.
2017/03/20 18:12:30 CLSRSC-594: Executing installation step 15 of 19: 'InstallKA'.
2017/03/20 18:12:38 CLSRSC-594: Executing installation step 16 of 19: 'InitConfig'.
CRS-2791: Starting shutdown of Oracle High Availability Services-managed resources on 'jytest1'
CRS-2793: Shutdown of Oracle High Availability Services-managed resources on 'jytest1' has completed
CRS-4133: Oracle High Availability Services has been stopped.
CRS-4123: Oracle High Availability Services has been started.
CRS-2672: Attempting to start 'ora.driver.afd' on 'jytest1'
CRS-2672: Attempting to start 'ora.evmd' on 'jytest1'
CRS-2672: Attempting to start 'ora.mdnsd' on 'jytest1'
CRS-2676: Start of 'ora.driver.afd' on 'jytest1' succeeded
CRS-2672: Attempting to start 'ora.cssdmonitor' on 'jytest1'
CRS-2676: Start of 'ora.mdnsd' on 'jytest1' succeeded
CRS-2676: Start of 'ora.evmd' on 'jytest1' succeeded
CRS-2672: Attempting to start 'ora.gpnpd' on 'jytest1'
CRS-2676: Start of 'ora.cssdmonitor' on 'jytest1' succeeded
CRS-2676: Start of 'ora.gpnpd' on 'jytest1' succeeded
CRS-2672: Attempting to start 'ora.gipcd' on 'jytest1'
CRS-2676: Start of 'ora.gipcd' on 'jytest1' succeeded
CRS-2672: Attempting to start 'ora.cssd' on 'jytest1'
CRS-2672: Attempting to start 'ora.diskmon' on 'jytest1'
CRS-2676: Start of 'ora.diskmon' on 'jytest1' succeeded
CRS-2676: Start of 'ora.cssd' on 'jytest1' succeeded

Disk label(s) created successfully. Check /u01/app/grid/cfgtoollogs/asmca/asmca-170320PM061328.log for details.
Disk groups created successfully. Check /u01/app/grid/cfgtoollogs/asmca/asmca-170320PM061328.log for details.

2017/03/20 18:15:29 CLSRSC-482: Running command: '/u01/app/product/12.2.0/crs/bin/ocrconfig -upgrade grid oinstall'
CRS-2672: Attempting to start 'ora.crf' on 'jytest1'
CRS-2672: Attempting to start 'ora.storage' on 'jytest1'
CRS-2676: Start of 'ora.storage' on 'jytest1' succeeded
CRS-2676: Start of 'ora.crf' on 'jytest1' succeeded
CRS-2672: Attempting to start 'ora.crsd' on 'jytest1'
CRS-2676: Start of 'ora.crsd' on 'jytest1' succeeded
CRS-4256: Updating the profile
Successful addition of voting disk b62d970610de4fa2bf2194b4ef533c34.
Successfully replaced voting disk group with +CRS.
CRS-4256: Updating the profile
CRS-4266: Voting file(s) successfully replaced
## STATE File Universal Id File Name Disk group
-- ----- ----------------- --------- ---------
1. ONLINE b62d970610de4fa2bf2194b4ef533c34 (AFD:CRS1) [CRS]
Located 1 voting disk(s).
CRS-2791: Starting shutdown of Oracle High Availability Services-managed resources on 'jytest1'
CRS-2673: Attempting to stop 'ora.crsd' on 'jytest1'
CRS-2677: Stop of 'ora.crsd' on 'jytest1' succeeded
CRS-2673: Attempting to stop 'ora.storage' on 'jytest1'
CRS-2673: Attempting to stop 'ora.crf' on 'jytest1'
CRS-2673: Attempting to stop 'ora.drivers.acfs' on 'jytest1'
CRS-2673: Attempting to stop 'ora.gpnpd' on 'jytest1'
CRS-2673: Attempting to stop 'ora.mdnsd' on 'jytest1'
CRS-2677: Stop of 'ora.drivers.acfs' on 'jytest1' succeeded
CRS-2677: Stop of 'ora.crf' on 'jytest1' succeeded
CRS-2677: Stop of 'ora.gpnpd' on 'jytest1' succeeded
CRS-2677: Stop of 'ora.storage' on 'jytest1' succeeded
CRS-2673: Attempting to stop 'ora.asm' on 'jytest1'
CRS-2677: Stop of 'ora.mdnsd' on 'jytest1' succeeded
CRS-2677: Stop of 'ora.asm' on 'jytest1' succeeded
CRS-2673: Attempting to stop 'ora.cluster_interconnect.haip' on 'jytest1'
CRS-2677: Stop of 'ora.cluster_interconnect.haip' on 'jytest1' succeeded
CRS-2673: Attempting to stop 'ora.ctssd' on 'jytest1'
CRS-2673: Attempting to stop 'ora.evmd' on 'jytest1'
CRS-2677: Stop of 'ora.ctssd' on 'jytest1' succeeded
CRS-2677: Stop of 'ora.evmd' on 'jytest1' succeeded
CRS-2673: Attempting to stop 'ora.cssd' on 'jytest1'
CRS-2677: Stop of 'ora.cssd' on 'jytest1' succeeded
CRS-2673: Attempting to stop 'ora.driver.afd' on 'jytest1'
CRS-2673: Attempting to stop 'ora.gipcd' on 'jytest1'
CRS-2677: Stop of 'ora.driver.afd' on 'jytest1' succeeded
CRS-2677: Stop of 'ora.gipcd' on 'jytest1' succeeded
CRS-2793: Shutdown of Oracle High Availability Services-managed resources on 'jytest1' has completed
CRS-4133: Oracle High Availability Services has been stopped.
2017/03/20 18:16:55 CLSRSC-594: Executing installation step 17 of 19: 'StartCluster'.
CRS-4123: Starting Oracle High Availability Services-managed resources
CRS-2672: Attempting to start 'ora.mdnsd' on 'jytest1'
CRS-2672: Attempting to start 'ora.evmd' on 'jytest1'
CRS-2676: Start of 'ora.mdnsd' on 'jytest1' succeeded
CRS-2676: Start of 'ora.evmd' on 'jytest1' succeeded
CRS-2672: Attempting to start 'ora.gpnpd' on 'jytest1'
CRS-2676: Start of 'ora.gpnpd' on 'jytest1' succeeded
CRS-2672: Attempting to start 'ora.gipcd' on 'jytest1'
CRS-2676: Start of 'ora.gipcd' on 'jytest1' succeeded
CRS-2672: Attempting to start 'ora.cssdmonitor' on 'jytest1'
CRS-2676: Start of 'ora.cssdmonitor' on 'jytest1' succeeded
CRS-2672: Attempting to start 'ora.cssd' on 'jytest1'
CRS-2672: Attempting to start 'ora.diskmon' on 'jytest1'
CRS-2676: Start of 'ora.diskmon' on 'jytest1' succeeded
CRS-2676: Start of 'ora.cssd' on 'jytest1' succeeded
CRS-2672: Attempting to start 'ora.cluster_interconnect.haip' on 'jytest1'
CRS-2672: Attempting to start 'ora.ctssd' on 'jytest1'
CRS-2676: Start of 'ora.ctssd' on 'jytest1' succeeded
CRS-2676: Start of 'ora.cluster_interconnect.haip' on 'jytest1' succeeded
CRS-2672: Attempting to start 'ora.asm' on 'jytest1'
CRS-2676: Start of 'ora.asm' on 'jytest1' succeeded
CRS-2672: Attempting to start 'ora.storage' on 'jytest1'
CRS-2676: Start of 'ora.storage' on 'jytest1' succeeded
CRS-2672: Attempting to start 'ora.crf' on 'jytest1'
CRS-2676: Start of 'ora.crf' on 'jytest1' succeeded
CRS-2672: Attempting to start 'ora.crsd' on 'jytest1'
CRS-2676: Start of 'ora.crsd' on 'jytest1' succeeded
CRS-6023: Starting Oracle Cluster Ready Services-managed resources
CRS-6017: Processing resource auto-start for servers: jytest1
CRS-6016: Resource auto-start has completed for server jytest1
CRS-6024: Completed start of Oracle Cluster Ready Services-managed resources
CRS-4123: Oracle High Availability Services has been started.
2017/03/20 18:18:43 CLSRSC-343: Successfully started Oracle Clusterware stack
2017/03/20 18:18:43 CLSRSC-594: Executing installation step 18 of 19: 'ConfigNode'.
CRS-2672: Attempting to start 'ora.ASMNET1LSNR_ASM.lsnr' on 'jytest1'
CRS-2676: Start of 'ora.ASMNET1LSNR_ASM.lsnr' on 'jytest1' succeeded
CRS-2672: Attempting to start 'ora.asm' on 'jytest1'
CRS-2676: Start of 'ora.asm' on 'jytest1' succeeded
CRS-2672: Attempting to start 'ora.CRS.dg' on 'jytest1'
CRS-2676: Start of 'ora.CRS.dg' on 'jytest1' succeeded
2017/03/20 18:21:45 CLSRSC-594: Executing installation step 19 of 19: 'PostConfig'.
2017/03/20 18:22:19 CLSRSC-325: Configure Oracle Grid Infrastructure for a Cluster ... succeeded
[root@jytest2 /]# ./u01/app/product/12.2.0/crs/root.sh
Performing root user operation.

The following environment variables are set as:
ORACLE_OWNER= grid
ORACLE_HOME= /u01/app/product/12.2.0/crs

Enter the full pathname of the local bin directory: [/usr/local/bin]:
Copying dbhome to /usr/local/bin ...
Copying oraenv to /usr/local/bin ...
Copying coraenv to /usr/local/bin ...

Creating /etc/oratab file...
Entries will be added to the /etc/oratab file as needed by
Database Configuration Assistant when a database is created
Finished running generic part of root script.
Now product-specific root actions will be performed.
Relinking oracle with rac_on option
Using configuration parameter file: /u01/app/product/12.2.0/crs/crs/install/crsconfig_params
The log of current session can be found at:
/u01/app/grid/crsdata/jytest2/crsconfig/rootcrs_jytest2_2017-03-20_06-28-04PM.log
2017/03/20 18:28:30 CLSRSC-594: Executing installation step 1 of 19: 'SetupTFA'.
2017/03/20 18:28:31 CLSRSC-4001: Installing Oracle Trace File Analyzer (TFA) Collector.
2017/03/20 18:29:23 CLSRSC-4002: Successfully installed Oracle Trace File Analyzer (TFA) Collector.
2017/03/20 18:29:23 CLSRSC-594: Executing installation step 2 of 19: 'ValidateEnv'.
2017/03/20 18:29:25 CLSRSC-363: User ignored prerequisites during installation
2017/03/20 18:29:25 CLSRSC-594: Executing installation step 3 of 19: 'CheckFirstNode'.
2017/03/20 18:29:27 CLSRSC-594: Executing installation step 4 of 19: 'GenSiteGUIDs'.
2017/03/20 18:29:27 CLSRSC-594: Executing installation step 5 of 19: 'SaveParamFile'.
2017/03/20 18:29:30 CLSRSC-594: Executing installation step 6 of 19: 'SetupOSD'.
2017/03/20 18:29:33 CLSRSC-594: Executing installation step 7 of 19: 'CheckCRSConfig'.
2017/03/20 18:29:33 CLSRSC-594: Executing installation step 8 of 19: 'SetupLocalGPNP'.
2017/03/20 18:29:35 CLSRSC-594: Executing installation step 9 of 19: 'ConfigOLR'.
2017/03/20 18:29:42 CLSRSC-594: Executing installation step 10 of 19: 'ConfigCHMOS'.
2017/03/20 18:29:42 CLSRSC-594: Executing installation step 11 of 19: 'CreateOHASD'.
2017/03/20 18:29:44 CLSRSC-594: Executing installation step 12 of 19: 'ConfigOHASD'.
2017/03/20 18:30:00 CLSRSC-330: Adding Clusterware entries to file 'oracle-ohasd.service'
2017/03/20 18:30:51 CLSRSC-594: Executing installation step 13 of 19: 'InstallAFD'.
2017/03/20 18:32:00 CLSRSC-594: Executing installation step 14 of 19: 'InstallACFS'.
CRS-2791: Starting shutdown of Oracle High Availability Services-managed resources on 'jytest2'
CRS-2793: Shutdown of Oracle High Availability Services-managed resources on 'jytest2' has completed
CRS-4133: Oracle High Availability Services has been stopped.
CRS-4123: Oracle High Availability Services has been started.
2017/03/20 18:32:29 CLSRSC-594: Executing installation step 15 of 19: 'InstallKA'.
2017/03/20 18:32:31 CLSRSC-594: Executing installation step 16 of 19: 'InitConfig'.
CRS-2791: Starting shutdown of Oracle High Availability Services-managed resources on 'jytest2'
CRS-2793: Shutdown of Oracle High Availability Services-managed resources on 'jytest2' has completed
CRS-4133: Oracle High Availability Services has been stopped.
CRS-4123: Oracle High Availability Services has been started.
CRS-2791: Starting shutdown of Oracle High Availability Services-managed resources on 'jytest2'
CRS-2673: Attempting to stop 'ora.drivers.acfs' on 'jytest2'
CRS-2677: Stop of 'ora.drivers.acfs' on 'jytest2' succeeded
CRS-2793: Shutdown of Oracle High Availability Services-managed resources on 'jytest2' has completed
CRS-4133: Oracle High Availability Services has been stopped.
2017/03/20 18:32:59 CLSRSC-594: Executing installation step 17 of 19: 'StartCluster'.
CRS-4123: Starting Oracle High Availability Services-managed resources
CRS-2672: Attempting to start 'ora.mdnsd' on 'jytest2'
CRS-2672: Attempting to start 'ora.evmd' on 'jytest2'
CRS-2676: Start of 'ora.mdnsd' on 'jytest2' succeeded
CRS-2676: Start of 'ora.evmd' on 'jytest2' succeeded
CRS-2672: Attempting to start 'ora.gpnpd' on 'jytest2'
CRS-2676: Start of 'ora.gpnpd' on 'jytest2' succeeded
CRS-2672: Attempting to start 'ora.gipcd' on 'jytest2'
CRS-2676: Start of 'ora.gipcd' on 'jytest2' succeeded
CRS-2672: Attempting to start 'ora.cssdmonitor' on 'jytest2'
CRS-2676: Start of 'ora.cssdmonitor' on 'jytest2' succeeded
CRS-2672: Attempting to start 'ora.cssd' on 'jytest2'
CRS-2672: Attempting to start 'ora.diskmon' on 'jytest2'
CRS-2676: Start of 'ora.diskmon' on 'jytest2' succeeded
CRS-2676: Start of 'ora.cssd' on 'jytest2' succeeded
CRS-2672: Attempting to start 'ora.cluster_interconnect.haip' on 'jytest2'
CRS-2672: Attempting to start 'ora.ctssd' on 'jytest2'
CRS-2676: Start of 'ora.ctssd' on 'jytest2' succeeded
CRS-2676: Start of 'ora.cluster_interconnect.haip' on 'jytest2' succeeded
CRS-2672: Attempting to start 'ora.asm' on 'jytest2'
CRS-2676: Start of 'ora.asm' on 'jytest2' succeeded
CRS-2672: Attempting to start 'ora.storage' on 'jytest2'
CRS-2676: Start of 'ora.storage' on 'jytest2' succeeded
CRS-2672: Attempting to start 'ora.crf' on 'jytest2'
CRS-2676: Start of 'ora.crf' on 'jytest2' succeeded
CRS-2672: Attempting to start 'ora.crsd' on 'jytest2'
CRS-2676: Start of 'ora.crsd' on 'jytest2' succeeded
CRS-6017: Processing resource auto-start for servers: jytest2
CRS-2673: Attempting to stop 'ora.LISTENER_SCAN1.lsnr' on 'jytest1'
CRS-2672: Attempting to start 'ora.net1.network' on 'jytest2'
CRS-2672: Attempting to start 'ora.ASMNET1LSNR_ASM.lsnr' on 'jytest2'
CRS-2677: Stop of 'ora.LISTENER_SCAN1.lsnr' on 'jytest1' succeeded
CRS-2676: Start of 'ora.net1.network' on 'jytest2' succeeded
CRS-2673: Attempting to stop 'ora.scan1.vip' on 'jytest1'
CRS-2672: Attempting to start 'ora.ons' on 'jytest2'
CRS-2677: Stop of 'ora.scan1.vip' on 'jytest1' succeeded
CRS-2672: Attempting to start 'ora.scan1.vip' on 'jytest2'
CRS-2676: Start of 'ora.scan1.vip' on 'jytest2' succeeded
CRS-2672: Attempting to start 'ora.LISTENER_SCAN1.lsnr' on 'jytest2'
CRS-2676: Start of 'ora.ons' on 'jytest2' succeeded
CRS-2676: Start of 'ora.ASMNET1LSNR_ASM.lsnr' on 'jytest2' succeeded
CRS-2672: Attempting to start 'ora.asm' on 'jytest2'
CRS-2676: Start of 'ora.LISTENER_SCAN1.lsnr' on 'jytest2' succeeded
CRS-2676: Start of 'ora.asm' on 'jytest2' succeeded
CRS-6016: Resource auto-start has completed for server jytest2
CRS-6024: Completed start of Oracle Cluster Ready Services-managed resources
CRS-4123: Oracle High Availability Services has been started.
2017/03/20 18:36:14 CLSRSC-343: Successfully started Oracle Clusterware stack
2017/03/20 18:36:14 CLSRSC-594: Executing installation step 18 of 19: 'ConfigNode'.
2017/03/20 18:36:32 CLSRSC-594: Executing installation step 19 of 19: 'PostConfig'.
2017/03/20 18:37:14 CLSRSC-325: Configure Oracle Grid Infrastructure for a Cluster ... succeeded

检查集群信息

[grid@jytest1 ~]$ crsctl stat res -t
--------------------------------------------------------------------------------
Name           Target  State        Server                   State details       
--------------------------------------------------------------------------------
Local Resources
--------------------------------------------------------------------------------
ora.ASMNET1LSNR_ASM.lsnr
               ONLINE  ONLINE       jytest1                  STABLE
               ONLINE  ONLINE       jytest2                  STABLE
ora.CRS.dg
               ONLINE  ONLINE       jytest1                  STABLE
               ONLINE  ONLINE       jytest2                  STABLE
ora.LISTENER.lsnr
               ONLINE  ONLINE       jytest1                  STABLE
               ONLINE  ONLINE       jytest2                  STABLE
ora.chad
               ONLINE  ONLINE       jytest1                  STABLE
               ONLINE  ONLINE       jytest2                  STABLE
ora.net1.network
               ONLINE  ONLINE       jytest1                  STABLE
               ONLINE  ONLINE       jytest2                  STABLE
ora.ons
               ONLINE  ONLINE       jytest1                  STABLE
               ONLINE  ONLINE       jytest2                  STABLE
ora.proxy_advm
               OFFLINE OFFLINE      jytest1                  STABLE
               OFFLINE OFFLINE      jytest2                  STABLE
--------------------------------------------------------------------------------
Cluster Resources
--------------------------------------------------------------------------------
ora.LISTENER_SCAN1.lsnr
      1        ONLINE  ONLINE       jytest2                  STABLE
ora.LISTENER_SCAN2.lsnr
      1        ONLINE  ONLINE       jytest1                  STABLE
ora.LISTENER_SCAN3.lsnr
      1        ONLINE  ONLINE       jytest1                  STABLE
ora.MGMTLSNR
      1        ONLINE  ONLINE       jytest1                  169.254.123.145 88.8
                                                             8.88.1,STABLE
ora.asm
      1        ONLINE  ONLINE       jytest1                  Started,STABLE
      2        ONLINE  ONLINE       jytest2                  Started,STABLE
      3        OFFLINE OFFLINE                               STABLE
ora.cvu
      1        ONLINE  ONLINE       jytest1                  STABLE
ora.jytest1.vip
      1        ONLINE  ONLINE       jytest1                  STABLE
ora.jytest2.vip
      1        ONLINE  ONLINE       jytest2                  STABLE
ora.mgmtdb
      1        ONLINE  ONLINE       jytest1                  Open,STABLE
ora.qosmserver
      1        ONLINE  ONLINE       jytest1                  STABLE
ora.scan1.vip
      1        ONLINE  ONLINE       jytest2                  STABLE
ora.scan2.vip
      1        ONLINE  ONLINE       jytest1                  STABLE
ora.scan3.vip
      1        ONLINE  ONLINE       jytest1                  STABLE
--------------------------------------------------------------------------------

安装数据库软件

[root@jytest1 Desktop]# su - oracle
Last login: Mon Mar 20 12:13:17 CST 2017 on pts/0
[oracle@jytest1 ~]$ cd /soft/database
[oracle@jytest1 database]$ export DISPLAY=:1

[oracle@jytest1 database]$ ./runInstaller

以 root用户在所有节点上执行以下脚本,先在主节点执行

[root@jytest1 /]# ./u01/app/oracle/product/12.2.0/db/root.sh
Performing root user operation.

The following environment variables are set as:
ORACLE_OWNER= oracle
ORACLE_HOME= /u01/app/oracle/product/12.2.0/db

Enter the full pathname of the local bin directory: [/usr/local/bin]:
The contents of "dbhome" have not changed. No need to overwrite.
The contents of "oraenv" have not changed. No need to overwrite.
The contents of "coraenv" have not changed. No need to overwrite.

Entries will be added to the /etc/oratab file as needed by
Database Configuration Assistant when a database is created
Finished running generic part of root script.
Now product-specific root actions will be performed.

[root@jytest2 /]# ./u01/app/oracle/product/12.2.0/db/root.sh
Performing root user operation.

The following environment variables are set as:
ORACLE_OWNER= oracle
ORACLE_HOME= /u01/app/oracle/product/12.2.0/db

Enter the full pathname of the local bin directory: [/usr/local/bin]:
The contents of "dbhome" have not changed. No need to overwrite.
The contents of "oraenv" have not changed. No need to overwrite.
The contents of "coraenv" have not changed. No need to overwrite.

Entries will be added to the /etc/oratab file as needed by
Database Configuration Assistant when a database is created
Finished running generic part of root script.
Now product-specific root actions will be performed.

创建磁盘组

[grid@jytest1 ~]$ export DISPLAY=:1
[grid@jytest1 ~]$ asmca

创建存储数据库文件的磁盘组DATA,TEST


创建数据库jy

[oracle@jytest1 database]$ dbca

检查数据库配置信息

[grid@jytest1 ~]$ srvctl config database -d jy
Database unique name: jy
Database name: jy
Oracle home: /u01/app/oracle/product/12.2.0/db
Oracle user: oracle
Spfile: +DATA/JY/PARAMETERFILE/spfile.272.939166623
Password file: +DATA/JY/PASSWORD/pwdjy.256.939165949
Domain:
Start options: open
Stop options: immediate
Database role: PRIMARY
Management policy: AUTOMATIC
Server pools:
Disk Groups: DATA
Mount point paths:
Services:
Type: RAC
Start concurrency:
Stop concurrency:
OSDBA group: dba
OSOPER group: oper
Database instances: jy1,jy2
Configured nodes: jytest1,jytest2
CSS critical: no
CPU count: 0
Memory target: 0
Maximum memory: 0
Default network number for database services:
Database is administrator managed
[grid@jytest1 ~]$ srvctl status database -d jy
Instance jy1 is running on node jytest1
Instance jy2 is running on node jytest2

检查集群信息

[grid@jytest1 ~]$ crsctl stat res -t
--------------------------------------------------------------------------------
Name           Target  State        Server                   State details       
--------------------------------------------------------------------------------
Local Resources
--------------------------------------------------------------------------------
ora.ASMNET1LSNR_ASM.lsnr
               ONLINE  ONLINE       jytest1                  STABLE
               ONLINE  ONLINE       jytest2                  STABLE
ora.CRS.dg
               ONLINE  ONLINE       jytest1                  STABLE
               ONLINE  ONLINE       jytest2                  STABLE
ora.DATA.dg
               ONLINE  ONLINE       jytest1                  STABLE
               ONLINE  ONLINE       jytest2                  STABLE
ora.LISTENER.lsnr
               ONLINE  ONLINE       jytest1                  STABLE
               ONLINE  ONLINE       jytest2                  STABLE
ora.TEST.dg
               ONLINE  ONLINE       jytest1                  STABLE
               ONLINE  ONLINE       jytest2                  STABLE
ora.chad
               ONLINE  ONLINE       jytest1                  STABLE
               ONLINE  ONLINE       jytest2                  STABLE
ora.net1.network
               ONLINE  ONLINE       jytest1                  STABLE
               ONLINE  ONLINE       jytest2                  STABLE
ora.ons
               ONLINE  ONLINE       jytest1                  STABLE
               ONLINE  ONLINE       jytest2                  STABLE
ora.proxy_advm
               OFFLINE OFFLINE      jytest1                  STABLE
               OFFLINE OFFLINE      jytest2                  STABLE
--------------------------------------------------------------------------------
Cluster Resources
--------------------------------------------------------------------------------
ora.LISTENER_SCAN1.lsnr
      1        ONLINE  ONLINE       jytest2                  STABLE
ora.LISTENER_SCAN2.lsnr
      1        ONLINE  ONLINE       jytest1                  STABLE
ora.LISTENER_SCAN3.lsnr
      1        ONLINE  ONLINE       jytest1                  STABLE
ora.MGMTLSNR
      1        ONLINE  ONLINE       jytest1                  169.254.123.145 88.8
                                                             8.88.1,STABLE
ora.asm
      1        ONLINE  ONLINE       jytest1                  Started,STABLE
      2        ONLINE  ONLINE       jytest2                  Started,STABLE
      3        OFFLINE OFFLINE                               STABLE
ora.cvu
      1        ONLINE  ONLINE       jytest1                  STABLE
ora.jy.db
      1        ONLINE  ONLINE       jytest1                  Open,HOME=/u01/app/o
                                                             racle/product/12.2.0
                                                             /db,STABLE
      2        ONLINE  ONLINE       jytest2                  Open,HOME=/u01/app/o
                                                             racle/product/12.2.0
                                                             /db,STABLE
ora.jytest1.vip
      1        ONLINE  ONLINE       jytest1                  STABLE
ora.jytest2.vip
      1        ONLINE  ONLINE       jytest2                  STABLE
ora.mgmtdb
      1        ONLINE  ONLINE       jytest1                  Open,STABLE
ora.qosmserver
      1        ONLINE  ONLINE       jytest1                  STABLE
ora.scan1.vip
      1        ONLINE  ONLINE       jytest2                  STABLE
ora.scan2.vip
      1        ONLINE  ONLINE       jytest1                  STABLE
ora.scan3.vip
      1        ONLINE  ONLINE       jytest1                  STABLE
--------------------------------------------------------------------------------

到此12CR2 RAC for Oracle Linux 7.1的安装完成!

One thought on “Oracle Database 12c Release 2 (12.2) RAC On Oracle Linux 7 Using VMware

  1. Τhanks ffor posting ***220;Oracle Database 12c Release 2 (12.2) RAC On Oracle Linux 7 Using VMware ***211; 但行好事 莫问前程***221; ***211; wasn***217;t a wastge of time!

Leave a Reply

Your email address will not be published.

Proudly powered by WordPress | Indrajeet by Sus Hill.