This post documents how to install a 2-node Oracle RAC cluster with Oracle 12.1.0.2 SE2 running on Oracle Linux 6.7 (OL6) with VirtualBox (VBOX) 5.0.4 on Windows 8.1. Using VirtualBox 5.0.4 implies in my lab that Linux virtual machines are no more able to use NAT IP addresses to connect to Internet. Root cause is not clear: it seems network routes are missing on Windows 8.1 host machine - but creating manually the missing routes does not help.
Part 1 details Linux installation and configuration and part 2 details Oracle software installation (Grid Infrastructure, Oracle Database) and database creation.
Disclaimer: this is only an example that can be used to setup a lab or test system: it is not designed to be used for production purpose.
Each RAC node must:
The cluster should have also 3 differents virtual IP adresses (VIP) to be used only by the SCAN listener: a single name should be resolved by 3 differents adresses.
The following table gives the SCAN network configuration used for this installation:
| SCAN | VIP 1 | VIP 2 | VIP 3 |
|---|---|---|---|
| ol6toc-scan | 192.168.56.45 | 192.168.56.46 | 192.168.56.47 |
A 2-node Oracle RAC cluster needs actually 3 machines (if you dedicate DNS server to a different virtual machine (VM)):
One of the very first thing to do is to choose hostnames and network IP addresses.
The following table gives the cluster network configuration needed for this Oracle installation:
| ol6dns01: DNS server | ol6tocn1: RAC node 1 | ol6tocn2: RAC node 2 | network interface | |
|---|---|---|---|---|
| Public IP | 192.168.56.13 | 192.168.56.41 | 192.168.56.42 | eth0 |
| Private IP | N/A | 192.168.43.41 | 192.168.43.42 | eth1 |
| Virtual IP (VIP) | N/A | 192.168.56.31 | 192.168.56.32 | assigned by OUI |
Note that Oracle Universal Installer (OUI) will assign network interface for cluster node virtual ip address during Grid Infrastructure installation.
The main steps of this system/network/storage installation and configuration part are the following:
1. configure DNS server machine
2. configure first cluster node with shared storage
3. clone first cluster node to second cluster node
4. run Cluster Verification Utility (CLUVFY) to make sure both cluster nodes are ready for Oracle sofware installation.
All commands with '#" prompt must be entered with root user account while all command with '$' prompt must be entered with oracle user account.
I have created first VBOX VM with following configuration:
When installing OL6 I have chosen in following order:
1. English as language installation
2. French as appropriate keyboard for the system
3. Basic Storage Devices as installation devices
4. ol6dns01.localdomain as computer name
5. Europe/Paris as time zone
6. root password
7. Use All Space as installation type
8. Basic Server as default installation type
9. bind-9.8.2-0.37.rc1.el6.x86_64 as optional package after clicking on Customize Now, selecting Servers and selecting Network Infrastrcucture Server
After OL6 installation and VM reboot, I have used system console to configure public network interface with /etc/syconfig/network-scripts/ifcfg-eth0 editing:
# cat /etc/sysconfig/network-scripts/ifcfg-eth0 DEVICE=eth0 HWADDR=08:00:27:D7:32:3D TYPE=Ethernet IPADDR=192.168.56.13 # ifup eth0 Determing if ip address 192.168.56.13 is already in use for device eth0... #
Now I can connect to ol6dns01 using SSH (Putty from Windows host) and check network configuration:
# ip addr
1: lo: mtu 65536 qdisc noqueue state UNKNOWN
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: eth0: mtu 1500 qdisc pfifo_fast state UP qlen 1000
link/ether 08:00:27:d7:32:3d brd ff:ff:ff:ff:ff:ff
inet 192.168.56.13/24 brd 192.168.56.255 scope global eth0
inet6 fe80::a00:27ff:fed7:323d/64 scope link
valid_lft forever preferred_lft forever
This OL6 installation is using about 2 GB of disk space:
# df -k
Filesystem 1K-blocks Used Available Use% Mounted on
/dev/mapper/vg_ol6dns01-lv_root
10508540 2088364 7863328 21% /
tmpfs 509128 0 509128 0% /dev/shm
/dev/sda1 487652 73043 384913 16% /boot
DNS server is not configured by default:
# chkconfig --list named named 0:off 1:off 2:off 3:off 4:off 5:off 6:off #
Here is my /etc/named.conf:
# cat /etc/named.conf
//
// named.conf
//
// Provided by Red Hat bind package to configure the ISC BIND named(8) DNS
// server as a caching only nameserver (as a localhost DNS resolver only).
//
// See /usr/share/doc/bind*/sample/ for example named configuration files.
//
options {
listen-on port 53 { 127.0.0.1;192.168.56.13; };
listen-on-v6 port 53 { ::1; };
directory "/var/named";
dump-file "/var/named/data/cache_dump.db";
statistics-file "/var/named/data/named_stats.txt";
memstatistics-file "/var/named/data/named_mem_stats.txt";
allow-query { localhost; };
recursion yes;
dnssec-enable yes;
dnssec-validation yes;
dnssec-lookaside auto;
/* Path to ISC DLV key */
bindkeys-file "/etc/named.iscdlv.key";
managed-keys-directory "/var/named/dynamic";
};
logging {
channel default_debug {
file "data/named.run";
severity dynamic;
};
};
zone "." IN {
type hint;
file "named.ca";
};
include "/etc/named.rfc1912.zones";
include "/etc/named.root.key";
zone "localdomain." IN {
type master;
file "localdomain.zone";
allow-update { none; };
allow-query { any; };
};
zone "56.168.192.in-addr.arpa." IN {
type master;
file "56.168.192.in-addr.arpa";
allow-update { none; };
};
#
Note that I have added the following sections to default /etc/named.conf for localdomain.com:
zone "localdomain." IN {
type master;
file "localdomain.zone";
allow-update { none; };
allow-query { any; };
};
zone "56.168.192.in-addr.arpa." IN {
type master;
file "56.168.192.in-addr.arpa";
allow-update { none; };
};
and I have added VM IP 192.168.56.13 address to:
listen-on port 53 { 127.0.0.1;192.168.56.13; };
I have created the file /var/named/localdomain.zone which contains:
# cat /var/named/localdomain.zone
$TTL 86400
@ IN SOA localhost root.localhost (
42 ; serial (d. adams)
3H ; refresh
15M ; retry
1W ; expiry
1D ) ; minimum
IN NS localhost
localhost IN A 127.0.0.1
ol6toc-scan IN A 192.168.56.45
ol6toc-scan IN A 192.168.56.46
ol6toc-scan IN A 192.168.56.47
ol6tocn1 IN A 192.168.56.41
ol6tocn2 IN A 192.168.56.42
#
I have also created /var/named/56.168.192.in-addr.arpa:
# cat 56.168.192.in-addr.arpa
$ORIGIN 56.168.192.in-addr.arpa.
$TTL 1H
@ IN SOA ol6dns01.localdomain. root.ol6dns01.localdomain. ( 2
3H
1H
1W
1H )
56.168.192.in-addr.arpa. IN NS ol6dns01.localdomain.
45 IN PTR ol6toc-scan.localdomain.
46 IN PTR ol6toc-scan.localdomain.
47 IN PTR ol6toc-scan.localdomain.
41 IN PTR ol6tocn1.localdomain.
42 IN PTR ol6tocn2.localdomain.
#
I have started named service with:
# chkconfig --add named # chkconfig --list named named 0:off 1:off 2:off 3:off 4:off 5:off 6:off # chkconfig named on # chkconfig --list named named 0:off 1:off 2:on 3:on 4:on 5:on 6:off # service named start Starting named: [ OK ] # service named status version: 9.8.2rc1-RedHat-9.8.2-0.37.rc1.el6 CPUs found: 1 worker threads: 1 number of zones: 21 debug level: 0 xfers running: 0 xfers deferred: 0 soa queries in progress: 0 query logging is OFF recursive clients: 0/0/1000 tcp clients: 0/100 server is up and running named (pid 2035) is running... [root@ol6dns01 ~]#
I have tested that DNS is working locally after modifying /etc/resolv.conf:
# cat /etc/resolv.conf search localdomain nameserver 192.168.56.13 # nslookup ol6toc-scan Server: 192.168.56.13 Address: 192.168.56.13#53 Name: ol6toc-scan.localdomain Address: 192.168.56.47 Name: ol6toc-scan.localdomain Address: 192.168.56.45 Name: ol6toc-scan.localdomain Address: 192.168.56.46
And I have tested that after VM reboot ...
# reboot
... named service has been automatically restarted:
# nslookup ol6toc-scan Server: 192.168.56.13 Address: 192.168.56.13#53 Name: ol6toc-scan.localdomain Address: 192.168.56.47 Name: ol6toc-scan.localdomain Address: 192.168.56.45 Name: ol6toc-scan.localdomain Address: 192.168.56.46 #
I have also disabled firewall for ol6dns01 so that cluster nodes will be able to query DNS:
# service iptables stop iptables: Setting chains to policy ACCEPT: filter [ OK ] iptables: Flushing firewall rules: [ OK ] iptables: Unloading modules: # chkconfig iptables off # chkconfig --list iptables iptables 0:off 1:off 2:off 3:off 4:off 5:off 6:off #
I have created first cluster node VM with following configuration:
When installing OL6 I have chosen in the following order:
1. English as language installation
2. French as appropriate keyboard for the system
3. Basic Storage Devices as installation devices
4. ol6tocn1.localdomain as computer name
5. Paris/France as time zone
6. root password
7. Use All Space as installation type
8. Database Server as default installation type
After OL6 installation and VM reboot, I have used system console to configure public network interface with /etc/sysconfig/network-scripts/ifcfg-eth0 editing (HWADDR setting is OK but I have added IPADDR and removed UUID, ONBOOT, NM_CONTROLLED and BOOTPROTO):
# cat /etc/sysconfig/network-scripts/ifcfg-eth0 DEVICE=eth0 HWADDR=08:00:27:65:D9:22 TYPE=Ethernet IPADDR=192.168.56.41 # ifup eth0 Determing if ip address 192.168.56.41 is already in use for device eth0... #
Now I connect with SSH (Putty from Windows host) to ol6tocn1 and check network configuration:
# ip addr
1: lo: mtu 65536 qdisc noqueue state UNKNOWN
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: eth0: mtu 1500 qdisc pfifo_fast state UP qlen 1000
link/ether 08:00:27:65:d9:22 brd ff:ff:ff:ff:ff:ff
inet 192.168.56.41/24 brd 192.168.56.255 scope global eth0
inet6 fe80::a00:27ff:fe65:d922/64 scope link
valid_lft forever preferred_lft forever
3: eth1: mtu 1500 qdisc noop state DOWN qlen 1000
link/ether 08:00:27:a0:e0:f9 brd ff:ff:ff:ff:ff:ff
#
I have configured second network interface for the RAC interconnect:
# cat /etc/sysconfig/network-scripts/ifcfg-eth1 DEVICE=eth1 HWADDR=08:00:27:A0:E0:F9 TYPE=Ethernet IPADDR=192.168.43.41 # ifup eth1 Determining if ip address 192.168.43.41 is already in use for device eth1... #
I have checked all network interfaces status with:
# ip addr
1: lo: mtu 65536 qdisc noqueue state UNKNOWN
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: eth0: mtu 1500 qdisc pfifo_fast state UP qlen 1000
link/ether 08:00:27:65:d9:22 brd ff:ff:ff:ff:ff:ff
inet 192.168.56.41/24 brd 192.168.56.255 scope global eth0
inet6 fe80::a00:27ff:fe65:d922/64 scope link
valid_lft forever preferred_lft forever
3: eth1: mtu 1500 qdisc pfifo_fast state UP qlen 1000
link/ether 08:00:27:a0:e0:f9 brd ff:ff:ff:ff:ff:ff
inet 192.168.43.41/24 brd 192.168.43.255 scope global eth1
inet6 fe80::a00:27ff:fea0:e0f9/64 scope link
valid_lft forever preferred_lft forever
#
I have added to /etc/resolv.conf the DNS server:
# cat /etc/resolv.conf search localdomain nameserver 192.168.56.13 #
I have checked name resolution:
# nslookup ol6toc-scan Server: 192.168.56.13 Address: 192.168.56.13#53 Name: ol6toc-scan.localdomain Address: 192.168.56.45 Name: ol6toc-scan.localdomain Address: 192.168.56.46 Name: ol6toc-scan.localdomain Address: 192.168.56.47 #
I have updated /etc/hosts with both cluster nodes IP addresses:
# cat /etc/hosts 127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4 ::1 localhost localhost.localdomain localhost6 localhost6.localdomain6 # # Public 192.168.56.41 ol6tocn1 ol6tocn1.localdomain 192.168.56.42 ol6tocn2 ol6tocn2.localdomain # Private 192.168.43.41 ol6tocn1-priv ol6tocn1-priv.localdomain 192.168.43.42 ol6tocn2-priv ol6tocn2-priv.localdomain # VIP 192.168.56.31 ol6tocn1-vip ol6tocn1-vip.localdomain 192.168.56.32 ol6tocn2-vip ol6tocn2-vip.localdomain #
I have disabled SELinux:
# cat /etc/selinux/config # This file controls the state of SELinux on the system. # SELINUX= can take one of these three values: # enforcing - SELinux security policy is enforced. # permissive - SELinux prints warnings instead of enforcing. # disabled - No SELinux policy is loaded. SELINUX=disabled # SELINUXTYPE= can take one of these two values: # targeted - Targeted processes are protected, # mls - Multi Level Security protection. SELINUXTYPE=targeted
I have also disabled firewall:
# chkconfig --list iptables iptables 0:off 1:off 2:on 3:on 4:on 5:on 6:off # chkconfig --del iptables # chkconfig --list iptables service iptables supports chkconfig, but is not referenced in any runlevel (run 'chkconfig --add iptables') #
I have installed Oracle 12c preinstallationRPM in 3 steps.
First step is to add Oracle Linux 6.7 ISO file as Virtual Optical Disk File in VM CDROM drive with VBOX GUI and to mount it on Linux:
# mount -t iso9660 /dev/cdrom /media/ mount: block device /dev/sr0 is write-protected, mounting read-only # ls -al /media total 1470 drwxr-xr-x. 12 root root 6144 Jul 28 08:49 . dr-xr-xr-x. 25 root root 4096 Sep 12 16:23 .. -rw-r--r--. 1 root root 44 Jul 28 08:49 .discinfo drwxr-xr-x. 3 root root 2048 Jul 28 08:48 EFI -rw-r--r--. 1 root root 8529 Jul 28 08:16 EULA -rw-r--r--. 1 root root 8529 Jul 28 08:16 eula.en_US -rw-r--r--. 1 root root 3334 Jul 28 08:16 eula.py -rw-r--r--. 1 root root 18390 Jul 28 08:16 GPL drwxr-xr-x. 3 root root 2048 Jul 28 08:18 HighAvailability drwxr-xr-x. 3 root root 2048 Jul 28 08:49 images drwxr-xr-x. 2 root root 2048 Jul 28 08:49 isolinux drwxr-xr-x. 3 root root 2048 Jul 28 08:18 LoadBalancer -rw-r--r--. 1 root root 98 Jul 28 08:49 media.repo drwxr-xr-x. 2 root root 706560 Jul 28 08:09 Packages -rw-r--r--. 1 root root 2619 Jul 28 08:16 README-en -rw-r--r--. 1 root root 7054 Jul 28 08:16 README-en.html -rw-r--r--. 1 root root 64617 Jul 28 08:16 RELEASE-NOTES-en -rw-r--r--. 1 root root 170750 Jul 28 08:16 RELEASE-NOTES-en.html -rw-r--r--. 1 root root 64617 Jul 28 08:16 RELEASE-NOTES-x86_64-en -rw-r--r--. 1 root root 170750 Jul 28 08:16 RELEASE-NOTES-x86_64-en.html -rw-r--r--. 1 root root 64617 Jul 28 08:16 RELEASE-NOTES-x86-en -rw-r--r--. 1 root root 170750 Jul 28 08:16 RELEASE-NOTES-x86-en.html lrwxrwxrwx. 1 root root 15 Jul 28 08:18 repodata -> Server/repodata drwxr-xr-x. 3 root root 2048 Jul 28 08:18 ResilientStorage -rw-r--r--. 1 root root 1011 Jul 28 08:16 RPM-GPG-KEY -rw-r--r--. 1 root root 1011 Jul 28 08:16 RPM-GPG-KEY-oracle drwxr-xr-x. 3 root root 2048 Jul 28 08:18 ScalableFileSystem drwxr-xr-x. 4 root root 2048 Jul 28 08:18 Server -rw-r--r--. 1 root root 108 Jul 28 08:16 supportinfo -r--r--r--. 1 root root 6517 Jul 28 08:49 TRANS.TBL -rw-r--r--. 1 root root 1546 Jul 28 08:49 .treeinfo drwxr-xr-x. 3 root root 2048 Jul 28 08:18 UEK3 #
Second step is to configure yum to use /media; I have created a new repo file /etc/yum.repos.d/myyum.repo:
# mv /etc/yum.repos.d/public-yum-ol6.repo /etc/yum.repos.d/public-yum-ol6.repo.orig # cat /etc/yum.repos.d/myyum.repo [OL67] name=Oracle Linux 6.7 x86_64 baseurl=file:///media gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY gpgcheck=1 enabled=1 # yum clean all Loaded plugins: security, ulninfo Cleaning repos: OL67 Cleaning up Everything # yum repolist Loaded plugins: security, ulninfo OL67 | 3.7 kB 00:00 ... OL67/primary_db | 3.0 MB 00:00 ... repo id repo name status OL67 Oracle Linux 6.7 x86_64 3,800 repolist: 3,800
I have installed Oracle 12c preinstallation RPM with:
# yum -y install oracle-rdbms-server-12cR1-preinstall Loaded plugins: security, ulninfo Setting up Install Process Resolving Dependencies --> Running transaction check ---> Package oracle-rdbms-server-12cR1-preinstall.x86_64 0:1.0-14.el6 will be installed --> Processing Dependency: xorg-x11-xauth for package: oracle-rdbms-server-12cR1-preinstall-1.0-14.el6.x86_64 --> Processing Dependency: xorg-x11-utils for package: oracle-rdbms-server-12cR1-preinstall-1.0-14.el6.x86_64 --> Processing Dependency: libstdc++-devel for package: oracle-rdbms-server-12cR1-preinstall-1.0-14.el6.x86_64 --> Processing Dependency: libaio-devel for package: oracle-rdbms-server-12cR1-preinstall-1.0-14.el6.x86_64 --> Processing Dependency: ksh for package: oracle-rdbms-server-12cR1-preinstall-1.0-14.el6.x86_64 --> Processing Dependency: gcc-c++ for package: oracle-rdbms-server-12cR1-preinstall-1.0-14.el6.x86_64 --> Processing Dependency: gcc for package: oracle-rdbms-server-12cR1-preinstall-1.0-14.el6.x86_64 --> Processing Dependency: compat-libstdc++-33 for package: oracle-rdbms-server-12cR1-preinstall-1.0-14.el6.x86_64 --> Processing Dependency: compat-libcap1 for package: oracle-rdbms-server-12cR1-preinstall-1.0-14.el6.x86_64 --> Running transaction check ---> Package compat-libcap1.x86_64 0:1.10-1 will be installed ---> Package compat-libstdc++-33.x86_64 0:3.2.3-69.el6 will be installed ---> Package gcc.x86_64 0:4.4.7-16.el6 will be installed --> Processing Dependency: cpp = 4.4.7-16.el6 for package: gcc-4.4.7-16.el6.x86_64 --> Processing Dependency: cloog-ppl >= 0.15 for package: gcc-4.4.7-16.el6.x86_64 ---> Package gcc-c++.x86_64 0:4.4.7-16.el6 will be installed --> Processing Dependency: libmpfr.so.1()(64bit) for package: gcc-c++-4.4.7-16.el6.x86_64 ---> Package ksh.x86_64 0:20120801-28.el6 will be installed ---> Package libaio-devel.x86_64 0:0.3.107-10.el6 will be installed ---> Package libstdc++-devel.x86_64 0:4.4.7-16.el6 will be installed ---> Package xorg-x11-utils.x86_64 0:7.5-6.el6 will be installed --> Processing Dependency: libdmx.so.1()(64bit) for package: xorg-x11-utils-7.5-6.el6.x86_64 --> Processing Dependency: libXxf86misc.so.1()(64bit) for package: xorg-x11-utils-7.5-6.el6.x86_64 --> Processing Dependency: libXxf86dga.so.1()(64bit) for package: xorg-x11-utils-7.5-6.el6.x86_64 ---> Package xorg-x11-xauth.x86_64 1:1.0.2-7.1.el6 will be installed --> Processing Dependency: libXmuu.so.1()(64bit) for package: 1:xorg-x11-xauth-1.0.2-7.1.el6.x86_64 --> Running transaction check ---> Package cloog-ppl.x86_64 0:0.15.7-1.2.el6 will be installed --> Processing Dependency: libppl_c.so.2()(64bit) for package: cloog-ppl-0.15.7-1.2.el6.x86_64 --> Processing Dependency: libppl.so.7()(64bit) for package: cloog-ppl-0.15.7-1.2.el6.x86_64 ---> Package cpp.x86_64 0:4.4.7-16.el6 will be installed ---> Package libXmu.x86_64 0:1.1.1-2.el6 will be installed ---> Package libXxf86dga.x86_64 0:1.1.4-2.1.el6 will be installed ---> Package libXxf86misc.x86_64 0:1.0.3-4.el6 will be installed ---> Package libdmx.x86_64 0:1.1.3-3.el6 will be installed ---> Package mpfr.x86_64 0:2.4.1-6.el6 will be installed --> Running transaction check ---> Package ppl.x86_64 0:0.10.2-11.el6 will be installed --> Finished Dependency Resolution Dependencies Resolved ========================================================================================================================================================== Package Arch Version Repository Size ========================================================================================================================================================== Installing: oracle-rdbms-server-12cR1-preinstall x86_64 1.0-14.el6 OL67 17 k Installing for dependencies: cloog-ppl x86_64 0.15.7-1.2.el6 OL67 93 k compat-libcap1 x86_64 1.10-1 OL67 17 k compat-libstdc++-33 x86_64 3.2.3-69.el6 OL67 183 k cpp x86_64 4.4.7-16.el6 OL67 3.7 M gcc x86_64 4.4.7-16.el6 OL67 10 M gcc-c++ x86_64 4.4.7-16.el6 OL67 4.7 M ksh x86_64 20120801-28.el6 OL67 759 k libXmu x86_64 1.1.1-2.el6 OL67 65 k libXxf86dga x86_64 1.1.4-2.1.el6 OL67 17 k libXxf86misc x86_64 1.0.3-4.el6 OL67 17 k libaio-devel x86_64 0.3.107-10.el6 OL67 13 k libdmx x86_64 1.1.3-3.el6 OL67 14 k libstdc++-devel x86_64 4.4.7-16.el6 OL67 1.6 M mpfr x86_64 2.4.1-6.el6 OL67 156 k ppl x86_64 0.10.2-11.el6 OL67 1.3 M xorg-x11-utils x86_64 7.5-6.el6 OL67 94 k xorg-x11-xauth x86_64 1:1.0.2-7.1.el6 OL67 34 k Transaction Summary ========================================================================================================================================================== Install 18 Package(s) Total download size: 23 M Installed size: 56 M Downloading Packages: ---------------------------------------------------------------------------------------------------------------------------------------------------------- Total 18 MB/s | 23 MB 00:01 warning: rpmts_HdrFromFdno: Header V3 RSA/SHA256 Signature, key ID ec551f03: NOKEY Retrieving key from file:///etc/pki/rpm-gpg/RPM-GPG-KEY Importing GPG key 0xEC551F03: Userid : Oracle OSS group (Open Source Software group) Package: rhn-client-tools-1.0.0.1-32.0.4.el6.noarch (@anaconda-OracleLinuxServer-201507280245.x86_64/6.7) From : /etc/pki/rpm-gpg/RPM-GPG-KEY Running rpm_check_debug Running Transaction Test Transaction Test Succeeded Running Transaction Installing : libstdc++-devel-4.4.7-16.el6.x86_64 1/18 Installing : mpfr-2.4.1-6.el6.x86_64 2/18 Installing : cpp-4.4.7-16.el6.x86_64 3/18 Installing : libdmx-1.1.3-3.el6.x86_64 4/18 Installing : libXmu-1.1.1-2.el6.x86_64 5/18 Installing : 1:xorg-x11-xauth-1.0.2-7.1.el6.x86_64 6/18 Installing : ksh-20120801-28.el6.x86_64 7/18 Installing : compat-libstdc++-33-3.2.3-69.el6.x86_64 8/18 Installing : libXxf86misc-1.0.3-4.el6.x86_64 9/18 Installing : libaio-devel-0.3.107-10.el6.x86_64 10/18 Installing : ppl-0.10.2-11.el6.x86_64 11/18 Installing : cloog-ppl-0.15.7-1.2.el6.x86_64 12/18 Installing : gcc-4.4.7-16.el6.x86_64 13/18 Installing : gcc-c++-4.4.7-16.el6.x86_64 14/18 Installing : libXxf86dga-1.1.4-2.1.el6.x86_64 15/18 Installing : xorg-x11-utils-7.5-6.el6.x86_64 16/18 Installing : compat-libcap1-1.10-1.x86_64 17/18 Installing : oracle-rdbms-server-12cR1-preinstall-1.0-14.el6.x86_64 18/18 Verifying : compat-libcap1-1.10-1.x86_64 1/18 Verifying : libXxf86dga-1.1.4-2.1.el6.x86_64 2/18 Verifying : xorg-x11-utils-7.5-6.el6.x86_64 3/18 Verifying : gcc-4.4.7-16.el6.x86_64 4/18 Verifying : ppl-0.10.2-11.el6.x86_64 5/18 Verifying : libaio-devel-0.3.107-10.el6.x86_64 6/18 Verifying : libXxf86misc-1.0.3-4.el6.x86_64 7/18 Verifying : mpfr-2.4.1-6.el6.x86_64 8/18 Verifying : compat-libstdc++-33-3.2.3-69.el6.x86_64 9/18 Verifying : ksh-20120801-28.el6.x86_64 10/18 Verifying : 1:xorg-x11-xauth-1.0.2-7.1.el6.x86_64 11/18 Verifying : oracle-rdbms-server-12cR1-preinstall-1.0-14.el6.x86_64 12/18 Verifying : cpp-4.4.7-16.el6.x86_64 13/18 Verifying : libstdc++-devel-4.4.7-16.el6.x86_64 14/18 Verifying : libXmu-1.1.1-2.el6.x86_64 15/18 Verifying : cloog-ppl-0.15.7-1.2.el6.x86_64 16/18 Verifying : libdmx-1.1.3-3.el6.x86_64 17/18 Verifying : gcc-c++-4.4.7-16.el6.x86_64 18/18 Installed: oracle-rdbms-server-12cR1-preinstall.x86_64 0:1.0-14.el6 Dependency Installed: cloog-ppl.x86_64 0:0.15.7-1.2.el6 compat-libcap1.x86_64 0:1.10-1 compat-libstdc++-33.x86_64 0:3.2.3-69.el6 cpp.x86_64 0:4.4.7-16.el6 gcc.x86_64 0:4.4.7-16.el6 gcc-c++.x86_64 0:4.4.7-16.el6 ksh.x86_64 0:20120801-28.el6 libXmu.x86_64 0:1.1.1-2.el6 libXxf86dga.x86_64 0:1.1.4-2.1.el6 libXxf86misc.x86_64 0:1.0.3-4.el6 libaio-devel.x86_64 0:0.3.107-10.el6 libdmx.x86_64 0:1.1.3-3.el6 libstdc++-devel.x86_64 0:4.4.7-16.el6 mpfr.x86_64 0:2.4.1-6.el6 ppl.x86_64 0:0.10.2-11.el6 xorg-x11-utils.x86_64 0:7.5-6.el6 xorg-x11-xauth.x86_64 1:1.0.2-7.1.el6 Complete! #
I have rebooted after removing ISO file from virtual drive:
# shutdown -r now
... and checked that SELinux is disabled:
# secon secon: SELinux is not enabled
Default Oracle Linux 6.7 kernel is Unbreakable Entreprise Kernel (UELK):
# uname -a Linux ol6tocn1.localdomain 3.8.13-68.3.4.el6uek.x86_64 #2 SMP Tue Jul 14 15:03:36 PDT 2015 x86_64 x86_64 x86_64 GNU/Linux
I have installed ASMLib packages after adding again ISO file to virtual cdrom drive:
# yum -y install oracleasm-support Loaded plugins: security, ulninfo Setting up Install Process Resolving Dependencies --> Running transaction check ---> Package oracleasm-support.x86_64 0:2.1.8-1.el6 will be installed --> Finished Dependency Resolution Dependencies Resolved =============================================================================================================== Package Arch Version Repository Size =============================================================================================================== Installing: oracleasm-support x86_64 2.1.8-1.el6 OL67 73 k Transaction Summary =============================================================================================================== Install 1 Package(s) Total download size: 73 k Installed size: 216 k Downloading Packages: Running rpm_check_debug Running Transaction Test Transaction Test Succeeded Running Transaction Installing : oracleasm-support-2.1.8-1.el6.x86_64 1/1 Verifying : oracleasm-support-2.1.8-1.el6.x86_64 1/1 Installed: oracleasm-support.x86_64 0:2.1.8-1.el6 Complete! #
I have downloaded from OTN oracleasmlib-2.0.4-1.el6.x86_64.rpm because it is not in Oracle Linux ISO file:
# yum -y install oracleasmlib-2.0.4-1.el6.x86_64.rpm Loaded plugins: security, ulninfo Setting up Install Process Examining oracleasmlib-2.0.4-1.el6.x86_64.rpm: oracleasmlib-2.0.4-1.el6.x86_64 Marking oracleasmlib-2.0.4-1.el6.x86_64.rpm to be installed Resolving Dependencies --> Running transaction check ---> Package oracleasmlib.x86_64 0:2.0.4-1.el6 will be installed --> Finished Dependency Resolution Dependencies Resolved =============================================================================================================== Package Arch Version Repository Size =============================================================================================================== Installing: oracleasmlib x86_64 2.0.4-1.el6 /oracleasmlib-2.0.4-1.el6.x86_64 27 k Transaction Summary =============================================================================================================== Install 1 Package(s) Total size: 27 k Installed size: 27 k Downloading Packages: Running rpm_check_debug Running Transaction Test Transaction Test Succeeded Running Transaction Installing : oracleasmlib-2.0.4-1.el6.x86_64 1/1 Verifying : oracleasmlib-2.0.4-1.el6.x86_64 1/1 Installed: oracleasmlib.x86_64 0:2.0.4-1.el6 Complete! #
I have shutdown Linux VM and removed ISO file from virtual drive:
# shutdown -n now
Even if I currently have only one cluster node I can start to configure shared storage because configuration will be kept by cloning first cluster node VM to second cluster node VM.
On my host system (Windows 8.1), I have created 3 virtual disks for 3 ASM disk groups:
>vboxmanage createhd --filename ol6tocn1.ol6tocn2.sata.1.vdi --size=10240 -format=VDI --variant fixe 0%...10%...20%...30%...40%...50%...60%...70%...80%...90%...100% Medium created. UUID: 59ac48b8-c743-4aad-af4b-4d0cc256bc22 >vboxmanage createhd --filename ol6tocn1.ol6tocn2.sata.2.vdi --size=10240 -format=VDI --variant fixe 0%...10%...20%...30%...40%...50%...60%...70%...80%...90%...100% Medium created. UUID: 352a960f-854d-400f-bd44-9910e9b01cbb >vboxmanage createhd --filename ol6tocn1.ol6tocn2.sata.3.vdi --size=10240 -format=VDI --variant fixe 0%...10%...20%...30%...40%...50%...60%...70%...80%...90%...100% Medium created. UUID: 6a89629a-db33-4df7-83f5-5f82107660c4 >
I have run from Windows VBOX commands to attach the virtual disks and make them shareable:
>vboxmanage storageattach ol6tocn1 --storagectl "SATA" --port 1 --device 0 --type hdd --medium ol6tocn1.ol6tocn2.sata.1.vdi --mtype shareable >vboxmanage storageattach ol6tocn1 --storagectl "SATA" --port 2 --device 0 --type hdd --medium ol6tocn1.ol6tocn2.sata.2.vdi --mtype shareable >vboxmanage storageattach ol6tocn1 --storagectl "SATA" --port 3 --device 0 --type hdd --medium ol6tocn1.ol6tocn2.sata.3.vdi --mtype shareable > >vboxmanage modifyhd ol6tocn1.ol6tocn2.sata.1.vdi -type shareable >vboxmanage modifyhd ol6tocn1.ol6tocn2.sata.2.vdi -type shareable >vboxmanage modifyhd ol6tocn1.ol6tocn2.sata.3.vdi -type shareable >
I have restarted ol6tocn1 VM and checked that the corresponding device files have been created:
# ls -al /dev/sd[bcd] brw-rw---- 1 root disk 8, 16 Sep 13 11:26 /dev/sdb brw-rw---- 1 root disk 8, 32 Sep 13 11:26 /dev/sdc brw-rw---- 1 root disk 8, 48 Sep 13 11:26 /dev/sdd #
I have created a single partition on each disk:
# fdisk /dev/sdb
Device contains neither a valid DOS partition table, nor Sun, SGI or OSF disklabel
Building a new DOS disklabel with disk identifier 0xad799975.
Changes will remain in memory only, until you decide to write them.
After that, of course, the previous content won't be recoverable.
Warning: invalid flag 0x0000 of partition table 4 will be corrected by w(rite)
WARNING: DOS-compatible mode is deprecated. It's strongly recommended to
switch off the mode (command 'c') and change display units to
sectors (command 'u').
Command (m for help): n
Command action
e extended
p primary partition (1-4)
p
Partition number (1-4): 1
First cylinder (1-1305, default 1):
Using default value 1
Last cylinder, +cylinders or +size{K,M,G} (1-1305, default 1305):
Using default value 1305
Command (m for help): w
The partition table has been altered!
Calling ioctl() to re-read partition table.
Syncing disks.
# fdisk /dev/sdc
Device contains neither a valid DOS partition table, nor Sun, SGI or OSF disklabel
Building a new DOS disklabel with disk identifier 0xbc140dc4.
Changes will remain in memory only, until you decide to write them.
After that, of course, the previous content won't be recoverable.
Warning: invalid flag 0x0000 of partition table 4 will be corrected by w(rite)
WARNING: DOS-compatible mode is deprecated. It's strongly recommended to
switch off the mode (command 'c') and change display units to
sectors (command 'u').
Command (m for help): n
Command action
e extended
p primary partition (1-4)
p
Partition number (1-4): 1
First cylinder (1-1305, default 1):
Using default value 1
Last cylinder, +cylinders or +size{K,M,G} (1-1305, default 1305):
Using default value 1305
Command (m for help): w
The partition table has been altered!
Calling ioctl() to re-read partition table.
Syncing disks.
# fdisk /dev/sdd
Device contains neither a valid DOS partition table, nor Sun, SGI or OSF disklabel
Building a new DOS disklabel with disk identifier 0x1c25e64a.
Changes will remain in memory only, until you decide to write them.
After that, of course, the previous content won't be recoverable.
Warning: invalid flag 0x0000 of partition table 4 will be corrected by w(rite)
WARNING: DOS-compatible mode is deprecated. It's strongly recommended to
switch off the mode (command 'c') and change display units to
sectors (command 'u').
Command (m for help): n
Command action
e extended
p primary partition (1-4)
p
Partition number (1-4): 1
First cylinder (1-1305, default 1):
Using default value 1
Last cylinder, +cylinders or +size{K,M,G} (1-1305, default 1305):
Using default value 1305
Command (m for help): w
The partition table has been altered!
Calling ioctl() to re-read partition table.
Syncing disks.
# ls -al /dev/sd[bcd]*
brw-rw---- 1 root disk 8, 16 Sep 13 11:29 /dev/sdb
brw-rw---- 1 root disk 8, 17 Sep 13 11:29 /dev/sdb1
brw-rw---- 1 root disk 8, 32 Sep 13 11:29 /dev/sdc
brw-rw---- 1 root disk 8, 33 Sep 13 11:29 /dev/sdc1
brw-rw---- 1 root disk 8, 48 Sep 13 11:29 /dev/sdd
brw-rw---- 1 root disk 8, 49 Sep 13 11:29 /dev/sdd1
#
I have configured ASMLib:
# /usr/sbin/oracleasm configure -i
Configuring the Oracle ASM library driver.
This will configure the on-boot properties of the Oracle ASM library
driver. The following questions will determine whether the driver is
loaded on boot and what permissions it will have. The current values
will be shown in brackets ('[]'). Hitting without typing an
answer will keep that current value. Ctrl-C will abort.
Default user to own the driver interface []: oracle
Default group to own the driver interface []: dba
Start Oracle ASM library driver on boot (y/n) [n]: y
Scan for Oracle ASM disks on boot (y/n) [y]: y
Writing Oracle ASM library driver configuration: done
# oracleasm init
Creating /dev/oracleasm mount point: /dev/oracleasm
Loading module "oracleasm": oracleasm
Configuring "oracleasm" to use device physical block size
Mounting ASMlib driver filesystem: /dev/oracleasm
I have created 3 ASM disks:
# oracleasm createdisk ASM1 /dev/sdb1 Writing disk header: done Instantiating disk: done # oracleasm createdisk ASM2 /dev/sdc1 Writing disk header: done Instantiating disk: done # oracleasm createdisk ASM3 /dev/sdd1 Writing disk header: done Instantiating disk: done # oracleasm listdisks ASM1 ASM2 ASM3
I have rebooted ...:
# shutdown -r now
... and checked that the ASM disks are still available:
# oracleasm status Checking if ASM is loaded: yes Checking if /dev/oracleasm is mounted: yes # oracleasm listdisks ASM1 ASM2 ASM3 #
I have shutdown ol6tocn1:
# shutdown -h now
I have first cloned the ol6tocn1 system disk:
>vboxmanage clonehd ol6tocn1.sata.0.vdi ol6tocn2.sata.0.vdi 0%...10%...20%...30%...40%...50%...60%...70%...80%...90%...100% Clone medium created in format 'VDI'. UUID: 879b618a-f25f-4adb-a75a-2749d7f1d2f1 >
Using VBOX GUI I have created a new VM named ol6otcn2 using ol6tocn2.sata.0.vdi as system disk.
I have set the same properties for ol6tocn2 as for ol6tocn1:
I have attached the 3 shared disks to ol6tocn2:
>vboxmanage storageattach ol6tocn2 --storagectl "SATA" --port 1 --device 0 --type hdd --medium ol6tocn1.ol6tocn2.sata.1.vdi --mtype shareable >vboxmanage storageattach ol6tocn2 --storagectl "SATA" --port 2 --device 0 --type hdd --medium ol6tocn1.ol6tocn2.sata.2.vdi --mtype shareable >vboxmanage storageattach ol6tocn2 --storagectl "SATA" --port 3 --device 0 --type hdd --medium ol6tocn1.ol6tocn2.sata.3.vdi --mtype shareable >
I have started ol6ocn2 and from the console modified the hostname:
# cat /etc/sysconfig/network NETWORKING=yes HOSTNAME=ol6tocn2.localdomain # oracle-rdbms-server-12cR1-preinstall : Add NOZEROCONF=yes NOZEROCONF=yes #
Cloning the Linux machine has created new network interfaces named eth2 and eth3 but for the RAC cluster we need to have the same network interface names for all cluster nodes namely eth0 and eth1.
From the console I have reconfigured the network interfaces.
First I have modified /etc/udev/rules.d/70-persistent-net.rules to have only eth0 and eth1 as network interfaces with the right MAC addresses displayed by ip addr output:
# cat /etc/udev/rules.d/70-persistent-net.rules
# This file was automatically generated by the /lib/udev/write_net_rules
# program, run by the persistent-net-generator.rules rules file.
#
# You can modify it, as long as you keep each rule on a single
# line, and change only the value of the NAME= key.
# PCI device 0x8086:0x100e (e1000)
SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*", ATTR{address}=="08:00:27:f8:79:80", ATTR{type}=="1", KERNEL=="eth*", NAME="eth0"
# PCI device 0x8086:0x100e (e1000)
SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*", ATTR{address}=="08:00:27:0b:5f:10", ATTR{type}=="1", KERNEL=="eth*", NAME="eth1"
#
I have also modified in /etc/sysconfig/network-scripts ifcfg-eth0 and ifcfg-eth1 scripts to use right MAC address for HWADDR setting and right IP address for IPADDR setting:
# cd /etc/sysconfig/network-scripts/ # cat ifcfg-eth0 DEVICE=eth0 HWADDR=08:00:27:F8:79:80 TYPE=Ethernet IPADDR=192.168.56.42 # cat ifcfg-eth1 DEVICE=eth1 HWADDR=08:00:27:0B:5F:10 TYPE=Ethernet IPADDR=192.168.43.42 #
and I have rebooted to be able to connect with SSH and to check that network interfaces status is also OK after reboot:
# hostname
ol6tocn2.localdomain
# ip addr
1: lo: mtu 65536 qdisc noqueue state UNKNOWN
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: eth1: mtu 1500 qdisc pfifo_fast state UP qlen 1000
link/ether 08:00:27:0b:5f:10 brd ff:ff:ff:ff:ff:ff
inet 192.168.43.42/24 brd 192.168.43.255 scope global eth1
inet6 fe80::a00:27ff:fe0b:5f10/64 scope link
valid_lft forever preferred_lft forever
3: eth0: mtu 1500 qdisc pfifo_fast state UP qlen 1000
link/ether 08:00:27:f8:79:80 brd ff:ff:ff:ff:ff:ff
inet 192.168.56.42/24 brd 192.168.56.255 scope global eth0
inet6 fe80::a00:27ff:fef8:7980/64 scope link
valid_lft forever preferred_lft forever
#
I have checked that /etc/resolv.conf has not been modified ...
# cat /etc/resolv.conf search localdomain nameserver 192.168.56.13
... and that DNS is OK:
# nslookup ol6toc-scan Server: 192.168.56.13 Address: 192.168.56.13#53 Name: ol6toc-scan.localdomain Address: 192.168.56.47 Name: ol6toc-scan.localdomain Address: 192.168.56.45 Name: ol6toc-scan.localdomain Address: 192.168.56.46 #
I have also checked that the ASM ddisks are available on ol6tocn2:
# oracleasm status Checking if ASM is loaded: yes Checking if /dev/oracleasm is mounted: yes # oracleasm listdisks ASM1 ASM2 ASM3 #
I have configured passwordless ssh between the 2 cluster nodes for oracle account.
First I have set oracle password on both nodes (note that oracle account has been created by the preinstallation RPM):
# passwd oracle Changing password for user oracle. New password: BAD PASSWORD: it is based on a dictionary word BAD PASSWORD: is too simple Retype new password: passwd: all authentication tokens updated successfully.
I have run with oracle account on ol6tocn1 :
[oracle@ol6tocn1 ~]$ ssh-keygen Generating public/private rsa key pair. Enter file in which to save the key (/home/oracle/.ssh/id_rsa): Created directory '/home/oracle/.ssh'. Enter passphrase (empty for no passphrase): Enter same passphrase again: Your identification has been saved in /home/oracle/.ssh/id_rsa. Your public key has been saved in /home/oracle/.ssh/id_rsa.pub. The key fingerprint is: 8e:d8:a5:c3:d5:36:1c:5a:83:e4:b3:03:42:c1:77:23 oracle@ol6tocn1.localdomain The key's randomart image is: +--[ RSA 2048]----+ | .o. . | | .. Eoo. | | ...o+.+ | | . . B o | | S = | | + * o . | | . * . | | . | | | +-----------------+ [oracle@ol6tocn1 ~]$ [oracle@ol6tocn1 ~]$ ssh-copy-id ol6tocn2 The authenticity of host 'ol6tocn2 (192.168.56.42)' can't be established. RSA key fingerprint is 13:b8:ca:a1:15:48:03:35:f4:5f:65:e8:01:6e:f2:44. Are you sure you want to continue connecting (yes/no)? yes Warning: Permanently added 'ol6tocn2,192.168.56.42' (RSA) to the list of known hosts. Now try logging into the machine, with "ssh 'ol6tocn2'", and check in: .ssh/authorized_keys to make sure we haven't added extra keys that you weren't expecting. [oracle@ol6tocn1 ~]$
I have run similar steps on cluster node 2:
[oracle@ol6tocn2 ~]$ ssh-keygen Generating public/private rsa key pair. Enter file in which to save the key (/home/oracle/.ssh/id_rsa): Enter passphrase (empty for no passphrase): Enter same passphrase again: Your identification has been saved in /home/oracle/.ssh/id_rsa. Your public key has been saved in /home/oracle/.ssh/id_rsa.pub. The key fingerprint is: 20:8f:3e:06:15:d9:2e:c0:91:75:75:68:ff:21:c3:7d oracle@ol6tocn2.localdomain The key's randomart image is: +--[ RSA 2048]----+ | ..+oo..... | | + .o. o. | | .o... o . | | ..+.. = o E | | . ... S + o | | o . | | + | | . . | | | +-----------------+ [oracle@ol6tocn2 ~]$ ssh-copy-id ol6tocn1 The authenticity of host 'ol6tocn1 (192.168.56.41)' can't be established. RSA key fingerprint is 13:b8:ca:a1:15:48:03:35:f4:5f:65:e8:01:6e:f2:44. Are you sure you want to continue connecting (yes/no)? yes Warning: Permanently added 'ol6tocn1,192.168.56.41' (RSA) to the list of known hosts. oracle@ol6tocn1's password: Now try logging into the machine, with "ssh 'ol6tocn1'", and check in: .ssh/authorized_keys to make sure we haven't added extra keys that you weren't expecting. [oracle@ol6tocn2 ~]$
I have checked that I can connect from each local node to each remote node without password:
[oracle@ol6tocn1 ~]$ ssh ol6tocn2 hostname ol6tocn2.localdomain [oracle@ol6tocn1 ~]$
and:
[oracle@ol6tocn2 ~]$ ssh ol6tocn1 hostname ol6tocn1.localdomain [oracle@ol6tocn2 ~]$
I have also enabled passwordless ssh on each node with local ssh connection (i.e. from ol6tocn1 to ol6tocn1 and from ol6tocn2 to ol6tocn2):
[oracle@ol6tocn1 ~]$ ssh-copy-id ol6tocn1 The authenticity of host 'ol6tocn1 (192.168.56.41)' can't be established. RSA key fingerprint is 13:b8:ca:a1:15:48:03:35:f4:5f:65:e8:01:6e:f2:44. Are you sure you want to continue connecting (yes/no)? yes Warning: Permanently added 'ol6tocn1,192.168.56.41' (RSA) to the list of known hosts. oracle@ol6tocn1's password: Now try logging into the machine, with "ssh 'ol6tocn1'", and check in: .ssh/authorized_keys to make sure we haven't added extra keys that you weren't expecting. [oracle@ol6tocn1 ~]$
and:
[oracle@ol6tocn2 ~]$ ssh-copy-id ol6tocn2 The authenticity of host 'ol6tocn2 (192.168.56.42)' can't be established. RSA key fingerprint is 13:b8:ca:a1:15:48:03:35:f4:5f:65:e8:01:6e:f2:44. Are you sure you want to continue connecting (yes/no)? yes Warning: Permanently added 'ol6tocn2,192.168.56.42' (RSA) to the list of known hosts. oracle@ol6tocn2's password: Now try logging into the machine, with "ssh 'ol6tocn2'", and check in: .ssh/authorized_keys to make sure we haven't added extra keys that you weren't expecting. [oracle@ol6tocn2 ~]$
I have checked that I connect locally on each node without password:
[oracle@ol6tocn1 ~]$ ssh ol6tocn1 hostname ol6tocn1.localdomain [oracle@ol6tocn1 ~]$
and:
[oracle@ol6tocn2 ~]$ ssh ol6tocn2 hostname ol6tocn2.localdomain [oracle@ol6tocn2 ~]$
I have disabled NTP configuration on both nodes with:
# mv /etc/ntp.conf /etc/ntp.conf.orig
I have uploaded Grid Infrastructure 12.1.0.2 media (note that there is only one such media that must be used for both Entreprise Edition and SE2) to first cluster node after creating /stage directory:
# mkdir /stage # chown oracle:dba /stage
I have run with oracle account:
$ unzip linuxamd64_12102_grid_1of2.zip $ unzip linuxamd64_12102_grid_2of2.zip
I have run CLUVFY :
$ ./grid/runcluvfy.sh stage -pre crsinst -n "ol6tocn1,ol6tocn2"
Performing pre-checks for cluster services setup
Checking node reachability...
Node reachability check passed from node "ol6tocn1"
Checking user equivalence...
User equivalence check passed for user "oracle"
Checking node connectivity...
Checking hosts config file...
Verification of the hosts config file successful
Node connectivity passed for subnet "192.168.56.0" with node(s) ol6tocn1,ol6tocn2
TCP connectivity check passed for subnet "192.168.56.0"
Node connectivity passed for subnet "192.168.43.0" with node(s) ol6tocn1,ol6tocn2
TCP connectivity check passed for subnet "192.168.43.0"
Interfaces found on subnet "192.168.56.0" that are likely candidates for a private interconnect are:
ol6tocn1 eth0:192.168.56.41
ol6tocn2 eth0:192.168.56.42
Interfaces found on subnet "192.168.43.0" that are likely candidates for a private interconnect are:
ol6tocn1 eth1:192.168.43.41
ol6tocn2 eth1:192.168.43.42
WARNING:
Could not find a suitable set of interfaces for VIPs
Checking subnet mask consistency...
Subnet mask consistency check passed for subnet "192.168.56.0".
Subnet mask consistency check passed for subnet "192.168.43.0".
Subnet mask consistency check passed.
Node connectivity check passed
Checking multicast communication...
Checking subnet "192.168.56.0" for multicast communication with multicast group "224.0.0.251"...
Check of subnet "192.168.56.0" for multicast communication with multicast group "224.0.0.251" passed.
Check of multicast communication passed.
Checking ASMLib configuration.
Check for ASMLib configuration passed.
Total memory check failed
Check failed on nodes:
ol6tocn2,ol6tocn1
Available memory check passed
Swap space check passed
Free disk space check passed for "ol6tocn2:/usr,ol6tocn2:/var,ol6tocn2:/etc,ol6tocn2:/sbin,ol6tocn2:/tmp"
Free disk space check passed for "ol6tocn1:/usr,ol6tocn1:/var,ol6tocn1:/etc,ol6tocn1:/sbin,ol6tocn1:/tmp"
Check for multiple users with UID value 54321 passed
User existence check passed for "oracle"
Group existence check passed for "oinstall"
Group existence check passed for "dba"
Membership check for user "oracle" in group "oinstall" [as Primary] passed
Membership check for user "oracle" in group "dba" passed
Run level check passed
Hard limits check passed for "maximum open file descriptors"
Soft limits check passed for "maximum open file descriptors"
Hard limits check passed for "maximum user processes"
Soft limits check passed for "maximum user processes"
System architecture check passed
Kernel version check passed
Kernel parameter check passed for "semmsl"
Kernel parameter check passed for "semmns"
Kernel parameter check passed for "semopm"
Kernel parameter check passed for "semmni"
Kernel parameter check passed for "shmmax"
Kernel parameter check passed for "shmmni"
Kernel parameter check passed for "shmall"
Kernel parameter check passed for "file-max"
Kernel parameter check passed for "ip_local_port_range"
Kernel parameter check passed for "rmem_default"
Kernel parameter check passed for "rmem_max"
Kernel parameter check passed for "wmem_default"
Kernel parameter check passed for "wmem_max"
Kernel parameter check passed for "aio-max-nr"
Kernel parameter check passed for "panic_on_oops"
Package existence check passed for "binutils"
Package existence check passed for "compat-libcap1"
Package existence check passed for "compat-libstdc++-33(x86_64)"
Package existence check passed for "libgcc(x86_64)"
Package existence check passed for "libstdc++(x86_64)"
Package existence check passed for "libstdc++-devel(x86_64)"
Package existence check passed for "sysstat"
Package existence check passed for "gcc"
Package existence check passed for "gcc-c++"
Package existence check passed for "ksh"
Package existence check passed for "make"
Package existence check passed for "glibc(x86_64)"
Package existence check passed for "glibc-devel(x86_64)"
Package existence check passed for "libaio(x86_64)"
Package existence check passed for "libaio-devel(x86_64)"
Package existence check passed for "nfs-utils"
Checking availability of ports "6200,6100" required for component "Oracle Notification Service (ONS)"
Port availability check passed for ports "6200,6100"
Checking availability of ports "42424" required for component "Oracle Cluster Synchronization Services (CSSD)"
Port availability check passed for ports "42424"
Check for multiple users with UID value 0 passed
Current group ID check passed
Starting check for consistency of primary group of root user
Check for consistency of root user's primary group passed
Starting Clock synchronization checks using Network Time Protocol(NTP)...
No NTP Daemons or Services were found to be running
Clock synchronization check using Network Time Protocol(NTP) passed
Core file name pattern consistency check passed.
User "oracle" is not part of "root" group. Check passed
Default user file creation mask check passed
Checking integrity of file "/etc/resolv.conf" across nodes
"domain" and "search" entries do not coexist in any "/etc/resolv.conf" file
All nodes have same "search" order defined in file "/etc/resolv.conf"
The DNS response time for an unreachable node is within acceptable limit on all nodes
checking DNS response from all servers in "/etc/resolv.conf"
Check for integrity of file "/etc/resolv.conf" passed
Time zone consistency check passed
Checking integrity of name service switch configuration file "/etc/nsswitch.conf" ...
All nodes have same "hosts" entry defined in file "/etc/nsswitch.conf"
Check for integrity of name service switch configuration file "/etc/nsswitch.conf" passed
Checking daemon "avahi-daemon" is not configured and running
Daemon not configured check passed for process "avahi-daemon"
Daemon not running check passed for process "avahi-daemon"
Starting check for /dev/shm mounted as temporary file system ...
Check for /dev/shm mounted as temporary file system passed
Starting check for /boot mount ...
Check for /boot mount passed
Starting check for zeroconf check ...
Check for zeroconf check passed
Pre-check for cluster services setup was unsuccessful on all the nodes.
$
I have ignored "WARNING: Could not find a suitable set of interfaces for VIPs" and "total memory check failed". See what Helmut Hutlzer (Oracle Corp. employee working for RAC team) has written about suitable address for VIPS.
Note the important change with respect to 12.1.0.1 RAC prerequisites: the ASM diskgroup to store OCR and voting disk must also store the GIMR database: this implies that the minimum disk size is about 6 GB.
The 2 cluster node VM are now ready for Grid Infrastructure installation.