If you read the Oracle database documentation, it is not clear: I have not found any clear answer for this question.
The purpose of this blog article is to try to give an answer to this question by tracing system calls run by a dedicated server process to check if we see some I/O system call and by checking /proc/pid/fd for open file descriptors.
I have used 2 environments: both are running Oracle Linux 6.3 64-bit with Oracle Grid Infrastructure 11.2.0.3 and Oracle database 11.2.0.3 and have a database using ASM:
To understand the process I am going to use, you need first to understand the following:
1. Even if you use ASM, you need to know that it is not the ASM instance that is doing the I/O work. I/O requests are still managed by the database instance on metadata given by ASM instance to database instance. This is clearly stated in the Concepts Guide:"ASM instances manage the metadata of the disk group and provide file layout information to the database instances. Database instances direct I/O to ASM disks without going through an ASM instance."
2. It is the dedicated server process executing the SQL statement for a given database client that is going to read needed data in the right datafiles. This is also clearly stated in the Concepts Guide:"For example, these processes parse SQL queries, place them in the shared pool, create and execute a query plan for each query, and read buffers from the database buffer cache or from disk."
Start a new database session:
SQL> @gpid SQL> select spid 2 from v$process p, v$session s 3 where p.addr = s.paddr 4 and s.sid = (select sid from v$mystat where rownum=1); SPID ------------------------ 2969
Start strace on the related dedicated server process:
# strace -p 2969 -o 2969.log Process 2969 attached - interrupt to quit
Run some database queries:
SQL> select name from v$datafile; NAME -------------------------------------------------------------------------------- +DATA/pr11/datafile/system.256.823295401 +DATA/pr11/datafile/sysaux.257.823295403 +DATA/pr11/datafile/undotbs1.258.823295403 +DATA/pr11/datafile/users.259.823295405 SQL> select path from v$asm_disk; PATH -------------------------------------------------------------------------------- /dev/asm-disk1 /dev/asm-disk2 SQL>
Check open file descriptors in /proc file system for the related dedicated server process PID:
# ls -al /proc/2969/fd total 0 dr-x------ 2 root root 0 Aug 17 14:46 . dr-xr-xr-x 7 oracle sysasm 0 Aug 17 14:46 .. lr-x------ 1 root root 64 Aug 17 14:47 0 -> /dev/null l-wx------ 1 root root 64 Aug 17 14:47 1 -> /dev/null l-wx------ 1 root root 64 Aug 17 14:47 12 -> pipe:[107956] l-wx------ 1 root root 64 Aug 17 14:47 2 -> /dev/null lrwx------ 1 root root 64 Aug 17 14:47 256 -> /dev/asm-disk1 lr-x------ 1 root root 64 Aug 17 14:47 3 -> /dev/null lr-x------ 1 root root 64 Aug 17 14:47 4 -> /dev/null lr-x------ 1 root root 64 Aug 17 14:47 5 -> /u01/oracle/db11203/rdbms/mesg/oraus.msb lr-x------ 1 root root 64 Aug 17 14:47 6 -> /proc/2969/fd lr-x------ 1 root root 64 Aug 17 14:47 7 -> /dev/zero lrwx------ 1 root root 64 Aug 17 14:47 8 -> /u01/oracle/admin/PR11/adump/PR11_ora_2969_1.aud lr-x------ 1 root root 64 Aug 17 14:47 9 -> pipe:[107955] #
We see that first ASM disk has been opened which shows that ASM without ASMLib is using the open system call. In the strace trace file we also see that pread system call has been used to read data on this device:
# grep 'pread(256' 2969.log pread(256, "\25\302\1\1\4\2\272 \v&\312]\251"..., 16384, 1348485120) = 16384 pread(256, "\25\302\20<\5\377\377\1\4zc\200G\1"..., 16384, 1350565888) = 16384 pread(256, "\25\302\22<\5\377\377\1\4\305\177\346}221"..., 16384, 1350598656) = 16384 pread(256, "\25\302\30:\5\377\377\1\4\33@ \16"..., 16384, 1351614464) = 16384 pread(256, "\25\302\1\1\4\2\272 \v&\312]\251"..., 16384, 1348485120) = 16384 pread(256, "\25\302\20<\5\377\377\1\4zc\200G\1"..., 16384, 1350565888) = 16384 pread(256, "\25\302\22<\5\377\377\1\4\305\177\346}221"..., 16384, 1350598656) = 16384 pread(256, "\25\302 ;\5\377\377\1\4\365\34\3\3\2+D"..., 16384, 1352663040) = 16384 #
Note that you can sometimes get the open system call trace for the ASM disk but I have not been able to have a reproducible test case for that.
Start a new database session:
SQL> @gpid SQL> select spid 2 from v$process p, v$session s 3 where p.addr = s.paddr 4 and s.sid = (select sid from v$mystat where rownum=1); SPID ------------------------ 3967 SQL>
Start strace on the related dedicated server process:
# strace -p 3967 -o 3967.log Process 3967 attached - interrupt to quit
Run some database queries:
SQL> select name from v$datafile; NAME -------------------------------------------------------------------------------- +DATA/pr11/datafile/system.256.823471309 +DATA/pr11/datafile/sysaux.257.823471311 +DATA/pr11/datafile/undotbs1.258.823471313 +DATA/pr11/datafile/users.259.823471315 SQL> select path from v$asm_disk; PATH -------------------------------------------------------------------------------- ORCL:ASM1 ORCL:ASM2 SQL>
We can see that ASMLib has used the open system call to access a file that is not a ASM disk device file:
# ls -al /proc/3967/fd total 0 dr-x------ 2 root root 0 Aug 17 17:00 . dr-xr-xr-x 8 oracle asmadmin 0 Aug 17 17:00 .. lr-x------ 1 root root 64 Aug 17 17:01 0 -> /dev/null l-wx------ 1 root root 64 Aug 17 17:01 1 -> /dev/null lrwx------ 1 root root 64 Aug 17 17:01 10 -> /dev/oracleasm/iid/0000000000000003 l-wx------ 1 root root 64 Aug 17 17:01 12 -> pipe:[80866] l-wx------ 1 root root 64 Aug 17 17:01 2 -> /dev/null lr-x------ 1 root root 64 Aug 17 17:01 3 -> /dev/null lr-x------ 1 root root 64 Aug 17 17:01 4 -> /dev/null lr-x------ 1 root root 64 Aug 17 17:01 5 -> /u01/oracle/db11203/rdbms/mesg/oraus.msb lr-x------ 1 root root 64 Aug 17 17:01 6 -> /proc/3967/fd lr-x------ 1 root root 64 Aug 17 17:01 7 -> /dev/zero lrwx------ 1 root root 64 Aug 17 17:01 8 -> /u01/oracle/admin/PR11/adump/PR11_ora_3967_1.aud lr-x------ 1 root root 64 Aug 17 17:01 9 -> pipe:[80865] # # file /dev/oracleasm/iid/0000000000000003 /dev/oracleasm/iid/0000000000000003: empty # file /dev/oracleasm/disks/ASM1 /dev/oracleasm/disks/ASM1: block special # file /dev/oracleasm/disks/ASM2 /dev/oracleasm/disks/ASM2: block special #
There is also no pread system call trace in strace.log:
# grep pread 3967.log #
Actually this file has just been created according to its open system call trace:
# grep open 3967.log
open("/opt/oracle/extapi/64/asm/orcl/1/libasm.so", O_RDONLY) = 10
open("/dev/oracleasm/.check_iid", O_RDWR) = 10
open("/dev/oracleasm/iid/0000000000000003", O_RDWR|O_CREAT, 0770) = 10
[root@ol6elsa1 tmp]#
It is clear that ASM without ASMlib is using OS system calls to execute I/O requests whereas ASM with ASMLib is not using the same system calls to execute I/O requests. This means that ASMLib is not using OS system calls: this is confirmed by authors of Oracle Automatic Storage Management Under-the-Hood & Practical Deployment Guide who wrote page 88 that "ASM accesses disks through the standard OS interfaces used by Oracle to access any file (unless an ASMLIB is used)".