The performance numbers obtained here with RapidDisk will always be consistent as opposed to the Flash memory where over time, and once the Flash storage enters its Programmable Erase (PE) cycles, the speeds drastically drop and provide inconsistent results. You can observe some basic numbers below. Note that sequential and random access I/O to both DRAM and Flash based technologies is transparent as there are no movable components to distinguish the two to provide differing results. The results that you see below for the sequential access I/O will not differ from the numbers obtained in random access I/O.
Contents |
Summary of Numbers Posted Below
| Profile | RxDSK | SATA (mechanical HDD) |
|---|---|---|
| 1M xfer 1G file 32 depth |
Seq Reads: 3.3 GB/s Seq Writes: 2.9 GB/s Ran Reads: 3.3 GB/s Ran Writes: 2.9 GB/s |
Seq Reads: 154 MB/s Seq Writes: 131 MB/s Ran Reads: 93 MB/s Ran Writes: 106 MB/s |
| 4K xfer 1G file 32 depth |
Seq Reads: 1.7 GB/s Seq Writes: 1.5 GB/s Ran Reads: 1.3 GB/s Ran Writes: 1.4 GB/s |
Seq Reads: 116 MB/s Seq Writes: 70 MB/s Ran Reads: 2.2 MB/s Ran Writes: 1.9 MB/s |
| Profile | RxCACHE | SATA (mechanical HDD) |
|---|---|---|
| 4K xfer 32M file 32 depth |
Seq Reads: 802 MB/s Seq Writes: 68 MB/s Ran Reads: 907 MB/s Ran Writes: 2.8 MB/s |
Seq Reads: 104 MB/s Seq Writes: 70 MB/s Ran Reads: 7.9 MB/s Ran Writes: 2.8 MB/s |
Note - Using RapidCache with 1 MByte transfers on the same hardware, reading cached data both sequentially and randomly can get as high as 1.0 - 1.2 GB/s and a bit more.
Using Fusion I/O's fio Benchmark Utility for RapidDisk
These numbers were obtained using a recent 64-bit system running a 3.1 GHz Xeon Quad Core processor. I used fio 2.0.3 and ran the test on volumes formatted and mounted with an Ext4 file system. Each job file ran 4 processes (one at a time): (1) Sequential Reads, (2) Random Reads, (3) Sequential Writes, and (4) Random Writes. Also note that I formatted the output to show each process group and results grouped together.
1M Transfer Size to a 1G File with up to 32 threads
On average the RapidDisk is consistent with sequential/random reads/writes at approximately 3.3 GB/s. The local magnetic SATA drive is performing sequential read/write operations at about 150 MB/s and less and the random at about 100 MB/s. Huge difference, huh? That is over 30x the speed!!!
Using a RapidDisk Volume
seq-read: (g=0): rw=read, bs=1M-1M/1M-1M, ioengine=libaio, iodepth=32 Run status group 0 (all jobs): READ: io=1055.0MB, aggrb=3187.4MB/s, minb=3263.9MB/s, maxb=3263.9MB/s, mint=331msec, maxt=331msec rand-read: (g=1): rw=randread, bs=1M-1M/1M-1M, ioengine=libaio, iodepth=32 Run status group 1 (all jobs): READ: io=1024.0MB, aggrb=3180.2MB/s, minb=3256.5MB/s, maxb=3256.5MB/s, mint=322msec, maxt=322msec seq-write: (g=2): rw=write, bs=1M-1M/1M-1M, ioengine=libaio, iodepth=32 Run status group 2 (all jobs): WRITE: io=1005.7MB, aggrb=2865.2MB/s, minb=2933.1MB/s, maxb=2933.1MB/s, mint=351msec, maxt=351msec rand-write: (g=3): rw=randwrite, bs=1M-1M/1M-1M, ioengine=libaio, iodepth=32 Run status group 3 (all jobs): WRITE: io=68608KB, aggrb=2791.7MB/s, minb=2858.7MB/s, maxb=2858.7MB/s, mint=24msec, maxt=24msec
Using the Local Magnetic SATA II Volume
seq-read: (g=0): rw=read, bs=1M-1M/1M-1M, ioengine=libaio, iodepth=32 Run status group 0 (all jobs): READ: io=1055.0MB, aggrb=150861KB/s, minb=154482KB/s, maxb=154482KB/s, mint=7161msec, maxt=7161msec rand-read: (g=1): rw=randread, bs=1M-1M/1M-1M, ioengine=libaio, iodepth=32 Run status group 1 (all jobs): READ: io=1024.0MB, aggrb=91722KB/s, minb=93924KB/s, maxb=93924KB/s, mint=11432msec, maxt=11432msec seq-write: (g=2): rw=write, bs=1M-1M/1M-1M, ioengine=libaio, iodepth=32 Run status group 2 (all jobs): WRITE: io=1055.0MB, aggrb=127456KB/s, minb=130515KB/s, maxb=130515KB/s, mint=8476msec, maxt=8476msec rand-write: (g=3): rw=randwrite, bs=1M-1M/1M-1M, ioengine=libaio, iodepth=32 Run status group 3 (all jobs): WRITE: io=1024.0MB, aggrb=103114KB/s, minb=105589KB/s, maxb=105589KB/s, mint=10169msec, maxt=10169msec
4K Transfer Size to a 1G File with up to 32 threads
On average the RapidDisk is consistent with sequential/random reads/writes at between 1.3 - 1.7 GB/s. The local magnetic SATA drive is performing sequential read/write operations at about 100 MB/s and the random at 2 MB/s and less. Still a huge difference, huh?
Using a RapidDisk Volume
seq-read: (g=0): rw=read, bs=4K-4K/4K-4K, ioengine=libaio, iodepth=32 Run status group 0 (all jobs): READ: io=1024.2MB, aggrb=1528.6MB/s, minb=1565.3MB/s, maxb=1565.3MB/s, mint=670msec, maxt=670msec rand-read: (g=1): rw=randread, bs=4K-4K/4K-4K, ioengine=libaio, iodepth=32 Run status group 1 (all jobs): READ: io=1024.0MB, aggrb=1302.9MB/s, minb=1334.7MB/s, maxb=1334.7MB/s, mint=786msec, maxt=786msec seq-write: (g=2): rw=write, bs=4K-4K/4K-4K, ioengine=libaio, iodepth=32 Run status group 2 (all jobs): WRITE: io=1005.7MB, aggrb=1449.8MB/s, minb=1483.9MB/s, maxb=1483.9MB/s, mint=694msec, maxt=694msec rand-write: (g=3): rw=randwrite, bs=4K-4K/4K-4K, ioengine=libaio, iodepth=32 Run status group 3 (all jobs): WRITE: io=68608KB, aggrb=1351.8MB/s, minb=1382.1MB/s, maxb=1382.1MB/s, mint=701msec, maxt=701msec
Using the Local Magnetic SATA II Volume
seq-read: (g=0): rw=read, bs=4K-4K/4K-4K, ioengine=libaio, iodepth=32 Run status group 0 (all jobs): READ: io=1024.2MB, aggrb=112872KB/s, minb=115581KB/s, maxb=115581KB/s, mint=9291msec, maxt=9291msec rand-read: (g=1): rw=randread, bs=4K-4K/4K-4K, ioengine=libaio, iodepth=32 Run status group 1 (all jobs): READ: io=129016KB, aggrb=2146KB/s, minb=2197KB/s, maxb=2197KB/s, mint=60111msec, maxt=60111msec seq-write: (g=2): rw=write, bs=4K-4K/4K-4K, ioengine=libaio, iodepth=32 Run status group 2 (all jobs): WRITE: io=1024.2MB, aggrb=69093KB/s, minb=70751KB/s, maxb=70751KB/s, mint=15178msec, maxt=15178msec rand-write: (g=3): rw=randwrite, bs=4K-4K/4K-4K, ioengine=libaio, iodepth=32 Run status group 3 (all jobs): WRITE: io=111336KB, aggrb=1853KB/s, minb=1898KB/s, maxb=1898KB/s, mint=60066msec, maxt=60066msec
Using Fusion I/O's fio Benchmark Utility for RapidCache
These numbers were obtained using an recent 64-bit system running an Intel Xeon 3.1 GHz Qaud Core processor. I used fio 2.0.3 and ran the test on volumes formatted and mounted with an Ext2 file system. Each job file ran 4 processes (one at a time): (1) Sequential Reads, (2) Random Reads, (3) Sequential Writes, and (4) Random Writes. Also note that I formatted the output to show each process group and results grouped together.
4K Transfer Size to a 32M File with up to 32 threads
This is configured with a 1 GByte rxdsk mapping to a 10 GByte partition on a 7.2k 3.0 Gbps spinning magnetic SATA disk drive. On average the RapidCache mapping is consistent with sequential/random reads at about 802 - 907 MB/s. A very high value when you see the results below for the non-cached reads (sequential: 104 MB/s and random 8 MB/s). Huge difference, huh? The writes are what is expected for write-through operation, that they are no different from writing straight to the disk device.
Using a RapidCache Mapping
seq-read: (g=0): rw=read, bs=4K-4K/4K-4K, ioengine=libaio, iodepth=32 Run status group 0 (all jobs): READ: io=32892KB, aggrb=783142KB/s, minb=801938KB/s, maxb=801938KB/s, mint=42msec, maxt=42msec rand-read: (g=1): rw=randread, bs=4K-4K/4K-4K, ioengine=libaio, iodepth=32 Run status group 1 (all jobs): READ: io=32768KB, aggrb=885621KB/s, minb=906876KB/s, maxb=906876KB/s, mint=37msec, maxt=37msec seq-write: (g=2): rw=write, bs=4K-4K/4K-4K, ioengine=libaio, iodepth=32 Run status group 2 (all jobs): WRITE: io=32892KB, aggrb=65915KB/s, minb=67497KB/s, maxb=67497KB/s, mint=499msec, maxt=499msec rand-write: (g=3): rw=randwrite, bs=4K-4K/4K-4K, ioengine=libaio, iodepth=32 Run status group 3 (all jobs): WRITE: io=32768KB, aggrb=2790KB/s, minb=2857KB/s, maxb=2857KB/s, mint=11741msec, maxt=11741msec
Writing straight to the drive without a RapidCache Mapping
seq-read: (g=0): rw=read, bs=4K-4K/4K-4K, ioengine=libaio, iodepth=32 Run status group 0 (all jobs): READ: io=32892KB, aggrb=102149KB/s, minb=104600KB/s, maxb=104600KB/s, mint=322msec, maxt=322msec rand-read: (g=1): rw=randread, bs=4K-4K/4K-4K, ioengine=libaio, iodepth=32 Run status group 1 (all jobs): READ: io=32768KB, aggrb=7726KB/s, minb=7911KB/s, maxb=7911KB/s, mint=4241msec, maxt=4241msec seq-write: (g=2): rw=write, bs=4K-4K/4K-4K, ioengine=libaio, iodepth=32 Run status group 2 (all jobs): WRITE: io=32892KB, aggrb=68668KB/s, minb=70316KB/s, maxb=70316KB/s, mint=479msec, maxt=479msec rand-write: (g=3): rw=randwrite, bs=4K-4K/4K-4K, ioengine=libaio, iodepth=32 Run status group 3 (all jobs): WRITE: io=32768KB, aggrb=2796KB/s, minb=2863KB/s, maxb=2863KB/s, mint=11719msec, maxt=11719msec
Note - Using RapidCache with 1 MByte transfers on the same hardware, reading cached data both sequentially and randomly can get as high as 1.0 - 1.2 GB/s and a bit more.
Using a Single Process of dd
These numbers were obtained using a recent 64-bit 3.1 GHz Xeon Quad Core processor based system.
Writes to the Physical Device node
Writes totaling 1 Gbyte of data to the physical rxdsk device node, at 512 bytes of transfer sizes:
$ sudo dd if=/dev/zero of=/dev/rxd0 dd: writing to `/dev/rxd0': No space left on device 2097153+0 records in 2097152+0 records out 1073741824 bytes (1.1 GB) copied, 1.90328 s, 564 MB/s
At 1 Kbytes of transfer sizes:
$ sudo dd if=/dev/zero of=/dev/rxd0 bs=1K dd: writing `/dev/rxd0': No space left on device 1048577+0 records in 1048576+0 records out 1073741824 bytes (1.1 GB) copied, 1.32526 s, 810 MB/s
At 4 Kbytes of transfer sizes:
$ sudo dd if=/dev/zero of=/dev/rxd0 bs=4K dd: writing `/dev/rxd0': No space left on device 262145+0 records in 262144+0 records out 1073741824 bytes (1.1 GB) copied, 0.667642 s, 1.6 GB/s
At 1 Mbytes of transfer sizes:
$ sudo dd if=/dev/zero of=/dev/rxd0 bs=1M dd: writing `/dev/rxd0': No space left on device 1025+0 records in 1024+0 records out 1073741824 bytes (1.1 GB) copied, 0.717855 s, 1.5 GB/s
At 4 Mbytes of transfer sizes:
$ sudo dd if=/dev/zero of=/dev/rxd0 bs=4M dd: writing `/dev/rxd0': No space left on device 257+0 records in 256+0 records out 1073741824 bytes (1.1 GB) copied, 0.92569 s, 1.2 GB/s
Notice that the total time taken is reduced to less than 1 second for a whole 1.1 GByte write when the transfer sizes align to the system's page sizes. A How many hard drives (including Flash-based SSDs do you know that can write over 1 GBytes of data in less than 1 second?!?
Writes to a File Over a File System
Writes to a file over a file system labeled on the rxdsk RAM disk and totaling 1 Gbyte, at 512 bytes of transfer sizes:
$ sudo dd if=/dev/zero of=/mnt/rxc/rxd.dat count=2097152 dd: writing to `/mnt/rxc/rxd.dat': No space left on device 2059585+0 records in 2059584+0 records out 1054507008 bytes (1.1 GB) copied, 1.55186 s, 680 MB/s
At 1 Kbytes of transfer sizes:
$ sudo dd if=/dev/zero of=/mnt/rxc/rxd.dat bs=1K count=1048576 dd: writing `/mnt/rxc/rxd.dat': No space left on device 1029793+0 records in 1029792+0 records out 1054507008 bytes (1.1 GB) copied, 1.03899 s, 1.0 GB/s
At 4 Kbytes of transfer sizes:
$ sudo dd if=/dev/zero of=/mnt/rxc/rxd.dat bs=4K count=262144 dd: writing `/mnt/rxc/rxd.dat': No space left on device 257449+0 records in 257448+0 records out 1054507008 bytes (1.1 GB) copied, 0.65012 s, 1.6 GB/s
At 1 Mbytes of transfer sizes:
$ sudo dd if=/dev/zero of=/mnt/rxc/rxd.dat bs=1M count=1024 dd: writing `/mnt/rxc/rxd.dat': No space left on device 1006+0 records in 1005+0 records out 1054507008 bytes (1.1 GB) copied, 0.628955 s, 1.7 GB/s
At 4 Mbytes of transfer sizes:
$ sudo dd if=/dev/zero of=/mnt/rxc/rxd.dat bs=4M count=256 dd: writing `/mnt/rxc/rxd.dat': No space left on device 252+0 records in 251+0 records out 1054507008 bytes (1.1 GB) copied, 0.862366 s, 1.2 GB/s
Again, notice that the total time taken is reduced to less than 1 second for a whole 1.1 GByte write when the transfer sizes align to the system's page sizes. Although the file system will provide some overhead processing, but it is still barely noticeable to the user. These are still high performing speeds providing almost instant results.