SplunkIt v2.0.2 Results & EC2 Storage Comparisons

With a new version of SplunkIt out the door, it was time to get new benchmark numbers, both for EC2 and the commodity hardware we tested in previous posts.  Now that SplunkIt is compatible with Windows and Splunk 5, we can get a good picture of how Splunk will perform across different operating systems and major Splunk versions.  Grab the new kit here.

All tests used a search user (splunkit-user) on a VMware ESX 4.1 instance with 1 Xeon 2.67GHz core, 2GB RAM, Linux 64bit, and Firefox 9.  All posted bonnie++ IOPS numbers were gathered by invoking bonnie++ in the following fashion, for those who are interested:

bonnie++ -d <PATH> -s <RAM*10> -qf -u root -n 50

In these first examples, we’ll see how your choice of EC2 storage may affect Splunk search & index performance.  We evaluated 3 different storage options: ephemeral instance storage, elastic block storage (EBS), and EBS with 800 provisioned IOPS.

As the results show, EBS provides a noticeable gain in search & index performance over ephemeral instance storage, on top of the redundancy that already comes with EBS.  Provisioned IOPS increases indexing performance even more, and should improve search performance as well if running many concurrent searches.  Standard EBS also appears to provide inconsistent I/O performance compared to provisioned IOPS, as the range of bonnie numbers & Amazon’s EBS documentation suggests.

Linux on EC2: c1.xlarge

System: EC2, c1.xlarge; CPU: 8 vCPU, 20 ECU; Memory: 7GB; OS: Linux 64-bit, Ubuntu 12.04LTS

Windows on EC2: c1.xlarge

System: EC2, c1.xlarge; CPU: 8 vCPU, 20 ECU; Memory: 7GB; OS: Windows Server 2008 R2 64-bit

Linux on EC2: other instance types

Storage: EBS; Bonnie IOPS: 90-220; OS: Linux 64-bit, Ubuntu 12.04LTS

These results highlight the substantial performance improvements introduced in Splunk 5 across linux and windows, especially using Elastic Block Storage (EBS) in EC2.  Down the road we may experiment with the hi1.4xlarge SSD instances & other EC2 optimizations to study their impact on searching & indexing.

Bare Metal: RAID 0 and RAID 1+0

We revisited the lab system we tested in our first results blog post, this time with SplunkIt v2.0.2.  We tested the kit with Splunk installed on RAID-0 and RAID-10 (1+0) partitions, with results posted below.

Indexing & search performance was comparable between the two, with RAID-10 providing a slight improvement in search times.  The mismatch in the number of disks – 6 in RAID-10 case, 12 in the RAID-0 case – may explain why RAID-10 didn’t outperform RAID-0 in all indexing tests.  We would expect a larger improvement in search performance from RAID-0 to RAID-10 if our data was spread across many more buckets.  Like the EC2 tests above, this test highlights the benefits of upgrading to Splunk 5.

System: HP DL380G7; CPU: 2×6 Xeon 2.67GHz; Memory: 12GB; OS: Linux 64-bit, Fedora 14

We’ll periodically post SplunkIt results for new configurations and hardware.  Feel free to send us questions, feedback, and your own test results at splunkit@splunk.com.