It seems like I just blogged about the general availability of the Splunk App for VMware 2.0. With typical Splunk speed, I’m here to introduce you to the latest and greatest of our now generally available 3.0 version of the App.
This release is all about making it fast, easy and efficient. Allow me to elaborate.
Easy to configure and manage:
The App now boasts of GUI-based configuration. Configure the collection nodes, data and log collection from one or more VCs and whitelist/blacklist hosts – all from the interface built into the App.
Installing the App is easy. In our beta cycles, the App took less than 60 minutes to get up and running across enterprise-class VMware deployments. One of our customers, RMS installed the App in over 265 ESXi hosts easily with clicks of buttons across their dev and production instances.
To validate correctness of incoming data and ongoing maintenance, we have install health reports that provide you detailed insights into the type of data coming in, last time data was collected, type of data received, data collection time stamps, data volumes by individual sourcetype and much more.
Easy to scale:
If you’ve been using our 2.0 version of the App, you know about the virtual forwarder appliance. The appliance collects performance and log data from the ESX/i hosts and tasks, events and topology information from VC.
This version has introduced the concept of a Data Collection Node (DCN), which replaces the virtual appliance from 2.0. The DCN collects performance metrics, tasks and events, inventory, hierarchy and topology information – all from VC.
This DCN ships as an appliance but can also be customized and installed on physical machines to increase scalability.
Why you ask? First, we’re adopting the VMware recommended approach, to benefit from the efficiencies built into VC since it already collects the metrics from the ESXi hosts. Second, when collecting data from the ESXi hosts, the vSphere API requires service accounts for each host. Customers have expressed their reluctance in having to configure one ESXi host at a time. And we listened.
Worried about performance impact to your VC? Don’t!
First, VC operations are prioritized over API calls. So, the API calls from the App do not limit or delay normal VC operations.
Second, the key really is that while the App continues to collect performance metrics at the deepest level of granularity (at 20-second intervals), we collect it every 3 minutes. Other types of metrics such as resources and clusters are collected less frequently and only deltas from a previously logged collection time-stamp are collected for inventory and hierarchy metrics. With all this data collection intelligence built into the App, we minimize the number of API calls made to VC.
Therefore, don’t fret about impact to VC performance and its operations.
Reduced data volumes:
All these efficiencies we’ve built into the data collection brings forth additional benefits – significant and dramatic reduction in data volumes. We’ve published numbers for your reference (and this includes ALL THE DATA the app collects, including logs from VC and ESXi hosts, performance metrics at 20-second granularity, tasks, events, inventory and hierarchy). Here is a gist.
vCenter Logs: ~15 MB per host per day per vCenter
ESXi host logs: ~135-235 MB per day
Host performance data: 10 MB per day
VM performance data: 3MB per day
Therefore, this averages out to ~250-300MG per host per day.
This makes data volume sizing a lot more predictable and with these kinds of numbers, it significantly reduces your license costs (we’re just thoughtful like that!). With the Splunk platform and the App for VMware, you can get comprehensive insights into your VMware infrastructure and other IT tiers, all within one single console.
Accelerated reporting:
In Splunk enterprise 5.0, we introduced the concept of tsidx stats – a time-series index file that stores data in a columnar structure. We’ve leveraged this capability in this release of the App allowing for faster aggregation and query performance on ad-hoc searches.
We’ve also made changes to the data formats. This not only makes it easier to navigate, extract and read the data, but it improves data extraction times thus accelerating reporting.
Industry standard adoption:
Logs from ESXi hosts and VCs are now captured via syslog. This also means that you can export logs in syslog format. Let’s say you need to send logs to VMware support, a simple search query and voila, send it in syslog format to VMware support. Our topology-based log browser reports, carried over from 2.0, allow you to narrow down log-related exceptions by affected VMware hierarchy components. You can use all this data to capture unique exceptions such as duplicate IPs, ISCSI reservation errors, errors related to storage access and much more.
This App is available for a 90-day free trial on Splunk Apps (splunkbase). So, get started right away. Access our documentation and release notes for guidance. If you have any questions, we’re here to help.
This release is an important step for us to get to where we want to be. So, yes, we’re very proud of this release. Having said that, we also believe in continual improvement. So, if you have more suggestions or feedback, email us at vmware@splunk.com.
Have fun with the App y’all. And if you’re coming to .conf2013, see you there.