Correlating Windows and VMware Host Information

When you install a new virtual host on VMware, you get to give it any name you want. The name has nothing to do with what is running on the host. How can we go from the Windows information to the VMware information? We’re here to help.

Let’s take a look at the VMware side of things for a moment. If you have the Splunk App for VMware installed, then you likely already have this information. The sourcetype is “vmware:inv:vm” and there is one event for every virtual host in there. Since we need a common field on which to correlate, I’m going to choose the network interface MAC Address. The “vmware:inv:vm” event is JSON data, so we need to use the spath command to extract the right information:

sourcetype=vmware:inv:vm macAddress 
    | spath moid 
    | spath output=mac path=changeSet.guest.net{}.macAddress 
    | spath output=vm_name path=changeSet.name 
    | stats values(mac) as mac by vm_name,host,moid 
    | mvexpand mac

What you will get is a table that provides the hypervisor and the name of the VM on that hypervisor for any given MAC address. However, we want to go further. We want to also correlate that information to the host information that is coming in from the Windows hosts. The Windows Universal Forwarder has a WinHostMon data input that we can use to provide this information. Since I generally recommend that all Windows hosts get the Splunk_TA_windows, I would also recommend you place this input definition in the local/inputs.conf file for this TA.

[WinHostMon://networkAdapter]
interval = 86400
sourcetype = WinHostMon:NetworkAdapter
type = networkAdapter

When this runs, you will get the MAC address of the network adapter together with the Windows host field.

03/04/2014 09:58:07.486 
Type=NetworkAdapter 
Name="Intel(R) 82579LM Gigabit Network Connection" 
ComputerName=ACME-001 
Manufacturer="Intel" 
ProductName="Intel(R) 82579LM Gigabit Network Connection" 
Status="OK" 
MACAddress="00:50:56:BE:73:A9"

We can set up a similar lookup to VMware as follows:

sourcetype=WinHostMon:NetworkAdapter 
    | stats values(MACAddress) as MACAddress by host 
    | mvexpand MACAddress

The only gotcha is that you will find that the MACAddress on the Windows side of things is one thing (00:50:56:BE:73:A9) and on the VMware side is something slightly different (00:50:56:be:73:a9). You need to choose one of them and convert the other. Fortunately, it’s a simple conversion. I chose altering the Windows side of things as follows:

sourcetype=WinHostMon:NetworkAdapter
    | eval mac=lower(MACAddress)
    | stats values(mac) as mac by host
    | mvexpand mac

Now we can correlate the two together. My preferred form of this is to generate a combined lookup that has the host, mac, vCenter host name and vm_name in the lookup:

(sourcetype=WinHostMon:NetworkAdapter) OR (sourcetype=vmware:inv:vm macAddress)
    | eval mac=lower(MACAddress)
    | spath moid 
    | spath output=mac path=changeSet.guest.net{}.macAddress 
    | spath output=vm_name path=changeSet.name 
    | eval vcenter=if(sourcetype="vmware:inv:vm",host,null())
    | eval w=if(sourcetype="WinHostMon:NetworkAdapter",host,null())
    | eval hostSystem=vcenter+"-"+moid
    | stats values(mac) as mac,values(w) as winhost by vm_name,hostSystem 
    | mvexpand mac

So, what can you do with this information. Well, the Splunk App for VMware has a nice drill down that allows you to drill into the host details from anywhere. In Simple XML you can add a <drilldown> target to /app/splunk_for_vmware/vm_detail?selectedVirtualMachine=VCENTER-MOID. I’ve included a hostSystem field for this purpose. All you need to do is to add a lookup to the end of your search:

...your-search... | lookup vmnethost host as winhost OUTPUT hostSystem

Now you can do the drilldown linkage within Simple XML. Clicking on your table takes you right into the virtual machine details page for your Windows server in the context of VMware. From there, you can start exploring the data in a VMware context.