How to easily monitor VMware using OMS LogAnalytics with the OpsLogix VMware Management Pack, the extended way!
At OpsLogix, we’re always looking to integrate our products within the latest markets. We’ve already created a fantastic VMware Management Pack for SCOM that allows you to monitor your VMware environment without any extra software installations on your VMware boxes. Yep! You’re reading it correctly. No extra software installation on your VMware production servers! Sounds good doesn’t it?
Since OMS (Operations Management Suite) is a brand new product from Microsoft, we made sure to also integrate our VMware Management Pack into it. I will show you how easy it is to collect all your VMware logs and us it for your analytics.
But that’s not all.
We also collect over 60 performance metrics for you to use to analyze the VMware health state. For example, Host / VM memory and CPU usage and datastore space etc. and of course all audit / task / alarm / snmp etc. events. Meaning that you can use it to meet all the requirements of your security auditing (NEN, SOX, ISO & etc.) And what’s really neat about our VMware monitoring solution, is that it doesn’t require a vCenter installation. You can also directly connect to a ESX(i) host without almost losing any monitoring features.
Before we start we must meet the following pre requirements:
- A working SCOM 2012 R2 environment
- vCenter or ESXi server Boxes access (read only required at top-level)
- An OpsLogix VMware Management Pack & license
- The OpsLogix OMS Extension Management Pack
- A Microsoft account (it can be live.com / hotmail.com or office365)
Step 1 – Setting up the OMS account
To set up your OMS account follow the on-boarding Instructions on page 3 by clicking here. It will take a few minutes of your time, plus it’s free of charge! There is only a restriction on the daily upload limit of 500 MB of data and a retention time of 7 days. But it’s a decent start for VMware event/performance analytics.
Step 2 – Add your SCOM management group to OMS
After finishing the instructions in step 1, check if you’ve also added your SCOM management group to OMS. Go to Overview -> Settings -> Connected Sources. If everything is done correctly, you should see your management group listed as green:
Step 3 – Setup of the VMware Monitoring
Follow the quick steps below, and also make sure to read the installation guide included in the software package:
- Open the Operations Manager Console as a SCOM administrator and Import the “OpsLogix VMware Management Pack” files included in the software package.
- Apply the VMware License using the OpsLogix Licensing dashboard in the SCOM Operations Management Console.
- Add the VMware instances that you want to monitor. You can do this in the SCOM console using the OpsLogix VMware configuration dashboard.
- Add the vCenter connection or the direct ESX(i) Host connection.
- If you’ve successfully added the VMware connection, you should see the monitoring being populated. Just wait until you see the vCenter/ESX Connections state view become active with a health state.
Step 4 – Setup the OMS VMware Monitoring
In this step we’re going to enable the VMware Logs to the OMS collection.
- Import the OpsLogix OMS VMware extension management pack, using the SCOM console. This is a separate download and not included in the VMware Management Pack, see ‘Pre requirements’.
- By default, only VMware connections that are members of the SCOM group “OpsLogix VMware OMS group” are enabled to collect the logs and performance metrics.
Step 5 – Viewing the OMS VMware logs and performance collection
Now let’s check if the VMware log and performance metrics are picked up by OMS.
- Open a web browser and go to: https://login.mms.microsoft.com/signin.aspx?ref=ms_mms
- Login using the Microsoft account you created in the step “Setup the OMS account”
- The main OMS dashboard should appear. Select “Log Search”.
- Before we continue in the OMS portal we are going to generate some test VMware Alerts so that OMS can have collected them. As a test we are going to do a vCenter log on.
- Just open a vCenter connection and provide a correct log on. This will generate a VMware log event that will be collected in the OMS.
- If you have setup a direct ESX(i) connection just do the same using the Web based or telnet access.
- Wait for a minute or five to let OMS pick up the generated alert. The event OMS collection interval is per 60 seconds. If everything is working you should see an event in the log search. Please not that this log events will NOT be visible in SCOM because we don’t store this alert into the SCOM databases. Also remember that we are collecting ALL VMware messages, so if we would do this, due to the amount of log messages we could kill the performance/free space of the SCOM environment. However, the OMS environment is exactly configured to handle this big data without any problems!
- Next we test if the VMware performance metrics are collected. The query we use to get the performance metrics is: Type=Perf (ObjectName = VM*) or (ObjectName = Host*) or (ObjectName = Datastore*)
As you can see the test events are collected in OMS. Next we are going to let OMS collect information during a period of couple of days so that we have some data to work with.
Step 7 – VMware OMS search query’s
We’re going to be writing some nice OMS query’s that can help you analyse the VMware event logs. Here are some examples:
Event log related
|All VMware Events||Type=Event EventLog=VMWare|
|All VMware Events with only the message and time||Type=Event EventLog= VMWare | Select Source , TimeGenerated , RenderedDescription|
|All VMware Events With the word Test in it||Type=Event EventLog= VMWare Test | Select Source , TimeGenerated , RenderedDescription|
|Count the number of messages grouped by event source||Type=Event EventLog= VMWare | measure count() by Source|
|Count per vCenter the alerts generated||Type=Event EventLog= VMWare Source= AlarmActionTriggeredEvent | measure count() by Computer|
|All Failed user logons to the vCenter or ESX(i) Hosts.||Type=Event EventLog=VMWare (Source=UserLoginSessionEvent Or Source=BadUsernameSessionEvent) | measure count() by Source|
Performance metric related
|All VMware performance counters||Type=Perf (ObjectName = VM*) or (ObjectName = Host*) or (ObjectName = Datastore*)|
|Virtual Machine CPU Usage Top over active time range||ObjectName=”VM.CPU” InstanceName=percent CounterName=”% Average CPU time” | measure percentile99(CounterValue) by Computer|
|VMware Host CPU Usage >80% over active time range||ObjectName=”Host.CPU” InstanceName=percent CounterName=”% CPU Usage” InstanceName=percent CounterValue > 80 |measure percentile99(CounterValue) by Computer|
|VMware Host Mem Usage >80% over active time range||Type=Perf ObjectName=”Host.Memory” InstanceName=percent CounterName=”Average % Memory Usage” CounterValue > 70 | measure percentile99(CounterValue) by Computer|
|Datastore % used space with a 1hour aggeration .||ObjectName=Datastore CounterName=”Datastore % used space” InstanceName=Total | measure percentile99(CounterValue) by Computer interval 1hour|
|Datastore % used space Top over active time range||ObjectName=Datastore CounterName=”Datastore % used space” InstanceName=Total | measure percentile99(CounterValue) by Computer|
Please keep in mind to also use the Table view instead of the default List view. In some outputs it makes it easier to read. You can also export it to Excel, which is a good option.
Step 8 – Dashboards
Last but not least we’ve made it possible to combine the most important information to be displayed into one dashboard as an overview.
An example of an end result could be as shown below:
Notice this example is not public yet, but in the meanwhile you can use the ‘normal’ OMS dashboard feature as an overview.
We’re done! Easy right? Hopefully this post was useful and you discovered the Power of OMS and VMware together using the OpsLogix VMware Management Pack.
This post is a working example of OMS and VMware. Since the OMS platform is still being extended it is possible that our solution has to be changed to reflect the OMS changes. At writing time all features described in this post are working. The described OpsLogix OMS extension Management Pack at this time is available for free to all customers having an OpsLogix VMware Management Pack license.