Exporting Content with vRealize Operations Manager Policies

Policies as we know are the control center of vRealize Operations Manager. In the vRealize Operations Webinar Series, we have been running for the entire year, we have seen that policies needs to be touched, configured, modified etc, with every feature or functionality available out of the box or being customized with vROps. This pretty much aligns with all the products of VMware. They all have a theme of Policy Based Control which allows you to built, run and operate the Software Defined Datacenter components by tying them down to your business policies.

With this article, I wanted to highlight, that policies is not about configuration only, they are also about content. In a recent project, I discovered that with vROps 6.2.1, you can utilize policies for exporting and importing content from one vROps instance to another. This includes:-

• Alert Definitions

• Symptom Definitions

• Recommendations, &,

• SuperMetrics

This in my opinion is a killer feature as it allows you to create content once and distribute the same by a simple export and import of a policy. This opens a lot of avenues to collaborate and share meaningful content with community.

While the above mentioned content can be exported & imported easily, things like Views, Dashboards and Reports still need to be exported and imported individually. I believe with time, VMware should work towards unifying & ease content management by making everything exportable and importable like LogInsight content packs.

I will not go into the details of how to Export & Import policies, since I have already done an article on this sometime back which is applicable today as well. I will utilize this feature and work on creating a collaborative content library where you can share content with policies.

Stay tuned for more fun on vROps!!!

vROps Webinar 2016 - Part 6 : Understanding vROps SuperMetric & Views

Just completed the editing and upload of the the latest episode of the vROps Webinar Series 2016. With this we have completed 6 sessions this year. I am glad that we are able to dish out sessions every month and the interest level in these sessions just keeps on growing. Looking forward to deliver more content for the rest of the year.

Session Details:- In this episode, we talk about content within vROps. To begin with, we speak about Views and SuperMetrics. With this you will get a lot of insights into how you can use these features to build some useful, Reports and Dashboards within your vROps environments. 

Once again I would like to thank my friend and partner in this project Simon as without him this would not be possible.

So without further ado, here is the recording for this session:

Note : It is recommended that you watch the video in HD quality for a great experience.

vROps Webinar 2016 - Announcing Part 6 : Understanding vROps SuperMetric & Views

Time to announce the next part of the year long webinar series on vRealize Operations Manager. This time around, Simon and I will take you to the world of Content within vROps. I will be honest in saying that this is an ocean in itself and we might not be able to touch upon all the aspects of developing content within vROps. During this effort we plan to start with giving you an overview of the various options available to develop content within vROps.

Among all those options available, we would talk in detail about SuperMetrics and Views and would continue to talk about other option such as Dashboards and Reports in the upcoming sessions. We want to ensure that we go through all the options so that it can help you develop useful content within your environments.

Day & Date          : Monday, 27^th June 2016

Time                     : 1:30 PM - 2:30 PM  (SGT)

Event                    : vROps Webinar 2016

Topic                     : Part 6 : Understanding vROps SuperMetric & Views

Speakers               : Simon Eady / Sunny Dua

WebEx Link          : JoinWebEx Meeting

NOTE - Don't forget to mark your calendars by saving the Date!! Feel free to forward the invite to anyone who might be interested. It's open to all!!


Sharing & Spread the Knowledge!!

Creating a Capacity & Performance Management Dashboard in vRealize Operations 6.x

A few days back I tweeted about a hybrid dashboard which would help you gauge the Capacity & Measure the performance of your virtual infrastructure from a single pane. I call it the Capacity & Performance Management Dashboard which has been jointly built by Iwan Rahabok & myself. 

In my last post I wrote about the Help Desk Dashboard which was discussed during our VMworld session. This dashboard was also discussed in the VMworld session which me & Iwan presented and with this article, I will quickly run through the description of this dashboard, the use cases it solves and finally a quick and easy method to import this into your instances of vRealize Operations Manager. This dashboard does have 15 Super Metrics, 6 Views and 1 Customized XML file which helps you build the entire dashboard. While if you build this from scratch, creating this dashboard can take hours, but I promise you will spend under 10 minutes to operationalize this in your environment.

To begin with let's have a quick look at the dashboard. :-

Here is a quick description of all the widgets of this dashboard:-

1- This lists all the clusters you are monitoring through vRealize Operations Manager 6.x

2- This widget display the key capacity metrics from a CPU, Memory & Storage perspective.

3- This widget provides the Peak & Average CPU Usage by an Virtual Machine in the selected cluster. This would basically give you an idea of how the virtual machines are using CPU in your environment. Is it Too Busy, Less Busy etc.

4- While the previous widget measure the usage, this widget looks at Peak & Average CPU Contention% which defines whether the demand of the CPU is being met without Contention at the CPU. Contention% is the key metric to measure CPU performance. 

5- This widget provides the Peak & Average Memory Usage by an Virtual Machine in the selected cluster. This would basically give you an idea of how the virtual machines are using RAM in your environment. Is it Too Busy, Less Busy etc.

6- Similar to CPU Contention, for Memory performance you need to measure the Memory Contention %. With this widget we measure the Peak & Average memory contention experienced by any virtual machine in the cluster. The contention will be seen as soon as you have memory over-commitment and more demand than supply which is not a great sign.

7- This widget shows the Peak & Average IOPS done by any virtual machine in the selected cluster. 

8- Alongside the IOPS you can see the Peak and Average Virtual Disk latency experienced by the virtual machines.

The above widgets showcase a number of metrics and most of them are Supermetrics which I will help you create in your environment!!

Here are the list of use cases where this dashboard can help you:-

1- Performance SLA Monitoring - If you have an agreed Performance SLA with the business measured in CPU & Memory Contention and Disk Latency then you can easily keep a track with this dashboard.

2- Performance Trending - Since this dashboard allows you to view the data from the past you can easily go back in time and look at how performance metrics trend in your environment.

3- Performance Driven Capacity Planning - This one is my favorite as you can now take capacity decisions based on your performance metrics. This basically means that if you need to ensure that as soon as you see performance metrics.

4- Capacity Overview - The capacity piece is quite obvious from the scorecard which talks about the consolidation ratios, summary of resources and the capacity remaining. With this in place you can easily see what kind of performance you are getting with over commitment of resources.

So if these reasons are compelling for you, let's go ahead and get you this dashboard. Go ahead and download the following files by clicking on the links below. Once you have these downloaded, we will begin creating the dashboard.

SUPERMETRICS

VIEWS

DASHBOARD

One you have these files, let's start with creating the dashboard. Follow the steps below:

IMPORT SUPERMETRICS & ENABLE THEM IN THE POLICY

1- Login to vROps with an account with Admin Privileges. Click on Content -> Super Metrics. Click on the Blue Wheel Icon and click on Import Super Metrics.

2 - Click the Browse button to go to SUPERMETRICS folder which you downloaded before and you need to select each super metric one by one and click on Open to Import all of them ONE by ONE.

3- One you import all of them, you will see a total of 15 Super Metrics in the list. Notice that these Super Metrics are already attached to the vCenter Adapter at the Cluster Compute Resource level.

4- Now let's enable these Super Metrics for collection in the Default Policy. Click on Administration -> Policies -> Policy Library -> Select the Default Policy - > Click on the Pencil shaped icon to Edit the policy.

5- Now click on the Collect Metrics and Properties option in the Edit Monitoring Policy Wizard and click on Collapse to see all the columns.

6- Click on Attribute type and uncheck everything except Supermetric to list all the of them.

7- In the Object Type select Cluster Compute Resource under the vCenter Adpater to list all the Super Metrics attached to a Cluster Object and you should see all the 15 super metrics which we imported earlier.

8- Finally, just select all the 15 metrics using CTRL + Mouse Left Click and click on Actions -> Enable to enable collection on all these super metrics.

Now that your Super Metrics are Ready, let's import the Views after 5 minutes or so.


IMPORTING VIEWS

 1- Click on Content -> Views -> Blue Wheel Icon -> Import

2- Browse to the file ALLVIEWS.xml which you downloaded before and click on open. Once the Views are imported you will see a successful message. Click on Done.

This would give you 6 new Views which the dashboard uses. Now let's create an XML for the Capacity Score card.

XML CREATION

1- Click on Content -> Manage Metric Config (You need vROps 6.0.2 or above), highlight the ResKndMetric Directory and click on the Green + Sign to add an XML.

2- Name the XML as CAPACITY-DATA.xml and click on OK.

3- Here is the XML you need to copy on the right pane. Please ensure that you use the right XML here. You have the following choices.

Choice 1 - If you have done a fresh install of vROps 6.1 then use this xml highlighted in BLUE Color:

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<AdapterKinds>
<AdapterKind adapterKindKey="VMWARE">
<ResourceKind resourceKindKey="ClusterComputeResource"> 

<Metric attrkey="summary|total_number_hosts" label="ESXi Hosts" unit=" " yellow="63" orange="64" red="65" />
<Metric attrkey="summary|total_number_datastores" label="Datastores" unit=" " yellow="255" orange="256" red="257" />
<Metric attrkey="summary|total_number_vms" label="VMs and Templates" unit=" " yellow="5000" orange="6000" red="8000" />

<Metric attrkey="Super Metric|sm_a5a9c11a-f27c-446c-8bd5-48a11124b543" label="Total Datastore Capacity(TB)" unit="" yellow="16383" orange="16384" red="16385" />
<Metric attrkey="Super Metric|sm_6ad28eb5-ef7e-4a88-8fb7-8b4225944f48" label="Used Datastore Capacity(TB)" unit="" yellow="16383" orange="16384" red="16385" />
<Metric attrkey="Super Metric|sm_53c99a82-ecd7-4053-93cf-08b54be817c0" label="Remaining Datastore Capacity(%)" unit="" yellow="15" orange="10" red="5" />
 
<Metric attrkey="summary|avg_vm_density" label="Running VMs Per Host" unit=":1" yellow="50" orange="60" red="70" />
<Metric attrkey="cpu|demand|vConsumption.per.pConsumption" label="vCPU : pCPU" unit=":1" yellow="7.1" orange="8" red="9" />
<Metric attrkey="mem|consumed|vConsumption.per.pConsumption" label="vMEM : pMEM" unit="memory" yellow="1.26" orange="1.5" red="2" />

<Metric attrkey="summary|capacityRemainingUsingConsumers_average" label="Total VMs Remaining" unit="" yellow="10" orange="5" red="0" />
<Metric attrkey="cpu|capacityRemainingUsingConsumers_average" label="CPU - VM Remaining" unit="" yellow="10" orange="5" red="0" />
<Metric attrkey="mem|capacityRemainingUsingConsumers_average" label="MEMORY - VM Remaining" unit="" yellow="10" orange="5" red="0" />

</ResourceKind>
</AdapterKind>
</AdapterKinds>

Choice 2 - If you are running on any version of vROps 6.0.x or upgraded to 6.1 from any of these versions then use the xml highlighted in RED

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<AdapterKinds>
<AdapterKind adapterKindKey="VMWARE">
<ResourceKind resourceKindKey="ClusterComputeResource"> 

<Metric attrkey="summary|total_number_hosts" label="ESXi Hosts" unit=" " yellow="63" orange="64" red="65" />
<Metric attrkey="summary|total_number_datastores" label="Datastores" unit=" " yellow="255" orange="256" red="257" />
<Metric attrkey="summary|total_number_vms" label="VMs and Templates" unit=" " yellow="5000" orange="6000" red="8000" />

<Metric attrkey="Super Metric|sm_a5a9c11a-f27c-446c-8bd5-48a11124b543" label="Total Datastore Capacity(TB)" unit="" yellow="16383" orange="16384" red="16385" />
<Metric attrkey="Super Metric|sm_6ad28eb5-ef7e-4a88-8fb7-8b4225944f48" label="Used Datastore Capacity(TB)" unit="" yellow="16383" orange="16384" red="16385" />
<Metric attrkey="Super Metric|sm_53c99a82-ecd7-4053-93cf-08b54be817c0" label="Remaining Datastore Capacity(%)" unit="" yellow="15" orange="10" red="5" />
 
<Metric attrkey="summary|avg_vm_density" label="Running VMs Per Host" unit=":1" yellow="50" orange="60" red="70" />
<Metric attrkey="cpu|demand|vConsumption.per.pConsumption" label="vCPU : pCPU" unit=":1" yellow="7.1" orange="8" red="9" />
<Metric attrkey="mem|demand|vConsumption.per.pConsumption" label="vMEM : pMEM" unit="memory" yellow="1.26" orange="1.5" red="2" />

<Metric attrkey="summary|capacityRemainingUsingConsumers_average" label="Total VMs Remaining" unit="" yellow="10" orange="5" red="0" />
<Metric attrkey="cpu|capacityRemainingUsingConsumers_average" label="CPU - VM Remaining" unit="" yellow="10" orange="5" red="0" />
<Metric attrkey="mem|capacityRemainingUsingConsumers_average" label="MEMORY - VM Remaining" unit="" yellow="10" orange="5" red="0" />

</ResourceKind>
</AdapterKind>

</AdapterKinds>

Once you have pasted the relevant xml click on Save.

IMPORT THE DASHBOARD

1- Click on Content -> Dashboards -> Blue Wheel Icon -> Import Dashboards

2- Browse to the downloaded file Cluster_cap_perf_db file and click on Open.

3- Once the Dashboard is imported successfully click on Done.

Phewww.. Guess what you have the dashboard now!! :-) Click on the Dashboard List option on the home page and you will see the Capacity Menu, click on that and you will see the Capacity & Performance Dashboard. Click on that to launch the dashboard. 

Click on any of your Clusters to see the capacity scores and the performance data. Please remember that these super metrics would start calculating now, hence you need to wait for a day or two to get some data around performance. Usually after a week or so, you will have some good data to solve those use cases which I mentioned above.

Alright, so this is done!! Do share you comments on how this dashboard helped you!!

Don't forget to share and spread the knowledge!!

Part 3 - vCOps Cluster Utilization Dashboard - Bringing it all together!

This is the last and final part of a 3 post series on creating a vCOps Cluster Utilization Dashboard. Before you read or implement this part on your instance of vCenter Operations Manager, you need to ensure that you follow the following articles:-

Part 1 - Creating a Cluster Utilization Dashboard in vCOps Custom User Interface! & 

Part 2 - Creating Supermetrics for Cluster Utilization Dashboard in vCOps!

So from the last part of this series you have the Supermetrics ID of all the Supermetrics which you have created in your environment. We have 8 supermetrics with id's from 1 to 8.

To begin with download the following XML file which has the code for the Utilization Metrics:-

Cluster-Wise-Util-Metric-File

Open this file in a notepad after downloading it and now replace the shown metrics with the supermetrics ID which you have captured in the part 2 of this post. For me the Supermetrics ID is from 1 to 8 & I have highlighted the same. You need to edit the file with the supermetric ID corresponding to your instance of vCOps.

WARNING - DO NOT CHANGE ANYTHING ELSE IN THE FILE NOR THE NAME OF THE FILE. LEAVE EVERYTHING AS IT IS.

Here is how the file will look like and the values you need to edit.

STEP 1 - Place the edited xml file in a specific location of the UI VM

1- Use an SCP software to login to the UI VM using the root credentials. I am using WinSCP.

2- Change the directory to the following location /usr/lib/vmware-vcops/tomcat-enterprise/webapps/vcops-custom/WEB-INF/classes/resources/reskndmetrics

3- Drag and drop the 3-Cluster Wise Utilization - int-file-ver2.xml file from your system where you downloaded it to this directory as shown in the screenshot below.

4- Right click on the target file and click on Properties to change the permission level to 644 (For Read & Execute rights) as shown below.

You are done with STEP 1. Let's look at STEP 2. Before that download the dashboard xml which you need to import from here:-

Cluster-Wise Utilization Dashboard

STEP 2 - Import the dashboard in vCOps Custom UI

1- Login to vCOps Custom UI with an ID with Administrative Privileges.

2- Click on the Import Option under Dashboard Tools.

3- Browse to the location where you have saved the 3-Cluster Wise Utilization-DB-ver2 and click on Import.

4- You will get a dialog box that your dashboard it successfully imported. Close the same and click on the Dashboards Menu to find a new dashboard named "CLUSTER WISE UTILIZATION"

5- Click on this and you will get the dashboard which we saw in the beginning of this post. Ain't it simple :-)

6- In case you do not see a blank dashboard, Edit the Resources Widget on the Left Pane, Browse to Resource Kinds - > Cluster Compute Resource -> Click on All Attributes once and click on OK. This will get you the list of the clusters and you will be good to go!!

Hope you enjoy this dashboard :-)

Closing this series. Will see you with another custom dashboard soon. Till then!!

********************************Update 29-July-2014*******************************

As per the comments from Jeff and Kenneth, post implementing the steps mentioned in this 3 part blog post, they had a Blue Box with a question mark and No Data written on the CPU-TOTAL CAPACITY and MEMORY-TOTAL CAPACITY

The Blue Box represents that the data is not being collected for the metric for that particular object. On investigation it was found that the metrics of CPU Total (Mhz) and Memory Total (Kb) were not being collected on the vCOps instance since they were running a balance profile for metrics collection. Here are the steps to enable collection.

Login to Custom UI - Environment - Configuration - Attribute Packages - Select Adapter Kind as VMware Adapter - Select Resource Kind as Cluster Compute Resource - Select Default Attributes and Click on OK - Expand CPU Usage and Check Total CPU - Expand Memory and Check Total Memory) - Click on Okay. Log out of vCOps and login after 5 minutes.

Share & Spread the Knowledge!!

Part 2 - Creating Supermetrics for Cluster Utilization Dashboard in vCOps!

Continuing from the previous part where I gave you an overview of the Cluster Utilization Dashboard, in this post we will create all the 8 Supermetrics required for this dashboard. We will start with one example, wherein I will give you step by step instructions and you would need to repeat the steps for the rest of the Supermetrics.

To begin with let's look at the Supermetrics which we need to create:-

CREATING THE SUPERMETRICS

Let's create the SM 1. Here we need to get the Total VMs a cluster can host, hence to get that number we will add Total VMs deployed so far with the Total Remaining. This is how the formula would look like:-

Note:- For someone new to super metric concept, I would recommend you read this white paper from VMware.

1- Login to vCOps Custom UI with a user with appropriate access rights.

2- Click on Environment -> Super Metrics

3- Click on the Add New icon on this window to create a new super metric.

4-  Configure your super metric as shown in figure below. Follow the steps from 1 to 5 and click on OK

5- Now you need to configure the rest of the 8 Super metrics the same was as I showed you the first one. Here are the formulas which can help you get to the correct metric.

SM 2

SM 3

SM 4

SM 5

SM 6

SM 7

SM 8

6- Once you have created all the Supermetrics, your list will appear as mine. Please make sure that you note down all the Supermetrics ID as we would need them in PART 3 when we bring this all together. For me they are Number 1 to 8.

CREATING A SUPERMETRICS PACKAGE AND ADDING THE SUPERMETRICS

We will now create a Supermetric package where we will add all these Supermetrics and then apply them on the clusters of my environment to start calculating this metric.

7- Click on Package Editor and click on the + symbol to add a package.

8- Since all these Supermetrics are for Cluster level, give it a name, I call it SM-CLUSTER. Put a check against all the supermetrics you want to add to this package. You can then setup DT configuration if required else just click on OK.

APPLYING THE SUPERMETRICS PACKAGE ON THE EXISTING & NEW CLUSTERS

Now that your Supermetrics package is ready, you need to apply the same on the existing clusters and also apply the same on the Cluster Compute Resource kind to ensure that all these supermetrics are calculated for any new cluster which you bring into vCenter Operations Manager. Here are the steps:-

9- Click on Environment -> Environment Overview.

10- On the left pane navigate to Resource Kinds -> Cluster Compute Resources -> Default Attributes as shown in the screenshot below. This will list all the existing clusters in your environment.

11- Select all the clusters and click on the Edit Button to edit the properties of the selected cluster. If you have a lot of clusters then you can select the 1st item (Press the CTRL + SHIFT) on the keyboard and select the last item. This will select all the clusters. Click on the Edit button as shown below.

12- Under Supermetric package apply the package, SM-CLUSTER which we created in Step 8 by clicking on the drop down arrow as shown below & click on OK. This will apply all the supermetrics on all the existing clusters.

13 - Finally, let us apply this package on the Cluster Compute Resource Kinds to ensure that every new cluster has these metrics by default. Click on Environment -> Configuration -> Resource Kinds Default.

14 - Select Cluster Compute Resource on the Left pane and Select the SM-CLUSTER package under super metric package & click on OK.

Alright, so we are done with the trivial part.. In the Next Part, I will help you bring together these Supermetrics and an XML file which I have created for you guys to download to get your final Cluster Utilization Dashboard.

Till then - stay tuned...

***Share & Spread the Knowledge***

Part 1 - Creating a Cluster Utilization Dashboard in vCOps Custom User Interface!

A few days back I wrote a post on creating a One-Click Cluster Capacity Dashboard which was liked by a number of people in the virtual world. In-fact it was so easy to download and replicate that within a few hours of rolling out that post, I could see a number of people coming back to me stating that they have deployed the Dashboard and have started looking at the Capacity numbers for their infrastructure. Many of them asked as to how can they create their own XMLs for creating similar dashboards. I wrote another post in a few days which spoke about Creating your own XMLs for Custom Dashboards and got some great feedback on that as well.

Doing this is so much fun, however more often it is difficult to think of the various problems you can solve with these Custom Dashboards. With my continuous engagements with the customers on the field, I get some fancy requests which keeps my brain working on digging out possibilities with vCenter Operations Manager. 

In my last engagement with a customer, I installed the Capacity Dashboard for him which I have already shared with you guys before. The immediate reaction of the customer after looking at the dashboard was to have something similar for his environment which could help him gauge the UTILIZATION of each cluster with a single click. This includes, CPU, Memory & Disk Utilization. The client was very clear with what he needs and that helped me create the following dashboard.

I will do this Dashboard in a 3 Part Series.

Part 1 - Introduction to the Cluster Utilization Dashboard (This post)
Part 2 - Creating the required Supermetrics for the Cluster Utilization Dashboard
Part 3 - Bringing it all together.

NOTE - This dashboard uses both Regular vCOps Derived Metrics and SuperMetrics. I have marked the Scorecard boxes which are using Supermetrics with the initials "SM <Number>". We would need to create these SuperMetrics before we could use the .XML file and import the Dashboard as we did previously. 

Let us first look at the features & use cases for this dashboard and then we would proceed with the steps on how to build the each component of this dashboard.

CLUSTER UTILIZATION DASHBOARD - FEATURES

1 - Gives you a single page view of the Utilization of all your vSphere Clusters being monitored by vCenter Operations Manager. (One of Multiple vCenter Servers). Click on a cluster on the left pane for which you want to know the UTILIZATION data and on the right pane you will see the same.

2- The 1st row gives you an overview on how many VMs can be deployed in a cluster based on the vCOps average profile, how many are deployed and how many more can be deployed based on the deployment trends.

3- Then you have 3 rows of CPU , Memory & Storage Total Capacity,  Allocated Capacity, Usage, Usage in percentage & finally the number of VMs remaining keeping a resource in mind such as CPU, Memory & Storage.

4- It is important to note that vCOps uses local storage if available on the ESXi hosts, hence the storage stats might be a bit skewed. You can create groups within vCOps to isolate local storage from SAN cluster wise but then it will not allow you to have a dynamic dashboard such as this. You will have to then create a separate dashboard for each of your cluster & create a new one every time you add a cluster. But with THIS cluster utilization dashboard everything is automatic and dynamic ;-)

5- This is the most asked Dashboard by CxO's as they want a one click summary of what they have, how much they have used and what is remaining. This is the LARGEST use case hands down.

6- Their are last 30 days trend-lines as well for the specific metric which makes it a bit more informative.

CLUSTER UTILIZATION DASHBOARD - SUPERMETRICS??

I wrote another post on vCenter Operations Manager some time back where I have showcased how to create Supermetrics. In this post, I will show you how to create one of the Supermetric through screenshots and then give you the formula for rest of the super metrics which you can create yourself following the example. If you closely look at the screenshot of the dashboard, you will notice that this dashboard displays 20 metrics in total. 8 out of 20 are Supermetrics. The table below shows what metrics & Supermetrics are being displayed.

Need less to say that the one in RED are the Supermetrics and the others are derived metrics. Before we bring this all together, let me share the steps to create a Supermetric, let's say SM 1

I will close this part right now and in PART 2, I will explain as to how we need to create all these Supermetrics. That should be out by tomorrow so stay tuned :-)


***Share & Spread the Knowledge***