Monday, June 1, 2020

Monitor Tanzu Kubernetes clusters using vRealize Operations


Whether it is containers or virtual machines, the end goal of organizations is to have a highly available, reliable and a scalable platform to run their business applications. With vSphere 7.0, VMware provides a unified platform to run your applications, by leveraging vSphere with Kubernetes.

In a previous post, we shared how vRealize Operations 8.1 & vRealize Operations Cloud can easily discover this new platform. Once discovered, the powerful analytics in vRealize Operations, unlock the use cases of monitoring, troubleshooting & capacity management for these new constructs. In the screenshot from my environment, you can see the newly discovered objects automatically tied to the vCenter inventory.

With this post, we will explore how vRealize Operations and vRealize Operations Cloud can monitor Tanzu Kubernetes cluster to provide your central IT teams with Full Stack Observability, from upstream Kubernetes applications; SDDC infrastructure, all the way down to the physical infrastructure. This will give the peace of mind you need to run your business applications with zero-blind spots and complete control.

What is a Tanzu Kubernetes cluster?

A Tanzu Kubernetes cluster is deployed using VMware Tanzu Kubernetes Grid. provides a consistent, upstream implementation of Kubernetes, that is tested, signed, and supported by VMware. A Tanzu Kubernetes cluster is an opinionated installation of Kubernetes open-source software that is built and supported by VMware.

Whether you deploy a Tanzu Kubernetes cluster on vSphere 7.0 using Tanzu Kubernetes Grid Service for vSphere, on AWS natively using Tanzu Kubernetes Grid or on VMware Cloud on AWS using Tanzu Kubernetes Grid Plus, vRealize Operations (On-Prem & Cloud), provides deep visibility into this new world. As central IT teams start to manage these Kubernetes environments, they can simply extend their existing investment in vRealize products to this new world and empower their teams to have an end to end visibility into business applications comprising of traditional VM based apps and modern microservices-based apps.

Monitoring Kubernetes with vRealize Operations


Let's start with the simple pre-requisites:

Prepping a Kubernetes Cluster for Monitoring

In my example, I am leveraging the awesome work done by William Lam, with his Tanzu Kubernetes Grid Demo Appliance fling and his blog post that explains how to leverage this appliance to deploy Kubernetes clusters on VMware Cloud on AWS. In addition to that, I would be using my instance of vRealize Operations Cloud. Let's get started.

With my Tanzu Kubernetes Grid Demo Appliance, I am pre-authenticated to my VMC on AWS SDDC. The first cluster that Tanzu Kubernetes Grid deployed here is a Management Cluster and in addition to that, I have deployed a couple of guest clusters for test and production use. Here are the clusters in my environment:

To list all available Kubernetes contexts, you can use the following command:

kubectl config get-contexts

I have already configured these 3 clusters to be monitored within vRealize Operations Cloud using the container management solution. You can see in the screenshot below:

Let's deploy a new Kubernetes Cluster and then we will add that to vRealize Operations Cloud for monitoring.


Step 1 - We will deploy a small development cluster using the following command

tkg create cluster --plan=dev tkg-cluster-03

Alright looks like my Kubernetes cluster "tkg-cluster-03" is up and running.


Step 2 - Let's create a vrops-cAdvisor.yaml file on this cluster and run it as a DaemonSet. Run the following commands

Switch context to the newly deployed cluster by running the command

kubectl config use-context tkg-cluster-03-admin@tkg-cluster-03

Now let's switch to the temp directory and create a vrops-cAdvisor.yaml file using VI command. You can choose to create this file elsewhere as well. Copy the content from the following text into the file into your vrops-cAdvisor.yaml file

If you wish, you can change the namespace where you want to deploy cAdvisor by editing the namespace option in the text. I am using the namespace named kube-system

Save the file using wq! and then lets run the following command to run the cAdvisor as a DaemonSet

kubectl apply -f vrops-cAdvisor.yaml

Alright, the cAdvisor sidecar is deployed. That was simple. You can run the following command to see if the containers are running

kubectl -n kube-system get pod

Step 3 - We would need the IP Address and the Credentials for this cluster to add this to vRealize Operations. This information is available in the config file. Run the following command to read the config file.

less .kube/config

We need 2 things from this config file.

  1. The IP Address of the newly deployed Kubernetes cluster. You can see all the guest clusters here and the latest one named "tkg-cluster-03" with its server URL : ""
  2. We need a way to authenticate against this guest cluster. vRealize Operations support following. For more information, see Kubernetes Authentication.

In my case, I will use the Client Certification Auth. Just copy the following 3 tokens and keep them, we will use them later. This part can be tricky so make sure you have the right tokens if using Client Cert Auth. You would need the following tokens Certificate Authority Data, Client Certificate Data and Client Key Data as shown below. Ensure to use the tokens for tkg-cluster-03 



Configure Kubernetes Adapter in vRealize Operations

Now that we have all the ingredients, let's get cooking.

Step 1 - Let's add this to vRealize Operations Cloud to begin monitoring. Click on Administration -> Other Accounts -> Click Add Account.

Step 2 - Select Kubernetes Adapter to fill in the following details. You can also see how I created the credentials using the cert data used before.

Note - The vCenter Server Advanced Setting step is optional, if your Kubernetes Cluster is running on vSphere like mine, you can simply add the vCenter Server here. If you are monitoring this vCenter Server with the same instance of vROps, the Container Solution will automatically connect the Kubernetes Nodes to vCenter Virtual Machines.

That's it. just 2 steps :-)



Using the Kubernetes Overview Dashboard and Troubleshooting Workbench for containers

This one is out of the box. Click on Dashboards -> Kubernetes Environment -> Kubernetes Overview

Since my Kubernetes cluster is running on a VMware Cloud AWS environment, let's see how that advanced setting creates the relation between the Kubernetes nodes and VMs. Select a node from Widget 5 and click on the Object Details icon from this dashboard.

From Object Details, click on the Metrics tab, and let's expand the related VM Kubernetes objects, here you can see the entire related inventory from a container, all the way to the VMC Org.


Here is another example of a Kubernetes application named "YELB", that I have deployed on my Kubernetes cluster. vRealize Operations automatically detects that and I can click on the Troubleshoot to start troubleshooting issues using the "Troubleshooting Workbench".


Yelb in Troubleshooting Workbench:



  • The container management solution of vRealize Operations provides deep visibility into any flavor of Kubernetes running on top of vSphere. Be it Tanzu Kubernetes Grid, deployed Kubernetes clusters, or Openshift, it does not matter.
  • When it comes to Container Operations for Central IT teams, full-stack visibility from applications to infrastructure is the need of the hour and with vRealize Operations and vRealize Operations Cloud, we provide this visibility by leveraging your existing investments in VMware.
  • Lastly, if you are running Kubernetes clusters on a non-vSphere platform such as AWS, Azure, or GCP, you can still leverage the container solution to get visibility into upstream Kubernetes.

Hope this helps. Leave your comments here or reach out via twitter @sunny_dua for any follow-up conversations. Happy Kubernetting!!


Tuesday, April 14, 2020

vRealize Operations Cloud - The one stop shop for multi-cloud operations is LIVE

About vRealize Operations Cloud:

Along with vRealize Operations 8.1, VMware just made the vRealize Operations Cloud service go live. VMware vRealize Operations Cloud is a cloud-delivered service that allows a customer’s infrastructure and operations teams to manage the enterprise’s VMware Cloud environment, whether in the customer’s own on-premises software-defined data center (“SDDC”) or in the customer’s hosted environment, such as VMware Cloud on AWS.

·       vRealize Operations Cloud has the same feature capabilities as the On-premises vRealize Operations Manager 8.1, with the operational agility and benefits of SaaS

·       The service is offered on the Cloud Service Platform thereby providing a consistent experience to customers who are consuming other VMware Cloud Services and leveraging common services for Identity Management, Billing, and Support.

·       vROps Cloud provides integrations with other vRealize SaaS Offerings namely vRealize Log Insight Cloud, vRealize Network Insight Cloud, and vRealize Automation Cloud

·       Exclusive in vROps Cloud: View logs in context of the selected object in vRealize Operations Cloud; Launch into the full capabilities of vRealize Log Insight Cloud at the click of a button

Purchasing vROps Cloud:

Refer to vROps Cloud Pricing for the latest information on pricing.

vROps Cloud Documentation & Resources:

vRealize Operations 8.1 is out. Here is everything you need to know.

VMware just announced the General Availability of vRealize Operations 8.1. I wanted to quickly share the high-level capabilities that are coming with this release. These are based on key themes of VMware's Cloud Operations vision. 

vRealize Operations Manager 8.1 delivers new and enhanced capabilities for self-driving operations to help customers optimize, plan, and scale VMware Cloud, which includes on-premises private cloud or VMware SDDC in multiple public clouds such as VMware Cloud on AWS, while at the same time unifying multi-cloud monitoring. Powered by artificial intelligence (AI), this release takes us one step closer to the Cloud Operations Vision of providing a unified operations platform, deliver continuous performance optimization, efficient capacity, and cost management, proactive planning, app-aware intelligent remediation, and integrated compliance. 

The key release themes include:

Integrated and Intelligent

  • Support for vSphere 7 with Kubernetes helps operationalize vSphere with Kubernetes by enabling you to monitor the health, performance, and the capacity of constructs such as Namespaces, Tanzu Kubernetes cluster, and vSphere Pods.
  • Self-driving Operations for VMware Cloud on AWS provides native support for VMware Cloud on AWS. It simplifies Cloud Account for VMware Cloud on AWS, leveraging VMware Cloud Service Portal Token and will be able to auto-Discover new SDDCs and incorporate for monitoring in a few simple steps.
  • Intelligent Alert importance ranking to help prioritize triggered alerts based on access and occurrence frequency analysis. 
  • Enhanced and updated management packs to support the new release. There are multiple enhancements to various management packs with this release. Refer to the VMware Product Interoperability Matrix for all VMware Management Packs supported with vRealize Operations Manager 8.1. The 3rd party authored Management Packs that are supported with 8.1 are listed in the VMware Compatibility Guide.
  • Native support for regulatory compliance packs 
  • Integrations. Integrations. Integrations

Easy Management
  • Simplified alert management with the ability to create and modify alert definitions and manage changes across alerts, symptoms, recommendations, policies, and notifications using a simplified workflow. This includes enhanced notification filters as well.
  • Simplified and intuitive summary pages for key objects to simplify monitoring and troubleshooting with quick access to key information.
  • Enhanced Dashboard Management capabilities such as search, arrange, maximize widgets, etc. to simplify the user experience.
  • Consistent navigation via improved menus across the workflows to enhance ease of use.

 Application-Aware and Multi-Cloud
  • Application Operations enhancements with the ability to do credential-less discovery of services leveraging VMware Tools.
  • Telegraf Agent lifecycle management with public API support for application monitoring at large scale.
  • Native Support for Azure via Cloud account and monitoring Azure services.
  • Enhanced Public cloud monitoring support with an introduction to Google Cloud Platform (GCP) integration in addition to existing support of AWS and Azure.
  • Enhanced cloud costing with Cloud Health perspectives.
  • Out of box dashboards, Reports and Alerts to manage VMware Cloud on AWS, Public Clouds, and vSphere with Kubernetes

For a detailed list of enhancements refer to the release notes.

·       Download Link: vRealize Operations 8.1 Download Landing Page

·       Release Notes:  Release Notes

Monday, October 28, 2019

vRealize Operations dashboards to monitor VMware Cloud on AWS

I have seen a number of asks for dashboards that can help with monitoring specific use cases related to VMware Cloud on AWS. With this post, I will share four such dashboards that I have been working on pertaining to VMC monitoring use cases. Special thanks to William Lam for guiding me with the best practices related to VMC monitoring. All those best practices were considered while creating these dashboards.

Along with the description and usage if these dashboards, this post will also provide the pre-requisites and few simple steps to import these dashboards in your environment.

About the dashboards

Once imported you will see the following four dashboards under a new dashboard group named "VMC Dashboards".

VMC Capacity Dashboard


This dashboard provides a capacity overview of each of your VMC SDDC. You can easily drill down into the capacity of all the underlying components such as Clusters, Hosts, VMs, Datastores, and Diskgroups.

How to use this dashboard?
  • The first 3 rows show you a card per VMC SDDC with 3 different dimensions. This includes Capacity Remaining, Time Remaining and VM Remaining.
  • Upon selecting an SDDC, you can see the clusters, hosts, VMs (both management and workload), datastores and disk groups. 
  • The key KPIs are color-coded to help identify capacity bottlenecks.

VMC Inventory Dashboard

This dashboard provides a quick overview of inventory of all your VMC SDDCs. The inventory includes:
  • vSphere Clusters
  • Datastores
  • Hosts
  • Virtual Machines
How to use this dashboard?
  • The first row shows you a card per VMC SDDC with number of virtual machines running in each SDDC. This also shows you a trend of virtual machine growth over the past 30 days.
  • Once you are close to the number of VMs supported per SDDC, the card will indicate that by changing colors.
  • Upon selecting a card, you can see the list of all the vSphere Clusters, Datastores, vSphere Hosts and VMs in that SDDC with key configuration details.
  • You can choose to export the desired list in a CSV format using the toolbars on the list.
  • You can also filter the list of vSphere Hosts and VMs by selecting a vSphere Cluster or list of VMs by selecting a vSphere Host.

VMC Management VM Dashboard


This dashboard helps you monitor the utilization and performance of the key management VMs running in your SDDC. The goal of this dashboard is to ensure that the management components such as vCenter and NSX are not facing any resource bottleneck from a CPU, Memory, Network and Storage perspective.

How to use this dashboard?

  • The first list provides all the management components in each SDDC with key CPU utilization and performance KPIs. Upon selecting a management VM, you can see the usage and performance trends of all the CPU cores.
  • The second list provides all the management components in each SDDC with key Memory utilization and performance KPIs. Upon selecting a management VM, you can see the memory usage and performance trends.
  • The third list provides all the management components in each SDDC with key Network utilization and performance KPIs. Upon selecting a management VM, you can see the network usage and performance trends.
  • The fourth list provides all the management components in each SDDC with key Storage utilization and performance KPIs. Upon selecting a management VM, you can see the storage usage and performance trends.

VMC Utilization and Performance Dashboard

This dashboard provides a utilization and performance overview of each SDDC based on heavy hitter VMs and impacted VMs over the last 30 days of utilization and performance KPIs

How to use this dashboard?

  • The first list shows the list of all the SDDC with aggregate CPU, Memory ad Storage utilization over the last 30 days with Maximum and 95th Percentile values.
  • Upon selecting an SDDC, you can see the list of top virtual machines which are consuming Compute, Network & Storage resources in each SDDC.
  • The dashboard has 2 sections thereafter. One shows the Compute (CPU & Memory) utilization and performance analysis and the second shows the Network and Storage utilization and performance analysis.
  • Each section is based on the last 30 days of data with 95th percentile transformation which is configurable as needed to Max, Average, Current, Standard Deviation or other mathematical transformations.
  • This data helps you find the victims and villains in your environment which are negatively impacting capacity or performance from a CPU, Memory, Storage or Network perspective.


The following pre-requisites should be taken care of before importing these dashboards.
  • These dashboards need either vRealize Operations version 7.5 or 8.0.
  • These dashboards are tested with VMC SDDC version 1.7 and above.
  • Both vCenter and vSAN adapter instances should be configured.
  • Need vRealize Operations Advanced edition or above.
  • Need appropriate vRealize Operations permissions to import and share.

Steps to import

1- Ensure that your VMC vCenters instances are configured with Cloud Type as "VMware Cloud on AWS". More details here.

2- Download this VMC Dashboard file and extract it to your desktop.

3- Import file to your vROps instance. Click on Dashboards - Views - Actions - Import Views

4- Import file to your vROps instance.  Click on Dashboards - Actions - Manage Dashboards - Action - Import Dashboards.

You should have your dashboards by now. Please note that these dashboards might take a few seconds to load for the first time. 

Hope this article helps you. Please share your comments in the comments section below or follow me on twitter to be updated on similar content - @sunny_dua.

Friday, October 18, 2019

Everything you need to know about the vRealize Operations 8.0 release

The wait is over. vRealize Operations 8.0 is NOW Available. Here are my prescribed next steps:

4- Ensure to run the Upgrade Assessment Tool, available on the download page:

      5- Check the compatibility with management packs you might have in your environment -
      6- Check product compatibility matrix (just in case) -

      7- Good news – Most of the VMware solution packs that go with vROps are already released and here are the links:

Solution Name
Release Notes & Download Link
vRealize Operations Management Pack for CloudHealth by VMware
vRealize Operations Management Pack for Storage Devices
vRealize Operations Management Pack for VMware Cloud on AWS
vRealize Operations Management Pack for NSX-T
vRealize Operations Management Pack for vCloud Director
vRealize Operations Management Pack for Skyline
vRealize Operations Management Pack for Container Monitoring
SDDC Health Monitoring Solution
vRealize Operations Management Pack for NSX for vSphere
vRealize Operations Management Pack for vRealize Orchestrator
vRealize Operations Management Pack for VMware Identity Manager
vRealize Operations Federation Management Pack
vRealize Operations Management Pack for VMware Integrated OpenStack
vRealize Operations Management Pack for Cloud Provider Pod

Happy Installing/Upgrading.

Hope this article helps you. Please share your comments in the comments section below or follow me on twitter to be updated on similar content - @sunny_dua.