Did You Know #2 - Leveraging vROps Remote Collectors for Local Adapters!

In this part of the "Did You Know" series, I will talk about a small architectural tip which will not only help you enhance the performance of your vRealize Operations Manager cluster, it will also save you from up-sizing the cluster from let's say, medium to large nodes and at the end of the day save a ton of CPU & Memory in the process.

Did you know that vRealize Operations Manager uses Remote Collectors for collecting data from a Remote Datacenter and send it over to the centralized vROps cluster. The diagram below shows the actual purpose for which a remote collector was introduced in vRealize Operations Manager:

In the above example, we have a vROps Cluster in Site A. This cluster consists of 2 or more nodes which have a local collector module on them. This collector module collects the data from the local data sources, which are also known as adapter instances. Some examples of an adapter instances would be vCenter Adapter, NSX Adapter, MPSD (management pack for Storage Devices) etc.

The Nodes of the cluster here have multiple roles to play. They not only collect the data from the data sources, they also have to crunch this data using the analytics engine, calculate dynamic thresholds, run the capacity engine and host all the data through the CASA and Web UI. 

On the other hand in Site B, we have a remote collector group with 2 or more remote collectors (in a HA mode). Their role is to collect the data from the Site B data sources using the Collector framework on each node and send that data over to the centralized cluster in Site A. The Remote Collectors are small form factor of the vROps appliances which are stateless and the only role thy have is to collect data. Here are a few facts which make them great for playing the role of a collector from a sizing standpoint.

The come in 2 form factors ***: 

SMALL: 2 vCPU / 4 GB RAM. A small RC can collect 1500 Objects (an object can be a VM, Datastore, ESXi Host, LUN, etc) and upto 600,000 metrics. (1 VM usually creates around 250 metrics).

LARGE: 4 vCPU / 16 GB RAM. A small RC can collect 12000 Objects and up-to 3,500,000 metrics.

We all know that the main cluster nodes of vROps can also do collection and as per the sizing guidelines, a medium node can collect up to 7000 Objects, while a large cluster node which is 16 vCPU / 48 GB of ram can also collect up to 12000 Objects.

***Reference VMware KB - https://kb.vmware.com/kb/2130551

SIZING SCENARIO:

Now imagine a scenario, where you have a 4 Medium Node cluster with 4 node in Site A. You have a vCenter Adapter instance which has more than 7000 Objects (5000 VMs, 2000 Datastores, 200 ESXI Hosts etc). In such a situation, in order to collect data from this vCenter Adapter instance, you would have to up size your cluster node to a large node. Since vROps cluster nodes have to be symmetrical, you would have to up-size all your cluster nodes to LARGE NODES. In this situation you would have to invest on 32 vCPU (8 per cluster node to reach 16 vCPUs) and 64 GB of RAM (16 per node to reach 48 GB per cluster node). This in most cases is a huge change since you would have to ensure you have enough resources in the under lying cluster. In some cases you might also go beyond the NUMA boundary which we all know has some performance impact from a CPU standpoint.

With all these concerns in place, it would an excellent opportunity to leverage the Remote Collectors in the local Site A as well. While the name says REMOTE, it is not necessary that remote collectors are deployed only on remote sites. They can also be utilized in a local site to collect data from adapter instances which can be large in size. Taken our example into consideration, we would just need 2 Remote Collectors ( 2 for high availability, in case one fails) to collect from the Site A vCenter. These 2 appliances will only cost as 8 vCPUs and 32 GB of RAM in total). This will reduce the resource requirements by more than half and also ensure that your cluster nodes have no pressure on collector and hence all that CPU and RAM can be utilized by the other roles on the vROps nodes which will eventually give better performance.

So here would be the new architecture with Remote Collectors Everywhere!!!

With this model, we have better performance, more scale and less hardware requirement for deploying large vROps Deployments. Another important thing to note is that you can always migrate from Design 1 to Design 2 or from Design 2 to Design 1 without any downtime or data loss. Hence if you are on the way to scale the environments being monitored by vROps, this tech tip would be very useful for you.

Hope this helps with day to day datacenter operations using vRealize Operations Manager.

Stay tuned for more goodies!

vROps Webinar 2016 : Part 5 - Design & Deployment Considerations

As promised, I am posting the recording for the 5th Session of vROps Webinar Series 2016. Both Simon and I successfully delivered the session on Design and Deployment considerations.

Session Details:- In this installment of the series, we discussed the steps and thought processes that should be used before and during the design and deployment of vRealize Operations Manager. During the session among other things we will cover the planning, core components, correct sizing, HA, clustering, DR and future growth.

Once again I would like to thank my friend and partner in this project Simon as without him this would not be possible.

So without further ado, here is the recording for this session:

Note : It is recommended that you watch the video in HD quality for a great experience.

vROps Webinar 2016 - Announcing Part 5 : Design & Deployment Considerations

Time to announce the next part of the year long webinar series on vRealize Operations Manager. This time around, Simon and I thought about discussing Architecture of vROps. To some, it might sound strange as for smaller deployments you might not have to worry about Sizing and Architecture much since it is pretty simple to install and configure a small or a medium node for a small shop. However as your monitoring needs grow and you start adding solutions for monitoring data sources beyond vSphere, you would need to think about scaling up or scaling out. As your monitoring environment weaves into your incident ticketing system, you would start to see the need to HA of vROps and as you have a DR strategy for your workloads, you will start thinking about DR for your operations tools as well. 

We have seen these questions and situations come up in many of our engagements and hence we thought that we should share some of our experience around this area. Below are the Webex Details:

Day & Date          : Friday, 27^th May 2016

Time                     : 2:00 PM - 3:00 PM  (SGT)

Event                    : vROps Webinar 2016

Topic                    : Part 5 : Design & Deployment Considerations

Speakers               : Simon Eady / Sunny Dua

WebEx Link          : Join WebEx meeting 

NOTE - Don't forget to mark your calendars by saving the calendar invite!! Feel free to forward the invite to anyone who might be interested. It's open to all!!


Sharing & Spread the Knowledge!!

Part 4 - vRealize Operations Manager Deployment Models!

Now that we are aware of the new features and the architecture of vRealize Operations Manager, with this post I will talk about the various deployment models which are possible while deploying vROps 6.0 in your infrastructure. I will be writing about the options available to those who are already running on vCOps 5.x for moving to vROps 6.0 in the next part, however with this part I am focused on talking about the deployment architecture of the solution by considering various deployment scenarios.

While talking about the Cluster Architecture of vROps, I also pointed out to the fact that the scalability and performance of vROps 6.0 comes form the in-memory database - "GemFire" which allows you to access hot data from the memory layer making it fast and also allows you to stripe this data across multiple nodes.

Here are a few design principles which we need to consider while choosing the deployment model:-

• vROps 6.0 Cluster Nodes are required to be in a Local Area Network.

• For collection from remote vCenter Servers you would require a REMOTE COLLECTOR

• Deploying a Cluster Architecture is not a compulsion. It is done only when scalability & resiliency is required.

• For 100% resiliency it is important to protect the MASTER NODE by using a MASTER REPLICA.

• Each node in a cluster has to be of the EXACT SAME configuration (CPU, MEMORY & STORAGE)

Now let's look at some of the deployment scenarios and what would be the architecture of vROps.

Deployment Example 1 - Multiple vCenter servers in one datacenter location!

With a single datacenter with one or Multiple vCenter Servers you would deploy either one or more than one node in the cluster depending on how you want to scale the cluster for monitoring more resources and also increasing the resiliency by enabling the HA feature. All the users login to this one site.

Deployment Example 2 - Multiple vCenters in one location & vCenters in many other different datacenter location.

With the above scenario, you can either choose to have an individual vROps instance (clustered or non-cluster) in each site or you could have one centralized vROps instance in the major location, while you can have remote collectors in each of the other sites which have vCenter Servers in them. If the centralized deployment is used then all the users will have a unified view through the centralized datacenter. With vROps instance in each site, the users will login to the site they wish to access.

Deployment Example 3 - Multiple vCenters in two main locations.

In this third and final example, each datacenter has a number of vCenter Servers hence you can choose to deploy a separate instance of vROps cluster in each datacenter. In this case, the users will login to their local vROps instance to access the resources they are monitoring.

With the next part, I will take you through the steps to install & configure the vROps appliance and then we will look through the various options available within the solution.

Till then..

Share & Spread the Knowledge!!