How Does vCenter Operations Manager Collects Data from a vSphere Environment?

This article is to ensure that I document a discussion I had a few days back with some Gurus of vCenter Operations Manager. The discussion ignited from a customer requirement where they wanted to know:

"If they can ETL (Extract, Transform & Load) 2 years old data from the vCenter Server DB into their freshly installed vCenter Operations Manager Environment."

Needless to say, customers see a great value in the Performance Analytic and Capacity Planning engine of vCenter Operations Manager. They want to ensure that they can use the historical data from VC and get some intelligence from the vCOps about the performance and capacity trends in their infrastructure in the past. Alongside, giving a peek into the history, this can also become a strong historical base for vCOps to provide future predictions which is another strong area of the product.

The requirement above immediately raises a few questions about the process of data collection:-

• Does vCOps talk to vCenter Database through the vCenter Adapter to collect the data? If yes, can I feed historical data to give vCOps a kick-start?

• What is the impact of Data Collection on vCenter Database? Do I need to increase resources CPU/Memory/IOPS to ensure that the performance is fine?

• Other questions are around Data Collection cycles, remote collection etc.

Well, to be honest there can be unique questions from each deployment. Therefore, instead of looking at the questions, let's look at the process of data collection.

To recap, let us first look at the architecture of vCOps vAPP which will give us a clear picture as to where does the collection happens. (I wrote an article a few months back about the architecture & sizing of vCOps in case you are interested.)

If you look at the red highlighted circle, you can see that the collector sits on the Analytics VM, from where is uses adapters to collect data from resources such as vCenter, vCloud Director, Configuration manager or even 3rd party sources.

Now if we look at the collection process specifically for the vCenter Adapter, this is how the data is collected:-

vCenter Operations Data Collection Process using vCenter Adapter

Well this explains how data is collected from a vSphere Environment by vCenter Operations Manager. Since this model does not talk to vCenter DB, it is quite clear that you cannot have an ETL approach which I discussed in the beginning of this article

To conclude the data gathering process of vCenter Operations Manager is completely through PerformanceManagerAPI where vCenter Server acts as a proxy and the data is pulled from the ESXi hosts.


Share and Spread the Knowledge. 

VMware Cloud Credibility: One-Stop-Shop To Learn About Cloud!

A few days back I was listening to VMware Communities Roundtable Podcast hosted by John Mark Troyer who works for VMware as a Director for the social media evangelist group. The podcast was of great interest to me as it discussed the VMware vExpert 2013 rewardees.

In-fact, once the results were out here, I was extremely happy and excited to see that, I have been honored with this title for the year 2013. Yes, I am a vExpert 2013 :-) 

One of the great thing about VMware technology and community is that this is an ocean full of knowledge and there is no end to learning. One of the greatest example of this is the VMware Cloud Cred Program which was discussed by John, Eric and Corey during this podcast. As soon as I heard about the program, I logged into the Cloud Cred Website to check what's cooking.

Once I registered and created my team and added some of my fellow vExperts and VMware enthusiasts to my team, I was exposed to the great collaborative features of this program, which not only allows you to work as a team, but also enables you to learn together and understand both basic and advance concepts of Virtualization & Cloud Computing.

What's fantastic is that this program is not only for Virtual-Nerds (name of our cloud cred team) but also for individuals who want to get started with Virtualization & Cloud Computing. This portal acts as a gateway which exposes you to a number of great resources whether its influential people on twitter, some great blog articles on cloud computing or some amazing social evangelism tasks. Well, I guess this would be my go to place for my daily dose of news on Cloud Computing and Virtualization. 

To put the icing on the cake, this program not only makes you learn, but also earn :-)

With each task you and your team performs, you get brownie points which can be later redeemed for some awesome goodies. The complete list of awards can be found HERE

The GRAND prize from this program is a fully paid trip to VMware Barcelona. Here are the details.

Learn More

Well I am 2 days old into this program and I have already won a number of gifts :-) Here is the list of my goodies.

To learn more about this program, have a look at the details in the video below and enroll and be a part of this wonderful opportunity to learn.

As always, Don't forget to Share and Spread the Knowledge. 

Hardware considerations while using VMware Site Recovery Manager!

Today, I received a query from a VMware partner around hardware requirements for VMware Site Recovery Manager. Although the SRM solution is hardware agnostic in most of the cases, there are a number of things one must be careful of before implementing Site Recovery Manager in there vSphere infrastructure. I replied to the query and I thought I should share this with the community as well to ensure they can read this and make use of it.

Here is the Query

Can someone throw some light on SRM hardware requirements on both sites? Is there stringent requirements of matching CPUs at both end. While DR runs lesser capacity than DC, we want to plan lower performing CPUs. Can this different generation CPUs will be taken care by SRM??

Here was my Response

While your query holds good for traditional physical DR, with SRM the entire equation of hardware dependency changes drastically. Site Recovery Manager is a disaster recovery solution which is completely hardware agnostic as it operates at the software layer of the data-center which is vSphere.

To make it more simple, you can have any brand/model/make/CPU of server running in the Protected Site, and fail-over your virtual machines from this Protected site to Recovery site on a ESXi hosts which is of a completely different make, model or CPU from the Protected site. The only requirement is that both these server types should be supported with the vSphere version which you are running on them, hence they should be on our HCL (Hardware Compatibility List) for servers. 

Next, you need to look at the capacity of the DR site. As you mentioned, the DR site can be made up of less expensive hardware, however, at the same time, you should have appropriate capacity calculations to ensure that the Virtual Machines, if failed over to the DR site during a disaster or if powered on at the DR site in a Test Recovery Mode, should have sufficient Compute and Storage resources to operate normally and support the respective business functions.

Last and the most important dependency on hardware is for storage. As you are aware, SRM with version 5.0 and above supports both Storage Based Replication(SBR) and  Host Based Replication (HBR or vSphere Replication), you have a choice here. If you chose SBR, you need to have an identical storage array on both the sites and a storage replication software which will enable this replication. In this case, you need to check our  HCL (Hardware Compatibility List), to ensure that you a supported SRA (Storage Replication Adapter) available for that storage array. In case you go with the built in HBR or VR, you do not have to worry about the storage hardware again, since the replication happens from the VMkernel layer from source to destination host over the Networking Protocols. Also remember that you need to select the right version of SRM to go with the version of vSphere which you are running. Look at the Product Interoperability matrix for this.

To summarize, since SRM is an Active/Passive DR solution, it does not need you to run identical hardware and you can run any supported hardware for server configuration. For storage you have 2 options HBR and SBR. To read more about vSphere Replication and searching VMware Hardware Compatibility Lists, you can refer to the following articles I wrote a few days back:-

Using vSphere Replication with VMware Site Recovery Manager !!

Using the VMware Hardware Compatibility List (HCL) and Product Interoperability Matrix!!

As always, Don't forget to Share and Spread the Knowledge.