Dividing Bandwidth of a 10 GB CNA Adapter for ESXi Networking andStorage!!

With most of my recent projects customers are moving towards the 10G converged adapters to achieve the benefits of consolidation of network and storage especially on Blade Server Architecture.

I am writing this post to provide you guidelines on how you can divide a 10GB CNA card on your ESXi server to meet all the network and storage requirements. Before that, let’s have a look at what is the 10Gig CNA and what are the brands available in the market available for this technology.

A CNA card a.k.a "Converged Network Adapter" is an I/O card on a X86 server, that combines the functionality of a host bus adapter (HBA) with a network interface controller (NIC). In other words it "converges" access to, respectively, a storage area network and a general-purpose computer network. As simple as it sounds, it makes things simple in the datacenter as well. Instead of running down those cables from each NIC card/FC HBA or iSCSI cards, you can just use a single cable to do all these tasks for you. This is because the CNA card is converged and can carry all the traffic on a single physical interface.

There are a number of manufacturers of such cards who either manufacture these cards themselves or just re-brand them with their logo and custom firmware.. Here are a few examples:-

- Cisco

- Dell

- HP

- Q-Logic

- Emulex

- IBM etc..

So as a customer you have a number of choices and it is important that you choose what fits your existing infrastructure or the new hardware if it is a Greenfield site.

Let's say you bought a CNA which gives you 4 virtual ports per physical port, let’s see how we can divide the bandwidth of this physical port amongst the virtual port to both Storage and Network Communication.

On the physical card the bandwidth can be divided like how it is shown in the figure below:-

Here, the CNA card has 2 physical port each with 10 GB bandwidth. I have further divided this card into 3 network cards and 1 FC HBA per physical port. Hence, I will have a total of 6 Network cards and 2 FC HBA per CAN card. If you like the concept of No Single Point of Failure (SPOF) and can afford another card, and then you would end up having 12 NIC Cards and 4 FC HBA Ports per Blade server.

Isn't that cool?? A Blade server which so many NICs. Well this can be used on Rack servers as well as it will also reduce the back-end cabling!

Now a last look at how I would use these NICs and FC Ports to configure the networking for the ESXi Server. The diagram below shows how I would configure the networking on my ESXi server to get the best possible configuration out of the available hardware resources.

The Diagram above clearly shows how we have divided this bandwidth amongst all the required port groups. If you have 2 such cards, you will have high resiliency in your design and the number of ports would double up providing better performance as well.

Remember you are free to toggle around the bandwidth for the Virtual NICs and Virtual FC HBA’s basis how much you want for your port groups. The bandwidths which I have mentioned above are a guideline and can be used as they fit in most the bills.

Hope this helps you design the network and storage with the 10GB Adapter without issues.

**************************************************

Update - 3rd April - Look at this new article - Dividing Bandwidth of a 10 GB CNA Adapter for ESXi Networking and Storage using Network I/O Control (NIOC) which talks about using Network IO Control to do the network segregation.

Using scripts to automate VMware Site Recovery Manager workflows & recovery steps!

While I am trying to cope up with projects, new VMware products, training and new initiatives for the VMware community, I quickly thought of coming back to the audience and talk about something which I have discussed before in one of my post and share my experience around that.

Well, I am talking about an article which I wrote on Guest IP Customization on VMware SRM. I would suggest you read that first here before reading this to make more sense out of what we are trying to achieve with this post. Here is the link -  Things to Know About Guest IP Customization in VMware Site Recovery Manager!

Coming back to where I left. We know that certain operating systems, mostly Unix based might not support OS Customization, because of which we are unable to use the Guest IP Customization option in the SRM workflow. This feature allows us to insert a new IP address using the OS API's on a virtual machine at the time of Test Recovery or Disaster Recovery workflow execution. If the OS does not support IP customization then you would have to manually change the IP address on the recovered virtual machine on the DR site, this will impact the overall RTO (Recovery Time Objective) to resume services on the DR site. Though this should not impact a Test Drill much, however this method is prone to human errors and can be painful to execute at the time of disaster.

VMware SRM gives a great feature which can be used to achieve the desired automation and agility for recovering VM's with the required IP settings for Guest OSes which do not support customization. If you ever notice the recovery workflow, you would see that you are able to add pre-power on and post-power on commands to a recovery workflow. These commands can then either talk to a central repository of scripts which can be created in the SRM server or on the recovered virtual machine itself. These commands can be called if you select the respective recovery plan which recovers the virtual machines in question. The screenshot below shows how this window looks like in the SRM recovery workflow:-

In this post we will learn how to use the option "COMMAND ON RECOVERED VM". Here is the situation.

SCENARIO==> You have a virtual machine protected using SRM which is running Oracle Linux. Once this VM gets recovered on the DR site during test recovery or disaster recovery, you want the IP address and other settings to change as soon as the VM comes up on the DR site. Since Guest IP Customization will not work on Oracle Linux you would get the following error if you try to use that method - "Error: The Guest operating system "oracleLinux64Guest" is not supported." How will we solve this issue?

SOLUTION==> To solve this issue we would use the Command on Recovered VM feature of SRM workflow. Here is how you navigate to this window:-

1	Click Recovery Plans in the left pane, and select the recovery plan which has the VM in question.
2	Click the Virtual Machines tab.
3	Right-click the virtual machine and click Configure.
4	Select Pre-Power On Steps or Post Power On Steps in the left pane, and click Add.
5	Select Command on Recovered VM.
6	In the Name text box, type a name for the step. "Command to call IP Address Change Script."
7	In the Content text box, type the commands for the step to run. /bin/sh<space>/etc/network-scripts/changeip.sh  (I will share the content of the script changeip.sh in a moment)
8	(Optional) Modify the Timeout setting.
9	Click OK to add the step to the recovery plan.
10	Click OK to reconfigure the virtual machine to run the command before or after it powers on. Here is the screenshot from my server:-

Here is how my script chnageip.sh looks like:-



#!/bin/bash
#################################################################
# Injecting network files for DR environment
/bin/cp -p /etc/network-scripts/network /etc/sysconfig/network 
/sbin/init 6
#End of script.

Let me describe this script. I am copying the files which I have already kept on the Virtual Machine on my primary site with the new IP settings on the following location - /etc/network-scripts/network. These files are replicated to the DR VM image automatically as a part of my replication plan, whether storage based or vSphere Replication.

With this script I am copying the 'network' file which has all the new settings to the /etc/sysconfig/ which will replace the existing IP settings of the Linux machine to the new IP address which we have defined in our DR Network file. After that the VM would reboot to power on with the new settings.

Similar to this script you can do a bunch of things by calling such scripts on the recovered VM's, such as:-

a) Change any other OS settings such as name, IP, network settings etc.
b) Run cron jobs or schedule tasks.
c) Run configuration files to make changes in an application configuration.
d) Do wonders ;-)


Well now when you run this recovery plan, your VM will power on on the DR site and then as soon as the VMware tools is up on the VM, SRM workflow will call this script and the script will change the network settings of the VM as scripted by you automatically. Automation and agility at its best. :-)

You might have questions about rights to run this script etc.. Remember VMware Tools is used here, so the script should have the rights same as the user account which was used to install VMware Tools, normally its domain admin on 'Windows' and root on 'Linux' OSes.

Well that's all for this time, I hope this would help you get things going on SRM and use the amazing options which SRM provides you to automate stuff. In my upcoming article, I would speak about how to use the option of using "Command on SRM server".

Do let me know if you have any questions or comments. Hope this helps..

A blog series on "Basics of VMware Virtualization" - Share your opinion!!

The idea about this small survey came out of some numerous instances where, college freshers, people in the industry, professionals from other technology sectors etc. have asked me about learning VMware technology.

I have answered these questions in a different way every-time. For someone who was not as lucky as me to join VMware and learn the technology from some amazing colleagues and mentors, how do I create a platform from where they can start from scratch and work there way up to be at par with Certified Professionals. As tempting as it looks for someone who wants to lean, this will take some tremendous amount of work and late nights for me as an author.

In order to ensure that this idea makes sense and would benefit the TO BE VMware community, I would request you to attend this small survey with just 2 questions and let me know your opinion. 

Your response would matter a lot to me.

Here is the link:-

http://www.sogosurvey.com/k/SsUSXPsRsPsPsP  

Thanks once again!!

Reclaiming Waste Capacity using vCOPS - How to Calculate Waste & Usage Accurately!

This article is coming out of a discussion which I had around a month ago with one of my colleague and then with a customer about the Capacity Management feature of vCenter Operations Manager (vCOPS).

If you are new to vCOPS, I would recommend reading my other posts on vCOPS which would make you familiar with this topic:

vCenter Operations Manager - Solving Performance, Capacity and Configuration Problems!!

Right Sizing vCenter Operations Manager vAPP For Efficient Performance !!

I believe this topic is worth writing about as this might help you understand how Capacity IQ which is now rolled up into vCenter Operations Manager (vCOPS), calculates the usage of resources for a Virtual Machine or for that matter any object in the vCenter. The resource usage ultimately helps the tool to monitor the Capacity Utilization over a period of time. This Capacity utilization leads to calculation of 2 minor badges:-

a) Reclaimable Waste, and

b) Density

These two values then roll up into a Main Badge known as "EFFICIENCY". A score of 100 on efficiency means that you are using the virtual infrastructure in the most appropriate way, and as that score starts reducing, you know that either you have virtual machines which are Over Sized or Under Sized which will lead to waste of resources or performance issues due to resource contention.

Below is a screenshot which shows how the efficiency badge and the sub-badges show in the vCenter Operations Dashboard.

At the end of the day, efficiency is the most important piece of information which the Capacity Management feature of vCOPS provides. From the perspective of an IT buyer, it becomes a tool which helps you to ensure that you do not waste any resources in your infrastructure by following primitive methods of resource allocation to servers and applications.

Hence, this allows you to right size your infrastructure as you operate and manage it. 

For example, a new application which needs to be deployed in your infrastructure needs a Windows 2008 R2 VM, with 4 vCPU and 16 GB of RAM as per the application owner. This might be a practice which is being carried forward by the application owner from the world of physical servers. However, with Virtual it is quite possible that the VM will never use the allocated capacity. The challenge is that how can we capture this data and present it back to the application owner.

vCOPS has the answer - Once this machine is created and the server goes into production, vCOPS would start monitoring this virtual machine on a regular basis and would capture data around utilization of CPU & Memory. After a period of 30 to 45 days, vCOPS would understand the capacity utilization patterns of this virtual machine. After this, a report in vCOPS about Reclaimable Waste will easily tell you about all the virtual machines which are over-sized on CPU or Memory. On the basis of this report you can reclaim the resource and save a lot of money for your organization by increasing the efficiency of the hardware. 

While I say this, it is important that you have the correct settings to monitor the utilization capacity and usage patterns of your virtual infrastructure. In a business environment where the servers work between 9 AM to 6 PM, Monday to Friday, it is important that you capture the utilization patterns during this period to calculate the reclaimable waste and density of the Virtual Machine. In such a scenario if you set the monitoring days and time to 24/7, you will end up capturing a lot of skewed data which does not reflect the correct business cycles. This will ultimately result in a very low efficiency and a huge amount of reclaimable waste which might not be TRUE otherwise.

To avoid such problems, follow the settings on the screenshot mentioned below and you should be good to go.

This would ensure that you capture the right data and process it into valid information which will help you manage capacity in your Virtual Infrastructure. It's important that any decision regarding capacity is not taken in a haste. Rather, we should ensure that we customize the settings for monitoring capacity on the basis of our own environment and then let the tool run for a period of 4 weeks to 6 weeks before you start looking into the results and begin to make changes for the betterment of your Virtual Infrastructure.

Hope this will help you learn more about what vCOPS can do for you and how you can do such tasks accurately.

Right Sizing vCenter Operations Manager vApp For Efficient Performance !!

Out of a recent engagement on vCenter Operations Manager, the most important discussion point which came out was the sizing of vCOPS vAPP. There were other discussions as well which are interesting and I would write and share about those facts, however I thought this should be a good start towards understanding what vCOPS can do for you and how the solution should be sized for best results.

If you are new to vCOPS and need to understand the basics about vCOPS, you should refer to one of my previous article about vCOPS - vCenter Operations Manager - Solving Performance, Capacity and Configuration Problems!!

After going through that post you would know that vCOPS is available as a packaged vAPP which consists of two Virtual Machines. These machines are called:-

Analytics Virtual Machine and UI Virtual Machine

The diagram below shows the architecture of this vAPP:-

In this article I will not explain each of these components as it is clearly defined in a free VMware MyLearn training on the following link - VMware vCenter Operations Manager Fundamentals [V5.X]. I would highly recommend this training to anyone who wants to learn about vCOPS to understand the basics of this solution.

Now, coming back to sizing these virtual machines. 

Analytics VM - On a high level you need to understand that the Analytics VM is the one which does most of the work and also gets all the data in form of Raw Metrics. All the algorithms regarding the Performance Analytics run in this virtual machine and the same are stored in separate database which are hosted on this virtual machine.

UI (User-Interface) VM - This is where the Capacity IQ is hosted along with the Admin User-Interface, vSphere User-Interface and the Custom User-Interface. The Capacity related data is stored in a database which is a part of this virtual machine.

Now, the question which rises out from here is that how do we size these virtual machines in terms of CPU & Memory resources and how much storage should be assigned to these Virtual Machines to ensure that all the components of this application are able to run successfully and also perform all the tasks which you expect them to.

Since we know that vCOPS extends its offering to vSphere and Non vSphere environments, we need to look at sizing requirements from both perspectives:-

a) Based on the number of VMs – Works well for a vSphere-centric Environment.

b) Based on the number of Metrics – Works well when adding non vSphere adapters

Let's look at the numbers for a vSphere Environment for CPU, RAM & Storage

In case you are using a collector to collect metrics data from Non-vSphere environment, then you would need to calculate and add CPU, Memory & Disk resources on the basis of the recommended numbers below:-

I hope this will help you to size your vCOPS vAPP appropriately at the time of deployment. Right sizing will ensure that you do not face any issues with this application while it's monitoring the performance, showcasing capacity and ensuring standard configurations and compliance in your vSphere & Non-vSphere environment.