VMware vSphere - What is the correct way to right size a virtual machine? (Situation 2: Sizing using Virtualization Assessment Tools)

VMware vSphere - What is the correct way to right size a virtual machine? (Situation 2: Sizing using Virtualization Assessment Tools)

I got some great feedback on my last post (Situation 1: Sizing for an RFP Response), so here I am continuing with the part 2 of this post. 

Now, if you are facing this situation, then you can be sure about the fact that you are already engaged in a project with the customer. The customer would now let you in his Datacenter and ask you to do whatever you can to find out what they are running and size the close to final target virtual infrastructure for the them.

For such a sizing we cannot depend on assumptions and we would need to be more realistic with the numbers. The only way to be realistic with the numbers, is to use tools which can go and sniff out the existing physical infrastructure and help you gather data which can be processed into useful information.

In this post, I will speak about the various options available, to collect this data and then see how to process it and come to final sizing of the virtual infrastructure. This will take you one step closer to the final goal of adopting or implementing Virtualization in a customer's environment.

Before we begin, I cannot stress more in saying that this is the most important step in the entire project as this would decide if the Virtualization initiative taken by the CIO of the organization is a success or a failure. Yes, sizing is very important. If you have a look at any virtual environment which has performance issues, 95% of the times this would be due to incorrect sizing of the virtual machines.

So lets begin the joyride....

Lets see what are the options available...

Now in the life-cycle of a Virtualization and consolidation project this phase is termed as ASSESSMENT. An assessment of the As-is state helps you find the GAP between the As-is and the To-Be (target) state. This process is called Gap Analysis. To perform such analysis for a Virtualization project, there are a number of tools available in the market. I will talk about the most famous and frequently used tools here:-

I - VMware Capacity Planner - This, I believe is the most widely used tool for carrying out a Virtualization assessment. As the name suggests this tool is from VMware. Originally this product was owned by a company named AOG. VMware has been quite strategic with its acquisitions and hence they were quick to realize the importance of this tool and acquired this in the year 2005 and named it VMware Capacity Planner.

To add to this, VMware decided to offer this tool for free to partners. So if you are a VMware partner and looking to assess a customer's environment, then you can actually do it for FREE. You just need to finish a MyLearn course which is self paced and free of cost. Here is the link to the course:-

http://mylearn.vmware.com/mgrReg/courses.cfm?ui=www_field&a=det&id_course=46564

Once done with this course you would need to go to the following link:--

https://optimize.vmware.com/index.cfm

Here you would be able to click on the link to "Request a New Account" (if you are a registered partner with VMware) and the rest of the process is pretty standard.

Once you have this account in place, you would be able to log in on the capacity planner portal and create assessment projects for Server Virtualization and Desktop Virtualization.

Let me get back to the track now. Using this tool would be simple enough once you finish that 5 hours self-paced course. The important thing is to ensure that you are using the tool to get you the best possible results. Here are a few pointers which I can give you:-

a) Ensure that the assessment period is a minimum of 4 weeks to get sufficient samples from the tools.

b) Try to cover the peaks and drops of the business during your assessment.

c) Be ready to make assumptions for certain periods which you cannot cover during the assessment, such as Quarter Ends, Year Ends etc. Also, make sure that your customer gives you the maximum inputs in order to make these assumptions as accurate as possible.

d) Needless to say that you would need Local Admin / Root credentials to ensure that you are able to collect the data from your physical workloads without any issues.

Now, this tool has one unique feature which might be a positive or a negative feature. The tool actually uses a local Data Collector which is a piece of software installed on a box which will be sitting in the datacenter of the customer and collecting all the performance values, to name a few important one's

a) CPU Utilization

b) Memory Utilization

c) Disk utilization & IOPS

d) Network Utilization

This data is then uploaded on to the VMware Data warehouse over the internet. This Data warehouse is linked to the VMware Capacity Planner portal. The beauty of the product is that this portal will automatically do the assessment of the data which is uploaded and at the end of the assessment will provide you with an assessment report which will go to the details of sizing the entire target ESXi infrastructure for you. Needless to say that it uses the utilization data of the current physical servers and uses its algorithms to define the size of these machines once they are virtualized. 

Along with sizing, this tool would also let you down-select the good candidates for Virtualization by looking at the utilization of the physical servers. This ability helps you to ensure that you are taking the right step forward towards Virtualization.

The only reason why I would not use this tool is if my customer is sensitive about uploading his inventory information to an unknown destination where it is processed. Though there are options to hide the information such as host name, IP addresses etc within the tool, this is more to do with the Security Standards of the customer than an actual risk. Hence there are customers who would not go this route.

Lets see how we work with such customers.

II -  Platespin Recon -  might help you in such a situation. This is not only an assessment tool but a tool which helps you to create Virtualization and migration projects on a whole. Moreover, this can be installed on premise i.e. in the customer datacenter and the entire assessment study can be done within the datacenter. This also links with the other product of Novell which is widely used for P2V's, V2V's, P2P's & V2P's. The product is call Platespin Migrate. 

But as we all know "Their are no Free Lunches", this tool comes with a cost. You would pay for the assessment and then if you would want to use the Migrate tool, you would have to shell out a few bucks for that as well. You can get more information about the pricing & packaging of these products on the Novell Website:- http://www.novell.com/solutions/virtualization-workload/

To know more about these products and how they compare with each other, you can visit the following blogs which have tons of great information about them:-

http://verbeiren.blogspot.com/2008/01/vmware-capacity-planner-vs-platespin.html

http://communities.vmware.com/thread/94640?start=0&tstart=0

Now, you would also find customers who would not want to use any tools, yes I have faced some of those as well. They could have there own reasons, but it does not help a consultant for sure. Lets see how to do your assessment in such situations.

Note: - This one is only applicable to Windows environment as we will be using the native performance tool aka PERFMON.

III - Perfmon (Performance Monitor) -  This utility is u sally leveraged by all the capacity assessment tools in order to capture data from Windows servers. 

Here are a few steps can use to configure Perfmon :

1 -      Click start, run, type Perfmon and click OK

2 -      Right click on User Defined as shown in the screenshot and click on “New->Data Collector Set”.

   3-      Type the server name as “<Servername>_Perfdata_DBS” in the name field and select the radio button “Create from a template (Recommended)” and click on Next.

   

    4- Click on Browse and select the file “Perfdata_DBS” and click on OPEN. (Perfdata_DBS) is a template which can be created in perfmon for collecting the following metrics:-

- Total CPU Peak Utilization: from “% Processor Time”  

- Memory Peak Utilization:  from “Available Mbytes”

- Disk Peak IOPS: from “Disk Transfers/sec” \

- Network Peak BPS: from “Bytes Total/sec” 

- CPU Queue Peak: from “Processor Queue Length” (optional)

- Disk Queue Peak: from ”Current Disk Queue Length” (optional)

    5-    Click next to select the output file location. 

    6-    Click Next and Click Finish on the following Screen   

    7-    On the Perfmon screen right click on the listed job and click Properties.

    8-   Click on Schedule TAB to define the job Schedule. Add a start DATE and an END Date.  (This period should be anywhere between 2 to 4 weeks. It’s good to have 4 weeks of DATA.)                     

    9-    Now right click on the JOB and click on START.

     Once the Job is ran for the defined period, fetch the .CSV file which has the results.

     Along with the performance data you can use tools such as WinAudit to collect information about the physcial servers. Now with these results you will be able to get the Peak utilization values. The best way to size with these numbers is to use the the percentile formula in mathematics. 

      In simpler terms, pick the 10% of values which are with peaks for all the counters. Average these values to get the Peak Average Utilization. Add a buffer to 20% to this data for each virtual machine (for CPU & Memory), this will help you derive the final size.

Below is how the VM resources are obtained

a) vCPU – The Peak CPU utilization will be obtained based on the 90^th Percentile. This figure will be in Mhz. An added buffer of 20% would be added to this value for a head room.  Determine the vCPU required for the VM by  dividing the CPU cycles required by the Mhz per core. Each VM will be given a minimum of 2 vCPU.

The vCPU allocated to the VM will be in increments of 2. This is to cater for applications which can only support even number for vCPU. e,g 2vCPU, 4vCPU, 6vCPU….

example :   Peak CPU utilization based on 90 percentile = 3550 Mhz

                 Utilization after adding 20% Buffer =  4260 Mhz

                 Mhz per Core            = 2660 (approximate value on the new server)

                 Proposed vCPU required    = 4260/2660

                                                         =  1.601

                                                         = 2 (rounded to the nearest 2vCPU)

b) Memory - The Peak Memory utilization will be obtained based on the 90^th Percentile. This figure will be in MB. An added buffer of 20% would be added to this value for a head room. This figure will be further rounded up to the nearest GB. Each VM will be given a minimum of 2 GB.

example :   Peak Memory utilization based on 90 percentile = 722 MB

                 Utilization after adding 20% Buffer =  866MB

                 Memory required after rounding to the nearest GB is 1024MB. 

So with the exercise you would be able to size each virtual machine. Though this is a crude way of doing things, but this might fetch the results for you to get the information you need. You can refer to my previous post to see how would you arrive to the number of ESXi servers required to run the virtual machines.

Well, I would end this series with this high level overview and approach. I will come back with some more topics which would help you understand and learn more about the Virtual World.

Till then...

Happy Virtualizing

Regards

Sunny Dua

Enter your email address:


Delivered by FeedBurner

VMware vSphere - What is the correct way to right size a virtual machine?? (Situation 1: Sizing for an RFP Response)

VMware vSphere - What is the correct way to right size a virtual machine?? (Situation 1: Sizing for an RFP Response)

Being in the field and working with a lot of VMware customers and partners, I often get questions about how to size my virtual machines. I often answer this question in different ways as the answer to this topic is quite subjective and the approach to look at this issue would change based on various situations.

Disclaimer:- These sizing methodologies are based on experiences and best practices, hence there is nothing right or wrong, this is scenario based, hence use this as a guideline and not a thumb rule.

For ease of understanding, let me break down the discussions into different situations. In this article I will speak about the first situation. 

Situation 1: Sizing for an RFP response::

This section is for consultants who work on responding to RFP's which have scope of Server Virtualization and Consolidation. Usually in a RFI or RFP situation, the amount of information given by customers is limited. On the basis of this limited information, the customers expect you to size and price the target virtual infrastructure. This situation can be divided into 2 scenarios:-

Scenario A- The inventory of existing servers is available. Such inventory includes data points like Server Count, CPU / Cores per Server, Provisioned Memory, Provisioned Storage etc. This data may or may not include the utilization data for these servers. Usually customers pull out this inventory using some existing tool in the infrastructure such as a ITM, SCOM etc. 

Lets see an example which will help is size a single server with this information:-

The current physical server has the following data available:-

Server Name - WEBROLE1

CPU Specs - 2 CPU, Dual core, 2.4 GHz

Memory - 8 GB

HDD - 500 GB

OS - Windows 2003 Standard Edition 

Utilization data (Assuming it is available)

Avg CPU Utilization - 5%

Avg Memory Utilization - 8%

Disk Utilization - 47%

Now lets look at the Average utilization in the real terms:-

Total CPU MHz available on the server = 2 CPU x 2 Cores x 2400 MHz = 9600MHz

Average used in MHz = 5% of 9600 Mhz = 480 Mhz 

Total Memory available on the server = 8192 MB

Average used in MB's= 8% of 8192 MB = 656 MB

Total Disk available on the server = 500 GB

Used in GB's = 235 GB

So now if you look at the utilization we get the following specs

CPU = 480 MHz

Memory = 656 MB

Disk = 235 GB (Actual Used)

Is this enough to size the target machine??  The answer is a BIG NO.

Though I said there are no THUMB RULES, there is one RULE which we should always consider.

"Never Size on Average Utilization" - "Always Size on Peak Utilization"

Hence if we have the the average utilization data, the target virtual machine CPU & Memory size CAN NOT be derived using the values as described above.

In such a scenario we will either use the PEAK UTILIZATION values, if given by the customer, else we would have to go into an assumption mode based on which you would size the VM.

Assuming the values given the example above are PEAK Utilization values then target utilization of the virtual machine would be:-

CPU = 480 MHz

Memory = 656 MB

Now, it is always a best practice to add a buffer to the Peak Utilization values. Hence add a buffer of 25% to the Peak Utilization Values:

CPU = 480 MHz + 25%    = 600 MHz

Memory = 656 MB + 25% = 820 MB

There are 2 reasons why we added this 25%:

i) The peak utilization data is a single peak point collected, however there could be multiple peak points  across business cycles which we needs to address, hence a buffer is always good.

ii) Its good to have some head room for situations where the memory utilization shoots up due to a misbehaved service, process, application etc.

Now lets consider the other factors which might influence this sizing:-

i) We should look at the OS requirement as well to ensure that we do not go below the requirement while right sizing the virtual machine. Hence in this case, for a Windows 2003 Standard, Microsoft recommends a minimum of 2 GB of RAM, hence the vRAM for the above example would trim down to a minimum of 2 GB.

ii) For number of vCPU's for the above example, we see that we need only 600 MHz of processing power to suffice the workload need, however we should ensure that we meet the minimum OS requirement to derive the required clock speed. At the same time the decision to introduce vSMP (more than one vCPU) in a virtual machine is a tricky one. Ensure to minimize the number of vCPU's to as low as possible to ensure that we are not introducing the factor of "CPU Ready TIME" on the virtual machine. For situations where the number of vCPU's on a ESXi server are more that the underlying hardware cores, Time Sharing is introduced. This time sharing can lead to CPU Ready time in order to schedule CPU threads during contention. For more on this read Performance Best Practices for VMware vSphere 5.0. This will help you understand why more is not always better. So in the above case since the server role was using multiple cores in the physical world, I would give it 2 vCPU's. Since this is an RFP stage, you might not have the privilege to speak to an application owner, hence please clearly state your assumptions and reasons for taking this value.

Hence, with the above example, we were able to size the target virtual machine's CPU and Memory Requirement. The Disk requirement is an open factor and most of the RFP's will have information on how would the customer want to work with the storage space of the servers during and post Virtualization. Using the above method you can size for all the physical machines which are available to derive the total CPU and Memory Requirements to run the workloads post Virtualization.

Phewww.....

Coming back to where we started!! What if you don't have the Peak Utilization? How do you size in such case??

The answer is simple, increase you buffer window from 25% to 35%. The additional 10% would be used to cater the needs of peaks in the workloads. As always, mention this clearly in the assumptions.

I know what you are thinking... What if the customer does not give you the utilization data at all.. This might sound like a GREY area, but trust me, I have encountered such cases very regularly. In such situations, you need to take the industry assumption that any physical server which is assessed has an average utilization of 2% to 15% (This will hold good for 95% of physical servers). Hence I would recommend that you take the highest value i.e. 15% and add a buffer of 35% (to size for Peaks + Head Room) as explained above. Based on this you would end up with a 50% utilization. Though this sounds on a higher side, you would have your chances to slim down this number during the due diligence phase of any engagement.

I guess the last possibility for an RFP would be around a scenario where a customer would not give you any data at all. The statement or the end goal of the customer might just say that he wants to Virtualize 1000 physical servers. That brings us to the Scenario B.

Scenario B - Though I don't like to be in this scenario, but you can't really help if a customer expects you to give sizing and pricing based on the count of servers he has.

Lets understand this by using an Example:-

XYZ LTD. has floated an RFP with a goal to virtualize 1000 Physical servers in his datacenter. The total usable capacity of the storage being used in 10 TB. 

With this information, we need to find out the compute requirements of the virtual machines running in the end state. Lets see how we break down this issue:-

First we would have to assume sizes of the virtual machines which might come up in the end state. Taking an assumption we divide the virtual machines in his environment into 3 categories based on sizes:-

Table - 1

Virtual Machine Type	# of vCPU's	vRAM (GB)
Large	8	16
Medium	4	8
Small	2	4

After this we should look to break down the environment into 4 different categories if not mentioned by the customer:-

Table - 2 

Workload Type	Category	Percentage
Platinum	Business Critical Workloads	10%
Gold	Production Workloads	40%
Silver	Infrastructure Workloads	20%
Bronze	Test/Dev/UAT	30%

Based on the table above if we divide the 1000 servers into the defined categories, we will get the following values:-

Table - 3

Workload Type	Category	# of Servers
Platinum	Business Critical Workloads	100
Gold	Production Workloads	400
Silver	Infrastructure Workloads	200
Bronze	Test/Dev/UAT	300

Now these tables should go into your assumptions and the numbers can absolutely change based on the data given to you by the customer. if I have to define the Work load type, here are some examples of each workload type:-

PLATINUM - Production Databases, ERP, SAP, Email & Collaboration, Other Customer apps  etc.

Gold - Web Servers, Applications (packaged or custom), vCenter, CRM etc.

Silver - Directory Services, Anti Virus, DHCP, DNS, Security, Patch Management etc.

Bronze - Test, UAT & Dev environments.


The next task is to give a Virtual Machine Type to a Workload Type. In simpler terms, what would be the size of the Platinum, Gold, Silver or Bronze virtual machines. We would need to take some assumptions here to derive these numbers. The assumptions I have taken are listed in table - 4. Your table might look different if you have some information from the customer about what is the distribution of sizes across workloads.

Table - 4 

	Workload Size Percentage
Workload Type	Large	Medium	Small
Platinum	50%	25%	25%
Gold	30%	35%	35%
Silver	20%	40%	40%
Bronze	30%	20%	50%

This will give you the numbers you are looking for. Lets calculate for the these 1000 servers and see how many vCPU's and vRAM we need for each category mentioned above.

Table - 5

	Workload Size Percentage
Workload Type	Large                Servers	Medium Servers	Small Servers	Total Servers
Platinum	50	25	25	100
Gold	120	140	140	400
Silver	40	80	80	200
Bronze	90	60	150	300
_____________ Total Servers	__________________                            300	_____________ 305	___________ 395	___________ 1000

Since we now know the size per category, here is total amount of vCPU & vRAM requirement as per Table -1.

Workload  Type	Total vCPUs	Total vRAM (GB)
Platinum	550	1100
Gold	1800	3600
Silver	800	1600
Bronze	1260	2520

Now comes the biggest questions, how many vCPUs per Core? Can we over-commit memory? Have you not faced such questions before? I bet you have. Here is a guideline which I tell the customers to follow:-

Table - 6

Workload Type	vCPU to Core Ratio	Memory Ratio
Platinum	1.75 vCPU for 1 Core	1 GB vRAM for 1 GB pRAM
Gold	2.5 vCPU for 1 Core	1 GB vRAM for 1 GB pRAM
Silver	3 vCPU for 1 Core	1.4 GB vRAM for 1 GB pRAM
Bronze	4 vCPU for 1 Core	1.6 GB vRAM for 1 GB pRAM

So lets see how much physical RAM and Cores we need for XYZ LTD's physical servers to Virtualize:-

Workload  Type	Physical Cores	Physical RAM (GB)
Platinum	315	1100
Gold	720	3600
Silver	267	1143
Bronze	315	1575
Total	1617	7418

Now if your target server is a 2 CPU 10 cores Blade with 96 GB of RAM, lets see how many such blades you would need to consolidate these 1000 Physical servers.

Total Number of Servers required = 1617 / 20 = 81 Servers approx.
The total physical RAM we get is = 81 x 96 = 7776 GB

Hence to Virtualize these 1000 Physical workloads, we would need 81 ESXi servers giving us a handsome consolidation ratio of 1:13 (approximately). Voila.. we are done here...

Though Licensing is a different topic, but you would need 162 vSphere 5.0 Enterprise Plus licenses to license these ESXi servers.

I hope this helps you size your solutions correctly and respond to those RFP's with more accurate and sensible information. I will talk about other situations in my upcoming articles. Till then..

Happy Virtualizing......

Regards
Sunny Dua

Enter your email address:


Delivered by FeedBurner