Running Scripts to Automate SRM Recovery Plans using Powershell / PowerCli !

In my last post, I took a deep dive on how to run commands within the Recovery Plan, which would call out scripts on the recovered Virtual Machines. I would highly recommend that you read that article before reading this one. Here is the link - Using scripts to automate VMware Site Recovery Manager workflows & recovery steps!

Now is the time to take this one level up and see how SRM can integrate with other powerful tools such as PowerShell and PowerCLI to create a centralized script repository on the SRM server. This repository can contain scripts which can be executed by the PowerShell scripting engine with the help of PowerCLI installed on the SRM Server.

I think a better approach would be to go ahead and take a scenario which needs such kind of scripting and I will showcase step by step configuration which would help use solve the problem through scripting.

Situation:-

An organization is using vSphere Replication to replicate the virtual machines from Site A (Protected Site) to Site B (Recovery Site). They have the SRM automation engine to perform Recovery or Test.

There are a few virtual machines protected at Site A with an application which generates a lot of garbage logs. These garbage logs are a part of the OS and the VM , hence this data is also replicated to the DR site using vSphere Replication.

Since this data is not required at the recovery site, we are simply wasting the replication bandwidth to send these logs which could be in GB’s in some cases per virtual machine. The organization wants to efficiently use the Network between sites and want to remove any replication overheads which might not be required in the Recovery Site.

Solution:-

The solution to this issue is that we keep these logs on a separate log partition (which is on a different VMDK). vSphere Replication allows us to go to the granularity of choosing the VMDK’s which you want to replicate from a Virtual machine.

Let’s say the name of the VM which we want to replicate without the logs is called “vmsunny” which has 2 Virtual Disks.

Disk 1 – vmsunny.vmdk

Disk 2 – vmsunny_1.vmdk

The Disk 1 has the OS, Application and Data while the Disk 2 has the logs which are of no use in the DR site.

In this case, we will just replicate the Disk 2, once to create a copy on the DR site and then when we configure vSphere Replication for this VM, we would not replicate the Disk 2 to save the replication bandwidth from next time onwards.

Now we will use the SRM scripting capabilities to add the disk 2 on the DR virtual machine whenever we run a Test Recovery or a Disaster Recovery. We would need to script this task so that it happens automatically with the Recovery Workflow.

How to Implement

Here are the logical steps which I would follow in this case:-

a)      Replicate the ‘vmsunny’ vm from Site A to Site B, replicate both Disk 1 and Disk 2

b)      Once both the disks are created and the replication is complete, pause the replication, go the Site B datastore where Disk 1 and Disk 2 are located and move the Disk 2 out of the VM folder (This is to ensure that Disk 2 is preserved when you un-replicate the Disk 2 in next step).

c)      Now go back to vSphere Replication and configure replication for vmsunny once again. This time, DO NOT replicate Disk 2 i.e. vmsunny_1.vmdk. This will remove this disk from replicating from now on.

d)     Remove this disk from the Protection group as well by using the Detach option.

e)      Now go back to the location where you moved the Disk 2 in the DR site. Move the Disk 2 back to the VM folder from where you moved it in Step 2.

f)       Resume the replication which you paused in Step 2.

Now we would configure the Recovery Plan for ‘vmsunny’ VM and add a command to call a script from the SRM Server. This script would add the Disk 2 on the VM in the DR site as soon as the Test Recovery or Disaster Recovery option is selected for that recovery plan. This would be a pre-power on script which would add the disk and then the VM would power on.

Preparing the SRM server for running scripts & Creating the Recovery Plan

Before you could call scripts from the SRM server, you need to configure Powershell and PowerCLI on the SRM server. This would allow the SRM server to run powershell scripts using the VIX API and call the virtual center to add a disk to a virtual machine.

Please follow the steps in the following article written by Alan Renouf:-

Back to Basics: Part 1 – Installing PowerCLI

This will help you prepare your SRM Server for running PowerCli scripts which would call the following script kept on a folder in the SRM server:-

++++++++++++++++++START OF SCRIPT++++++++++++++++++

add-pssnapin VMware.VimAutomation.Core

Connect-VIServer -Server <vCenter Server Name> -WarningAction SilentlyContinue

New-HardDisk -VM "Virtual Machine Name" -Persistence IndependentPersistent -DiskPath "PATH TO  VMDK AT DR SITE"

++++++++++++++++++END OF SCRIPT++++++++++++++++++

Sample Example:-

add-pssnapin VMware.VimAutomation.Core

Connect-VIServer -Server mylabvcenter.lab.com -WarningAction SilentlyContinue

New-HardDisk -VM "vmsunny" -Persistence IndependentPersistent -DiskPath "[EVA DISK1] vmsunny/vmsunny_1.vmdk"

This script would be placed on a folder in the SRM Server, let’s say it is located on the SRM server on c:\scripts folder. In order to call this script we would add this to the SRM recovery plan as a pre-power on script as shown in the screenshot below:-

Let us assume that the script which we need to call is located on C:\scripts with the name as sunnyvm.ps1. We will call this script as shown in the screenshot below:-

Here we are calling the powershell application which is calling sunnyvm.ps1 located on the SRM server.

sunnyvm.ps1 contains the following script as stated before:-

add-pssnapin VMware.VimAutomation.Core

Connect-VIServer -Server mylabvcenter.lab.com -WarningAction SilentlyContinue

New-HardDisk -VM "vmsunny" -Persistence IndependentPersistent -DiskPath "[EVA DISK1] vmsunny/vmsunny_1.vmdk"

This would add vmsunny_1.vmdk to the sunnyvm and we then the vm will boot up. Now all the logging can be done on this disk when the VM is either recovered at the DR site or is powered on in a Test Recovery Mode.

Using this engine, you can do other powerful things in your SRM environment, such as Re-sizing vCPU or Memory allocations, add/remove disks or any  automation task which would make your recovery automatic and error free. Please feel free to share this article and your thoughts using the comment window.

Using scripts to automate VMware Site Recovery Manager workflows & recovery steps!

While I am trying to cope up with projects, new VMware products, training and new initiatives for the VMware community, I quickly thought of coming back to the audience and talk about something which I have discussed before in one of my post and share my experience around that.

Well, I am talking about an article which I wrote on Guest IP Customization on VMware SRM. I would suggest you read that first here before reading this to make more sense out of what we are trying to achieve with this post. Here is the link -  Things to Know About Guest IP Customization in VMware Site Recovery Manager!

Coming back to where I left. We know that certain operating systems, mostly Unix based might not support OS Customization, because of which we are unable to use the Guest IP Customization option in the SRM workflow. This feature allows us to insert a new IP address using the OS API's on a virtual machine at the time of Test Recovery or Disaster Recovery workflow execution. If the OS does not support IP customization then you would have to manually change the IP address on the recovered virtual machine on the DR site, this will impact the overall RTO (Recovery Time Objective) to resume services on the DR site. Though this should not impact a Test Drill much, however this method is prone to human errors and can be painful to execute at the time of disaster.

VMware SRM gives a great feature which can be used to achieve the desired automation and agility for recovering VM's with the required IP settings for Guest OSes which do not support customization. If you ever notice the recovery workflow, you would see that you are able to add pre-power on and post-power on commands to a recovery workflow. These commands can then either talk to a central repository of scripts which can be created in the SRM server or on the recovered virtual machine itself. These commands can be called if you select the respective recovery plan which recovers the virtual machines in question. The screenshot below shows how this window looks like in the SRM recovery workflow:-

In this post we will learn how to use the option "COMMAND ON RECOVERED VM". Here is the situation.

SCENARIO==> You have a virtual machine protected using SRM which is running Oracle Linux. Once this VM gets recovered on the DR site during test recovery or disaster recovery, you want the IP address and other settings to change as soon as the VM comes up on the DR site. Since Guest IP Customization will not work on Oracle Linux you would get the following error if you try to use that method - "Error: The Guest operating system "oracleLinux64Guest" is not supported." How will we solve this issue?

SOLUTION==> To solve this issue we would use the Command on Recovered VM feature of SRM workflow. Here is how you navigate to this window:-

1	Click Recovery Plans in the left pane, and select the recovery plan which has the VM in question.
2	Click the Virtual Machines tab.
3	Right-click the virtual machine and click Configure.
4	Select Pre-Power On Steps or Post Power On Steps in the left pane, and click Add.
5	Select Command on Recovered VM.
6	In the Name text box, type a name for the step. "Command to call IP Address Change Script."
7	In the Content text box, type the commands for the step to run. /bin/sh<space>/etc/network-scripts/changeip.sh  (I will share the content of the script changeip.sh in a moment)
8	(Optional) Modify the Timeout setting.
9	Click OK to add the step to the recovery plan.
10	Click OK to reconfigure the virtual machine to run the command before or after it powers on. Here is the screenshot from my server:-

Here is how my script chnageip.sh looks like:-



#!/bin/bash
#################################################################
# Injecting network files for DR environment
/bin/cp -p /etc/network-scripts/network /etc/sysconfig/network 
/sbin/init 6
#End of script.

Let me describe this script. I am copying the files which I have already kept on the Virtual Machine on my primary site with the new IP settings on the following location - /etc/network-scripts/network. These files are replicated to the DR VM image automatically as a part of my replication plan, whether storage based or vSphere Replication.

With this script I am copying the 'network' file which has all the new settings to the /etc/sysconfig/ which will replace the existing IP settings of the Linux machine to the new IP address which we have defined in our DR Network file. After that the VM would reboot to power on with the new settings.

Similar to this script you can do a bunch of things by calling such scripts on the recovered VM's, such as:-

a) Change any other OS settings such as name, IP, network settings etc.
b) Run cron jobs or schedule tasks.
c) Run configuration files to make changes in an application configuration.
d) Do wonders ;-)


Well now when you run this recovery plan, your VM will power on on the DR site and then as soon as the VMware tools is up on the VM, SRM workflow will call this script and the script will change the network settings of the VM as scripted by you automatically. Automation and agility at its best. :-)

You might have questions about rights to run this script etc.. Remember VMware Tools is used here, so the script should have the rights same as the user account which was used to install VMware Tools, normally its domain admin on 'Windows' and root on 'Linux' OSes.

Well that's all for this time, I hope this would help you get things going on SRM and use the amazing options which SRM provides you to automate stuff. In my upcoming article, I would speak about how to use the option of using "Command on SRM server".

Do let me know if you have any questions or comments. Hope this helps..