Tuesday, October 29, 2013

Using Powershell to Create / Clone vCenter folders from Old vCenter to New...vCenter 5.1 migrations part 4

Wow, ok so yeah I have been lazy lately. This work stuff keeps getting in my way. We left off a couple months ago with the export/import of clusters. When we actually executed this in production the export and import tasks were separated because additional steps are needed prior to the import/creation of the clusters, but to keep within the topic I thought it best to post them together last time.
This post is one of those that would need to be done prior to the creation of clusters. As the title indicates we will be creating all of the folder objects in this post. The concept behind the folders is pretty simple, we just need to understand that there are 4 different folder types:
1. Network
2. HostAndCluster
3. Datastore
4. VM
Let's jump right in...

param(
 [string] $SourceVIserver = $(Read-Host -prompt "Source vCenter?"),
 [string] $DestinationVIserver = $(Read-Host -prompt "Destination vCenter?")
 )
$svc=connect-viserver $SourceVIserver
$dvc=connect-viserver $DestinationVIserver

$folderpaths = @()
$excludedNames = "Datacenters","vm","host","Datastore","Network"
$folders = get-folder -server $svc|?{$excludednames -notcontains $_.name}
foreach ($folder in $folders)
{
 $output = ""|select name, path, type
 $output.name = $folder.name
 $output.type = $folder.type
 $path = $folder.name
 $fld = $folder.extensiondata
 while ($fld.parent)
 {
  $fld = get-view $fld.parent -server $svc
  if ($excludedNames -notcontains $fld.name){$path = $fld.name + "\" + $path}
 }
 $output.path = $path
 $folderpaths += $output
}

$folderpaths = $folderpaths|sort path
foreach ($folderpath in $folderpaths)
{
 switch ($folderpath.type)
 {
  "Network" {$parent = get-folder -server $dvc -name "Network"}
  "HostAndCluster" {$parent = get-folder -server $dvc -name "Host"}
  "Datastore" {$parent = get-folder -server $dvc -name "Datastore"}
  "VM" {$parent = get-folder -server $dvc -name "vm"}
 }
 $fullpath = ($folderpath.path.split("\"))
 $depth = $fullpath.count
 for ($i = 1; $i -lt $depth; $i++)
 {
  if ($depth - $i -eq 1){new-folder -location $parent -name $folderpath.name}
  else{$parent = get-folder -location $parent -name $fullpath[$i]}
 }
}

disconnect-viserver * -confirm:$false -force

Breaking this down:
Lines 1-6, source and destination parameters
Lines 8-9, initialize variables
Line 10, get all folders from source vCenter, exclude system parent folders
Lines 11-25, loop through nested folders to create an array object with the full path for each folder
Lines 27-44, Create folders in their proper location
Line 46, disconnect from vCenters

Tuesday, August 6, 2013

Using Powershell to Create/Clone Clusters to new vCenter...vCenter 5.1 migrations part 3

Ok, so it has been a awhile, for good reasons though...vacation was good!  Somehow we admins, when we leave all of our work is waiting for us when we return...and then some (can I get an amen)?  Now that I am caught up with my daily admin duties lets take a look at the next steps.

It saved some code by putting together the cluster creation for both SDRS clusters and DRS clusters in one small script.  This script would be needed to create the clusters prior to importing any settings for either DRS or SDRS.  I will be doing the SDRS post next.

This script assumes that both source and target vCenters are up and running as it will connect to both and read from the source and write to the target:


param(
 [Parameter(Mandatory=$true)] $SourceVIserver = $(Read-Host -prompt "VIserver?"),
 [Parameter(Mandatory=$true)] $DestinationVIserver = $(Read-Host -prompt "VIserver?")
 )

$svc=connect-viserver $SourceVIserver
$dvc=connect-viserver $DestinationVIserver

#Clone Clusters from source to target
$sclusters=get-cluster -server $svc|sort name
$dc=get-datacenter -server $dvc

foreach ($cluster in $sclusters)
{
 new-cluster -location $dc -server $dvc -name ($cluster.name) -DRSEnabled -DRSautomationlevel "PartiallyAutomated" -confirm:$false
}

$dscs=get-datastorecluster -server $svc|sort name
foreach($dsc in $dscs)
{
 $newdscFolder = get-folder -server $dvc -name ((get-folder -server $svc -id ($dsc.extensiondata.parent)).name)
 $dscl=new-datastorecluster -location $newdscFolder -name ($dsc.name)
 $dscl|set-datastorecluster -SdrsAutomationLevel Manual
}

disconnect-viserver * -confirm:$false -force


*** Notice that when I created the DRS clusters I forced the cluster DRS automation level to Partial, the last thing I wanted was anything moving around during my upgrade.

The first foreach loop creates the DRS clusters and the second creates the SDRS clusters.  There is a bit more code in the second loop because our SDRS clusters are contained in folders for security and organization reasons.  We of course wanted things to be back the way they were.  This could have been done for the DRS clusters, but our DRS clusters were not in folders, so I just kept the code shorter.

Ok, so now we have:
- VM Details in a CSV file (post)
- DRS details in a CSV file (post)
- DRS and SDRS clusters created in our new vCenter

Thursday, June 20, 2013

Using PowerCLI to export DRS details...vCenter 5.1 migration part 2

Ok, this past week has been busy as I still have tons of hosts to get up to ESXi 5.1...I'm just so glad vCenter is done :)  Now there are quite a few details that can be contained within DRS and I knew what settings I used and wrote the scripts specifically for those.

For DRS, I just needed to know the rules (affinity and anti-affinity)

Lets look at how I dumped the DRS rules (I'm obviously not the only one who has done this):


param(
 [Parameter(Mandatory=$true)][string] $VIserver = $(Read-Host -prompt "vCenter?")
)

$vc=connect-viserver $VIserver

$outfile = "rules_$($vc.name).csv"
$rules = get-cluster -server $vc| Get-DrsRule
$output=@()
if ($rules -ne $NULL)
{
 foreach($rule in $rules){
   $line=""|select Cluster,Name,Enabled,KeepTogether,VM1,VM2
   $line.cluster = (Get-View -Id $rule.ClusterId).Name
   $line.name = $rule.Name 
   $line.Enabled = $rule.Enabled
   $line.KeepTogether = $rule.KeepTogether
   $line.VM1 = (get-view -id ($rule.VMIds)[0]).name
   $line.VM2 = (get-view -id ($rule.VMIds)[1]).name
   $output+=$line
 }
}
$output|export-csv -notypeinformation $outfile
disconnect-viserver * -confirm:$false -force

...now, of course an import script would make this so much more useful.

param(
 [Parameter(Mandatory=$true)] [string] $inputfile = $(Read-Host -prompt "input file?"),
 [Parameter(Mandatory=$true)] $VIserver = $(Read-Host -prompt "VIserver?")
 )

$vc=connect-viserver $VIserver
$rules=import-csv -path $inputfile
if($rules -eq $NULL){throw "Could not read $inputfile"}

$clusters=$rules|select cluster|sort cluster|Get-Unique -AsString|%{$_.cluster}
foreach ($cluster in $clusters)
{
 $clusterrules = $rules|?{$_.cluster -eq $cluster}
 $clusterobj=get-cluster $cluster
 foreach ($rule in $clusterrules)
 {new-drsrule -cluster $clusterobj -Name $rule.name -Enabled ([system.convert]::toBoolean($rule.enabled)) `
  -KeepTogether ([system.convert]::toBoolean($rule.keeptogether)) -VM (get-vm $rule.vm1,$rule.vm2)} 
}

disconnect-viserver * -confirm:$false -force

Alright, there we have it. First script dumps ALL rules for all clusters in a vCenter, while the second script creates those rules on the corresponding clusters in the new vCenter. I know I haven't gone over the cluster creation scripts yet, but I thought it was better to go over the export/import in one post, not exactly chronological, very much related.

Thursday, June 6, 2013

Using Powershell to Export VM details...vCenter 5.1 migration part 1

Ok, I really have no idea how many posts this is going to take, but I want to make sure they aren't too long and hopefully they will be useful for more than JUST vCenter migrations (that was my point in making them in different scripts).

The purpose of this first script was to have a COPY of what all of the VM details were before the migration started.  Thankfully I had most of this written prior to our migration, as I ran it weekly for DR purposes.  Here it is:


param(
 [string] $location=".",
 [array] $vcenters)
 
filter Get-FolderPath {
    $_ | % {
        $row = "" | select Name, Path
        $row.Name = $_.Name
        $current = $_.folder
        $path = ""
        do {
            $parent = $current
            if($parent.Name -ne "vm"){$path = $parent.Name + "\" + $path}
            $current = $current.Parent
        } while ($current.Parent -ne $null)
        $row.Path = $path
        $row
    }
}

function get-vCenterDetails
{
 param($vcenters)
 
 $allvmdetails=@()
 $vcs = Connect-VIServer $vcenters
 foreach ($vc in $vcs)
 {
  $DCs=Get-Datacenter -server $vc|Sort-Object name
  foreach ($DC in $DCs)
  {
   Write-Host "Getting DC $DC"
   Get-Cluster -Location $DC|sort-object name| %{
    $cluster = $_
    
    Write-Host "Getting cluster $cluster"
    $vmhosts=Get-vmhost -Location $cluster|Sort-Object name
        
    write-host "Getting VM details..."
    $allvmdetails += $cluster | Get-VM|Sort-Object name|select @{n="vmname";e={$_.name}}, `
     @{n="Datacenter";e={$DC.name}}, `
     powerstate, `
     @{n="OS";e={$_.guest.OSFullName}}, `
     version, `
     @{n="Folder";e={($_|get-folderpath).path}}, `
     @{n="ToolsStatus";e={$_.guest.ExtensionData.ToolsStatus}}, `
     VMhost, `
     @{n="Cluster";e={$cluster.name}}, `
     NumCPU, `
     @{n="MemMB";e={$_.memorymb}}, `
     @{n="DiskGB";e={[Math]::Round((($_.HardDisks | Measure-Object -Property CapacityKB -Sum).Sum / 1MB),2)}}, `
     Notes
   }
  }
 }
 $allvmdetails
}

$date=get-date -Format yyyy.MM.dd
write-host "Getting Host and VM details: $(get-date)"
$vcenterdetails=get-vcenterdetails $vcenters
$vcenterdetails|export-csv -NoTypeInformation "$location\VMware_vmdetails_$date.csv"
write-host "Done: $(get-date)"

Disconnect-VIServer * -Confirm:$false -force

You must have the PowerCLI snap-in loaded to run the above, here is the usage (assuming you save the file as get-vmdetails.ps1):
PS c:\> .\get-vmdetails.ps1 -location c:\reportfolder -vcenters MyvCenter1,MyvCenter2,MyvCenter3

I wasn't planning on a detailed explanation of all of the scripts that I post, only due the number of great resources out there, but please feel free to ask questions if you have them.  The brief summary of what the script is doing would be:

  • Connect to your vCenter(s)
  • Loop through each Cluster within each Datacenter within each vCenter
  • Gather the following details for each VM: Name, DataCenter, PowerState, OS, Folder, ToolsStatus, Cluster, Host, HW Version, Memory, CPU, Disk, Notes
  • Export this as a CSV file
This was my first step in recording where my VMs were located so I could parse this file later after the VMs were connected to a new vCenter I could put them back into the same folder structure.

Part 2 coming soon...

Friday, May 31, 2013

Have you upgraded to vCenter 5.1 yet ???

VMware vCenter 5.1 upgrade or migrate?

Well after looking over all of the requirements and the massive changes in the design with the:
  • Inventory Service
  • Single Sign On
  • vCenter Service
  • Web Service
There was no way I really wanted to try an upgrade, but there was also no way that I wanted to start from scratch as I have 2 vCenters in different data centers with ~ 5,000 virtual machines...so now what?  

PowerCLI to the rescue

Well I knew there was much work to be done and I wasn't finding anyone on the net that was attempting it...not sure why.  The benefits of being able to stand up a new environment and getting it all working, especially with SSO, before even starting the migration was priceless.  I am not going to go into all of the details here, but I will be following up with other posts with all of the scripts that I had to write to make this a seemless, no downtime migration from vCenter 5.0 to vCenter 5.1.

Keep in mind that this migration was to new operating systems, new databases, new...everything.

Here are the overall steps that brought us to success:

1. Build a new clean setup of vCenter 5.1 on Windows 2008 R2
    • vCenter / Web services share an OS
    • DB server runs solo
    • SSO and SSO DB share an OS, this was for isolation and eventual replication needs, I really wanted SSO self contained
NOTE: Yes, we are using our Enterprise CA signed certificates, but this is not the post to go into that in depth...maybe later

2. Now it was time to document all of what needed to be brought from the old vCenter to the new, keeping in mind of course that we would be losing all historic stats and logs...not that they would be going away, but if we needed to get to them we would have to keep the old vCenter until we were alright with saying goodbye to all of our old data (not a big deal).  Here are the high level steps we took:

  • Manually create Access Roles to match your old environment
  • Export All VM details from source vCenter (namely folder location)
  • Make sure to bring over all license keys from old vCenter to new
  • Copy the vCenter ID to the new vCenter so our MAC addresses stay consistent
  • Manually created the Data Center object in the new vCenter with the same name as the old...could have been scripted, wasn't worth it
  • Run script to export all storage DRS clusters to a CSV file 
  • Run scipt to export all DRS cluster details to a CSV file
  • Run script to clone folder structure to the new vCenter, this included all folders in all categories: Network, VMs and Templates, Hosts and Clusters, Datastores
  • Run script to create all DVSwitches and portgroups in new vCenter, cloned from old vCenter
  • Run script create all cluster objects (DRS and SDRS) in new vCenter, making sure to leave HA and DRS off until all migrations have been completed
Now here is where it can get a bit crazy
  • Run script to:
    • Disconnect a host from source vCenter
    • Connect to new vCenter
    • Connect that new host to the new DVSwitch 
    • Remove one of the redundant uplinks from the now orphaned DVSwitch(s)
    • Connect it to the new DVSwitch(s)
    • Migrate all of the VM NICs to the portgroups on the NEW DVSwitches
    • Remove the remaining DVSwitch(s) uplinks from the orphaned DVSwitch(s) and add them as uplinks on the new DVSwitch(s)
    • ...loop through all hosts
  • Run script to import old DRS rules
  • Run script to import Datastores to SDRS clusters
  • Run script to move VMs to their proper VM folders
  • Run script to set permissions on all objects to mirror old vCenter
  • Manually re-created all custom alarms
That was about it.  We had some small issues due to some odd misconfigurations that we found, but we did all of this in up time and it was just plain AWESOME!!!

I will be following up with future posts with the scripts that I wrote and some other issues that we saw...like the new DVPortgroups now reject MAC address changes.


Thursday, May 30, 2013

Just getting started

Ok, so I was inspired today after a VMUG event to get blogging and share all of the things I am working on in hopes that it helps someone, but also as a place I can point to when I can't remember how I did something.

My focus will be primarily around server, network and storage virtualization.  I really enjoy automation through PowerCLI and using Powershell in general.  I have a heavy IT background since the early 90's, focusing primarily in the DOS and Windows at that time.  My interests have drifted from software to hardware, networking a little Database, but lately has been strongly into virtualization and Enterprise Architecture.

Looking to start technical posts as often as I can... probably weekly.