Enhanced UNMAP script using with PowerCLI and RESTful API

***********UPDATE PLEASE REFER TO THE POST AT THIS LINK FOR UPDATED INFORMATION ON THIS SCRIPT***********************

The most common request I get for scripts here at Pure Storage is an UNMAP script using PowerCLI. I have a basic one here that does the trick–UNMAPs Pure Storage volumes in a vCenter. That being said it is pretty dumb–doesn’t tell you much about what happened other than what volumes it is reclaiming (or not reclaiming) and moves on. A few requests have come in recently for something a little more in-depth. Most notably the ability to see how much space has been reclaimed. This information cannot be gathered from the VMware side of things–it has to come from the FlashArray.

There are two options here–either use our REST APIs or use our PowerShell toolkit to get this information (which just wraps the REST calls). For this script I chose to use the REST API directly from within PowerShell. What this script does is:

  1. Connects to the vCenter and FlashArray
  2. Finds all of the datastores and counts how many are actually Pure Storage volumes (NAA comparison)
  3. Iterates through all of the datastores
  4. Skips it if it is not Pure
  5. If it is, the current data reduction ratio is reported and so the is current physical written capacity on the FlashArray.
  6. Runs UNMAP on the datastore
  7. Reports the new data reduction and physical space after UNMAP completes and how much was reclaimed.
  8. Repeats for the rest of the volumes.

The script reports all of this to the console window, but it always throws it in a log file through add-content. If you don’t want it to return the info to the console, simply delete the write-host lines. If you don’t want it to log, delete the add-content lines.

There are a few required parameters–vCenter information (IP, username, password), FlashArray info (IP, username, password), UNMAP block count and a log file location. These are hard-coded parameters, but that can easily be changed by altering it to a read-host.

You may also note that after each UNMAP the script sleeps for 60 seconds–I do this so I make sure the FlashArray has time to update its information right after the UNMAP. 60 seconds is VERY conservative–probably 10 or so is fine, so feel free to mess with that number if you don’t like waiting. I also have another sleep at the end of each datastore operation to give a quick chance to review the latest results before it starts spewing the next datastore information on the screen (note this update didn’t make it into the video demo below–it doesn’t wait after each datastore).

See the script in action below. Essentially I am deleting a bunch of VMs across 4 datastores and then running the UNMAP. You can see the space get reclaimed on the FlashArray.

Note: You need particular access (see a blog post about that here) to vCenter to run UNMAP. For the FlashArray only Read Only is needed (higher of course is fine too).

Get the script here:

https://github.com/codyhosterman/powercli/blob/master/unmapsdk.ps1

Reclaiming in-guest capacity with VMware and Pure Storage

UPDATE: In-guest UNMAP is now supported in a VM and sDelete and such is no longer required. Please refer to these posts:

In-Guest UNMAP Fix in ESXi 6.5 Part I: Windows

In-Guest UNMAP Fix in ESXi 6.5 Part II: Linux

Direct Guest OS UNMAP in vSphere 6.0

Reclaiming “dirty” or “dead” space is a topic that goes by my desk quite often these days–since the FlashArray is a data reduction array it is especially important that space is not wasted on the array–throws off the economics etc. Therefore UNMAP is an important VAAI feature to leverage in any AFA environment. Supporting UNMAP is definitely table stakes for AFAs.

Note–I am doing to use the terms “dead”, “dirty” and “stranded” to define space that needs to be reclaimed interchangeably. So anyways…

Unfortunately UNMAP in its current form does not satisfy all of the reclamation use cases. UNMAP will only reclaim space on any array when capacity is cleared from the VMFS volume–so when a VM (or virtual disk) is deleted or migrated elsewhere. It does not have the ability to reclaim space when data is “deleted” inside a virtual machine by the guest OS when using virtual disks. VMware does not know this capacity has been cleared and neither can the array. So until this virtual disk is deleted or moved the capacity cannot be reclaimed with UNMAP. So to be clear, UNMAP with vmkfstools (in ESXi 5.0/5.1) or esxcli (in ESXi 5.5) does not allow you to reclaim space that remains stranded inside of virtual disks.

Continue reading “Reclaiming in-guest capacity with VMware and Pure Storage”