I posted shortly after ESXi 6.0 came out a while back explaining how to do in-guest UNMAP with Windows. See the original post here:
The high-level workflow if you don’t want to read the post is:
- You delete a file in Windows
- Run Disk Optimizer to reclaim the space
- Windows issues UNMAP to the filesystem
- ESXi shrinks the virtual disk
- If EnableBlockDelete is enabled on the ESXi hosts, ESXi will issue UNMAP to reclaim the space on the array
This had a few requirements:
- ESXi 6.0+
- VM hardware version 11+
- Thin virtual disk
- CBT cannot be enabled (though this restriction is removed in ESXi 6.5 see this post)
One thing really bothered me about this, which is why doesn’t automatic UNMAP work? In Windows 2012 R2 and later, by default, when you delete a file UNMAP is supposed to be issued automatically to reclaim the space. This never seemed to work.
Well I finally found my problem. Long story short, make sure when formatting your NTFS that the allocation unit is 64K. Anything else will prevent automatic in-guest UNMAP with virtual disks from Windows.
So make sure your NTFS volumes are using a 64K allocation unit.
UPDATE: This is fixed in 6.5 Patch 1! You no longer need the right allocation unit size! See this post: In-Guest UNMAP Fix in ESXi 6.5 Part I: Windows
NOTE: This behavior is common to ESXi 6.0, 6.0 U2 and 6.5. The reason I am blogging about it just now is that I just figured it out.
This is a common SQL best practice so many of you probably are have NTFS configured as such. To check, here is a short PowerShell script from this blog post that will do it easily:
$wql = "SELECT Label, Blocksize, Name FROM Win32_Volume WHERE FileSystem='NTFS'" 2 Get-WmiObject -Query $wql -ComputerName '.' | Select-Object Label, Blocksize, Name
Paste that into a PowerShell window and you will get your answer.
Note all of this was on Windows 2012 R2. I need to still test Windows 10 and 2016.
For details on some testing: read on…
I was testing some other UNMAP stuff and I couldn’t get UNMAP to work fully.
I had copied an ISO to a thin virtual disk which made it 3.1 GB in size:
The virtual disk:
I would permanently delete the file and then run the disk optimizer tool, not everything would be reclaimed though. I got back about half of what I expected.
About 1.5 GB was returned. Not perfect but something.
So during testing I decided to try deleting the virtual disk instead of removing and adding an entirely new virtual disk.
I noticed that the remaining 1.5 GB virtual disk, was now ~80 MB. It had fully reclaimed! But how/why?
Well I first thought that maybe I didn’t wait long enough for UNMAP to finish. But I quickly proved that wrong. Then after a quick search I found this:
I had completely forgot about that behavior. NTFS format triggers a fully UNMAP first, then it allocates its metadata, in this case about 87 MB. You can see it goes basically down to zero then up to 87, as I would run the ls -s command during the format and it goes from 1.5 GB down to 30 MB then finishes at 87 MB.
Since it was essentially issuing UNMAP to a raw device, it got be thinking about alignment issues and how with the new Linux in-guest it has to be aligned at 1 MB for UNMAP to work. So I started trying different NTFS allocation units. Starting with 64k-which is SQL server recommendation. Sure enough, when I set it to that automatic UNMAP worked, and it worked well. As soon as I deleted my file(s) I got 100% of my space back–my virtual disk shrunk down to the original size.
So my recommendation is to make sure you format NTFS using a 64k allocation unit. Most applications recommend it anyways, so you might as well. For UNMAP, it is far more efficient (you get all of your space back) and it is automatic (as soon as you delete a file it will reclaim the space and shrink the virtual disk.
Stay tuned on more details on why this is required.