Category Archives: ESXi

Recent ESXi 6 Storage Bugs and the FlashArray

As you might be aware, there have been a few storage-related issues with ESXi 6.0 as of late:

Accidental PDL during dropped paths:

Storage PDL responses may not trigger path failover in vSphere 6.0 (2144657)

Host issues during smartd inquiries:

Issuing a 0x85 SCSI Command from a VMware ESXi 6.0 host results in a PDL error (2133286)

The question that comes up for the Pure Storage FlashArray is are we susceptible? The short answer is no. Let’s explain why. Continue reading Recent ESXi 6 Storage Bugs and the FlashArray

Semi-transparent failover with VMFS and Active/Passive Replication

So in a blog series that I started a few weeks back (still working on finishing it), I wrote about managing snapshots and resignaturing of VMFS volumes. One of the posts was dedicated to why I would choose resignaturing over force mounting almost all of the time.

An obvious question after that post is, well when would I want to force mount? There is a situation where i think it is a decent option. A failover situation where the recovery site is the same site as the production site, in terms of compute/vCenter. The storage is what fails over to another array. This is a situation I see increasingly common as network pipes are getting bigger.

Continue reading Semi-transparent failover with VMFS and Active/Passive Replication

VMFS Snapshots and the FlashArray Part I: Mounting an unresolved VMFS

This is part 1 of this 7 part series. Questions around managing VMFS snapshots have been cropping up a lot lately and I realized I didn’t have a lot of specific Pure Storage and VMware resignaturing information out there. Especially around scripting all of this and the various options to do this. So I put a long series out here about how to do all of this. Let’s start with what an unresolved VMFS is and how to mount it.

The series being:

  1. Mounting an unresolved VMFS
  2. Why not force mount?
  3. Why might a VMFS resignature operation fail?
  4. How to correlate a VMFS and a FlashArray volume
  5. How to snapshot a VMFS on the FlashArray
  6. How to mount a VMFS FlashArray snapshot
  7. Restoring a single VM from a FlashArray snapshot

Continue reading VMFS Snapshots and the FlashArray Part I: Mounting an unresolved VMFS

UNMAP Block Count Behavior Change in ESXi 5.5 P3+

I recently was doing some troubleshooting for a customer that was using my UNMAP PowerCLI script and discovered a change in ESXi 5.5+ UNMAP. The issue was that the script was taking quite a while to complete. After some logic optimizations and increasing timeouts the script was sped up a bit and less timeout errors occurred, but a bunch of the UNMAP operations were still taking a lot longer than expected. Eventually we threw our hands up and said it was good enough. A bit more recently, I was testing a 3rd party UNMAP tool and ran into similar behavior so I dug into it a bit more and found some semi-unexpected changes in how UNMAP works, specifically the behavior when leveraging non-default block iteration counts. Continue reading UNMAP Block Count Behavior Change in ESXi 5.5 P3+

Add Storage Wizard Slowness and Unresolved VMFS Volumes

This week I received a question from a customer about some slowness in the vSphere “Add Storage” wizard they were seeing. This is a problem that has occurred over the years quite a few times for a variety of different reasons. VMware has fixed most of them, this latest reason luckily was known and has a relatively simple solution. An option called VMFS.UnresolvedVolumeLiveCheck.

option

Continue reading Add Storage Wizard Slowness and Unresolved VMFS Volumes

XCOPY Improvement in vSphere 6.0 for Thin Virtual Disks

Here is another “look what I found” storage-related post for vSphere 6. Once again, I am still looking into exact design changes, so this is what I observed and my educated guess on how it was done. Look for more details as time wears on.

***This blog post really turned out longer than I expected, probably should have been a two parter, so I apologize for the length.***

Like usual, let me wax historical for a bit… A little over a year ago, in my previous job, I wrote a proposal document to VMware to improve how they handled XCOPY. XCOPY, as you may be aware, is the SCSI command used by ESXi to clone/Storage vMotion/deploy from template VMs on a compatible array. It seems that in vSphere 6.0 VMware implemented these requests (my good friend Drew Tonnesen recently blogged on this). My request centered around three things:

  1. Allow XCOPY to use a much larger transfer size (current maximum is 16 MB) a.k.a, how much space a single XCOPY SCSI command can describe. Things like Microsoft ODX can handle XCOPY sizes up to 256 MB for example (though the ODX implementation is a bit different).
  2. Allow ESXi to query the Maximum Segment Length during an Extended Copy (XCOPY) Receive Copy Results and use that value. This value tells ESXi what to use as a maximum transfer size. This will allow the end user to avoid the hassle of having to deal with manual transfer size changes.
  3. Allow for thin virtual disks to leverage a larger transfer size than 1 MB.

The first two are currently supported in a very limited fashion by VMware right now, (but stay tuned on this!) so for this post I am going to focus on the thin virtual disk enhancement and what it means on the FlashArray.

Continue reading XCOPY Improvement in vSphere 6.0 for Thin Virtual Disks

Updated VMware and Pure Storage Best Practices Guide

Update: Please see this page for latest updates on best practices and relevant links.

Quick post here. I have updated the Pure Storage FlashArray Best Practices Guide for VMware vSphere. Not a total overhaul but there are some changes to note.

cover

Updates include:

  • New information for vSphere 6.0 This mostly focuses on what supports vSphere 6.0 and re-enforcing that current best practices remain the same. Expect a lot more vSphere 6 content though in forthcoming updates. As new storage features are tested and considered in the latest version of the VMware platfom they will be included in this guide, such as VVols.
  • Queue Depth changes are no longer mentioned in this document. Messing with this is considered a tweak that most people will not need. Don’t broke what isn’t broken is the mantra here.
  • More instruction on iSCSI setup and clarified instruction.
  • General tightening and simplification of the document
  • New content pack for Log Insight (which will be out soon). The changes are detailed in the document

 

Direct Guest OS UNMAP in vSphere 6.0

This is certainly not my first post about UNMAP and I am pretty sure it will not be my last, but I think this is one of the more interesting updates of late. vSphere 6.0 has a new feature that supports the ability for direct UNMAP operations from inside a virtual machine issued from a Guest OS. Importantly this is now supported using a virtual disk instead of the traditional requirement of a raw device mapping.

Continue reading Direct Guest OS UNMAP in vSphere 6.0

ESXi IO Operations Limit Parameter and IO Balance

Quick post here. I am working on updating some documentation and I wanted to add a bit more color to a section on changing the IO Operations limit for ESXi NMP Round Robin devices. The Pure Storage recommendation is to change this value to one from the default of 1,000. Therefore, ESXi will switch logical paths after each I/O instead of 1,000. There are some performance benefits to this and some evidence for improved failover time (in the case of a path failure) with this setting. I am not going to get into the veracity of these benefits right now. What I wanted to share here is that there is no doubt changing this to 1 makes a big difference to I/O balance on the array itself. Continue reading ESXi IO Operations Limit Parameter and IO Balance

Pure Storage and VMware VAAI

Today I posted a new document to our repository on purestorage.com: Pure Storage and VMware Storage APIs for Array Integration—VAAI. This is a new white paper that describes in detail the VAAI block primitives that VMware offers and that we support. Furthermore, performance expectations are described, comparing before/after and how the operations do at scale. There are some best practices listed as well, the why and how of those recommendations are also described within.

I have to say, especially when it comes to XCOPY, I have never seen a storage array do so well with it. It is really quite impressive how fast XCOPY sessions complete and how scaling it up (in terms of numbers of VMs or size of the VMDKs) doesn’t weaken the process at all. The main purpose of this post is to alert you to the new document but I will go over some high level performance pieces of information as well. Read the document for the details and more.


vaai_pdf_cover

Continue reading Pure Storage and VMware VAAI