What’s New in Purity 6.1: NVMe-oF/Fibre Channel

NVMe. A continued march to rid ourselves of the vestigial SCSI standard. As I have said in the past, SCSI was designed for spinning disk–where performance and density are not friendly to one another. NVMe, however, was built for flash. The FlashArray was built for, well, flash. Shocking, I know.

Putting SCSI in front of flash, at any layer, constricts what performance density can be offered. It isn’t just about latency–but throughput/IOPS per GB. A spinning disk can get larger, but it really doesn’t get faster. Flash performance scales much better with capacity however. So larger flash drives don’t get slower per GB. But this really requires the HW and the SW to take advantage of it. SCSI has bottlenecks–queue limits that are low. NVMe has fantastically larger queues. It opens up the full performance, and specifically performance density of your flash, and in turn, your array. We added NVMe to our NVRAM, then our internal flash to the chassis, then NVMe-oF to our expansion shelves, then NVMe-oF to our front end from the host. The next step is to work with our partners to enable NVMe in their stack. We worked with VMware to release it in ESXi 7.0. More info on all of this in the following posts:

Continue reading “What’s New in Purity 6.1: NVMe-oF/Fibre Channel”

Configuring FlashArray Offload to AWS S3

I am working on some new integration efforts around a feature we call “offload” and figured since I am setting it up, I might as well document it.

For the un-initiated we have a feature called Snapshot Offload that creates snapshots of one or more sets of volumes in a write-consistent manner and sends them to a non-FlashArray target (similar to our snapshot replication from one FlashArray to another conceptually). Those snapshots can then be ingested back to that FlashArray, a different one, or a Cloud Block Store instance in AWS or Azure.

Snapshot offload allows you to send to either some object store, so AWS S3, Azure Blob, GCP Object, or a random NFS target:

For this post I am going to walk through creating an AWS S3 bucket, providing credentials, and then configuring a protection group.

Continue reading “Configuring FlashArray Offload to AWS S3”

ESXi NVMe-oF Namespace IDs, LUNs, and other Identifiers

In the world of SCSI, a storage device is generally addressed by two things:

  1. LUN–Logical Unit Number. This is a number used to address the device down a specific path to a specific array, for a specific host. So it is not a unique number really, it is not guaranteed to be unique to an array, to a host, or a volume. So for every path to a volume there could be a different LUN number. Think of it like a street address. 100 Maple St. There are many “100 Maple Streets”. So it requires the city, the state/province/etc, the country to really be meaningful. And a street name can change. So can other things. So it can usually get you want you want, but it isn’t guaranteed.
  2. Serial number. This is a globally unique identifier of the volume. This means it is entirely unique for that volume and it cannot be change. It is the same for everyone and everything who uses that volume. To continue our metaphor, look at it like the GPS coordinates of the house instead of the address. It will get you where you need, always.

So how does this change with NVMe? Well these things still exist, but how they interact is…different.

Now, first, let me remind that generally these concepts are vendor neutral, but how things are generated, reported, and even sometimes named vary. So I write this for Pure Storage, so keep that in mind.

Continue reading “ESXi NVMe-oF Namespace IDs, LUNs, and other Identifiers”

Pure Storage Plugin for the vSphere Client 4.5.0 Release

Howdy doody folks. Lots of releases coming down the pipe in short order and the latest is well the latest release of the Pure Storage Plugin for the vSphere Client. This may be our last release of it in this architecture (though we may have one or so more depending on things) in favor of the new preferred client-side architecture that VMware released in 6.7. Details on that here if you are curious.

Anyways, what’s new in this plugin?

The release notes are here:

https://support.purestorage.com/Solutions/VMware_Platform_Guide/Release_Notes_for_VMware_Solutions/Release_Notes%3A_Pure_Storage_Plugin_for_the_vSphere_Client#4.5.0_Release_Notes

But in short, five things:

  1. Improved protection group import wizard. This feature pulls in FlashArray protection groups and converts them into vVol storage policies. This was, rudimentary at best previously, and is now a full-blown, much more flexible wizard.
  2. Native performance charts. Previously performance charts for datastores (where we showed FlashArray performance stats in the vSphere Client) was actually an iframe we pulled from our GUI. This was a poor decision. We have re-done this entirely from the ground up and now pull the stats from the REST API and draw them natively using the Clarity UI. Furthermore, there are now way more stats shown too.
  3. Datastore connectivity management. A few releases ago we added a feature to add an existing datastore to new compute, but it wasn’t particularly flexible and it wasn’t helpful if there were connectivity issues and didn’t provide good insight into what was already connected. We now have an entirely new page that focuses on this.
  4. Host management. This has been entirely revamped. Initially host management was laser focused on one use case: connecting a cluster to a new FlashArray. But no ability to add/remove a host or make adjustments. And like above, no good insight into current configuration. The host and cluster objects now have their own page with extensive controls.
  5. vVol Datastore Summary. This shows some basic information around the vVol datastore object

First off how do you install? The easiest method is PowerShell. See details (and other options) here:

https://support.purestorage.com/Solutions/VMware_Platform_Guide/User_Guides_for_VMware_Solutions/Using_the_Pure_Storage_Plugin_for_the_vSphere_Client/vSphere_Plugin_User_Guide%3A_Installing_the_vSphere_Plugin

Continue reading “Pure Storage Plugin for the vSphere Client 4.5.0 Release”

Managing vVol Storage Policies with PowerShell

I just posted about some new cmdlets here:

Also in that release are a few more cmdlets concerning storage policy creation, editing, and assignment. They were built to make the process easier–the original cmdlets and their use is certainly an option–and for very specific things you might want to do they might be necessary, but the vast majority of common operations can be more easily achieved with these.

As always, to install run:

Install-Module PureStorage.FlashArray.VMware

Or to upgrade:

Update-Module PureStorage.FlashArray.VMware

These modules are open source, so if you just want to use my code or open an RFE or issue go here:

https://github.com/PureStorage-OpenConnect/PureStorage.FlashArray.VMware/

For detailed help on a cmdlet, run Get-Help

Continue reading “Managing vVol Storage Policies with PowerShell”

New vVol Replication PowerShell Cmdlets

Happy New Year everyone! Let’s work to make 2021 a better year.

In furtherance of that goal, I have put out a few new vVol-related PowerShell cmdlets! So baby steps I guess.

The following are the new cmdlets:

Basics:

  • Get-PfaVvolStorageArray

Replication:

  • Get-PfaVvolReplicationGroup
  • Get-PfaVvolReplicationGroupPartner
  • Get-PfaVvolFaultDomain

Storage Policy Management:

  • Build-PfaVvolStoragePolicyConfig
  • Edit-PfaVvolStoragePolicy
  • Get-PfaVvolStoragePolicy
  • New-PfaVvolStoragePolicy
  • Set-PfaVvolVmStoragePolicy

Now to walk through how to use them. This post will talk about the basics and the replication cmdlets. The next post will talk about the profile cmdlets.

Continue reading “New vVol Replication PowerShell Cmdlets”

Strong Consistency in S3 and Cloud Block Store

From my armchair in the past few weeks, I have been watching the myriad of announcements at re:Invent by AWS and a few things caught my eye (well a lot of things did, but a few in particular to storage).

The first thing to note was the change in the consistency model in S3. Up until now, consistency was “eventual” within S3 for certain operations like changes to a file, and there are a ton of posts that do a great job of explaining this. One is below:

https://dhruvsharma-50981.medium.com/s3-eventual-data-consistency-model-issues-and-tackling-them-47093365a595

Google “S3 eventual consistency” and you will find tons of examples.

At a high level, when modifying or deleting objects the change may not be immediately reflected. So on an immediate subsequent read, you may not get what you wrote. For busy environments with high change rates/modifications this could lead to corruption. So you needed to understand the behavior and build to or around it.

At re:Invent, AWS announced that strong consistency is now supported for S3 operations:

https://aws.amazon.com/blogs/aws/amazon-s3-update-strong-read-after-write-consistency/

https://aws.amazon.com/s3/consistency/

What you write is what you get. Fantastic!

So what does this have to do with Cloud Block Store? Read on.

Continue reading “Strong Consistency in S3 and Cloud Block Store”

Exporting/Importing a Certificate with the Pure1 PowerShell Module

When deployed on Windows, the Pure1 PowerShell Module takes advantage of Windows-based certificates in the user (or specified) certificate store. On Linux or MacOS, it uses RSA private key pairs.

To relocate authentication on a Non-Windows machine to another non-Windows machine, you just copy the private key from wherever it is to the target. For Windows though you need to export the cert (which has a private key) from the certificate store, then you can copy the file to wherever.

In the latest release of the Pure1 PowerShell module (1.4.3.1) there is a new feature to do that for you–or at least simplify the process of exporting the cert with the right settings.

Let’s walk through exporting and then importing the cert. In a future post I will go into some of the other enhancements in this release in more detail.

As always the repo is here (and release notes) and it is best installed/updated via the PowerShell Gallery:

install-module PureStorage.Pure1
or
update-module PureStorage.Pure1

https://github.com/PureStorage-OpenConnect/PureStorage.Pure1

Continue reading “Exporting/Importing a Certificate with the Pure1 PowerShell Module”

Improved Certificate Management for Pure1 PowerShell Module

I have made some basic improvements to the Pure1 PowerShell Module that should make it easier to authenticate and connect.

In the 1.4.1.1 release there are two new cmdlets:

Get-PureOneCertificate

Set-PureOneDefaultCertificate

And some minor changes to:

New-PureOneCertificate

New-PureOneConnection

Continue reading “Improved Certificate Management for Pure1 PowerShell Module”

Using PowerShell with Tanzu and the Kubernetes API

Sounds like a silly thing, but we all have to start somewhere. Generally when I dig into something new, I like to start from a place I know well. So when it comes to using a new API, I like to use a tool I know how to use. Kubernetes–and its API is fairly new to me from a hands-on perspective. PowerShell, however, is not. I have decent handle on that. So seems to me a good place to start with the k8s API.

I don’t know if this is the best way, or even a good way, but it does work. And there is also this:

https://www.powershellgallery.com/packages/Microsoft.PowerShell.KubeCtl/0.0.3

But I am trying to learn authentication and the finer points of the API, so I like to start with first principles.

Create a Service Account

So the first step is to create a service account. So create a new file and then in that, enter in the following information, replacing the username and/or namespace with whatever you want:

vim newuseracct.yml

Then apply it:

Again using your favorite editor, create a new file:

vim newuser.yaml

This will apply the cluster admin role to that account. Replace the username, the namespace or even role as needed.

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: pscody
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: cluster-admin
subjects:
  - kind: ServiceAccount
    name: pscody
    namespace: kube-system

Now apply it:

kubectl apply -f newuser.yaml

Retrieve the Token

Once the account is created, you need the token. Run:

kubectl get serviceaccounts pscody -o yaml --namespace kube-system

Replacing the username and the namespace as needed.

Under the secrets, grab the name. In my case it is “pscody-token-s4lvz”.

Then run:

kubectl -n kube-system describe secret pscody-token-s4lvz

Copy everything in the token.

Get Server Address

Now you need to get the server address for the cluster where you created the user. So if you don’t know, look at the context via kubectl config get-contexts:

Then, run kubectl config view and pull the server address for the corresponding cluster, so for mine it is cody-dev so the address is https://10.21.202.237:6443

Connect with Invoke-RestMethod

Now head over to PowerShell!

First, store your token in an object, I will use $token.

Then we need to form the header as a bearer token:

$k8sheader = @{authorization="Bearer $($token)"}

This is the format needed to authenticate with that token.

Now you are ready!

To pull the storage classes for instance run:

Invoke-RestMethod -Method GET -Uri https://10.21.202.237:6443/apis/storage.k8s.io/v1/storageclasses -Headers $k8sheader -SkipCertificateCheck

You will need skip certificate check for now–I didn’t configure the certificate checking yet.

If we store the response in an object we can more easily dig in:

And find my default storage class.

Definitely a lot more for me to learn here, but it is a start!