Configuring Pure1 VM Analytics

If you are Pure Storage customer and you recently logged into Pure1 (which is our cloud-based management/analytics/support portal), you might have noticed a notification the first time you logged in this week:

What is this? Well this is the great thing about Pure1–we just add features all of the time and this is an exciting one. Two years or so ago I wrote this blog post:

Understanding VMware ESXi Queuing and the FlashArray

That went into storage troubleshooting with ESXi. Got a lot of positive feedback on that and also a lot of questions–it was a situation where people definitely needed more helpful tools to identify performance issues, or simply just to track who was doing what and where.

https://blog.purestorage.com/vm-analytics-now-available-in-pure1/

This week we introduced what we call “VM Analytics” to Pure1. All customers get it for free.

How does it work? Well, first, you configure a collector. There are two options for this. We can either run the collector inside of a FlashArray (it runs inside of our Purity/Run platform as a docker container) or you can instantiate the container on docker yourself as an off array collector. The collector then is authorized to one or more vCenters (read-only creds are fine) and pulls in your VM topologies and metrics. It pulls in:

  • VMs
  • Virtual Disks
  • Datastores
  • Hosts
  • Clusters
  • Datacenters

From a metric standpoint, it pulls in the relevant metrics for VMs, Datastores, Virtual disks, and hosts:

  • IOPS (R and W)
  • Throughput (R and W)
  • Latency (R and W)
  • CPU
  • Memory

Each vCenter is polled every 10 minutes and then this information then gets dialed home back to Pure1 hourly. Inside of Pure1, you can now see you VMware topology. Sort VMs or their disks by latency, IOPS etc. What VM is where? What datastore is on what FlashArray? What are the latency differences through the stack?

So before we dive into how you use it, let’s go through setting it up.

As I mentioned there are two methods for the collector on array or off array. For on array, just simply open a Pure support case and ask for it. What are the requirements?

  • Purity 4.10.x (preferably latest)

  • M20 or higher

Fairly straight forward. If you do not have at least one array that matches those requirements, fear not–the off array collector is for you. Let’s walk through setup.

On-Array Collector vs Off-Array

Personally I am a fan of the off array collector–it is super easy to setup, you just need a vanilla Ubuntu VM and the collector takes only a few minutes to setup at most. And you can do it entirely yourself, no need to call support.

I would recommend the on-array in two scenarios:

  1. Your network cannot route out. In this case your Ubuntu VM cannot talk to the internet and cannot download what you need for the collector.
  2. You cant/dont use Ubuntu.

Install the On Array Collector

Call Pure support–they do all the work for you.

Default credentials are pureuser/pureuser. You can find the connection IP to SSH to by logging into your FlashArray GUI:

Install the Off Array Collector

For a VERY DETAILED step-by-step process go see this blog post

https://www.codyhosterman.com/2018/10/configuring-pure1-vm-analytics-detailed-guide/

If you know how to install/configure Ubuntu this post should do just fine, so continue on.

NOTE: if you had support configure the on array collector you can skip all of this and go to the section on Configuring the collector.

The step is to login to pure1.

https://pure1.purestorage.com/ 

Click on VM Topology then the gear icon that says “Collector configuration” when you hover over it.

In the pop-up that appears, click Create Collector and enter a name for your collector.

There are no real requirements on this name, this is just a friendly name for you to use as a reference.

This will add a collector instance. If you hover to the right of it, you can click on “View installation script”. 

Click copy and save this for later.

The next step is to configure my collector. In this case, I have a Ubuntu 16.04.1 VM:

A pretty bare bones Ubuntu installation at this point. The following are the requirements:

  • Ubuntu 16.04

  • x86_64 cpu with at least 2 cores Minimum 2GB RAM

  • Docker EE 1.11.2 or newer, or CE 17.03 or newer

  • python 3.5.2 or later

  • logrotate, and cron installed and running

I have python 3.5.2 already:

If you do not have it installed, install it. I won’t cover it here but there are plenty of blogs out there that walk through that–it is pretty straight forward.

Next to install Docker–I will go with CE. I will also not walk through this, because there is a great and accurate resource here:

https://docs.docker.com/install/linux/docker-ce/ubuntu/#install-docker-ce-1

This walks through installing it. Follow ALL of the instructions (step 1-4 under SET UP THE REPOSITORY and step 1 and 2 under INSTALL DOCKER CE).

Do not skip the apt-get updates (there are two). Configure the repository and then install Docker. Just run the commands listed in that page. Takes a minute or so.

When you are done you should have Docker ready to go:

Lastly, set logrotate to hourly:

Now install the collector. Grab that script you copied from Pure1. Mine is:

env COLLECTOR_AUTH_KEY=dxxxxxxxxxxxxxxxxxx8 bash -c "$(wget -O - https://static.pure1.purestorage.com/vm-analytics-collector/install.sh)"

Run that now. You will likely have to add sudo to it. If you forget, it will remind you.

This will not take long (10-20 seconds). It will download the necessary files and install the collector.

Done!

Configure the Collector

If you are using the off-array collector, just SSH into that VM. If you are using the on-array collector, SSH into the Docker VM support configured for you.

Now authenticate with vCenter! I will add three of mine using the purevmanalytics command:

You should add sudo here too likely.

sudo purevmanalytics connect --hostname <hostname>--username <username> --password <password>

You can review the status with purevmanalytics list:

You’re done! You should see data starting to populate in Pure1 in an hour or so.

Verifying/Checking Logs

The collector puts its logs here:

/var/log/docker/vm_analytics

You can look at the phonehome status in the ccm log and general collection information in the vmanalytics.log file.

Also you can check in Pure1 for the last time the collector has phoned home:

Note that it might take a few hours for the first metrics to show up. Even if you force a phonehome. If the logs have not rolled over yet, nothing will be sent.

FAQ

Some basic questions:

Do I need many collectors?

No usually. As long as the collector can reach all of your vCenters, you do not need to have more than one. If the networks are firewalled, then you might though if a given collector and not reach a certain vCenter.

Do I need to authenticate my FlashArrays?

No. They dial home themselves, Pure1 already has their information/metrics.

What are the network requirements?

Refer to this:

https://support.purestorage.com/Pure1/Pure1_Manage/01_Pure1_Manage_User_Guide/Pure1_Manage_-_VM_Topology

Can I force a phone home?

Yes for the off-array collector only though. purevmanalytics phonehome.

What FlashArray models does it support?

While the collector can only run on certain FlashArray models–all support FlashArrays are supported with the feature itself. So 400 series, //m and //x. They will all show up and be available in the Pure1 topology.

VMware support?

vCenter 5.5 and later.

Does it support VVols? RDMs? VMFS?

Today this supports VMFS and RDMs. VVols support is coming.

Hyper-V?

Not today (2018). This is VMware only at this point, but it will change.

In the next post, I will go over using it and what you can see in Pure1.

 

24 thoughts on “Configuring Pure1 VM Analytics”

  1. Hi mate,

    With the Off Array Collector are you able to add multiple SANs to a single Ubuntu server under a separate container or will you require one server per SAN?

    1. The considerations are twofold. First off the collector doesn’t need to talk to any FlashArrays—it only collects from vCenters. It then dials that information home. The FlashArrays dial home their own information separately. So it only needs access to your vCenters. So you only need more than one collector if you have vCenters that are on completely different networks. If a collector can route to every vCenter over the IP network you really only need one. That’s the primary consideration. The other consideration is if you have multiple organizations within Pure1. If you have more than one organization in Pure1 it means that different users or groups from your company can login to Pure1 but they see totally distinct FlashArrays. In this case you would need a different collector for each org. In general, most customers will likely need only one.

      1. Thanks for the clarification, the information i got from Pure support was one collector per array, but since it is brand new they may not have all the information yet.

  2. Hi.
    Can you tell me what firewall rules are required ?
    Can it work through a proxy server ?
    Our implementation is behind a corporate firewall and proxy server.

  3. Strangely the installation works with a 18.04 but no data goes in Pure1. I follow the documentation with a 16.04 and it works perfectly fine.
    Have a nice Weekend

    1. Just found out that 18.04 does not currently work, engineering is working on fixing it for the next release. Apparently there are some issues with DNS configuration that need to be resolved.There is a workaround though:

      The workaround is to:

      Deactivate systemd-resolved:
      pure@docker:~$ sudo systemctl disable systemd-resolved.service

      Stop systemd-resolved:
      pure@docker:~$ sudo service systemd-resolved stop

      Remove the symlink to /etc/resolv.conf
      pure@docker:~$ sudo rm /etc/resolv.conf

      Create a new resolv.conf file
      pure@docker:~$ sudo touch /etc/resolv.conf

      Configure your nameservers

      Your /etc/resolv.conf should look like this afterwards:
      pure@docker:~$ cat /etc/resolv.conf
      nameserver 8.8.8.8
      search mydomain.com

      Re-Install the VM Analytics Collector:
      pure@docker:~$ sudo env COLLECTOR_AUTH_KEY= bash -c “$(wget -O – https://static.pure1.purestorage.com/vm-analytics-collector/install.sh)”

  4. Hey Cody,
    Just wanted to say thanks for the blog post on this.
    Saved me a call to support or some google time around setting logrotate to hourly.

    1. We have something in the works–we want to do OVAs right and we have a plan for it. But waiting for it would delay this significantly. So stay tuned.

      1. Thanks for a cool tool. We would be fine with an OVA. If this isn’t possible, we would like to get the required modules listed so that we can deploy this in RedHat. Either option is fine.

        1. You’re welcome! We are actively working on both of those options. I suspect RedHat will come sooner than the OVA though.

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.