Category: Pure Storage

JSON Web Tokens (JWT) are part of the mechanism that we (and many modern REST implementations) use to authorize connections. I think the term authorize is the key here. Authenticate vs. Authorize. Think of it in a similar way to when you log into a website. You initially login (authenticate) with a website with a user name and password. But the next time you go to it, or re-launch your browser you don’t have to. Why because you already authenticated. An authorization is stored in a cookie so you don’t have to again. For at least a certain amount of time or for the length of that browser session etc.

So a JWT can be analogous to a cookie.

A good post on that here:

https://hashedin.com/blog/auth-headers-vs-jwt-vs-sessions-choosing-right-auth-technique-for-apis/

This is often somewhat abstracted, but not always. If you want to directly authenticate to Pure1, for instance, you need to create a JWT. So let’s dig into that process. Then let’s talk about troubleshooting techniques for a rejected JWT.

The Anatomy of a JWT

So what is in a JWT? Well the data can vary, but in this case I will be talking about the data required by Pure1.

There are three parts:

1. Headers
2. Payload
3. Signature

The headers indicate what type of encryption is used in the signature.

The payload indicates the information required by the authenticator. Expiration. User. Key. Whatever.

The signature is the encrypted string that consists of the header plus the payload data. So an example.

For Pure1, the header looks like so. Always:

{
  "alg": "RS256",
  "typ": "JWT"
}

Basically saying use RSA 256 bit encryption for this JWT.

The payload is always structured the same, but the data varies:

{
  "iss": "pure1:apikey:tzckd2UsXGmNFecK",
  "iat": 1604684213,
  "exp": 1607276213
}

The iss property is a Pure1 key assigned to an application. The IAT property is the current epoch time, and the exp property is the expiration of this key. So the JWT cannot be used after that time to authorize any more connections.

As you might notice, this information is formatted in JSON. But it is not sent that way. The data is sent via https, so it needs to be what is called URL encoded.

So the above actually looks like below:

eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJwdXJlMTphcGlrZXk6dHpja2QyVXNYR21ORmVjSyIsImlhdCI6MTYwNDY4NDIxMywiZXhwIjoxNjA3Mjc2MjEzfQ

Lets break that down. There are two parts, separated by a period.

eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCJ9

Which is the headers after URL encoding. And:

eyJpc3MiOiJwdXJlMTphcGlrZXk6dHpja2QyVXNYR21ORmVjSyIsImlhdCI6MTYwNDY4NDIxMywiZXhwIjoxNjA3Mjc2MjEzfQ

Which is the payload after URL encoding.

But that is not the full JWT. The full JWT looks like below:

eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJwdXJlMTphcGlrZXk6dHpja2QyVXNYR21ORmVjSyIsImlhdCI6MTYwNDY4NDIxMywiZXhwIjoxNjA3Mjc2MjEzfQ.uFAJ-XS0e3mcvp_xGWRCMqaRQ8iXazp9MaGGeLmNeVsWI2vhNeVD2_NrHiLhAphDjHrQLmuKY9clWh2R3WJOLrwTR3N_1aaYGKCMjLXfDuXgcHD980RgsXCM0axIsdX9KKmnyVDbcuipiy1yUxyxRiCtd6o2OBgF9H3bZoGTSbTiPPRHHoyAwEKaThqXLFVB3jUrxukvMAuH9mTATXi7cfxplPJZixj3mhi0-IaZ4UKYXHFi9vUyOg7wfWvtev45E6SHMV8ye9tevrh8zG5jWDJYmOBqdRtxSgXXKE-lVg3qiEXToSddq0Xe4rRnS4avgZjBWPENaDkMKBug8KBnqg

There is a third section. This is the first two parts that have been encrypted via a RSA 256 bit private key.

So basically the header and payload is sent twice. Unencrypted, but URL encoded and encrypted AND URL encoded.

A good way to decode this is via a website jwt.io.

This site will interpret the unencrypted, but URL encoded data and display them in the right hand side.

How Pure1 checks the JWT

So how does Pure1 check the JWT? Well a few things happen. Before a JWT can be used for authorization, you must first create a RSA 256 bit private key. Let’s say our key is this:

-----BEGIN RSA PRIVATE KEY-----
MIIEpAIBAAKCAQEA4WVi9HtenBdPUbZKvjOoefYxUsNOt+eUTPAMWU3dNoUR/2pk
oy1i2+iOBwkQgh5veyKZpQQCjEnyGKeliZEkMPh2bJgFuKKyG4L0wgU1v7AEsuhQ
oNg7mdpPUlmIiIkVU91mtLxjES95AXl0A0ojkUFe5JsE71Qt+2GgVJoB7fRh2l8e
Qnvje9tcfGFaAKoW63FmalNrZq8FjDbhQ4N3AFANClfRlP7Ru3QLGbPWl3WMdGkj
ZU7STYON3fg3TN7K/95xRheOqpbZ/JpL9FHrwZNiSx4hY82pjqDrdeF2mGDVILMa
3FQIHNBqNyY1ORkLc8EW+wKJk+8OsNwbr4mjWwIDAQABAoIBAF0kEnJJEN2b3FE9
7UEQA8zyaYtz3fGtJuFsSQ4WWRWQg9FyW8f6l2GchiRuIH5EqXjO4YMJVyw12m9J
x+RDMBt0cSuK3sJfqQ4pXY9Xq9BconRz/+2XrXl0d1bmXL9fNosDcM+7EY0mevCb
Cc/aMTxzMQ1y6eUkeiPpQeH0ClwT3CTQ/ovZcIrZUSb0JYUuev5AuS7PFmA/fIIM
Bi4e+PufWm2kpSY5lzCsiB7E2da7xvBH9V5pmy1/5tgD9zIKm9y9jJ6Qek3cz728
MivfbkKqp5rjIVABPIYWEGA8pt5itbOSUgkkfmgLF8iVoDVA21U0o2Z1bi1RP2Y2
QK9vD5ECgYEA+7jCFZPP4RhDHCGjXBObS6mzJu+ELUmfgh1hjyY+0+ZUiWaj3txc
89uAMjw43KbGTI7sywHrqPluOiu/HH891CVndhy+2sTnMYw9DCek5AzsPjeVvmD+
kKTFuCqs90xN1TVP+PEb28AR13uVPsWl2U0ClwtGtqcuRzmw5Za2cn0CgYEA5ToV
2KxT5+F4HKiB3ibyUhC5R/kFOcWCpBsr1JSDbk/D/PdBWqfbJ/G582pHmq3Wk4Pe
L02Ppmp+vbNLBAWg++0cd6dI7WLBNMAvHt2LHL8jBsIUqf2V+onnsUvgPmJ2Xc1A
aB7FFKF4h56IbcZD3qny60pqbfZOR8hVZmdSvLcCgYEApoDhsJAjU0tPKM5/ViqJ
8JSBsmGAVsx6PADfRWcQ6+1RQcmo34N34L3yoEgBfMK8LGvl6aqNjSngZY4GTf1E
ko1jUFdXTzIrkzSmNOIMSeTaZ5Uw8csK1/aF6nogCzxnx34KNiqWONdlddmDMRBf
0csEyZBl5+/Rxr88acE1UakCgYEAlyfGYCrC6ZV0bvFvMd7cy+DbTOqY14+2piCE
tpl4UbWnt5gUMIOfsKdlx2296fHYHU5HpUmwAD0Zdl5UiUJs9bNVolFddaw5W1+f
surEYSBpKHuyxKXD0II/llpCdODYzOIcJoyjyQbxQ/z2WezztFC9LwrTQ8+3rv4b
0nsvw+ECgYBjGKeA0sll4DfEU4UJpj1xwIG3HYR0Tw2lkBFmoCGlAowVbQqdKtn0
aEGHlLGxzarv0CBGXpVvR+q4fCf/vycJZUc7pQR8r6VmEpy952p2uyo+o3debOvL
4/Nbh8GZ3I7CsHUDnHLw2sBa/Ojv4yf5Q2N1XURlBsv1ynDoZ+5EQg==
-----END RSA PRIVATE KEY-----

This key is used to create the signature portion of the JWT. More on that in a bit.

From the private key, a public key is created. This is used to decipher anything signed by the private key. So a public key for the above private key is the following:

-----BEGIN PUBLIC KEY-----
MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEA4WVi9HtenBdPUbZKvjOo
efYxUsNOt+eUTPAMWU3dNoUR/2pkoy1i2+iOBwkQgh5veyKZpQQCjEnyGKeliZEk
MPh2bJgFuKKyG4L0wgU1v7AEsuhQoNg7mdpPUlmIiIkVU91mtLxjES95AXl0A0oj
kUFe5JsE71Qt+2GgVJoB7fRh2l8eQnvje9tcfGFaAKoW63FmalNrZq8FjDbhQ4N3
AFANClfRlP7Ru3QLGbPWl3WMdGkjZU7STYON3fg3TN7K/95xRheOqpbZ/JpL9FHr
wZNiSx4hY82pjqDrdeF2mGDVILMa3FQIHNBqNyY1ORkLc8EW+wKJk+8OsNwbr4mj
WwIDAQAB
-----END PUBLIC KEY-----

So the public key is entered into Pure1.

Once uploaded, Pure1 generates an Application ID that corresponds to that public key.

In this case: pure1:apikey:tzckd2UsXGmNFecK. So let’s walk through the authorization process.

1. Create a data payload that specifies the application ID. Also specifying a time that this payload is valid for.
2. Sign the header + data payload.
3. URL encode the header, the payload and the signature. Separated each by a period.
4. Send that to Pure1.
5. Pure1 de-encodes the payload (not decrypts! remember the first part is just encoded, not encrypted) and finds the application ID
6. It then looks for the respective public key associated with that application ID.
7. It takes the third part of the JWT (the signature) and decrypts it using the public key.
8. If the result matches the first two parts of the JWT (the header and payload) then the JWT is considered authorized and the connection is allowed.

Verifying a JWT

Okay, so you’ve created a JWT. But you are having some kind of a problem.

There are four common issues:

1. You’ve used the wrong API key
2. You’ve entered the wrong public key into Pure1
3. You are using the wrong private key
4. The JSON is malformed

There are other possibilities like the wrong type of RSA key etc, but that issue should be prevented when you try to add the key into Pure1.

Verifying the API Key

So let’s take the JWT from earlier:

eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJwdXJlMTphcGlrZXk6dHpja2QyVXNYR21ORmVjSyIsImlhdCI6MTYwNDY4NDIxMywiZXhwIjoxNjA3Mjc2MjEzfQ.uFAJ-XS0e3mcvp_xGWRCMqaRQ8iXazp9MaGGeLmNeVsWI2vhNeVD2_NrHiLhAphDjHrQLmuKY9clWh2R3WJOLrwTR3N_1aaYGKCMjLXfDuXgcHD980RgsXCM0axIsdX9KKmnyVDbcuipiy1yUxyxRiCtd6o2OBgF9H3bZoGTSbTiPPRHHoyAwEKaThqXLFVB3jUrxukvMAuH9mTATXi7cfxplPJZixj3mhi0-IaZ4UKYXHFi9vUyOg7wfWvtev45E6SHMV8ye9tevrh8zG5jWDJYmOBqdRtxSgXXKE-lVg3qiEXToSddq0Xe4rRnS4avgZjBWPENaDkMKBug8KBnqg

Navigate the the website jwt.io. Paste the first two parts of the JWT into the left panel:

This will automatically decode them into the header and payload. If it looks different or weird:

It is formatted incorrectly or encoded incorrectly. If it is formatted correctly, the main thing you want to check here, is that the API key is correct.

Verifying the Public Key

Next the signature itself.

Now paste the FULL JWT into the left panel:

Note that the signature is noted as invalid. This is because there is no key to check it with. For this you need the public key.

My public key is the following:

-----BEGIN PUBLIC KEY-----
MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEA4WVi9HtenBdPUbZKvjOo
efYxUsNOt+eUTPAMWU3dNoUR/2pkoy1i2+iOBwkQgh5veyKZpQQCjEnyGKeliZEk
MPh2bJgFuKKyG4L0wgU1v7AEsuhQoNg7mdpPUlmIiIkVU91mtLxjES95AXl0A0oj
kUFe5JsE71Qt+2GgVJoB7fRh2l8eQnvje9tcfGFaAKoW63FmalNrZq8FjDbhQ4N3
AFANClfRlP7Ru3QLGbPWl3WMdGkjZU7STYON3fg3TN7K/95xRheOqpbZ/JpL9FHr
wZNiSx4hY82pjqDrdeF2mGDVILMa3FQIHNBqNyY1ORkLc8EW+wKJk+8OsNwbr4mj
WwIDAQAB
-----END PUBLIC KEY-----

Take that and paste it into the public key box. If this is the right public key, it will turn to signature verified. If it does not, you are using the wrong public key. In this case

Verifying the Private Key

You can also work it backwards.

If this is the header and payload I want to sign:

{
  "alg": "RS256",
  "typ": "JWT"
}

{
  "iss": "pure1:apikey:tzckd2UsXGmNFecK",
  "iat": 1604684213,
  "exp": 1607276213
}

Clear everything in the site:

Then put the header and payload in:

Now paste in the private key:

It will create the JWT for you. If that JWT is different than one you generated elsewhere your other JWT was incorrectly created.

You can add in your public key to ensure it is all good:

What to do with a bad JWT

So if you get an authorization error with Pure1 what should you do? Make sure the combination that you are using is correct: right API key, right public key, right private key. Figure out which one is wrong. The simplest thing often is to start over: create a new key pair, add the public one into Pure1, and retry.

I was deploying a new Tanzu Kubernetes Cluster the other day with:

tkg create cluster cody-prod --plan prod --vsphere-controlplane-endpoint-ip 10.21.202.236

Oh great, it worked without sudo!

Except not quite:

The VMs were all deployed and good, but it looks like it did not quite have the permissions to save the information to my local config. This is likely salvageable, but since it is new, just as simple to delete it and re-run with sudo

sudo tkg delete cluster cody-prod

Deletes the VMs and removes the references. The likely solution here is folder permissions, but for now just use sudo with tkg when provisioning new resources.

The recent release of VMware Cloud Foundation (version 4.1) added support for Virtual Volumes as a principle storage option via either iSCSI or Fibre Channel. See some more information on that here:

vVols, please report to the Principal’s Office! VCF 4.1 and vVols!

When you deploy a new workload domain in VCF, a new vCenter gets deployed and along with that a new cluster with hosts, and all of the various related pieces (NSX etc.). Before you can deploy a WD, you must have of course hosts–and ESXi needs to be installed on these hosts. Depending on what storage option you choose, there are a few requirements for these hosts besides just having ESXi.

Continue reading “Preparing for a vVol VMware Cloud Foundation Workload Domain with PowerShell”