Hosting a KMS on the encrypted VxRail cluster, with Key Persistence

In my previous post (VxRail cluster encryption with the vSphere Native Key Provider) I spoke about how to encrypt your VxRail cluster with the vSphere Native Key Provider that was introduced in vSphere 7.0 U2.

It is a great feature and the post shows how easy it is to set it up and use it to encrypt your cluster. However… what if you do need more functionality, that only a Key Management Server (KMS) can offer? Oh, and I don’t want to spend more cost on an extra host (outside my cluster) to host the KMS appliance? I can’t put it on the actual cluster itself, that it is providing the encryption keys for! Or can I …?

Key Persistence With vSphere 7.0 U2, VMware also introduced key persistence. This means that “encrypted virtual machines and virtual TPMs can continue to function even when the key server is temporarily offline or unavailable. The ESXi hosts can persist the encryption keys to continue encryption and vTPM operations.” Your encrypted workloads will now keep running without interruption when the KMS is not available, even during host reboots. This feature does require TPM 2.0 to be present on the motherboard and enabled. It also requires the new Key Persistence feature to be enabled on the hosts, which can be done using the esxcli.

So after I thought it was great and easy to use the Native Key Provider (NKP) to encrypt my VxRail cluster, I thought why not encrypt the cluster with a proper KMS and host the KMS on one of the nodes of the cluster itself? It goes against the usual guidelines of good architecture, but … nothing was discovered by doing nothing! ;)

At this point, once the nodes come back up, you will notice that they all generate a host attestation alarm. This is expected and happens because Secure Boot is not enabled:

On each host, using esxcli, I also made sure that the system settings encryption mode was set to TPM:

Note that if TPM Security is not enabled properly in the BIOS, then setting encryption mode to TPM will fail:

Once TPM Security is enabled and encryption mode is set to TPM, you can enable the key persistence feature on the host:

Next I deployed a CloudLink KMS on the cluster:

You can see here that it is hosted on one of the nodes of the cluster:

Finally it was time to do the test. It’s a simple one, but it proves the point. I gave the node that is hosting the KMS a surprise boot and it came back up and running without any issues. It never interrupted any workload on the VxRail cluster.

Conclusion Similar to the Native Key Provider, the new Key Persistence feature is something that goes well with the overall goal of VxRail – making IT simpler and easier!

This feature allows you to use a KMS and simply host it on the cluster that makes use of its encryption services. In light of the focus on cyber crime, this is definitely an interesting feature.