Secrets in HashiCorp Vault

IdentityMesh ships with three secret-store backends:

This document covers the HashiCorp Vault provider.

Why HashiCorp Vault

The DPAPI provider encrypts each secret blob with the host’s LocalMachine key. That key never leaves the host, so the blobs are not portable: a re-image, a DR rebuild, or a cluster failover to a second host all require re-provisioning every connector secret by hand.

Azure Key Vault solves the portability problem in Azure environments. HashiCorp Vault is the equivalent option for shops that don’t run on Azure or that already operate Vault as their organisation-wide secret broker — including AWS, GCP, and pure on-prem deployments where an Azure dependency would be out of place.

Enabling HashiCorp Vault

In the sync engine’s appsettings.json:

{
  "Secrets": {
    "Provider": "HashiCorpVault",
    "HashiCorpVault": {
      "Address": "https://vault.example.com:8200",
      "RoleId": "<approle-role-id>",
      "SecretId": "<approle-secret-id>",
      "MountPath": "secret",
      "PathPrefix": "identitymesh/"
    }
  }
}

When Secrets:Provider is unset or any value other than HashiCorpVault (or AzureKeyVault), IdentityMesh uses the DPAPI store and the HashiCorp Vault configuration is ignored. So flipping the switch is the only deployment change required.

Configuration keys

The engine fails fast at startup with a clear InvalidOperationException if Address is missing or if neither Token nor (RoleId + SecretId) are configured.

Authentication

Two auth methods are supported:

1. Token auth — set Secrets:HashiCorpVault:Token to a Vault token. Convenient for dev workstations and first-touch provisioning, but the token has whatever lifetime Vault gives it, so this is not the recommended production setup.

2. AppRole auth — set Secrets:HashiCorpVault:RoleId and Secrets:HashiCorpVault:SecretId. The engine exchanges them for a short-lived token at startup and on renewal. This is the recommended production configuration: the role ID is non-secret and lives in config, the secret ID is delivered out-of-band (CD pipeline, response wrapping, or Vault Agent sidecar) and rotates independently of the engine.

Whichever identity the engine ends up using needs a Vault policy that allows create, update, and read on <MountPath>/data/<PathPrefix>im-* (and delete if you intend to prune secrets out-of-band).

Storage layout: KV v2

IdentityMesh writes to the KV v2 secrets engine. Each secret is stored as a single dictionary key:

{
  "value": "<base64-encoded byte[]>"
}

KV v2 versions every write, so re-setting the same ref creates a new version rather than overwriting in place — useful for audit trails and emergency rollback. The engine always reads the current version.

Path mapping

Callers pass arbitrary refs (e.g. secret://ad/svc/password). Vault paths are hierarchical (slashes have semantic meaning), so mirroring those slashes into Vault would create awkward, unreviewable hierarchies and would couple the on-vault layout to how callers happen to format their refs today. Instead, IdentityMesh maps each ref to a stable flat path:

secretRef  →  PathPrefix + "im-" + lowercase-hex(SHA-256(secretRef))[:32]

For example, with PathPrefix = "identitymesh/", secret://ad/svc/password becomes identitymesh/im-3f2a1b... — predictable, collision-resistant (128 bits), and never truncates a long ref. The engine logs the mapping at INFO when it writes a secret, so you can correlate refs to vault paths when needed.

The mapping is one-way (you can’t recover the original ref from the vault path). Keep the ref strings in your connector configs as the canonical identifier; the vault path is an implementation detail.

Provisioning a secret

Secrets are still provisioned via the Admin UI / Admin API exactly as they are with the DPAPI and Key Vault stores. The engine handles the encoding-and-storage step transparently:

1. Operator enters the cleartext in the Admin UI.
2. The Admin API calls ISecretStore.SetAsync(ref, bytes).
3. With Secrets:Provider = HashiCorpVault, that resolves to the HashiCorp Vault store, which Base64-encodes the bytes and calls WriteSecretAsync against the configured KV v2 mount.
4. The connector config keeps only the ref. Rotation is ref-stable: re-setting the same ref creates a new KV v2 version, the engine always reads the current version.

What if the vault is unreachable

The engine attempts to read the secret on each connector run. If Vault is unreachable (network outage, sealed vault, AppRole revoked, mount deleted) the connector run fails with the underlying VaultApiException surfaced. The watermark is not advanced, so the next run re-tries from the last good checkpoint.

Common failure modes to monitor:

• Vault sealed — every read fails with HTTP 503. Unseal the vault; the engine recovers on the next run.
• Token / AppRole secret expired — every read fails with HTTP 403. Re-issue credentials and restart the engine.
• Mount path moved — every read fails with HTTP 404. IdentityMesh treats 404 as “secret not present” (consistent with the Azure store), so connector runs will see refs as unprovisioned. Restore the mount or update MountPath.

For longer outages, vault availability — not IdentityMesh — is the SLA you should monitor. Use Vault’s own health endpoints and metrics to decide whether the engine is the right place to alert.

Migration from DPAPI to HashiCorp Vault

There is no automated migration today. The procedure is:

1. Provision the Vault mount and a policy granting the engine’s AppRole create, update, and read on the IdentityMesh path prefix.
2. Take the engine offline.
3. For each secret in the existing DPAPI store, re-enter the cleartext via the Admin UI. With Secrets:Provider already flipped to HashiCorpVault in the engine’s config, the new value lands in Vault under its mapped path.
4. Restart the engine.

The DPAPI rows in IM_Secrets remain in the database after migration. They are unused but harmless; an explicit cleanup pass can purge them once you’re confident the Vault store is the source of truth.

Related

• secrets-and-dpapi.md — the default on-prem provider.
• secrets-keyvault.md — the Azure Key Vault provider.
• secret-rotation.md — rotation runbook (ref-stable rotation works the same way for all three providers).
• deployment-architecture.md — where secrets fit in the broader topology.