mirror of
https://github.com/rustfs/rustfs.git
synced 2026-01-16 17:20:33 +00:00
* feat(kms): implement key management service with local and vault backends Signed-off-by: junxiang Mu <1948535941@qq.com> * feat(kms): enhance security with zeroize for sensitive data and improve key management Signed-off-by: junxiang Mu <1948535941@qq.com> * remove Hashi word Signed-off-by: junxiang Mu <1948535941@qq.com> * refactor: remove unused request structs from kms handlers Signed-off-by: junxiang Mu <1948535941@qq.com> --------- Signed-off-by: junxiang Mu <1948535941@qq.com>
2.6 KiB
2.6 KiB
KMS Troubleshooting
Use this checklist to diagnose and resolve common KMS-related issues.
Quick Diagnostics
- Check status
awscurl --service s3 --region us-east-1 \ --access_key admin --secret_key admin \ http://localhost:9000/rustfs/admin/v3/kms/status - Inspect logs – enable
RUST_LOG=rustfs::kms=debug. - Verify backend reachability – for Vault, run
vault statusand test network connectivity. - Run a smoke test – upload a small object with
--server-side-encryption AES256.
Common Issues
| Symptom | Likely Cause | Resolution |
|---|---|---|
status: NotConfigured |
KMS was never configured or configuration failed. | POST /kms/configure, then /kms/start. Check logs for JSON parsing errors. |
healthy: false |
Backend health probe failed; Vault sealed or filesystem inaccessible. | Unseal Vault, confirm permissions on the key directory, re-run /kms/start. |
InternalError: failed to create key |
Backend rejected the request (e.g. Vault policy). | Review Vault audit logs and ensure the RustFS policy has transit/keys/* access. |
AccessDenied when downloading SSE-C objects |
Missing/incorrect SSE-C headers. | Provide the same x-amz-server-side-encryption-customer-* headers used during upload. |
| Multipart SSE-C download truncated | Parts smaller than 5 MiB stored inline; older builds mishandled them. | Re-upload with ≥5 MiB parts or upgrade to the latest RustFS build. |
Operation not permitted during tests |
OS sandbox blocked launching vault or rustfs. |
Re-run tests with elevated permissions (cargo test ... with sandbox overrides). |
KMS key directory is required for local backend |
Started RustFS with --kms-backend local but no --kms-key-dir. |
Supply the flag or use the dynamic API to set the directory before calling /kms/start. |
Clearing the Cache
If data keys become stale (e.g. after manual rotation in Vault), clear the cache:
awscurl --service s3 --region us-east-1 \
--access_key admin --secret_key admin \
-X POST http://localhost:9000/rustfs/admin/v3/kms/clear-cache
Resetting the Service
POST /kms/stopPOST /kms/configurewith the known-good payloadPOST /kms/start- Verify with
/kms/status
Support Data Collection
When opening an issue, capture:
- Output of
/kms/statusand/kms/config - Relevant RustFS logs (
rustfs::kms=*) - Vault audit log snippets (if using Vault)
- The SSE headers used by the failing client request
Providing these artifacts drastically speeds up triage.