mirror of
https://github.com/rustfs/rustfs.git
synced 2026-01-16 17:20:33 +00:00
fix: restore localized samples in tests (#749)
* fix: restore required localized examples * style: fix formatting issues
This commit is contained in:
@@ -1,139 +1,141 @@
|
||||
# RustFS 性能测试指南
|
||||
# RustFS Performance Testing Guide
|
||||
|
||||
本文档提供了对 RustFS 进行性能测试和性能分析的完整方法和工具。
|
||||
This document describes the recommended tools and workflows for benchmarking RustFS and analyzing performance bottlenecks.
|
||||
|
||||
## 概览
|
||||
## Overview
|
||||
|
||||
RustFS 提供了多种性能测试和分析工具:
|
||||
RustFS exposes several complementary tooling options:
|
||||
|
||||
1. **性能分析(Profiling)** - 使用内置的 pprof 接口收集 CPU 性能数据
|
||||
2. **负载测试(Load Testing)** - 使用多种客户端工具模拟高并发请求
|
||||
3. **监控和分析** - 查看性能指标和识别性能瓶颈
|
||||
1. **Profiling** – collect CPU samples through the built-in `pprof` endpoints.
|
||||
2. **Load testing** – drive concurrent requests with dedicated client utilities.
|
||||
3. **Monitoring and analysis** – inspect collected metrics to locate hotspots.
|
||||
|
||||
## 前置条件
|
||||
## Prerequisites
|
||||
|
||||
### 1. 启用性能分析
|
||||
### 1. Enable profiling support
|
||||
|
||||
在启动 RustFS 时,需要设置环境变量启用性能分析功能:
|
||||
Set the profiling environment variable before launching RustFS:
|
||||
|
||||
```bash
|
||||
export RUSTFS_ENABLE_PROFILING=true
|
||||
./rustfs
|
||||
```
|
||||
|
||||
### 2. 安装依赖工具
|
||||
### 2. Install required tooling
|
||||
|
||||
确保系统中安装了以下工具:
|
||||
Make sure the following dependencies are available:
|
||||
|
||||
```bash
|
||||
# 基础工具
|
||||
curl # HTTP 请求
|
||||
jq # JSON 处理 (可选)
|
||||
# Base tools
|
||||
curl # HTTP requests
|
||||
jq # JSON processing (optional)
|
||||
|
||||
# 分析工具
|
||||
go # Go pprof 工具 (可选,用于 protobuf 格式)
|
||||
python3 # Python 负载测试脚本
|
||||
# Analysis tools
|
||||
go # Go pprof CLI (optional, required for protobuf output)
|
||||
python3 # Python load-testing scripts
|
||||
|
||||
# macOS 用户
|
||||
# macOS users
|
||||
brew install curl jq go python3
|
||||
|
||||
# Ubuntu/Debian 用户
|
||||
# Ubuntu/Debian users
|
||||
sudo apt-get install curl jq golang-go python3
|
||||
```
|
||||
|
||||
## 性能测试方法
|
||||
## Performance Testing Methods
|
||||
|
||||
### 方法 1:使用专业脚本(推荐)
|
||||
### Method 1: Use the dedicated profiling script (recommended)
|
||||
|
||||
项目提供了完整的性能分析脚本:
|
||||
The repository ships with a helper script for common profiling flows:
|
||||
|
||||
```bash
|
||||
# 查看脚本帮助
|
||||
# Show command help
|
||||
./scripts/profile_rustfs.sh help
|
||||
|
||||
# 检查性能分析状态
|
||||
# Check profiler status
|
||||
./scripts/profile_rustfs.sh status
|
||||
|
||||
# 收集火焰图(30秒)
|
||||
# Capture a 30 second flame graph
|
||||
./scripts/profile_rustfs.sh flamegraph
|
||||
|
||||
# 收集 protobuf 格式性能数据
|
||||
# Download protobuf-formatted samples
|
||||
./scripts/profile_rustfs.sh protobuf
|
||||
|
||||
# 收集两种格式的性能数据
|
||||
# Collect both formats
|
||||
./scripts/profile_rustfs.sh both
|
||||
|
||||
# 自定义参数
|
||||
# Provide custom arguments
|
||||
./scripts/profile_rustfs.sh -d 60 -u http://192.168.1.100:9000 both
|
||||
```
|
||||
|
||||
### 方法 2:使用 Python 综合测试
|
||||
### Method 2: Run the Python end-to-end tester
|
||||
|
||||
Python 脚本提供了负载测试和性能分析的一体化解决方案:
|
||||
A Python utility combines background load generation with profiling:
|
||||
|
||||
```bash
|
||||
# 运行综合性能分析
|
||||
# Launch the integrated test harness
|
||||
python3 test_load.py
|
||||
```
|
||||
|
||||
此脚本会:
|
||||
1. 启动后台负载测试(多线程 S3 操作)
|
||||
2. 并行收集性能分析数据
|
||||
3. 生成火焰图用于分析
|
||||
The script will:
|
||||
|
||||
### 方法 3:使用简单负载测试
|
||||
1. Launch multi-threaded S3 operations as load.
|
||||
2. Pull profiling samples in parallel.
|
||||
3. Produce a flame graph for investigation.
|
||||
|
||||
对于快速测试,可以使用 bash 脚本:
|
||||
### Method 3: Simple shell-based load test
|
||||
|
||||
For quick smoke checks, a lightweight bash script is also provided:
|
||||
|
||||
```bash
|
||||
# 运行简单负载测试
|
||||
# Execute a lightweight benchmark
|
||||
./simple_load_test.sh
|
||||
```
|
||||
|
||||
## 性能分析输出格式
|
||||
## Profiling Output Formats
|
||||
|
||||
### 1. 火焰图(SVG 格式)
|
||||
### 1. Flame graph (SVG)
|
||||
|
||||
- **用途**: 可视化 CPU 使用情况
|
||||
- **文件**: `rustfs_profile_TIMESTAMP.svg`
|
||||
- **查看方式**: 使用浏览器打开 SVG 文件
|
||||
- **分析要点**:
|
||||
- 宽度表示 CPU 使用时间
|
||||
- 高度表示调用栈深度
|
||||
- 点击可以放大特定函数
|
||||
- **Purpose**: Visualize CPU time distribution.
|
||||
- **File name**: `rustfs_profile_TIMESTAMP.svg`
|
||||
- **How to view**: Open the SVG in a browser.
|
||||
- **Interpretation tips**:
|
||||
- Width reflects CPU time per function.
|
||||
- Height illustrates call-stack depth.
|
||||
- Click to zoom into specific frames.
|
||||
|
||||
```bash
|
||||
# 在浏览器中打开
|
||||
# Example: open the file in a browser
|
||||
open profiles/rustfs_profile_20240911_143000.svg
|
||||
```
|
||||
|
||||
### 2. Protobuf 格式
|
||||
### 2. Protobuf samples
|
||||
|
||||
- **用途**: 使用 Go pprof 工具进行详细分析
|
||||
- **文件**: `rustfs_profile_TIMESTAMP.pb`
|
||||
- **分析工具**: `go tool pprof`
|
||||
- **Purpose**: Feed data to the `go tool pprof` command.
|
||||
- **File name**: `rustfs_profile_TIMESTAMP.pb`
|
||||
- **Tooling**: `go tool pprof`
|
||||
|
||||
```bash
|
||||
# 使用 Go pprof 分析
|
||||
# Analyze the protobuf output
|
||||
go tool pprof profiles/rustfs_profile_20240911_143000.pb
|
||||
|
||||
# pprof 常用命令
|
||||
(pprof) top # 显示 CPU 使用率最高的函数
|
||||
(pprof) list func # 显示指定函数的源代码
|
||||
(pprof) web # 生成 web 界面(需要 graphviz)
|
||||
(pprof) png # 生成 PNG 图片
|
||||
(pprof) help # 查看所有命令
|
||||
# Common pprof commands
|
||||
(pprof) top # Show hottest call sites
|
||||
(pprof) list func # Display annotated source for a function
|
||||
(pprof) web # Launch the web UI (requires graphviz)
|
||||
(pprof) png # Render a PNG flame chart
|
||||
(pprof) help # List available commands
|
||||
```
|
||||
|
||||
## API 接口使用
|
||||
## API Usage
|
||||
|
||||
### 检查性能分析状态
|
||||
### Check profiling status
|
||||
|
||||
```bash
|
||||
curl "http://127.0.0.1:9000/rustfs/admin/debug/pprof/status"
|
||||
```
|
||||
|
||||
返回示例:
|
||||
Sample response:
|
||||
|
||||
```json
|
||||
{
|
||||
"enabled": "true",
|
||||
@@ -141,186 +143,187 @@ curl "http://127.0.0.1:9000/rustfs/admin/debug/pprof/status"
|
||||
}
|
||||
```
|
||||
|
||||
### 收集性能数据
|
||||
### Capture profiling data
|
||||
|
||||
```bash
|
||||
# 收集 30 秒的火焰图
|
||||
# Fetch a 30-second flame graph
|
||||
curl "http://127.0.0.1:9000/rustfs/admin/debug/pprof/profile?seconds=30&format=flamegraph" \
|
||||
-o profile.svg
|
||||
|
||||
# 收集 protobuf 格式数据
|
||||
# Fetch protobuf output
|
||||
curl "http://127.0.0.1:9000/rustfs/admin/debug/pprof/profile?seconds=30&format=protobuf" \
|
||||
-o profile.pb
|
||||
```
|
||||
|
||||
**参数说明**:
|
||||
- `seconds`: 收集时长(1-300 秒)
|
||||
- `format`: 输出格式(`flamegraph`/`svg` 或 `protobuf`/`pb`)
|
||||
**Parameters**
|
||||
- `seconds`: Duration between 1 and 300 seconds.
|
||||
- `format`: Output format (`flamegraph`/`svg` or `protobuf`/`pb`).
|
||||
|
||||
## 负载测试场景
|
||||
## Load Testing Scenarios
|
||||
|
||||
### 1. S3 API 负载测试
|
||||
### 1. S3 API workload
|
||||
|
||||
使用 Python 脚本进行完整的 S3 操作负载测试:
|
||||
Use the Python harness to exercise a complete S3 workflow:
|
||||
|
||||
```python
|
||||
# 基本配置
|
||||
# Basic configuration
|
||||
tester = S3LoadTester(
|
||||
endpoint="http://127.0.0.1:9000",
|
||||
access_key="rustfsadmin",
|
||||
access_key="rustfsadmin",
|
||||
secret_key="rustfsadmin"
|
||||
)
|
||||
|
||||
# 运行负载测试
|
||||
# 4 个线程,每个线程执行 10 次操作
|
||||
# Execute the load test
|
||||
# Four threads, ten operations each
|
||||
tester.run_load_test(num_threads=4, operations_per_thread=10)
|
||||
```
|
||||
|
||||
每次操作包括:
|
||||
1. 上传 1MB 对象
|
||||
2. 下载对象
|
||||
3. 删除对象
|
||||
Each iteration performs:
|
||||
1. Upload a 1 MB object.
|
||||
2. Download the object.
|
||||
3. Delete the object.
|
||||
|
||||
### 2. 自定义负载测试
|
||||
### 2. Custom load scenarios
|
||||
|
||||
```bash
|
||||
# 创建测试桶
|
||||
# Create a test bucket
|
||||
curl -X PUT "http://127.0.0.1:9000/test-bucket"
|
||||
|
||||
# 并发上传测试
|
||||
# Concurrent uploads
|
||||
for i in {1..10}; do
|
||||
echo "test data $i" | curl -X PUT "http://127.0.0.1:9000/test-bucket/object-$i" -d @- &
|
||||
done
|
||||
wait
|
||||
|
||||
# 并发下载测试
|
||||
# Concurrent downloads
|
||||
for i in {1..10}; do
|
||||
curl "http://127.0.0.1:9000/test-bucket/object-$i" > /dev/null &
|
||||
done
|
||||
wait
|
||||
```
|
||||
|
||||
## 性能分析最佳实践
|
||||
## Profiling Best Practices
|
||||
|
||||
### 1. 测试环境准备
|
||||
### 1. Environment preparation
|
||||
|
||||
- 确保 RustFS 已启用性能分析: `RUSTFS_ENABLE_PROFILING=true`
|
||||
- 使用独立的测试环境,避免其他程序干扰
|
||||
- 确保有足够的磁盘空间存储分析文件
|
||||
- Confirm that `RUSTFS_ENABLE_PROFILING=true` is set.
|
||||
- Use an isolated benchmark environment to avoid interference.
|
||||
- Reserve disk space for generated profile artifacts.
|
||||
|
||||
### 2. 数据收集建议
|
||||
### 2. Data collection tips
|
||||
|
||||
- **预热阶段**: 先运行 5-10 分钟的轻量负载
|
||||
- **数据收集**: 在稳定负载下收集 30-60 秒的性能数据
|
||||
- **多次采样**: 收集多个样本进行对比分析
|
||||
- **Warm-up**: Run a light workload for 5–10 minutes before sampling.
|
||||
- **Sampling window**: Capture 30–60 seconds under steady load.
|
||||
- **Multiple samples**: Take several runs to compare results.
|
||||
|
||||
### 3. 分析重点
|
||||
### 3. Analysis focus areas
|
||||
|
||||
在火焰图中重点关注:
|
||||
When inspecting flame graphs, pay attention to:
|
||||
|
||||
1. **宽度最大的函数** - CPU 使用时间最长
|
||||
2. **平顶函数** - 可能的性能瓶颈
|
||||
3. **深度调用栈** - 可能的递归或复杂逻辑
|
||||
4. **意外的系统调用** - I/O 或内存分配问题
|
||||
1. **The widest frames** – most CPU time consumed.
|
||||
2. **Flat plateaus** – likely bottlenecks.
|
||||
3. **Deep call stacks** – recursion or complex logic.
|
||||
4. **Unexpected syscalls** – I/O stalls or allocation churn.
|
||||
|
||||
### 4. 常见性能问题
|
||||
### 4. Common issues
|
||||
|
||||
- **锁竞争**: 查找 `std::sync` 相关函数
|
||||
- **内存分配**: 查找 `alloc` 相关函数
|
||||
- **I/O 等待**: 查找文件系统或网络 I/O 函数
|
||||
- **序列化开销**: 查找 JSON/XML 解析函数
|
||||
- **Lock contention**: Investigate frames under `std::sync`.
|
||||
- **Memory allocation**: Search for `alloc`-related frames.
|
||||
- **I/O wait**: Review filesystem or network call stacks.
|
||||
- **Serialization overhead**: Look for JSON/XML parsing hotspots.
|
||||
|
||||
## 故障排除
|
||||
## Troubleshooting
|
||||
|
||||
### 1. 性能分析未启用
|
||||
### 1. Profiling disabled
|
||||
|
||||
错误信息:`{"enabled":"false"}`
|
||||
Error: `{"enabled":"false"}`
|
||||
|
||||
**Fix**:
|
||||
|
||||
解决方案:
|
||||
```bash
|
||||
export RUSTFS_ENABLE_PROFILING=true
|
||||
# 重启 RustFS
|
||||
# Restart RustFS
|
||||
```
|
||||
|
||||
### 2. 连接被拒绝
|
||||
### 2. Connection refused
|
||||
|
||||
错误信息:`Connection refused`
|
||||
Error: `Connection refused`
|
||||
|
||||
检查项:
|
||||
- RustFS 是否正在运行
|
||||
- 端口是否正确(默认 9000)
|
||||
- 防火墙设置
|
||||
**Checklist**:
|
||||
- Confirm RustFS is running.
|
||||
- Ensure the port number is correct (default 9000).
|
||||
- Verify firewall rules.
|
||||
|
||||
### 3. 分析文件过大
|
||||
### 3. Oversized profile output
|
||||
|
||||
如果生成的分析文件过大:
|
||||
- 减少收集时间(如 15-30 秒)
|
||||
- 降低负载测试的并发度
|
||||
- 使用 protobuf 格式而非 SVG
|
||||
If artifacts become too large:
|
||||
- Shorten the capture window (e.g., 15–30 seconds).
|
||||
- Reduce load-test concurrency.
|
||||
- Prefer protobuf output instead of SVG.
|
||||
|
||||
## 配置参数
|
||||
## Configuration Parameters
|
||||
|
||||
### 环境变量
|
||||
### Environment variables
|
||||
|
||||
| 变量 | 默认值 | 描述 |
|
||||
| Variable | Default | Description |
|
||||
|------|--------|------|
|
||||
| `RUSTFS_ENABLE_PROFILING` | `false` | 启用性能分析 |
|
||||
| `RUSTFS_URL` | `http://127.0.0.1:9000` | RustFS 服务器地址 |
|
||||
| `PROFILE_DURATION` | `30` | 性能数据收集时长(秒) |
|
||||
| `OUTPUT_DIR` | `./profiles` | 输出文件目录 |
|
||||
| `RUSTFS_ENABLE_PROFILING` | `false` | Enable profiling support |
|
||||
| `RUSTFS_URL` | `http://127.0.0.1:9000` | RustFS endpoint |
|
||||
| `PROFILE_DURATION` | `30` | Profiling duration in seconds |
|
||||
| `OUTPUT_DIR` | `./profiles` | Output directory |
|
||||
|
||||
### 脚本参数
|
||||
### Script arguments
|
||||
|
||||
```bash
|
||||
./scripts/profile_rustfs.sh [OPTIONS] [COMMAND]
|
||||
|
||||
OPTIONS:
|
||||
-u, --url URL RustFS URL
|
||||
-d, --duration SECONDS Profile duration
|
||||
-d, --duration SECONDS Profile duration
|
||||
-o, --output DIR Output directory
|
||||
|
||||
COMMANDS:
|
||||
status 检查状态
|
||||
flamegraph 收集火焰图
|
||||
protobuf 收集 protobuf 数据
|
||||
both 收集两种格式(默认)
|
||||
status Check profiler status
|
||||
flamegraph Collect a flame graph
|
||||
protobuf Collect protobuf samples
|
||||
both Collect both formats (default)
|
||||
```
|
||||
|
||||
## 输出文件位置
|
||||
## Output Locations
|
||||
|
||||
- **脚本输出**: `./profiles/` 目录
|
||||
- **Python 脚本**: `/tmp/rustfs_profiles/` 目录
|
||||
- **文件命名**: `rustfs_profile_TIMESTAMP.{svg|pb}`
|
||||
- **Script output**: `./profiles/`
|
||||
- **Python script**: `/tmp/rustfs_profiles/`
|
||||
- **File naming**: `rustfs_profile_TIMESTAMP.{svg|pb}`
|
||||
|
||||
## 示例工作流程
|
||||
## Example Workflow
|
||||
|
||||
1. **启动 RustFS**:
|
||||
1. **Launch RustFS**
|
||||
```bash
|
||||
RUSTFS_ENABLE_PROFILING=true ./rustfs
|
||||
```
|
||||
|
||||
2. **验证性能分析可用**:
|
||||
2. **Verify profiling availability**
|
||||
```bash
|
||||
./scripts/profile_rustfs.sh status
|
||||
```
|
||||
|
||||
3. **开始负载测试**:
|
||||
3. **Start a load test**
|
||||
```bash
|
||||
python3 test_load.py &
|
||||
```
|
||||
|
||||
4. **收集性能数据**:
|
||||
4. **Collect samples**
|
||||
```bash
|
||||
./scripts/profile_rustfs.sh -d 60 both
|
||||
```
|
||||
|
||||
5. **分析结果**:
|
||||
5. **Inspect the results**
|
||||
```bash
|
||||
# 查看火焰图
|
||||
# Review the flame graph
|
||||
open profiles/rustfs_profile_*.svg
|
||||
|
||||
# 或使用 pprof 分析
|
||||
|
||||
# Or analyze the protobuf output
|
||||
go tool pprof profiles/rustfs_profile_*.pb
|
||||
```
|
||||
|
||||
通过这个完整的性能测试流程,你可以系统地分析 RustFS 的性能特征,识别瓶颈,并进行有针对性的优化。
|
||||
Following this workflow helps you understand RustFS performance characteristics, locate bottlenecks, and implement targeted optimizations.
|
||||
|
||||
258
docs/README.md
258
docs/README.md
@@ -1,239 +1,239 @@
|
||||
# RustFS 文档中心
|
||||
# RustFS Documentation Center
|
||||
|
||||
欢迎来到 RustFS 分布式文件系统文档中心!
|
||||
Welcome to the RustFS distributed file system documentation center!
|
||||
|
||||
## 📚 文档导航
|
||||
## 📚 Documentation Navigation
|
||||
|
||||
### 🔐 KMS (密钥管理服务)
|
||||
### 🔐 KMS (Key Management Service)
|
||||
|
||||
RustFS KMS 提供企业级密钥管理和数据加密服务。
|
||||
RustFS KMS delivers enterprise-grade key management and data encryption.
|
||||
|
||||
| 文档 | 描述 | 适用场景 |
|
||||
| Document | Description | Audience |
|
||||
|------|------|----------|
|
||||
| [KMS 使用指南](./kms/README.md) | 完整的 KMS 使用文档,包含快速开始、配置和部署 | 所有用户必读 |
|
||||
| [HTTP API 接口](./kms/http-api.md) | HTTP REST API 接口文档和使用示例 | 管理员和运维 |
|
||||
| [编程 API 接口](./kms/api.md) | Rust 库编程接口和代码示例 | 开发者集成 |
|
||||
| [配置参考](./kms/configuration.md) | 完整的配置选项和环境变量说明 | 系统管理员 |
|
||||
| [故障排除](./kms/troubleshooting.md) | 常见问题诊断和解决方案 | 运维人员 |
|
||||
| [安全指南](./kms/security.md) | 安全最佳实践和合规指导 | 安全架构师 |
|
||||
| [KMS User Guide](./kms/README.md) | Comprehensive KMS guide with quick start, configuration, and deployment steps | Required reading for all users |
|
||||
| [HTTP API Reference](./kms/http-api.md) | HTTP REST API reference with usage examples | Administrators and operators |
|
||||
| [Programming API Reference](./kms/api.md) | Rust library APIs and code samples | Developers |
|
||||
| [Configuration Reference](./kms/configuration.md) | Complete configuration options and environment variables | System administrators |
|
||||
| [Troubleshooting](./kms/troubleshooting.md) | Diagnosis tips and solutions for common issues | Operations engineers |
|
||||
| [Security Guide](./kms/security.md) | Security best practices and compliance guidance | Security architects |
|
||||
|
||||
## 🚀 快速开始
|
||||
## 🚀 Quick Start
|
||||
|
||||
### 1. KMS 5分钟快速部署
|
||||
### 1. Deploy KMS in 5 Minutes
|
||||
|
||||
**生产环境(使用 Vault)**
|
||||
**Production (Vault backend)**
|
||||
|
||||
```bash
|
||||
# 1. 启用 Vault 功能编译
|
||||
# 1. Enable the Vault feature flag
|
||||
cargo build --features vault --release
|
||||
|
||||
# 2. 配置环境变量
|
||||
# 2. Configure environment variables
|
||||
export RUSTFS_VAULT_ADDRESS=https://vault.company.com:8200
|
||||
export RUSTFS_VAULT_TOKEN=hvs.CAESIJ...
|
||||
|
||||
# 3. 启动服务
|
||||
# 3. Launch the service
|
||||
./target/release/rustfs server
|
||||
```
|
||||
|
||||
**开发测试(使用本地后端)**
|
||||
**Development & Testing (Local backend)**
|
||||
|
||||
```bash
|
||||
# 1. 编译测试版本
|
||||
# 1. Build a release binary
|
||||
cargo build --release
|
||||
|
||||
# 2. 配置本地存储
|
||||
# 2. Configure local storage
|
||||
export RUSTFS_KMS_BACKEND=Local
|
||||
export RUSTFS_KMS_LOCAL_KEY_DIR=/tmp/rustfs-keys
|
||||
|
||||
# 3. 启动服务
|
||||
# 3. Launch the service
|
||||
./target/release/rustfs server
|
||||
```
|
||||
|
||||
### 2. S3 兼容加密
|
||||
### 2. S3-Compatible Encryption
|
||||
|
||||
```bash
|
||||
# 上传加密文件
|
||||
# Upload an encrypted object
|
||||
curl -X PUT https://rustfs.company.com/bucket/sensitive.txt \
|
||||
-H "x-amz-server-side-encryption: AES256" \
|
||||
--data-binary @sensitive.txt
|
||||
|
||||
# 自动解密下载
|
||||
# Download with automatic decryption
|
||||
curl https://rustfs.company.com/bucket/sensitive.txt
|
||||
```
|
||||
|
||||
## 🏗️ 架构概览
|
||||
## 🏗️ Architecture Overview
|
||||
|
||||
### KMS 三层安全架构
|
||||
### Three-Layer KMS Security Architecture
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────┐
|
||||
│ 应用层 │
|
||||
│ Application Layer │
|
||||
│ ┌─────────────┐ ┌─────────────┐ │
|
||||
│ │ S3 API │ │ REST API │ │
|
||||
│ │ S3 API │ │ REST API │ │
|
||||
│ └─────────────┘ └─────────────┘ │
|
||||
├─────────────────────────────────────────────────┤
|
||||
│ 加密层 │
|
||||
│ ┌─────────────┐ 加密 ┌─────────────────┐ │
|
||||
│ │ 对象数据 │ ◄───► │ 数据密钥 (DEK) │ │
|
||||
│ └─────────────┘ └─────────────────┘ │
|
||||
│ Encryption Layer │
|
||||
│ ┌─────────────┐ Encrypt ┌─────────────────┐ │
|
||||
│ │ Object Data │ ◄──────► │ Data Key (DEK) │ │
|
||||
│ └─────────────┘ └─────────────────┘ │
|
||||
├─────────────────────────────────────────────────┤
|
||||
│ 密钥管理层 │
|
||||
│ ┌─────────────────┐ 加密 ┌──────────────┐ │
|
||||
│ │ 数据密钥 (DEK) │ ◄────│ 主密钥 │ │
|
||||
│ └─────────────────┘ │ (Vault/HSM) │ │
|
||||
│ └──────────────┘ │
|
||||
│ Key Management Layer │
|
||||
│ ┌─────────────────┐ Encrypt ┌──────────────┐ │
|
||||
│ │ Data Key (DEK) │ ◄───────│ Master Key │ │
|
||||
│ └─────────────────┘ │ (Vault/HSM) │ │
|
||||
│ └──────────────┘ │
|
||||
└─────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
### 核心特性
|
||||
### Key Features
|
||||
|
||||
- ✅ **多层加密**: Master Key → DEK → Object Data
|
||||
- ✅ **高性能**: 1MB 流式加密,支持大文件
|
||||
- ✅ **多后端**: Vault (生产) + Local (测试)
|
||||
- ✅ **S3 兼容**: 支持标准 SSE-S3/SSE-KMS 头
|
||||
- ✅ **企业级**: 审计、监控、合规支持
|
||||
- ✅ **Multi-layer encryption**: Master Key → DEK → Object Data
|
||||
- ✅ **High performance**: 1 MB streaming encryption with large file support
|
||||
- ✅ **Multiple backends**: Vault (production) + Local (testing)
|
||||
- ✅ **S3 compatibility**: Supports standard SSE-S3/SSE-KMS headers
|
||||
- ✅ **Enterprise-ready**: Auditing, monitoring, and compliance features
|
||||
|
||||
## 📖 学习路径
|
||||
## 📖 Learning Paths
|
||||
|
||||
### 👨💻 开发者
|
||||
### 👨💻 Developers
|
||||
|
||||
1. 阅读 [编程 API 接口](./kms/api.md) 了解 Rust 库使用
|
||||
2. 查看代码示例学习集成方法
|
||||
3. 参考 [故障排除](./kms/troubleshooting.md) 解决问题
|
||||
1. Read the [Programming API Reference](./kms/api.md) to learn the Rust library
|
||||
2. Review the sample code to understand integration patterns
|
||||
3. Consult [Troubleshooting](./kms/troubleshooting.md) when issues occur
|
||||
|
||||
### 👨💼 系统管理员
|
||||
### 👨💼 System Administrators
|
||||
|
||||
1. 从 [KMS 使用指南](./kms/README.md) 开始
|
||||
2. 学习 [HTTP API 接口](./kms/http-api.md) 进行管理
|
||||
3. 详细阅读 [配置参考](./kms/configuration.md)
|
||||
4. 设置监控和日志
|
||||
1. Start with the [KMS User Guide](./kms/README.md)
|
||||
2. Learn the [HTTP API Reference](./kms/http-api.md) for management tasks
|
||||
3. Study the [Configuration Reference](./kms/configuration.md) in depth
|
||||
4. Configure monitoring and logging
|
||||
|
||||
### 👨🔧 运维工程师
|
||||
### 👨🔧 Operations Engineers
|
||||
|
||||
1. 熟悉 [HTTP API 接口](./kms/http-api.md) 进行日常管理
|
||||
2. 掌握 [故障排除](./kms/troubleshooting.md) 技能
|
||||
3. 了解 [安全指南](./kms/security.md) 要求
|
||||
4. 建立运维流程
|
||||
1. Become familiar with the [HTTP API Reference](./kms/http-api.md) for day-to-day work
|
||||
2. Master the [Troubleshooting](./kms/troubleshooting.md) procedures
|
||||
3. Understand the requirements in the [Security Guide](./kms/security.md)
|
||||
4. Establish operational runbooks
|
||||
|
||||
### 🔒 安全架构师
|
||||
### 🔒 Security Architects
|
||||
|
||||
1. 深入学习 [安全指南](./kms/security.md)
|
||||
2. 评估威胁模型和风险
|
||||
3. 制定安全策略
|
||||
1. Dive into the [Security Guide](./kms/security.md)
|
||||
2. Evaluate threat models and risk posture
|
||||
3. Define security policies
|
||||
|
||||
## 🤝 贡献指南
|
||||
## 🤝 Contribution Guide
|
||||
|
||||
我们欢迎社区贡献!
|
||||
We welcome community contributions!
|
||||
|
||||
### 文档贡献
|
||||
### Documentation Contributions
|
||||
|
||||
```bash
|
||||
# 1. Fork 项目
|
||||
# 1. Fork the repository
|
||||
git clone https://github.com/your-username/rustfs.git
|
||||
|
||||
# 2. 创建文档分支
|
||||
# 2. Create a documentation branch
|
||||
git checkout -b docs/improve-kms-guide
|
||||
|
||||
# 3. 编辑文档
|
||||
# 编辑 docs/kms/ 下的 Markdown 文件
|
||||
# 3. Edit the documentation
|
||||
# Update Markdown files under docs/kms/
|
||||
|
||||
# 4. 提交更改
|
||||
# 4. Commit the changes
|
||||
git add docs/
|
||||
git commit -m "docs: improve KMS configuration examples"
|
||||
|
||||
# 5. 创建 Pull Request
|
||||
# 5. Open a Pull Request
|
||||
gh pr create --title "Improve KMS documentation"
|
||||
```
|
||||
|
||||
### 文档规范
|
||||
### Documentation Guidelines
|
||||
|
||||
- 使用清晰的标题和结构
|
||||
- 提供可运行的代码示例
|
||||
- 包含适当的警告和提示
|
||||
- 支持多种使用场景
|
||||
- 保持内容最新
|
||||
- Use clear headings and structure
|
||||
- Provide runnable code examples
|
||||
- Include warnings and tips where appropriate
|
||||
- Support multiple usage scenarios
|
||||
- Keep the content up to date
|
||||
|
||||
## 📞 支持与反馈
|
||||
## 📞 Support & Feedback
|
||||
|
||||
### 获取帮助
|
||||
### Getting Help
|
||||
|
||||
- **GitHub Issues**: https://github.com/rustfs/rustfs/issues
|
||||
- **讨论区**: https://github.com/rustfs/rustfs/discussions
|
||||
- **文档问题**: 在相关文档页面创建 Issue
|
||||
- **安全问题**: security@rustfs.com
|
||||
- **Discussion Forum**: https://github.com/rustfs/rustfs/discussions
|
||||
- **Documentation Questions**: Open an issue on the relevant document
|
||||
- **Security Concerns**: security@rustfs.com
|
||||
|
||||
### 问题报告模板
|
||||
### Issue Reporting Template
|
||||
|
||||
报告问题时请提供:
|
||||
When reporting a problem, please provide:
|
||||
|
||||
```markdown
|
||||
**环境信息**
|
||||
- RustFS 版本: v1.0.0
|
||||
- 操作系统: Ubuntu 20.04
|
||||
- Rust 版本: 1.75.0
|
||||
**Environment**
|
||||
- RustFS version: v1.0.0
|
||||
- Operating system: Ubuntu 20.04
|
||||
- Rust version: 1.75.0
|
||||
|
||||
**问题描述**
|
||||
简要描述遇到的问题...
|
||||
**Issue Description**
|
||||
Summarize the problem you encountered...
|
||||
|
||||
**重现步骤**
|
||||
1. 步骤一
|
||||
2. 步骤二
|
||||
3. 步骤三
|
||||
**Reproduction Steps**
|
||||
1. Step one
|
||||
2. Step two
|
||||
3. Step three
|
||||
|
||||
**期望行为**
|
||||
描述期望的正确行为...
|
||||
**Expected Behavior**
|
||||
Describe what you expected to happen...
|
||||
|
||||
**实际行为**
|
||||
描述实际发生的情况...
|
||||
**Actual Behavior**
|
||||
Describe what actually happened...
|
||||
|
||||
**相关日志**
|
||||
**Relevant Logs**
|
||||
```bash
|
||||
# 粘贴相关日志
|
||||
# Paste relevant log excerpts
|
||||
```
|
||||
|
||||
**附加信息**
|
||||
其他可能有用的信息...
|
||||
**Additional Information**
|
||||
Any other details that may help...
|
||||
```
|
||||
|
||||
## 📈 版本历史
|
||||
## 📈 Release History
|
||||
|
||||
| 版本 | 发布日期 | 主要特性 |
|
||||
| Version | Release Date | Highlights |
|
||||
|------|----------|----------|
|
||||
| v1.0.0 | 2024-01-15 | 🎉 首个正式版本,完整 KMS 功能 |
|
||||
| v0.9.0 | 2024-01-01 | 🔐 KMS 系统重构,性能优化 |
|
||||
| v0.8.0 | 2023-12-15 | ⚡ 流式加密,1MB 块大小优化 |
|
||||
| v1.0.0 | 2024-01-15 | 🎉 First official release with full KMS functionality |
|
||||
| v0.9.0 | 2024-01-01 | 🔐 KMS system refactor with performance optimizations |
|
||||
| v0.8.0 | 2023-12-15 | ⚡ Streaming encryption with 1 MB block size tuning |
|
||||
|
||||
## 🗺️ 开发路线图
|
||||
## 🗺️ Roadmap
|
||||
|
||||
### 即将发布 (v1.1.0)
|
||||
### Coming Soon (v1.1.0)
|
||||
|
||||
- [ ] 密钥自动轮转
|
||||
- [ ] HSM 集成支持
|
||||
- [ ] Web UI 管理界面
|
||||
- [ ] 更多合规性支持 (SOC2, HIPAA)
|
||||
- [ ] Automatic key rotation
|
||||
- [ ] HSM integration support
|
||||
- [ ] Web UI management console
|
||||
- [ ] Additional compliance support (SOC2, HIPAA)
|
||||
|
||||
### 长期规划
|
||||
### Long-Term Plans
|
||||
|
||||
- [ ] 多租户密钥隔离
|
||||
- [ ] 密钥导入/导出工具
|
||||
- [ ] 性能基准测试套件
|
||||
- [ ] Multi-tenant key isolation
|
||||
- [ ] Key import/export tooling
|
||||
- [ ] Performance benchmarking suite
|
||||
- [ ] Kubernetes Operator
|
||||
|
||||
## 📋 文档反馈
|
||||
## 📋 Documentation Feedback
|
||||
|
||||
帮助我们改进文档!
|
||||
Help us improve the documentation!
|
||||
|
||||
**这些文档对您有帮助吗?**
|
||||
- 👍 很有帮助
|
||||
- 👌 基本满意
|
||||
- 👎 需要改进
|
||||
**Was this documentation helpful?**
|
||||
- 👍 Very helpful
|
||||
- 👌 Mostly satisfied
|
||||
- 👎 Needs improvement
|
||||
|
||||
**改进建议**:
|
||||
请在 GitHub Issues 中提出具体的改进建议。
|
||||
**Suggestions for improvement:**
|
||||
Share specific ideas via GitHub Issues.
|
||||
|
||||
---
|
||||
|
||||
**最后更新**: 2024-01-15
|
||||
**文档版本**: v1.0.0
|
||||
**Last Updated**: 2024-01-15
|
||||
**Documentation Version**: v1.0.0
|
||||
|
||||
*感谢使用 RustFS!我们致力于为您提供最好的分布式文件系统解决方案。*
|
||||
*Thank you for using RustFS! We are committed to delivering the best distributed file system solution.*
|
||||
|
||||
File diff suppressed because it is too large
Load Diff
Reference in New Issue
Block a user