Compare commits

..

12 Commits

Author SHA1 Message Date
安正超
e9c9a2d1f2 fix: simplify Docker entrypoint following efficient user switching pattern (#421)
* fix: simplify Docker entrypoint following efficient user switching pattern

- Remove ALL file permission modifications (no chown at all)
- Use chroot --userspec or gosu to switch user context
- Extremely simple and fast implementation
- Zero filesystem modifications for permissions

Fixes #388

* Update entrypoint.sh

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update entrypoint.sh

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update entrypoint.sh

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* wip

* wip

* wip

---------

Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2025-08-19 22:58:54 +08:00
0xdx2
3ebab98d2d feat: include user-defined metadata in S3 response (#431) 2025-08-19 22:09:50 +08:00
weisd
10c949af62 fix:make bucket exists (#428) 2025-08-19 16:14:59 +08:00
reigadegr
4a3325276d fix(ecstore): add async-recursion to resolve nightly trait solver reg… (#415)
* fix(ecstore): add async-recursion to resolve nightly trait solver regression

The newest nightly compiler switched to the new trait solver, which
currently rejects async recursive functions that were previously accepted.
This causes the following compilation failures:

- `LocalDisk::delete_file()`
- `LocalDisk::scan_dir()`

Add `async-recursion` as a workspace dependency and annotate both functions with `#[async_recursion]` so that the crate compiles cleanly with the latest nightly and will continue to build once the new solver lands in stable.

Signed-off-by: reigadegr <2722688642@qq.com>

* fix: resolve duplicate bound error in scan_dir function

Replaced inline trait bounds with where clause to avoid duplication caused by macro expansion.

Signed-off-by: reigadegr <2722688642@qq.com>

---------

Signed-off-by: reigadegr <2722688642@qq.com>
Co-authored-by: 安正超 <anzhengchao@gmail.com>
2025-08-18 20:58:05 +08:00
majinghe
c5f6c66f72 feat: extend rustfs mcp with bucket creation and deletion (#416)
* feat: extend rustfs mcp with bucket creation and deletion

* update file to fix pipeline error

* change variable name to fix pipeline error
2025-08-18 09:06:55 +08:00
shiro.lee
c7c149975b fix: the automatic logout issue and user list display failure on Windows systems (#353) (#343) (#403)
Co-authored-by: 安正超 <anzhengchao@gmail.com>
2025-08-14 00:20:27 +08:00
安正超
d552210b59 feat: translate chinese to english (#402)
* Checkpoint before follow-up message

Co-authored-by: anzhengchao <anzhengchao@gmail.com>

* Translate project documentation and comments from Chinese to English

Co-authored-by: anzhengchao <anzhengchao@gmail.com>

* Fix typo: "unparseable" to "unparsable" in version test comment

Co-authored-by: anzhengchao <anzhengchao@gmail.com>

* Refactor compression test code with minor syntax improvements

Co-authored-by: anzhengchao <anzhengchao@gmail.com>

---------

Co-authored-by: Cursor Agent <cursoragent@cursor.com>
2025-08-14 00:19:01 +08:00
安正超
581607da6a feat: optimize AI rules with unified .rules.md (#401)
* feat: optimize AI rules with unified .rules.md and entry points

- Create .rules.md as the central AI coding rules file
- Add .copilot-rules.md as GitHub Copilot entry point
- Add CLAUDE.md as Claude AI entry point
- Incorporate principles from rustfs.com project
- Add three critical rules:
  1. Use English for all code comments and documentation
  2. Clean up temporary scripts after use
  3. Only make confident modifications

* Update CLAUDE.md

---------

Co-authored-by: Cursor Agent <cursoragent@cursor.com>
2025-08-14 00:18:09 +08:00
安正超
e95107f7d6 fix: separate RELEASE tag and VERSION in Docker build (#399)
- RELEASE: GitHub release tag without 'v' prefix (e.g., 1.0.0-alpha.42)
- VERSION: filename version with 'v' prefix (e.g., v1.0.0-alpha.42)
- Download URL uses RELEASE for path, VERSION for filename
- Fixes incorrect URL generation that was adding extra 'v' prefix

Co-authored-by: Cursor Agent <cursoragent@cursor.com>
2025-08-13 22:49:48 +08:00
安正超
a693cb52f3 feat: change Docker build to download from GitHub releases instead of dl.rustfs.com (#398)
- Modified Dockerfile to download pre-built binaries from GitHub releases
- For latest releases, use GitHub API to find the correct download URL
- For specific versions, construct the GitHub release URL directly
- Updated docker-buildx.sh script messages to reflect new download source
- This change addresses security concerns about potential tampering with binaries from dl.rustfs.com

Co-authored-by: Cursor Agent <cursoragent@cursor.com>
2025-08-13 22:00:41 +08:00
houseme
2c7366038e modify protobuf version from to 2025-08-13 01:01:50 +08:00
houseme
1cc6dfde87 modify protobuf version from 31.1 to 31.0 2025-08-13 00:58:22 +08:00
41 changed files with 1544 additions and 638 deletions

58
.copilot-rules.md Normal file
View File

@@ -0,0 +1,58 @@
# GitHub Copilot Rules for RustFS Project
## Core Rules Reference
This project follows the comprehensive AI coding rules defined in `.rules.md`. Please refer to that file for the complete set of development guidelines, coding standards, and best practices.
## Copilot-Specific Configuration
When using GitHub Copilot for this project, ensure you:
1. **Review the unified rules**: Always check `.rules.md` for the latest project guidelines
2. **Follow branch protection**: Never attempt to commit directly to main/master branch
3. **Use English**: All code comments, documentation, and variable names must be in English
4. **Clean code practices**: Only make modifications you're confident about
5. **Test thoroughly**: Ensure all changes pass formatting, linting, and testing requirements
## Quick Reference
### Critical Rules
- 🚫 **NEVER commit directly to main/master branch**
-**ALWAYS work on feature branches**
- 📝 **ALWAYS use English for code and documentation**
- 🧹 **ALWAYS clean up temporary files after use**
- 🎯 **ONLY make confident, necessary modifications**
### Pre-commit Checklist
```bash
# Before committing, always run:
cargo fmt --all
cargo clippy --all-targets --all-features -- -D warnings
cargo check --all-targets
cargo test
```
### Branch Workflow
```bash
git checkout main
git pull origin main
git checkout -b feat/your-feature-name
# Make your changes
git add .
git commit -m "feat: your feature description"
git push origin feat/your-feature-name
gh pr create
```
## Important Notes
- This file serves as an entry point for GitHub Copilot
- All detailed rules and guidelines are maintained in `.rules.md`
- Updates to coding standards should be made in `.rules.md` to ensure consistency across all AI tools
- When in doubt, always refer to `.rules.md` for authoritative guidance
## See Also
- [.rules.md](./.rules.md) - Complete AI coding rules and guidelines
- [CONTRIBUTING.md](./CONTRIBUTING.md) - Contribution guidelines
- [README.md](./README.md) - Project overview and setup instructions

702
.rules.md Normal file
View File

@@ -0,0 +1,702 @@
# RustFS Project AI Coding Rules
## 🚨🚨🚨 CRITICAL DEVELOPMENT RULES - ZERO TOLERANCE 🚨🚨🚨
### ⛔️ ABSOLUTE PROHIBITION: NEVER COMMIT DIRECTLY TO MASTER/MAIN BRANCH ⛔️
**🔥 THIS IS THE MOST CRITICAL RULE - VIOLATION WILL RESULT IN IMMEDIATE REVERSAL 🔥**
- **🚫 ZERO DIRECT COMMITS TO MAIN/MASTER BRANCH - ABSOLUTELY FORBIDDEN**
- **🚫 ANY DIRECT COMMIT TO MAIN BRANCH MUST BE IMMEDIATELY REVERTED**
- **🚫 NO EXCEPTIONS FOR HOTFIXES, EMERGENCIES, OR URGENT CHANGES**
- **🚫 NO EXCEPTIONS FOR SMALL CHANGES, TYPOS, OR DOCUMENTATION UPDATES**
- **🚫 NO EXCEPTIONS FOR ANYONE - MAINTAINERS, CONTRIBUTORS, OR ADMINS**
### 📋 MANDATORY WORKFLOW - STRICTLY ENFORCED
**EVERY SINGLE CHANGE MUST FOLLOW THIS WORKFLOW:**
1. **Check current branch**: `git branch` (MUST NOT be on main/master)
2. **Switch to main**: `git checkout main`
3. **Pull latest**: `git pull origin main`
4. **Create feature branch**: `git checkout -b feat/your-feature-name`
5. **Make changes ONLY on feature branch**
6. **Test thoroughly before committing**
7. **Commit and push to feature branch**: `git push origin feat/your-feature-name`
8. **Create Pull Request**: Use `gh pr create` (MANDATORY)
9. **Wait for PR approval**: NO self-merging allowed
10. **Merge through GitHub interface**: ONLY after approval
### 🔒 ENFORCEMENT MECHANISMS
- **Branch protection rules**: Main branch is protected
- **Pre-commit hooks**: Will block direct commits to main
- **CI/CD checks**: All PRs must pass before merging
- **Code review requirement**: At least one approval needed
- **Automated reversal**: Direct commits to main will be automatically reverted
## 🎯 Core AI Development Principles
### Five Execution Steps
#### 1. Task Analysis and Planning
- **Clear Objectives**: Deeply understand task requirements and expected results before starting coding
- **Plan Development**: List specific files, components, and functions that need modification, explaining the reasons for changes
- **Risk Assessment**: Evaluate the impact of changes on existing functionality, develop rollback plans
#### 2. Precise Code Location
- **File Identification**: Determine specific files and line numbers that need modification
- **Impact Analysis**: Avoid modifying irrelevant files, clearly state the reason for each file modification
- **Minimization Principle**: Unless explicitly required by the task, do not create new abstraction layers or refactor existing code
#### 3. Minimal Code Changes
- **Focus on Core**: Only write code directly required by the task
- **Avoid Redundancy**: Do not add unnecessary logs, comments, tests, or error handling
- **Isolation**: Ensure new code does not interfere with existing functionality, maintain code independence
#### 4. Strict Code Review
- **Correctness Check**: Verify the correctness and completeness of code logic
- **Style Consistency**: Ensure code conforms to established project coding style
- **Side Effect Assessment**: Evaluate the impact of changes on downstream systems
#### 5. Clear Delivery Documentation
- **Change Summary**: Detailed explanation of all modifications and reasons
- **File List**: List all modified files and their specific changes
- **Risk Statement**: Mark any assumptions or potential risk points
### Core Principles
- **🎯 Precise Execution**: Strictly follow task requirements, no arbitrary innovation
- **⚡ Efficient Development**: Avoid over-design, only do necessary work
- **🛡️ Safe and Reliable**: Always follow development processes, ensure code quality and system stability
- **🔒 Cautious Modification**: Only modify when clearly knowing what needs to be changed and having confidence
### Additional AI Behavior Rules
1. **Use English for all code comments and documentation** - All comments, variable names, function names, documentation, and user-facing text in code should be in English
2. **Clean up temporary scripts after use** - Any temporary scripts, test files, or helper files created during AI work should be removed after task completion
3. **Only make confident modifications** - Do not make speculative changes or "convenient" modifications outside the task scope. If uncertain about a change, ask for clarification rather than guessing
## Project Overview
RustFS is a high-performance distributed object storage system written in Rust, compatible with S3 API. The project adopts a modular architecture, supporting erasure coding storage, multi-tenant management, observability, and other enterprise-level features.
## Core Architecture Principles
### 1. Modular Design
- Project uses Cargo workspace structure, containing multiple independent crates
- Core modules: `rustfs` (main service), `ecstore` (erasure coding storage), `common` (shared components)
- Functional modules: `iam` (identity management), `madmin` (management interface), `crypto` (encryption), etc.
- Tool modules: `cli` (command line tool), `crates/*` (utility libraries)
### 2. Asynchronous Programming Pattern
- Comprehensive use of `tokio` async runtime
- Prioritize `async/await` syntax
- Use `async-trait` for async methods in traits
- Avoid blocking operations, use `spawn_blocking` when necessary
### 3. Error Handling Strategy
- **Use modular, type-safe error handling with `thiserror`**
- Each module should define its own error type using `thiserror::Error` derive macro
- Support error chains and context information through `#[from]` and `#[source]` attributes
- Use `Result<T>` type aliases for consistency within each module
- Error conversion between modules should use explicit `From` implementations
- Follow the pattern: `pub type Result<T> = core::result::Result<T, Error>`
- Use `#[error("description")]` attributes for clear error messages
- Support error downcasting when needed through `other()` helper methods
- Implement `Clone` for errors when required by the domain logic
## Code Style Guidelines
### 1. Formatting Configuration
```toml
max_width = 130
fn_call_width = 90
single_line_let_else_max_width = 100
```
### 2. **🔧 MANDATORY Code Formatting Rules**
**CRITICAL**: All code must be properly formatted before committing. This project enforces strict formatting standards to maintain code consistency and readability.
#### Pre-commit Requirements (MANDATORY)
Before every commit, you **MUST**:
1. **Format your code**:
```bash
cargo fmt --all
```
2. **Verify formatting**:
```bash
cargo fmt --all --check
```
3. **Pass clippy checks**:
```bash
cargo clippy --all-targets --all-features -- -D warnings
```
4. **Ensure compilation**:
```bash
cargo check --all-targets
```
#### Quick Commands
Use these convenient Makefile targets for common tasks:
```bash
# Format all code
make fmt
# Check if code is properly formatted
make fmt-check
# Run clippy checks
make clippy
# Run compilation check
make check
# Run tests
make test
# Run all pre-commit checks (format + clippy + check + test)
make pre-commit
# Setup git hooks (one-time setup)
make setup-hooks
```
### 3. Naming Conventions
- Use `snake_case` for functions, variables, modules
- Use `PascalCase` for types, traits, enums
- Constants use `SCREAMING_SNAKE_CASE`
- Global variables prefix `GLOBAL_`, e.g., `GLOBAL_Endpoints`
- Use meaningful and descriptive names for variables, functions, and methods
- Avoid meaningless names like `temp`, `data`, `foo`, `bar`, `test123`
- Choose names that clearly express the purpose and intent
### 4. Type Declaration Guidelines
- **Prefer type inference over explicit type declarations** when the type is obvious from context
- Let the Rust compiler infer types whenever possible to reduce verbosity and improve maintainability
- Only specify types explicitly when:
- The type cannot be inferred by the compiler
- Explicit typing improves code clarity and readability
- Required for API boundaries (function signatures, public struct fields)
- Needed to resolve ambiguity between multiple possible types
### 5. Documentation Comments
- Public APIs must have documentation comments
- Use `///` for documentation comments
- Complex functions add `# Examples` and `# Parameters` descriptions
- Error cases use `# Errors` descriptions
- Always use English for all comments and documentation
- Avoid meaningless comments like "debug 111" or placeholder text
### 6. Import Guidelines
- Standard library imports first
- Third-party crate imports in the middle
- Project internal imports last
- Group `use` statements with blank lines between groups
## Asynchronous Programming Guidelines
### 1. Trait Definition
```rust
#[async_trait::async_trait]
pub trait StorageAPI: Send + Sync {
async fn get_object(&self, bucket: &str, object: &str) -> Result<ObjectInfo>;
}
```
### 2. Error Handling
```rust
// Use ? operator to propagate errors
async fn example_function() -> Result<()> {
let data = read_file("path").await?;
process_data(data).await?;
Ok(())
}
```
### 3. Concurrency Control
- Use `Arc` and `Mutex`/`RwLock` for shared state management
- Prioritize async locks from `tokio::sync`
- Avoid holding locks for long periods
## Logging and Tracing Guidelines
### 1. Tracing Usage
```rust
#[tracing::instrument(skip(self, data))]
async fn process_data(&self, data: &[u8]) -> Result<()> {
info!("Processing {} bytes", data.len());
// Implementation logic
}
```
### 2. Log Levels
- `error!`: System errors requiring immediate attention
- `warn!`: Warning information that may affect functionality
- `info!`: Important business information
- `debug!`: Debug information for development use
- `trace!`: Detailed execution paths
### 3. Structured Logging
```rust
info!(
counter.rustfs_api_requests_total = 1_u64,
key_request_method = %request.method(),
key_request_uri_path = %request.uri().path(),
"API request processed"
);
```
## Error Handling Guidelines
### 1. Error Type Definition
```rust
// Use thiserror for module-specific error types
#[derive(thiserror::Error, Debug)]
pub enum MyError {
#[error("IO error: {0}")]
Io(#[from] std::io::Error),
#[error("Storage error: {0}")]
Storage(#[from] ecstore::error::StorageError),
#[error("Custom error: {message}")]
Custom { message: String },
#[error("File not found: {path}")]
FileNotFound { path: String },
#[error("Invalid configuration: {0}")]
InvalidConfig(String),
}
// Provide Result type alias for the module
pub type Result<T> = core::result::Result<T, MyError>;
```
### 2. Error Helper Methods
```rust
impl MyError {
/// Create error from any compatible error type
pub fn other<E>(error: E) -> Self
where
E: Into<Box<dyn std::error::Error + Send + Sync>>,
{
MyError::Io(std::io::Error::other(error))
}
}
```
### 3. Error Context and Propagation
```rust
// Use ? operator for clean error propagation
async fn example_function() -> Result<()> {
let data = read_file("path").await?;
process_data(data).await?;
Ok(())
}
// Add context to errors
fn process_with_context(path: &str) -> Result<()> {
std::fs::read(path)
.map_err(|e| MyError::Custom {
message: format!("Failed to read {}: {}", path, e)
})?;
Ok(())
}
```
## Performance Optimization Guidelines
### 1. Memory Management
- Use `Bytes` instead of `Vec<u8>` for zero-copy operations
- Avoid unnecessary cloning, use reference passing
- Use `Arc` for sharing large objects
### 2. Concurrency Optimization
```rust
// Use join_all for concurrent operations
let futures = disks.iter().map(|disk| disk.operation());
let results = join_all(futures).await;
```
### 3. Caching Strategy
- Use `LazyLock` for global caching
- Implement LRU cache to avoid memory leaks
## Testing Guidelines
### 1. Unit Tests
```rust
#[cfg(test)]
mod tests {
use super::*;
use test_case::test_case;
#[tokio::test]
async fn test_async_function() {
let result = async_function().await;
assert!(result.is_ok());
}
#[test_case("input1", "expected1")]
#[test_case("input2", "expected2")]
fn test_with_cases(input: &str, expected: &str) {
assert_eq!(function(input), expected);
}
}
```
### 2. Integration Tests
- Use `e2e_test` module for end-to-end testing
- Simulate real storage environments
### 3. Test Quality Standards
- Write meaningful test cases that verify actual functionality
- Avoid placeholder or debug content like "debug 111", "test test", etc.
- Use descriptive test names that clearly indicate what is being tested
- Each test should have a clear purpose and verify specific behavior
- Test data should be realistic and representative of actual use cases
## Cross-Platform Compatibility Guidelines
### 1. CPU Architecture Compatibility
- **Always consider multi-platform and different CPU architecture compatibility** when writing code
- Support major architectures: x86_64, aarch64 (ARM64), and other target platforms
- Use conditional compilation for architecture-specific code:
```rust
#[cfg(target_arch = "x86_64")]
fn optimized_x86_64_function() { /* x86_64 specific implementation */ }
#[cfg(target_arch = "aarch64")]
fn optimized_aarch64_function() { /* ARM64 specific implementation */ }
#[cfg(not(any(target_arch = "x86_64", target_arch = "aarch64")))]
fn generic_function() { /* Generic fallback implementation */ }
```
### 2. Platform-Specific Dependencies
- Use feature flags for platform-specific dependencies
- Provide fallback implementations for unsupported platforms
- Test on multiple architectures in CI/CD pipeline
### 3. Endianness Considerations
- Use explicit byte order conversion when dealing with binary data
- Prefer `to_le_bytes()`, `from_le_bytes()` for consistent little-endian format
- Use `byteorder` crate for complex binary format handling
### 4. SIMD and Performance Optimizations
- Use portable SIMD libraries like `wide` or `packed_simd`
- Provide fallback implementations for non-SIMD architectures
- Use runtime feature detection when appropriate
## Security Guidelines
### 1. Memory Safety
- Disable `unsafe` code (workspace.lints.rust.unsafe_code = "deny")
- Use `rustls` instead of `openssl`
### 2. Authentication and Authorization
```rust
// Use IAM system for permission checks
let identity = iam.authenticate(&access_key, &secret_key).await?;
iam.authorize(&identity, &action, &resource).await?;
```
## Configuration Management Guidelines
### 1. Environment Variables
- Use `RUSTFS_` prefix
- Support both configuration files and environment variables
- Provide reasonable default values
### 2. Configuration Structure
```rust
#[derive(Debug, Deserialize, Clone)]
pub struct Config {
pub address: String,
pub volumes: String,
#[serde(default)]
pub console_enable: bool,
}
```
## Dependency Management Guidelines
### 1. Workspace Dependencies
- Manage versions uniformly at workspace level
- Use `workspace = true` to inherit configuration
### 2. Feature Flags
```rust
[features]
default = ["file"]
gpu = ["dep:nvml-wrapper"]
kafka = ["dep:rdkafka"]
```
## Deployment and Operations Guidelines
### 1. Containerization
- Provide Dockerfile and docker-compose configuration
- Support multi-stage builds to optimize image size
### 2. Observability
- Integrate OpenTelemetry for distributed tracing
- Support Prometheus metrics collection
- Provide Grafana dashboards
### 3. Health Checks
```rust
// Implement health check endpoint
async fn health_check() -> Result<HealthStatus> {
// Check component status
}
```
## Code Review Checklist
### 1. **Code Formatting and Quality (MANDATORY)**
- [ ] **Code is properly formatted** (`cargo fmt --all --check` passes)
- [ ] **All clippy warnings are resolved** (`cargo clippy --all-targets --all-features -- -D warnings` passes)
- [ ] **Code compiles successfully** (`cargo check --all-targets` passes)
- [ ] **Pre-commit hooks are working** and all checks pass
- [ ] **No formatting-related changes** mixed with functional changes (separate commits)
### 2. Functionality
- [ ] Are all error cases properly handled?
- [ ] Is there appropriate logging?
- [ ] Is there necessary test coverage?
### 3. Performance
- [ ] Are unnecessary memory allocations avoided?
- [ ] Are async operations used correctly?
- [ ] Are there potential deadlock risks?
### 4. Security
- [ ] Are input parameters properly validated?
- [ ] Are there appropriate permission checks?
- [ ] Is information leakage avoided?
### 5. Cross-Platform Compatibility
- [ ] Does the code work on different CPU architectures (x86_64, aarch64)?
- [ ] Are platform-specific features properly gated with conditional compilation?
- [ ] Is byte order handling correct for binary data?
- [ ] Are there appropriate fallback implementations for unsupported platforms?
### 6. Code Commits and Documentation
- [ ] Does it comply with [Conventional Commits](https://www.conventionalcommits.org/en/v1.0.0/)?
- [ ] Are commit messages concise and under 72 characters for the title line?
- [ ] Commit titles should be concise and in English, avoid Chinese
- [ ] Is PR description provided in copyable markdown format for easy copying?
## Common Patterns and Best Practices
### 1. Resource Management
```rust
// Use RAII pattern for resource management
pub struct ResourceGuard {
resource: Resource,
}
impl Drop for ResourceGuard {
fn drop(&mut self) {
// Clean up resources
}
}
```
### 2. Dependency Injection
```rust
// Use dependency injection pattern
pub struct Service {
config: Arc<Config>,
storage: Arc<dyn StorageAPI>,
}
```
### 3. Graceful Shutdown
```rust
// Implement graceful shutdown
async fn shutdown_gracefully(shutdown_rx: &mut Receiver<()>) {
tokio::select! {
_ = shutdown_rx.recv() => {
info!("Received shutdown signal");
// Perform cleanup operations
}
_ = tokio::time::sleep(SHUTDOWN_TIMEOUT) => {
warn!("Shutdown timeout reached");
}
}
}
```
## Domain-Specific Guidelines
### 1. Storage Operations
- All storage operations must support erasure coding
- Implement read/write quorum mechanisms
- Support data integrity verification
### 2. Network Communication
- Use gRPC for internal service communication
- HTTP/HTTPS support for S3-compatible API
- Implement connection pooling and retry mechanisms
### 3. Metadata Management
- Use FlatBuffers for serialization
- Support version control and migration
- Implement metadata caching
## Branch Management and Development Workflow
### Branch Management
- **🚨 CRITICAL: NEVER modify code directly on main or master branch - THIS IS ABSOLUTELY FORBIDDEN 🚨**
- **⚠️ ANY DIRECT COMMITS TO MASTER/MAIN WILL BE REJECTED AND MUST BE REVERTED IMMEDIATELY ⚠️**
- **🔒 ALL CHANGES MUST GO THROUGH PULL REQUESTS - NO DIRECT COMMITS TO MAIN UNDER ANY CIRCUMSTANCES 🔒**
- **Always work on feature branches - NO EXCEPTIONS**
- Always check the .rules.md file before starting to ensure you understand the project guidelines
- **MANDATORY workflow for ALL changes:**
1. `git checkout main` (switch to main branch)
2. `git pull` (get latest changes)
3. `git checkout -b feat/your-feature-name` (create and switch to feature branch)
4. Make your changes ONLY on the feature branch
5. Test thoroughly before committing
6. Commit and push to the feature branch
7. **Create a pull request for code review - THIS IS THE ONLY WAY TO MERGE TO MAIN**
8. **Wait for PR approval before merging - NEVER merge your own PRs without review**
- Use descriptive branch names following the pattern: `feat/feature-name`, `fix/issue-name`, `refactor/component-name`, etc.
- **Double-check current branch before ANY commit: `git branch` to ensure you're NOT on main/master**
- **Pull Request Requirements:**
- All changes must be submitted via PR regardless of size or urgency
- PRs must include comprehensive description and testing information
- PRs must pass all CI/CD checks before merging
- PRs require at least one approval from code reviewers
- Even hotfixes and emergency changes must go through PR process
- **Enforcement:**
- Main branch should be protected with branch protection rules
- Direct pushes to main should be blocked by repository settings
- Any accidental direct commits to main must be immediately reverted via PR
### Development Workflow
## 🎯 **Core Development Principles**
- **🔴 Every change must be precise - don't modify unless you're confident**
- Carefully analyze code logic and ensure complete understanding before making changes
- When uncertain, prefer asking users or consulting documentation over blind modifications
- Use small iterative steps, modify only necessary parts at a time
- Evaluate impact scope before changes to ensure no new issues are introduced
- **🚀 GitHub PR creation prioritizes gh command usage**
- Prefer using `gh pr create` command to create Pull Requests
- Avoid having users manually create PRs through web interface
- Provide clear and professional PR titles and descriptions
- Using `gh` commands ensures better integration and automation
## 📝 **Code Quality Requirements**
- Use English for all code comments, documentation, and variable names
- Write meaningful and descriptive names for variables, functions, and methods
- Avoid meaningless test content like "debug 111" or placeholder values
- Before each change, carefully read the existing code to ensure you understand the code structure and implementation, do not break existing logic implementation, do not introduce new issues
- Ensure each change provides sufficient test cases to guarantee code correctness
- Do not arbitrarily modify numbers and constants in test cases, carefully analyze their meaning to ensure test case correctness
- When writing or modifying tests, check existing test cases to ensure they have scientific naming and rigorous logic testing, if not compliant, modify test cases to ensure scientific and rigorous testing
- **Before committing any changes, run `cargo clippy --all-targets --all-features -- -D warnings` to ensure all code passes Clippy checks**
- After each development completion, first git add . then git commit -m "feat: feature description" or "fix: issue description", ensure compliance with [Conventional Commits](https://www.conventionalcommits.org/en/v1.0.0/)
- **Keep commit messages concise and under 72 characters** for the title line, use body for detailed explanations if needed
- After each development completion, first git push to remote repository
- After each change completion, summarize the changes, do not create summary files, provide a brief change description, ensure compliance with [Conventional Commits](https://www.conventionalcommits.org/en/v1.0.0/)
- Provide change descriptions needed for PR in the conversation, ensure compliance with [Conventional Commits](https://www.conventionalcommits.org/en/v1.0.0/)
- **Always provide PR descriptions in English** after completing any changes, including:
- Clear and concise title following Conventional Commits format
- Detailed description of what was changed and why
- List of key changes and improvements
- Any breaking changes or migration notes if applicable
- Testing information and verification steps
- **Provide PR descriptions in copyable markdown format** enclosed in code blocks for easy one-click copying
## 🚫 AI Documentation Generation Restrictions
### Forbidden Summary Documents
- **Strictly forbidden to create any form of AI-generated summary documents**
- **Do not create documents containing large amounts of emoji, detailed formatting tables and typical AI style**
- **Do not generate the following types of documents in the project:**
- Benchmark summary documents (BENCHMARK*.md)
- Implementation comparison analysis documents (IMPLEMENTATION_COMPARISON*.md)
- Performance analysis report documents
- Architecture summary documents
- Feature comparison documents
- Any documents with large amounts of emoji and formatted content
- **If documentation is needed, only create when explicitly requested by the user, and maintain a concise and practical style**
- **Documentation should focus on actually needed information, avoiding excessive formatting and decorative content**
- **Any discovered AI-generated summary documents should be immediately deleted**
### Allowed Documentation Types
- README.md (project introduction, keep concise)
- Technical documentation (only create when explicitly needed)
- User manual (only create when explicitly needed)
- API documentation (generated from code)
- Changelog (CHANGELOG.md)
These rules should serve as guiding principles when developing the RustFS project, ensuring code quality, performance, and maintainability.

68
CLAUDE.md Normal file
View File

@@ -0,0 +1,68 @@
# Claude AI Rules for RustFS Project
## Core Rules Reference
This project follows the comprehensive AI coding rules defined in `.rules.md`. Please refer to that file for the complete set of development guidelines, coding standards, and best practices.
## Claude-Specific Configuration
When using Claude for this project, ensure you:
1. **Review the unified rules**: Always check `.rules.md` for the latest project guidelines
2. **Follow branch protection**: Never attempt to commit directly to main/master branch
3. **Use English**: All code comments, documentation, and variable names must be in English
4. **Clean code practices**: Only make modifications you're confident about
5. **Test thoroughly**: Ensure all changes pass formatting, linting, and testing requirements
6. **Clean up after yourself**: Remove any temporary scripts or test files created during the session
## Quick Reference
### Critical Rules
- 🚫 **NEVER commit directly to main/master branch**
-**ALWAYS work on feature branches**
- 📝 **ALWAYS use English for code and documentation**
- 🧹 **ALWAYS clean up temporary files after use**
- 🎯 **ONLY make confident, necessary modifications**
### Pre-commit Checklist
```bash
# Before committing, always run:
cargo fmt --all
cargo clippy --all-targets --all-features -- -D warnings
cargo check --all-targets
cargo test
```
### Branch Workflow
```bash
git checkout main
git pull origin main
git checkout -b feat/your-feature-name
# Make your changes
git add .
git commit -m "feat: your feature description"
git push origin feat/your-feature-name
gh pr create
```
## Claude-Specific Best Practices
1. **Task Analysis**: Always thoroughly analyze the task before starting implementation
2. **Minimal Changes**: Make only the necessary changes to accomplish the task
3. **Clear Communication**: Provide clear explanations of changes and their rationale
4. **Error Prevention**: Verify code correctness before suggesting changes
5. **Documentation**: Ensure all code changes are properly documented in English
## Important Notes
- This file serves as an entry point for Claude AI
- All detailed rules and guidelines are maintained in `.rules.md`
- Updates to coding standards should be made in `.rules.md` to ensure consistency across all AI tools
- When in doubt, always refer to `.rules.md` for authoritative guidance
- Claude should prioritize code quality, safety, and maintainability over speed
## See Also
- [.rules.md](./.rules.md) - Complete AI coding rules and guidelines
- [CONTRIBUTING.md](./CONTRIBUTING.md) - Contribution guidelines
- [README.md](./README.md) - Project overview and setup instructions

1
Cargo.lock generated
View File

@@ -8205,6 +8205,7 @@ name = "rustfs-ecstore"
version = "0.0.5"
dependencies = [
"async-channel",
"async-recursion",
"async-trait",
"aws-sdk-s3",
"base64 0.22.1",

View File

@@ -1,48 +1,56 @@
# Multi-stage build for RustFS production image
# Build stage: Download and extract RustFS binary
# -------------------
# Build stage
# -------------------
FROM alpine:3.22 AS build
# Build arguments for platform and release
ARG TARGETARCH
ARG RELEASE=latest
# Install minimal dependencies for downloading and extracting
RUN apk add --no-cache ca-certificates curl unzip
# Create build directory
WORKDIR /build
# Set architecture-specific variables
RUN if [ "$TARGETARCH" = "amd64" ]; then \
echo "x86_64-musl" > /tmp/arch; \
elif [ "$TARGETARCH" = "arm64" ]; then \
echo "aarch64-musl" > /tmp/arch; \
# Download and extract release package matching current TARGETARCH
# - If RELEASE=latest: take first tag_name from /releases (may include pre-releases)
# - Otherwise use specified tag (e.g. v0.1.2)
RUN set -eux; \
case "$TARGETARCH" in \
amd64) ARCH_SUBSTR="x86_64-musl" ;; \
arm64) ARCH_SUBSTR="aarch64-musl" ;; \
*) echo "Unsupported TARGETARCH=$TARGETARCH" >&2; exit 1 ;; \
esac; \
if [ "$RELEASE" = "latest" ]; then \
TAG="$(curl -fsSL https://api.github.com/repos/rustfs/rustfs/releases \
| grep -o '"tag_name": "[^"]*"' | cut -d'"' -f4 | head -n 1)"; \
else \
echo "unsupported" > /tmp/arch; \
fi
RUN ARCH=$(cat /tmp/arch) && \
if [ "$ARCH" = "unsupported" ]; then \
echo "Unsupported architecture: $TARGETARCH" && exit 1; \
fi && \
if [ "${RELEASE}" = "latest" ]; then \
VERSION="latest"; \
else \
VERSION="v${RELEASE#v}"; \
fi && \
BASE_URL="https://dl.rustfs.com/artifacts/rustfs/release" && \
PACKAGE_NAME="rustfs-linux-${ARCH}-${VERSION}.zip" && \
DOWNLOAD_URL="${BASE_URL}/${PACKAGE_NAME}" && \
echo "Downloading ${PACKAGE_NAME} from ${DOWNLOAD_URL}" >&2 && \
curl -f -L "${DOWNLOAD_URL}" -o rustfs.zip && \
unzip rustfs.zip -d /build && \
chmod +x /build/rustfs && \
rm rustfs.zip || { echo "Failed to download or extract ${PACKAGE_NAME}" >&2; exit 1; }
TAG="$RELEASE"; \
fi; \
echo "Using tag: $TAG (arch pattern: $ARCH_SUBSTR)"; \
# Find download URL in assets list for this tag that contains arch substring and ends with .zip
URL="$(curl -fsSL "https://api.github.com/repos/rustfs/rustfs/releases/tags/$TAG" \
| grep -o "\"browser_download_url\": \"[^\"]*${ARCH_SUBSTR}[^\"]*\\.zip\"" \
| cut -d'"' -f4 | head -n 1)"; \
if [ -z "$URL" ]; then echo "Failed to locate release asset for $ARCH_SUBSTR at tag $TAG" >&2; exit 1; fi; \
echo "Downloading: $URL"; \
curl -fL "$URL" -o rustfs.zip; \
unzip -q rustfs.zip -d /build; \
# If binary is not in root directory, try to locate and move from zip to /build/rustfs
if [ ! -x /build/rustfs ]; then \
BIN_PATH="$(unzip -Z -1 rustfs.zip | grep -E '(^|/)rustfs$' | head -n 1 || true)"; \
if [ -n "$BIN_PATH" ]; then \
mkdir -p /build/.tmp && unzip -q rustfs.zip "$BIN_PATH" -d /build/.tmp && \
mv "/build/.tmp/$BIN_PATH" /build/rustfs; \
fi; \
fi; \
[ -x /build/rustfs ] || { echo "rustfs binary not found in asset" >&2; exit 1; }; \
chmod +x /build/rustfs; \
rm -rf rustfs.zip /build/.tmp || true
# Runtime stage: Configure runtime environment
FROM alpine:3.22.1
# Build arguments and labels
# -------------------
# Runtime stage
# -------------------
FROM alpine:3.22
ARG RELEASE=latest
ARG BUILD_DATE
ARG VCS_REF
@@ -50,7 +58,7 @@ ARG VCS_REF
LABEL name="RustFS" \
vendor="RustFS Team" \
maintainer="RustFS Team <dev@rustfs.com>" \
version="${RELEASE}" \
version="v${RELEASE#v}" \
release="${RELEASE}" \
build-date="${BUILD_DATE}" \
vcs-ref="${VCS_REF}" \
@@ -59,43 +67,37 @@ LABEL name="RustFS" \
url="https://rustfs.com" \
license="Apache-2.0"
# Install runtime dependencies
RUN echo "https://dl-cdn.alpinelinux.org/alpine/v3.20/community" >> /etc/apk/repositories && \
apk update && \
apk add --no-cache ca-certificates bash gosu coreutils shadow && \
# Install only runtime requirements: certificates and coreutils (provides chroot --userspec)
RUN apk add --no-cache ca-certificates coreutils && \
addgroup -g 1000 rustfs && \
adduser -u 1000 -G rustfs -s /bin/bash -D rustfs
adduser -u 1000 -G rustfs -s /sbin/nologin -D rustfs
# Copy CA certificates and RustFS binary from build stage
# Copy binary and entry script (ensure fixed entrypoint.sh exists in repository)
COPY --from=build /etc/ssl/certs/ca-certificates.crt /etc/ssl/certs/
COPY --from=build /build/rustfs /usr/bin/rustfs
# Copy entry point script
COPY entrypoint.sh /entrypoint.sh
# Set permissions
RUN chmod +x /usr/bin/rustfs /entrypoint.sh && \
mkdir -p /data /logs && \
chown rustfs:rustfs /data /logs && \
chmod 700 /data /logs
chmod 0750 /data /logs
# Environment variables (credentials should be set via environment or secrets)
ENV RUSTFS_ADDRESS=:9000 \
RUSTFS_ACCESS_KEY=rustfsadmin \
RUSTFS_SECRET_KEY=rustfsadmin \
RUSTFS_CONSOLE_ENABLE=true \
RUSTFS_VOLUMES=/data \
RUST_LOG=warn \
RUSTFS_OBS_LOG_DIRECTORY=/logs \
RUSTFS_SINKS_FILE_PATH=/logs
# Default environment (can be overridden in docker run/compose)
ENV RUSTFS_ADDRESS=":9000" \
RUSTFS_ACCESS_KEY="rustfsadmin" \
RUSTFS_SECRET_KEY="rustfsadmin" \
RUSTFS_CONSOLE_ENABLE="true" \
RUSTFS_VOLUMES="/data" \
RUST_LOG="warn" \
RUSTFS_OBS_LOG_DIRECTORY="/logs" \
RUSTFS_SINKS_FILE_PATH="/logs" \
RUSTFS_USERNAME="rustfs" \
RUSTFS_GROUPNAME="rustfs" \
RUSTFS_UID="1000" \
RUSTFS_GID="1000"
# Expose port
EXPOSE 9000
# Volumes for data and logs
VOLUME ["/data", "/logs"]
# Set entry point
ENTRYPOINT ["/entrypoint.sh"]
CMD ["/usr/bin/rustfs"]

View File

@@ -1,80 +1,88 @@
# syntax=docker/dockerfile:1.6
# Multi-stage Dockerfile for RustFS - LOCAL DEVELOPMENT ONLY
#
# ⚠️ IMPORTANT: This Dockerfile is for local development and testing only.
# ⚠️ It builds RustFS from source code and is NOT used in CI/CD pipelines.
# ⚠️ CI/CD pipeline uses pre-built binaries from Dockerfile instead.
# IMPORTANT: This Dockerfile builds RustFS from source for local development and testing.
# CI/CD uses the production Dockerfile with prebuilt binaries instead.
#
# Usage for local development:
# Example:
# docker build -f Dockerfile.source -t rustfs:dev-local .
# docker run --rm -p 9000:9000 rustfs:dev-local
#
# Supports cross-compilation for amd64 and arm64 architectures
# Supports cross-compilation for amd64 and arm64 via TARGETPLATFORM.
ARG TARGETPLATFORM
ARG BUILDPLATFORM
# -----------------------------
# Build stage
FROM --platform=$BUILDPLATFORM rust:1.88-bookworm AS builder
# -----------------------------
FROM rust:1.88-bookworm AS builder
# Re-declare build arguments after FROM (required for multi-stage builds)
# Re-declare args after FROM
ARG TARGETPLATFORM
ARG BUILDPLATFORM
# Debug: Print platform information
RUN echo "🐳 Build Info: BUILDPLATFORM=$BUILDPLATFORM, TARGETPLATFORM=$TARGETPLATFORM"
# Debug: print platforms
RUN echo "Build info -> BUILDPLATFORM=${BUILDPLATFORM}, TARGETPLATFORM=${TARGETPLATFORM}"
# Install required build dependencies
RUN apt-get update && apt-get install -y \
wget \
git \
# Install build toolchain and headers
# Use distro packages for protoc/flatc to avoid host-arch mismatch
RUN set -eux; \
export DEBIAN_FRONTEND=noninteractive; \
apt-get update; \
apt-get install -y --no-install-recommends \
build-essential \
ca-certificates \
curl \
unzip \
gcc \
git \
pkg-config \
libssl-dev \
lld \
&& rm -rf /var/lib/apt/lists/*
protobuf-compiler \
flatbuffers-compiler; \
rm -rf /var/lib/apt/lists/*
# Note: sccache removed for simpler builds
# Install cross-compilation tools for ARM64
RUN if [ "$TARGETPLATFORM" = "linux/arm64" ]; then \
apt-get update && \
apt-get install -y gcc-aarch64-linux-gnu && \
rm -rf /var/lib/apt/lists/*; \
# Optional: cross toolchain for aarch64 (only when targeting linux/arm64)
RUN set -eux; \
if [ "${TARGETPLATFORM:-linux/amd64}" = "linux/arm64" ]; then \
export DEBIAN_FRONTEND=noninteractive; \
apt-get update; \
apt-get install -y --no-install-recommends gcc-aarch64-linux-gnu; \
rm -rf /var/lib/apt/lists/*; \
fi
# Install protoc
RUN wget https://github.com/protocolbuffers/protobuf/releases/download/v31.1/protoc-31.1-linux-x86_64.zip \
&& unzip protoc-31.1-linux-x86_64.zip -d protoc3 \
&& mv protoc3/bin/* /usr/local/bin/ && chmod +x /usr/local/bin/protoc \
&& mv protoc3/include/* /usr/local/include/ && rm -rf protoc-31.1-linux-x86_64.zip protoc3
# Install flatc
RUN wget https://github.com/google/flatbuffers/releases/download/v25.2.10/Linux.flatc.binary.g++-13.zip \
&& unzip Linux.flatc.binary.g++-13.zip \
&& mv flatc /usr/local/bin/ && chmod +x /usr/local/bin/flatc && rm -rf Linux.flatc.binary.g++-13.zip
# Set up Rust targets based on platform
RUN set -e && \
PLATFORM="${TARGETPLATFORM:-linux/amd64}" && \
echo "🎯 Setting up Rust target for platform: $PLATFORM" && \
case "$PLATFORM" in \
"linux/amd64") rustup target add x86_64-unknown-linux-gnu ;; \
"linux/arm64") rustup target add aarch64-unknown-linux-gnu ;; \
*) echo "❌ Unsupported platform: $PLATFORM" && exit 1 ;; \
# Add Rust targets based on TARGETPLATFORM
RUN set -eux; \
case "${TARGETPLATFORM:-linux/amd64}" in \
linux/amd64) rustup target add x86_64-unknown-linux-gnu ;; \
linux/arm64) rustup target add aarch64-unknown-linux-gnu ;; \
*) echo "Unsupported TARGETPLATFORM=${TARGETPLATFORM}" >&2; exit 1 ;; \
esac
# Set up environment for cross-compilation
# Cross-compilation environment (used only when targeting aarch64)
ENV CARGO_TARGET_AARCH64_UNKNOWN_LINUX_GNU_LINKER=aarch64-linux-gnu-gcc
ENV CC_aarch64_unknown_linux_gnu=aarch64-linux-gnu-gcc
ENV CXX_aarch64_unknown_linux_gnu=aarch64-linux-gnu-g++
WORKDIR /usr/src/rustfs
# Copy all source code
# Layered copy to maximize caching:
# 1) top-level manifests
COPY Cargo.toml Cargo.lock ./
# 2) workspace member manifests (adjust if workspace layout changes)
COPY rustfs/Cargo.toml rustfs/Cargo.toml
COPY crates/*/Cargo.toml crates/
COPY cli/rustfs-gui/Cargo.toml cli/rustfs-gui/Cargo.toml
# Pre-fetch dependencies for better caching
RUN --mount=type=cache,target=/usr/local/cargo/registry \
--mount=type=cache,target=/usr/local/cargo/git \
cargo fetch --locked || true
# 3) copy full sources (this is the main cache invalidation point)
COPY . .
# Configure cargo for optimized builds
# Cargo build configuration for lean release artifacts
ENV CARGO_NET_GIT_FETCH_WITH_CLI=true \
CARGO_REGISTRIES_CRATES_IO_PROTOCOL=sparse \
CARGO_INCREMENTAL=0 \
@@ -82,75 +90,92 @@ ENV CARGO_NET_GIT_FETCH_WITH_CLI=true \
CARGO_PROFILE_RELEASE_SPLIT_DEBUGINFO=off \
CARGO_PROFILE_RELEASE_STRIP=symbols
# Generate protobuf code
RUN cargo run --bin gproto
# Generate protobuf/flatbuffers code (uses protoc/flatc from distro)
RUN --mount=type=cache,target=/usr/local/cargo/registry \
--mount=type=cache,target=/usr/local/cargo/git \
--mount=type=cache,target=/usr/src/rustfs/target \
cargo run --bin gproto
# Build the actual application with optimizations
RUN case "$TARGETPLATFORM" in \
"linux/amd64") \
echo "🔨 Building for amd64..." && \
rustup target add x86_64-unknown-linux-gnu && \
cargo build --release --target x86_64-unknown-linux-gnu --bin rustfs -j $(nproc) && \
cp target/x86_64-unknown-linux-gnu/release/rustfs /usr/local/bin/rustfs \
;; \
"linux/arm64") \
echo "🔨 Building for arm64..." && \
rustup target add aarch64-unknown-linux-gnu && \
cargo build --release --target aarch64-unknown-linux-gnu --bin rustfs -j $(nproc) && \
cp target/aarch64-unknown-linux-gnu/release/rustfs /usr/local/bin/rustfs \
;; \
*) \
echo "❌ Unsupported platform: $TARGETPLATFORM" && exit 1 \
;; \
# Build RustFS (target depends on TARGETPLATFORM)
RUN --mount=type=cache,target=/usr/local/cargo/registry \
--mount=type=cache,target=/usr/local/cargo/git \
--mount=type=cache,target=/usr/src/rustfs/target \
set -eux; \
case "${TARGETPLATFORM:-linux/amd64}" in \
linux/amd64) \
echo "Building for x86_64-unknown-linux-gnu"; \
cargo build --release --locked --target x86_64-unknown-linux-gnu --bin rustfs -j "$(nproc)"; \
install -m 0755 target/x86_64-unknown-linux-gnu/release/rustfs /usr/local/bin/rustfs \
;; \
linux/arm64) \
echo "Building for aarch64-unknown-linux-gnu"; \
cargo build --release --locked --target aarch64-unknown-linux-gnu --bin rustfs -j "$(nproc)"; \
install -m 0755 target/aarch64-unknown-linux-gnu/release/rustfs /usr/local/bin/rustfs \
;; \
*) \
echo "Unsupported TARGETPLATFORM=${TARGETPLATFORM}" >&2; exit 1 \
;; \
esac
# Runtime stage - Ubuntu minimal for better compatibility
# -----------------------------
# Runtime stage (Ubuntu minimal)
# -----------------------------
FROM ubuntu:22.04
# Install runtime dependencies
RUN apt-get update && apt-get install -y \
ARG BUILD_DATE
ARG VCS_REF
LABEL name="RustFS (dev-local)" \
maintainer="RustFS Team" \
build-date="${BUILD_DATE}" \
vcs-ref="${VCS_REF}" \
description="RustFS - local development image built from source (NOT for production)."
# Minimal runtime deps: certificates + tzdata + coreutils (for chroot --userspec)
RUN set -eux; \
export DEBIAN_FRONTEND=noninteractive; \
apt-get update; \
apt-get install -y --no-install-recommends \
ca-certificates \
tzdata \
wget \
coreutils \
passwd \
&& rm -rf /var/lib/apt/lists/*
coreutils; \
rm -rf /var/lib/apt/lists/*
# Create rustfs user and group
RUN groupadd -g 1000 rustfs && \
useradd -d /app -g rustfs -u 1000 -s /bin/bash rustfs
# Create a conventional runtime user/group (final switch happens in entrypoint via chroot --userspec)
RUN set -eux; \
groupadd -g 1000 rustfs; \
useradd -u 1000 -g rustfs -M -s /usr/sbin/nologin rustfs
WORKDIR /app
# Create data directories
RUN mkdir -p /data/rustfs{0,1,2,3} && \
chown -R rustfs:rustfs /data /app
# Prepare data/log directories with sane defaults
RUN set -eux; \
mkdir -p /data /logs; \
chown -R rustfs:rustfs /data /logs /app; \
chmod 0750 /data /logs
# Copy binary from builder stage
COPY --from=builder /usr/local/bin/rustfs /app/rustfs
RUN chmod +x /app/rustfs && chown rustfs:rustfs /app/rustfs
# Copy entrypoint script
# Copy the freshly built binary and the entrypoint
COPY --from=builder /usr/local/bin/rustfs /usr/bin/rustfs
COPY entrypoint.sh /entrypoint.sh
RUN chmod +x /entrypoint.sh
RUN chmod +x /usr/bin/rustfs /entrypoint.sh
# Switch to non-root user
USER rustfs
# Default environment (override in docker run/compose as needed)
ENV RUSTFS_ADDRESS=":9000" \
RUSTFS_ACCESS_KEY="rustfsadmin" \
RUSTFS_SECRET_KEY="rustfsadmin" \
RUSTFS_CONSOLE_ENABLE="true" \
RUSTFS_VOLUMES="/data" \
RUST_LOG="warn" \
RUSTFS_OBS_LOG_DIRECTORY="/logs" \
RUSTFS_SINKS_FILE_PATH="/logs" \
RUSTFS_USERNAME="rustfs" \
RUSTFS_GROUPNAME="rustfs" \
RUSTFS_UID="1000" \
RUSTFS_GID="1000"
# Expose ports
EXPOSE 9000
VOLUME ["/data", "/logs"]
# Environment variables
ENV RUSTFS_ACCESS_KEY=rustfsadmin \
RUSTFS_SECRET_KEY=rustfsadmin \
RUSTFS_ADDRESS=":9000" \
RUSTFS_CONSOLE_ENABLE=true \
RUSTFS_VOLUMES=/data \
RUST_LOG=warn
# Volume for data
VOLUME ["/data"]
# Set entrypoint and default command
# Keep root here; entrypoint will drop privileges using chroot --userspec
ENTRYPOINT ["/entrypoint.sh"]
CMD ["/app/rustfs"]
CMD ["/usr/bin/rustfs"]

152
Makefile
View File

@@ -1,5 +1,5 @@
###########
# 远程开发,需要 VSCode 安装 Dev Containers, Remote SSH, Remote Explorer
# Remote development requires VSCode with Dev Containers, Remote SSH, Remote Explorer
# https://code.visualstudio.com/docs/remote/containers
###########
DOCKER_CLI ?= docker
@@ -23,7 +23,7 @@ fmt-check:
.PHONY: clippy
clippy:
@echo "🔍 Running clippy checks..."
cargo clippy --fix --allow-dirty
cargo clippy --fix --allow-dirty
cargo clippy --all-targets --all-features -- -D warnings
.PHONY: check
@@ -126,7 +126,7 @@ docker-buildx-push:
.PHONY: docker-buildx-version
docker-buildx-version:
@if [ -z "$(VERSION)" ]; then \
echo "❌ 错误: 请指定版本, 例如: make docker-buildx-version VERSION=v1.0.0"; \
echo "❌ Error: Please specify version, example: make docker-buildx-version VERSION=v1.0.0"; \
exit 1; \
fi
@echo "🏗️ Building multi-architecture production Docker images (version: $(VERSION))..."
@@ -135,7 +135,7 @@ docker-buildx-version:
.PHONY: docker-buildx-push-version
docker-buildx-push-version:
@if [ -z "$(VERSION)" ]; then \
echo "❌ 错误: 请指定版本, 例如: make docker-buildx-push-version VERSION=v1.0.0"; \
echo "❌ Error: Please specify version, example: make docker-buildx-push-version VERSION=v1.0.0"; \
exit 1; \
fi
@echo "🚀 Building and pushing multi-architecture production Docker images (version: $(VERSION))..."
@@ -168,11 +168,11 @@ docker-dev-local:
.PHONY: docker-dev-push
docker-dev-push:
@if [ -z "$(REGISTRY)" ]; then \
echo "❌ 错误: 请指定镜像仓库, 例如: make docker-dev-push REGISTRY=ghcr.io/username"; \
echo "❌ Error: Please specify registry, example: make docker-dev-push REGISTRY=ghcr.io/username"; \
exit 1; \
fi
@echo "🚀 Building and pushing multi-architecture development Docker images..."
@echo "💡 推送到仓库: $(REGISTRY)"
@echo "💡 Pushing to registry: $(REGISTRY)"
$(DOCKER_CLI) buildx build \
--platform linux/amd64,linux/arm64 \
--file $(DOCKERFILE_SOURCE) \
@@ -210,7 +210,9 @@ docker-build-production:
docker-build-source:
@echo "🏗️ Building single-architecture source Docker image..."
@echo "💡 Consider using 'make docker-dev-local' for multi-arch support"
$(DOCKER_CLI) build -f $(DOCKERFILE_SOURCE) -t rustfs:source .
DOCKER_BUILDKIT=1 $(DOCKER_CLI) build \
--build-arg BUILDKIT_INLINE_CACHE=1 \
-f $(DOCKERFILE_SOURCE) -t rustfs:source .
# ========================================================================================
# Development Environment
@@ -249,7 +251,7 @@ dev-env-restart: dev-env-stop dev-env-start
.PHONY: docker-inspect-multiarch
docker-inspect-multiarch:
@if [ -z "$(IMAGE)" ]; then \
echo "❌ 错误: 请指定镜像, 例如: make docker-inspect-multiarch IMAGE=rustfs/rustfs:latest"; \
echo "❌ Error: Please specify image, example: make docker-inspect-multiarch IMAGE=rustfs/rustfs:latest"; \
exit 1; \
fi
@echo "🔍 Inspecting multi-architecture image: $(IMAGE)"
@@ -277,93 +279,93 @@ build-cross-all:
.PHONY: help-build
help-build:
@echo "🔨 RustFS 构建帮助:"
@echo "🔨 RustFS Build Help:"
@echo ""
@echo "🚀 本地构建 (推荐使用):"
@echo " make build # 构建 RustFS 二进制文件 (默认包含 console)"
@echo " make build-dev # 开发模式构建"
@echo " make build-musl # 构建 x86_64 musl 版本"
@echo " make build-gnu # 构建 x86_64 GNU 版本"
@echo " make build-musl-arm64 # 构建 aarch64 musl 版本"
@echo " make build-gnu-arm64 # 构建 aarch64 GNU 版本"
@echo "🚀 Local Build (Recommended):"
@echo " make build # Build RustFS binary (includes console by default)"
@echo " make build-dev # Development mode build"
@echo " make build-musl # Build x86_64 musl version"
@echo " make build-gnu # Build x86_64 GNU version"
@echo " make build-musl-arm64 # Build aarch64 musl version"
@echo " make build-gnu-arm64 # Build aarch64 GNU version"
@echo ""
@echo "🐳 Docker 构建:"
@echo " make build-docker # 使用 Docker 容器构建"
@echo " make build-docker BUILD_OS=ubuntu22.04 # 指定构建系统"
@echo "🐳 Docker Build:"
@echo " make build-docker # Build using Docker container"
@echo " make build-docker BUILD_OS=ubuntu22.04 # Specify build system"
@echo ""
@echo "🏗️ 跨架构构建:"
@echo " make build-cross-all # 构建所有架构的二进制文件"
@echo "🏗️ Cross-architecture Build:"
@echo " make build-cross-all # Build binaries for all architectures"
@echo ""
@echo "🔧 直接使用 build-rustfs.sh 脚本:"
@echo " ./build-rustfs.sh --help # 查看脚本帮助"
@echo " ./build-rustfs.sh --no-console # 构建时跳过 console 资源"
@echo " ./build-rustfs.sh --force-console-update # 强制更新 console 资源"
@echo " ./build-rustfs.sh --dev # 开发模式构建"
@echo " ./build-rustfs.sh --sign # 签名二进制文件"
@echo " ./build-rustfs.sh --platform x86_64-unknown-linux-gnu # 指定目标平台"
@echo " ./build-rustfs.sh --skip-verification # 跳过二进制验证"
@echo "🔧 Direct usage of build-rustfs.sh script:"
@echo " ./build-rustfs.sh --help # View script help"
@echo " ./build-rustfs.sh --no-console # Build without console resources"
@echo " ./build-rustfs.sh --force-console-update # Force update console resources"
@echo " ./build-rustfs.sh --dev # Development mode build"
@echo " ./build-rustfs.sh --sign # Sign binary files"
@echo " ./build-rustfs.sh --platform x86_64-unknown-linux-gnu # Specify target platform"
@echo " ./build-rustfs.sh --skip-verification # Skip binary verification"
@echo ""
@echo "💡 build-rustfs.sh 脚本提供了更多选项、智能检测和二进制验证功能"
@echo "💡 build-rustfs.sh script provides more options, smart detection and binary verification"
.PHONY: help-docker
help-docker:
@echo "🐳 Docker 多架构构建帮助:"
@echo "🐳 Docker Multi-architecture Build Help:"
@echo ""
@echo "🚀 生产镜像构建 (推荐使用 docker-buildx.sh):"
@echo " make docker-buildx # 构建生产多架构镜像(不推送)"
@echo " make docker-buildx-push # 构建并推送生产多架构镜像"
@echo " make docker-buildx-version VERSION=v1.0.0 # 构建指定版本"
@echo " make docker-buildx-push-version VERSION=v1.0.0 # 构建并推送指定版本"
@echo "🚀 Production Image Build (Recommended to use docker-buildx.sh):"
@echo " make docker-buildx # Build production multi-arch image (no push)"
@echo " make docker-buildx-push # Build and push production multi-arch image"
@echo " make docker-buildx-version VERSION=v1.0.0 # Build specific version"
@echo " make docker-buildx-push-version VERSION=v1.0.0 # Build and push specific version"
@echo ""
@echo "🔧 开发/源码镜像构建 (本地开发测试):"
@echo " make docker-dev # 构建开发多架构镜像(无法本地加载)"
@echo " make docker-dev-local # 构建开发单架构镜像(本地加载)"
@echo " make docker-dev-push REGISTRY=xxx # 构建并推送开发镜像"
@echo "🔧 Development/Source Image Build (Local development testing):"
@echo " make docker-dev # Build dev multi-arch image (cannot load locally)"
@echo " make docker-dev-local # Build dev single-arch image (local load)"
@echo " make docker-dev-push REGISTRY=xxx # Build and push dev image"
@echo ""
@echo "🏗️ 本地生产镜像构建 (替代方案):"
@echo " make docker-buildx-production-local # 本地构建生产单架构镜像"
@echo "🏗️ Local Production Image Build (Alternative):"
@echo " make docker-buildx-production-local # Build production single-arch image locally"
@echo ""
@echo "📦 单架构构建 (传统方式):"
@echo " make docker-build-production # 构建单架构生产镜像"
@echo " make docker-build-source # 构建单架构源码镜像"
@echo "📦 Single-architecture Build (Traditional way):"
@echo " make docker-build-production # Build single-arch production image"
@echo " make docker-build-source # Build single-arch source image"
@echo ""
@echo "🚀 开发环境管理:"
@echo " make dev-env-start # 启动开发容器环境"
@echo " make dev-env-stop # 停止开发容器环境"
@echo " make dev-env-restart # 重启开发容器环境"
@echo "🚀 Development Environment Management:"
@echo " make dev-env-start # Start development container environment"
@echo " make dev-env-stop # Stop development container environment"
@echo " make dev-env-restart # Restart development container environment"
@echo ""
@echo "🔧 辅助工具:"
@echo " make build-cross-all # 构建所有架构的二进制文件"
@echo " make docker-inspect-multiarch IMAGE=xxx # 检查镜像的架构支持"
@echo "🔧 Auxiliary Tools:"
@echo " make build-cross-all # Build binaries for all architectures"
@echo " make docker-inspect-multiarch IMAGE=xxx # Check image architecture support"
@echo ""
@echo "📋 环境变量:"
@echo " REGISTRY 镜像仓库地址 (推送时需要)"
@echo " DOCKERHUB_USERNAME Docker Hub 用户名"
@echo " DOCKERHUB_TOKEN Docker Hub 访问令牌"
@echo " GITHUB_TOKEN GitHub 访问令牌"
@echo "📋 Environment Variables:"
@echo " REGISTRY Image registry address (required for push)"
@echo " DOCKERHUB_USERNAME Docker Hub username"
@echo " DOCKERHUB_TOKEN Docker Hub access token"
@echo " GITHUB_TOKEN GitHub access token"
@echo ""
@echo "💡 建议:"
@echo " - 生产用途: 使用 docker-buildx* 命令 (基于预编译二进制)"
@echo " - 本地开发: 使用 docker-dev* 命令 (从源码构建)"
@echo " - 开发环境: 使用 dev-env-* 命令管理开发容器"
@echo "💡 Suggestions:"
@echo " - Production use: Use docker-buildx* commands (based on precompiled binaries)"
@echo " - Local development: Use docker-dev* commands (build from source)"
@echo " - Development environment: Use dev-env-* commands to manage dev containers"
.PHONY: help
help:
@echo "🦀 RustFS Makefile 帮助:"
@echo "🦀 RustFS Makefile Help:"
@echo ""
@echo "📋 主要命令分类:"
@echo " make help-build # 显示构建相关帮助"
@echo " make help-docker # 显示 Docker 相关帮助"
@echo "📋 Main Command Categories:"
@echo " make help-build # Show build-related help"
@echo " make help-docker # Show Docker-related help"
@echo ""
@echo "🔧 代码质量:"
@echo " make fmt # 格式化代码"
@echo " make clippy # 运行 clippy 检查"
@echo " make test # 运行测试"
@echo " make pre-commit # 运行所有预提交检查"
@echo "🔧 Code Quality:"
@echo " make fmt # Format code"
@echo " make clippy # Run clippy checks"
@echo " make test # Run tests"
@echo " make pre-commit # Run all pre-commit checks"
@echo ""
@echo "🚀 快速开始:"
@echo " make build # 构建 RustFS 二进制"
@echo " make docker-dev-local # 构建开发 Docker 镜像(本地)"
@echo " make dev-env-start # 启动开发环境"
@echo "🚀 Quick Start:"
@echo " make build # Build RustFS binary"
@echo " make docker-dev-local # Build development Docker image (local)"
@echo " make dev-env-start # Start development environment"
@echo ""
@echo "💡 更多帮助请使用 'make help-build' 'make help-docker'"
@echo "💡 For more help use 'make help-build' or 'make help-docker'"

View File

@@ -158,7 +158,7 @@ pub fn Home() -> Element {
Meta {
name: "description",
// TODO: translate to english
content: "RustFS RustFS 用热门安全的 Rust 语言开发,兼容 S3 协议。适用于 AI/ML 及海量数据存储、大数据、互联网、工业和保密存储等全部场景。近乎免费使用。遵循 Apache 2 协议,支持国产保密设备和系统。",
content: "RustFS is developed in the popular and secure Rust language, compatible with S3 protocol. Suitable for all scenarios including AI/ML and massive data storage, big data, internet, industrial and secure storage. Nearly free to use. Follows Apache 2 license, supports domestic security devices and systems.",
}
div { class: "min-h-screen flex flex-col items-center bg-white",
div { class: "absolute top-4 right-6 flex space-x-2",

View File

@@ -36,7 +36,7 @@ pub fn Navbar() -> Element {
pub struct LoadingSpinnerProps {
#[props(default = true)]
loading: bool,
#[props(default = "正在处理中...")]
#[props(default = "Processing...")]
text: &'static str,
}

View File

@@ -63,7 +63,7 @@ pub fn Setting() -> Element {
let config = config.read().clone();
spawn(async move {
if let Err(e) = service.read().restart(config).await {
ServiceManager::show_error(&format!("发送重启命令失败:{e}"));
ServiceManager::show_error(&format!("Failed to send restart command: {e}"));
}
// reset the status when you're done
loading.set(false);
@@ -209,7 +209,7 @@ pub fn Setting() -> Element {
}
LoadingSpinner {
loading: loading.read().to_owned(),
text: "服务处理中...",
text: "Service processing...",
}
}
}

View File

@@ -139,7 +139,7 @@ impl RustFSConfig {
if !stored_config.address.is_empty() && stored_config.address != Self::DEFAULT_ADDRESS_VALUE {
config.address = stored_config.address;
let (host, port) = Self::extract_host_port(config.address.as_str())
.ok_or_else(|| format!("无法从地址 '{}' 中提取主机和端口", config.address))?;
.ok_or_else(|| format!("Unable to extract host and port from address '{}'", config.address))?;
config.host = host.to_string();
config.port = port.to_string();
}
@@ -538,17 +538,17 @@ mod tests {
address: "127.0.0.1:9000".to_string(),
host: "127.0.0.1".to_string(),
port: "9000".to_string(),
access_key: "用户名".to_string(),
secret_key: "密码 123".to_string(),
domain_name: "测试.com".to_string(),
volume_name: "/数据/存储".to_string(),
access_key: "username".to_string(),
secret_key: "password123".to_string(),
domain_name: "test.com".to_string(),
volume_name: "/data/storage".to_string(),
console_address: "127.0.0.1:9001".to_string(),
};
assert_eq!(config.access_key, "用户名");
assert_eq!(config.secret_key, "密码 123");
assert_eq!(config.domain_name, "测试.com");
assert_eq!(config.volume_name, "/数据/存储");
assert_eq!(config.access_key, "username");
assert_eq!(config.secret_key, "password123");
assert_eq!(config.domain_name, "test.com");
assert_eq!(config.volume_name, "/data/storage");
}
#[test]

View File

@@ -81,7 +81,7 @@ pub enum ServiceCommand {
/// success: true,
/// start_time: chrono::Local::now(),
/// end_time: chrono::Local::now(),
/// message: "服务启动成功".to_string(),
/// message: "Service started successfully".to_string(),
/// };
///
/// println!("{:?}", result);
@@ -175,7 +175,7 @@ impl ServiceManager {
/// ```
async fn prepare_service() -> Result<PathBuf, Box<dyn Error>> {
// get the user directory
let home_dir = dirs::home_dir().ok_or("无法获取用户目录")?;
let home_dir = dirs::home_dir().ok_or("Unable to get user directory")?;
let rustfs_dir = home_dir.join("rustfs");
let bin_dir = rustfs_dir.join("bin");
let data_dir = rustfs_dir.join("data");
@@ -247,23 +247,23 @@ impl ServiceManager {
match cmd {
ServiceCommand::Start(config) => {
if let Err(e) = Self::start_service(&config).await {
Self::show_error(&format!("启动服务失败:{e}"));
Self::show_error(&format!("Failed to start service: {e}"));
}
}
ServiceCommand::Stop => {
if let Err(e) = Self::stop_service().await {
Self::show_error(&format!("停止服务失败:{e}"));
Self::show_error(&format!("Failed to stop service: {e}"));
}
}
ServiceCommand::Restart(config) => {
if Self::check_service_status().await.is_some() {
if let Err(e) = Self::stop_service().await {
Self::show_error(&format!("重启服务失败:{e}"));
Self::show_error(&format!("Failed to restart service: {e}"));
continue;
}
}
if let Err(e) = Self::start_service(&config).await {
Self::show_error(&format!("重启服务失败:{e}"));
Self::show_error(&format!("Failed to restart service: {e}"));
}
}
}
@@ -295,7 +295,7 @@ impl ServiceManager {
async fn start_service(config: &RustFSConfig) -> Result<(), Box<dyn Error>> {
// Check if the service is already running
if let Some(existing_pid) = Self::check_service_status().await {
return Err(format!("服务已经在运行,PID: {existing_pid}").into());
return Err(format!("Service is already running, PID: {existing_pid}").into());
}
// Prepare the service program
@@ -307,16 +307,16 @@ impl ServiceManager {
}
// Extract the port from the configuration
let main_port = Self::extract_port(&config.address).ok_or("无法解析主服务端口")?;
let console_port = Self::extract_port(&config.console_address).ok_or("无法解析控制台端口")?;
let main_port = Self::extract_port(&config.address).ok_or("Unable to parse main service port")?;
let console_port = Self::extract_port(&config.console_address).ok_or("Unable to parse console port")?;
let host = config.address.split(':').next().ok_or("无法解析主机地址")?;
let host = config.address.split(':').next().ok_or("Unable to parse host address")?;
// Check the port
let ports = vec![main_port, console_port];
for port in ports {
if Self::is_port_in_use(host, port).await {
return Err(format!("端口 {port} 已被占用").into());
return Err(format!("Port {port} is already in use").into());
}
}
@@ -339,12 +339,12 @@ impl ServiceManager {
// Check if the service started successfully
if Self::is_port_in_use(host, main_port).await {
Self::show_info(&format!("服务启动成功!进程 ID: {process_pid}"));
Self::show_info(&format!("Service started successfully! Process ID: {process_pid}"));
Ok(())
} else {
child.kill().await?;
Err("服务启动失败".into())
Err("Service failed to start".into())
}
}
@@ -378,13 +378,13 @@ impl ServiceManager {
// Verify that the service is indeed stopped
tokio::time::sleep(Duration::from_secs(1)).await;
if Self::check_service_status().await.is_some() {
return Err("服务停止失败".into());
return Err("Service failed to stop".into());
}
Self::show_info("服务已成功停止");
Self::show_info("Service stopped successfully");
Ok(())
} else {
Err("服务未运行".into())
Err("Service is not running".into())
}
}
@@ -411,7 +411,7 @@ impl ServiceManager {
/// ```
pub(crate) fn show_error(message: &str) {
rfd::MessageDialog::new()
.set_title("错误")
.set_title("Error")
.set_description(message)
.set_level(rfd::MessageLevel::Error)
.show();
@@ -426,7 +426,7 @@ impl ServiceManager {
/// ```
pub(crate) fn show_info(message: &str) {
rfd::MessageDialog::new()
.set_title("成功")
.set_title("Success")
.set_description(message)
.set_level(rfd::MessageLevel::Info)
.show();
@@ -475,7 +475,7 @@ impl ServiceManager {
self.command_tx.send(ServiceCommand::Start(config.clone())).await?;
let host = &config.host;
let port = config.port.parse::<u16>().expect("无效的端口号");
let port = config.port.parse::<u16>().expect("Invalid port number");
// wait for the service to actually start
let mut retries = 0;
while retries < 30 {
@@ -486,14 +486,14 @@ impl ServiceManager {
success: true,
start_time,
end_time,
message: "服务启动成功".to_string(),
message: "Service started successfully".to_string(),
});
}
tokio::time::sleep(Duration::from_secs(1)).await;
retries += 1;
}
Err("服务启动超时".into())
Err("Service start timeout".into())
}
/// Stop the service
@@ -537,14 +537,14 @@ impl ServiceManager {
success: true,
start_time,
end_time,
message: "服务停止成功".to_string(),
message: "Service stopped successfully".to_string(),
});
}
tokio::time::sleep(Duration::from_secs(1)).await;
retries += 1;
}
Err("服务停止超时".into())
Err("Service stop timeout".into())
}
/// Restart the service
@@ -590,7 +590,7 @@ impl ServiceManager {
self.command_tx.send(ServiceCommand::Restart(config.clone())).await?;
let host = &config.host;
let port = config.port.parse::<u16>().expect("无效的端口号");
let port = config.port.parse::<u16>().expect("Invalid port number");
// wait for the service to restart
let mut retries = 0;
@@ -602,8 +602,8 @@ impl ServiceManager {
Err(e) => {
error!("save config error: {}", e);
self.command_tx.send(ServiceCommand::Stop).await?;
Self::show_error("保存配置失败");
return Err("保存配置失败".into());
Self::show_error("Failed to save configuration");
return Err("Failed to save configuration".into());
}
}
let end_time = chrono::Local::now();
@@ -611,13 +611,13 @@ impl ServiceManager {
success: true,
start_time,
end_time,
message: "服务重启成功".to_string(),
message: "Service restarted successfully".to_string(),
});
}
tokio::time::sleep(Duration::from_secs(1)).await;
retries += 1;
}
Err("服务重启超时".into())
Err("Service restart timeout".into())
}
}
@@ -802,10 +802,10 @@ mod tests {
success: true,
start_time: chrono::Local::now(),
end_time: chrono::Local::now(),
message: "操作成功 🎉".to_string(),
message: "Operation successful 🎉".to_string(),
};
assert_eq!(result.message, "操作成功 🎉");
assert_eq!(result.message, "Operation successful 🎉");
assert!(result.success);
}

View File

@@ -23,7 +23,7 @@ use tracing_subscriber::util::SubscriberInitExt;
/// that rotates log files daily
pub fn init_logger() -> WorkerGuard {
// configuring rolling logs rolling by day
let home_dir = dirs::home_dir().expect("无法获取用户目录");
let home_dir = dirs::home_dir().expect("Unable to get user directory");
let rustfs_dir = home_dir.join("rustfs");
let logs_dir = rustfs_dir.join("logs");
let file_appender = RollingFileAppender::builder()

View File

@@ -100,6 +100,7 @@ rustfs-rio.workspace = true
rustfs-signer.workspace = true
rustfs-checksums.workspace = true
futures-util.workspace = true
async-recursion.workspace = true
[target.'cfg(not(windows))'.dependencies]
nix = { workspace = true }

View File

@@ -440,6 +440,7 @@ impl LocalDisk {
}
#[tracing::instrument(level = "debug", skip(self))]
#[async_recursion::async_recursion]
pub async fn delete_file(
&self,
base_path: &PathBuf,
@@ -803,13 +804,17 @@ impl LocalDisk {
Ok(())
}
async fn scan_dir<W: AsyncWrite + Unpin>(
#[async_recursion::async_recursion]
async fn scan_dir<W>(
&self,
current: &mut String,
opts: &WalkDirOptions,
out: &mut MetacacheWriter<W>,
objs_returned: &mut i32,
) -> Result<()> {
) -> Result<()>
where
W: AsyncWrite + Unpin + Send,
{
let forward = {
opts.forward_to.as_ref().filter(|v| v.starts_with(&*current)).map(|v| {
let forward = v.trim_start_matches(&*current);

View File

@@ -177,15 +177,17 @@ impl S3PeerSys {
let pools = cli.get_pools();
let idx = i;
if pools.unwrap_or_default().contains(&idx) {
per_pool_errs.push(errors[j].as_ref());
per_pool_errs.push(errors[j].clone());
}
// TODO: reduceWriteQuorumErrs
if let Some(pool_err) =
reduce_write_quorum_errs(&per_pool_errs, BUCKET_OP_IGNORED_ERRS, (per_pool_errs.len() / 2) + 1)
{
return Err(pool_err);
}
}
}
// TODO:
Ok(())
}
pub async fn list_bucket(&self, opts: &BucketOptions) -> Result<Vec<BucketInfo>> {
@@ -387,7 +389,6 @@ impl PeerS3Client for LocalPeerS3Client {
if opts.force_create && matches!(e, Error::VolumeExists) {
return Ok(());
}
Err(e)
}
}
@@ -405,7 +406,9 @@ impl PeerS3Client for LocalPeerS3Client {
}
}
// TODO: reduceWriteQuorumErrs
if let Some(err) = reduce_write_quorum_errs(&errs, BUCKET_OP_IGNORED_ERRS, (local_disks.len() / 2) + 1) {
return Err(err);
}
Ok(())
}

View File

@@ -1221,7 +1221,7 @@ impl StorageAPI for ECStore {
}
if let Err(err) = self.peer_sys.make_bucket(bucket, opts).await {
let err = err.into();
let err = to_object_err(err.into(), vec![bucket]);
if !is_err_bucket_exists(&err) {
let _ = self
.delete_bucket(
@@ -1234,7 +1234,6 @@ impl StorageAPI for ECStore {
)
.await;
}
return Err(err);
};

View File

@@ -172,7 +172,12 @@ impl ObjectStore {
}
if let Some(info) = v.item {
let name = info.name.trim_start_matches(&prefix).trim_end_matches(SLASH_SEPARATOR);
let object_name = if cfg!(target_os = "windows") {
info.name.replace('\\', "/")
} else {
info.name
};
let name = object_name.trim_start_matches(&prefix).trim_end_matches(SLASH_SEPARATOR);
let _ = sender
.send(StringOrErr {
item: Some(name.to_owned()),

View File

@@ -231,6 +231,20 @@ Retrieve an object from S3 with two operation modes: read content directly or do
- `local_path` (string, optional): Local file path (required when mode is "download")
- `max_content_size` (number, optional): Maximum content size in bytes for read mode (default: 1MB)
### `create_bucket`
Create a new S3 bucket with the specified name.
**Parameters:**
- `bucket_name` (string): Source S3 bucket.
### `delete_bucket`
Delete the specified S3 bucket. If the bucket is not empty, the deletion will fail. You should delete all objects and objects inside them before calling this method.**WARNING: This operation will permanently delete the bucket and all objects within it!**
- `bucket_name` (string): Source S3 bucket.
## Architecture
The MCP server is built with a modular architecture:

View File

@@ -151,6 +151,36 @@ impl S3Client {
Ok(Self { client })
}
pub async fn create_bucket(&self, bucket_name: &str) -> Result<BucketInfo> {
info!("Creating S3 bucket: {}", bucket_name);
self.client
.create_bucket()
.bucket(bucket_name)
.send()
.await
.context(format!("Failed to create S3 bucket: {bucket_name}"))?;
info!("Bucket '{}' created successfully", bucket_name);
Ok(BucketInfo {
name: bucket_name.to_string(),
creation_date: None, // Creation date not returned by create_bucket
})
}
pub async fn delete_bucket(&self, bucket_name: &str) -> Result<()> {
info!("Deleting S3 bucket: {}", bucket_name);
self.client
.delete_bucket()
.bucket(bucket_name)
.send()
.await
.context(format!("Failed to delete S3 bucket: {bucket_name}"))?;
info!("Bucket '{}' deleted successfully", bucket_name);
Ok(())
}
pub async fn list_buckets(&self) -> Result<Vec<BucketInfo>> {
debug!("Listing S3 buckets");

View File

@@ -54,6 +54,18 @@ pub struct UploadFileRequest {
pub cache_control: Option<String>,
}
#[derive(Serialize, Deserialize, JsonSchema)]
pub struct CreateBucketReqeust {
#[schemars(description = "Name of the S3 bucket to create")]
pub bucket_name: String,
}
#[derive(Serialize, Deserialize, JsonSchema)]
pub struct DeleteBucketReqeust {
#[schemars(description = "Name of the S3 bucket to delete")]
pub bucket_name: String,
}
#[derive(Serialize, Deserialize, JsonSchema)]
pub struct GetObjectRequest {
#[schemars(description = "Name of the S3 bucket")]
@@ -110,6 +122,53 @@ impl RustfsMcpServer {
})
}
#[tool(description = "Create a new S3 bucket with the specified name")]
pub async fn create_bucket(&self, Parameters(req): Parameters<CreateBucketReqeust>) -> String {
info!("Executing create_bucket tool for bucket: {}", req.bucket_name);
match self.s3_client.create_bucket(&req.bucket_name).await {
Ok(_) => {
format!("Successfully created bucket: {}", req.bucket_name)
}
Err(e) => {
format!("Failed to create bucket '{}': {:?}", req.bucket_name, e)
}
}
}
#[tool(description = "Delete an existing S3 bucket with the specified name")]
pub async fn delete_bucket(&self, Parameters(req): Parameters<DeleteBucketReqeust>) -> String {
info!("Executing delete_bucket tool for bucket: {}", req.bucket_name);
// check if bucket is empty, if not, can not delete bucket directly.
let object_result = match self
.s3_client
.list_objects_v2(&req.bucket_name, ListObjectsOptions::default())
.await
{
Ok(result) => result,
Err(e) => {
error!("Failed to list objects in bucket '{}': {:?}", req.bucket_name, e);
return format!("Failed to list objects in bucket '{}': {:?}", req.bucket_name, e);
}
};
if !object_result.objects.is_empty() {
error!("Bucket '{}' is not empty", req.bucket_name);
return format!("Failed to delete bucket '{}': bucket is not empty", req.bucket_name);
}
// delete the bucket.
match self.s3_client.delete_bucket(&req.bucket_name).await {
Ok(_) => {
format!("Successfully deleted bucket: {}", req.bucket_name)
}
Err(e) => {
format!("Failed to delete bucket '{}': {:?}", req.bucket_name, e)
}
}
}
#[tool(description = "List all S3 buckets accessible with the configured credentials")]
pub async fn list_buckets(&self) -> String {
info!("Executing list_buckets tool");
@@ -667,4 +726,20 @@ mod tests {
assert_eq!(read_mode_deser, GetObjectMode::Read);
assert_eq!(download_mode_deser, GetObjectMode::Download);
}
#[test]
fn test_bucket_creation() {
let request = CreateBucketReqeust {
bucket_name: "test-bucket".to_string(),
};
assert_eq!(request.bucket_name, "test-bucket");
}
#[test]
fn test_bucket_deletion() {
let request = DeleteBucketReqeust {
bucket_name: "test-bucket".to_string(),
};
assert_eq!(request.bucket_name, "test-bucket");
}
}

View File

@@ -556,19 +556,19 @@ mod tests {
fn test_index_add() -> io::Result<()> {
let mut index = Index::new();
// 测试添加第一个索引
// Test adding first index
index.add(100, 1000)?;
assert_eq!(index.info.len(), 1);
assert_eq!(index.info[0].compressed_offset, 100);
assert_eq!(index.info[0].uncompressed_offset, 1000);
// 测试添加相同未压缩偏移量的索引
// Test adding index with same uncompressed offset
index.add(200, 1000)?;
assert_eq!(index.info.len(), 1);
assert_eq!(index.info[0].compressed_offset, 200);
assert_eq!(index.info[0].uncompressed_offset, 1000);
// 测试添加新的索引(确保距离足够大)
// Test adding new index (ensure distance is large enough)
index.add(300, 2000 + MIN_INDEX_DIST)?;
assert_eq!(index.info.len(), 2);
assert_eq!(index.info[1].compressed_offset, 300);
@@ -581,14 +581,14 @@ mod tests {
fn test_index_add_errors() {
let mut index = Index::new();
// 添加初始索引
// Add initial index
index.add(100, 1000).unwrap();
// 测试添加更小的未压缩偏移量
// Test adding smaller uncompressed offset
let err = index.add(200, 500).unwrap_err();
assert_eq!(err.kind(), io::ErrorKind::InvalidData);
// 测试添加更小的压缩偏移量
// Test adding smaller compressed offset
let err = index.add(50, 2000).unwrap_err();
assert_eq!(err.kind(), io::ErrorKind::InvalidData);
}
@@ -599,22 +599,22 @@ mod tests {
index.total_uncompressed = 1000 + MIN_INDEX_DIST * 3;
index.total_compressed = 5000;
// 添加一些测试数据,确保索引间距满足 MIN_INDEX_DIST 要求
// Add some test data, ensure index spacing meets MIN_INDEX_DIST requirement
index.add(100, 1000)?;
index.add(300, 1000 + MIN_INDEX_DIST)?;
index.add(500, 1000 + MIN_INDEX_DIST * 2)?;
// 测试查找存在的偏移量
// Test finding existing offset
let (comp, uncomp) = index.find(1500)?;
assert_eq!(comp, 100);
assert_eq!(uncomp, 1000);
// 测试查找边界值
// Test finding boundary value
let (comp, uncomp) = index.find(1000 + MIN_INDEX_DIST)?;
assert_eq!(comp, 300);
assert_eq!(uncomp, 1000 + MIN_INDEX_DIST);
// 测试查找最后一个索引
// Test finding last index
let (comp, uncomp) = index.find(1000 + MIN_INDEX_DIST * 2)?;
assert_eq!(comp, 500);
assert_eq!(uncomp, 1000 + MIN_INDEX_DIST * 2);
@@ -628,16 +628,16 @@ mod tests {
index.total_uncompressed = 10000;
index.total_compressed = 5000;
// 测试未初始化的索引
// Test uninitialized index
let uninit_index = Index::new();
let err = uninit_index.find(1000).unwrap_err();
assert_eq!(err.kind(), io::ErrorKind::Other);
// 测试超出范围的偏移量
// Test offset out of range
let err = index.find(15000).unwrap_err();
assert_eq!(err.kind(), io::ErrorKind::UnexpectedEof);
// 测试负数偏移量
// Test negative offset
let err = match index.find(-1000) {
Ok(_) => panic!("should be error"),
Err(e) => e,
@@ -650,15 +650,15 @@ mod tests {
let mut index = Index::new();
index.est_block_uncomp = MIN_INDEX_DIST;
// 添加超过最大索引数量的条目,确保间距满足 MIN_INDEX_DIST 要求
// Add entries exceeding maximum index count, ensure spacing meets MIN_INDEX_DIST requirement
for i in 0..MAX_INDEX_ENTRIES + 100 {
index.add(i as i64 * 100, i as i64 * MIN_INDEX_DIST).unwrap();
}
// 手动调用 reduce 方法
// Manually call reduce method
index.reduce();
// 验证索引数量是否被正确减少
// Verify index count has been correctly reduced
assert!(index.info.len() <= MAX_INDEX_ENTRIES);
}
@@ -666,16 +666,16 @@ mod tests {
fn test_index_json() -> io::Result<()> {
let mut index = Index::new();
// 添加一些测试数据
// Add some test data
index.add(100, 1000)?;
index.add(300, 2000 + MIN_INDEX_DIST)?;
// 测试 JSON 序列化
// Test JSON serialization
let json = index.to_json().unwrap();
let json_str = String::from_utf8(json).unwrap();
println!("json_str: {json_str}");
// 验证 JSON 内容
// Verify JSON content
assert!(json_str.contains("\"compressed\": 100"));
assert!(json_str.contains("\"uncompressed\": 1000"));

View File

@@ -443,7 +443,7 @@ mod tests {
let mut compressed = Vec::new();
compress_reader.read_to_end(&mut compressed).await.unwrap();
// DecompressReader解包
// DecompressReader unpacking
let mut decompress_reader = DecompressReader::new(Cursor::new(compressed.clone()), CompressionAlgorithm::Gzip);
let mut decompressed = Vec::new();
decompress_reader.read_to_end(&mut decompressed).await.unwrap();
@@ -460,7 +460,7 @@ mod tests {
let mut compressed = Vec::new();
compress_reader.read_to_end(&mut compressed).await.unwrap();
// DecompressReader解包
// DecompressReader unpacking
let mut decompress_reader = DecompressReader::new(Cursor::new(compressed.clone()), CompressionAlgorithm::Deflate);
let mut decompressed = Vec::new();
decompress_reader.read_to_end(&mut decompressed).await.unwrap();

View File

@@ -223,7 +223,7 @@ mod tests {
let n = etag_reader.read_to_end(&mut buf).await.unwrap();
assert_eq!(n, data.len());
assert_eq!(&buf, data);
// 校验通过etag应等于expected
// Verification passed, etag should equal expected
assert_eq!(etag_reader.try_resolve_etag(), Some(expected));
}
@@ -236,7 +236,7 @@ mod tests {
let mut etag_reader = EtagReader::new(reader, Some(wrong_checksum));
let mut buf = Vec::new();
// 校验失败,应该返回InvalidData错误
// Verification failed, should return InvalidData error
let err = etag_reader.read_to_end(&mut buf).await.unwrap_err();
assert_eq!(err.kind(), std::io::ErrorKind::InvalidData);
}

View File

@@ -123,7 +123,7 @@ mod tests {
let hardlimit = HardLimitReader::new(reader, 3);
let mut r = hardlimit;
let mut buf = vec![0u8; 10];
// 读取超限,应该返回错误
// Reading exceeds limit, should return error
let err = match read_full(&mut r, &mut buf).await {
Ok(n) => {
println!("Read {n} bytes");

View File

@@ -53,9 +53,9 @@ pub trait QueryExecution: Send + Sync {
fn query_type(&self) -> QueryType {
QueryType::Batch
}
// 开始
// Start
async fn start(&self) -> QueryResult<Output>;
// 停止
// Stop
fn cancel(&self) -> QueryResult<()>;
}

View File

@@ -87,15 +87,15 @@ impl PhysicalPlanner for DefaultPhysicalPlanner {
logical_plan: &LogicalPlan,
session: &SessionCtx,
) -> QueryResult<Arc<dyn ExecutionPlan>> {
// 将扩展的物理计划优化规则注入 df session state
// Inject extended physical plan optimization rules into df's session state
let new_state = SessionStateBuilder::new_from_existing(session.inner().clone())
.with_physical_optimizer_rules(self.ext_physical_optimizer_rules.clone())
.build();
// 通过扩展的物理计划转换规则构造 df Physical Planner
// Construct df's Physical Planner with extended physical plan transformation rules
let planner = DFDefaultPhysicalPlanner::with_extension_planners(self.ext_physical_transform_rules.clone());
// 执行 df 的物理计划规划及优化
// Execute df's physical plan planning and optimization
planner
.create_physical_plan(logical_plan, &new_state)
.await

View File

@@ -289,44 +289,44 @@ mod tests {
CompressionAlgorithm::Snappy,
];
println!("\n压缩算法基准测试结果:");
println!("\nCompression algorithm benchmark results:");
println!(
"{:<10} {:<10} {:<15} {:<15} {:<15}",
"数据大小", "算法", "压缩时间(ms)", "压缩后大小", "压缩率"
"Data Size", "Algorithm", "Compress Time(ms)", "Compressed Size", "Compression Ratio"
);
for size in sizes {
// 生成可压缩的数据(重复的文本模式)
// Generate compressible data (repeated text pattern)
let pattern = b"Hello, this is a test pattern that will be repeated multiple times to create compressible data. ";
let data: Vec<u8> = pattern.iter().cycle().take(size).copied().collect();
for algo in algorithms {
// 压缩测试
// Compression test
let start = Instant::now();
let compressed = compress_block(&data, algo);
let compress_time = start.elapsed();
let compression_time = start.elapsed();
// 解压测试
// Decompression test
let start = Instant::now();
let _decompressed = decompress_block(&compressed, algo).unwrap();
let _decompress_time = start.elapsed();
let _decompression_time = start.elapsed();
// 计算压缩率
// Calculate compression ratio
let compression_ratio = (size as f64 / compressed.len() as f64) as f32;
println!(
"{:<10} {:<10} {:<15.2} {:<15} {:<15.2}x",
format!("{}KB", size / 1024),
algo.as_str(),
compress_time.as_secs_f64() * 1000.0,
compression_time.as_secs_f64() * 1000.0,
compressed.len(),
compression_ratio
);
// 验证解压结果
// Verify decompression result
assert_eq!(_decompressed, data);
}
println!(); // 添加空行分隔不同大小的结果
println!(); // Add blank line to separate results of different sizes
}
}
}

View File

@@ -110,18 +110,18 @@ use siphasher::sip::SipHasher;
pub const EMPTY_STRING_SHA256_HASH: &str = "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855";
pub fn sip_hash(key: &str, cardinality: usize, id: &[u8; 16]) -> usize {
// 你的密钥,必须是 16 字节
// Your key, must be 16 bytes
// 计算字符串的 SipHash 值
// Calculate SipHash value of the string
let result = SipHasher::new_with_key(id).hash(key.as_bytes());
result as usize % cardinality
(result as usize) % cardinality
}
pub fn crc_hash(key: &str, cardinality: usize) -> usize {
let mut hasher = Hasher::new(); // 创建一个新的哈希器
let mut hasher = Hasher::new(); // Create a new hasher
hasher.update(key.as_bytes()); // 更新哈希状态,添加数据
hasher.update(key.as_bytes()); // Update hash state, add data
let checksum = hasher.finalize();

View File

@@ -599,7 +599,7 @@ mod tests {
#[test]
fn test_format_consistency_with_extensions() {
// 测试格式与扩展名的一致性
// Test format consistency with extensions
let consistency_tests = vec![
(CompressionFormat::Gzip, "gz"),
(CompressionFormat::Bzip2, "bz2"),
@@ -721,20 +721,20 @@ mod tests {
#[test]
fn test_compression_format_clone_and_copy() {
// 测试 CompressionFormat 是否可以被复制
// Test if CompressionFormat can be copied
let format = CompressionFormat::Gzip;
let format_copy = format;
// 验证复制后的值相等
// Verify copied values are equal
assert_eq!(format, format_copy);
// 验证原值仍然可用
// Verify original value is still usable
assert_eq!(format, CompressionFormat::Gzip);
}
#[test]
fn test_compression_format_match_exhaustiveness() {
// 测试 match 语句的完整性
// Test match statement completeness
fn handle_format(format: CompressionFormat) -> &'static str {
match format {
CompressionFormat::Gzip => "gzip",
@@ -748,7 +748,7 @@ mod tests {
}
}
// 测试所有变体都有对应的处理
// Test all variants have corresponding handlers
assert_eq!(handle_format(CompressionFormat::Gzip), "gzip");
assert_eq!(handle_format(CompressionFormat::Bzip2), "bzip2");
assert_eq!(handle_format(CompressionFormat::Zip), "zip");
@@ -760,10 +760,10 @@ mod tests {
#[test]
fn test_extension_parsing_performance() {
// 测试扩展名解析的性能(简单的性能测试)
// Test extension parsing performance (simple performance test)
let extensions = vec!["gz", "bz2", "zip", "xz", "zlib", "zst", "unknown"];
// 多次调用以测试性能一致性
// Multiple calls to test performance consistency
for _ in 0..1000 {
for ext in &extensions {
let _format = CompressionFormat::from_extension(ext);
@@ -775,7 +775,7 @@ mod tests {
#[test]
fn test_format_default_behavior() {
// 测试格式的默认行为
// Test format default behavior
let unknown_extensions = vec!["", "txt", "doc", "pdf", "unknown_ext"];
for ext in unknown_extensions {
@@ -786,7 +786,7 @@ mod tests {
#[test]
fn test_compression_level() {
// 测试压缩级别
// Test compression level
let default_level = CompressionLevel::default();
assert_eq!(default_level, CompressionLevel::Default);
@@ -800,7 +800,7 @@ mod tests {
#[test]
fn test_format_extension() {
// 测试格式扩展名获取
// Test format extension retrieval
assert_eq!(CompressionFormat::Gzip.extension(), "gz");
assert_eq!(CompressionFormat::Bzip2.extension(), "bz2");
assert_eq!(CompressionFormat::Zip.extension(), "zip");
@@ -813,7 +813,7 @@ mod tests {
#[test]
fn test_format_is_supported() {
// 测试格式支持检查
// Test format support check
assert!(CompressionFormat::Gzip.is_supported());
assert!(CompressionFormat::Bzip2.is_supported());
assert!(CompressionFormat::Zip.is_supported());
@@ -826,7 +826,7 @@ mod tests {
#[test]
fn test_format_from_path() {
// 测试从路径识别格式
// Test format recognition from path
use std::path::Path;
assert_eq!(CompressionFormat::from_path("file.gz"), CompressionFormat::Gzip);
@@ -840,7 +840,7 @@ mod tests {
#[tokio::test]
async fn test_get_encoder_supported_formats() {
// 测试支持的格式能够创建编码器
// Test supported formats can create encoders
use std::io::Cursor;
let output = Vec::new();
@@ -853,7 +853,7 @@ mod tests {
#[tokio::test]
async fn test_get_encoder_unsupported_formats() {
// 测试不支持的格式返回错误
// Test unsupported formats return errors
use std::io::Cursor;
let output1 = Vec::new();
@@ -900,7 +900,7 @@ mod tests {
#[test]
fn test_decompressor_creation() {
// 测试解压缩器创建
// Test decompressor creation
let decompressor = Decompressor::new(CompressionFormat::Gzip);
assert_eq!(decompressor.format, CompressionFormat::Gzip);
@@ -910,7 +910,7 @@ mod tests {
#[test]
fn test_zip_entry_creation() {
// 测试 ZIP 条目信息创建
// Test ZIP entry info creation
let entry = ZipEntry {
name: "test.txt".to_string(),
size: 1024,
@@ -928,7 +928,7 @@ mod tests {
#[test]
fn test_compression_level_variants() {
// 测试压缩级别的所有变体
// Test all compression level variants
let levels = vec![
CompressionLevel::Fastest,
CompressionLevel::Best,
@@ -938,14 +938,14 @@ mod tests {
];
for level in levels {
// 验证每个级别都有对应的 Debug 实现
// Verify each level has corresponding Debug implementation
let _debug_str = format!("{level:?}");
}
}
#[test]
fn test_format_comprehensive_coverage() {
// 测试格式的全面覆盖
// Test comprehensive format coverage
let all_formats = vec![
CompressionFormat::Gzip,
CompressionFormat::Bzip2,
@@ -958,13 +958,13 @@ mod tests {
];
for format in all_formats {
// 验证每个格式都有扩展名
// Verify each format has an extension
let _ext = format.extension();
// 验证支持状态检查
// Verify support status check
let _supported = format.is_supported();
// 验证 Debug 实现
// Verify Debug implementation
let _debug = format!("{format:?}");
}
}
@@ -975,7 +975,7 @@ mod tests {
// use std::path::Path;
// use tokio::fs::File;
// let input_path = "/Users/weisd/Downloads/wsd.tar.gz"; // 替换为你的压缩文件路径
// let input_path = "/Users/weisd/Downloads/wsd.tar.gz"; // Replace with your compressed file path
// let f = File::open(input_path).await?;
@@ -994,8 +994,8 @@ mod tests {
// )
// .await
// {
// Ok(_) => println!("解压成功!"),
// Err(e) => println!("解压失败:{}", e),
// Ok(_) => println!("Decompression successful!"),
// Err(e) => println!("Decompression failed: {}", e),
// }
// Ok(())

View File

@@ -1,22 +1,22 @@
# RustFS 管理员用户名
# RustFS admin username
RUSTFS_ROOT_USER=rustfsadmin
# RustFS 管理员密码
# RustFS admin password
RUSTFS_ROOT_PASSWORD=rustfsadmin
# 数据卷配置示例路径:deploy/data/rustfs.env
# RustFS 数据卷存储路径,支持多卷配置,vol1 vol4
# Data volume configuration example path: deploy/data/rustfs.env
# RustFS data volume storage path, supports multi-volume configuration, vol1 to vol4
RUSTFS_VOLUMES="./deploy/deploy/vol{1...4}"
# RustFS 服务启动参数,指定监听地址和端口
# RustFS service startup parameters, specify listening address and port
RUSTFS_OPTS="--address :9000"
# RustFS 服务监听地址和端口
# RustFS service listening address and port
RUSTFS_ADDRESS=":9000"
# 是否启用 RustFS 控制台功能
# Whether to enable RustFS console functionality
RUSTFS_CONSOLE_ENABLE=true
# RustFS 服务域名配置
# RustFS service domain configuration
RUSTFS_SERVER_DOMAINS=127.0.0.1:9000
# RustFS 许可证内容
# RustFS license content
RUSTFS_LICENSE="license content"
# 可观测性配置Endpointhttp://localhost:4317
# Observability configuration Endpoint: http://localhost:4317
RUSTFS_OBS_ENDPOINT=http://localhost:4317
# TLS 证书目录路径:deploy/certs
# TLS certificate directory path: deploy/certs
RUSTFS_TLS_PATH=/etc/default/tls

View File

@@ -150,11 +150,7 @@ build_and_push() {
else
print_message $RED "❌ Failed to build latest variant"
print_message $YELLOW "💡 Note: Make sure rustfs binaries are available at:"
if [ "$CHANNEL" = "dev" ]; then
print_message $YELLOW " https://dl.rustfs.com/artifacts/rustfs/dev/"
else
print_message $YELLOW " https://dl.rustfs.com/artifacts/rustfs/release/"
fi
print_message $YELLOW " https://github.com/rustfs/rustfs/releases"
exit 1
fi
@@ -183,11 +179,7 @@ build_and_push() {
else
print_message $RED "❌ Failed to build release variant"
print_message $YELLOW "💡 Note: Make sure rustfs binaries are available at:"
if [ "$CHANNEL" = "dev" ]; then
print_message $YELLOW " https://dl.rustfs.com/artifacts/rustfs/dev/"
else
print_message $YELLOW " https://dl.rustfs.com/artifacts/rustfs/release/"
fi
print_message $YELLOW " https://github.com/rustfs/rustfs/releases"
exit 1
fi
else
@@ -248,7 +240,7 @@ done
# Main execution
main() {
print_message $BLUE "🐳 RustFS Docker Buildx Build Script"
print_message $YELLOW "📋 Build Strategy: Uses pre-built binaries from dl.rustfs.com"
print_message $YELLOW "📋 Build Strategy: Uses pre-built binaries from GitHub Releases"
print_message $YELLOW "🚀 Production images only - optimized for distribution"
echo ""

165
entrypoint.sh Normal file → Executable file
View File

@@ -1,104 +1,81 @@
#!/bin/bash
#!/bin/sh
set -e
APP_USER=rustfs
APP_GROUP=rustfs
APP_UID=${PUID:-1000}
APP_GID=${PGID:-1000}
# Parse RUSTFS_VOLUMES into array (support space, comma, tab as separator)
VOLUME_RAW="${RUSTFS_VOLUMES:-/data}"
# Replace comma and tab with space, then split
VOLUME_RAW=$(echo "$VOLUME_RAW" | tr ',\t' ' ')
read -ra ALL_VOLUMES <<< "$VOLUME_RAW"
# Only keep local volumes (start with /, not http/https)
LOCAL_VOLUMES=()
for vol in "${ALL_VOLUMES[@]}"; do
if [[ "$vol" =~ ^/ ]] && [[ ! "$vol" =~ ^https?:// ]]; then
# Not a URL (http/https), just a local path
LOCAL_VOLUMES+=("$vol")
fi
# If it's a URL (http/https), skip
# If it's an empty string, skip
# If it's a local path, keep
# (We don't support other protocols here)
done
# Always ensure /logs is included for permission fix
include_logs=1
for vol in "${LOCAL_VOLUMES[@]}"; do
if [ "$vol" = "/logs" ]; then
include_logs=0
break
fi
done
if [ $include_logs -eq 1 ]; then
LOCAL_VOLUMES+=("/logs")
# 1) Normalize command:
# - No arguments: default to execute rustfs
# - First argument starts with '-': treat as rustfs arguments, auto-prefix rustfs
# - First argument is 'rustfs': replace with absolute path to avoid PATH interference
if [ $# -eq 0 ] || [ "${1#-}" != "$1" ]; then
set -- /usr/bin/rustfs "$@"
elif [ "$1" = "rustfs" ]; then
shift
set -- /usr/bin/rustfs "$@"
fi
# Try to update rustfs UID/GID if needed (requires root and shadow tools)
update_user_group_ids() {
local uid="$1"
local gid="$2"
local user="$3"
local group="$4"
local updated=0
if [ "$(id -u "$user")" != "$uid" ]; then
if command -v usermod >/dev/null 2>&1; then
echo "🔧 Updating UID of $user to $uid"
usermod -u "$uid" "$user"
updated=1
# 2) Parse and create local mount directories (ignore http/https), ensure /logs is included
VOLUME_RAW="${RUSTFS_VOLUMES:-/data}"
# Convert comma/tab to space
VOLUME_LIST=$(echo "$VOLUME_RAW" | tr ',\t' ' ')
LOCAL_VOLUMES=""
for vol in $VOLUME_LIST; do
case "$vol" in
/*)
case "$vol" in
http://*|https://*) : ;;
*) LOCAL_VOLUMES="$LOCAL_VOLUMES $vol" ;;
esac
;;
*)
: # skip non-local paths
;;
esac
done
# Ensure /logs is included
case " $LOCAL_VOLUMES " in
*" /logs "*) : ;;
*) LOCAL_VOLUMES="$LOCAL_VOLUMES /logs" ;;
esac
echo "Initializing mount directories:$LOCAL_VOLUMES"
for vol in $LOCAL_VOLUMES; do
if [ ! -d "$vol" ]; then
echo " mkdir -p $vol"
mkdir -p "$vol"
# If target user is specified, try to set directory owner to that user (non-recursive to avoid large disk overhead)
if [ -n "$RUSTFS_UID" ] && [ -n "$RUSTFS_GID" ]; then
chown "$RUSTFS_UID:$RUSTFS_GID" "$vol" 2>/dev/null || true
elif [ -n "$RUSTFS_USERNAME" ] && [ -n "$RUSTFS_GROUPNAME" ]; then
chown "$RUSTFS_USERNAME:$RUSTFS_GROUPNAME" "$vol" 2>/dev/null || true
fi
fi
if [ "$(id -g "$group")" != "$gid" ]; then
if command -v groupmod >/dev/null 2>&1; then
echo "🔧 Updating GID of $group to $gid"
groupmod -g "$gid" "$group"
updated=1
done
# 3) Default credentials warning
if [ "${RUSTFS_ACCESS_KEY}" = "rustfsadmin" ] || [ "${RUSTFS_SECRET_KEY}" = "rustfsadmin" ]; then
echo "!!!WARNING: Using default RUSTFS_ACCESS_KEY or RUSTFS_SECRET_KEY. Override them in production!"
fi
# 4) Start with specified user
docker_switch_user() {
if [ -n "${RUSTFS_USERNAME}" ] && [ -n "${RUSTFS_GROUPNAME}" ]; then
if [ -n "${RUSTFS_UID}" ] && [ -n "${RUSTFS_GID}" ]; then
# Execute with numeric UID:GID directly (doesn't depend on user existing in system)
exec chroot --userspec="${RUSTFS_UID}:${RUSTFS_GID}" / "$@"
else
# When only names are provided, create minimal passwd/group entries with 1000:1000; deduplicate before writing
if ! grep -q "^${RUSTFS_USERNAME}:" /etc/passwd 2>/dev/null; then
echo "${RUSTFS_USERNAME}:x:1000:1000:${RUSTFS_USERNAME}:/nonexistent:/sbin/nologin" >> /etc/passwd
fi
if ! grep -q "^${RUSTFS_GROUPNAME}:" /etc/group 2>/dev/null; then
echo "${RUSTFS_GROUPNAME}:x:1000:" >> /etc/group
fi
exec chroot --userspec="${RUSTFS_USERNAME}:${RUSTFS_GROUPNAME}" / "$@"
fi
else
# If no user is specified, keep as root (container has minimal privilege practices that can be configured separately)
exec "$@"
fi
return $updated
}
echo "📦 Initializing mount directories: ${LOCAL_VOLUMES[*]}"
for vol in "${LOCAL_VOLUMES[@]}"; do
if [ ! -d "$vol" ]; then
echo "📁 Creating directory: $vol"
mkdir -p "$vol"
fi
# Alpine busybox stat does not support -c, coreutils is required
dir_uid=$(stat -c '%u' "$vol")
dir_gid=$(stat -c '%g' "$vol")
if [ "$dir_uid" != "$APP_UID" ] || [ "$dir_gid" != "$APP_GID" ]; then
if [[ "$SKIP_CHOWN" != "true" ]]; then
# Prefer to update rustfs user/group UID/GID
update_user_group_ids "$dir_uid" "$dir_gid" "$APP_USER" "$APP_GROUP" || \
{
echo "🔧 Fixing ownership for: $vol$APP_USER:$APP_GROUP"
if [[ -n "$CHOWN_RECURSION_DEPTH" ]]; then
echo "🔧 Applying ownership fix with recursion depth: $CHOWN_RECURSION_DEPTH"
find "$vol" -mindepth 0 -maxdepth "$CHOWN_RECURSION_DEPTH" -exec chown "$APP_USER:$APP_GROUP" {} \;
else
echo "🔧 Applying ownership fix recursively (full depth)"
chown -R "$APP_USER:$APP_GROUP" "$vol"
fi
}
else
echo "⚠️ SKIP_CHOWN is enabled. Skipping ownership fix for: $vol"
fi
fi
chmod 700 "$vol"
done
# Warn if default credentials are used
if [[ "$RUSTFS_ACCESS_KEY" == "rustfsadmin" || "$RUSTFS_SECRET_KEY" == "rustfsadmin" ]]; then
echo "⚠️ WARNING: Using default RUSTFS_ACCESS_KEY or RUSTFS_SECRET_KEY"
echo "⚠️ It is strongly recommended to override these values in production!"
fi
echo "🚀 Starting application: $*"
exec gosu "$APP_USER" "$@"
echo "Starting: $*"
docker_switch_user "$@"

View File

@@ -311,7 +311,7 @@ impl S3 for FS {
.make_bucket(
&bucket,
&MakeBucketOptions {
force_create: true,
force_create: false, // TODO: force support
lock_enabled: object_lock_enabled_for_bucket.is_some_and(|v| v),
..Default::default()
},
@@ -984,6 +984,7 @@ impl S3 for FS {
accept_ranges: Some("bytes".to_string()),
content_range,
e_tag: info.etag,
metadata: Some(info.user_defined),
..Default::default()
};

View File

@@ -7,21 +7,21 @@ type VersionParseResult = Result<(u32, u32, u32, Option<String>), Box<dyn std::e
#[allow(clippy::const_is_empty)]
pub fn get_version() -> String {
// 获取最新的 tag
// Get the latest tag
if let Ok(latest_tag) = get_latest_tag() {
// 检查当前 commit 是否比最新 tag 更新
// Check if current commit is newer than the latest tag
if is_head_newer_than_tag(&latest_tag) {
// 如果当前 commit 更新,则提升版本号
// If current commit is newer, increment the version number
if let Ok(new_version) = increment_version(&latest_tag) {
return format!("refs/tags/{new_version}");
}
}
// 如果当前 commit 就是最新 tag或者版本提升失败返回当前 tag
// If current commit is the latest tag, or version increment failed, return current tag
return format!("refs/tags/{latest_tag}");
}
// 如果没有 tag使用原来的逻辑
// If no tag exists, use original logic
if !build::TAG.is_empty() {
format!("refs/tags/{}", build::TAG)
} else if !build::SHORT_COMMIT.is_empty() {
@@ -31,7 +31,7 @@ pub fn get_version() -> String {
}
}
/// 获取最新的 git tag
/// Get the latest git tag
fn get_latest_tag() -> Result<String, Box<dyn std::error::Error>> {
let output = Command::new("git").args(["describe", "--tags", "--abbrev=0"]).output()?;
@@ -43,7 +43,7 @@ fn get_latest_tag() -> Result<String, Box<dyn std::error::Error>> {
}
}
/// 检查当前 HEAD 是否比指定的 tag 更新
/// Check if current HEAD is newer than specified tag
fn is_head_newer_than_tag(tag: &str) -> bool {
let output = Command::new("git")
.args(["merge-base", "--is-ancestor", tag, "HEAD"])
@@ -55,23 +55,23 @@ fn is_head_newer_than_tag(tag: &str) -> bool {
}
}
/// 提升版本号(增加 patch 版本)
/// Increment version number (increase patch version)
fn increment_version(version: &str) -> Result<String, Box<dyn std::error::Error>> {
// 解析版本号,例如 "1.0.0-alpha.19" -> (1, 0, 0, Some("alpha.19"))
// Parse version number, e.g. "1.0.0-alpha.19" -> (1, 0, 0, Some("alpha.19"))
let (major, minor, patch, pre_release) = parse_version(version)?;
// 如果有预发布标识符,则增加预发布版本号
// If there's a pre-release identifier, increment the pre-release version number
if let Some(pre) = pre_release {
if let Some(new_pre) = increment_pre_release(&pre) {
return Ok(format!("{major}.{minor}.{patch}-{new_pre}"));
}
}
// 否则增加 patch 版本号
// Otherwise increment patch version number
Ok(format!("{major}.{minor}.{}", patch + 1))
}
/// 解析版本号
/// Parse version number
pub fn parse_version(version: &str) -> VersionParseResult {
let parts: Vec<&str> = version.split('-').collect();
let base_version = parts[0];
@@ -89,9 +89,9 @@ pub fn parse_version(version: &str) -> VersionParseResult {
Ok((major, minor, patch, pre_release))
}
/// 增加预发布版本号
/// Increment pre-release version number
fn increment_pre_release(pre_release: &str) -> Option<String> {
// 处理形如 "alpha.19" 的预发布版本
// Handle pre-release versions like "alpha.19"
let parts: Vec<&str> = pre_release.split('.').collect();
if parts.len() == 2 {
if let Ok(num) = parts[1].parse::<u32>() {
@@ -99,7 +99,7 @@ fn increment_pre_release(pre_release: &str) -> Option<String> {
}
}
// 处理形如 "alpha19" 的预发布版本
// Handle pre-release versions like "alpha19"
if let Some(pos) = pre_release.rfind(|c: char| c.is_alphabetic()) {
let prefix = &pre_release[..=pos];
let suffix = &pre_release[pos + 1..];
@@ -208,14 +208,14 @@ mod tests {
#[test]
fn test_parse_version() {
// 测试标准版本解析
// Test standard version parsing
let (major, minor, patch, pre_release) = parse_version("1.0.0").unwrap();
assert_eq!(major, 1);
assert_eq!(minor, 0);
assert_eq!(patch, 0);
assert_eq!(pre_release, None);
// 测试预发布版本解析
// Test pre-release version parsing
let (major, minor, patch, pre_release) = parse_version("1.0.0-alpha.19").unwrap();
assert_eq!(major, 1);
assert_eq!(minor, 0);
@@ -225,32 +225,32 @@ mod tests {
#[test]
fn test_increment_pre_release() {
// 测试 alpha.19 -> alpha.20
// Test alpha.19 -> alpha.20
assert_eq!(increment_pre_release("alpha.19"), Some("alpha.20".to_string()));
// 测试 beta.5 -> beta.6
// Test beta.5 -> beta.6
assert_eq!(increment_pre_release("beta.5"), Some("beta.6".to_string()));
// 测试无法解析的情况
// Test unparsable case
assert_eq!(increment_pre_release("unknown"), None);
}
#[test]
fn test_increment_version() {
// 测试预发布版本递增
// Test pre-release version increment
assert_eq!(increment_version("1.0.0-alpha.19").unwrap(), "1.0.0-alpha.20");
// 测试标准版本递增
// Test standard version increment
assert_eq!(increment_version("1.0.0").unwrap(), "1.0.1");
}
#[test]
fn test_version_format() {
// 测试版本格式是否以 refs/tags/ 开头
// Test if version format starts with refs/tags/
let version = get_version();
assert!(version.starts_with("refs/tags/") || version.starts_with("@"));
// 如果是 refs/tags/ 格式,应该包含版本号
// If it's refs/tags/ format, should contain version number
if let Some(version_part) = version.strip_prefix("refs/tags/") {
assert!(!version_part.is_empty());
}
@@ -258,14 +258,14 @@ mod tests {
#[test]
fn test_current_version_output() {
// 显示当前版本输出
// Display current version output
let version = get_version();
println!("Current version: {version}");
// 验证版本格式
// Verify version format
assert!(version.starts_with("refs/tags/") || version.starts_with("@"));
// 如果是 refs/tags/ 格式,验证版本号不为空
// If it's refs/tags/ format, verify version number is not empty
if let Some(version_part) = version.strip_prefix("refs/tags/") {
assert!(!version_part.is_empty());
println!("Version part: {version_part}");

View File

@@ -14,12 +14,12 @@
for i in {0..3}; do
DIR="/data/rustfs$i"
echo "处理 $DIR"
echo "Processing $DIR"
if [ -d "$DIR" ]; then
echo "清空 $DIR"
echo "Clearing $DIR"
sudo rm -rf "$DIR"/* "$DIR"/.[!.]* "$DIR"/..?* 2>/dev/null || true
echo "已清空 $DIR"
echo "Cleared $DIR"
else
echo "$DIR 不存在,跳过"
echo "$DIR does not exist, skipping"
fi
done

View File

@@ -14,32 +14,32 @@
# limitations under the License.
# 脚本名称:scp_to_servers.sh
# Script name: scp_to_servers.sh
rm ./target/x86_64-unknown-linux-gnu/release/rustfs.zip
# 压缩./target/x86_64-unknown-linux-gnu/release/rustfs
# Compress ./target/x86_64-unknown-linux-gnu/release/rustfs
zip -j ./target/x86_64-unknown-linux-gnu/release/rustfs.zip ./target/x86_64-unknown-linux-gnu/release/rustfs
# 上传到服务器
# Upload to server
LOCAL_FILE="./target/x86_64-unknown-linux-gnu/release/rustfs.zip"
REMOTE_PATH="~"
# 必须传入IP参数否则报错退出
# IP parameter must be provided, otherwise exit with error
if [ -z "$1" ]; then
echo "用法: $0 <server_ip>"
echo "请传入目标服务器IP地址"
echo "Usage: $0 <server_ip>"
echo "Please provide target server IP address"
exit 1
fi
SERVER_LIST=("root@$1")
# 遍历服务器列表
# Iterate through server list
for SERVER in "${SERVER_LIST[@]}"; do
echo "正在将文件复制到服务器:$SERVER 目标路径:$REMOTE_PATH"
echo "Copying file to server: $SERVER target path: $REMOTE_PATH"
scp "$LOCAL_FILE" "${SERVER}:${REMOTE_PATH}"
if [ $? -eq 0 ]; then
echo "成功复制到 $SERVER"
echo "Successfully copied to $SERVER"
else
echo "复制到 $SERVER 失败"
echo "Failed to copy to $SERVER"
fi
done

View File

@@ -16,9 +16,9 @@
# ps -ef | grep rustfs | awk '{print $2}'| xargs kill -9
# 本地 rustfs.zip 路径
# Local rustfs.zip path
ZIP_FILE="./rustfs.zip"
# 解压目标
# Unzip target
UNZIP_TARGET="./"
@@ -35,119 +35,119 @@ SERVER_LIST=(
REMOTE_TMP="~/rustfs"
# 部署 rustfs 到所有服务器
# Deploy rustfs to all servers
deploy() {
echo "解压 $ZIP_FILE ..."
echo "Unzipping $ZIP_FILE ..."
unzip -o "$ZIP_FILE" -d "$UNZIP_TARGET"
if [ $? -ne 0 ]; then
echo "解压失败,退出"
echo "Unzip failed, exiting"
exit 1
fi
LOCAL_RUSTFS="${UNZIP_TARGET}rustfs"
if [ ! -f "$LOCAL_RUSTFS" ]; then
echo "未找到解压后的 rustfs 文件,退出"
echo "Unzipped rustfs file not found, exiting"
exit 1
fi
for SERVER in "${SERVER_LIST[@]}"; do
echo "上传 $LOCAL_RUSTFS $SERVER:$REMOTE_TMP"
echo "Uploading $LOCAL_RUSTFS to $SERVER:$REMOTE_TMP"
scp "$LOCAL_RUSTFS" "${SERVER}:${REMOTE_TMP}"
if [ $? -ne 0 ]; then
echo "上传到 $SERVER 失败,跳过"
echo "Upload to $SERVER failed, skipping"
continue
fi
echo "$SERVER 上操作 systemctl 和文件替换"
echo "Operating systemctl and file replacement on $SERVER"
ssh "$SERVER" bash <<EOF
set -e
echo "停止 rustfs 服务"
echo "Stopping rustfs service"
sudo systemctl stop rustfs || true
echo "覆盖 /usr/local/bin/rustfs"
echo "Overwriting /usr/local/bin/rustfs"
sudo cp ~/rustfs /usr/local/bin/rustfs
sudo chmod +x /usr/local/bin/rustfs
echo "启动 rustfs 服务"
echo "Starting rustfs service"
sudo systemctl start rustfs
echo "检测 rustfs 服务状态"
echo "Checking rustfs service status"
sudo systemctl status rustfs --no-pager --lines=10
EOF
if [ $? -eq 0 ]; then
echo "$SERVER 部署并重启 rustfs 成功"
echo "$SERVER deployed and restarted rustfs successfully"
else
echo "$SERVER 部署或重启 rustfs 失败"
echo "$SERVER failed to deploy or restart rustfs"
fi
done
}
# 清空 /data/rustfs0~3 目录下所有文件(包括隐藏文件)
# Clear all files (including hidden files) in /data/rustfs0~3 directories
clear_data_dirs() {
for SERVER in "${SERVER_LIST[@]}"; do
echo "清空 $SERVER:/data/rustfs0~3 下所有文件"
echo "Clearing all files in $SERVER:/data/rustfs0~3"
ssh "$SERVER" bash <<EOF
for i in {0..3}; do
DIR="/data/rustfs$i"
echo "处理 $DIR"
if [ -d "$DIR" ]; then
echo "清空 $DIR"
sudo rm -rf "$DIR"/* "$DIR"/.[!.]* "$DIR"/..?* 2>/dev/null || true
echo "已清空 $DIR"
DIR="/data/rustfs\$i"
echo "Processing \$DIR"
if [ -d "\$DIR" ]; then
echo "Clearing \$DIR"
sudo rm -rf "\$DIR"/* "\$DIR"/.[!.]* "\$DIR"/..?* 2>/dev/null || true
echo "Cleared \$DIR"
else
echo "$DIR 不存在,跳过"
echo "\$DIR does not exist, skipping"
fi
done
EOF
done
}
# 控制 rustfs 服务
# Control rustfs service
stop_rustfs() {
for SERVER in "${SERVER_LIST[@]}"; do
echo "停止 $SERVER rustfs 服务"
echo "Stopping $SERVER rustfs service"
ssh "$SERVER" "sudo systemctl stop rustfs"
done
}
start_rustfs() {
for SERVER in "${SERVER_LIST[@]}"; do
echo "启动 $SERVER rustfs 服务"
echo "Starting $SERVER rustfs service"
ssh "$SERVER" "sudo systemctl start rustfs"
done
}
restart_rustfs() {
for SERVER in "${SERVER_LIST[@]}"; do
echo "重启 $SERVER rustfs 服务"
echo "Restarting $SERVER rustfs service"
ssh "$SERVER" "sudo systemctl restart rustfs"
done
}
# 向所有服务器追加公钥到 ~/.ssh/authorized_keys
# Append public key to ~/.ssh/authorized_keys on all servers
add_ssh_key() {
if [ -z "$2" ]; then
echo "用法: $0 addkey <pubkey_file>"
echo "Usage: $0 addkey <pubkey_file>"
exit 1
fi
PUBKEY_FILE="$2"
if [ ! -f "$PUBKEY_FILE" ]; then
echo "指定的公钥文件不存在: $PUBKEY_FILE"
echo "Specified public key file does not exist: $PUBKEY_FILE"
exit 1
fi
PUBKEY_CONTENT=$(cat "$PUBKEY_FILE")
for SERVER in "${SERVER_LIST[@]}"; do
echo "追加公钥到 $SERVER:~/.ssh/authorized_keys"
echo "Appending public key to $SERVER:~/.ssh/authorized_keys"
ssh "$SERVER" "mkdir -p ~/.ssh && chmod 700 ~/.ssh && echo '$PUBKEY_CONTENT' >> ~/.ssh/authorized_keys && chmod 600 ~/.ssh/authorized_keys"
if [ $? -eq 0 ]; then
echo "$SERVER 公钥追加成功"
echo "$SERVER public key appended successfully"
else
echo "$SERVER 公钥追加失败"
echo "$SERVER public key append failed"
fi
done
}
monitor_logs() {
for SERVER in "${SERVER_LIST[@]}"; do
echo "监控 $SERVER:/var/logs/rustfs/rustfs.log ..."
echo "Monitoring $SERVER:/var/logs/rustfs/rustfs.log ..."
ssh "$SERVER" "tail -F /var/logs/rustfs/rustfs.log" |
sed "s/^/[$SERVER] /" &
done
@@ -156,32 +156,32 @@ monitor_logs() {
set_env_file() {
if [ -z "$2" ]; then
echo "用法: $0 setenv <env_file>"
echo "Usage: $0 setenv <env_file>"
exit 1
fi
ENV_FILE="$2"
if [ ! -f "$ENV_FILE" ]; then
echo "指定的环境变量文件不存在: $ENV_FILE"
echo "Specified environment variable file does not exist: $ENV_FILE"
exit 1
fi
for SERVER in "${SERVER_LIST[@]}"; do
echo "上传 $ENV_FILE $SERVER:~/rustfs.env"
echo "Uploading $ENV_FILE to $SERVER:~/rustfs.env"
scp "$ENV_FILE" "${SERVER}:~/rustfs.env"
if [ $? -ne 0 ]; then
echo "上传到 $SERVER 失败,跳过"
echo "Upload to $SERVER failed, skipping"
continue
fi
echo "覆盖 $SERVER:/etc/default/rustfs"
echo "Overwriting $SERVER:/etc/default/rustfs"
ssh "$SERVER" "sudo mv ~/rustfs.env /etc/default/rustfs"
if [ $? -eq 0 ]; then
echo "$SERVER /etc/default/rustfs 覆盖成功"
echo "$SERVER /etc/default/rustfs overwritten successfully"
else
echo "$SERVER /etc/default/rustfs 覆盖失败"
echo "$SERVER /etc/default/rustfs overwrite failed"
fi
done
}
# 主命令分发
# Main command dispatcher
case "$1" in
deploy)
deploy
@@ -208,6 +208,6 @@ case "$1" in
set_env_file "$@"
;;
*)
echo "用法: $0 {deploy|clear|stop|start|restart|addkey <pubkey_file>|monitor_logs|setenv <env_file>}"
echo "Usage: $0 {deploy|clear|stop|start|restart|addkey <pubkey_file>|monitor_logs|setenv <env_file>}"
;;
esac

View File

@@ -26,6 +26,6 @@ fi
echo "Creating log directory if it does not exist..."
mkdir -p "$current_dir/deploy/logs/notify"
# 启动 webhook 服务器
# Start webhook server
echo "Starting webhook server..."
cargo run --example webhook -p rustfs-notify &

View File

@@ -47,19 +47,19 @@ export RUSTFS_ADDRESS=":9000"
export RUSTFS_CONSOLE_ENABLE=true
export RUSTFS_CONSOLE_ADDRESS=":9001"
# export RUSTFS_SERVER_DOMAINS="localhost:9000"
# HTTPS 证书目录
# HTTPS certificate directory
# export RUSTFS_TLS_PATH="./deploy/certs"
# 可观测性 相关配置信息
#export RUSTFS_OBS_ENDPOINT=http://localhost:4317 # OpenTelemetry Collector 的地址
#export RUSTFS_OBS_USE_STDOUT=false # 是否使用标准输出
#export RUSTFS_OBS_SAMPLE_RATIO=2.0 # 采样率0.0-1.0之间0.0表示不采样1.0表示全部采样
#export RUSTFS_OBS_METER_INTERVAL=1 # 采样间隔,单位为秒
#export RUSTFS_OBS_SERVICE_NAME=rustfs # 服务名称
#export RUSTFS_OBS_SERVICE_VERSION=0.1.0 # 服务版本
export RUSTFS_OBS_ENVIRONMENT=develop # 环境名称
export RUSTFS_OBS_LOGGER_LEVEL=info # 日志级别,支持 trace, debug, info, warn, error
export RUSTFS_OBS_LOCAL_LOGGING_ENABLED=true # 是否启用本地日志记录
# Observability related configuration
#export RUSTFS_OBS_ENDPOINT=http://localhost:4317 # OpenTelemetry Collector address
#export RUSTFS_OBS_USE_STDOUT=false # Whether to use standard output
#export RUSTFS_OBS_SAMPLE_RATIO=2.0 # Sample ratio, between 0.0-1.0, 0.0 means no sampling, 1.0 means full sampling
#export RUSTFS_OBS_METER_INTERVAL=1 # Sampling interval in seconds
#export RUSTFS_OBS_SERVICE_NAME=rustfs # Service name
#export RUSTFS_OBS_SERVICE_VERSION=0.1.0 # Service version
export RUSTFS_OBS_ENVIRONMENT=develop # Environment name
export RUSTFS_OBS_LOGGER_LEVEL=info # Log level, supports trace, debug, info, warn, error
export RUSTFS_OBS_LOCAL_LOGGING_ENABLED=true # Whether to enable local logging
export RUSTFS_OBS_LOG_DIRECTORY="$current_dir/deploy/logs" # Log directory
export RUSTFS_OBS_LOG_ROTATION_TIME="hour" # Log rotation time unit, can be "second", "minute", "hour", "day"
export RUSTFS_OBS_LOG_ROTATION_SIZE_MB=100 # Log rotation size in MB
@@ -89,34 +89,34 @@ export OTEL_INSTRUMENTATION_SCHEMA_URL="https://opentelemetry.io/schemas/1.31.0"
export OTEL_INSTRUMENTATION_ATTRIBUTES="env=production"
# notify
export RUSTFS_NOTIFY_WEBHOOK_ENABLE="on" # 是否启用 webhook 通知
export RUSTFS_NOTIFY_WEBHOOK_ENDPOINT="http://[::]:3020/webhook" # webhook 通知地址
export RUSTFS_NOTIFY_WEBHOOK_ENABLE="on" # Whether to enable webhook notification
export RUSTFS_NOTIFY_WEBHOOK_ENDPOINT="http://[::]:3020/webhook" # Webhook notification address
export RUSTFS_NOTIFY_WEBHOOK_QUEUE_DIR="$current_dir/deploy/logs/notify"
export RUSTFS_NOTIFY_WEBHOOK_ENABLE_PRIMARY="on" # 是否启用 webhook 通知
export RUSTFS_NOTIFY_WEBHOOK_ENDPOINT_PRIMARY="http://[::]:3020/webhook" # webhook 通知地址
export RUSTFS_NOTIFY_WEBHOOK_ENABLE_PRIMARY="on" # Whether to enable webhook notification
export RUSTFS_NOTIFY_WEBHOOK_ENDPOINT_PRIMARY="http://[::]:3020/webhook" # Webhook notification address
export RUSTFS_NOTIFY_WEBHOOK_QUEUE_DIR_PRIMARY="$current_dir/deploy/logs/notify"
export RUSTFS_NOTIFY_WEBHOOK_ENABLE_MASTER="on" # 是否启用 webhook 通知
export RUSTFS_NOTIFY_WEBHOOK_ENDPOINT_MASTER="http://[::]:3020/webhook" # webhook 通知地址
export RUSTFS_NOTIFY_WEBHOOK_ENABLE_MASTER="on" # Whether to enable webhook notification
export RUSTFS_NOTIFY_WEBHOOK_ENDPOINT_MASTER="http://[::]:3020/webhook" # Webhook notification address
export RUSTFS_NOTIFY_WEBHOOK_QUEUE_DIR_MASTER="$current_dir/deploy/logs/notify"
export RUSTFS_NS_SCANNER_INTERVAL=60 # 对象扫描间隔时间,单位为秒
export RUSTFS_NS_SCANNER_INTERVAL=60 # Object scanning interval in seconds
# exportRUSTFS_SKIP_BACKGROUND_TASK=true
export RUSTFS_COMPRESSION_ENABLED=true # 是否启用压缩
export RUSTFS_COMPRESSION_ENABLED=true # Whether to enable compression
#export RUSTFS_REGION="us-east-1"
# 事件消息配置
# Event message configuration
#export RUSTFS_EVENT_CONFIG="./deploy/config/event.example.toml"
if [ -n "$1" ]; then
export RUSTFS_VOLUMES="$1"
fi
# 启动 webhook 服务器
# Start webhook server
#cargo run --example webhook -p rustfs-notify &
# 启动主服务
# Start main service
cargo run --bin rustfs

View File

@@ -1,54 +0,0 @@
#!/bin/bash
echo "🔍 验证所有PR分支的CI状态..."
branches=(
"feature/add-auth-module-tests"
"feature/add-storage-core-tests"
"feature/add-admin-handlers-tests"
"feature/add-server-components-tests"
"feature/add-integration-tests"
)
cd /workspace
for branch in "${branches[@]}"; do
echo ""
echo "🌟 检查分支: $branch"
git checkout $branch 2>/dev/null
echo "📝 检查代码格式..."
if cargo fmt --all --check; then
echo "✅ 代码格式正确"
else
echo "❌ 代码格式有问题"
fi
echo "🔧 检查基本编译..."
if cargo check --quiet; then
echo "✅ 基本编译通过"
else
echo "❌ 编译失败"
fi
echo "🧪 运行核心测试..."
if timeout 60 cargo test --lib --quiet 2>/dev/null; then
echo "✅ 核心测试通过"
else
echo "⚠️ 测试超时或失败(可能是依赖问题)"
fi
done
echo ""
echo "🎉 所有分支检查完毕!"
echo ""
echo "📋 PR状态总结:"
echo "- PR #309: feature/add-auth-module-tests"
echo "- PR #313: feature/add-storage-core-tests"
echo "- PR #314: feature/add-admin-handlers-tests"
echo "- PR #315: feature/add-server-components-tests"
echo "- PR #316: feature/add-integration-tests"
echo ""
echo "✅ 所有冲突已解决,代码已格式化"
echo "🔗 请检查GitHub上的CI状态"