* feat: disable Docker builds for development versions
- Remove dev-latest, main-latest, and dev-* version options from manual triggers
- Skip Docker builds for development versions in workflow_run events
- Only build Docker images for releases (v1.0.0) and prereleases (v1.0.0-alpha1)
- Simplify tags generation logic by removing development branch handling
- Update workflow documentation to reflect release-only Docker strategy
BREAKING CHANGE: Development Docker images are no longer built automatically
* feat: remove dev channel support from Dockerfile
- Remove CHANNEL build argument (no longer needed)
- Simplify download logic to only support release channel
- Remove dev-specific package download paths
- Update BASE_URL to point directly to release directory
- Remove channel label from Docker image metadata
- Streamline version handling (latest vs specific release)
This aligns with the workflow changes that disabled dev Docker builds.
- Replace manual cut -c1-7 with git rev-parse --short in docker.yml
- Ensures consistent short SHA length between build.yml and docker.yml
- Git automatically adjusts length for uniqueness, preventing conflicts
- Remove linux_builds_success output and related variables
- Simplify build-docker condition to only check should_build
- The should_build check already includes workflow success verification
- Reduce code complexity while maintaining the same functionality
* fix: correct YAML indentation error in docker workflow
- Fix incorrect indentation at line 237 in .github/workflows/docker.yml
- Step 'Extract metadata and generate tags' had 12 spaces instead of 6
- This was causing YAML syntax validation to fail
* fix: restore unified build-rustfs task with correct YAML syntax
- Revert complex job separation back to single build-rustfs task
- Maintain Linux and macOS builds in unified matrix
- Fix YAML indentation and syntax issues
- Docker builds will use only Linux binaries as designed in Dockerfile
* feat: implement precise Docker build triggering using workflow_run
- Use workflow_run event to trigger Docker builds independently
- Add precise Linux build status checking via GitHub API
- Only trigger Docker builds when both Linux architectures succeed
- Remove coupling between build.yml and docker.yml workflows
- Improve TARGETPLATFORM consistency in Dockerfile
This resolves the issue where Docker builds would trigger even if
Linux ARM64 builds failed, causing missing binary artifacts during
multi-architecture Docker image creation.
- Fix incorrect indentation at line 237 in .github/workflows/docker.yml
- Step 'Extract metadata and generate tags' had 12 spaces instead of 6
- This was causing YAML syntax validation to fail
- Change docker.yml to use workflow_call triggered by build.yml
- Remove redundant force_build parameter from build.yml
- Simplify build_docker parameter (build implies push in CI/CD)
- Add proper dependency chain: build.yml -> docker.yml -> registry
- Update documentation to reflect new architecture
- Mark Dockerfile.source as local development only
- Remove cargo.config.toml file that was causing build issues
- Restore .github/workflows/build.yml to working state from commit 2e9792577f
- These changes ensure the build system works correctly again
* fix: Resolve zstd-sys Zig compilation issues
- Remove specific Zig version constraint in action.yml to use default version
- Clean up duplicate environment variable settings in build-rustfs.sh
- Add CARGO_TARGET_*_LINKER environment variables for better cross-compilation support
- Optimize build configuration for consistent cross-platform compilation
Fixes compilation issues with zstd-sys when using Zig cross-compilation.
Aligns with previously working configuration that uses default Zig version.
* fix: Restore working build configuration from 4fb4b353
- Restore matrix.cross parameter to differentiate cross-compilation
- Use simple cargo zigbuild instead of complex build-rustfs.sh script
- Remove unnecessary zstd dependencies from action.yml
- Restore console asset download step
- Use correct target directory path for packaging
- Align with known working configuration from commit 4fb4b353
This reverts to the proven working build approach that successfully
performed cross-platform compilation.
* fix: Align build-rustfs.sh with working version logic
- Simplify build logic to match working version 4fb4b353
- Use exact same build commands as the working build.yml:
* cargo build for native compilation
* cargo zigbuild for Linux ARM64 cross-compilation
* cross build for Windows ARM64 cross-compilation
- Remove complex environment variable setup that caused conflicts
- Add touch rustfs/build.rs to match working version
- Use -p rustfs --bins flag consistent with working version
This ensures build-rustfs.sh (if used) follows the proven working approach.
- Update to mlugg/setup-zig@v2 for better stability and features
- Use Zig 0.13.0 for improved musl target support
- Add system zstd libraries (libzstd-dev, zstd) to Ubuntu dependencies
- Configure environment variables for zstd-sys to use pkg-config
- Enable pkg-config feature for zstd dependency to prefer system library
- Add proper C/C++ compiler configuration for musl targets
Fixes the 'error: unable to parse target query x86_64-unknown-linux-musl: UnknownOperatingSystem'
compilation error in zstd-sys during cross-compilation.
- Add generate_sha256() function to handle cross-platform SHA256 generation
- Use shasum -a 256 on macOS instead of sha256sum
- Use sha256sum on Linux with shasum as fallback
- Replace direct sha256sum usage in build script with new function
- Fixes 'sha256sum: command not found' error on macOS builds
* feat: implement Docker improvements and binary build scripts
This commit transforms the RustFS Docker build system to follow MinIO's best practices:
## 🏗️ Binary Build Script (build-rustfs.sh)
- Create independent binary compilation script for multi-platform builds
- Support x86_64 and aarch64 Linux musl targets
- Include checksum generation and optional binary signing
- Support cross-compilation and upload functionality
- Automated target installation and environment setup
## 🐳 Docker Improvements
- Rewrite Dockerfiles to download precompiled binaries instead of building from source
- Follow MinIO's approach for security and binary verification
- Add comprehensive LABEL metadata (version, build-date, vcs-ref)
- Implement proper environment variable management
- Add signature verification with minisign (commented for future use)
- Include static curl download for minimal runtime dependencies
## 🚀 Enhanced Build Script (docker-buildx.sh)
- Inspired by MinIO's docker-buildx.sh for consistency and reliability
- Support multiple platforms with proper build arguments
- Auto-detect git versions and pass metadata to containers
- Improved error messages with helpful troubleshooting hints
- Cleanup and cache management between builds
## 🛠️ Supporting Scripts
- scripts/download-static-curl.sh: Download statically compiled curl
- scripts/setup-test-binaries.sh: Create test binaries for local development
## 📋 Key Benefits
- Faster Docker builds (download vs compile)
- Better security with signature verification
- Consistent with industry standards (MinIO approach)
- Proper multi-platform support
- Enhanced metadata and traceability
- Independent binary distribution capability
* feat: update Docker files to use Aliyun OSS for binary downloads
* feat: merge stash with OSS binary download improvements
- Remove old build_rustfs.sh script
- Keep Aliyun OSS download URLs for binary retrieval
- Maintain Docker build improvements from stash
- Resolve merge conflicts between stash and OSS updates
* feat: improve build-rustfs.sh with auto platform detection
- Auto-detect current platform using uname (like old build_rustfs.sh)
- Default to building for current platform only
- Add --all-platforms flag for cross-compilation to Linux musl targets
- Support macOS (darwin) and Linux platforms
- Auto-enable cross compilation when needed
- Provide better usage examples and platform detection info
This makes the script much more user-friendly by default while
maintaining flexibility for cross-compilation scenarios.
* refactor: simplify build-rustfs.sh for CI/CD pipeline usage
- Remove cross-compilation complexity (each CI runner builds natively)
- Focus on single platform builds per runner
- Remove --all-platforms and --cross options
- Simplify to match CI/CD workflow where:
* Linux x86_64 runner builds Linux x86_64 binary
* Linux ARM64 runner builds Linux ARM64 binary
* macOS x86_64 runner builds macOS x86_64 binary
* macOS ARM64 runner builds macOS ARM64 binary
- Keep signing and upload functionality for release CI
- Make the script's purpose and usage clearer
This aligns with the user's understanding that build scripts should
focus on native compilation for the current platform only.
* feat: update download server domain to dl.rustfs.com
- Update Dockerfile to use dl.rustfs.com/dev/ for development binaries
- Update Dockerfile.release to use dl.rustfs.com/release/ for release binaries
- Update docker-buildx.sh error messages with new URLs
- Update build-rustfs.sh upload target to dl.rustfs.com
- Update test scripts to reference new domain
- Clean up remaining git conflict markers
This centralizes all binary downloads through the official
dl.rustfs.com domain instead of direct OSS access.
* fix: correct dl.rustfs.com path structure to include /artifacts/rustfs/
- Update all download URLs to use correct path structure:
* Dev: https://dl.rustfs.com/artifacts/rustfs/dev/
* Release: https://dl.rustfs.com/artifacts/rustfs/release/
- Test confirmed both paths return HTTP 200 with application/zip content-type
- Update Dockerfile, Dockerfile.release, docker-buildx.sh, and build-rustfs.sh
- Update test scripts with correct base path
The dl.rustfs.com domain requires the /artifacts/rustfs/ prefix
to access the binary files correctly.
* feat: refactor Dockerfile to download binaries from GitHub Releases
- Changed binary download source from dl.rustfs.com to GitHub Releases
- Added support for latest release auto-detection via GitHub API
- Enhanced error handling with detailed messages and helpful links
- Added optional checksum verification using SHA256SUMS
- Improved architecture support for amd64 and arm64
- Removed unnecessary minisign installation
- Added jq dependency for JSON parsing
* feat: consolidate Docker build to use single Dockerfile
- Removed Dockerfile.release and use unified Dockerfile instead
- Updated docker-buildx.sh to use single Dockerfile with build args
- Both latest and release variants now use GitHub Releases
- Simplified build process and reduced maintenance overhead
- Updated error messages to point to GitHub releases
* chore: remove unused Dockerfile.obs
- Removed Dockerfile.obs as it's no longer needed
- Simplified Docker build configuration
* feat: unify Docker prebuild variants to use GitHub Releases
- Updated .docker/alpine/Dockerfile.prebuild to download from GitHub Releases
- Updated .docker/ubuntu/Dockerfile.prebuild to download from GitHub Releases
- All prebuild variants now consistently use GitHub Releases as binary source
- Added checksum verification for all prebuild variants
- Updated .docker/README.md to reflect unified GitHub Releases approach
- Improved error handling and user guidance in all prebuild Dockerfiles
* feat: major Docker structure simplification and consolidation
## 🎯 Simplified Docker Structure
Moved from complex multi-directory structure to clean root-level organization:
### Before:
- Dockerfile (production)
- .docker/alpine/Dockerfile.prebuild (duplicate)
- .docker/alpine/Dockerfile.source
- .docker/ubuntu/Dockerfile.prebuild (duplicate)
- .docker/ubuntu/Dockerfile.source
- .docker/ubuntu/Dockerfile.dev
### After:
- Dockerfile (production - Alpine + GitHub Releases)
- Dockerfile.source (source build - Ubuntu + cross-compilation)
- Dockerfile.dev (development - Ubuntu + full toolchain)
## 🔧 Key Changes
- **Eliminated Duplicates**: Removed redundant prebuild variants
- **Moved Core Files**: Dockerfile.{source,dev} now in root directory
- **Unified Configuration**: cargo.config.toml moved to root
- **Updated References**: Fixed all GitHub Actions and docker-compose paths
- **Simplified CI Matrix**: Reduced from 5 to 3 Docker variants
## 📦 Preserved Valuable Diversity
- **Production**: Alpine-based for minimal size
- **Source**: Ubuntu-based with cross-compilation support
- **Development**: Ubuntu-based with full development tools
## 🚀 Benefits
- ✅ Cleaner project structure
- ✅ Easier maintenance and navigation
- ✅ Reduced CI/CD complexity
- ✅ Faster build matrix execution
- ✅ Maintained functionality and flexibility
* chore: remove duplicate cargo.config.toml from .docker directory
The file is now in the root directory and no longer needed in .docker/
* fix: update all references to removed Dockerfile files
- Updated .docker/compose/README.md to reference Dockerfile.source instead of Dockerfile.obs
- Updated docker-compose.yml to use Dockerfile.source instead of Dockerfile.dev
- Updated scripts/build-docker-multiarch.sh to use Dockerfile.source for devenv builds
- Updated .github/workflows/docker.yml to use Dockerfile.source for dev builds
- Updated Makefile to use Dockerfile.source for init-devenv target
- Updated .docker/README.md to remove references to non-existent Dockerfile.dev
- Ensured all Docker configurations consistently use the unified Dockerfile structure
* chore: remove unnecessary console static assets download
- Remove obsolete download steps from build.yml and performance.yml
- Console static assets are already embedded via rust-embed in rustfs/static/
- The download from dl.rustfs.com is no longer needed as project contains complete console assets
- This improves build reliability and reduces external dependencies
- Replaced with verification steps that confirm embedded assets are present
* feat: update Makefile and README.md for new Docker build system
- Updated Makefile to use unified Docker build system:
- Replace references to non-existent Dockerfile.ubuntu22.04 and Dockerfile.rockylinux9.3
- Add new docker-buildx targets using docker-buildx.sh script
- Deprecate old docker-build-multiarch targets with warnings
- Add docker-build-production and docker-build-source targets
- Update help-docker with new command structure
- Updated README.md with docker-buildx.sh usage:
- Add comprehensive Docker build from source section
- Document multi-architecture build capabilities
- Include both script and Make target examples
- Show registry flexibility and build optimization features
- Update step numbers in quickstart guide
- Improve developer experience with clear documentation and updated tooling
- Maintain backward compatibility with deprecation warnings
* feat: integrate console assets download into build-rustfs.sh
- Added console download functionality to build-rustfs.sh:
- New flags: --download-console, --force-console-update, --console-version
- Intelligent detection of existing console assets
- Retry logic with fallback error handling
- Consistent with Docker build asset management
- Updated scripts to use unified build process:
- scripts/static.sh: Now uses build-rustfs.sh for console downloads
- scripts/run.sh: Uses build-rustfs.sh instead of direct curl
- scripts/run.ps1: Updated with guidance for Windows users
- Benefits:
- Unified asset management across all build processes
- Consistent version handling and retry logic
- Eliminates duplicate download logic
- Better error handling and user feedback
- Preparation for CI/CD integration
- Removed unused download-static-curl.sh script
This change centralizes console asset management and prepares for
streamlined CI/CD processes where build-rustfs.sh becomes the
single point of truth for binary and asset builds.
* fix: update PowerShell script to use unified console asset management
- Updated scripts/run.ps1 to use build-rustfs.sh for console asset downloads
- Added guidance for Windows users to use the unified build script
- Maintains consistency across all platform-specific scripts
* feat: add binary verification to build script
- Add verify_binary function to test built binaries
- Test --help and --version commands
- Verify binary structure with readelf/otool
- Add --skip-verification option for cross-compilation
- Include verification status in build output
- Automatic error handling if verification fails
* feat: add platform selection support to build script
- Add --platform parameter to build-rustfs.sh for target platform selection
- Implement cross-compilation support with automatic 'cross' tool detection
- Auto-enable --skip-verification for cross-compilation scenarios
- Update all Makefile build targets to use unified build-rustfs.sh script
- Add helpful error messages and suggestions for cross-compilation failures
- Update help documentation with platform selection examples
- Improve build consistency across different architectures
* feat: modernize CI/CD build process with build-rustfs.sh
- Replace manual cargo build commands with unified build-rustfs.sh script
- Simplify matrix configuration by removing cross-compilation flags
- Ensure consistency between local and CI/CD builds
- Automatic cross-compilation tool detection and installation
- Built-in binary verification for quality assurance
- Unified console asset management
- Better error handling and suggestions
Benefits:
- Consistent build process across all environments
- Automatic detection and handling of cross-compilation scenarios
- Built-in quality checks with binary verification
- Reduced CI/CD configuration complexity
- Better maintainability with single source of truth for build logic
* feat: optimize CI/CD workspace path management
- Add WORKSPACE_DIR environment variable to cache github.workspace
- Set default working-directory at job level for consistency
- Use explicit workspace paths in critical operations
- Improve reliability and maintainability of CI/CD paths
- Ensure consistent behavior across different GitHub Actions environments
Benefits:
- More explicit and reliable path handling
- Better maintainability with centralized workspace reference
- Reduced risk of path-related issues in CI/CD
- Consistent working directory across all job steps
* refactor: simplify CI/CD path management - remove redundant workspace references
- Remove unnecessary WORKSPACE_DIR environment variable
- Remove redundant defaults.run.working-directory setting
- Use relative paths since GITHUB_WORKSPACE is the default working directory
- Follow GitHub Actions best practices by leveraging default behavior
As per GitHub Actions documentation, GITHUB_WORKSPACE is already the default
working directory, so explicit specification is unnecessary in most cases.
* docs: update Docker README to reflect current project state
- Fix directory structure: remove non-existent nginx/ directory
- Correct base OS: Dockerfile.source uses Debian Bookworm, not Ubuntu 22.04
- Add docker-buildx.sh script documentation
- Update Docker tag examples to match actual CI/CD workflows
- Add CI/CD integration section explaining automated builds
- Document build variants and manual build options
- Reflect current project architecture and tooling
These updates ensure the documentation accurately represents the current
Docker build system and CI/CD workflows.
* fix: update Docker command in rustfs README
- Replace quay.io registry with Docker Hub (rustfs/rustfs:latest)
- Remove separate console port 9001, console now runs on main port 9000
- Add both Docker and Podman examples for user choice
- Fix console access URL to use unified port
This aligns with the recent console port consolidation changes
and the project's move to Docker Hub as the primary registry.
* wip
* fix: remove unnecessary entrypoint.sh and fix Docker paths
* Update Dockerfile
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* cleanup: remove unused DOCKERFILE_PATH variable from Makefile
* feat: update Docker build to use dl.rustfs.com for binary downloads
- Replace GitHub releases download with dl.rustfs.com
- Add CHANNEL parameter support (release/dev)
- Update docker-buildx.sh to support channel-specific builds
- Improve error messages with new download URLs
- Support both latest and specific version downloads
- Add channel validation in build script
---------
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
This commit introduces two key enhancements: the integration of Grafana Tempo for distributed tracing and the implementation of a dual-stack TCP listener for improved network compatibility.
- **Observability**:
- Adds the `tempo` service to the `docker-compose.yml` observability stack.
- Tempo is configured to collect and store traces, integrating with the existing OpenTelemetry setup.
- A custom `tempo-entrypoint.sh` script is included to manage volume permissions on startup.
- **Networking**:
- Modifies `http.rs` to support dual-stack (IPv4/IPv6) connections on a single socket.
- By setting the `IPV6_V6ONLY` socket option to `false`, the server can now accept both IPv6 and IPv4-mapped IPv6 traffic, enhancing cross-platform support.
* feat: enhance Docker build system with advanced version selection
## New Features
- Add force_rebuild parameter for Docker workflow manual triggers
- Improve version pattern matching with better regex validation
- Add comprehensive Docker Build Guide documentation
- Enhanced logging and error reporting for build process
- Support for prerelease version detection (alpha, beta, rc)
## Improvements
- Better version pattern validation for releases and dev builds
- More detailed build logs with context and warnings
- Clear documentation for all Docker image variants and use cases
- Updated README with Docker version examples and guide reference
## Documentation
- New comprehensive Docker Build Guide (docs/DOCKER_BUILD_GUIDE.md)
- Updated README with version-specific Docker examples
- Workflow dependency diagram and troubleshooting guide
- Complete reference for all supported version patterns
This enhancement provides a robust, well-documented Docker build system
that supports flexible version selection while maintaining deterministic
build behavior without fallback mechanisms.
* fix: simplify dev version regex pattern in docker workflow
* fix: simplify version number regex pattern in docker workflow
* feat: remove docs directory
* fix: correct SHA length matching in main-latest filename generation
* refactor: use bash string operations instead of sed for main-latest filename generation
* refactor: simplify filename generation by removing redundant intermediate variables
* feat: add dev-latest version generation for all development builds
* feat: add dev-latest support to Docker workflow
* feat: enhance Docker build system with advanced version selection
## New Features
- Add force_rebuild parameter for Docker workflow manual triggers
- Improve version pattern matching with better regex validation
- Add comprehensive Docker Build Guide documentation
- Enhanced logging and error reporting for build process
- Support for prerelease version detection (alpha, beta, rc)
## Improvements
- Better version pattern validation for releases and dev builds
- More detailed build logs with context and warnings
- Clear documentation for all Docker image variants and use cases
- Updated README with Docker version examples and guide reference
## Documentation
- New comprehensive Docker Build Guide (docs/DOCKER_BUILD_GUIDE.md)
- Updated README with version-specific Docker examples
- Workflow dependency diagram and troubleshooting guide
- Complete reference for all supported version patterns
This enhancement provides a robust, well-documented Docker build system
that supports flexible version selection while maintaining deterministic
build behavior without fallback mechanisms.
* fix: simplify dev version regex pattern in docker workflow
* fix: simplify version number regex pattern in docker workflow
* feat: remove docs directory
* fix: correct SHA length matching in main-latest filename generation
* feat: enhance Docker build system with advanced version selection
## New Features
- Add force_rebuild parameter for Docker workflow manual triggers
- Improve version pattern matching with better regex validation
- Add comprehensive Docker Build Guide documentation
- Enhanced logging and error reporting for build process
- Support for prerelease version detection (alpha, beta, rc)
## Improvements
- Better version pattern validation for releases and dev builds
- More detailed build logs with context and warnings
- Clear documentation for all Docker image variants and use cases
- Updated README with Docker version examples and guide reference
## Documentation
- New comprehensive Docker Build Guide (docs/DOCKER_BUILD_GUIDE.md)
- Updated README with version-specific Docker examples
- Workflow dependency diagram and troubleshooting guide
- Complete reference for all supported version patterns
This enhancement provides a robust, well-documented Docker build system
that supports flexible version selection while maintaining deterministic
build behavior without fallback mechanisms.
* fix: simplify dev version regex pattern in docker workflow
* fix: simplify version number regex pattern in docker workflow
* feat: remove docs directory
- Add version input parameter to docker.yml workflow_dispatch
- Support main-latest, latest, dev-xxx, and specific version patterns
- Remove complex fallback mechanism from all Dockerfile variants
- Add clear error handling with helpful user guidance
- Create main-latest versions for development builds
- Ensure Docker builds require explicit VERSION parameter
- Update all Docker variants (production, alpine, ubuntu) consistently
This change solves the build dependency issue where Docker builds
could fail when expected binary artifacts don't exist, by providing
a clean version selection mechanism without unpredictable fallbacks.
- Add automatic creation of latest version files for release and prerelease builds
- Simplify installation script by providing direct latest URLs
- Support rustfs-linux-{arch}-latest.zip naming convention
- Improve build artifact management and user experience
* feat: implement multi-channel release system with artifact naming
- Add dedicated release.yml workflow for handling GitHub releases
- Refactor build.yml to support dev/release/prerelease artifact naming
- Update docker.yml to support version-specific image tagging
- Implement artifact naming rules:
- Dev: rustfs-{platform}-{arch}-dev-{sha}.zip
- Release: rustfs-{platform}-{arch}-v{version}.zip
- Prerelease: rustfs-{platform}-{arch}-v{version}.zip
- Add OSS upload directory separation (dev/ vs release/)
- Only stable releases update latest.json and create latest tags
- Separate GitHub Release creation from build workflow
- Add comprehensive build summaries and status reporting
This enables proper multi-channel distribution with clear artifact
identification and prevents confusion between dev and stable releases.
* fix: support version tags without v prefix (1.0.0 instead of v1.0.0)
- Update trigger patterns from 'v*.*.*' to '*.*.*' in all workflows
- Fix version extraction logic to handle tags without v prefix
- Maintain backward compatibility with existing logic
Note: Artifact naming still includes 'v' prefix for clarity
(e.g., tag '1.0.0' creates 'rustfs-linux-x86_64-v1.0.0.zip')
* feat: update Dockerfile to support multi-channel release system
- Add build arguments for VERSION, BUILD_TYPE, and TARGETARCH
- Support dynamic artifact download based on build type:
- Development: downloads from artifacts/rustfs/dev/
- Release: downloads from artifacts/rustfs/release/
- Auto-generate correct filenames based on new naming convention:
- Dev: rustfs-linux-{arch}-dev-{sha}.zip
- Release: rustfs-linux-{arch}-v{version}.zip
- Add architecture mapping for multi-platform builds
- Pass BUILD_TYPE parameter from docker.yml workflow
- Improve error handling with helpful download path suggestions
This ensures Docker images use the correct pre-built binaries
from the new multi-channel release system.
* feat: optimize and consolidate Dockerfile structure
## Major Improvements:
### ✅ Created Missing Files
- Add .docker/Dockerfile.alpine for lightweight Alpine-based builds
- Support both pre-built binary download and source compilation
### 🔧 Fixed Critical Issues
- Fix Dockerfile.obs: ubuntu:latest → ubuntu:22.04 (stable version)
- Add proper security practices (non-root user, health checks)
- Add proper error handling and environment variables
### 🗑️ Eliminated Redundancy
- Remove .docker/Dockerfile.ubuntu22.04 (duplicate of devenv)
- Update docker.yml workflow to use devenv for ubuntu variant
- Consolidate similar functionality into fewer, better files
### 🚀 Enhanced Functionality
- Make devenv Dockerfile dual-purpose (dev environment + runtime)
- Add VERSION/BUILD_TYPE support for dynamic binary downloads
- Improve security with proper user management
- Add comprehensive health checks and error handling
### 📊 Final Dockerfile Structure:
1. Dockerfile (production, Alpine-based, pre-built binaries)
2. Dockerfile.multi-stage (full source builds, Ubuntu-based)
3. Dockerfile.obs (observability builds, Ubuntu-based)
4. .docker/Dockerfile.alpine (lightweight Alpine variant)
5. .docker/Dockerfile.devenv (development + ubuntu variant)
6. .docker/Dockerfile.rockylinux9.3 (RockyLinux variant)
This reduces redundancy while maintaining all necessary build variants
and improving maintainability across the entire container ecosystem.
* refactor: streamline Dockerfile structure and remove unused files
## 🎯 Major Cleanup:
### 🗑️ Removed Unused Files (2 files)
- Delete Dockerfile.obs (not referenced anywhere)
- Delete .docker/Dockerfile.rockylinux9.3 (not referenced anywhere)
### 📁 Reorganized File Layout
- Move Dockerfile.multi-stage → .docker/Dockerfile.multi-stage
- Update docker-compose.yml to use new path
- Keep main Dockerfile in root (production use)
- Consolidate variants in .docker/ directory
### ✅ Final Clean Structure:
### 📊 Before vs After:
- **Before**: 7 files (1 missing, 2 unused, scattered layout)
- **After**: 4 files (all used, organized layout)
- **Reduction**: 43% fewer files, 100% utilization
This eliminates confusion and reduces maintenance overhead while
keeping all actually needed functionality intact.
* refactor: implement comprehensive Docker tag strategy with production variant
- Restore production variant as default with explicit naming
- Add support for prerelease channels (alpha, beta, rc)
- Implement rolling development tags (dev, dev-variant)
- Support semantic versioning with variant combinations
- Update documentation with complete tag strategy examples
- Align with GPT-suggested comprehensive tagging approach
Tag examples:
- rustfs/rustfs:1.2.3 (main production)
- rustfs/rustfs:1.2.3-production (explicit production)
- rustfs/rustfs:1.2.3-alpine (Alpine variant)
- rustfs/rustfs:alpha (latest alpha)
- rustfs/rustfs:dev (latest development)
- rustfs/rustfs:dev-13e4a0b (specific commit)
* perf: optimize Docker build speed with comprehensive caching and compilation improvements
- Add dual caching strategy: GitHub Actions + Registry cache
- Implement sccache for Rust compilation caching across builds
- Configure parallel compilation with all available CPU cores
- Add optimized cargo configuration for faster builds
- Enable sparse registry protocol for dependency resolution
- Configure LLD linker for faster linking
- Add BuildKit optimizations with inline cache
- Disable provenance/SBOM generation for faster builds
- Document build performance improvements and timings
Performance improvements:
- Source builds: ~40-50% faster with cache hits
- Pre-built binaries: ~30-40% faster
- Parallel matrix builds reduce total CI time significantly
- Registry cache provides persistent cross-run benefits
* refactor: consolidate Docker variants and eliminate duplication
- Replace root Dockerfile with enhanced Alpine prebuild version
- Remove redundant alpine variant from build matrix
- Root Dockerfile now includes:
- Non-root user security
- Health checks
- Better error handling
- protoc/flatc tool support
- Update documentation to reflect simplified 4-variant strategy
- Remove duplicate .docker/alpine/Dockerfile.prebuild
Build matrix now:
- production (root Dockerfile - Alpine prebuild)
- alpine-source (Alpine source build)
- ubuntu (Ubuntu prebuild)
- ubuntu-source (Ubuntu source build)
Benefits:
- Eliminates functional duplication
- Improves security with non-root execution
- Maintains same image variants with better quality
- Simplifies maintenance
* fix: restore alpine variant for better user choice
- Restore alpine variant (rustfs/rustfs:1.2.3-alpine)
- Re-add .docker/alpine/Dockerfile.prebuild
- Update build matrix to include 5 variants again:
- production (default)
- alpine (explicit Alpine choice)
- alpine-source (Alpine source build)
- ubuntu (Ubuntu pre-built)
- ubuntu-source (Ubuntu source build)
- Update documentation to reflect restored alpine tags
- Fix build performance table to include all variants
User feedback: Alpine variant provides explicit choice even if
similar to production variant. Better UX with clear options.
* fix: remove redundant rustup target add commands in Alpine Dockerfiles
- Remove 'rustup target add x86_64-unknown-linux-musl' from Alpine source build
- Remove redundant target add from Alpine prebuild fallback path
- Remove redundant target add from root Dockerfile fallback path
Reason: rust:alpine base image already has x86_64-unknown-linux-musl
as the default target since Alpine uses musl libc by default.
Thanks to @houseme for spotting this redundancy in code review.
* fix: add missing RUSTFS_VOLUMES environment variable in Dockerfiles
- Add RUSTFS_VOLUMES=/data to all Dockerfile variants
- This fixes the issue where CMD ['/app/rustfs'] was used without providing the required volumes parameter
- The volumes parameter is required by the application and can be provided via command line or RUSTFS_VOLUMES environment variable
* fix: update docker-compose configurations to ensure all environments work correctly
- Added missing access key and secret key environment variables to docker-compose.yaml
- This ensures the distributed test environment has proper authentication credentials
- Complementary fix to the previous Dockerfile updates for consistent configuration
* fix: recreate missing Dockerfile.obs with complete content
- The file was accidentally left empty after initial creation
- Now contains proper Ubuntu-based configuration for observability environment
- Includes all necessary environment variables including RUSTFS_VOLUMES
- Supports docker-compose-obs.yaml configuration
* refactor: organize Docker Compose configurations and eliminate duplication
- Move specialized configurations to .docker/compose/ directory
- Rename docker-compose.yaml → docker-compose.cluster.yaml (distributed testing)
- Rename docker-compose-obs.yaml → docker-compose.observability.yaml (observability testing)
- Keep docker-compose.yml as the main production configuration
- Add comprehensive README explaining different configuration purposes
- Eliminates confusion between similar filenames
- Provides clear guidance on when to use each configuration
* fix: correct relative paths in moved Docker Compose configurations
- Fix binary volume mount paths in docker-compose.cluster.yaml (./target → ../../target)
- Fix Dockerfile.obs context path in docker-compose.observability.yaml (. → ../..)
- Fix observability config file paths (./.docker → ../../.docker)
- Update README.md with correct usage instructions for new locations
- All configurations now correctly reference files relative to their new positions
* refactor: move Dockerfile.obs to .docker/compose/ directory for better organization
- Move Dockerfile.obs from root to .docker/compose/ directory
- Update all dockerfile references in docker-compose.observability.yaml
- Keep related files (Dockerfile.obs + docker-compose.observability.yaml) together
- Clean up root directory by removing specialized-purpose Dockerfile
- Update README.md to document new file organization
- Improves project structure and file discoverability
* refactor: improve Docker build configuration for better clarity
- Move Dockerfile.obs back to project root for simpler build context
- Update docker-compose.observability.yaml to use cleaner dockerfile reference
- Change from '.docker/compose/Dockerfile.obs' to simply 'Dockerfile.obs'
- Maintain context as '../..' for access to project files
- Remove redundant Dockerfile.obs documentation from compose README
- This follows Docker best practices: simple context + Dockerfile at context root
* wip
* refactor(config): Unify S3 API and Console ports
This commit streamlines the server configuration by unifying the S3 API and the WebUI (Console) to serve on a single port.
Previously, the console was managed by separate configuration options (`RUSTFS_CONSOLE_ENABLE` and `RUSTFS_CONSOLE_ADDRESS`), requiring a distinct port. This added complexity to deployment and configuration.
With this change:
- The `RUSTFS_CONSOLE_ADDRESS` and `RUSTFS_CONSOLE_FS_ENDPOINT` environment variables are removed.
- The WebUI is now always available and served directly from the main application port defined by `RUSTFS_ADDRESS`.
- This simplifies setup, reduces the number of exposed ports, and makes the application easier to manage and deploy, especially in containerized environments.
Users should update their startup scripts and remove the deprecated `RUSTFS_CONSOLE_*` variables.
* improve docker comprose config file and remove docs dir
This commit streamlines the server configuration by unifying the S3 API and the WebUI (Console) to serve on a single port.
Previously, the console was managed by separate configuration options (`RUSTFS_CONSOLE_ENABLE` and `RUSTFS_CONSOLE_ADDRESS`), requiring a distinct port. This added complexity to deployment and configuration.
With this change:
- The `RUSTFS_CONSOLE_ADDRESS` and `RUSTFS_CONSOLE_FS_ENDPOINT` environment variables are removed.
- The WebUI is now always available and served directly from the main application port defined by `RUSTFS_ADDRESS`.
- This simplifies setup, reduces the number of exposed ports, and makes the application easier to manage and deploy, especially in containerized environments.
Users should update their startup scripts and remove the deprecated `RUSTFS_CONSOLE_*` variables.
* feat: add browser redirect layer to route GET requests to console
* refactor: move RedirectLayer to separate layer.rs file
* feat: restrict redirect layer to only handle root path and index.html
* feat: restrict redirect layer to only handle root path /rustfs and index.html
* fix unzip error
* fix url change error
fix url change error
* Simplify user experience and integrate console and endpoint
Simplify user experience and integrate console and endpoint
* merge console router
* make code happy
* Scanner (#156)
* feat: integrate CancellationToken for unified background services management
- Consolidate data scanner and auto heal cancellation tokens into single unified token
- Move GLOBAL_BACKGROUND_SERVICES_CANCEL_TOKEN to global.rs for centralized management
- Add graceful shutdown support to MRF heal routine with MinIO-compatible logic
- Implement heal_routine_with_cancel method preserving original healing logic
- Update main.rs to use unified background services shutdown mechanism
- Enhance error handling with proper ecstore Result types
- Fix clippy warnings for needless return statements
- Maintain backward compatibility while adding modern cancellation support
This change provides a cleaner architecture for background service lifecycle management
and ensures all healing services can be gracefully shut down through a single token.
Signed-off-by: junxiang Mu <1948535941@qq.com>
* fix: Refact heal and scanner design
Signed-off-by: junxiang Mu <1948535941@qq.com>
* refact: step 2
Signed-off-by: junxiang Mu <1948535941@qq.com>
* feat: refactor scanner module and add data usage statistics
- Move scanner code to scanner/ subdirectory for better organization
- Add data usage statistics collection and persistence
- Implement histogram support for size and version distribution
- Add global cancel token management for scanner operations
- Integrate scanner with ECStore for comprehensive data analysis
- Update error handling and improve test isolation
- Add data usage API endpoints and backend integration
Signed-off-by: junxiang Mu <1948535941@qq.com>
* Chore: fix ref and fix comment
Signed-off-by: junxiang Mu <1948535941@qq.com>
* fix: fix clippy
Signed-off-by: junxiang Mu <1948535941@qq.com>
---------
Signed-off-by: junxiang Mu <1948535941@qq.com>
Co-authored-by: dandan <dandan@dandandeMac-Studio.local>
---------
Signed-off-by: junxiang Mu <1948535941@qq.com>
Co-authored-by: guojidan <63799833+guojidan@users.noreply.github.com>
Co-authored-by: dandan <dandan@dandandeMac-Studio.local>
* feat: integrate CancellationToken for unified background services management
- Consolidate data scanner and auto heal cancellation tokens into single unified token
- Move GLOBAL_BACKGROUND_SERVICES_CANCEL_TOKEN to global.rs for centralized management
- Add graceful shutdown support to MRF heal routine with MinIO-compatible logic
- Implement heal_routine_with_cancel method preserving original healing logic
- Update main.rs to use unified background services shutdown mechanism
- Enhance error handling with proper ecstore Result types
- Fix clippy warnings for needless return statements
- Maintain backward compatibility while adding modern cancellation support
This change provides a cleaner architecture for background service lifecycle management
and ensures all healing services can be gracefully shut down through a single token.
Signed-off-by: junxiang Mu <1948535941@qq.com>
* fix: Refact heal and scanner design
Signed-off-by: junxiang Mu <1948535941@qq.com>
* refact: step 2
Signed-off-by: junxiang Mu <1948535941@qq.com>
* feat: refactor scanner module and add data usage statistics
- Move scanner code to scanner/ subdirectory for better organization
- Add data usage statistics collection and persistence
- Implement histogram support for size and version distribution
- Add global cancel token management for scanner operations
- Integrate scanner with ECStore for comprehensive data analysis
- Update error handling and improve test isolation
- Add data usage API endpoints and backend integration
Signed-off-by: junxiang Mu <1948535941@qq.com>
* Chore: fix ref and fix comment
Signed-off-by: junxiang Mu <1948535941@qq.com>
* fix: fix clippy
Signed-off-by: junxiang Mu <1948535941@qq.com>
---------
Signed-off-by: junxiang Mu <1948535941@qq.com>
Co-authored-by: dandan <dandan@dandandeMac-Studio.local>
Added language selection links to the README for easier access to translated versions: German, Spanish, French, Japanese, Korean, Portuguese, and Russian.
- Replace paths-ignore with paths for more precise control
- Only trigger on Rust source files, Cargo files, and workflow itself
- Improve efficiency by avoiding unnecessary performance tests
- Follow best practices for targeted workflow execution
* feat: ensure workflows never skip execution during version releases
- Modified skip-duplicate-actions to never skip when pushing tags
- Updated all workflow jobs to force execution for tag pushes (version releases)
- Ensures complete CI/CD pipeline execution for releases including:
- All tests and lint checks
- Multi-platform builds
- GUI builds
- Release asset creation
- OSS uploads
This guarantees that version releases always undergo full validation
and build processes, maintaining release quality and consistency.
* refactor: simplify workflow skip logic using do_not_skip parameter
- Replace complex conditional expressions with do_not_skip: ['release', 'push']
- Add skip-duplicate-actions to docker.yml workflow
- Ensure all workflows use consistent skip mechanism
- Maintain release and tag push execution guarantee
- Simplify job conditions by removing redundant tag checks
This change makes workflows more maintainable and follows
official skip-duplicate-actions best practices.
* feat: enhance build and release workflow with multi-platform support
- Add Windows support (x86_64 and ARM64) to build matrix
- Add macOS Intel x86_64 support alongside Apple Silicon
- Improve cross-platform builds with proper toolchain selection
- Use GitHub CLI (gh) for release management instead of GitHub Actions
- Add automatic checksum generation (SHA256/SHA512) for all binaries
- Support different archive formats per platform (zip for Windows, tar.gz for Unix)
- Add comprehensive release notes with installation guides
- Enhanced error handling for console assets download
- Platform-specific build information in packages
- Support both binary and GUI application releases
- Update OSS upload to handle multiple file formats
This brings RustFS builds up to enterprise-grade standards with:
- 6 binary targets (Linux x86_64/ARM64, macOS x86_64/ARM64, Windows x86_64/ARM64)
- Professional release management with checksums
- User-friendly installation instructions
- Multi-platform GUI applications
* feat: add core development principles to cursor rules
- Add precision-first development principle: 每次改动都要精准,没把握就别改
- Add GitHub CLI priority rule: GitHub PR 创建优先使用 gh 命令
- Emphasize careful analysis before making changes
- Promote use of gh commands for better automation and integration
* refactor: translate cursor rules to English
- Translate core development principles from Chinese to English
- Maintain consistency with project's English-first policy
- Update 'Every change must be precise' principle
- Update 'GitHub PR creation prioritizes gh command usage' rule
- Ensure all cursor rules are in English for better accessibility
* fix: prevent workflow changes from triggering CI/CD pipelines
- Add .github/** to paths-ignore in build.yml workflow
- Add .github/** to paths-ignore in docker.yml workflow
- Update skip-duplicate paths_ignore to include .github files
- Workflow changes should not trigger performance, build, or docker workflows
- Saves unnecessary CI/CD resource usage when updating workflow configurations
- Consistent with performance.yml which already ignores .github/**
* feat: enhance build and release workflow with multi-platform support
- Add Windows support (x86_64 and ARM64) to build matrix
- Add macOS Intel x86_64 support alongside Apple Silicon
- Improve cross-platform builds with proper toolchain selection
- Use GitHub CLI (gh) for release management instead of GitHub Actions
- Add automatic checksum generation (SHA256/SHA512) for all binaries
- Support different archive formats per platform (zip for Windows, tar.gz for Unix)
- Add comprehensive release notes with installation guides
- Enhanced error handling for console assets download
- Platform-specific build information in packages
- Support both binary and GUI application releases
- Update OSS upload to handle multiple file formats
This brings RustFS builds up to enterprise-grade standards with:
- 6 binary targets (Linux x86_64/ARM64, macOS x86_64/ARM64, Windows x86_64/ARM64)
- Professional release management with checksums
- User-friendly installation instructions
- Multi-platform GUI applications
* feat: add core development principles to cursor rules
- Add precision-first development principle: 每次改动都要精准,没把握就别改
- Add GitHub CLI priority rule: GitHub PR 创建优先使用 gh 命令
- Emphasize careful analysis before making changes
- Promote use of gh commands for better automation and integration
* refactor: translate cursor rules to English
- Translate core development principles from Chinese to English
- Maintain consistency with project's English-first policy
- Update 'Every change must be precise' principle
- Update 'GitHub PR creation prioritizes gh command usage' rule
- Ensure all cursor rules are in English for better accessibility
- Fix performance.yml: now ignores *.md, README*, and docs/**
- Fix build.yml: now ignores documentation files and images
- Fix docker.yml: prevent Docker builds on README changes
- Replace 'paths:' with 'paths-ignore:' to properly exclude docs
- Reduces unnecessary CI runs for documentation-only PRs
This resolves the issue where README changes triggered expensive
CI pipelines including Performance Testing and Docker builds.
* docs: simplify all crates README files
- Remove extensive code examples and detailed documentation
- Convert to minimal module introductions with core feature lists
- Direct users to main RustFS repository for comprehensive docs
- Updated 20 crate README files for consistency and brevity
Files updated:
- crates/rio/README.md (415→15 lines)
- crates/s3select-api/README.md (592→15 lines)
- crates/s3select-query/README.md (658→15 lines)
- crates/signer/README.md (407→15 lines)
- crates/utils/README.md (395→15 lines)
- crates/workers/README.md (463→15 lines)
- crates/zip/README.md (408→15 lines)
* docs: restore original headers in crates README files
- Add back RustFS logo image and CI badges
- Restore formatted headers and structured layout
- Keep simplified content with module introductions
- Maintain consistent documentation structure across all crates
All 20 crate README files now have proper headers while keeping
the simplified content that directs users to the main repository.
* rules: enforce PR-only workflow for main branch
- Strengthen rule that ALL changes must go through pull requests
- Explicitly forbid direct commits to main branch under any circumstances
- Add comprehensive PR requirements and enforcement guidelines
- Clarify that PRs are the ONLY way to merge to main branch
- Add requirement for PR approval before merging
- Include enforcement mechanisms for branch protection
- Remove invalid github-token parameter from arduino/setup-protoc action
- Fix cross-compilation RUSTFLAGS issue by conditionally setting target-cpu=native
- Update workflow tag triggers from v* to * for non-v prefixed tags
- Optimize Zig and cargo-zigbuild installation using official actions
This resolves build failures in aarch64-unknown-linux-musl target where
zig was receiving invalid x86_64 CPU flags during cross-compilation.
- Fix random number generation API usage
- Adjust sleep calculation to follow MinIO pattern
- Ensure proper random range for scanner cycles
Signed-off-by: junxiang Mu <1948535941@qq.com>
* fix: restore Zig and cargo-zigbuild caching in GitHub Actions setup action
Use mlugg/setup-zig and taiki-e/cache-cargo-install-action to speed up cross-compilation tool installation and avoid repeated downloads. All comments and code are in English.
* fix: use correct taiki-e/install-action for cargo-zigbuild
Use taiki-e/install-action@cargo-zigbuild instead of taiki-e/cache-cargo-install-action@v2 to match the original implementation from PR #77.
* refactor: remove explicit Zig version to use latest stable
Move the construction of the hybrid service stack, including all middleware and the RPC service, from the main `run` function into the `process_connection` function.
This change ensures that each incoming connection gets its own isolated service instance. This improves modularity by making the connection handling logic more self-contained and simplifies the main server loop.
Key changes:
- The `hybrid_service` and `rpc_service` are now created inside `process_connection`.
- The `run` function's responsibility is reduced to accepting TCP connections and spawning tasks for `process_connection`.
- Only execute builds on tag push, scheduled runs, or commit message contains --build
- Add latest.json version tracking to rustfs-version OSS bucket
- Create GitHub Release with all build artifacts automatically
- Update comments to English for consistency
- Reduce unnecessary CI resource usage while maintaining automation
This commit introduces a significant reorganization of the project structure to improve maintainability and clarity.
Key changes include:
- Adjusted the directory layout for a more logical module organization.
- Removed unused crate dependencies, reducing the overall project size and potentially speeding up build times.
- Updated import paths and configuration files to reflect the structural changes.
* chore: Add copyright and license headers
This commit adds the Apache 2.0 license and a copyright notice to the header of all source files. This ensures that the licensing and copyright information is clearly stated within the codebase.
* cargo fmt
* fix
* fmt
* fix clippy
- Add #[allow(clippy::too_many_arguments)] for functions with 8 parameters
- Replace vec initialization with array where appropriate
- Fix test code warnings by adding #[allow] attributes
- Improve code quality while maintaining functionality
- Change console_enable default value from false to true
- This makes the console server available by default for better user experience
- Users can still disable it by setting RUSTFS_CONSOLE_ENABLE=false
- Refactor `unmarshal_msg` to use `rmp_serde::from_slice` instead of manual parsing
- Add serde field renaming attributes to `FileMetaVersion` struct
- Remove 428 lines of manual MessagePack parsing code
- Improve string comparison using `!object.is_empty()` instead of `object != ""`
- Update volume directory numbering in run script from test{0..4} to test{1..4}
- Clean up unused imports and code
Fixes#517
- Refactor find_version_index to accept Uuid directly instead of string
- Split from_meta_cache_entries_sorted into separate methods for versions and infos
- Add support for after_version_id filtering in version listing
- Fix S3 error code for replication configuration not found
- Improve error handling and logging levels
- Clean up import statements and remove debug code
- Remove specific bucket metadata error variants (BucketPolicyNotFound, etc.)
- Unify all configuration-not-found errors to use StorageError::ConfigNotFound
- Simplify error handling in bucket metadata system by removing redundant error conversions
- Clean up error matching patterns in metadata getter methods
- Remove debug print statement from get_bucket_targets_config
- Update error handling in S3 API layer to use unified ConfigNotFound error
This change improves code maintainability by reducing error type complexity
while preserving the same error semantics for API consumers.
- Remove reed-solomon-erasure dependency and all related code
- Simplify ReedSolomonEncoder from enum to struct with SIMD-only implementation
- Eliminate all conditional compilation (#[cfg(feature = ...)])
- Add instance caching with RwLock-based encoder/decoder reuse
- Implement reset mechanism to avoid unnecessary allocations
- Ensure thread safety with proper cache management
- Update documentation and benchmark scripts for SIMD-only approach
- Apply code formatting across all files
Breaking Changes:
- Removes support for reed-solomon-erasure feature flag
- API remains compatible but implementation is now SIMD-only
Performance Impact:
- Improved encoding/decoding performance through SIMD optimization
- Reduced memory allocations via instance caching
- Enhanced thread safety and concurrency support
* chore(ci): upgrade protoc from 30.2 to 31.1
- Update protoc version in GitHub Actions setup workflow
- Use arduino/setup-protoc@v3 to install the latest protoc version
- Ensure compatibility with current project requirements
- Improve proto file compilation performance and stability
This upgrade aligns our development environment with the latest protobuf standards.
* modify package version
* refactor(deps): centralize crate versions in root Cargo.toml
- Move all dependency versions to workspace.dependencies section
- Standardize AWS SDK and related crates versions
- Update tokio, bytes, and futures crates to latest stable versions
- Ensure consistent version use across all workspace members
- Implement workspace inheritance for common dependencies
This change simplifies dependency management and ensures version consistency across the project.
* fix
* modify
* chore(ci): upgrade protoc from 30.2 to 31.1
- Update protoc version in GitHub Actions setup workflow
- Use arduino/setup-protoc@v3 to install the latest protoc version
- Ensure compatibility with current project requirements
- Improve proto file compilation performance and stability
This upgrade aligns our development environment with the latest protobuf standards.
* modify package version
* refactor(deps): centralize crate versions in root Cargo.toml
- Move all dependency versions to workspace.dependencies section
- Standardize AWS SDK and related crates versions
- Update tokio, bytes, and futures crates to latest stable versions
- Ensure consistent version use across all workspace members
- Implement workspace inheritance for common dependencies
This change simplifies dependency management and ensures version consistency across the project.
* fix
* modify
- Update opentelemetry from 0.29.1 to 0.30.0
- Update related opentelemetry dependencies for compatibility
- Ensure compatibility with existing observability implementation
- Improve tracing and metrics collection capabilities
This upgrade provides better performance and stability for our observability stack.
* chore(ci): upgrade protoc from 30.2 to 31.1
- Update protoc version in GitHub Actions setup workflow
- Use arduino/setup-protoc@v3 to install the latest protoc version
- Ensure compatibility with current project requirements
- Improve proto file compilation performance and stability
This upgrade aligns our development environment with the latest protobuf standards.
* modify package version
* refactor(deps): centralize crate versions in root Cargo.toml
- Move all dependency versions to workspace.dependencies section
- Standardize AWS SDK and related crates versions
- Update tokio, bytes, and futures crates to latest stable versions
- Ensure consistent version use across all workspace members
- Implement workspace inheritance for common dependencies
This change simplifies dependency management and ensures version consistency across the project.
* fix
* modify
- Update opentelemetry from 0.29.1 to 0.30.0
- Update related opentelemetry dependencies for compatibility
- Ensure compatibility with existing observability implementation
- Improve tracing and metrics collection capabilities
This upgrade provides better performance and stability for our observability stack.
- Replace fixed `flexi_logger::Duplicate::Info` with dynamic level from config
- Convert logger_level string to corresponding LevelFilter enum
- Ensure terminal output respects the same log level as file logs
- Fix documentation to reflect the dynamic level behavior
- Create English and Chinese README files for the openobserve-otel directory
- Document configuration details for both OpenObserve and OTel Collector
- Include setup instructions and application integration examples
- Add badges for both OpenObserve and OpenTelemetry projects
- Add 2-minute timeout to samply record command with proper error handling
- Improve test volume directory creation
- Add workflow_dispatch for manual triggering
- Add job timeout of 10 minutes
- Set environment variables to match run.sh configuration
- Add run number to artifact name for better identification
- Add proper error checking and output when profiling fails
- Set artifact retention period to 7 days
This change simplifies the crate name to better reflect its core functionality
as the event handling system for RustFS. The renamed package maintains all
existing functionality while improving naming consistency across the project.
- Updated all imports and references to use the new crate name
- Maintained API compatibility with existing implementations
- Updated tests to reflect the name change
* init rustfs config
* improve code for rustfs-config crate
* add
* improve code for comment
* fix: modify rustfs-config crate name
* add default fn
* improve error logger
* fix: modify docker config yaml
* improve code for config
* feat: restrict kafka feature to Linux only
- Add target-specific feature configuration in Cargo.toml for obs and event-notifier crates
- Implement conditional compilation for kafka feature only on Linux systems
- Add appropriate error handling for non-Linux platforms
- Ensure backward compatibility with existing code
* refactor(ci): optimize build workflow for better efficiency
- Integrate GUI build steps into main build-rustfs job
- Add conditional GUI build execution based on tag releases
- Simplify workflow by removing redundant build-rustfs-gui job
- Copy binary directly to embedded-rustfs directory without downloading artifacts
- Update merge job dependency to only rely on build-rustfs
- Improve cross-platform compatibility for Windows binary naming (.exe)
- Streamline artifact uploading and OSS publishing process
- Maintain consistent conditional logic for release operations
* refactor(ci): optimize build workflow for better efficiency
- Integrate GUI build steps into main build-rustfs job
- Add conditional GUI build execution based on tag releases
- Simplify workflow by removing redundant build-rustfs-gui job
- Copy binary directly to embedded-rustfs directory without downloading artifacts
- Update merge job dependency to only rely on build-rustfs
- Improve cross-platform compatibility for Windows binary naming (.exe)
- Streamline artifact uploading and OSS publishing process
- Maintain consistent conditional logic for release operations
* fix(ci): add repo-token to setup-protoc action for authentication
- Add GITHUB_TOKEN parameter to arduino/setup-protoc@v3 action
- Ensure proper authentication for Protoc installation in CI workflow
- Maintain consistent setup across different CI environments
* modify config
* improve readme.md
* remove env config relation
* add allow(dead_code)
- Add GITHUB_TOKEN parameter to arduino/setup-protoc@v3 action
- Ensure proper authentication for Protoc installation in CI workflow
- Maintain consistent setup across different CI environments
- Integrate GUI build steps into main build-rustfs job
- Add conditional GUI build execution based on tag releases
- Simplify workflow by removing redundant build-rustfs-gui job
- Copy binary directly to embedded-rustfs directory without downloading artifacts
- Update merge job dependency to only rely on build-rustfs
- Improve cross-platform compatibility for Windows binary naming (.exe)
- Streamline artifact uploading and OSS publishing process
- Maintain consistent conditional logic for release operations
- Integrate GUI build steps into main build-rustfs job
- Add conditional GUI build execution based on tag releases
- Simplify workflow by removing redundant build-rustfs-gui job
- Copy binary directly to embedded-rustfs directory without downloading artifacts
- Update merge job dependency to only rely on build-rustfs
- Improve cross-platform compatibility for Windows binary naming (.exe)
- Streamline artifact uploading and OSS publishing process
- Maintain consistent conditional logic for release operations
Added support for both ARM64 Linux variants to the CI/CD build pipeline:
1. Enabled the previously commented `aarch64-unknown-linux-gnu` target build
2. Re-enabled the `aarch64-unknown-linux-musl` target build
3. Updated the build matrix to ensure proper runner selection:
- Ubuntu runners build all Linux targets
- macOS runners build only Apple Silicon targets
4. Maintained compatibility with the existing build scripts and packaging process
This expansion gives users more options for deploying on ARM64 Linux platforms, supporting both glibc and musl libc environments for maximum compatibility and performance.
Added ARM64 macOS (Apple Silicon) build target support to the CI/CD pipeline by:
1. Including `aarch64-apple-darwin` as a new build variant in the build matrix
2. Adding proper exclusion rules to ensure the target only runs on macOS runners
3. Ensuring compatibility with the existing build scripts and packaging process
This change enables native builds for Apple Silicon Macs, improving performance for users with M1/M2/M3/M4 processors while maintaining the same artifact organization and deployment process.
* add tracing instrument
* fix rebalance/decom
* modify Telemetry filter order
* feat: improve address binding and port handling mechanism (#366)
* feat: improve address binding and port handling mechanism
1. Add support for ":port" format to enable dual-stack binding (IPv4/IPv6)
2. Implement automatic port allocation when port 0 is specified
3. Optimize server startup process with unified address resolution
4. Enhance error handling and logging for address resolution
5. Improve graceful shutdown with signal listening
6. Clean up commented code in console.rs
Files:
- ecstore/src/utils/net.rs
- rustfs/src/console.rs
- rustfs/src/main.rs
Branch: feature/server-and-console-port
* improve code for console
* improve code
* improve code for console and net.rs
* Update rustfs/src/main.rs
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* Update rustfs/src/utils/mod.rs
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
---------
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* upgrade config file
* modify
* fix readme
Signed-off-by: junxiang Mu <1948535941@qq.com>
* improve readme.md
* improve code for readme.md
add chinese readme.md
* Implement Storage Service Event Notification System
Added event notification capability to the storage module, enabling the storage service to publish object operation events. Key changes include:
1. Created `event_notifier` module providing core functionality:
- `create_metadata()` - Creates event metadata objects with default configuration ID
- `send_event()` - Asynchronously sends event notifications with error handling
2. Integrated the `rustfs_event_notifier` library:
- Supports object creation, deletion, and access events
- Provides event metadata building and management
- Includes proper error propagation
These changes enable the system to trigger notifications when storage operations occur, facilitating auditing, monitoring, and integration with other systems.
* fix
---------
Signed-off-by: junxiang Mu <1948535941@qq.com>
Co-authored-by: weisd <im@weisd.in>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: junxiang Mu <1948535941@qq.com>
* feat: improve address binding and port handling mechanism
1. Add support for ":port" format to enable dual-stack binding (IPv4/IPv6)
2. Implement automatic port allocation when port 0 is specified
3. Optimize server startup process with unified address resolution
4. Enhance error handling and logging for address resolution
5. Improve graceful shutdown with signal listening
6. Clean up commented code in console.rs
Files:
- ecstore/src/utils/net.rs
- rustfs/src/console.rs
- rustfs/src/main.rs
Branch: feature/server-and-console-port
* improve code for console
* improve code
* improve code for console and net.rs
* Update rustfs/src/main.rs
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* Update rustfs/src/utils/mod.rs
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
---------
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* improve code for opentelemetry and add system metrics
* feat: add TraceLayer for HTTP service and improve metrics
- Add TraceLayer to HTTP server for request tracing
- Implement system metrics for process monitoring
- Optimize init_telemetry method for better resource management
- Add graceful shutdown handling for telemetry components
- Fix GracefulShutdown ownership issues with Arc wrapper
* improve code for init_process_observer
* remove tomlfmt.toml
* Translation comment
* improve code for console CompressionLayer params
- Add TraceLayer to HTTP server for request tracing
- Implement system metrics for process monitoring
- Optimize init_telemetry method for better resource management
- Add graceful shutdown handling for telemetry components
- Fix GracefulShutdown ownership issues with Arc wrapper
* init event notifer
* feat: implement event notification system
- Add core event notification interfaces
- Support multiple notification backends:
- Webhook (default)
- Kafka
- MQTT
- HTTP Producer
- Implement configurable event filtering
- Add async event dispatching with backpressure handling
- Provide serialization/deserialization for event payloads
This module enables system events to be published to various endpoints
with consistent delivery guarantees and failure handling.
* feat(event-notifier): improve notification system initialization safety
- Add READY atomic flag to track full initialization status
- Implement initialize_safe and start_safe methods with mutex protection
- Add wait_until_ready function with configurable timeout
- Create initialize_and_start_with_ready_check helper method
- Replace sleep-based waiting with proper readiness checks
- Add safety checks before sending events
- Replace chrono with std::time for time handling
- Update error handling to provide clear initialization status
This change reduces race conditions in multi-threaded environments
and ensures events are only processed when the system is fully ready.
* fix sql
Signed-off-by: junxiang Mu <1948535941@qq.com>
* move protobuf generate into bin crate
Signed-off-by: junxiang Mu <1948535941@qq.com>
* improve Cargo toml
* improve code
* improve code for global
* improve code
* feat(event-notifier): improve environment variable handling
- Fix deserialization error when parsing config from environment variables
- Add proper array format support for adapters configuration
- Update environment variable examples with correct format
- Improve documentation for configuration loading
- Implement helper functions for environment variable validation
This change fixes the "invalid type: map, expected a sequence" error
by ensuring proper formatting of array-type fields in environment variables.
* feat: integrate event-notifier system with rustfs
- Rename package from rustfs-event-notifier to event-notifier for consistency
- Add shutdown hooks for event notification system in main process
- Handle graceful termination of notification services on server shutdown
- Implement initialization and configuration loading for event notification
- Fix environment variable configuration to properly parse adapter arrays
- Update example code to demonstrate proper configuration usage
This change ensures proper integration between rustfs and the event
notification system, with clean startup and shutdown sequences to prevent
resource leaks during application lifecycle.
* feat: improve webhook server and run script integration
- Enhance webhook example with proper shutdown handling using tokio::select!
- Update run.sh to automatically start webhook server alongside main service
- Add event notification configuration to run.sh using environment variables
- Set proper port bindings to ensure webhook server starts on port 3000
- Improve console output for better debugging experience
- Fix race condition during service startup and shutdown
This change ensures proper integration between the webhook server and
the main rustfs service, providing a seamless development experience
with automatic service discovery and clean termination.
* improve for logger
* improve code for global.rs
* fix: modify webhook port
* fix
---------
Signed-off-by: junxiang Mu <1948535941@qq.com>
Co-authored-by: junxiang Mu <1948535941@qq.com>
- Enhance webhook example with proper shutdown handling using tokio::select!
- Update run.sh to automatically start webhook server alongside main service
- Add event notification configuration to run.sh using environment variables
- Set proper port bindings to ensure webhook server starts on port 3000
- Improve console output for better debugging experience
- Fix race condition during service startup and shutdown
This change ensures proper integration between the webhook server and
the main rustfs service, providing a seamless development experience
with automatic service discovery and clean termination.
- Rename package from rustfs-event-notifier to event-notifier for consistency
- Add shutdown hooks for event notification system in main process
- Handle graceful termination of notification services on server shutdown
- Implement initialization and configuration loading for event notification
- Fix environment variable configuration to properly parse adapter arrays
- Update example code to demonstrate proper configuration usage
This change ensures proper integration between rustfs and the event
notification system, with clean startup and shutdown sequences to prevent
resource leaks during application lifecycle.
- Fix deserialization error when parsing config from environment variables
- Add proper array format support for adapters configuration
- Update environment variable examples with correct format
- Improve documentation for configuration loading
- Implement helper functions for environment variable validation
This change fixes the "invalid type: map, expected a sequence" error
by ensuring proper formatting of array-type fields in environment variables.
- Add READY atomic flag to track full initialization status
- Implement initialize_safe and start_safe methods with mutex protection
- Add wait_until_ready function with configurable timeout
- Create initialize_and_start_with_ready_check helper method
- Replace sleep-based waiting with proper readiness checks
- Add safety checks before sending events
- Replace chrono with std::time for time handling
- Update error handling to provide clear initialization status
This change reduces race conditions in multi-threaded environments
and ensures events are only processed when the system is fully ready.
- Add core event notification interfaces
- Support multiple notification backends:
- Webhook (default)
- Kafka
- MQTT
- HTTP Producer
- Implement configurable event filtering
- Add async event dispatching with backpressure handling
- Provide serialization/deserialization for event payloads
This module enables system events to be published to various endpoints
with consistent delivery guarantees and failure handling.
- Change build target from `x86_64-unknown-linux-gnu` to `x86_64-unknown-linux-musl`
- Default install `x86_64-unknown-linux-musl` toolchain in setup action
- Add `musl-tools` and `build-essential` to system dependencies
* feature/observability: (27 commits)
modify default value
TryInto cover
upgrade reqwest version from 0.12.12 to 0.12.15
improve code for FileSink
improve code for config and FileSink
webhook add auth_token
upgrade docker images
feat(obs): enhance OpenTelemetry configuration and logging
merge main
feat: add metrics_handler
add prometheus
upgrade opentelemetry create from 0.28.0 to 0.29.0
update .gitignore
replace log to tracing
improve code for observability
improve code for main
feat(observability): add obs_config option and document stdout export
improve logger entry for Observability
improve log struct
improve code
...
Improve observability setup with the following changes:
- Replace static OnceCell with tokio::sync::OnceCell for guard management
- Add logger_level to OtelConfig for configurable tracing verbosity
- Improve telemetry initialization with better error handling
- Enhance logging filters and span configuration
Breaking Changes:
- Configuration now requires logger_level field
- Global guard management uses async-safe primitives
Example config:
observability:
endpoint: "http://localhost:4317"
logger_level: "debug" # New required field
- Add obs_config parameter to config struct with default path
- Document how to modify use_stdout value in README.md
- Support configuring observability via file or environment variables
This change helps users configure telemetry output destination for better
observability options in different deployment scenarios.
1. Added a new build process `build-rustfs-gui` in `build.yaml` to streamline the build operations for the RustFS GUI.
2. Optimized `cli/rustfs-gui/utils/helper.rs` by using `rust-embed` to embed the `rustfs` resources directly into the binary.
Ensure the static directory is copied as a sibling to the rustfs binary
in the artifact package, maintaining the proper directory structure
for the application to locate its resources.
- Fix artifact pattern matching in merge step to use consistent name pattern
- Standardize artifact naming convention with clear format
- Improve static assets handling with simpler download and extraction
- Add 7-day retention policy for artifacts to manage storage
- Use output variables to ensure consistency between artifact names and paths
- Optimize package creation process to eliminate redundant nesting
- Enhance error handling and script readability
- Implemented `KafkaAuditTarget` to send audit entries to a Kafka topic.
- Implemented `WebhookAuditTarget` to send audit entries to a specified webhook URL.
- Updated `AuditLogger` to support multiple audit targets including Kafka and Webhook.
- Added examples and documentation for the new audit targets.
- Add initialization logic for the `rustfs-logging` crate.
- Provide examples for logging utilities.
- Include tests for logging and telemetry functionalities.
- Ensure proper configuration of dependencies in `Cargo.toml`.
* Improve init_logger function and format code with cargo fmt
- Enhance `init_logger` function for better logging configuration.
- Apply `cargo fmt` to format the codebase.
* Refactor main.rs for clarity and modularity in rustfs-gui
- Move `init_logger` function to `utils/logger.rs` for better separation of concerns.
- Create a dedicated `router` module in `router/router.rs`.
- Update `main.rs` to use the new `logger` and `router` modules.
- Apply `cargo fmt` to format the codebase.
- Create `SettingViews` component in `views/setting.rs`.
- Update `views/mod.rs` to include the new `SettingViews` component.
- Update routing in `main.rs` to include the new route for `SettingViews`.
- Create `Home` component in `components/Home.rs`.
- Create `Navbar` component in `components/navbar.rs`.
- Create `HomeView` component in `views/home.rs`.
- Update `views/mod.rs` to include new views.
- Update routing in `main.rs` to include new routes for `HomeView`.
* Add new CSS, JS, and icon assets to rustfs-gui
- Add `navbar.css` for styling the navigation bar.
- Add `sts.js` for handling tab switching and password toggling.
- Include new icon assets in `assets/icons` directory.
- Update `Dioxus.toml` to reference new assets.
* Remove unused icons from dioxus.toml
- Remove `assets/icons/icon_*.png` from the icon list.
* Initialize rustfs-gui crate
- Set up the `rustfs-gui` package in the workspace.
- Add `dioxus` dependency with `router` feature.
- Configure workspace settings for edition, license, repository, and rust-version.
- Include lints configuration for the workspace.
* Add utility functions and configurations for rustfs-gui
- Implement `RustFSConfig` struct with default values and methods for loading, saving, and clearing configurations.
- Add `ServiceCommand` enum for managing service commands.
- Implement `ServiceManager` struct for starting, stopping, and restarting the service.
- Include helper functions for checking service status, preparing the service, downloading files, and unzipping files.
- Add logging and error handling for service operations.
- Set up the `rustfs-gui` package in the workspace.
- Add `dioxus` dependency with `router` feature.
- Configure workspace settings for edition, license, repository, and rust-version.
- Include lints configuration for the workspace.
7. **Commit and push to feature branch**: `git push origin feat/your-feature-name`
8. **Create Pull Request**: Use `gh pr create` (MANDATORY)
9. **Wait for PR approval**: NO self-merging allowed
10. **Merge through GitHub interface**: ONLY after approval
### 🔒 ENFORCEMENT MECHANISMS
- **Branch protection rules**: Main branch is protected
- **Pre-commit hooks**: Will block direct commits to main
- **CI/CD checks**: All PRs must pass before merging
- **Code review requirement**: At least one approval needed
- **Automated reversal**: Direct commits to main will be automatically reverted
## Project Overview
RustFS is a high-performance distributed object storage system written in Rust, compatible with S3 API. The project adopts a modular architecture, supporting erasure coding storage, multi-tenant management, observability, and other enterprise-level features.
- All storage operations must support erasure coding
- Implement read/write quorum mechanisms
- Support data integrity verification
### 2. Network Communication
- Use gRPC for internal service communication
- HTTP/HTTPS support for S3-compatible API
- Implement connection pooling and retry mechanisms
### 3. Metadata Management
- Use FlatBuffers for serialization
- Support version control and migration
- Implement metadata caching
@@ -540,40 +834,94 @@ These rules should serve as guiding principles when developing the RustFS projec
### 4. Code Operations
#### Branch Management
- **🚨 CRITICAL: NEVER modify code directly on main or master branch - THIS IS ABSOLUTELY FORBIDDEN 🚨**
- **⚠️ ANY DIRECT COMMITS TO MASTER/MAIN WILL BE REJECTED AND MUST BE REVERTED IMMEDIATELY ⚠️**
- **Always work on feature branches - NO EXCEPTIONS**
- Always check the .cursorrules file before starting to ensure you understand the project guidelines
- **MANDATORY workflow for ALL changes:**
- **🚨 CRITICAL: NEVER modify code directly on main or master branch - THIS IS ABSOLUTELY FORBIDDEN 🚨**
- **⚠️ ANY DIRECT COMMITS TO MASTER/MAIN WILL BE REJECTED AND MUST BE REVERTED IMMEDIATELY ⚠️**
- **🔒 ALL CHANGES MUST GO THROUGH PULL REQUESTS - NO DIRECT COMMITS TO MAIN UNDER ANY CIRCUMSTANCES 🔒**
- **Always work on feature branches - NO EXCEPTIONS**
- Always check the .cursorrules file before starting to ensure you understand the project guidelines
- **MANDATORY workflow for ALL changes:**
1. `git checkout main` (switch to main branch)
2. `git pull` (get latest changes)
3. `git checkout -b feat/your-feature-name` (create and switch to feature branch)
4. Make your changes ONLY on the feature branch
5. Test thoroughly before committing
6. Commit and push to the feature branch
7. Create a pull request for code review
- Use descriptive branch names following the pattern: `feat/feature-name`, `fix/issue-name`, `refactor/component-name`, etc.
- **Double-check current branch before ANY commit: `git branch` to ensure you're NOT on main/master**
- Ensure all changes are made on feature branches and merged through pull requests
7. **Create a pull request for code review - THIS IS THE ONLY WAY TO MERGE TO MAIN**
8. **Wait for PR approval before merging - NEVER merge your own PRs without review**
- Use descriptive branch names following the pattern: `feat/feature-name`, `fix/issue-name`, `refactor/component-name`, etc.
- **Double-check current branch before ANY commit: `git branch` to ensure you're NOT on main/master**
- **Pull Request Requirements:**
- All changes must be submitted via PR regardless of size or urgency
- PRs must include comprehensive description and testing information
- PRs must pass all CI/CD checks before merging
- PRs require at least one approval from code reviewers
- Even hotfixes and emergency changes must go through PR process
- **Enforcement:**
- Main branch should be protected with branch protection rules
- Direct pushes to main should be blocked by repository settings
- Any accidental direct commits to main must be immediately reverted via PR
#### Development Workflow
- Use English for all code comments, documentation, and variable names
- Write meaningful and descriptive names for variables, functions, and methods
- Avoid meaningless test content like "debug 111" or placeholder values
- Before each change, carefully read the existing code to ensure you understand the code structure and implementation, do not break existing logic implementation, do not introduce new issues
- Ensure each change provides sufficient test cases to guarantee code correctness
- Do not arbitrarily modify numbers and constants in test cases, carefully analyze their meaning to ensure test case correctness
- When writing or modifying tests, check existing test cases to ensure they have scientific naming and rigorous logic testing, if not compliant, modify test cases to ensure scientific and rigorous testing
- **Before committing any changes, run `cargo clippy --all-targets --all-features -- -D warnings` to ensure all code passes Clippy checks**
- After each development completion, first git add . then git commit -m "feat: feature description" or "fix: issue description", ensure compliance with [Conventional Commits](https://www.conventionalcommits.org/en/v1.0.0/)
- **Keep commit messages concise and under 72 characters** for the title line, use body for detailed explanations if needed
- After each development completion, first git push to remote repository
- After each change completion, summarize the changes, do not create summary files, provide a brief change description, ensure compliance with [Conventional Commits](https://www.conventionalcommits.org/en/v1.0.0/)
- Provide change descriptions needed for PR in the conversation, ensure compliance with [Conventional Commits](https://www.conventionalcommits.org/en/v1.0.0/)
- **Always provide PR descriptions in English** after completing any changes, including:
- Clear and concise title following Conventional Commits format
- Detailed description of what was changed and why
- List of key changes and improvements
- Any breaking changes or migration notes if applicable
- Testing information and verification steps
- **Provide PR descriptions in copyable markdown format** enclosed in code blocks for easy one-click copying
## 🎯 **Core Development Principles**
- **🔴 Every change must be precise - don't modify unless you're confident**
- Carefully analyze code logic and ensure complete understanding before making changes
- When uncertain, prefer asking users or consulting documentation over blind modifications
- Use small iterative steps, modify only necessary parts at a time
- Evaluate impact scope before changes to ensure no new issues are introduced
- Prefer using `gh pr create` command to create Pull Requests
- Avoid having users manually create PRs through web interface
- Provide clear and professional PR titles and descriptions
- Using `gh` commands ensures better integration and automation
## 📝 **Code Quality Requirements**
- Use English for all code comments, documentation, and variable names
- Write meaningful and descriptive names for variables, functions, and methods
- Avoid meaningless test content like "debug 111" or placeholder values
- Before each change, carefully read the existing code to ensure you understand the code structure and implementation, do not break existing logic implementation, do not introduce new issues
- Ensure each change provides sufficient test cases to guarantee code correctness
- Do not arbitrarily modify numbers and constants in test cases, carefully analyze their meaning to ensure test case correctness
- When writing or modifying tests, check existing test cases to ensure they have scientific naming and rigorous logic testing, if not compliant, modify test cases to ensure scientific and rigorous testing
- **Before committing any changes, run `cargo clippy --all-targets --all-features -- -D warnings` to ensure all code passes Clippy checks**
- After each development completion, first git add . then git commit -m "feat: feature description" or "fix: issue description", ensure compliance with [Conventional Commits](https://www.conventionalcommits.org/en/v1.0.0/)
- **Keep commit messages concise and under 72 characters** for the title line, use body for detailed explanations if needed
- After each development completion, first git push to remote repository
- After each change completion, summarize the changes, do not create summary files, provide a brief change description, ensure compliance with [Conventional Commits](https://www.conventionalcommits.org/en/v1.0.0/)
- Provide change descriptions needed for PR in the conversation, ensure compliance with [Conventional Commits](https://www.conventionalcommits.org/en/v1.0.0/)
- **Always provide PR descriptions in English** after completing any changes, including:
- Clear and concise title following Conventional Commits format
- Detailed description of what was changed and why
- List of key changes and improvements
- Any breaking changes or migration notes if applicable
- Testing information and verification steps
- **Provide PR descriptions in copyable markdown format** enclosed in code blocks for easy one-click copying
Thank you for your contribution! Please ensure your PR follows the community standards ([CODE_OF_CONDUCT.md](CODE_OF_CONDUCT.md)) and sign the CLA if this is your first contribution.
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
name:"issue-translator"
on:
issue_comment:
types:[created]
issues:
types:[opened]
jobs:
build:
runs-on:ubuntu-latest
steps:
- uses:usthe/issues-translate-action@v2.7
with:
IS_MODIFY_TITLE:false
# not require, default false, . Decide whether to modify the issue title
# if true, the robot account @Issues-translate-bot must have modification permissions, invite @Issues-translate-bot to your project or use your custom bot.
CUSTOM_BOT_NOTE:Bot detected the issue body's language is not English, translate it automatically.
# not require. Customize the translation robot prefix message.
Thank you for your interest in contributing documentation and related software code to a project hosted or managed by RustFS. In order to clarify the intellectual property license granted with Contributions from any person or entity, RustFS must have a Contributor License Agreement (“CLA”) on file that has been signed by each Contributor, indicating agreement to the license terms below. This version of the Contributor License Agreement allows an individual to submit Contributions to the applicable project. If you are making a submission on behalf of a legal entity, then you should sign the separate Corporate Contributor License Agreement.
You accept and agree to the following terms and conditions for Your present and future Contributions submitted to RustFS. You hereby irrevocably assign and transfer to RustFS all right, title, and interest in and to Your Contributions, including all copyrights and other intellectual property rights therein.
Definitions
“You” (or “Your”) shall mean the copyright owner or legal entity authorized by the copyright owner that is making this Agreement with RustFS. For legal entities, the entity making a Contribution and all other entities that control, are controlled by, or are under common control with that entity are considered to be a single Contributor. For the purposes of this definition, “control” means (i) the power, direct or indirect, to cause the direction or management of such entity, whether by contract or otherwise, or (ii) ownership of fifty percent (50%) or more of the outstanding shares, or (iii) beneficial ownership of such entity.
“Contribution” shall mean any original work of authorship, including any modifications or additions to an existing work, that is intentionally submitted by You to RustFS for inclusion in, or documentation of, any of the products or projects owned or managed by RustFS (the “Work”), including without limitation any Work described in Schedule A. For the purposes of this definition, “submitted” means any form of electronic or written communication sent to RustFS or its representatives, including but not limited to communication on electronic mailing lists, source code control systems, and issue tracking systems that are managed by, or on behalf of, RustFS for the purpose of discussing and improving the Work.
Assignment of Copyright
Subject to the terms and conditions of this Agreement, You hereby irrevocably assign and transfer to RustFS all right, title, and interest in and to Your Contributions, including all copyrights and other intellectual property rights therein, for the entire term of such rights, including all renewals and extensions. You agree to execute all documents and take all actions as may be reasonably necessary to vest in RustFS the ownership of Your Contributions and to assist RustFS in perfecting, maintaining, and enforcing its rights in Your Contributions.
Grant of Patent License
Subject to the terms and conditions of this Agreement, You hereby grant to RustFS and to recipients of documentation and software distributed by RustFS a perpetual, worldwide, non-exclusive, no-charge, royalty-free, irrevocable (except as stated in this section) patent license to make, have made, use, offer to sell, sell, import, and otherwise transfer the Work, where such license applies only to those patent claims licensable by You that are necessarily infringed by Your Contribution(s) alone or by combination of Your Contribution(s) with the Work to which such Contribution(s) was submitted. If any entity institutes patent litigation against You or any other entity (including a cross-claim or counterclaim in a lawsuit) alleging that your Contribution, or the Work to which you have contributed, constitutes direct or contributory patent infringement, then any patent licenses granted to that entity under this Agreement for that Contribution or Work shall terminate as of the date such litigation is filed.
You represent that you are legally entitled to grant the above assignment and license.
You represent that each of Your Contributions is Your original creation (see section 7 for submissions on behalf of others). You represent that Your Contribution submissions include complete details of any third-party license or other restriction (including, but not limited to, related patents and trademarks) of which you are personally aware and which are associated with any part of Your Contributions.
You are not expected to provide support for Your Contributions, except to the extent You desire to provide support. You may provide support for free, for a fee, or not at all. Unless required by applicable law or agreed to in writing, You provide Your Contributions on an “AS IS” BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied, including, without limitation, any warranties or conditions of TITLE, NON- INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A PARTICULAR PURPOSE.
Should You wish to submit work that is not Your original creation, You may submit it to RustFS separately from any Contribution, identifying the complete details of its source and of any license or other restriction (including, but not limited to, related patents, trademarks, and license agreements) of which you are personally aware, and conspicuously marking the work as “Submitted on behalf of a third-party: [named here]”.
You agree to notify RustFS of any facts or circumstances of which you become aware that would make these representations inaccurate in any respect.
Modification of CLA
RustFS reserves the right to update or modify this CLA in the future. Any updates or modifications to this CLA shall apply only to Contributions made after the effective date of the revised CLA. Contributions made prior to the update shall remain governed by the version of the CLA that was in effect at the time of submission. It is not necessary for all Contributors to re-sign the CLA when the CLA is updated or modified.
Governing Law and Dispute Resolution
This Agreement will be governed by and construed in accordance with the laws of the People’s Republic of China excluding that body of laws known as conflict of laws. The parties expressly agree that the United Nations Convention on Contracts for the International Sale of Goods will not apply. Any legal action or proceeding arising under this Agreement will be brought exclusively in the courts located in Beijing, China, and the parties hereby irrevocably consent to the personal jurisdiction and venue therein.
For your reading convenience, this Agreement is written in parallel English and Chinese sections. To the extent there is a conflict between the English and Chinese sections, the English sections shall govern.
echo"💡 Make sure the package ${PACKAGE_NAME} exists";\
echo"🔗 Check: ${DOWNLOAD_URL}";\
exit 1;\
fi&&\
unzip /build/rustfs.zip -d /build &&\
chmod +x /build/rustfs &&\
rm /build/rustfs.zip &&\
echo"✅ Successfully downloaded and extracted rustfs binary"
# Runtime stage
FROMalpine:latest
# RUN apk add --no-cache <package-name>
# Set build arguments and labels
ARGRELEASE=latest
ARG BUILD_DATE
ARG VCS_REF
WORKDIR/app
LABELname="RustFS"\
vendor="RustFS Team"\
maintainer="RustFS Team <dev@rustfs.com>"\
version="${RELEASE}"\
release="${RELEASE}"\
build-date="${BUILD_DATE}"\
vcs-ref="${VCS_REF}"\
summary="RustFS is a high-performance distributed object storage system written in Rust, compatible with S3 API."\
description="RustFS is a high-performance distributed object storage software built using Rust. It supports erasure coding storage, multi-tenant management, observability, and other enterprise-level features."\
url="https://rustfs.com"\
license="Apache-2.0"
RUN mkdir -p /data/rustfs0 /data/rustfs1 /data/rustfs2 /data/rustfs3
RustFS is a high-performance distributed object storage software built using Rust, one of the most popular languages worldwide. Along with MinIO, it shares a range of advantages such as simplicity, S3 compatibility, open-source nature, support for data lakes, AI, and big data. Furthermore, it has a better and more user-friendly open-source license in comparison to other storage systems, being constructed under the Apache license. As Rust serves as its foundation, RustFS provides faster speed and safer distributed features for high-performance object storage.
```bash
cargo run --bin gproto
```
> ⚠️ **RustFS is under rapid development. Do NOT use in production environments!**
#### Using Docker for Prerequisites
## Features
```yaml
- uses:arduino/setup-protoc@v3
with:
version:"30.2"
- **High Performance**: Built with Rust, ensuring speed and efficiency.
- **Distributed Architecture**: Scalable and fault-tolerant design for large-scale deployments.
- **S3 Compatibility**: Seamless integration with existing S3-compatible applications.
- **Data Lake Support**: Optimized for big data and AI workloads.
- **Open Source**: Licensed under Apache 2.0, encouraging community contributions and transparency.
- **User-Friendly**: Designed with simplicity in mind, making it easy to deploy and manage.
| Developed based on Rust language, memory is safer | Developed in Go or C, with potential issues like memory GC/leaks |
| Does not report logs to third-party countries | Reporting logs to other third countries may violate national security laws |
| Licensed under Apache, more business-friendly | AGPL V3 License and other License, polluted open source and License traps, infringement of intellectual property rights |
| Comprehensive S3 support, works with domestic and international cloud providers | Full support for S3, but no local cloud vendor support |
| Rust-based development, strong support for secure and innovative devices | Poor support for edge gateways and secure innovative devices|
| Stable commercial prices, free community support | High pricing, with costs up to $250,000 for 1PiB |
| No risk | Intellectual property risks and risks of prohibited uses |
make docker-buildx-version VERSION=v1.0.0 # Build specific version
make help-docker # Show all Docker-related commands
```
4. **Access the Console**: Open your web browser and navigate to `http://localhost:9000` to access the RustFS console, default username and password is `rustfsadmin` .
5.**Create a Bucket**: Use the console to create a new bucket for your objects.
6.**Upload Objects**: You can upload files directly through the console or use S3-compatible APIs to interact with your RustFS instance.
## Documentation
For detailed documentation, including configuration options, API references, and advanced usage, please visit our [Documentation](https://docs.rustfs.com).
## Getting Help
If you have any questions or need assistance, you can:
- Check the [FAQ](https://github.com/rustfs/rustfs/discussions/categories/q-a) for common issues and solutions.
- Join our [GitHub Discussions](https://github.com/rustfs/rustfs/discussions) to ask questions and share your experiences.
- Open an issue on our [GitHub Issues](https://github.com/rustfs/rustfs/issues) page for bug reports or feature requests.
## Links
- [Documentation](https://docs.rustfs.com) - The manual you should read
- [Changelog](https://github.com/rustfs/rustfs/releases) - What we broke and fixed
- [GitHub Discussions](https://github.com/rustfs/rustfs/discussions) - Where the community lives
RustFS is a community-driven project, and we appreciate all contributions. Check out the [Contributors](https://github.com/rustfs/rustfs/graphs/contributors) page to see the amazing people who have helped make RustFS better.
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
modconfig;
modhelper;
modlogger;
Some files were not shown because too many files have changed in this diff
Show More
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.