mirror of
https://github.com/rustfs/rustfs.git
synced 2026-01-17 09:40:32 +00:00
* feat: implement multi-channel release system with artifact naming
- Add dedicated release.yml workflow for handling GitHub releases
- Refactor build.yml to support dev/release/prerelease artifact naming
- Update docker.yml to support version-specific image tagging
- Implement artifact naming rules:
- Dev: rustfs-{platform}-{arch}-dev-{sha}.zip
- Release: rustfs-{platform}-{arch}-v{version}.zip
- Prerelease: rustfs-{platform}-{arch}-v{version}.zip
- Add OSS upload directory separation (dev/ vs release/)
- Only stable releases update latest.json and create latest tags
- Separate GitHub Release creation from build workflow
- Add comprehensive build summaries and status reporting
This enables proper multi-channel distribution with clear artifact
identification and prevents confusion between dev and stable releases.
* fix: support version tags without v prefix (1.0.0 instead of v1.0.0)
- Update trigger patterns from 'v*.*.*' to '*.*.*' in all workflows
- Fix version extraction logic to handle tags without v prefix
- Maintain backward compatibility with existing logic
Note: Artifact naming still includes 'v' prefix for clarity
(e.g., tag '1.0.0' creates 'rustfs-linux-x86_64-v1.0.0.zip')
* feat: update Dockerfile to support multi-channel release system
- Add build arguments for VERSION, BUILD_TYPE, and TARGETARCH
- Support dynamic artifact download based on build type:
- Development: downloads from artifacts/rustfs/dev/
- Release: downloads from artifacts/rustfs/release/
- Auto-generate correct filenames based on new naming convention:
- Dev: rustfs-linux-{arch}-dev-{sha}.zip
- Release: rustfs-linux-{arch}-v{version}.zip
- Add architecture mapping for multi-platform builds
- Pass BUILD_TYPE parameter from docker.yml workflow
- Improve error handling with helpful download path suggestions
This ensures Docker images use the correct pre-built binaries
from the new multi-channel release system.
* feat: optimize and consolidate Dockerfile structure
## Major Improvements:
### ✅ Created Missing Files
- Add .docker/Dockerfile.alpine for lightweight Alpine-based builds
- Support both pre-built binary download and source compilation
### 🔧 Fixed Critical Issues
- Fix Dockerfile.obs: ubuntu:latest → ubuntu:22.04 (stable version)
- Add proper security practices (non-root user, health checks)
- Add proper error handling and environment variables
### 🗑️ Eliminated Redundancy
- Remove .docker/Dockerfile.ubuntu22.04 (duplicate of devenv)
- Update docker.yml workflow to use devenv for ubuntu variant
- Consolidate similar functionality into fewer, better files
### 🚀 Enhanced Functionality
- Make devenv Dockerfile dual-purpose (dev environment + runtime)
- Add VERSION/BUILD_TYPE support for dynamic binary downloads
- Improve security with proper user management
- Add comprehensive health checks and error handling
### 📊 Final Dockerfile Structure:
1. Dockerfile (production, Alpine-based, pre-built binaries)
2. Dockerfile.multi-stage (full source builds, Ubuntu-based)
3. Dockerfile.obs (observability builds, Ubuntu-based)
4. .docker/Dockerfile.alpine (lightweight Alpine variant)
5. .docker/Dockerfile.devenv (development + ubuntu variant)
6. .docker/Dockerfile.rockylinux9.3 (RockyLinux variant)
This reduces redundancy while maintaining all necessary build variants
and improving maintainability across the entire container ecosystem.
* refactor: streamline Dockerfile structure and remove unused files
## 🎯 Major Cleanup:
### 🗑️ Removed Unused Files (2 files)
- Delete Dockerfile.obs (not referenced anywhere)
- Delete .docker/Dockerfile.rockylinux9.3 (not referenced anywhere)
### 📁 Reorganized File Layout
- Move Dockerfile.multi-stage → .docker/Dockerfile.multi-stage
- Update docker-compose.yml to use new path
- Keep main Dockerfile in root (production use)
- Consolidate variants in .docker/ directory
### ✅ Final Clean Structure:
### 📊 Before vs After:
- **Before**: 7 files (1 missing, 2 unused, scattered layout)
- **After**: 4 files (all used, organized layout)
- **Reduction**: 43% fewer files, 100% utilization
This eliminates confusion and reduces maintenance overhead while
keeping all actually needed functionality intact.
* refactor: implement comprehensive Docker tag strategy with production variant
- Restore production variant as default with explicit naming
- Add support for prerelease channels (alpha, beta, rc)
- Implement rolling development tags (dev, dev-variant)
- Support semantic versioning with variant combinations
- Update documentation with complete tag strategy examples
- Align with GPT-suggested comprehensive tagging approach
Tag examples:
- rustfs/rustfs:1.2.3 (main production)
- rustfs/rustfs:1.2.3-production (explicit production)
- rustfs/rustfs:1.2.3-alpine (Alpine variant)
- rustfs/rustfs:alpha (latest alpha)
- rustfs/rustfs:dev (latest development)
- rustfs/rustfs:dev-13e4a0b (specific commit)
* perf: optimize Docker build speed with comprehensive caching and compilation improvements
- Add dual caching strategy: GitHub Actions + Registry cache
- Implement sccache for Rust compilation caching across builds
- Configure parallel compilation with all available CPU cores
- Add optimized cargo configuration for faster builds
- Enable sparse registry protocol for dependency resolution
- Configure LLD linker for faster linking
- Add BuildKit optimizations with inline cache
- Disable provenance/SBOM generation for faster builds
- Document build performance improvements and timings
Performance improvements:
- Source builds: ~40-50% faster with cache hits
- Pre-built binaries: ~30-40% faster
- Parallel matrix builds reduce total CI time significantly
- Registry cache provides persistent cross-run benefits
* refactor: consolidate Docker variants and eliminate duplication
- Replace root Dockerfile with enhanced Alpine prebuild version
- Remove redundant alpine variant from build matrix
- Root Dockerfile now includes:
- Non-root user security
- Health checks
- Better error handling
- protoc/flatc tool support
- Update documentation to reflect simplified 4-variant strategy
- Remove duplicate .docker/alpine/Dockerfile.prebuild
Build matrix now:
- production (root Dockerfile - Alpine prebuild)
- alpine-source (Alpine source build)
- ubuntu (Ubuntu prebuild)
- ubuntu-source (Ubuntu source build)
Benefits:
- Eliminates functional duplication
- Improves security with non-root execution
- Maintains same image variants with better quality
- Simplifies maintenance
* fix: restore alpine variant for better user choice
- Restore alpine variant (rustfs/rustfs:1.2.3-alpine)
- Re-add .docker/alpine/Dockerfile.prebuild
- Update build matrix to include 5 variants again:
- production (default)
- alpine (explicit Alpine choice)
- alpine-source (Alpine source build)
- ubuntu (Ubuntu pre-built)
- ubuntu-source (Ubuntu source build)
- Update documentation to reflect restored alpine tags
- Fix build performance table to include all variants
User feedback: Alpine variant provides explicit choice even if
similar to production variant. Better UX with clear options.
* fix: remove redundant rustup target add commands in Alpine Dockerfiles
- Remove 'rustup target add x86_64-unknown-linux-musl' from Alpine source build
- Remove redundant target add from Alpine prebuild fallback path
- Remove redundant target add from root Dockerfile fallback path
Reason: rust:alpine base image already has x86_64-unknown-linux-musl
as the default target since Alpine uses musl libc by default.
Thanks to @houseme for spotting this redundancy in code review.
* fix: add missing RUSTFS_VOLUMES environment variable in Dockerfiles
- Add RUSTFS_VOLUMES=/data to all Dockerfile variants
- This fixes the issue where CMD ['/app/rustfs'] was used without providing the required volumes parameter
- The volumes parameter is required by the application and can be provided via command line or RUSTFS_VOLUMES environment variable
* fix: update docker-compose configurations to ensure all environments work correctly
- Added missing access key and secret key environment variables to docker-compose.yaml
- This ensures the distributed test environment has proper authentication credentials
- Complementary fix to the previous Dockerfile updates for consistent configuration
* fix: recreate missing Dockerfile.obs with complete content
- The file was accidentally left empty after initial creation
- Now contains proper Ubuntu-based configuration for observability environment
- Includes all necessary environment variables including RUSTFS_VOLUMES
- Supports docker-compose-obs.yaml configuration
* refactor: organize Docker Compose configurations and eliminate duplication
- Move specialized configurations to .docker/compose/ directory
- Rename docker-compose.yaml → docker-compose.cluster.yaml (distributed testing)
- Rename docker-compose-obs.yaml → docker-compose.observability.yaml (observability testing)
- Keep docker-compose.yml as the main production configuration
- Add comprehensive README explaining different configuration purposes
- Eliminates confusion between similar filenames
- Provides clear guidance on when to use each configuration
* fix: correct relative paths in moved Docker Compose configurations
- Fix binary volume mount paths in docker-compose.cluster.yaml (./target → ../../target)
- Fix Dockerfile.obs context path in docker-compose.observability.yaml (. → ../..)
- Fix observability config file paths (./.docker → ../../.docker)
- Update README.md with correct usage instructions for new locations
- All configurations now correctly reference files relative to their new positions
* refactor: move Dockerfile.obs to .docker/compose/ directory for better organization
- Move Dockerfile.obs from root to .docker/compose/ directory
- Update all dockerfile references in docker-compose.observability.yaml
- Keep related files (Dockerfile.obs + docker-compose.observability.yaml) together
- Clean up root directory by removing specialized-purpose Dockerfile
- Update README.md to document new file organization
- Improves project structure and file discoverability
* refactor: improve Docker build configuration for better clarity
- Move Dockerfile.obs back to project root for simpler build context
- Update docker-compose.observability.yaml to use cleaner dockerfile reference
- Change from '.docker/compose/Dockerfile.obs' to simply 'Dockerfile.obs'
- Maintain context as '../..' for access to project files
- Remove redundant Dockerfile.obs documentation from compose README
- This follows Docker best practices: simple context + Dockerfile at context root
* wip
222 lines
5.6 KiB
YAML
222 lines
5.6 KiB
YAML
# Copyright 2024 RustFS Team
|
|
#
|
|
# Licensed under the Apache License, Version 2.0 (the "License");
|
|
# you may not use this file except in compliance with the License.
|
|
# You may obtain a copy of the License at
|
|
#
|
|
# http://www.apache.org/licenses/LICENSE-2.0
|
|
#
|
|
# Unless required by applicable law or agreed to in writing, software
|
|
# distributed under the License is distributed on an "AS IS" BASIS,
|
|
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
|
# See the License for the specific language governing permissions and
|
|
# limitations under the License.
|
|
|
|
version: "3.8"
|
|
|
|
services:
|
|
# RustFS main service
|
|
rustfs:
|
|
security_opt:
|
|
- "no-new-privileges:true"
|
|
image: rustfs/rustfs:latest
|
|
container_name: rustfs-server
|
|
build:
|
|
context: .
|
|
dockerfile: .docker/ubuntu/Dockerfile.source
|
|
args:
|
|
TARGETPLATFORM: linux/amd64
|
|
ports:
|
|
- "9000:9000" # S3 API port
|
|
environment:
|
|
- RUSTFS_VOLUMES=/data/rustfs0,/data/rustfs1,/data/rustfs2,/data/rustfs3
|
|
- RUSTFS_ADDRESS=0.0.0.0:9000
|
|
- RUSTFS_CONSOLE_ENABLE=true
|
|
- RUSTFS_ACCESS_KEY=rustfsadmin
|
|
- RUSTFS_SECRET_KEY=rustfsadmin
|
|
- RUSTFS_LOG_LEVEL=info
|
|
- RUSTFS_OBS_ENDPOINT=http://otel-collector:4317
|
|
volumes:
|
|
- rustfs_data_0:/data/rustfs0
|
|
- rustfs_data_1:/data/rustfs1
|
|
- rustfs_data_2:/data/rustfs2
|
|
- rustfs_data_3:/data/rustfs3
|
|
- ./logs:/app/logs
|
|
networks:
|
|
- rustfs-network
|
|
restart: unless-stopped
|
|
healthcheck:
|
|
test:
|
|
[
|
|
"CMD",
|
|
"wget",
|
|
"--no-verbose",
|
|
"--tries=1",
|
|
"--spider",
|
|
"http://localhost:9000/health",
|
|
]
|
|
interval: 30s
|
|
timeout: 10s
|
|
retries: 3
|
|
start_period: 40s
|
|
depends_on:
|
|
- otel-collector
|
|
|
|
# Development environment
|
|
rustfs-dev:
|
|
image: rustfs/rustfs:devenv
|
|
container_name: rustfs-dev
|
|
build:
|
|
context: .
|
|
dockerfile: .docker/ubuntu/Dockerfile.dev
|
|
# Pure development environment
|
|
ports:
|
|
- "9010:9000"
|
|
environment:
|
|
- RUSTFS_VOLUMES=/data/rustfs0,/data/rustfs1
|
|
- RUSTFS_ADDRESS=0.0.0.0:9000
|
|
- RUSTFS_CONSOLE_ENABLE=true
|
|
- RUSTFS_ACCESS_KEY=devadmin
|
|
- RUSTFS_SECRET_KEY=devadmin
|
|
- RUSTFS_LOG_LEVEL=debug
|
|
volumes:
|
|
- .:/app # Mount source code to /app for development
|
|
- rustfs_dev_data:/data
|
|
networks:
|
|
- rustfs-network
|
|
restart: unless-stopped
|
|
profiles:
|
|
- dev
|
|
|
|
# OpenTelemetry Collector
|
|
otel-collector:
|
|
image: otel/opentelemetry-collector-contrib:latest
|
|
container_name: otel-collector
|
|
command:
|
|
- --config=/etc/otelcol-contrib/otel-collector.yml
|
|
volumes:
|
|
- ./.docker/observability/otel-collector.yml:/etc/otelcol-contrib/otel-collector.yml:ro
|
|
ports:
|
|
- "4317:4317" # OTLP gRPC receiver
|
|
- "4318:4318" # OTLP HTTP receiver
|
|
- "8888:8888" # Prometheus metrics
|
|
- "8889:8889" # Prometheus exporter metrics
|
|
networks:
|
|
- rustfs-network
|
|
restart: unless-stopped
|
|
profiles:
|
|
- observability
|
|
|
|
# Jaeger for tracing
|
|
jaeger:
|
|
image: jaegertracing/all-in-one:latest
|
|
container_name: jaeger
|
|
ports:
|
|
- "16686:16686" # Jaeger UI
|
|
- "14250:14250" # Jaeger gRPC
|
|
environment:
|
|
- COLLECTOR_OTLP_ENABLED=true
|
|
networks:
|
|
- rustfs-network
|
|
restart: unless-stopped
|
|
profiles:
|
|
- observability
|
|
|
|
# Prometheus for metrics
|
|
prometheus:
|
|
image: prom/prometheus:latest
|
|
container_name: prometheus
|
|
ports:
|
|
- "9090:9090"
|
|
volumes:
|
|
- ./.docker/observability/prometheus.yml:/etc/prometheus/prometheus.yml:ro
|
|
- prometheus_data:/prometheus
|
|
command:
|
|
- "--config.file=/etc/prometheus/prometheus.yml"
|
|
- "--storage.tsdb.path=/prometheus"
|
|
- "--web.console.libraries=/etc/prometheus/console_libraries"
|
|
- "--web.console.templates=/etc/prometheus/consoles"
|
|
- "--storage.tsdb.retention.time=200h"
|
|
- "--web.enable-lifecycle"
|
|
networks:
|
|
- rustfs-network
|
|
restart: unless-stopped
|
|
profiles:
|
|
- observability
|
|
|
|
# Grafana for visualization
|
|
grafana:
|
|
image: grafana/grafana:latest
|
|
container_name: grafana
|
|
ports:
|
|
- "3000:3000"
|
|
environment:
|
|
- GF_SECURITY_ADMIN_USER=admin
|
|
- GF_SECURITY_ADMIN_PASSWORD=admin
|
|
volumes:
|
|
- grafana_data:/var/lib/grafana
|
|
- ./.docker/observability/grafana/provisioning:/etc/grafana/provisioning:ro
|
|
- ./.docker/observability/grafana/dashboards:/var/lib/grafana/dashboards:ro
|
|
networks:
|
|
- rustfs-network
|
|
restart: unless-stopped
|
|
profiles:
|
|
- observability
|
|
|
|
# Redis for caching (optional)
|
|
redis:
|
|
image: redis:7-alpine
|
|
container_name: redis
|
|
ports:
|
|
- "6379:6379"
|
|
volumes:
|
|
- redis_data:/data
|
|
networks:
|
|
- rustfs-network
|
|
restart: unless-stopped
|
|
profiles:
|
|
- cache
|
|
|
|
# NGINX reverse proxy (optional)
|
|
nginx:
|
|
image: nginx:alpine
|
|
container_name: nginx-proxy
|
|
ports:
|
|
- "80:80"
|
|
- "443:443"
|
|
volumes:
|
|
- ./.docker/nginx/nginx.conf:/etc/nginx/nginx.conf:ro
|
|
- ./.docker/nginx/ssl:/etc/nginx/ssl:ro
|
|
networks:
|
|
- rustfs-network
|
|
restart: unless-stopped
|
|
profiles:
|
|
- proxy
|
|
depends_on:
|
|
- rustfs
|
|
|
|
networks:
|
|
rustfs-network:
|
|
driver: bridge
|
|
ipam:
|
|
config:
|
|
- subnet: 172.20.0.0/16
|
|
|
|
volumes:
|
|
rustfs_data_0:
|
|
driver: local
|
|
rustfs_data_1:
|
|
driver: local
|
|
rustfs_data_2:
|
|
driver: local
|
|
rustfs_data_3:
|
|
driver: local
|
|
rustfs_dev_data:
|
|
driver: local
|
|
prometheus_data:
|
|
driver: local
|
|
grafana_data:
|
|
driver: local
|
|
redis_data:
|
|
driver: local
|