Compare commits

..

16 Commits

Author SHA1 Message Date
houseme
20cd117aa6 improve code for dockerfile (#256)
* improve code for dockerfile

* Update Dockerfile

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* improve code for file name

* improve code for dockerfile

* fix

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2025-07-18 15:53:00 +08:00
houseme
fc8931d69f improve code for dockerfile (#253)
* improve code for dockerfile

* Update Dockerfile

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2025-07-18 11:05:00 +08:00
weisd
0167b2decd fix: optimize RPC connection management and prevent race conditions (#252) 2025-07-18 10:41:00 +08:00
weisd
e67980ff3c Fix/range content length (#251)
* fix:getobject range length
2025-07-17 23:25:21 +08:00
weisd
96760bba5a fix:getobject range length (#250) 2025-07-17 23:14:19 +08:00
overtrue
2501d7d241 fix: remove branch restriction from Docker workflow_run trigger
The Docker workflow was not triggering for tag-based releases because it had
'branches: [main]' restriction in the workflow_run configuration. When pushing
tags, the triggering workflow runs on the tag, not on main branch.

Changes:
- Remove 'branches: [main]' from workflow_run trigger
- Simplify tag detection using github.event.workflow_run context instead of API calls
- Use official workflow_run event properties (head_branch, event) for reliable detection
- Support both 'refs/tags/VERSION' and direct 'VERSION' formats
- Add better logging for debugging workflow trigger issues

This fixes the issue where Docker images were not built for tagged releases.
2025-07-17 08:13:34 +08:00
overtrue
55b84262b5 fix: use GitHub API for reliable tag detection in Docker workflow
- Replace git commands with GitHub API calls for tag detection
- Add proper commit checkout for workflow_run events
- Use gh CLI and curl fallback for better reliability
- Add debug output to help troubleshoot tag detection issues

This should fix the issue where Docker builds were not triggered for tagged releases
due to missing tag information in the workflow_run environment.
2025-07-17 08:01:33 +08:00
overtrue
ce4252eb1a fix: correct Docker workflow trigger logic for tag-based releases
BREAKING CHANGE: Fixed Docker workflow that was incorrectly skipping builds for tagged releases
- Fix logic to detect tag pushes using git refs instead of branch names
- Properly identify tag pushes vs branch pushes using git show-ref
- Support both v-prefixed and bare version formats
- Ensure Docker images are built for all tagged releases including prereleases
2025-07-17 07:46:54 +08:00
overtrue
db708917b4 docs: update .docker/README.md to reflect simplified Makefile commands
- Add new Makefile Commands section with simplified docker-dev* commands
- Update Development Workflow to use new dev-env-* commands
- Update directory structure (remove deleted alpine/ directory)
- Reorganize build instructions to prioritize Makefile over direct scripts
- Add Common Development Tasks section with make help commands
2025-07-17 07:30:13 +08:00
overtrue
8ddb45627d refactor: simplify Docker build commands and fix version matching
- Remove obsolete .docker/alpine/Dockerfile.protoc (superseded by Dockerfile.source)
- Simplify Makefile commands by removing backward compatibility aliases
  * Replace docker-buildx-source* with shorter docker-dev* commands
  * Replace start/stop with explicit dev-env-start/dev-env-stop commands
- Fix Docker workflow version matching logic to correctly distinguish:
  * 1.0.0 vs 1.0.0-alpha.11 (prerelease detection)
  * Support both v1.0.0 and 1.0.0 formats (with/without v prefix)
  * Reorder case patterns to match prereleases before releases

BREAKING CHANGE: Removed legacy command aliases
- Use 'make docker-dev-local' instead of 'make docker-buildx-source-local'
- Use 'make dev-env-start' instead of 'make start'
2025-07-17 07:29:00 +08:00
overtrue
550c225b79 wip 2025-07-17 07:07:02 +08:00
overtrue
0d46b550a8 refactor: merge release workflow into build workflow and clean up
- Merge release logic into build.yml to avoid cross-workflow artifact access issues
- Add release jobs (create-release, upload-release-assets, update-latest-version, publish-release) that run only for tag pushes
- Use standard actions/download-artifact@v4 within the same workflow (no cross-workflow limitations)
- Deprecate standalone release.yml workflow with warning job and confirmation requirement
- Remove references to deleted release-notes-template.md file from both workflows
- Update build summary messages to reflect integrated release process

This resolves the 'Prepare release assets' failure by eliminating the need for cross-workflow artifact access.
2025-07-17 07:06:51 +08:00
overtrue
0693cca1a4 fix: resolve workflow_run artifact access issue in release pipeline
- Replace actions/download-artifact@v4 with GitHub API calls to access artifacts from triggering workflow
- Add proper permissions (contents: read, actions: read) to prepare-assets job
- Handle both workflow_run and workflow_dispatch trigger scenarios
- Fix the root cause: workflow_run events cannot access artifacts from triggering workflows using standard download-artifact action

Fixes the 'Prepare release assets' step failure by implementing cross-workflow artifact access through GitHub API.
2025-07-17 06:58:09 +08:00
安正超
0d9f9e381a refactor: use workflow_run trigger for release workflow to eliminate timing issues (#241)
* fix: use correct tag reference in release workflow wait-for-artifacts step

- Change ref from github.ref to needs.release-check.outputs.tag
- Fix issue where wait-on-check-action receives full git reference (refs/tags/1.0.0-alpha.21)
  instead of clean tag name (1.0.0-alpha.21)
- This resolves timeout errors when waiting for build artifacts during release process

Fixes the release workflow failure for tag 1.0.0-alpha.21

* refactor: use workflow_run trigger for release workflow instead of push

- Replace push trigger with workflow_run to eliminate timing issues
- Release workflow now triggers only after Build workflow completes successfully
- Remove wait-for-artifacts step completely (no longer needed)
- Add should_release condition to control release execution
- Support both tag pushes and manual releases via workflow_dispatch
- Align with docker.yml pattern for better reliability

This completely resolves the release workflow timeout issues by ensuring
build artifacts are always available before the release process starts.

Fixes the fundamental timing issue where release.yml and build.yml
were racing against each other when triggered by the same tag push.
2025-07-17 06:48:09 +08:00
安正超
6c7aa5a7ae fix: use correct tag reference in release workflow wait-for-artifacts step (#240)
- Change ref from github.ref to needs.release-check.outputs.tag
- Fix issue where wait-on-check-action receives full git reference (refs/tags/1.0.0-alpha.21)
  instead of clean tag name (1.0.0-alpha.21)
- This resolves timeout errors when waiting for build artifacts during release process

Fixes the release workflow failure for tag 1.0.0-alpha.21
2025-07-17 06:36:57 +08:00
overtrue
a27d935925 wip 2025-07-17 06:31:25 +08:00
18 changed files with 861 additions and 755 deletions

View File

@@ -6,10 +6,10 @@ This directory contains Docker configuration files and supporting infrastructure
```
rustfs/
├── Dockerfile # Production image (Alpine + GitHub Releases)
├── Dockerfile.source # Source build (Debian + cross-compilation)
├── cargo.config.toml # Rust cargo configuration
├── Dockerfile # Production image (Alpine + pre-built binaries)
├── Dockerfile.source # Development image (Debian + source build)
├── docker-buildx.sh # Multi-architecture build script
├── Makefile # Build automation with simplified commands
└── .docker/ # Supporting infrastructure
├── observability/ # Monitoring and observability configs
├── compose/ # Docker Compose configurations
@@ -64,7 +64,11 @@ docker run rustfs/rustfs:main-latest # Main branch latest
### Development Environment
```bash
# Start development container
# Quick setup using Makefile (recommended)
make docker-dev-local # Build development image locally
make dev-env-start # Start development container
# Manual Docker commands
docker run -it -v $(pwd):/workspace -p 9000:9000 rustfs/rustfs:latest-dev
# Build from source locally
@@ -76,9 +80,33 @@ docker-compose up rustfs-dev
## 🏗️ Build Arguments and Scripts
### Using docker-buildx.sh (Recommended)
### Using Makefile Commands (Recommended)
For multi-architecture builds, use the provided script:
The easiest way to build images using simplified commands:
```bash
# Development images (build from source)
make docker-dev-local # Build for local use (single arch)
make docker-dev # Build multi-arch (for CI/CD)
make docker-dev-push REGISTRY=xxx # Build and push to registry
# Production images (using pre-built binaries)
make docker-buildx # Build multi-arch production images
make docker-buildx-push # Build and push production images
make docker-buildx-version VERSION=v1.0.0 # Build specific version
# Development environment
make dev-env-start # Start development container
make dev-env-stop # Stop development container
make dev-env-restart # Restart development container
# Help
make help-docker # Show all Docker-related commands
```
### Using docker-buildx.sh (Advanced)
For direct script usage and advanced scenarios:
```bash
# Build latest version for all architectures
@@ -147,17 +175,51 @@ Architecture is automatically detected during build using Docker's `TARGETARCH`
## 🛠️ Development Workflow
For local development and testing:
### Quick Start with Makefile (Recommended)
```bash
# Quick development setup
docker-compose up rustfs-dev
# 1. Start development environment
make dev-env-start
# Custom source build
docker build -f Dockerfile.source -t rustfs:custom .
# 2. Your development container is now running with:
# - Port 9000 exposed for RustFS
# - Port 9010 exposed for admin console
# - Current directory mounted as /workspace
# 3. Stop when done
make dev-env-stop
```
### Manual Development Setup
```bash
# Build development image from source
make docker-dev-local
# Or use traditional Docker commands
docker build -f Dockerfile.source -t rustfs:dev .
# Run with development tools
docker run -it -v $(pwd):/workspace rustfs:custom bash
docker run -it -v $(pwd):/workspace -p 9000:9000 rustfs:dev bash
# Or use docker-compose for complex setups
docker-compose up rustfs-dev
```
### Common Development Tasks
```bash
# Build and test locally
make build # Build binary natively
make docker-dev-local # Build development Docker image
make test # Run tests
make fmt # Format code
make clippy # Run linter
# Get help
make help # General help
make help-docker # Docker-specific help
make help-build # Build-specific help
```
## 🚀 CI/CD Integration

View File

@@ -1,40 +0,0 @@
FROM alpine:3.18
ENV LANG C.UTF-8
# Install base dependencies
RUN apk add --no-cache \
wget \
git \
curl \
unzip \
gcc \
musl-dev \
pkgconfig \
openssl-dev \
dbus-dev \
wayland-dev \
webkit2gtk-4.1-dev \
build-base \
linux-headers
# install protoc
RUN wget https://github.com/protocolbuffers/protobuf/releases/download/v30.2/protoc-30.2-linux-x86_64.zip \
&& unzip protoc-30.2-linux-x86_64.zip -d protoc3 \
&& mv protoc3/bin/* /usr/local/bin/ && chmod +x /usr/local/bin/protoc \
&& mv protoc3/include/* /usr/local/include/ && rm -rf protoc-30.2-linux-x86_64.zip protoc3
# install flatc
RUN wget https://github.com/google/flatbuffers/releases/download/v24.3.25/Linux.flatc.binary.g++-13.zip \
&& unzip Linux.flatc.binary.g++-13.zip \
&& mv flatc /usr/local/bin/ && chmod +x /usr/local/bin/flatc && rm -rf Linux.flatc.binary.g++-13.zip
# install rust
RUN curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- -y
# Set PATH for rust
ENV PATH="/root/.cargo/bin:${PATH}"
COPY .docker/cargo.config.toml /root/.cargo/config.toml
WORKDIR /root/rustfs

View File

@@ -471,12 +471,12 @@ jobs:
"release")
echo "🚀 Release build artifacts have been uploaded to OSS release directory"
echo "✅ This build is ready for production use"
echo "🏷️ GitHub Release will be created automatically by the release workflow"
echo "🏷️ GitHub Release will be created in this workflow"
;;
"prerelease")
echo "🧪 Prerelease build artifacts have been uploaded to OSS release directory"
echo "⚠️ This is a prerelease build - use with caution"
echo "🏷️ GitHub Release will be created automatically by the release workflow"
echo "🏷️ GitHub Release will be created in this workflow"
;;
esac
@@ -489,3 +489,264 @@ jobs:
else
echo "❌ Docker image build will be skipped due to build failure"
fi
# Create GitHub Release (only for tag pushes)
create-release:
name: Create GitHub Release
needs: [build-check, build-rustfs]
if: startsWith(github.ref, 'refs/tags/') && needs.build-check.outputs.build_type != 'development'
runs-on: ubuntu-latest
permissions:
contents: write
outputs:
release_id: ${{ steps.create.outputs.release_id }}
release_url: ${{ steps.create.outputs.release_url }}
steps:
- name: Checkout repository
uses: actions/checkout@v4
with:
fetch-depth: 0
- name: Create GitHub Release
id: create
env:
GH_TOKEN: ${{ github.token }}
run: |
TAG="${{ needs.build-check.outputs.version }}"
VERSION="${{ needs.build-check.outputs.version }}"
IS_PRERELEASE="${{ needs.build-check.outputs.is_prerelease }}"
BUILD_TYPE="${{ needs.build-check.outputs.build_type }}"
# Determine release type for title
if [[ "$BUILD_TYPE" == "prerelease" ]]; then
if [[ "$TAG" == *"alpha"* ]]; then
RELEASE_TYPE="alpha"
elif [[ "$TAG" == *"beta"* ]]; then
RELEASE_TYPE="beta"
elif [[ "$TAG" == *"rc"* ]]; then
RELEASE_TYPE="rc"
else
RELEASE_TYPE="prerelease"
fi
else
RELEASE_TYPE="release"
fi
# Check if release already exists
if gh release view "$TAG" >/dev/null 2>&1; then
echo "Release $TAG already exists"
RELEASE_ID=$(gh release view "$TAG" --json databaseId --jq '.databaseId')
RELEASE_URL=$(gh release view "$TAG" --json url --jq '.url')
else
# Get release notes from tag message
RELEASE_NOTES=$(git tag -l --format='%(contents)' "${TAG}")
if [[ -z "$RELEASE_NOTES" || "$RELEASE_NOTES" =~ ^[[:space:]]*$ ]]; then
if [[ "$IS_PRERELEASE" == "true" ]]; then
RELEASE_NOTES="Pre-release ${VERSION} (${RELEASE_TYPE})"
else
RELEASE_NOTES="Release ${VERSION}"
fi
fi
# Create release title
if [[ "$IS_PRERELEASE" == "true" ]]; then
TITLE="RustFS $VERSION (${RELEASE_TYPE})"
else
TITLE="RustFS $VERSION"
fi
# Create the release
PRERELEASE_FLAG=""
if [[ "$IS_PRERELEASE" == "true" ]]; then
PRERELEASE_FLAG="--prerelease"
fi
gh release create "$TAG" \
--title "$TITLE" \
--notes "$RELEASE_NOTES" \
$PRERELEASE_FLAG \
--draft
RELEASE_ID=$(gh release view "$TAG" --json databaseId --jq '.databaseId')
RELEASE_URL=$(gh release view "$TAG" --json url --jq '.url')
fi
echo "release_id=$RELEASE_ID" >> $GITHUB_OUTPUT
echo "release_url=$RELEASE_URL" >> $GITHUB_OUTPUT
echo "Created release: $RELEASE_URL"
# Prepare and upload release assets
upload-release-assets:
name: Upload Release Assets
needs: [build-check, build-rustfs, create-release]
if: startsWith(github.ref, 'refs/tags/') && needs.build-check.outputs.build_type != 'development'
runs-on: ubuntu-latest
permissions:
contents: write
actions: read
steps:
- name: Checkout repository
uses: actions/checkout@v4
- name: Download all build artifacts
uses: actions/download-artifact@v4
with:
path: ./artifacts
pattern: rustfs-*
merge-multiple: true
- name: Prepare release assets
id: prepare
run: |
VERSION="${{ needs.build-check.outputs.version }}"
TAG="${{ needs.build-check.outputs.version }}"
mkdir -p ./release-assets
# Copy and verify artifacts
ASSETS_COUNT=0
for file in ./artifacts/*.zip; do
if [[ -f "$file" ]]; then
cp "$file" ./release-assets/
ASSETS_COUNT=$((ASSETS_COUNT + 1))
fi
done
if [[ $ASSETS_COUNT -eq 0 ]]; then
echo "❌ No artifacts found!"
exit 1
fi
cd ./release-assets
# Generate checksums
if ls *.zip >/dev/null 2>&1; then
sha256sum *.zip > SHA256SUMS
sha512sum *.zip > SHA512SUMS
fi
# Create signature placeholder files
for file in *.zip; do
echo "# Signature for $file" > "${file}.asc"
echo "# GPG signature will be added in future versions" >> "${file}.asc"
done
echo "📦 Prepared assets:"
ls -la
echo "🔢 Asset count: $ASSETS_COUNT"
- name: Upload to GitHub Release
env:
GH_TOKEN: ${{ github.token }}
run: |
TAG="${{ needs.build-check.outputs.version }}"
cd ./release-assets
# Upload all files
for file in *; do
if [[ -f "$file" ]]; then
echo "📤 Uploading $file..."
gh release upload "$TAG" "$file" --clobber
fi
done
echo "✅ All assets uploaded successfully"
# Update latest.json for stable releases only
update-latest-version:
name: Update Latest Version
needs: [build-check, upload-release-assets]
if: startsWith(github.ref, 'refs/tags/') && needs.build-check.outputs.is_prerelease == 'false'
runs-on: ubuntu-latest
steps:
- name: Update latest.json
env:
OSS_ACCESS_KEY_ID: ${{ secrets.ALICLOUDOSS_KEY_ID }}
OSS_ACCESS_KEY_SECRET: ${{ secrets.ALICLOUDOSS_KEY_SECRET }}
run: |
if [[ -z "$OSS_ACCESS_KEY_ID" ]]; then
echo "⚠️ OSS credentials not available, skipping latest.json update"
exit 0
fi
VERSION="${{ needs.build-check.outputs.version }}"
TAG="${{ needs.build-check.outputs.version }}"
# Install ossutil
OSSUTIL_VERSION="2.1.1"
OSSUTIL_ZIP="ossutil-${OSSUTIL_VERSION}-linux-amd64.zip"
OSSUTIL_DIR="ossutil-${OSSUTIL_VERSION}-linux-amd64"
curl -o "$OSSUTIL_ZIP" "https://gosspublic.alicdn.com/ossutil/v2/${OSSUTIL_VERSION}/${OSSUTIL_ZIP}"
unzip "$OSSUTIL_ZIP"
chmod +x "${OSSUTIL_DIR}/ossutil"
# Create latest.json
cat > latest.json << EOF
{
"version": "${VERSION}",
"tag": "${TAG}",
"release_date": "$(date -u +%Y-%m-%dT%H:%M:%SZ)",
"release_type": "stable",
"download_url": "https://github.com/${{ github.repository }}/releases/tag/${TAG}"
}
EOF
# Upload to OSS
./${OSSUTIL_DIR}/ossutil cp latest.json oss://rustfs-version/latest.json --force
echo "✅ Updated latest.json for stable release $VERSION"
# Publish release (remove draft status)
publish-release:
name: Publish Release
needs: [build-check, create-release, upload-release-assets]
if: startsWith(github.ref, 'refs/tags/') && needs.build-check.outputs.build_type != 'development'
runs-on: ubuntu-latest
permissions:
contents: write
steps:
- name: Checkout repository
uses: actions/checkout@v4
- name: Update release notes and publish
env:
GH_TOKEN: ${{ github.token }}
run: |
TAG="${{ needs.build-check.outputs.version }}"
VERSION="${{ needs.build-check.outputs.version }}"
IS_PRERELEASE="${{ needs.build-check.outputs.is_prerelease }}"
BUILD_TYPE="${{ needs.build-check.outputs.build_type }}"
# Determine release type
if [[ "$BUILD_TYPE" == "prerelease" ]]; then
if [[ "$TAG" == *"alpha"* ]]; then
RELEASE_TYPE="alpha"
elif [[ "$TAG" == *"beta"* ]]; then
RELEASE_TYPE="beta"
elif [[ "$TAG" == *"rc"* ]]; then
RELEASE_TYPE="rc"
else
RELEASE_TYPE="prerelease"
fi
else
RELEASE_TYPE="release"
fi
# Get original release notes from tag
ORIGINAL_NOTES=$(git tag -l --format='%(contents)' "${TAG}")
if [[ -z "$ORIGINAL_NOTES" || "$ORIGINAL_NOTES" =~ ^[[:space:]]*$ ]]; then
if [[ "$IS_PRERELEASE" == "true" ]]; then
ORIGINAL_NOTES="Pre-release ${VERSION} (${RELEASE_TYPE})"
else
ORIGINAL_NOTES="Release ${VERSION}"
fi
fi
# Publish the release (remove draft status)
gh release edit "$TAG" --draft=false
echo "🎉 Released $TAG successfully!"
echo "📄 Release URL: ${{ needs.create-release.outputs.release_url }}"

View File

@@ -38,7 +38,6 @@ on:
workflow_run:
workflows: ["Build and Release"]
types: [completed]
branches: [main]
# Manual trigger with same parameters for consistency
workflow_dispatch:
inputs:
@@ -83,6 +82,8 @@ jobs:
uses: actions/checkout@v4
with:
fetch-depth: 0
# For workflow_run events, checkout the specific commit that triggered the workflow
ref: ${{ github.event.workflow_run.head_sha || github.sha }}
- name: Check build conditions
id: check
@@ -114,32 +115,62 @@ jobs:
# Use Git to generate consistent short SHA (ensures uniqueness like build.yml)
short_sha=$(git rev-parse --short "${{ github.event.workflow_run.head_sha }}")
# Determine build type based on branch and commit
if [[ "${{ github.event.workflow_run.head_branch }}" == "main" ]]; then
build_type="development"
version="dev-${short_sha}"
# Skip Docker build for development builds
should_build=false
echo "⏭️ Skipping Docker build for development version (main branch)"
elif [[ "${{ github.event.workflow_run.event }}" == "push" ]] && [[ "${{ github.event.workflow_run.head_branch }}" =~ ^refs/tags/ ]]; then
# Tag push - only build for releases and prereleases
tag_name="${{ github.event.workflow_run.head_branch }}"
version="${tag_name#refs/tags/}"
if [[ "$version" == *"alpha"* ]] || [[ "$version" == *"beta"* ]] || [[ "$version" == *"rc"* ]]; then
build_type="prerelease"
is_prerelease=true
echo "🧪 Building Docker image for prerelease: $version"
# Determine build type based on triggering workflow event and ref
triggering_event="${{ github.event.workflow_run.event }}"
head_branch="${{ github.event.workflow_run.head_branch }}"
echo "🔍 Analyzing triggering workflow:"
echo " 📋 Event: $triggering_event"
echo " 🌿 Head branch: $head_branch"
echo " 📎 Head SHA: ${{ github.event.workflow_run.head_sha }}"
# Check if this was triggered by a tag push
if [[ "$triggering_event" == "push" ]]; then
# For tag pushes, head_branch will be like "refs/tags/v1.0.0" or just "v1.0.0"
if [[ "$head_branch" == refs/tags/* ]]; then
# Extract tag name from refs/tags/TAG_NAME
tag_name="${head_branch#refs/tags/}"
version="$tag_name"
elif [[ "$head_branch" =~ ^v?[0-9]+\.[0-9]+\.[0-9]+ ]]; then
# Direct tag name like "v1.0.0" or "1.0.0-alpha.1"
version="$head_branch"
elif [[ "$head_branch" == "main" ]]; then
# Regular branch push to main
build_type="development"
version="dev-${short_sha}"
should_build=false
echo "⏭️ Skipping Docker build for development version (main branch push)"
else
build_type="release"
create_latest=true
echo "🚀 Building Docker image for release: $version"
# Other branch push
build_type="development"
version="dev-${short_sha}"
should_build=false
echo "⏭️ Skipping Docker build for development version (branch: $head_branch)"
fi
# If we extracted a version (tag), determine release type
if [[ -n "$version" ]] && [[ "$version" != "dev-${short_sha}" ]]; then
# Remove 'v' prefix if present for consistent version format
if [[ "$version" == v* ]]; then
version="${version#v}"
fi
if [[ "$version" == *"alpha"* ]] || [[ "$version" == *"beta"* ]] || [[ "$version" == *"rc"* ]]; then
build_type="prerelease"
is_prerelease=true
echo "🧪 Building Docker image for prerelease: $version"
else
build_type="release"
create_latest=true
echo "🚀 Building Docker image for release: $version"
fi
fi
else
# Non-push events
build_type="development"
version="dev-${short_sha}"
# Skip Docker build for development builds
should_build=false
echo "⏭️ Skipping Docker build for development version"
echo "⏭️ Skipping Docker build for development version (event: $triggering_event)"
fi
echo "🔄 Build triggered by workflow_run:"
@@ -169,21 +200,23 @@ jobs:
create_latest=true
echo "🚀 Building with latest stable release version"
;;
v[0-9]*)
build_type="release"
create_latest=true
echo "📦 Building with specific release version: $input_version"
;;
v*alpha*|v*beta*|v*rc*)
# Prerelease versions (must match first, more specific)
v*alpha*|v*beta*|v*rc*|*alpha*|*beta*|*rc*)
build_type="prerelease"
is_prerelease=true
echo "🧪 Building with prerelease version: $input_version"
;;
# Release versions (match after prereleases, more general)
v[0-9]*|[0-9]*.*.*)
build_type="release"
create_latest=true
echo "📦 Building with specific release version: $input_version"
;;
*)
# Invalid version for Docker build
should_build=false
echo "❌ Invalid version for Docker build: $input_version"
echo "⚠️ Only release versions (latest, v1.0.0) and prereleases (v1.0.0-alpha1) are supported"
echo "⚠️ Only release versions (latest, v1.0.0, 1.0.0) and prereleases (v1.0.0-alpha1, 1.0.0-beta2) are supported"
;;
esac
fi

View File

@@ -1,78 +0,0 @@
## RustFS ${VERSION_CLEAN}
${ORIGINAL_NOTES}
---
### 🚀 Quick Download
**Linux (Static Binaries - No Dependencies):**
```bash
# x86_64 (Intel/AMD)
curl -LO https://github.com/rustfs/rustfs/releases/download/${VERSION}/rustfs-x86_64-unknown-linux-musl.zip
unzip rustfs-x86_64-unknown-linux-musl.zip
sudo mv rustfs /usr/local/bin/
# ARM64 (Graviton, Apple Silicon VMs)
curl -LO https://github.com/rustfs/rustfs/releases/download/${VERSION}/rustfs-aarch64-unknown-linux-musl.zip
unzip rustfs-aarch64-unknown-linux-musl.zip
sudo mv rustfs /usr/local/bin/
```
**macOS:**
```bash
# Apple Silicon (M1/M2/M3)
curl -LO https://github.com/rustfs/rustfs/releases/download/${VERSION}/rustfs-aarch64-apple-darwin.zip
unzip rustfs-aarch64-apple-darwin.zip
sudo mv rustfs /usr/local/bin/
# Intel
curl -LO https://github.com/rustfs/rustfs/releases/download/${VERSION}/rustfs-x86_64-apple-darwin.zip
unzip rustfs-x86_64-apple-darwin.zip
sudo mv rustfs /usr/local/bin/
```
### 📁 Available Downloads
| Platform | Architecture | File | Description |
|----------|-------------|------|-------------|
| Linux | x86_64 | `rustfs-x86_64-unknown-linux-musl.zip` | Static binary, no dependencies |
| Linux | ARM64 | `rustfs-aarch64-unknown-linux-musl.zip` | Static binary, no dependencies |
| macOS | Apple Silicon | `rustfs-aarch64-apple-darwin.zip` | Native binary, ZIP archive |
| macOS | Intel | `rustfs-x86_64-apple-darwin.zip` | Native binary, ZIP archive |
### 🔐 Verification
Download checksums and verify your download:
```bash
# Download checksums
curl -LO https://github.com/rustfs/rustfs/releases/download/${VERSION}/SHA256SUMS
# Verify (Linux)
sha256sum -c SHA256SUMS --ignore-missing
# Verify (macOS)
shasum -a 256 -c SHA256SUMS --ignore-missing
```
### 🛠️ System Requirements
- **Linux**: Any distribution with glibc 2.17+ (CentOS 7+, Ubuntu 16.04+)
- **macOS**: 10.15+ (Catalina or later)
- **Windows**: Windows 10 version 1809 or later
### 📚 Documentation
- [Installation Guide](https://github.com/rustfs/rustfs#installation)
- [Quick Start](https://github.com/rustfs/rustfs#quick-start)
- [Configuration](https://github.com/rustfs/rustfs/blob/main/docs/)
- [API Documentation](https://docs.rs/rustfs)
### 🆘 Support
- 🐛 [Report Issues](https://github.com/rustfs/rustfs/issues)
- 💬 [Community Discussions](https://github.com/rustfs/rustfs/discussions)
- 📖 [Documentation](https://github.com/rustfs/rustfs/tree/main/docs)

View File

@@ -1,353 +0,0 @@
# Copyright 2024 RustFS Team
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
name: Release
on:
push:
tags: ["*.*.*"]
workflow_dispatch:
inputs:
tag:
description: "Tag to create release for"
required: true
type: string
env:
CARGO_TERM_COLOR: always
jobs:
# Determine release type
release-check:
name: Release Type Check
runs-on: ubuntu-latest
outputs:
tag: ${{ steps.check.outputs.tag }}
version: ${{ steps.check.outputs.version }}
is_prerelease: ${{ steps.check.outputs.is_prerelease }}
release_type: ${{ steps.check.outputs.release_type }}
steps:
- name: Determine release type
id: check
run: |
if [[ "${{ github.event_name }}" == "workflow_dispatch" ]]; then
TAG="${{ github.event.inputs.tag }}"
else
TAG="${GITHUB_REF#refs/tags/}"
fi
VERSION="${TAG}"
# Check if this is a prerelease
IS_PRERELEASE=false
RELEASE_TYPE="release"
if [[ "$TAG" == *"alpha"* ]] || [[ "$TAG" == *"beta"* ]] || [[ "$TAG" == *"rc"* ]]; then
IS_PRERELEASE=true
if [[ "$TAG" == *"alpha"* ]]; then
RELEASE_TYPE="alpha"
elif [[ "$TAG" == *"beta"* ]]; then
RELEASE_TYPE="beta"
elif [[ "$TAG" == *"rc"* ]]; then
RELEASE_TYPE="rc"
fi
fi
echo "tag=$TAG" >> $GITHUB_OUTPUT
echo "version=$VERSION" >> $GITHUB_OUTPUT
echo "is_prerelease=$IS_PRERELEASE" >> $GITHUB_OUTPUT
echo "release_type=$RELEASE_TYPE" >> $GITHUB_OUTPUT
echo "📦 Release Type: $RELEASE_TYPE"
echo "🏷️ Tag: $TAG"
echo "🔢 Version: $VERSION"
echo "🚀 Is Prerelease: $IS_PRERELEASE"
# Create GitHub Release
create-release:
name: Create GitHub Release
needs: release-check
runs-on: ubuntu-latest
permissions:
contents: write
outputs:
release_id: ${{ steps.create.outputs.release_id }}
release_url: ${{ steps.create.outputs.release_url }}
steps:
- name: Checkout repository
uses: actions/checkout@v4
with:
fetch-depth: 0
- name: Create GitHub Release
id: create
env:
GH_TOKEN: ${{ github.token }}
run: |
TAG="${{ needs.release-check.outputs.tag }}"
VERSION="${{ needs.release-check.outputs.version }}"
IS_PRERELEASE="${{ needs.release-check.outputs.is_prerelease }}"
RELEASE_TYPE="${{ needs.release-check.outputs.release_type }}"
# Check if release already exists
if gh release view "$TAG" >/dev/null 2>&1; then
echo "Release $TAG already exists"
RELEASE_ID=$(gh release view "$TAG" --json databaseId --jq '.databaseId')
RELEASE_URL=$(gh release view "$TAG" --json url --jq '.url')
else
# Get release notes from tag message
RELEASE_NOTES=$(git tag -l --format='%(contents)' "${TAG}")
if [[ -z "$RELEASE_NOTES" || "$RELEASE_NOTES" =~ ^[[:space:]]*$ ]]; then
if [[ "$IS_PRERELEASE" == "true" ]]; then
RELEASE_NOTES="Pre-release ${VERSION} (${RELEASE_TYPE})"
else
RELEASE_NOTES="Release ${VERSION}"
fi
fi
# Create release title
if [[ "$IS_PRERELEASE" == "true" ]]; then
TITLE="RustFS $VERSION (${RELEASE_TYPE})"
else
TITLE="RustFS $VERSION"
fi
# Create the release
PRERELEASE_FLAG=""
if [[ "$IS_PRERELEASE" == "true" ]]; then
PRERELEASE_FLAG="--prerelease"
fi
gh release create "$TAG" \
--title "$TITLE" \
--notes "$RELEASE_NOTES" \
$PRERELEASE_FLAG \
--draft
RELEASE_ID=$(gh release view "$TAG" --json databaseId --jq '.databaseId')
RELEASE_URL=$(gh release view "$TAG" --json url --jq '.url')
fi
echo "release_id=$RELEASE_ID" >> $GITHUB_OUTPUT
echo "release_url=$RELEASE_URL" >> $GITHUB_OUTPUT
echo "Created release: $RELEASE_URL"
# Wait for build artifacts from build.yml
wait-for-artifacts:
name: Wait for Build Artifacts
needs: release-check
runs-on: ubuntu-latest
steps:
- name: Wait for build workflow
uses: lewagon/wait-on-check-action@v1.3.1
with:
ref: ${{ needs.release-check.outputs.tag }}
check-name: "Build RustFS"
repo-token: ${{ secrets.GITHUB_TOKEN }}
wait-interval: 30
allowed-conclusions: success
# Download and prepare release assets
prepare-assets:
name: Prepare Release Assets
needs: [release-check, wait-for-artifacts]
runs-on: ubuntu-latest
outputs:
assets_prepared: ${{ steps.prepare.outputs.assets_prepared }}
steps:
- name: Checkout repository
uses: actions/checkout@v4
- name: Download artifacts from build workflow
uses: actions/download-artifact@v4
with:
path: ./artifacts
pattern: rustfs-*
merge-multiple: true
- name: Prepare release assets
id: prepare
run: |
VERSION="${{ needs.release-check.outputs.version }}"
TAG="${{ needs.release-check.outputs.tag }}"
mkdir -p ./release-assets
# Copy and verify artifacts
ASSETS_COUNT=0
for file in ./artifacts/rustfs-*.zip; do
if [[ -f "$file" ]]; then
cp "$file" ./release-assets/
ASSETS_COUNT=$((ASSETS_COUNT + 1))
fi
done
if [[ $ASSETS_COUNT -eq 0 ]]; then
echo "❌ No artifacts found!"
exit 1
fi
cd ./release-assets
# Generate checksums
if ls *.zip >/dev/null 2>&1; then
sha256sum *.zip > SHA256SUMS
sha512sum *.zip > SHA512SUMS
fi
# TODO: Add GPG signing for signatures
# For now, create placeholder signature files
for file in *.zip; do
echo "# Signature for $file" > "${file}.asc"
echo "# GPG signature will be added in future versions" >> "${file}.asc"
done
echo "assets_prepared=true" >> $GITHUB_OUTPUT
echo "📦 Prepared assets:"
ls -la
echo "🔢 Asset count: $ASSETS_COUNT"
- name: Upload prepared assets
uses: actions/upload-artifact@v4
with:
name: release-assets-${{ needs.release-check.outputs.tag }}
path: ./release-assets/
retention-days: 30
# Upload assets to GitHub Release
upload-assets:
name: Upload Release Assets
needs: [release-check, create-release, prepare-assets]
runs-on: ubuntu-latest
permissions:
contents: write
steps:
- name: Download prepared assets
uses: actions/download-artifact@v4
with:
name: release-assets-${{ needs.release-check.outputs.tag }}
path: ./release-assets
- name: Upload to GitHub Release
env:
GH_TOKEN: ${{ github.token }}
run: |
TAG="${{ needs.release-check.outputs.tag }}"
cd ./release-assets
# Upload all files
for file in *; do
if [[ -f "$file" ]]; then
echo "📤 Uploading $file..."
gh release upload "$TAG" "$file" --clobber
fi
done
echo "✅ All assets uploaded successfully"
# Update latest.json for stable releases only
update-latest:
name: Update Latest Version
needs: [release-check, upload-assets]
if: needs.release-check.outputs.is_prerelease == 'false'
runs-on: ubuntu-latest
steps:
- name: Update latest.json
env:
OSS_ACCESS_KEY_ID: ${{ secrets.ALICLOUDOSS_KEY_ID }}
OSS_ACCESS_KEY_SECRET: ${{ secrets.ALICLOUDOSS_KEY_SECRET }}
run: |
if [[ -z "$OSS_ACCESS_KEY_ID" ]]; then
echo "⚠️ OSS credentials not available, skipping latest.json update"
exit 0
fi
VERSION="${{ needs.release-check.outputs.version }}"
TAG="${{ needs.release-check.outputs.tag }}"
# Install ossutil
OSSUTIL_VERSION="2.1.1"
OSSUTIL_ZIP="ossutil-${OSSUTIL_VERSION}-linux-amd64.zip"
OSSUTIL_DIR="ossutil-${OSSUTIL_VERSION}-linux-amd64"
curl -o "$OSSUTIL_ZIP" "https://gosspublic.alicdn.com/ossutil/v2/${OSSUTIL_VERSION}/${OSSUTIL_ZIP}"
unzip "$OSSUTIL_ZIP"
chmod +x "${OSSUTIL_DIR}/ossutil"
# Create latest.json
cat > latest.json << EOF
{
"version": "${VERSION}",
"tag": "${TAG}",
"release_date": "$(date -u +%Y-%m-%dT%H:%M:%SZ)",
"release_type": "stable",
"download_url": "https://github.com/${{ github.repository }}/releases/tag/${TAG}"
}
EOF
# Upload to OSS
./${OSSUTIL_DIR}/ossutil cp latest.json oss://rustfs-version/latest.json --force
echo "✅ Updated latest.json for stable release $VERSION"
# Publish release (remove draft status)
publish-release:
name: Publish Release
needs: [release-check, create-release, upload-assets]
runs-on: ubuntu-latest
permissions:
contents: write
steps:
- name: Checkout repository
uses: actions/checkout@v4
- name: Update release notes and publish
env:
GH_TOKEN: ${{ github.token }}
run: |
TAG="${{ needs.release-check.outputs.tag }}"
VERSION="${{ needs.release-check.outputs.version }}"
IS_PRERELEASE="${{ needs.release-check.outputs.is_prerelease }}"
RELEASE_TYPE="${{ needs.release-check.outputs.release_type }}"
# Get original release notes from tag
ORIGINAL_NOTES=$(git tag -l --format='%(contents)' "${TAG}")
if [[ -z "$ORIGINAL_NOTES" || "$ORIGINAL_NOTES" =~ ^[[:space:]]*$ ]]; then
if [[ "$IS_PRERELEASE" == "true" ]]; then
ORIGINAL_NOTES="Pre-release ${VERSION} (${RELEASE_TYPE})"
else
ORIGINAL_NOTES="Release ${VERSION}"
fi
fi
# Use release notes template if available
if [[ -f ".github/workflows/release-notes-template.md" ]]; then
# Substitute variables in template
sed -e "s/\${VERSION}/$TAG/g" \
-e "s/\${VERSION_CLEAN}/$VERSION/g" \
-e "s/\${ORIGINAL_NOTES}/$(echo "$ORIGINAL_NOTES" | sed 's/[[\.*^$()+?{|]/\\&/g')/g" \
.github/workflows/release-notes-template.md > enhanced_notes.md
# Update release notes
gh release edit "$TAG" --notes-file enhanced_notes.md
fi
# Publish the release (remove draft status)
gh release edit "$TAG" --draft=false
echo "🎉 Released $TAG successfully!"
echo "📄 Release URL: ${{ needs.create-release.outputs.release_url }}"

247
Cargo.lock generated
View File

@@ -455,7 +455,10 @@ dependencies = [
"serde_repr",
"tokio",
"url",
"zbus 5.8.0",
"wayland-backend",
"wayland-client",
"wayland-protocols",
"zbus 5.9.0",
]
[[package]]
@@ -512,7 +515,7 @@ dependencies = [
"futures-lite",
"parking",
"polling",
"rustix 1.0.7",
"rustix 1.0.8",
"slab",
"tracing",
"windows-sys 0.59.0",
@@ -544,7 +547,7 @@ dependencies = [
"cfg-if",
"event-listener",
"futures-lite",
"rustix 1.0.7",
"rustix 1.0.8",
"tracing",
]
@@ -571,7 +574,7 @@ dependencies = [
"cfg-if",
"futures-core",
"futures-io",
"rustix 1.0.7",
"rustix 1.0.8",
"signal-hook-registry",
"slab",
"windows-sys 0.59.0",
@@ -673,9 +676,9 @@ checksum = "c08606f8c3cbf4ce6ec8e28fb0014a2c086708fe954eaa885384a6165172e7e8"
[[package]]
name = "aws-credential-types"
version = "1.2.3"
version = "1.2.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "687bc16bc431a8533fe0097c7f0182874767f920989d7260950172ae8e3c4465"
checksum = "b68c2194a190e1efc999612792e25b1ab3abfefe4306494efaaabc25933c0cbe"
dependencies = [
"aws-smithy-async",
"aws-smithy-runtime-api",
@@ -708,9 +711,9 @@ dependencies = [
[[package]]
name = "aws-runtime"
version = "1.5.8"
version = "1.5.9"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "4f6c68419d8ba16d9a7463671593c54f81ba58cab466e9b759418da606dcc2e2"
checksum = "b2090e664216c78e766b6bac10fe74d2f451c02441d43484cd76ac9a295075f7"
dependencies = [
"aws-credential-types",
"aws-sigv4",
@@ -733,9 +736,9 @@ dependencies = [
[[package]]
name = "aws-sdk-s3"
version = "1.96.0"
version = "1.98.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "6e25d24de44b34dcdd5182ac4e4c6f07bcec2661c505acef94c0d293b65505fe"
checksum = "029e89cae7e628553643aecb3a3f054a0a0912ff0fd1f5d6a0b4fda421dce64b"
dependencies = [
"aws-credential-types",
"aws-runtime",
@@ -806,9 +809,9 @@ dependencies = [
[[package]]
name = "aws-smithy-checksums"
version = "0.63.4"
version = "0.63.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "244f00666380d35c1c76b90f7b88a11935d11b84076ac22a4c014ea0939627af"
checksum = "5ab9472f7a8ec259ddb5681d2ef1cb1cf16c0411890063e67cdc7b62562cc496"
dependencies = [
"aws-smithy-http",
"aws-smithy-types",
@@ -826,9 +829,9 @@ dependencies = [
[[package]]
name = "aws-smithy-eventstream"
version = "0.60.9"
version = "0.60.10"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "338a3642c399c0a5d157648426110e199ca7fd1c689cc395676b81aa563700c4"
checksum = "604c7aec361252b8f1c871a7641d5e0ba3a7f5a586e51b66bc9510a5519594d9"
dependencies = [
"aws-smithy-types",
"bytes",
@@ -837,9 +840,9 @@ dependencies = [
[[package]]
name = "aws-smithy-http"
version = "0.62.1"
version = "0.62.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "99335bec6cdc50a346fda1437f9fefe33abf8c99060739a546a16457f2862ca9"
checksum = "43c82ba4cab184ea61f6edaafc1072aad3c2a17dcf4c0fce19ac5694b90d8b5f"
dependencies = [
"aws-smithy-eventstream",
"aws-smithy-runtime-api",
@@ -865,7 +868,7 @@ dependencies = [
"aws-smithy-async",
"aws-smithy-runtime-api",
"aws-smithy-types",
"h2 0.3.26",
"h2 0.3.27",
"h2 0.4.11",
"http 0.2.12",
"http 1.3.1",
@@ -1574,23 +1577,12 @@ dependencies = [
[[package]]
name = "chrono-tz"
version = "0.10.3"
version = "0.10.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "efdce149c370f133a071ca8ef6ea340b7b88748ab0810097a9e2976eaa34b4f3"
checksum = "a6139a8597ed92cf816dfb33f5dd6cf0bb93a6adc938f11039f371bc5bcd26c3"
dependencies = [
"chrono",
"chrono-tz-build",
"phf 0.11.3",
]
[[package]]
name = "chrono-tz-build"
version = "0.4.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "8f10f8c9340e31fc120ff885fcdb54a0b48e474bbd77cab557f0c30a3e569402"
dependencies = [
"parse-zoneinfo",
"phf_codegen 0.11.3",
"phf 0.12.1",
]
[[package]]
@@ -1850,18 +1842,18 @@ dependencies = [
[[package]]
name = "const-str"
version = "0.6.2"
version = "0.6.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9e991226a70654b49d34de5ed064885f0bef0348a8e70018b8ff1ac80aa984a2"
checksum = "041fbfcf8e7054df725fb9985297e92422cdc80fcf313665f5ca3d761bb63f4c"
dependencies = [
"const-str-proc-macro",
]
[[package]]
name = "const-str-proc-macro"
version = "0.6.2"
version = "0.6.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f0d1c4c3cb85e5856b34e829af0035d7154f8c2889b15bbf43c8a6c6786dcab5"
checksum = "f801882b7ecd4188f4bca0317f34e022d623590d85893d7024b18d14f2a3b40b"
dependencies = [
"proc-macro2",
"quote",
@@ -2031,9 +2023,9 @@ dependencies = [
[[package]]
name = "crc32fast"
version = "1.4.2"
version = "1.5.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "a97769d94ddab943e4510d138150169a2758b5ef3eb191a9ee688de3e23ef7b3"
checksum = "9481c1c90cbf2ac953f07c8d4a58aa3945c425b7185c9154d67a65e4230da511"
dependencies = [
"cfg-if",
]
@@ -3384,18 +3376,6 @@ version = "0.2.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "bd0c93bb4b0c6d9b77f4435b0ae98c24d17f1c45b2ff844c6151a07256ca923b"
[[package]]
name = "dispatch2"
version = "0.2.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "1a0d569e003ff27784e0e14e4a594048698e0c0f0b66cabcb51511be55a7caa0"
dependencies = [
"bitflags 2.9.1",
"block2 0.6.1",
"libc",
"objc2 0.6.1",
]
[[package]]
name = "dispatch2"
version = "0.3.0"
@@ -3403,6 +3383,8 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "89a09f22a6c6069a18470eb92d2298acf25463f14256d24778e1230d789a2aec"
dependencies = [
"bitflags 2.9.1",
"block2 0.6.1",
"libc",
"objc2 0.6.1",
]
@@ -3417,6 +3399,15 @@ dependencies = [
"syn 2.0.104",
]
[[package]]
name = "dlib"
version = "0.5.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "330c60081dcc4c72131f8eb70510f1ac07223e5d4163db481a04a0befcffa412"
dependencies = [
"libloading 0.8.8",
]
[[package]]
name = "dlopen2"
version = "0.7.0"
@@ -3440,6 +3431,12 @@ dependencies = [
"syn 2.0.104",
]
[[package]]
name = "downcast-rs"
version = "1.2.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "75b325c5dbd37f80359721ad39aca5a29fb04c89279657cffdda8736d0c0b9d2"
[[package]]
name = "dpi"
version = "0.1.2"
@@ -4375,9 +4372,9 @@ dependencies = [
[[package]]
name = "h2"
version = "0.3.26"
version = "0.3.27"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "81fe527a889e1532da5c525686d96d4c2e74cdd345badf8dfef9f6b39dd5f5e8"
checksum = "0beca50380b1fc32983fc1cb4587bfa4bb9e78fc259aad4a0032d2080309222d"
dependencies = [
"bytes",
"fnv",
@@ -4623,7 +4620,7 @@ dependencies = [
"futures-channel",
"futures-core",
"futures-util",
"h2 0.3.26",
"h2 0.3.27",
"http 0.2.12",
"http-body 0.4.6",
"httparse",
@@ -6212,7 +6209,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "1c10c2894a6fed806ade6027bcd50662746363a9589d3ec9d9bef30a4e4bc166"
dependencies = [
"bitflags 2.9.1",
"dispatch2 0.3.0",
"dispatch2",
"objc2 0.6.1",
]
@@ -6611,15 +6608,6 @@ dependencies = [
"zstd",
]
[[package]]
name = "parse-zoneinfo"
version = "0.3.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "1f2a05b18d44e2957b88f96ba460715e295bc1d7510468a2f3d3b44535d26c24"
dependencies = [
"regex",
]
[[package]]
name = "password-hash"
version = "0.5.0"
@@ -6728,11 +6716,11 @@ dependencies = [
[[package]]
name = "phf"
version = "0.11.3"
version = "0.12.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "1fd6780a80ae0c52cc120a26a1a42c1ae51b247a253e4e06113d23d2c2edd078"
checksum = "913273894cec178f401a31ec4b656318d95473527be05c0752cc41cdc32be8b7"
dependencies = [
"phf_shared 0.11.3",
"phf_shared 0.12.1",
]
[[package]]
@@ -6755,16 +6743,6 @@ dependencies = [
"phf_shared 0.10.0",
]
[[package]]
name = "phf_codegen"
version = "0.11.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "aef8048c789fa5e851558d709946d6d79a8ff88c0440c587967f8e94bfb1216a"
dependencies = [
"phf_generator 0.11.3",
"phf_shared 0.11.3",
]
[[package]]
name = "phf_generator"
version = "0.8.0"
@@ -6836,6 +6814,15 @@ dependencies = [
"siphasher 1.0.1",
]
[[package]]
name = "phf_shared"
version = "0.12.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "06005508882fb681fd97892ecff4b7fd0fee13ef1aa569f8695dae7ab9099981"
dependencies = [
"siphasher 1.0.1",
]
[[package]]
name = "pin-project"
version = "1.1.10"
@@ -6967,7 +6954,7 @@ dependencies = [
"concurrent-queue",
"hermit-abi",
"pin-project-lite",
"rustix 1.0.7",
"rustix 1.0.8",
"tracing",
"windows-sys 0.59.0",
]
@@ -7647,13 +7634,13 @@ dependencies = [
[[package]]
name = "rfd"
version = "0.15.3"
version = "0.15.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "80c844748fdc82aae252ee4594a89b6e7ebef1063de7951545564cbc4e57075d"
checksum = "ef2bee61e6cffa4635c72d7d81a84294e28f0930db0ddcb0f66d10244674ebed"
dependencies = [
"ashpd 0.11.0",
"block2 0.6.1",
"dispatch2 0.2.0",
"dispatch2",
"js-sys",
"log",
"objc2 0.6.1",
@@ -8107,7 +8094,7 @@ dependencies = [
"dirs",
"hex",
"keyring",
"rfd 0.15.3",
"rfd 0.15.4",
"rust-embed",
"rust-i18n",
"serde",
@@ -8443,15 +8430,15 @@ dependencies = [
[[package]]
name = "rustix"
version = "1.0.7"
version = "1.0.8"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "c71e83d6afe7ff64890ec6b71d6a69bb8a610ab78ce364b3352876bb4c801266"
checksum = "11181fbabf243db407ef8df94a6ce0b2f9a733bd8be4ad02b4eda9602296cac8"
dependencies = [
"bitflags 2.9.1",
"errno",
"libc",
"linux-raw-sys 0.9.4",
"windows-sys 0.59.0",
"windows-sys 0.60.2",
]
[[package]]
@@ -8675,6 +8662,12 @@ dependencies = [
"windows-sys 0.59.0",
]
[[package]]
name = "scoped-tls"
version = "1.0.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e1cf6437eb19a8f4a6cc0f7dca544973b0b78843adbfeb3683d1a94a0024a294"
[[package]]
name = "scopeguard"
version = "1.2.0"
@@ -9698,7 +9691,7 @@ dependencies = [
"fastrand",
"getrandom 0.3.3",
"once_cell",
"rustix 1.0.7",
"rustix 1.0.8",
"windows-sys 0.59.0",
]
@@ -10079,7 +10072,7 @@ dependencies = [
"serde_spanned",
"toml_datetime",
"toml_write",
"winnow 0.7.11",
"winnow 0.7.12",
]
[[package]]
@@ -10769,6 +10762,66 @@ dependencies = [
"web-sys",
]
[[package]]
name = "wayland-backend"
version = "0.3.10"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "fe770181423e5fc79d3e2a7f4410b7799d5aab1de4372853de3c6aa13ca24121"
dependencies = [
"cc",
"downcast-rs",
"rustix 0.38.44",
"scoped-tls",
"smallvec",
"wayland-sys",
]
[[package]]
name = "wayland-client"
version = "0.31.10"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "978fa7c67b0847dbd6a9f350ca2569174974cd4082737054dbb7fbb79d7d9a61"
dependencies = [
"bitflags 2.9.1",
"rustix 0.38.44",
"wayland-backend",
"wayland-scanner",
]
[[package]]
name = "wayland-protocols"
version = "0.32.8"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "779075454e1e9a521794fed15886323ea0feda3f8b0fc1390f5398141310422a"
dependencies = [
"bitflags 2.9.1",
"wayland-backend",
"wayland-client",
"wayland-scanner",
]
[[package]]
name = "wayland-scanner"
version = "0.31.6"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "896fdafd5d28145fce7958917d69f2fd44469b1d4e861cb5961bcbeebc6d1484"
dependencies = [
"proc-macro2",
"quick-xml 0.37.5",
"quote",
]
[[package]]
name = "wayland-sys"
version = "0.31.6"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "dbcebb399c77d5aa9fa5db874806ee7b4eba4e73650948e8f93963f128896615"
dependencies = [
"dlib",
"log",
"pkg-config",
]
[[package]]
name = "web-sys"
version = "0.3.77"
@@ -11440,9 +11493,9 @@ dependencies = [
[[package]]
name = "winnow"
version = "0.7.11"
version = "0.7.12"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "74c7b26e3480b707944fc872477815d29a8e429d2f93a1ce000f5fa84a15cbcd"
checksum = "f3edebf492c8125044983378ecb5766203ad3b4c2f7a922bd7dd207f6d443e95"
dependencies = [
"memchr",
]
@@ -11542,7 +11595,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "af3a19837351dc82ba89f8a125e22a3c475f05aba604acc023d62b2739ae2909"
dependencies = [
"libc",
"rustix 1.0.7",
"rustix 1.0.8",
]
[[package]]
@@ -11641,9 +11694,9 @@ dependencies = [
[[package]]
name = "zbus"
version = "5.8.0"
version = "5.9.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "597f45e98bc7e6f0988276012797855613cd8269e23b5be62cc4e5d28b7e515d"
checksum = "4bb4f9a464286d42851d18a605f7193b8febaf5b0919d71c6399b7b26e5b0aad"
dependencies = [
"async-broadcast",
"async-recursion",
@@ -11661,8 +11714,8 @@ dependencies = [
"tracing",
"uds_windows",
"windows-sys 0.59.0",
"winnow 0.7.11",
"zbus_macros 5.8.0",
"winnow 0.7.12",
"zbus_macros 5.9.0",
"zbus_names 4.2.0",
"zvariant 5.6.0",
]
@@ -11682,9 +11735,9 @@ dependencies = [
[[package]]
name = "zbus_macros"
version = "5.8.0"
version = "5.9.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e5c8e4e14dcdd9d97a98b189cd1220f30e8394ad271e8c987da84f73693862c2"
checksum = "ef9859f68ee0c4ee2e8cde84737c78e3f4c54f946f2a38645d0d4c7a95327659"
dependencies = [
"proc-macro-crate 3.3.0",
"proc-macro2",
@@ -11714,7 +11767,7 @@ checksum = "7be68e64bf6ce8db94f63e72f0c7eb9a60d733f7e0499e628dfab0f84d6bcb97"
dependencies = [
"serde",
"static_assertions",
"winnow 0.7.11",
"winnow 0.7.12",
"zvariant 5.6.0",
]
@@ -11906,7 +11959,7 @@ dependencies = [
"enumflags2",
"serde",
"url",
"winnow 0.7.11",
"winnow 0.7.12",
"zvariant_derive 5.6.0",
"zvariant_utils 3.2.0",
]
@@ -11959,5 +12012,5 @@ dependencies = [
"serde",
"static_assertions",
"syn 2.0.104",
"winnow 0.7.11",
"winnow 0.7.12",
]

View File

@@ -108,8 +108,8 @@ cfg-if = "1.0.1"
chacha20poly1305 = { version = "0.10.1" }
chrono = { version = "0.4.41", features = ["serde"] }
clap = { version = "4.5.41", features = ["derive", "env"] }
const-str = { version = "0.6.2", features = ["std", "proc"] }
crc32fast = "1.4.2"
const-str = { version = "0.6.3", features = ["std", "proc"] }
crc32fast = "1.5.0"
criterion = { version = "0.5", features = ["html_reports"] }
dashmap = "6.1.0"
datafusion = "46.0.1"
@@ -196,7 +196,7 @@ reqwest = { version = "0.12.22", default-features = false, features = [
"json",
"blocking",
] }
rfd = { version = "0.15.3", default-features = false, features = [
rfd = { version = "0.15.4", default-features = false, features = [
"xdg-portal",
"tokio",
] }

View File

@@ -1,121 +1,94 @@
# Multi-stage build for RustFS production image
FROM alpine:latest AS build
# Build arguments - use TARGETPLATFORM for consistency with Dockerfile.source
# Build stage: Download and extract RustFS binary
FROM alpine:3.22 AS build
# Build arguments for platform and release
ARG TARGETPLATFORM
ARG BUILDPLATFORM
ARG RELEASE=latest
# Install dependencies for downloading and verifying binaries
RUN apk add --no-cache \
ca-certificates \
curl \
bash \
wget \
unzip \
jq
# Install minimal dependencies for downloading and extracting
RUN apk add --no-cache ca-certificates curl unzip
# Create build directory
WORKDIR /build
# Map TARGETPLATFORM to architecture format used in builds
# Map TARGETPLATFORM to architecture
RUN case "${TARGETPLATFORM}" in \
"linux/amd64") ARCH="x86_64" ;; \
"linux/arm64") ARCH="aarch64" ;; \
*) echo "Unsupported platform: ${TARGETPLATFORM}" && exit 1 ;; \
*) echo "Unsupported platform: ${TARGETPLATFORM}" >&2 && exit 1 ;; \
esac && \
echo "ARCH=${ARCH}" > /build/arch.env
# Download rustfs binary from dl.rustfs.com (release channel only)
# Download and extract RustFS binary
RUN . /build/arch.env && \
BASE_URL="https://dl.rustfs.com/artifacts/rustfs/release" && \
PLATFORM="linux" && \
if [ "${RELEASE}" = "latest" ]; then \
# Download latest release version \
PACKAGE_NAME="rustfs-${PLATFORM}-${ARCH}-latest.zip"; \
DOWNLOAD_URL="${BASE_URL}/${PACKAGE_NAME}"; \
echo "📥 Downloading latest release build: ${PACKAGE_NAME}"; \
else \
# Download specific release version \
PACKAGE_NAME="rustfs-${PLATFORM}-${ARCH}-v${RELEASE}.zip"; \
DOWNLOAD_URL="${BASE_URL}/${PACKAGE_NAME}"; \
echo "📥 Downloading specific release version: ${PACKAGE_NAME}"; \
fi && \
echo "🔗 Download URL: ${DOWNLOAD_URL}" && \
curl -f -L "${DOWNLOAD_URL}" -o /build/rustfs.zip && \
if [ ! -f /build/rustfs.zip ] || [ ! -s /build/rustfs.zip ]; then \
echo "❌ Failed to download binary package"; \
echo "💡 Make sure the package ${PACKAGE_NAME} exists"; \
echo "🔗 Check: ${DOWNLOAD_URL}"; \
exit 1; \
fi && \
unzip /build/rustfs.zip -d /build && \
PACKAGE_NAME="rustfs-linux-${ARCH}-${RELEASE}.zip" && \
DOWNLOAD_URL="${BASE_URL}/${PACKAGE_NAME}" && \
echo "Downloading ${PACKAGE_NAME} from ${DOWNLOAD_URL}" >&2 && \
curl -f -L "${DOWNLOAD_URL}" -o rustfs.zip && \
unzip rustfs.zip -d /build && \
chmod +x /build/rustfs && \
rm /build/rustfs.zip && \
echo "✅ Successfully downloaded and extracted rustfs binary"
rm rustfs.zip || { echo "Failed to download or extract ${PACKAGE_NAME}" >&2; exit 1; }
# Runtime stage
FROM alpine:latest
# Runtime stage: Configure runtime environment
FROM alpine:3.22.1
# Set build arguments and labels
# Build arguments and labels
ARG RELEASE=latest
ARG BUILD_DATE
ARG VCS_REF
LABEL name="RustFS" \
vendor="RustFS Team" \
maintainer="RustFS Team <dev@rustfs.com>" \
version="${RELEASE}" \
release="${RELEASE}" \
build-date="${BUILD_DATE}" \
vcs-ref="${VCS_REF}" \
summary="RustFS is a high-performance distributed object storage system written in Rust, compatible with S3 API." \
description="RustFS is a high-performance distributed object storage software built using Rust. It supports erasure coding storage, multi-tenant management, observability, and other enterprise-level features." \
url="https://rustfs.com" \
license="Apache-2.0"
vendor="RustFS Team" \
maintainer="RustFS Team <dev@rustfs.com>" \
version="${RELEASE}" \
release="${RELEASE}" \
build-date="${BUILD_DATE}" \
vcs-ref="${VCS_REF}" \
summary="High-performance distributed object storage system compatible with S3 API" \
description="RustFS is a distributed object storage system written in Rust, supporting erasure coding, multi-tenant management, and observability." \
url="https://rustfs.com" \
license="Apache-2.0"
# Install runtime dependencies
RUN apk add --no-cache \
ca-certificates \
curl \
tzdata \
bash \
&& addgroup -g 1000 rustfs \
&& adduser -u 1000 -G rustfs -s /bin/sh -D rustfs
RUN echo "https://dl-cdn.alpinelinux.org/alpine/v3.20/community" >> /etc/apk/repositories && \
apk update && \
apk add --no-cache ca-certificates bash gosu && \
addgroup -g 1000 rustfs && \
adduser -u 1000 -G rustfs -s /bin/bash -D rustfs
# Environment variables
ENV RUSTFS_ACCESS_KEY=rustfsadmin \
# Copy CA certificates and RustFS binary from build stage
COPY --from=build /etc/ssl/certs/ca-certificates.crt /etc/ssl/certs/
COPY --from=build /build/rustfs /usr/bin/rustfs
# Copy entry point script
COPY entrypoint.sh /entrypoint.sh
# Set permissions
RUN chmod +x /usr/bin/rustfs /entrypoint.sh && \
mkdir -p /data /logs && \
chown rustfs:rustfs /data /logs && \
chmod 700 /data /logs
# Environment variables (credentials should be set via environment or secrets)
ENV RUSTFS_ADDRESS=":9000" \
RUSTFS_ACCESS_KEY=rustfsadmin \
RUSTFS_SECRET_KEY=rustfsadmin \
RUSTFS_ADDRESS=":9000" \
RUSTFS_CONSOLE_ENABLE=true \
RUSTFS_VOLUMES=/data \
RUST_LOG=warn
# Set permissions for /usr/bin (similar to MinIO's approach)
RUN chmod -R 755 /usr/bin
# Copy CA certificates and binaries from build stage
COPY --from=build /etc/ssl/certs/ca-certificates.crt /etc/ssl/certs/
COPY --from=build /build/rustfs /usr/bin/
# Set executable permissions
RUN chmod +x /usr/bin/rustfs
# Create data directory
RUN mkdir -p /data /config && chown -R rustfs:rustfs /data /config
# Switch to non-root user
USER rustfs
# Set working directory
WORKDIR /data
RUST_LOG=warn \
RUSTFS_OBS_LOG_DIRECTORY=/logs \
RUSTFS_SINKS_FILE_PATH=/logs
# Expose port
EXPOSE 9000
# Volumes for data and logs
VOLUME ["/data", "/logs"]
# Volume for data
VOLUME ["/data"]
# Set entrypoint
ENTRYPOINT ["/usr/bin/rustfs"]
# Set entry point
ENTRYPOINT ["/entrypoint.sh"]
CMD ["/usr/bin/rustfs"]

192
Makefile
View File

@@ -46,21 +46,6 @@ setup-hooks:
chmod +x .git/hooks/pre-commit
@echo "✅ Git hooks setup complete!"
.PHONY: init-devenv
init-devenv:
$(DOCKER_CLI) build -t $(IMAGE_NAME) -f Dockerfile.source .
$(DOCKER_CLI) stop $(CONTAINER_NAME)
$(DOCKER_CLI) rm $(CONTAINER_NAME)
$(DOCKER_CLI) run -d --name $(CONTAINER_NAME) -p 9010:9010 -p 9000:9000 -v $(shell pwd):/root/s3-rustfs -it $(IMAGE_NAME)
.PHONY: start
start:
$(DOCKER_CLI) start $(CONTAINER_NAME)
.PHONY: stop
stop:
$(DOCKER_CLI) stop $(CONTAINER_NAME)
.PHONY: e2e-server
e2e-server:
sh $(shell pwd)/scripts/run.sh
@@ -80,8 +65,6 @@ build-dev:
@echo "🔨 Building RustFS in development mode..."
./build-rustfs.sh --dev
# Docker-based build (alternative approach)
# Usage: make BUILD_OS=ubuntu22.04 build-docker
# Output: target/ubuntu22.04/release/rustfs
@@ -98,13 +81,13 @@ build-docker:
.PHONY: build-musl
build-musl:
@echo "🔨 Building rustfs for x86_64-unknown-linux-musl..."
@echo "💡 On macOS/Windows, use 'make build-docker' or 'make docker-buildx' instead"
@echo "💡 On macOS/Windows, use 'make build-docker' or 'make docker-dev' instead"
./build-rustfs.sh --platform x86_64-unknown-linux-musl
.PHONY: build-gnu
build-gnu:
@echo "🔨 Building rustfs for x86_64-unknown-linux-gnu..."
@echo "💡 On macOS/Windows, use 'make build-docker' or 'make docker-buildx' instead"
@echo "💡 On macOS/Windows, use 'make build-docker' or 'make docker-dev' instead"
./build-rustfs.sh --platform x86_64-unknown-linux-gnu
.PHONY: deploy-dev
@@ -112,15 +95,19 @@ deploy-dev: build-musl
@echo "🚀 Deploying to dev server: $${IP}"
./scripts/dev_deploy.sh $${IP}
# Multi-architecture Docker build targets (NEW: using docker-buildx.sh)
# ========================================================================================
# Docker Multi-Architecture Builds (Primary Methods)
# ========================================================================================
# Production builds using docker-buildx.sh (for CI/CD and production)
.PHONY: docker-buildx
docker-buildx:
@echo "🏗️ Building multi-architecture Docker images with buildx..."
@echo "🏗️ Building multi-architecture production Docker images with buildx..."
./docker-buildx.sh
.PHONY: docker-buildx-push
docker-buildx-push:
@echo "🚀 Building and pushing multi-architecture Docker images with buildx..."
@echo "🚀 Building and pushing multi-architecture production Docker images with buildx..."
./docker-buildx.sh --push
.PHONY: docker-buildx-version
@@ -129,7 +116,7 @@ docker-buildx-version:
echo "❌ 错误: 请指定版本, 例如: make docker-buildx-version VERSION=v1.0.0"; \
exit 1; \
fi
@echo "🏗️ Building multi-architecture Docker images (version: $(VERSION))..."
@echo "🏗️ Building multi-architecture production Docker images (version: $(VERSION))..."
./docker-buildx.sh --release $(VERSION)
.PHONY: docker-buildx-push-version
@@ -138,21 +125,114 @@ docker-buildx-push-version:
echo "❌ 错误: 请指定版本, 例如: make docker-buildx-push-version VERSION=v1.0.0"; \
exit 1; \
fi
@echo "🚀 Building and pushing multi-architecture Docker images (version: $(VERSION))..."
@echo "🚀 Building and pushing multi-architecture production Docker images (version: $(VERSION))..."
./docker-buildx.sh --release $(VERSION) --push
# Development/Source builds using direct buildx commands
.PHONY: docker-dev
docker-dev:
@echo "🏗️ Building multi-architecture development Docker images with buildx..."
@echo "💡 This builds from source code and is intended for local development and testing"
@echo "⚠️ Multi-arch images cannot be loaded locally, use docker-dev-push to push to registry"
$(DOCKER_CLI) buildx build \
--platform linux/amd64,linux/arm64 \
--file $(DOCKERFILE_SOURCE) \
--tag rustfs:source-latest \
--tag rustfs:dev-latest \
.
.PHONY: docker-dev-local
docker-dev-local:
@echo "🏗️ Building single-architecture development Docker image for local use..."
@echo "💡 This builds from source code for the current platform and loads locally"
$(DOCKER_CLI) buildx build \
--file $(DOCKERFILE_SOURCE) \
--tag rustfs:source-latest \
--tag rustfs:dev-latest \
--load \
.
.PHONY: docker-dev-push
docker-dev-push:
@if [ -z "$(REGISTRY)" ]; then \
echo "❌ 错误: 请指定镜像仓库, 例如: make docker-dev-push REGISTRY=ghcr.io/username"; \
exit 1; \
fi
@echo "🚀 Building and pushing multi-architecture development Docker images..."
@echo "💡 推送到仓库: $(REGISTRY)"
$(DOCKER_CLI) buildx build \
--platform linux/amd64,linux/arm64 \
--file $(DOCKERFILE_SOURCE) \
--tag $(REGISTRY)/rustfs:source-latest \
--tag $(REGISTRY)/rustfs:dev-latest \
--push \
.
# Local production builds using direct buildx (alternative to docker-buildx.sh)
.PHONY: docker-buildx-production-local
docker-buildx-production-local:
@echo "🏗️ Building single-architecture production Docker image locally..."
@echo "💡 Alternative to docker-buildx.sh for local testing"
$(DOCKER_CLI) buildx build \
--file $(DOCKERFILE_PRODUCTION) \
--tag rustfs:production-latest \
--tag rustfs:latest \
--load \
--build-arg RELEASE=latest \
.
# ========================================================================================
# Single Architecture Docker Builds (Traditional)
# ========================================================================================
.PHONY: docker-build-production
docker-build-production:
@echo "🏗️ Building production Docker image..."
@echo "🏗️ Building single-architecture production Docker image..."
@echo "💡 Consider using 'make docker-buildx-production-local' for multi-arch support"
$(DOCKER_CLI) build -f $(DOCKERFILE_PRODUCTION) -t rustfs:latest .
.PHONY: docker-build-source
docker-build-source:
@echo "🏗️ Building source Docker image..."
@echo "🏗️ Building single-architecture source Docker image..."
@echo "💡 Consider using 'make docker-dev-local' for multi-arch support"
$(DOCKER_CLI) build -f $(DOCKERFILE_SOURCE) -t rustfs:source .
# ========================================================================================
# Development Environment
# ========================================================================================
.PHONY: dev-env-start
dev-env-start:
@echo "🚀 Starting development environment..."
$(DOCKER_CLI) buildx build \
--file $(DOCKERFILE_SOURCE) \
--tag rustfs:dev \
--load \
.
$(DOCKER_CLI) stop $(CONTAINER_NAME) 2>/dev/null || true
$(DOCKER_CLI) rm $(CONTAINER_NAME) 2>/dev/null || true
$(DOCKER_CLI) run -d --name $(CONTAINER_NAME) \
-p 9010:9010 -p 9000:9000 \
-v $(shell pwd):/workspace \
-it rustfs:dev
.PHONY: dev-env-stop
dev-env-stop:
@echo "🛑 Stopping development environment..."
$(DOCKER_CLI) stop $(CONTAINER_NAME) 2>/dev/null || true
$(DOCKER_CLI) rm $(CONTAINER_NAME) 2>/dev/null || true
.PHONY: dev-env-restart
dev-env-restart: dev-env-stop dev-env-start
# ========================================================================================
# Build Utilities
# ========================================================================================
.PHONY: docker-inspect-multiarch
docker-inspect-multiarch:
@if [ -z "$(IMAGE)" ]; then \
@@ -165,7 +245,7 @@ docker-inspect-multiarch:
.PHONY: build-cross-all
build-cross-all:
@echo "🔧 Building all target architectures..."
@echo "💡 On macOS/Windows, use 'make docker-buildx' for reliable multi-arch builds"
@echo "💡 On macOS/Windows, use 'make docker-dev' for reliable multi-arch builds"
@echo "🔨 Generating protobuf code..."
cargo run --bin gproto || true
@echo "🔨 Building x86_64-unknown-linux-musl..."
@@ -174,6 +254,10 @@ build-cross-all:
./build-rustfs.sh --platform aarch64-unknown-linux-gnu
@echo "✅ All architectures built successfully!"
# ========================================================================================
# Help and Documentation
# ========================================================================================
.PHONY: help-build
help-build:
@echo "🔨 RustFS 构建帮助:"
@@ -206,23 +290,61 @@ help-build:
help-docker:
@echo "🐳 Docker 多架构构建帮助:"
@echo ""
@echo "🚀 推荐使用 (新的 docker-buildx 方式):"
@echo " make docker-buildx # 构建多架构镜像(不推送)"
@echo " make docker-buildx-push # 构建并推送多架构镜像"
@echo "🚀 生产镜像构建 (推荐使用 docker-buildx.sh):"
@echo " make docker-buildx # 构建生产多架构镜像(不推送)"
@echo " make docker-buildx-push # 构建并推送生产多架构镜像"
@echo " make docker-buildx-version VERSION=v1.0.0 # 构建指定版本"
@echo " make docker-buildx-push-version VERSION=v1.0.0 # 构建并推送指定版本"
@echo ""
@echo "🏗️ 单架构构建:"
@echo " make docker-build-production # 构建生产环境镜像"
@echo " make docker-build-source # 构建源码构建镜像"
@echo "🔧 开发/源码镜像构建 (本地开发测试):"
@echo " make docker-dev # 构建开发多架构镜像(无法本地加载)"
@echo " make docker-dev-local # 构建开发单架构镜像(本地加载)"
@echo " make docker-dev-push REGISTRY=xxx # 构建并推送开发镜像"
@echo ""
@echo "🏗️ 本地生产镜像构建 (替代方案):"
@echo " make docker-buildx-production-local # 本地构建生产单架构镜像"
@echo ""
@echo "📦 单架构构建 (传统方式):"
@echo " make docker-build-production # 构建单架构生产镜像"
@echo " make docker-build-source # 构建单架构源码镜像"
@echo ""
@echo "🚀 开发环境管理:"
@echo " make dev-env-start # 启动开发容器环境"
@echo " make dev-env-stop # 停止开发容器环境"
@echo " make dev-env-restart # 重启开发容器环境"
@echo ""
@echo "🔧 辅助工具:"
@echo " make build-cross-all # 构建所有架构的二进制文件"
@echo " make docker-inspect-multiarch IMAGE=xxx # 检查镜像的架构支持"
@echo ""
@echo "📋 环境变量 (在推送时需要设置):"
@echo "📋 环境变量:"
@echo " REGISTRY 镜像仓库地址 (推送时需要)"
@echo " DOCKERHUB_USERNAME Docker Hub 用户名"
@echo " DOCKERHUB_TOKEN Docker Hub 访问令牌"
@echo " GITHUB_TOKEN GitHub 访问令牌"
@echo ""
@echo "💡 更多详情请参考项目根目录的 docker-buildx.sh 脚本"
@echo "💡 建议:"
@echo " - 生产用途: 使用 docker-buildx* 命令 (基于预编译二进制)"
@echo " - 本地开发: 使用 docker-dev* 命令 (从源码构建)"
@echo " - 开发环境: 使用 dev-env-* 命令管理开发容器"
.PHONY: help
help:
@echo "🦀 RustFS Makefile 帮助:"
@echo ""
@echo "📋 主要命令分类:"
@echo " make help-build # 显示构建相关帮助"
@echo " make help-docker # 显示 Docker 相关帮助"
@echo ""
@echo "🔧 代码质量:"
@echo " make fmt # 格式化代码"
@echo " make clippy # 运行 clippy 检查"
@echo " make test # 运行测试"
@echo " make pre-commit # 运行所有预提交检查"
@echo ""
@echo "🚀 快速开始:"
@echo " make build # 构建 RustFS 二进制"
@echo " make docker-dev-local # 构建开发 Docker 镜像(本地)"
@echo " make dev-env-start # 启动开发环境"
@echo ""
@echo "💡 更多帮助请使用 'make help-build' 或 'make help-docker'"

View File

@@ -108,14 +108,26 @@ pub const DEFAULT_CONSOLE_ADDRESS: &str = concat!(":", DEFAULT_CONSOLE_PORT);
/// It is used to store the logs of the application.
/// Default value: rustfs.log
/// Environment variable: RUSTFS_OBSERVABILITY_LOG_FILENAME
pub const DEFAULT_LOG_FILENAME: &str = "rustfs.log";
pub const DEFAULT_LOG_FILENAME: &str = "rustfs";
/// Default OBS log filename for rustfs
/// This is the default log filename for OBS.
/// It is used to store the logs of the application.
/// Default value: rustfs.log
pub const DEFAULT_OBS_LOG_FILENAME: &str = concat!(DEFAULT_LOG_FILENAME, ".log");
/// Default sink file log file for rustfs
/// This is the default sink file log file for rustfs.
/// It is used to store the logs of the application.
/// Default value: rustfs-sink.log
pub const DEFAULT_SINK_FILE_LOG_FILE: &str = concat!(DEFAULT_LOG_FILENAME, "-sink.log");
/// Default log directory for rustfs
/// This is the default log directory for rustfs.
/// It is used to store the logs of the application.
/// Default value: logs
/// Environment variable: RUSTFS_OBSERVABILITY_LOG_DIRECTORY
pub const DEFAULT_LOG_DIR: &str = "deploy/logs";
pub const DEFAULT_LOG_DIR: &str = "/logs";
/// Default log rotation size mb for rustfs
/// This is the default log rotation size for rustfs.

View File

@@ -14,7 +14,8 @@
use rustfs_config::{
APP_NAME, DEFAULT_LOG_DIR, DEFAULT_LOG_FILENAME, DEFAULT_LOG_KEEP_FILES, DEFAULT_LOG_LEVEL, DEFAULT_LOG_ROTATION_SIZE_MB,
DEFAULT_LOG_ROTATION_TIME, ENVIRONMENT, METER_INTERVAL, SAMPLE_RATIO, SERVICE_VERSION, USE_STDOUT,
DEFAULT_LOG_ROTATION_TIME, DEFAULT_OBS_LOG_FILENAME, DEFAULT_SINK_FILE_LOG_FILE, ENVIRONMENT, METER_INTERVAL, SAMPLE_RATIO,
SERVICE_VERSION, USE_STDOUT,
};
use serde::{Deserialize, Serialize};
use std::env;
@@ -104,7 +105,7 @@ impl OtelConfig {
log_filename: env::var("RUSTFS_OBS_LOG_FILENAME")
.ok()
.and_then(|v| v.parse().ok())
.or(Some(DEFAULT_LOG_FILENAME.to_string())),
.or(Some(DEFAULT_OBS_LOG_FILENAME.to_string())),
log_rotation_size_mb: env::var("RUSTFS_OBS_LOG_ROTATION_SIZE_MB")
.ok()
.and_then(|v| v.parse().ok())
@@ -210,16 +211,15 @@ pub struct FileSinkConfig {
impl FileSinkConfig {
pub fn get_default_log_path() -> String {
let temp_dir = env::temp_dir().join("rustfs");
let temp_dir = env::temp_dir().join(DEFAULT_LOG_FILENAME);
if let Err(e) = std::fs::create_dir_all(&temp_dir) {
eprintln!("Failed to create log directory: {e}");
return "rustfs/rustfs.log".to_string();
return DEFAULT_LOG_DIR.to_string();
}
temp_dir
.join("rustfs.log")
.join(DEFAULT_SINK_FILE_LOG_FILE)
.to_str()
.unwrap_or("rustfs/rustfs.log")
.unwrap_or(DEFAULT_LOG_DIR)
.to_string()
}
pub fn new() -> Self {

View File

@@ -77,8 +77,7 @@ impl Logger {
}
/// Asynchronous logging of unified log entries
#[tracing::instrument(skip(self), fields(log_source = "logger"))]
#[tracing::instrument(level = "error", skip_all)]
#[tracing::instrument(skip_all, fields(log_source = "logger"))]
pub async fn log_entry(&self, entry: UnifiedLogEntry) -> Result<(), GlobalError> {
// Extract information for tracing based on entry type
match &entry {

View File

@@ -14,6 +14,7 @@
use crate::{AppConfig, SinkConfig, UnifiedLogEntry};
use async_trait::async_trait;
use rustfs_config::DEFAULT_SINK_FILE_LOG_FILE;
use std::sync::Arc;
#[cfg(feature = "file")]
@@ -71,7 +72,7 @@ pub async fn create_sinks(config: &AppConfig) -> Vec<Arc<dyn Sink>> {
SinkConfig::File(file_config) => {
tracing::debug!("FileSink: Using path: {}", file_config.path);
match file::FileSink::new(
file_config.path.clone(),
format!("{}/{}", file_config.path.clone(), DEFAULT_SINK_FILE_LOG_FILE),
file_config.buffer_size.unwrap_or(8192),
file_config.flush_interval_ms.unwrap_or(1000),
file_config.flush_threshold.unwrap_or(100),

View File

@@ -39,14 +39,21 @@ pub async fn node_service_time_out_client(
Box<dyn Error>,
> {
let token: MetadataValue<_> = "rustfs rpc".parse()?;
let channel = match GLOBAL_Conn_Map.read().await.get(addr) {
Some(channel) => channel.clone(),
let channel = { GLOBAL_Conn_Map.read().await.get(addr).cloned() };
let channel = match channel {
Some(channel) => channel,
None => {
let connector = Endpoint::from_shared(addr.to_string())?.connect_timeout(Duration::from_secs(60));
connector.connect().await?
let channel = connector.connect().await?;
{
GLOBAL_Conn_Map.write().await.insert(addr.to_string(), channel.clone());
}
channel
}
};
GLOBAL_Conn_Map.write().await.insert(addr.to_string(), channel.clone());
// let timeout_channel = Timeout::new(channel, Duration::from_secs(60));
Ok(NodeServiceClient::with_interceptor(

51
entrypoint.sh Normal file
View File

@@ -0,0 +1,51 @@
#!/bin/bash
set -e
# Function to adjust rustfs user/group to match mounted directory ownership
fix_permissions() {
local dir="$1"
local user="rustfs"
local group="rustfs"
# Skip if directory doesn't exist or isn't mounted
if [ ! -d "$dir" ]; then
echo "Directory $dir does not exist, creating it"
mkdir -p "$dir"
chown "$user:$group" "$dir"
chmod 700 "$dir"
return
fi
# Get directory ownership
local dir_uid=$(stat -c %u "$dir")
local dir_gid=$(stat -c %g "$dir")
# If directory is owned by root or inaccessible, skip adjustment
if [ "$dir_uid" = "0" ] || [ "$dir_gid" = "0" ]; then
echo "Warning: Directory $dir is owned by root, skipping UID/GID adjustment"
chown "$user:$group" "$dir"
chmod 700 "$dir"
return
fi
# Adjust rustfs user/group to match directory ownership
if [ "$dir_uid" != "$(id -u $user)" ]; then
echo "Adjusting UID of $user to $dir_uid for $dir"
usermod -u "$dir_uid" "$user"
fi
if [ "$dir_gid" != "$(id -g $group)" ]; then
echo "Adjusting GID of $group to $dir_gid for $dir"
groupmod -g "$dir_gid" "$group"
fi
# Ensure permissions are correct
chown "$user:$group" "$dir"
chmod 700 "$dir"
}
# Fix permissions for /data and /logs
fix_permissions "/data"
fix_permissions "/logs"
# Run RustFS as the rustfs user
exec gosu rustfs:rustfs "$@"

View File

@@ -792,11 +792,6 @@ impl S3 for FS {
};
let last_modified = info.mod_time.map(Timestamp::from);
let body = Some(StreamingBlob::wrap(bytes_stream(
ReaderStream::with_capacity(reader.stream, DEFAULT_READ_BUFFER_SIZE),
info.size as usize,
)));
let mut rs = rs;
if let Some(part_number) = part_number {
@@ -805,17 +800,25 @@ impl S3 for FS {
}
}
let mut content_length = info.size as i64;
let content_range = if let Some(rs) = rs {
let total_size = info.get_actual_size().map_err(ApiError::from)?;
let (start, length) = rs.get_offset_length(total_size as i64).map_err(ApiError::from)?;
content_length = length;
Some(format!("bytes {}-{}/{}", start, start as i64 + length - 1, total_size))
} else {
None
};
let body = Some(StreamingBlob::wrap(bytes_stream(
ReaderStream::with_capacity(reader.stream, DEFAULT_READ_BUFFER_SIZE),
content_length as usize,
)));
let output = GetObjectOutput {
body,
content_length: Some(info.size as i64),
content_length: Some(content_length),
last_modified,
content_type,
accept_ranges: Some("bytes".to_string()),

View File

@@ -51,7 +51,7 @@ export RUSTFS_CONSOLE_ADDRESS=":9001"
# export RUSTFS_TLS_PATH="./deploy/certs"
# 可观测性 相关配置信息
export RUSTFS_OBS_ENDPOINT=http://localhost:4317 # OpenTelemetry Collector 的地址
#export RUSTFS_OBS_ENDPOINT=http://localhost:4317 # OpenTelemetry Collector 的地址
#export RUSTFS_OBS_USE_STDOUT=false # 是否使用标准输出
#export RUSTFS_OBS_SAMPLE_RATIO=2.0 # 采样率0.0-1.0之间0.0表示不采样1.0表示全部采样
#export RUSTFS_OBS_METER_INTERVAL=1 # 采样间隔,单位为秒