From 8c4735ff884ffb027cccca592d7bd20629ce0f81 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=E5=AE=89=E6=AD=A3=E8=B6=85?= Date: Thu, 5 Mar 2026 17:25:37 +0800 Subject: [PATCH] docs: scope AGENTS instructions by directory (#2083) --- .github/AGENTS.md | 30 +++++++ AGENTS.md | 150 +++++++++++++---------------------- crates/AGENTS.md | 20 +++++ crates/config/AGENTS.md | 28 +++++++ crates/e2e_test/AGENTS.md | 26 ++++++ crates/ecstore/AGENTS.md | 28 +++++++ crates/iam/AGENTS.md | 27 +++++++ crates/kms/AGENTS.md | 27 +++++++ crates/policy/AGENTS.md | 26 ++++++ rustfs/src/admin/AGENTS.md | 28 +++++++ rustfs/src/storage/AGENTS.md | 28 +++++++ 11 files changed, 321 insertions(+), 97 deletions(-) create mode 100644 .github/AGENTS.md create mode 100644 crates/AGENTS.md create mode 100644 crates/config/AGENTS.md create mode 100644 crates/e2e_test/AGENTS.md create mode 100644 crates/ecstore/AGENTS.md create mode 100644 crates/iam/AGENTS.md create mode 100644 crates/kms/AGENTS.md create mode 100644 crates/policy/AGENTS.md create mode 100644 rustfs/src/admin/AGENTS.md create mode 100644 rustfs/src/storage/AGENTS.md diff --git a/.github/AGENTS.md b/.github/AGENTS.md new file mode 100644 index 00000000..92c3ef72 --- /dev/null +++ b/.github/AGENTS.md @@ -0,0 +1,30 @@ +# GitHub Workflow Instructions + +Applies to `.github/` and repository pull-request operations. + +## Pull Requests + +- PR titles and descriptions must be in English. +- Use `.github/pull_request_template.md` for every PR body. +- Keep all template section headings. +- Use `N/A` for non-applicable sections. +- Include verification commands in the PR details. +- For `gh pr create` and `gh pr edit`, always write markdown body to a file and pass `--body-file`. +- Do not use multiline inline `--body`; backticks and shell expansion can corrupt content or trigger unintended commands. +- Recommended pattern: + - `cat > /tmp/pr_body.md <<'EOF'` + - `...markdown...` + - `EOF` + - `gh pr create ... --body-file /tmp/pr_body.md` + +## CI Alignment + +When changing CI-sensitive behavior, keep local validation aligned with `.github/workflows/ci.yml`. + +Current `test-and-lint` gate includes: + +- `cargo nextest run --all --exclude e2e_test` +- `cargo test --all --doc` +- `cargo fmt --all --check` +- `cargo clippy --all-targets --all-features -- -D warnings` +- `./scripts/check_layer_dependencies.sh` diff --git a/AGENTS.md b/AGENTS.md index 58a53f2b..3dc66c75 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -1,111 +1,67 @@ -# Repository Guidelines +# RustFS Agent Instructions (Global) -This file provides guidance for AI agents and developers working with code in this repository. +This root file keeps repository-wide rules only. +Use the nearest subdirectory `AGENTS.md` for path-specific guidance. -## Project Overview +## Rule Precedence -RustFS is a high-performance distributed object storage software built with Rust, providing S3-compatible APIs and advanced features like data lakes, AI, and big data support. It's designed as an alternative to MinIO with better performance and a more business-friendly Apache 2.0 license. +1. System/developer instructions. +2. This file (global defaults). +3. The nearest `AGENTS.md` in the current path (more specific scope wins). -## ⚠️ Pre-Commit Checklist (MANDATORY) +If repo-level instructions conflict, follow the nearest file and keep behavior aligned with CI. + +## Communication and Language + +- Respond in the same language used by the requester. +- Keep source code, comments, commit messages, and PR title/body in English. + +## Sources of Truth + +- Workspace layout and crate membership: `Cargo.toml` (`[workspace].members`) +- Local quality commands: `Makefile` and `.config/make/` +- CI quality gates: `.github/workflows/ci.yml` +- PR template: `.github/pull_request_template.md` + +Avoid duplicating long crate lists or command matrices in instruction files. +Reference the source files above instead. + +## Mandatory Before Commit + +Run and pass: -**Before EVERY commit, you MUST run and pass ALL of the following:** ```bash -cargo fmt --all --check # Code formatting -cargo clippy --all-targets --all-features -- -D warnings # Lints -cargo test --workspace --exclude e2e_test # Unit tests +make pre-commit ``` -Or simply run `make pre-commit` which covers all checks. **DO NOT commit if any check fails.** -## Communication Rules +If `make` is unavailable, run the equivalent checks defined under `.config/make/`. +Do not commit when required checks fail. -- Respond to the user in the same language used by the user. -- Code and documentation must be written in English only. -- **Pull Request titles and descriptions must be written in English** to ensure consistency and accessibility for all contributors. +## Git and PR Baseline -## Project Structure +- Use feature branches based on the latest `main`. +- Follow Conventional Commits, with subject length <= 72 characters. +- Keep PR title and description in English. +- Use `.github/pull_request_template.md` and keep all section headings. +- Use `N/A` for non-applicable template sections. +- Include verification commands in the PR description. +- When using `gh pr create`/`gh pr edit`, use `--body-file` instead of inline `--body` for multiline markdown. -The workspace root hosts shared dependencies in `Cargo.toml`. The service binary lives under `rustfs/src/main.rs`, while reusable crates sit in `crates/` (including `ecstore`, `iam`, `kms`, `madmin`, `s3select-api`, `s3select-query`, `config`, `crypto`, `lock`, `filemeta`, `rio`, `common`, `protos`, `audit-logger`, `notify`, `obs`, `workers`, `appauth`, `ahm`, `mcp`, `signer`, `checksums`, `utils`, `zip`, `targets`, and `e2e_test`). Deployment manifests are under `deploy/`, Docker assets sit at the root, and automation lives in `scripts/`. +## Security Baseline -### Core Architecture +- Never commit secrets, credentials, or key material. +- Use environment variables or vault tooling for sensitive configuration. +- For localhost-sensitive tests, verify proxy settings to avoid traffic leakage. -- **Main Binary (`rustfs/`):** Entry point at `rustfs/src/main.rs`, includes admin, auth, config, server, storage, license management, and profiling modules. -- **Key Crates:** Cargo workspace with 25+ crates supporting S3-compatible APIs, erasure coding storage, IAM, KMS, S3 Select, and observability. -- **Build System:** Custom `build-rustfs.sh` script, multi-architecture Docker builds, Make/Just task runners, cross-compilation support. +## Scoped Guidance in This Repository -## Build, Test, and Development Commands - -### Quick Commands -- `cargo check --all-targets` - Fast validation -- `cargo build --release` or `make build` - Release build -- `./build-rustfs.sh --dev` - Development build with debug symbols -- `make pre-commit` - Run all quality checks (fmt, clippy, check, test) - -### Testing -- `cargo test --workspace --exclude e2e_test` - Unit tests -- `cargo test --package e2e_test` - E2E tests -- For KMS tests: `NO_PROXY=127.0.0.1,localhost HTTP_PROXY= HTTPS_PROXY= http_proxy= https_proxy= cargo test --package e2e_test test_local_kms_end_to_end -- --nocapture --test-threads=1` - -### Cross-Platform Builds -- `./build-rustfs.sh --platform x86_64-unknown-linux-musl` - Build for musl -- `./build-rustfs.sh --platform aarch64-unknown-linux-gnu` - Build for ARM64 -- `make build-cross-all` - Build all supported architectures - -## Coding Style & Safety Requirements - -- **Formatting:** Follow `rustfmt.toml` (130-column width). Use `snake_case` for items, `PascalCase` for types, `SCREAMING_SNAKE_CASE` for constants. -- **Safety:** `unsafe_code = "deny"` enforced at workspace level. Never use `unwrap()`, `expect()`, or panic-inducing code except in tests. -- **Error Handling:** Prefer `anyhow` for applications, `thiserror` for libraries. Use proper error handling with `Result` and `Option`. -- **Async:** Keep async code non-blocking. Offload CPU-heavy work with `tokio::task::spawn_blocking` when necessary. -- **Language:** Code comments, function names, variable names, and all text in source files must be in English only. - -## Testing Guidelines - -Co-locate unit tests with their modules and give behavior-led names. Integration suites belong in each crate's `tests/` directory, while exhaustive end-to-end scenarios live in `crates/e2e_test/`. When fixing bugs or adding features, include regression tests that capture the new behavior. - -## KMS (Key Management Service) - -- **Implementation:** Complete with Local and Vault backends, auto-configures on startup with `--kms-enable` flag. -- **Encryption:** Full S3-compatible server-side encryption (SSE-S3, SSE-KMS, SSE-C). -- **Testing:** Comprehensive E2E tests in `crates/e2e_test/src/kms/`. Requires proxy bypass for local testing. -- **Key Files:** `crates/kms/`, `rustfs/src/storage/ecfs.rs`, `rustfs/src/admin/handlers/kms*.rs` - -## Environment Variables - -- Global configuration environment variables must use flat `RUSTFS_*` names (no module segments), such as `RUSTFS_REGION`, `RUSTFS_ADDRESS`, `RUSTFS_VOLUMES`, and `RUSTFS_LICENSE`. -- `RUSTFS_SCANNER_ENABLED` - Enable/disable background data scanner (default: true) -- `RUSTFS_HEAL_ENABLED` - Enable/disable auto-heal functionality (default: true) -- Deprecated aliases (for pre-beta compatibility) are documented in [the config module README](crates/config/README.md#environment-variable-naming-conventions) and must log warnings when used. -- For KMS tests: `NO_PROXY=127.0.0.1,localhost` and clear proxy environment variables - -## Commit & Pull Request Guidelines - -Work on feature branches (e.g., `feat/...`) after syncing `main`. Follow Conventional Commits under 72 characters. Each commit must compile, format cleanly, and pass `make pre-commit`. - -**Pull Request Requirements:** -- PR titles and descriptions **MUST be written in English** -- Open PRs with a concise summary, note verification commands, link relevant issues -- Follow the PR template format and fill in all required sections -- Wait for reviewer approval before merging - -### PR Template Enforcement (MANDATORY) - -- Always use `.github/pull_request_template.md` when creating or updating a PR body. -- Keep all template section headings in the final PR description (do not replace with custom sections only). -- If a section is not applicable, explicitly write `N/A` instead of removing the section. -- Before requesting review, verify the PR body still contains all template sections. -- If a PR is created without the template, immediately update the PR body to match the template before any further action. - -## Security & Configuration Tips - -Do not commit secrets or cloud credentials; prefer environment variables or vault tooling. Review IAM- and KMS-related changes with a second maintainer. Confirm proxy settings before running sensitive tests to avoid leaking traffic outside localhost. - -## Important Reminders - -- Always compile after code changes: Use `cargo build` to catch errors early -- Don't bypass tests: All functionality must be properly tested, not worked around -- Use proper error handling: Never use `unwrap()` or `expect()` in production code (except tests) -- Do what has been asked; nothing more, nothing less -- NEVER create files unless they're absolutely necessary for achieving your goal -- ALWAYS prefer editing an existing file to creating a new one -- NEVER proactively create documentation files (*.md) or README files unless explicitly requested -- NEVER commit PR description files (e.g., PR_DESCRIPTION.md): These are temporary reference files for creating pull requests and should remain local only +- `.github/AGENTS.md` +- `crates/AGENTS.md` +- `crates/config/AGENTS.md` +- `crates/ecstore/AGENTS.md` +- `crates/e2e_test/AGENTS.md` +- `crates/iam/AGENTS.md` +- `crates/kms/AGENTS.md` +- `crates/policy/AGENTS.md` +- `rustfs/src/admin/AGENTS.md` +- `rustfs/src/storage/AGENTS.md` diff --git a/crates/AGENTS.md b/crates/AGENTS.md new file mode 100644 index 00000000..aa02b510 --- /dev/null +++ b/crates/AGENTS.md @@ -0,0 +1,20 @@ +# Crates Instructions + +Applies to all paths under `crates/`. + +## Library Design + +- Treat crate code as reusable library code by default. +- Prefer `thiserror` for library-facing error types. +- Do not use `unwrap()`, `expect()`, or panic-driven control flow outside tests. + +## Testing + +- Keep unit tests close to the module they test. +- Keep integration tests under each crate's `tests/` directory. +- Add regression tests for bug fixes and behavior changes. + +## Async and Performance + +- Keep async paths non-blocking. +- Move CPU-heavy operations out of async hot paths with `tokio::task::spawn_blocking` when appropriate. diff --git a/crates/config/AGENTS.md b/crates/config/AGENTS.md new file mode 100644 index 00000000..279b3ffa --- /dev/null +++ b/crates/config/AGENTS.md @@ -0,0 +1,28 @@ +# Config Crate Instructions + +Applies to `crates/config/`. + +## Environment Variable Naming + +- Global configuration variables must use flat `RUSTFS_*` names. +- Do not introduce module-segmented names such as `RUSTFS_CONFIG_*`. + +Canonical examples: + +- `RUSTFS_REGION` +- `RUSTFS_ADDRESS` +- `RUSTFS_VOLUMES` +- `RUSTFS_LICENSE` +- `RUSTFS_SCANNER_ENABLED` +- `RUSTFS_HEAL_ENABLED` + +## Compatibility + +- Deprecated aliases must keep warning behavior. +- Document aliases in `crates/config/README.md`. +- Any alias change should include tests for both canonical and deprecated forms. + +## Source of Truth + +- Constants: `crates/config/src/constants/app.rs` +- Naming conventions: `crates/config/README.md#environment-variable-naming-conventions` diff --git a/crates/e2e_test/AGENTS.md b/crates/e2e_test/AGENTS.md new file mode 100644 index 00000000..b23c827f --- /dev/null +++ b/crates/e2e_test/AGENTS.md @@ -0,0 +1,26 @@ +# E2E Test Crate Instructions + +Applies to `crates/e2e_test/`. + +## Test Reliability + +- Keep end-to-end tests deterministic and environment-aware. +- Prefer readiness checks and explicit polling over fixed sleep-based timing assumptions. +- Ensure tests isolate resources and clean up temporary state. + +## Environment Safety + +- For local KMS-related E2E runs, keep proxy bypass settings: + - `NO_PROXY=127.0.0.1,localhost` + - clear `HTTP_PROXY` and `HTTPS_PROXY` +- Do not hardcode machine-specific paths or credentials. + +## Scope and Cost + +- Place exhaustive integration behavior here; keep unit behavior in source crates. +- Keep new E2E scenarios focused and avoid redundant overlap with existing suites. + +## Suggested Validation + +- `cargo test --package e2e_test` +- Full gate before commit: `make pre-commit` diff --git a/crates/ecstore/AGENTS.md b/crates/ecstore/AGENTS.md new file mode 100644 index 00000000..06a577cb --- /dev/null +++ b/crates/ecstore/AGENTS.md @@ -0,0 +1,28 @@ +# ECStore Crate Instructions + +Applies to `crates/ecstore/`. + +## Data Durability and Integrity + +- Do not weaken quorum checks, bitrot checks, or metadata validation paths. +- Treat any change affecting read/write/repair correctness as high risk and test accordingly. +- Prefer explicit failure over silent data corruption or implicit success. + +## Performance-Critical Paths + +- Be careful with allocations and locking in hot paths. +- Keep network and disk operations async-friendly; avoid introducing unnecessary blocking. +- Benchmark-sensitive changes should include measurable rationale. + +## Cross-Module Coordination + +- Validate behavior impacts on: + - `rustfs/src/storage/` + - `crates/filemeta/` + - `crates/heal/` + - `crates/checksums/` + +## Suggested Validation + +- `cargo test -p rustfs-ecstore` +- Full gate before commit: `make pre-commit` diff --git a/crates/iam/AGENTS.md b/crates/iam/AGENTS.md new file mode 100644 index 00000000..c9c8332f --- /dev/null +++ b/crates/iam/AGENTS.md @@ -0,0 +1,27 @@ +# IAM Crate Instructions + +Applies to `crates/iam/`. + +## Security Boundaries + +- Treat IAM changes as security-sensitive by default. +- Never log secrets, tokens, private claims, or credential material. +- Keep deny/allow evaluation behavior consistent with existing tests unless explicitly changing policy semantics. + +## Contract Stability + +- Preserve compatibility of user/group/policy attachment behavior and token claim handling. +- When changing IAM interfaces, verify impacted call sites in: + - `rustfs/src/auth.rs` + - `rustfs/src/admin/` + - `crates/policy/` + +## Error Handling + +- Return explicit errors; do not use panic-driven control flow outside tests. +- Keep error messages actionable but avoid leaking sensitive context. + +## Suggested Validation + +- `cargo test -p rustfs-iam` +- Full gate before commit: `make pre-commit` diff --git a/crates/kms/AGENTS.md b/crates/kms/AGENTS.md new file mode 100644 index 00000000..3f6cb014 --- /dev/null +++ b/crates/kms/AGENTS.md @@ -0,0 +1,27 @@ +# KMS Crate Instructions + +Applies to `crates/kms/`. + +## Change Coordination + +When changing key-management behavior, verify compatibility with: + +- `rustfs/src/storage/ecfs.rs` +- `rustfs/src/admin/handlers/kms.rs` +- `rustfs/src/admin/handlers/kms_dynamic.rs` +- `rustfs/src/admin/handlers/kms_keys.rs` +- `rustfs/src/admin/handlers/kms_management.rs` + +## Security + +- Never log plaintext keys, key material, or sensitive request payloads. +- Prefer explicit error propagation over panic paths. + +## Testing + +For local KMS end-to-end tests, keep proxy bypass settings: + +```bash +NO_PROXY=127.0.0.1,localhost HTTP_PROXY= HTTPS_PROXY= http_proxy= https_proxy= \ +cargo test --package e2e_test test_local_kms_end_to_end -- --nocapture --test-threads=1 +``` diff --git a/crates/policy/AGENTS.md b/crates/policy/AGENTS.md new file mode 100644 index 00000000..75abf24e --- /dev/null +++ b/crates/policy/AGENTS.md @@ -0,0 +1,26 @@ +# Policy Crate Instructions + +Applies to `crates/policy/`. + +## Policy Semantics + +- Keep action/resource/condition evaluation semantics stable unless compatibility changes are explicitly requested. +- Avoid introducing permissive fallbacks that may expand access unexpectedly. +- Any semantic change must include focused regression tests. + +## Integration Awareness + +- Validate compatibility with IAM and request-auth call sites: + - `crates/iam/` + - `rustfs/src/auth.rs` + - `rustfs/src/storage/access.rs` + +## Error and Observability + +- Prefer explicit, typed errors over implicit fallback behavior. +- Log policy evaluation diagnostics without exposing sensitive credential content. + +## Suggested Validation + +- `cargo test -p rustfs-policy` +- Full gate before commit: `make pre-commit` diff --git a/rustfs/src/admin/AGENTS.md b/rustfs/src/admin/AGENTS.md new file mode 100644 index 00000000..4564baa2 --- /dev/null +++ b/rustfs/src/admin/AGENTS.md @@ -0,0 +1,28 @@ +# Admin Module Instructions + +Applies to `rustfs/src/admin/`. + +## Auth and Authorization + +- Do not bypass authentication or authorization checks for admin routes. +- Keep admin permission checks aligned with IAM and policy evaluation behavior. +- Any change to security-sensitive route behavior must include tests. + +## API and Routing Stability + +- Preserve existing admin endpoint contracts unless a deliberate compatibility change is requested. +- Update route registration and handler tests when adding or changing endpoints. +- Coordinate cross-module changes with: + - `crates/iam/` + - `crates/policy/` + - `crates/kms/` + +## Operational Safety + +- Avoid exposing secrets or sensitive internals in admin responses and logs. +- Keep health/readiness/admin status responses deterministic and machine-readable. + +## Suggested Validation + +- Admin handler and routing tests under `rustfs/src/admin/` +- Full gate before commit: `make pre-commit` diff --git a/rustfs/src/storage/AGENTS.md b/rustfs/src/storage/AGENTS.md new file mode 100644 index 00000000..7817bad2 --- /dev/null +++ b/rustfs/src/storage/AGENTS.md @@ -0,0 +1,28 @@ +# Storage Module Instructions + +Applies to `rustfs/src/storage/`. + +## Correctness First + +- Preserve object data integrity checks, versioning behavior, and S3-compatible response semantics. +- Do not weaken validation paths for range reads, multipart behavior, or metadata consistency. +- If behavior changes, add regression tests near the affected storage module. + +## Security and Encryption + +- Never log plaintext customer keys, decrypted key material, or sensitive SSE request payloads. +- Keep SSE behavior aligned with KMS and admin handler integrations. +- Coordinate related changes with: + - `crates/kms/` + - `rustfs/src/admin/handlers/kms*.rs` + - `crates/ecstore/` + +## Performance and Async + +- Keep I/O paths non-blocking where possible. +- Avoid adding extra allocations or blocking work in hot object read/write paths without clear justification. + +## Suggested Validation + +- Targeted module tests in `rustfs/src/storage/*_test.rs` +- Full gate before commit: `make pre-commit`