Commit Graph

1855 Commits

Author SHA1 Message Date
shiro.lee
f049c656d9 fix: list_objects does not return common_prefixes field. (#543) (#554)
Co-authored-by: loverustfs <155562731+loverustfs@users.noreply.github.com>
2025-09-18 07:27:37 +08:00
majinghe
65dd947350 add tls support for docker compose (#553)
* add tls support for docker compose

* update docker compose file with comment
2025-09-17 22:45:23 +08:00
0xdx2
57f082ee2b fix: enforce max-keys limit to 1000 in S3 implementation (#549)
Co-authored-by: damon <damonxue2@gmail.com>
1.0.0-alpha.57
2025-09-16 18:02:24 +08:00
weisd
ae7e86d7ef refactor: simplify initialization flow and modernize string formatting (#548) 2025-09-16 15:44:50 +08:00
houseme
a12a3bedc3 feat(obs): optimize WriteMode selection logic in init_telemetry (#546)
- Refactor WriteMode selection to ensure all variables moved into thread closures are owned types, preventing lifetime issues.
- Simplify and clarify WriteMode assignment for production and non-production environments.
- Improve code readability and maintainability for logger initialization.
2025-09-16 08:25:37 +08:00
Copilot
cafec06b7e [Optimization] Enhance obs module telemetry.rs with environment-aware logging and production security (#539)
* Initial plan

* Implement environment-aware logging with production stdout auto-disable

Co-authored-by: houseme <4829346+houseme@users.noreply.github.com>

* add mimalloc crate

* fix

* improve code

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: houseme <4829346+houseme@users.noreply.github.com>
Co-authored-by: houseme <housemecn@gmail.com>
Co-authored-by: loverustfs <155562731+loverustfs@users.noreply.github.com>
2025-09-15 14:52:20 +08:00
Parm Gill
1770679e66 Adding a toggle for update check (#532) 2025-09-14 22:26:48 +08:00
jon
a4fbf596e6 add startup logo (#528)
* add startup logo

* Replace logo ASCII art in main.rs

---------

Co-authored-by: houseme <housemecn@gmail.com>
Co-authored-by: 安正超 <anzhengchao@gmail.com>
2025-09-14 12:04:00 +08:00
houseme
3f717292bf feat(console): support accessing console endpoint via port 9000 (#533)
* fix(main): update dns_init startup logic and remove unused code in http.rs

- Refactored the DNS resolver initialization logic in `main.rs` for improved startup reliability and error handling.
- Removed unused code from `http.rs` to keep the codebase clean and maintainable.

* feat(console): support accessing console endpoint via port 9000

- Added compatibility to allow console access through port 9000.
- Improved endpoint detection and routing for console service on standard and custom ports.
- Enhanced user experience for environments using port 9000 as the default access point.
1.0.0-alpha.56
2025-09-14 01:14:14 +08:00
houseme
73f0ecbf8f fix(main): update dns_init startup logic and remove unused code in http.rs (#531)
- Refactored the DNS resolver initialization logic in `main.rs` for improved startup reliability and error handling.
- Removed unused code from `http.rs` to keep the codebase clean and maintainable.
2025-09-13 23:43:25 +08:00
houseme
0c3079ae5e remove deps (#529) 2025-09-13 21:06:40 +08:00
majinghe
ebf30b0db5 update docker compose usage guidence in READEME file (#526) 2025-09-13 16:24:36 +08:00
Copilot
29c004d935 feat: enhance console separation with enterprise-grade security, monitoring, and advanced tower-http integration (#513)
* Initial plan

* feat: implement console service separation from endpoint

Co-authored-by: houseme <4829346+houseme@users.noreply.github.com>

* feat: add console separation documentation and tests

Co-authored-by: houseme <4829346+houseme@users.noreply.github.com>

* feat: enhance console separation with configurable CORS and improved Docker support

Co-authored-by: houseme <4829346+houseme@users.noreply.github.com>

* feat: implement enhanced console separation with security hardening and monitoring

Co-authored-by: houseme <4829346+houseme@users.noreply.github.com>

* refactor: implement console TLS following endpoint logic and improve configuration

Co-authored-by: houseme <4829346+houseme@users.noreply.github.com>

* add tower-http feature "timeout|limit"

* add dependencies crates `axum-server`

* refactor: reconstruct console server with enhanced tower-http features and environment variables

Co-authored-by: houseme <4829346+houseme@users.noreply.github.com>

* upgrade dep

* improve code for dns and console port `:9001`

* improve code

* fix

* docs: comprehensive improvement of console separation documentation and Docker deployment standards

Co-authored-by: houseme <4829346+houseme@users.noreply.github.com>

* fmt

* add logs

* improve code for Config handler

* remove logs

* fix

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: houseme <4829346+houseme@users.noreply.github.com>
Co-authored-by: houseme <housemecn@gmail.com>
1.0.0-alpha.55
2025-09-13 14:48:14 +08:00
majinghe
4595bf7db6 fix docker compose running with no such file error (#519)
* fix docker compose running with no such file error

* fix observability docker compose
1.0.0-alpha.54
2025-09-13 13:04:06 +08:00
guojidan
f372ccf4a8 disable pprof on win (#524)
Signed-off-by: junxiang Mu <1948535941@qq.com>
1.0.0-alpha.53
2025-09-12 18:43:45 +08:00
guojidan
9ce867f585 feat(lock): Optimize lock management performance in high-concurrency scenarios (#523)
Increase the size of the notification pool to reduce the thundering herd effect under high concurrency
Implement an adaptive timeout mechanism that dynamically adjusts based on system load and priority
Add a lock protection mechanism to prevent premature cleanup of active locks
Add lock acquisition methods for high-priority and critical-priority locks
Improve the cleanup strategy to be more conservative under high load
Add detailed debug logs to assist in diagnosing lock issues

Signed-off-by: junxiang Mu <1948535941@qq.com>
2025-09-12 18:17:07 +08:00
guojidan
124c31a68b refactor(profiling): Remove performance profiling support for Windows and optimize dependency management (#518)
Remove the pprof performance profiling functionality on the Windows platform, as this platform does not support the relevant features
Move the pprof dependency to the platform-specific configuration for non-Windows systems
Update the performance profiling endpoint handling logic to distinguish between platform support statuses
Add the CLAUDE.md document to explain project build and architecture information

Signed-off-by: RustFS Developer <dandan@rustfs.com>
Co-authored-by: RustFS Developer <dandan@rustfs.com>
2025-09-12 09:11:44 +08:00
guojidan
62a01f3801 Performance: improve (#514)
* Performance: improve

Signed-off-by: junxiang Mu <1948535941@qq.com>

* remove dirty

Signed-off-by: junxiang Mu <1948535941@qq.com>

* fix some err

Signed-off-by: junxiang Mu <1948535941@qq.com>

---------

Signed-off-by: junxiang Mu <1948535941@qq.com>
2025-09-11 19:48:28 +08:00
weisd
70e6bec2a4 feat:admin auth (#512)
* feat:admin auth

* fix:#509
1.0.0-alpha.52
2025-09-11 16:49:07 +08:00
guojidan
cf863ba059 feat(lock): Add support for disabling lock manager (#511)
* feat(lock): Add support for disabling lock manager
Implement control of lock system activation and deactivation via environment variables
Add DisabledLockManager for lock-free operation scenarios
Introduce LockManager trait to uniformly manage different lock managers

Signed-off-by: junxiang Mu <1948535941@qq.com>

* refactor(lock): Optimize implementation of global lock manager and parsing of boolean environment variables
Refactor the implementation of the global lock manager: wrap FastObjectLockManager with Arc and add the as_fast_lock_manager method
Extract the boolean environment variable parsing logic into an independent function parse_bool_env_var

Signed-off-by: junxiang Mu <1948535941@qq.com>

---------

Signed-off-by: junxiang Mu <1948535941@qq.com>
2025-09-11 13:46:06 +08:00
guojidan
d4beb1cc0b Fix lock (#510)
* Refactor: reimplement lock

Signed-off-by: junxiang Mu <1948535941@qq.com>

* Fix: fix test case failed

Signed-off-by: junxiang Mu <1948535941@qq.com>

* Improve: lock pref

Signed-off-by: junxiang Mu <1948535941@qq.com>

* fix(lock): Fix resource cleanup issue when batch lock acquisition fails
Ensure that the locks already acquired are properly released when batch lock acquisition fails to avoid memory leaks
Improve the lock protection mechanism to prevent double release issues
Add complete Apache license declarations to all files

Signed-off-by: junxiang Mu <1948535941@qq.com>

---------

Signed-off-by: junxiang Mu <1948535941@qq.com>
2025-09-11 12:10:35 +08:00
0xdx2
971e74281c fix:Fix some errors tested in mint (#507)
* refactor: replace new_object_layer_fn with get_validated_store for bucket validation

* feat: add validation for object tagging limits and uniqueness

* Apply suggestion from @Copilot

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Apply suggestion from @Copilot

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* feat: add EntityTooSmall error for multipart uploads and update error handling

* feat: validate max_parts input range for S3 multipart uploads

* Update rustfs/src/storage/ecfs.rs

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* fix: optimize tag key and value length validation checks

---------

Co-authored-by: damon <damonxue2@gmail.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2025-09-10 22:22:29 +08:00
Copilot
ca9a2b6ab9 feat: Implement enhanced DNS resolver with hickory-resolver, TLS support, and layered fallback for Kubernetes environments (#505)
* Initial plan

* feat: Implement layered DNS resolver with caching and validation

Co-authored-by: houseme <4829346+houseme@users.noreply.github.com>

* feat: Integrate DNS resolver into main application and fix formatting

Co-authored-by: houseme <4829346+houseme@users.noreply.github.com>

* feat: Implement enhanced DNS resolver with Moka cache and layered fallback

Co-authored-by: houseme <4829346+houseme@users.noreply.github.com>

* feat: Implement hickory-resolver with TLS support for enhanced DNS resolution

Co-authored-by: houseme <4829346+houseme@users.noreply.github.com>

* upgrade

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: houseme <4829346+houseme@users.noreply.github.com>
Co-authored-by: houseme <housemecn@gmail.com>
2025-09-10 21:16:33 +08:00
houseme
4e00110bfe add bucket notification configuration (#502) 2025-09-10 00:56:27 +08:00
安正超
9c97524c3b feat: consolidate AI rules into unified AGENTS.md (#501)
- Merge all AI rules from .rules.md, .cursorrules, and CLAUDE.md into AGENTS.md
- Add competitor keyword prohibition rules (minio, ceph, swift, etc.)
- Simplify rules by removing overly detailed code examples
- Integrate new development principles as highest priority
- Remove old tool-specific rule files
- Fix clippy warnings for format string improvements
2025-09-09 21:36:34 +08:00
guojidan
14a8802ce7 Fix: fix collect usage data (#500)
Signed-off-by: junxiang Mu <1948535941@qq.com>
2025-09-09 18:39:51 +08:00
guojidan
9d5ed1acac Feature/scanner performance optimization (#498)
* Refactor: reimplement scanner

Signed-off-by: RustFS Developer <dandan@rustfs.com>

* comment lock

Signed-off-by: junxiang Mu <1948535941@qq.com>

* remove dirty file

Signed-off-by: junxiang Mu <1948535941@qq.com>

* Fix: fix rebase

* fix(scanner): Improve error handling and logging

Signed-off-by: junxiang Mu <1948535941@qq.com>

---------

Signed-off-by: RustFS Developer <dandan@rustfs.com>
Signed-off-by: junxiang Mu <1948535941@qq.com>
Co-authored-by: RustFS Developer <dandan@rustfs.com>
1.0.0-alpha.51
2025-09-08 18:35:45 +08:00
0xdx2
44f3eb7244 Fix: add support for additional AWS S3 storage classes and validation logic (#487)
* Fix: add pagination fields to S3 response

* Fix: add support for additional AWS S3 storage classes and validation logic

* Fix: improve handling of optional fields in S3 response

---------

Co-authored-by: DamonXue <damonxue2@gmail.com>
1.0.0-alpha.50
2025-09-05 09:50:41 +08:00
weisd
01b2623f66 Fix/response (#485)
* fix:list_parts response

* fix:list_objects skip delete_marker
2025-09-03 17:52:31 +08:00
dependabot[bot]
cf4d63795f build(deps): bump crc-fast from 1.4.0 to 1.5.0 in the dependencies group (#481)
Bumps the dependencies group with 1 update: [crc-fast](https://github.com/awesomized/crc-fast-rust).


Updates `crc-fast` from 1.4.0 to 1.5.0
- [Release notes](https://github.com/awesomized/crc-fast-rust/releases)
- [Changelog](https://github.com/awesomized/crc-fast-rust/blob/main/CHANGELOG.md)
- [Commits](https://github.com/awesomized/crc-fast-rust/compare/1.4.0...1.5.0)

---
updated-dependencies:
- dependency-name: crc-fast
  dependency-version: 1.5.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: dependencies
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: weisd <im@weisd.in>
2025-09-03 17:30:08 +08:00
WenTao
0efc818635 Fix Windows path separator issue using PathBuf (#482)
* Update mod.rs

The following code uses a separator that is not compatible with Windows:

format!("{}/{}", file_config.path.clone(), rustfs_config::DEFAULT_SINK_FILE_LOG_FILE)


Change it to the following code:


std::path::Path::new(&file_config.path)
    .join(rustfs_config::DEFAULT_SINK_FILE_LOG_FILE)
    .to_string_lossy()
    .to_string()

* Replaced format! macro with PathBuf::join to fix path separator issue on Windows.Tested on Windows 10 with Rust 1.85.0, paths now correctly use \ separator.
2025-09-03 15:25:08 +08:00
weisd
c9d26c6e88 Fix/delete version (#484)
* fix:delete_version

* fix:test_lifecycle_expiry_basic

---------

Co-authored-by: likewu <likewu@126.com>
2025-09-03 15:12:58 +08:00
likewu
087df484a3 Fix/ilm (#478) 2025-09-02 18:18:26 +08:00
houseme
04bf4b0f98 feat: add S3 object legal hold and retention management APIs (#476)
* add bucket rule

* translation

* improve code for event notice add rule
1.0.0-alpha.49
2025-09-02 00:14:10 +08:00
likewu
7462be983a Feature up/ilm (#470)
* fix delete-marker expiration. add api_restore.

* time retry object upload

* lock file

* make fmt

* restore object

* serde-rs-xml -> quick-xml

* scanner_item prefix object_name

* object_path

* object_name

* fi version_purge_status

* old_dir None

Co-authored-by: houseme <housemecn@gmail.com>
2025-09-01 16:11:28 +08:00
houseme
5264503e47 build(deps): bump aws-config and clap upgrade version (#472) 2025-08-30 20:30:46 +08:00
dependabot[bot]
3b8cb0df41 build(deps): bump tracing-subscriber in the cargo group (#471)
Bumps the cargo group with 1 update: [tracing-subscriber](https://github.com/tokio-rs/tracing).


Updates `tracing-subscriber` from 0.3.19 to 0.3.20
- [Release notes](https://github.com/tokio-rs/tracing/releases)
- [Commits](https://github.com/tokio-rs/tracing/compare/tracing-subscriber-0.3.19...tracing-subscriber-0.3.20)

---
updated-dependencies:
- dependency-name: tracing-subscriber
  dependency-version: 0.3.20
  dependency-type: direct:production
  dependency-group: cargo
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-08-30 19:02:26 +08:00
houseme
9aebef31ff refactor(admin/event): optimize notification target routing and logic handling (#463)
* add

* fix

* add target arns list

* improve code for arns

* upgrade crates version

* fix

* improve import code mod.rs

* fix

* improve

* improve code

* improve code

* fix

* fmt
1.0.0-alpha.48
2025-08-27 09:39:25 +08:00
zzhpro
c2d782bed1 feat: support conditional writes (#409)
* feat: support conditional writes

* refactor: avoid using unwrap

* fix: obtain lock before check in CompleteMultiPartUpload

* refactor: do not obtain a lock when getting object meta

* fix: avoid using unwrap and modifying incoming arguments

* test: add e2e tests for conditional writes

---------

Co-authored-by: guojidan <63799833+guojidan@users.noreply.github.com>
Co-authored-by: 安正超 <anzhengchao@gmail.com>
Co-authored-by: loverustfs <155562731+loverustfs@users.noreply.github.com>
1.0.0-alpha.47
2025-08-25 18:35:24 -07:00
likewu
e00f5be746 Fix/addtier (#454)
* fix retry

* fmt

* fix

* fix

* fix

---------

Co-authored-by: loverustfs <155562731+loverustfs@users.noreply.github.com>
2025-08-25 10:24:48 +08:00
shiro.lee
e23297f695 fix: add the default port number to the given server domains (#373) (#458) 2025-08-25 07:49:36 +08:00
0xdx2
d6840a6e04 feat: add support for range requests in upload_part_copy and implement parse_copy_source_range function (#453)
* feat: add support for range requests in upload_part_copy and implement parse_copy_source_range function

* style: format debug and error logging for improved readability

* feat: implement parse_copy_source_range function and improve error handling in range requests

* Update rustfs/src/storage/ecfs.rs

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* fix: correct return type in parse_copy_source_range function

* fix: remove unnecessary unwrap in parse_copy_source_range tests

* fix: simplify etag comparison in copy condition validation

---------

Co-authored-by: DamonXue <damonxue2@gmail.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: loverustfs <155562731+loverustfs@users.noreply.github.com>
2025-08-24 10:54:48 +08:00
houseme
3557a52dc4 Potential fix for code scanning alert no. 7: Workflow does not contain permissions (#457)
Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
2025-08-24 10:10:04 +08:00
houseme
fd2aab2bd9 fix:revet #443 #446 (#452)
* fix: revet #443 #446

* fix
2025-08-23 17:30:06 +08:00
houseme
f1c50fcb74 fix:Workflow does not contain permissions (#451) 2025-08-23 12:35:23 +08:00
houseme
bdcba3460e Potential fix for code scanning alert no. 13: Code injection (#447)
Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
Co-authored-by: 安正超 <anzhengchao@gmail.com>
2025-08-23 10:05:00 +08:00
houseme
8857f31b07 Comment out error log for missing subscribers (#448) 2025-08-22 21:15:46 +08:00
loverustfs
5b85bf7a00 lock: dedicate unlock worker to thread runtime; robust fallback in Drop (#446)
* lock: dedicate unlock worker to thread runtime; robust fallback in Drop

* Update crates/lock/src/guard.rs

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update crates/lock/src/guard.rs

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update crates/lock/src/guard.rs

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Refactor logging in UNLOCK_TX error handling

Removed redundant logging of lock_id in warning message.

---------

Co-authored-by: houseme <housemecn@gmail.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
1.0.0-alpha.46
2025-08-22 16:51:56 +08:00
loverustfs
46bd75c0f8 ahm(scanner): throttle scanning, skip recently-modified objects, and … (#443)
* ahm(scanner): throttle scanning, skip recently-modified objects, and gate missing-object heals to deep mode; adjust conservative defaults

Signed-off-by: loverustfs <hello@rustfs.com>

* ecstore: enable virtual-host AUTO heuristics and URL building; signer: fix SigV2 canonical resource for vhost; add unit tests

* ecstore: AUTO virtual-host style URL selection; signer: SigV2 canonical resource fixes for vhost; tests added.\nahm: fix clippy drop_non_drop; integration tests robust to existing bucket; ignore flaky lifecycle test.\nMakefile: test target falls back to cargo test when nextest missing.\npre-commit: all checks green.

---------

Signed-off-by: loverustfs <hello@rustfs.com>
2025-08-22 16:03:29 +08:00
houseme
5fc5dd0fd9 Remove rustfs-gui module (#445)
This commit completely removes the rustfs-gui module from the project. The deletion includes:

- All source code files (*.rs) and associated resources
- GUI-specific dependencies from Cargo.toml
- Build scripts and configuration files specific to the GUI module
- Documentation and assets related to the graphical interface

The removal is performed because:
- The GUI component is no longer maintained
- Focus is shifting to core functionality and CLI interface
- Limited resources available for GUI development and maintenance

The core filesystem functionality remains available through the rustfs library and CLI interface.
2025-08-22 09:15:22 +08:00