59 Commits

Author SHA1 Message Date
yxrxy
d3cff7d033 feat(webdav): add WebDAV protocol gateway (#2158)
Signed-off-by: yxrxy <1532529704@qq.com>
Co-authored-by: houseme <housemecn@gmail.com>
Co-authored-by: 马登山 <Cxymds@qq.com>
Co-authored-by: heihutu <30542132+heihutu@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: 安正超 <anzhengchao@gmail.com>
2026-03-14 23:06:53 +08:00
安正超
8c4735ff88 docs: scope AGENTS instructions by directory (#2083) 2026-03-05 17:25:37 +08:00
GatewayJ
2cb8db36a5 fix(iam): user group policy and delete group (fixes #2028) (#2043)
Co-authored-by: GatewayJ <8352692332qq.com>
2026-03-02 14:19:01 +08:00
GatewayJ
ba32fd9d96 fix(s3): allow anonymous access when PublicAccessBlock config is miss… (#2039)
Co-authored-by: GatewayJ <8352692332qq.com>
2026-03-02 11:37:00 +08:00
唐小鸭
568c07ced9 fix: implement handling for "aws-chunked" Content-Encoding (#2009)
Co-authored-by: 安正超 <anzhengchao@gmail.com>
2026-03-01 01:22:12 +08:00
houseme
d17d2083d4 feat(targets): enhance webhook TLS support with custom CA and skip-verify (#1994)
Co-authored-by: Copilot <198982749+Copilot@users.noreply.github.com>
Co-authored-by: houseme <4829346+houseme@users.noreply.github.com>
Co-authored-by: heihutu <heihutu@gmail.com>
2026-02-27 21:24:49 +08:00
Niraj Yadav
8f00d1fbb0 feat(admin): implement handler for delete group (#1901)
Signed-off-by: Niraj Yadav <niryadav@redhat.com>
Signed-off-by: heihutu <30542132+heihutu@users.noreply.github.com>
Co-authored-by: heihutu <30542132+heihutu@users.noreply.github.com>
Co-authored-by: yxrxy <yxrxytrigger@gmail.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: 安正超 <anzhengchao@gmail.com>
2026-02-24 13:00:46 +08:00
loverustfs
5d737eaeb7 fix(ecstore): invalidate GlobalFileCache after write_all_private to fix DeleteMarker visibility (#1890) 2026-02-20 22:47:45 +08:00
loverustfs
cf633569a2 fix: remove duplicate common prefixes for slash delimiter (#1797) (#1841)
Signed-off-by: houseme <housemecn@gmail.com>
Co-authored-by: houseme <housemecn@gmail.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: heihutu <30542132+heihutu@users.noreply.github.com>
Co-authored-by: Copilot <198982749+Copilot@users.noreply.github.com>
Co-authored-by: houseme <4829346+houseme@users.noreply.github.com>
2026-02-17 21:36:31 +08:00
yxrxy
da58f8e291 fix: Allow non-admin users to read bucket quota configuration. (#1759)
Co-authored-by: houseme <housemecn@gmail.com>
Co-authored-by: heihutu <30542132+heihutu@users.noreply.github.com>
2026-02-10 11:23:21 +08:00
houseme
0b870d6301 build(deps): bump the dependencies group with 19 updates (#1745) 2026-02-07 12:22:14 +08:00
yxrxy
5c2eda356e feat: migrate FTP/SFTP to protocols crate and update dependencies (#1580)
Signed-off-by: yxrxy <yxrxytrigger@gmail.com>
Signed-off-by: houseme <housemecn@gmail.com>
Signed-off-by: heihutu <30542132+heihutu@users.noreply.github.com>
Co-authored-by: houseme <housemecn@gmail.com>
Co-authored-by: heihutu <30542132+heihutu@users.noreply.github.com>
Co-authored-by: loverustfs <hello@rustfs.com>
2026-02-06 20:58:42 +08:00
LeonWang0735
a2b88a79ec test(e2e_test): add automated cluster environment for conditional PUT race test (#1673)
Co-authored-by: houseme <housemecn@gmail.com>
Co-authored-by: loverustfs <hello@rustfs.com>
2026-02-06 15:01:59 +08:00
loverustfs
a4563f7b41 fix: return null versionId when suspended (#1066) (#1709) 2026-02-04 09:22:52 +08:00
LeonWang0735
ec4458f846 Fix/correctly handle terraform s3 backend with versioned bucket (#1686)
Co-authored-by: loverustfs <hello@rustfs.com>
2026-02-02 12:43:07 +08:00
houseme
90ed75a3dc refactor(utils/os): Optimize Windows OS utilities and add safety comments (#1671)
Co-authored-by: weisd <weishidavip@163.com>
2026-01-30 16:13:42 +08:00
weisd
dce117840c refactor: NamespaceLock (nslock), AHM→Heal Crate, and Lock/Clippy Fixes (#1664)
Co-authored-by: Copilot <198982749+Copilot@users.noreply.github.com>
Co-authored-by: weisd <2057561+weisd@users.noreply.github.com>
Co-authored-by: houseme <housemecn@gmail.com>
2026-01-30 13:13:41 +08:00
安正超
4890fb25c1 fix: listobjects v2 pagination (#1607) 2026-01-25 09:30:08 +08:00
GatewayJ
9285acba06 feat: object retention (#1589) 2026-01-24 22:12:45 +08:00
LeonWang0735
db5e72e475 Fix/correctly handle compression (#1594) 2026-01-24 10:55:10 +08:00
yxrxy
29d86036b1 feat: implement bucket quota system (#1461)
Signed-off-by: yxrxy <1532529704@qq.com>
Co-authored-by: loverustfs <hello@rustfs.com>
2026-01-12 11:42:07 +08:00
loverustfs
703d961168 fix: honor bucket policy for authenticated users (#1460)
Co-authored-by: GatewayJ <835269233@qq.com>
2026-01-10 20:01:28 +08:00
weisd
5660208e89 Refactor RPC Authentication System for Improved Maintainability (#1391) 2026-01-05 19:51:51 +08:00
dependabot[bot]
1d6c8750e7 build(deps): bump the dependencies group with 2 updates (#1383)
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: houseme <housemecn@gmail.com>
2026-01-05 15:33:57 +08:00
yxrxy
38c2d74d36 fix: fix FTPS/SFTP download issues and optimize S3Client caching (#1353)
Co-authored-by: houseme <housemecn@gmail.com>
Co-authored-by: loverustfs <hello@rustfs.com>
2026-01-04 17:28:18 +08:00
yxrxy
cf53a9d84a chore: replace native-tls with pure rustls for FTPS/SFTP e2e tests (#1334)
Signed-off-by: yxrxy <1532529704@qq.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2026-01-02 11:08:28 +08:00
houseme
8d7cd4cb1b chore: upgrade dependencies and migrate to aws-lc-rs (#1333) 2026-01-02 00:02:34 +08:00
yxrxy
b8aa8214e2 Feat/ftps&sftp (#1308)
[feat] ftp / sftp
2025-12-31 09:01:15 +08:00
loverustfs
e3a0a07495 fix: ensure version_id is returned in S3 response headers (#1272)
Co-authored-by: houseme <housemecn@gmail.com>
Co-authored-by: 安正超 <anzhengchao@gmail.com>
2025-12-28 09:41:32 +08:00
loverustfs
af5c0b13ef fix: HeadObject returns 404 for deleted objects with versioning enabled (#1229)
Co-authored-by: houseme <housemecn@gmail.com>
2025-12-22 20:43:00 +08:00
yxrxy
352035a06f feat: Implement AWS policy variables support (#1131)
Co-authored-by: houseme <housemecn@gmail.com>
Co-authored-by: loverustfs <hello@rustfs.com>
2025-12-16 13:32:01 +08:00
yihong
67095c05f9 fix: update tool chain make everything happy (#1134)
Signed-off-by: yihong0618 <zouzou0208@gmail.com>
2025-12-13 20:32:42 +08:00
guojidan
fba201df3d fix: harden data usage aggregation and cache handling (#1102)
Signed-off-by: junxiang Mu <1948535941@qq.com>
Co-authored-by: loverustfs <hello@rustfs.com>
2025-12-11 09:55:25 +08:00
Jörg Thalheim
2c86fe30ec Content encoding (#1089)
Signed-off-by: Jörg Thalheim <joerg@thalheim.io>
Co-authored-by: loverustfs <hello@rustfs.com>
2025-12-10 15:21:51 +08:00
Copilot
20961d7c91 Add comprehensive special character handling with validation refactoring and extensive test coverage (#1078)
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: houseme <4829346+houseme@users.noreply.github.com>
Co-authored-by: houseme <housemecn@gmail.com>
2025-12-09 13:40:29 +08:00
houseme
069194f553 Fix/getobjectlength (#920)
* fix getobject content length resp

* Fix regression in exception handling for non-existent key with enhanced compression predicate and metadata improvements (#915)

* Initial plan

* Fix GetObject regression by excluding error responses from compression

The issue was that CompressionLayer was attempting to compress error responses,
which could cause Content-Length header mismatches. By excluding 4xx and 5xx
responses from compression, we ensure error responses (like NoSuchKey) are sent
correctly without body truncation.

Co-authored-by: houseme <4829346+houseme@users.noreply.github.com>

* Enhance NoSuchKey fix with improved compression predicate and comprehensive tests

- Enhanced ShouldCompress predicate with size-based exclusion (< 256 bytes)
- Added detailed documentation explaining the compression logic
- Added debug logging for better observability
- Created comprehensive test suite with 4 test cases:
  - test_get_deleted_object_returns_nosuchkey
  - test_head_deleted_object_returns_nosuchkey
  - test_get_nonexistent_object_returns_nosuchkey
  - test_multiple_gets_deleted_object
- Added extensive inline documentation and comments
- Created docs/fix-nosuchkey-regression.md with full analysis

Co-authored-by: houseme <4829346+houseme@users.noreply.github.com>

* Add compression best practices documentation

Added comprehensive guide covering:
- Best practices for HTTP response compression
- Common pitfalls and solutions
- Performance considerations and trade-offs
- Testing guidelines and examples
- Monitoring and alerting recommendations
- Migration guide for existing services

Co-authored-by: houseme <4829346+houseme@users.noreply.github.com>

* fix

* fmt

* fmt

* Fix/objectdelete (#917)

* fix getobject content length resp

* fix delete object

---------

Co-authored-by: houseme <housemecn@gmail.com>

* Add comprehensive analysis of NoSuchKey fix and related improvements

Created detailed documentation analyzing:
- HTTP compression layer fix (primary issue)
- Content-length calculation fix from PR #917
- Delete object metadata fixes from PR #917
- How all components work together
- Complete scenario walkthrough
- Performance impact analysis
- Testing strategy and deployment checklist

This ties together all the changes in the PR branch including the merged
improvements from PR #917.

Co-authored-by: houseme <4829346+houseme@users.noreply.github.com>

* replace `once_cell` to `std`

* fmt

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: houseme <4829346+houseme@users.noreply.github.com>
Co-authored-by: houseme <housemecn@gmail.com>
Co-authored-by: weisd <im@weisd.in>

* fmt

---------

Co-authored-by: weisd <weishidavip@163.com>
Co-authored-by: Copilot <198982749+Copilot@users.noreply.github.com>
Co-authored-by: houseme <4829346+houseme@users.noreply.github.com>
Co-authored-by: weisd <im@weisd.in>
2025-11-24 18:56:34 +08:00
安正超
dd47fcf2a8 fix: restore localized samples in tests (#749)
* fix: restore required localized examples

* style: fix formatting issues
2025-10-29 13:16:31 +08:00
houseme
c5264f9703 improve code for metrics and switch tokio-tar to astral-tokio-tar (#705)
* improve code for metrics and switch tokio-tar to astral-tokio-tar

* remove log

* fix
2025-10-24 13:07:56 +08:00
houseme
42d3645d6f fix(targets): make target removal and reload transactional; prevent reappearing entries (#662)
* feat: improve code for notify

* upgrade starshard version

* upgrade version

* Fix ETag format to comply with HTTP standards by wrapping with quotes (#592)

* Initial plan

* Fix ETag format to comply with HTTP standards by wrapping with quotes

Co-authored-by: overtrue <1472352+overtrue@users.noreply.github.com>

* bufigx

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: overtrue <1472352+overtrue@users.noreply.github.com>
Co-authored-by: overtrue <anzhengchao@gmail.com>

* Improve lock (#596)

* improve lock

Signed-off-by: Mu junxiang <1948535941@qq.com>

* feat(tests): add wait_for_object_absence helper and improve lifecycle test reliability

Signed-off-by: Mu junxiang <1948535941@qq.com>

* chore: remove dirty docs

Signed-off-by: Mu junxiang <1948535941@qq.com>

---------

Signed-off-by: Mu junxiang <1948535941@qq.com>

* feat(append): implement object append operations with state tracking (#599)

* feat(append): implement object append operations with state tracking

Signed-off-by: junxiang Mu <1948535941@qq.com>

* chore: rebase

Signed-off-by: junxiang Mu <1948535941@qq.com>

---------

Signed-off-by: junxiang Mu <1948535941@qq.com>

* build(deps): upgrade s3s (#595)

Co-authored-by: loverustfs <155562731+loverustfs@users.noreply.github.com>

* fix: validate mqtt broker

* improve code for `import`

* fix

* improve

* remove logger from `rustfs-obs` crate

* remove code for config Observability

* fix

* improve code

* fix comment

* up

* up

* upgrade version

* fix

* fmt

* upgrade tokio version to 1.48.0

* upgrade `datafusion` and `reed-solomon-simd` version

* fix

* fmt

* improve code for notify webhook example

* improve code

* fix

* fix

* fmt

---------

Signed-off-by: Mu junxiang <1948535941@qq.com>
Signed-off-by: junxiang Mu <1948535941@qq.com>
Co-authored-by: Copilot <198982749+Copilot@users.noreply.github.com>
Co-authored-by: overtrue <1472352+overtrue@users.noreply.github.com>
Co-authored-by: overtrue <anzhengchao@gmail.com>
Co-authored-by: guojidan <63799833+guojidan@users.noreply.github.com>
Co-authored-by: Nugine <nugine@foxmail.com>
Co-authored-by: loverustfs <155562731+loverustfs@users.noreply.github.com>
2025-10-17 15:34:53 +08:00
安正超
639bf0c233 Revert "feat(append): implement object append operations with state tracking (#599)" (#646)
This reverts commit 4f73760a45.
2025-10-12 23:47:51 +08:00
Nugine
626c7ed34a fix: CompleteMultipartUpload encryption (#626) 2025-10-08 20:27:40 +08:00
guojidan
4f73760a45 feat(append): implement object append operations with state tracking (#599)
* feat(append): implement object append operations with state tracking

Signed-off-by: junxiang Mu <1948535941@qq.com>

* chore: rebase

Signed-off-by: junxiang Mu <1948535941@qq.com>

---------

Signed-off-by: junxiang Mu <1948535941@qq.com>
2025-09-27 20:06:26 -07:00
guojidan
be66cf8bd3 Improve lock (#596)
* improve lock

Signed-off-by: Mu junxiang <1948535941@qq.com>

* feat(tests): add wait_for_object_absence helper and improve lifecycle test reliability

Signed-off-by: Mu junxiang <1948535941@qq.com>

* chore: remove dirty docs

Signed-off-by: Mu junxiang <1948535941@qq.com>

---------

Signed-off-by: Mu junxiang <1948535941@qq.com>
2025-09-27 17:57:56 -07:00
weisd
90f21a9102 refactor: Reimplement bucket replication system with enhanced architecture (#590)
* feat:refactor replication

* use aws sdk for replication client

* refactor/replication

* merge main

* fix lifecycle test
2025-09-26 14:27:53 +08:00
houseme
9b7f4d477a Fix Tokio Runtime Initialization: Remove Private API Usage and Ensure IO Enabled (#587)
* fix: remove code

* improve code for tokio runtime config

* improve code for main

* fix: add tokio enable_all

* upgrade version

* improve for Cargo.toml
2025-09-24 22:23:31 +08:00
guojidan
ef0dbaaeb5 feat(encryption): add managed encryption support for SSE-S3 and SSE-KMS (#583)
Signed-off-by: junxiang Mu <1948535941@qq.com>
2025-09-24 02:09:04 -07:00
Copilot
29b0935be7 RustFS rustfs-audit Complete Implementation with Enterprise Observability (#557)
* Initial plan

* Implement core audit system with multi-target fan-out and configuration management

Co-authored-by: houseme <4829346+houseme@users.noreply.github.com>

* Changes before error encountered

Co-authored-by: houseme <4829346+houseme@users.noreply.github.com>

* Complete audit system with comprehensive observability and test coverage

Co-authored-by: houseme <4829346+houseme@users.noreply.github.com>

* improve code

* fix

* improve code

* fix test

* fix test

* fix

* add `rustfs-audit` to `rustfs`

* upgrade crate version

* fmt

* fmt

* fix

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: houseme <4829346+houseme@users.noreply.github.com>
Co-authored-by: houseme <housemecn@gmail.com>
2025-09-24 08:23:46 +08:00
guojidan
9ddf6a011d feature: support kms && encryt (#573)
* feat(kms): implement key management service with local and vault backends

Signed-off-by: junxiang Mu <1948535941@qq.com>

* feat(kms): enhance security with zeroize for sensitive data and improve key management

Signed-off-by: junxiang Mu <1948535941@qq.com>

* remove Hashi word

Signed-off-by: junxiang Mu <1948535941@qq.com>

* refactor: remove unused request structs from kms handlers

Signed-off-by: junxiang Mu <1948535941@qq.com>

---------

Signed-off-by: junxiang Mu <1948535941@qq.com>
2025-09-22 17:53:05 +08:00
zzhpro
c2d782bed1 feat: support conditional writes (#409)
* feat: support conditional writes

* refactor: avoid using unwrap

* fix: obtain lock before check in CompleteMultiPartUpload

* refactor: do not obtain a lock when getting object meta

* fix: avoid using unwrap and modifying incoming arguments

* test: add e2e tests for conditional writes

---------

Co-authored-by: guojidan <63799833+guojidan@users.noreply.github.com>
Co-authored-by: 安正超 <anzhengchao@gmail.com>
Co-authored-by: loverustfs <155562731+loverustfs@users.noreply.github.com>
2025-08-25 18:35:24 -07:00
junxiang Mu
e369e9f481 Feature: lock support auto release
Signed-off-by: junxiang Mu <1948535941@qq.com>
2025-08-09 17:52:08 +08:00