sdgoij/rustfs
1
0
Fork 0
mirror of https://github.com/rustfs/rustfs.git synced 2026-01-17 01:30:33 +00:00
Code Issues Packages Projects Releases Wiki Activity
1,752 Commits 19 Branches 80 Tags
c5df1f92c2cb107cb828ecc01dd97136c6c9db78
Commit Graph

1752 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
houseme
c5df1f92c2 refactor: replace lazy_static with LazyLock and notify crate registry create_targets_from_config (#311) 
* improve code for notify

* improve code for logger and fix typo (#272)

* Add GNU to  build.yml (#275)

* fix unzip error

* fix url change error

fix url change error

* Simplify user experience and integrate console and endpoint

Simplify user experience and integrate console and endpoint

* Add gnu to  build.yml

* upgrade version

* feat: add `cargo clippy --fix --allow-dirty` to pre-commit command (#282)

Resolves #277

- Add --fix flag to automatically fix clippy warnings
- Add --allow-dirty flag to run on dirty Git trees
- Improves code quality in pre-commit workflow

* fix: the issue where preview fails when the path length exceeds 255 characters (#280)

* fix

* fix: improve Windows build support and CI/CD workflow (#283)

- Fix Windows zip command issue by using PowerShell Compress-Archive
- Add Windows support for OSS upload with ossutil
- Replace Chinese comments with English in build.yml
- Fix bash syntax error in package_zip function
- Improve code formatting and consistency
- Update various configuration files for better cross-platform support

Resolves Windows build failures in GitHub Actions.

* fix: update link in README.md leading to a 404 error (#285)

* add rustfs.spec for rustfs (#103)

add support on loongarch64

* improve cargo.lock

* build(deps): bump the dependencies group with 5 updates (#289)

Bumps the dependencies group with 5 updates:

| Package | From | To |
| --- | --- | --- |
| [hyper-util](https://github.com/hyperium/hyper-util) | `0.1.15` | `0.1.16` |
| [rand](https://github.com/rust-random/rand) | `0.9.1` | `0.9.2` |
| [serde_json](https://github.com/serde-rs/json) | `1.0.140` | `1.0.141` |
| [strum](https://github.com/Peternator7/strum) | `0.27.1` | `0.27.2` |
| [sysinfo](https://github.com/GuillaumeGomez/sysinfo) | `0.36.0` | `0.36.1` |


Updates `hyper-util` from 0.1.15 to 0.1.16
- [Release notes](https://github.com/hyperium/hyper-util/releases)
- [Changelog](https://github.com/hyperium/hyper-util/blob/master/CHANGELOG.md)
- [Commits](https://github.com/hyperium/hyper-util/compare/v0.1.15...v0.1.16)

Updates `rand` from 0.9.1 to 0.9.2
- [Release notes](https://github.com/rust-random/rand/releases)
- [Changelog](https://github.com/rust-random/rand/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rust-random/rand/compare/rand_core-0.9.1...rand_core-0.9.2)

Updates `serde_json` from 1.0.140 to 1.0.141
- [Release notes](https://github.com/serde-rs/json/releases)
- [Commits](https://github.com/serde-rs/json/compare/v1.0.140...v1.0.141)

Updates `strum` from 0.27.1 to 0.27.2
- [Release notes](https://github.com/Peternator7/strum/releases)
- [Changelog](https://github.com/Peternator7/strum/blob/master/CHANGELOG.md)
- [Commits](https://github.com/Peternator7/strum/compare/v0.27.1...v0.27.2)

Updates `sysinfo` from 0.36.0 to 0.36.1
- [Changelog](https://github.com/GuillaumeGomez/sysinfo/blob/master/CHANGELOG.md)
- [Commits](https://github.com/GuillaumeGomez/sysinfo/compare/v0.36.0...v0.36.1)

---
updated-dependencies:
- dependency-name: hyper-util
  dependency-version: 0.1.16
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: dependencies
- dependency-name: rand
  dependency-version: 0.9.2
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: dependencies
- dependency-name: serde_json
  dependency-version: 1.0.141
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: dependencies
- dependency-name: strum
  dependency-version: 0.27.2
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: dependencies
- dependency-name: sysinfo
  dependency-version: 0.36.1
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: dependencies
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* improve code for logger

* improve

* upgrade

* refactor: 优化构建工作流，统一 latest 文件处理和简化制品上传 (#293)

* Refactor: DatabaseManagerSystem as global

Signed-off-by: junxiang Mu <1948535941@qq.com>

* fix: fmt

Signed-off-by: junxiang Mu <1948535941@qq.com>

* Test: add e2e_test for s3select

Signed-off-by: junxiang Mu <1948535941@qq.com>

* Test: add test script for e2e

Signed-off-by: junxiang Mu <1948535941@qq.com>

* improve code for registry and intergation

* improve code for registry `create_targets_from_config`

* fix

* Feature up/ilm (#305)

* fix

* fix

* fix

* fix delete-marker expiration. add api_restore.

* fix

* time retry object upload

* lock file

* make fmt

* fix

* restore object

* fix

* fix

* serde-rs-xml -> quick-xml

* fix

* checksum

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* transfer lang to english

* upgrade clap version from 4.5.41 to 4.5.42

* refactor: replace `lazy_static` with `LazyLock`

* add router

* fix: modify comment

* improve code

* fix typos

* fix

* fix: modify name and fmt

* improve code for registry

* fix test

---------

Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: junxiang Mu <1948535941@qq.com>
Co-authored-by: loverustfs <155562731+loverustfs@users.noreply.github.com>
Co-authored-by: 安正超 <anzhengchao@gmail.com>
Co-authored-by: shiro.lee <69624924+shiroleeee@users.noreply.github.com>
Co-authored-by: Marco Orlandin <mipnamic@mipnamic.net>
Co-authored-by: zhangwenlong <zhangwenlong@loongson.cn>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: junxiang Mu <1948535941@qq.com>
Co-authored-by: likewu <likewu@126.com>
 2025-07-30 19:02:10 +08:00
wangsl
4f1770d3fe feat:add mcp integration (#300) 
* add list_buckets mcp server

* add list_objects mcp

* add upload object mcp

* add get object mcp

* add list_buckets mcp server

* fix: resolve clippy warnings in rustfs-mcp-server

* fix: rename mcp package

* fix

* fix:remove useless comment

* feat:add mcp doc
 2025-07-30 14:25:01 +08:00
likewu
d56cee26db Feature up/ilm (#305) 
* fix

* fix

* fix

* fix delete-marker expiration. add api_restore.

* fix

* time retry object upload

* lock file

* make fmt

* fix

* restore object

* fix

* fix

* serde-rs-xml -> quick-xml

* fix

* checksum

* fix

* fix

* fix

* fix

* fix

* fix

* fix
 2025-07-29 14:21:19 +08:00
weisd
56fd8132e9 fix:#303 returns empty when querying an empty or not dir (#304) 1.0.0-alpha.38 2025-07-28 16:17:40 +08:00
guojidan
35daa74430 Merge pull request #302 from guojidan/lock 
Lock: add transactional
 2025-07-28 12:00:44 +08:00
junxiang Mu
dc156fb4cd Fix: clippy 
Signed-off-by: junxiang Mu <1948535941@qq.com>
 2025-07-28 11:38:42 +08:00
junxiang Mu
de905a878c Cargo: use workspace dependence 
Signed-off-by: junxiang Mu <1948535941@qq.com>
 2025-07-28 11:02:40 +08:00
junxiang Mu
f3252f989b Test: Add e2e test case for lock transactional 
Signed-off-by: junxiang Mu <1948535941@qq.com>
 2025-07-28 11:00:10 +08:00
junxiang Mu
01a2afca9a lock: Add transactional 
Signed-off-by: junxiang Mu <1948535941@qq.com>
 2025-07-28 10:59:43 +08:00
guojidan
a4fe68ad21 Merge pull request #301 from guojidan/improve-sql 
s3Select: add unit test case
 2025-07-28 09:56:10 +08:00
junxiang Mu
c03f86b23c s3Select: add unit test case 
Signed-off-by: junxiang Mu <1948535941@qq.com>
 2025-07-28 09:19:47 +08:00
guojidan
5667f324ae Merge pull request #297 from guojidan/improve-sql 
Test: Add e2e_test case for sql && add script for e2e_test
 2025-07-25 17:16:41 +08:00
junxiang Mu
bcd806796f Test: add test script for e2e 
Signed-off-by: junxiang Mu <1948535941@qq.com>
 2025-07-25 16:52:06 +08:00
junxiang Mu
612404c47f Test: add e2e_test for s3select 
Signed-off-by: junxiang Mu <1948535941@qq.com>
 2025-07-25 15:07:44 +08:00
guojidan
85388262b3 Merge pull request #294 from guojidan/improve-sql 
Refactor: DatabaseManagerSystem as global
 2025-07-25 08:33:54 +08:00
junxiang Mu
25a4503285 fix: fmt 
Signed-off-by: junxiang Mu <1948535941@qq.com>
 2025-07-25 08:18:14 +08:00
安正超
526c4d5a61 refactor: 优化构建工作流，统一 latest 文件处理和简化制品上传 (#293) 2025-07-25 01:10:04 +08:00
junxiang Mu
addc964d56 Refactor: DatabaseManagerSystem as global 
Signed-off-by: junxiang Mu <1948535941@qq.com>
 2025-07-24 17:12:51 +08:00
loverustfs
371119f733 GNU to MUSL modify Dockerfile 1.0.0-alpha.37 2025-07-24 16:36:15 +08:00
guojidan
021abc0398 Merge pull request #292 from guojidan/Arc 
Chore: remove dirty file(cache.rs)
 2025-07-24 16:32:20 +08:00
junxiang Mu
0672b6dd3e Chore: remove dirty file(cache.rs) 
Signed-off-by: junxiang Mu <1948535941@qq.com>
 2025-07-24 14:57:48 +08:00
guojidan
1372dc2857 Merge pull request #288 from guojidan/scanner 
Refactor: Scanner
 2025-07-24 14:42:54 +08:00
houseme
77bc9af109 Update Cargo.toml 2025-07-24 14:14:12 +08:00
junxiang Mu
91b1c84430 rebase 
Signed-off-by: junxiang Mu <1948535941@qq.com>
 2025-07-24 12:18:05 +08:00
junxiang Mu
b667927216 fix fmt 
Signed-off-by: junxiang Mu <1948535941@qq.com>
 2025-07-24 12:14:28 +08:00
junxiang Mu
29795fac51 fix Cargo.toml 
Signed-off-by: junxiang Mu <1948535941@qq.com>
 2025-07-24 12:14:28 +08:00
junxiang Mu
2ce7e01f55 Chore: remove dirty file(heal) 
Signed-off-by: junxiang Mu <1948535941@qq.com>
 2025-07-24 12:14:27 +08:00
junxiang Mu
4fefd63a5b rebase 
Signed-off-by: junxiang Mu <1948535941@qq.com>
 2025-07-24 12:14:05 +08:00
junxiang Mu
2a8c46874d fix: auto heal when xl.meta lose 
Signed-off-by: junxiang Mu <1948535941@qq.com>
 2025-07-24 12:14:05 +08:00
junxiang Mu
b8b5511b68 fix: heal data part lose 
Signed-off-by: junxiang Mu <1948535941@qq.com>
 2025-07-24 12:14:05 +08:00
junxiang Mu
bdaee228db fix(ahm): adjust test expectations for missing xl.meta recovery scenario 
Signed-off-by: junxiang Mu <1948535941@qq.com>
 2025-07-24 12:14:05 +08:00
junxiang Mu
d562620e99 fix: implement uses_data_dir method 
Signed-off-by: junxiang Mu <1948535941@qq.com>
 2025-07-24 12:14:05 +08:00
junxiang Mu
69b0c828c9 fix: scanner add heal bucket 
Signed-off-by: junxiang Mu <1948535941@qq.com>
 2025-07-24 12:14:05 +08:00
junxiang Mu
2bfd1efb9b Fix: fix add heal_manager into scanner when scanner start 
Signed-off-by: junxiang Mu <1948535941@qq.com>
 2025-07-24 12:14:05 +08:00
junxiang Mu
0854e6b921 Chore: rename init_heal_manager_with_channel 
Signed-off-by: junxiang Mu <1948535941@qq.com>
 2025-07-24 12:14:05 +08:00
junxiang Mu
b907f4e61b refactor(ahm): remove obsolete scanner/data_usage.rs after data usage refactor 
Signed-off-by: junxiang Mu <1948535941@qq.com>
 2025-07-24 12:14:05 +08:00
junxiang Mu
6ec568459c chore: update admin handlers, lockfile, and minor fixes 
Signed-off-by: junxiang Mu <1948535941@qq.com>
 2025-07-24 12:14:05 +08:00
junxiang Mu
ea210d52dc refactor(heal): unify heal request interface, add disk field, update ahm/ecstore/common for erasure set healing 
Signed-off-by: junxiang Mu <1948535941@qq.com>
 2025-07-24 12:14:03 +08:00
junxiang Mu
3d3c6e4e06 chore(protos): update proto definitions, remove ns_scanner, fix codegen and formatting 
Signed-off-by: junxiang Mu <1948535941@qq.com>
 2025-07-24 12:12:49 +08:00
junxiang Mu
e7d0a8d4b9 feat: integrate global metrics system into AHM scanner 
- Add global metrics system to common crate for cross-module usage
- Integrate global metrics collection into AHM scanner operations
- Update ECStore to use common metrics system instead of local implementation
- Add chrono dependency to AHM crate for timestamp handling
- Re-export IlmAction from common metrics in ECStore lifecycle module
- Update scanner methods to use global metrics for cycle, disk, and volume scans
- Maintain backward compatibility with local metrics collector
- Fix clippy warnings and ensure proper code formatting

This change enables unified metrics collection across the entire RustFS system,
allowing better monitoring and observability of scanner operations.

Signed-off-by: junxiang Mu <1948535941@qq.com>
 2025-07-24 12:12:49 +08:00
junxiang Mu
7d3b2b774c fix heal disk 
Signed-off-by: junxiang Mu <1948535941@qq.com>
 2025-07-24 12:12:49 +08:00
junxiang Mu
aed8f52423 refactor: integrate disk healing into erasure set healing 
- Remove HealType::Disk and related disk-specific healing methods
- Integrate disk format healing into heal_erasure_set with include_format_heal option
- Update auto disk scanner to use ErasureSet heal type instead of Disk heal
- Fix disk status change event handling to use ErasureSet heal requests
- Add proper bucket list retrieval for auto healing scenarios
- Update data scanner to submit ErasureSet heal tasks for offline disks
- Remove duplicate healing logic between Disk and ErasureSet types
- Ensure all healing operations go through unified ErasureSet healing path
 2025-07-24 12:12:49 +08:00
junxiang Mu
c49414f6ac fix: resolve test conflicts and improve data scanner functionality 
- Fix multi-threaded test conflicts in AHM heal integration tests
- Remove global environment sharing to prevent test state pollution
- Fix test_all_disk_method by clearing global disk map before test
- Improve data scanner and cache value implementations
- Update dependencies and resolve clippy warnings

Signed-off-by: junxiang Mu <1948535941@qq.com>
 2025-07-24 12:12:49 +08:00
junxiang Mu
8e766b90cd feat: implement heal channel mechanism for admin-ahm communication 
- Add global unbounded channel in common crate for heal requests
- Implement channel processor in ahm to handle heal commands
- Add Start/Query/Cancel commands support via channel
- Integrate heal manager initialization in main.rs
- Replace direct MRF calls with channel-based heal requests in ecstore
- Support advanced heal options including pool_index and set_index
- Enable admin handlers to send heal requests via channel
 2025-07-24 12:12:49 +08:00
junxiang Mu
3409cd8dff feat(ahm): add HealingTracker support & complete fresh-disk healing 
• Introduce ecstore HealingTracker into ahm crate; load/init/save tracker
• Re-implement heal_fresh_disk to use heal_erasure_set with tracker
• Enhance auto-disk scanner: detect unformatted disks via get_disk_id()
• Remove DataUsageCache handling for now
• Refactor imports & types, clean up duplicate constants
 2025-07-24 12:12:49 +08:00
junxiang Mu
f4973a681c feat: implement complete ahm heal system with ecstore integration 
- Add comprehensive heal storage API with ECStore integration
- Implement heal object, bucket, disk, metadata, and EC decode operations
- Add heal task management with progress tracking and statistics
- Optimize heal manager by removing unnecessary workers
- Add integration tests for core heal functionality (heal_object, heal_bucket, heal_format)
- Integrate with ecstore's native heal commands for actual repair operations

Signed-off-by: junxiang Mu <1948535941@qq.com>
 2025-07-24 12:12:49 +08:00
junxiang Mu
4fb3d187d0 feat: implement heal subsystem for automatic data repair 
- Add heal module with core types (HealType, HealRequest, HealTask)
- Implement HealManager for task scheduling and execution
- Add HealStorageAPI trait and ECStoreHealStorage implementation
- Integrate heal capabilities into scanner for automatic repair
- Support multiple heal types: object, bucket, disk, metadata, MRF, EC decode
- Add progress tracking and event system for heal operations
- Merge heal and scanner error types for unified error handling
- Include comprehensive logging and metrics for heal operations

Signed-off-by: junxiang Mu <1948535941@qq.com>
 2025-07-24 12:12:49 +08:00
dandan
0aff736efd Chore: fix ref and fix comment 
Signed-off-by: junxiang Mu <1948535941@qq.com>
 2025-07-24 12:12:49 +08:00
dandan
2aa7a631ef feat: refactor scanner module and add data usage statistics 
- Move scanner code to scanner/ subdirectory for better organization
- Add data usage statistics collection and persistence
- Implement histogram support for size and version distribution
- Add global cancel token management for scanner operations
- Integrate scanner with ECStore for comprehensive data analysis
- Update error handling and improve test isolation
- Add data usage API endpoints and backend integration

Signed-off-by: junxiang Mu <1948535941@qq.com>
 2025-07-24 12:12:49 +08:00
dandan
b40ef147a9 refact: step 2 
Signed-off-by: junxiang Mu <1948535941@qq.com>
 2025-07-24 12:12:49 +08:00