Commit Graph

71 Commits

Author SHA1 Message Date
DamonXue
9948b1f709 feat: Implement KMS configuration management and dynamic updates
- Added a new KMS configuration subsystem to support dynamic KMS settings.
- Introduced a global ConfigManager for managing KMS and general configurations.
- Implemented KMS configuration validation and connection testing.
- Created REST API endpoints for retrieving and updating KMS configurations.
- Enhanced the existing configuration handling to include KMS-specific parameters.
- Updated the Cargo.toml to include the new KMS feature.
- Added comprehensive tests for KMS configuration management and validation.
2025-06-07 20:56:22 +08:00
houseme
ff264f3385 feat(obs): upgrade OpenTelemetry dependencies to latest version (#447)
- Update opentelemetry from 0.29.1 to 0.30.0
- Update related opentelemetry dependencies for compatibility
- Ensure compatibility with existing observability implementation
- Improve tracing and metrics collection capabilities

This upgrade provides better performance and stability for our observability stack.
2025-06-07 00:10:20 +08:00
houseme
751abeca2b set logger level from RUST_LOG 2025-06-06 16:19:17 +08:00
houseme
9cc34f9f01 improve code for reqwest feature 2025-06-06 14:35:40 +08:00
houseme
1c5ba761ef upgrade version 2025-06-06 13:49:31 +08:00
houseme
cdd41752e0 improve code for obs crate 2025-05-30 13:04:24 +08:00
houseme
5e3c97b0e9 fix(obs): align stdout log level with configured logger_level
- Replace fixed `flexi_logger::Duplicate::Info` with dynamic level from config
- Convert logger_level string to corresponding LevelFilter enum
- Ensure terminal output respects the same log level as file logs
- Fix documentation to reflect the dynamic level behavior
2025-05-30 00:22:21 +08:00
houseme
2f20fe9749 docs(.docker): add bilingual README for OpenObserve+OpenTelemetry setup
- Create English and Chinese README files for the openobserve-otel directory
- Document configuration details for both OpenObserve and OTel Collector
- Include setup instructions and application integration examples
- Add badges for both OpenObserve and OpenTelemetry projects
2025-05-29 23:48:11 +08:00
houseme
1d45123968 fix 2025-05-29 22:30:33 +08:00
houseme
88b2d893dc fix 2025-05-29 21:15:45 +08:00
houseme
8887b7ea90 improve code for obs 2025-05-29 21:10:04 +08:00
houseme
2fd5ef75cf modify config for obs 2025-05-29 13:24:56 +08:00
安正超
52da2569dd Merge pull request #437 from rustfs/feat/add-formatting-rules-and-type-inference
feat: add comprehensive formatting rules and type inference guidelines
2025-05-28 16:20:59 +08:00
overtrue
3a32517f79 feat: add comprehensive formatting rules and type inference guidelines 2025-05-28 16:04:38 +08:00
houseme
d5539bf22f add workflow Samply action and modify console address port 9001 2025-05-28 16:02:53 +08:00
overtrue
5ef7571211 fix: resolve all doctest failures in rustfs-obs crate 2025-05-28 15:29:09 +08:00
安正超
4cd2c8e99c refactor: remove dead code comment in observability config test 2025-05-28 14:36:38 +08:00
安正超
b50e88a5a0 fix: resolve all remaining test failures and Clippy warnings 2025-05-28 11:28:43 +08:00
安正超
9ec22255e0 fix: resolve all Clippy warnings across codebase - Fixed field reassignment warnings in ecstore/src/file_meta.rs by using struct initialization instead of default + field assignment - Fixed overly complex boolean expression in ecstore/src/utils/os/mod.rs by removing meaningless assertion - Replaced manual Default implementation with derive in crates/zip/src/lib.rs - Updated io::Error usage to use io::Error::other() instead of deprecated pattern - Removed useless assertions and clone-on-copy warnings - Fixed unwrap usage by replacing with expect() providing meaningful error messages - Fixed useless vec usage by using array repeat instead - All code now passes comprehensive Clippy check with --all-targets --all-features -- -D warnings 2025-05-28 11:00:07 +08:00
overtrue
670636b3aa feat: add comprehensive test coverage for config module 2025-05-27 23:11:29 +08:00
overtrue
86a99d214c feat: add comprehensive test coverage for utils certs module 2025-05-27 22:44:30 +08:00
houseme
366fd98aeb format comment 2025-05-27 13:56:19 +08:00
houseme
f1ef7149e3 improve code for otel (#418) 2025-05-26 12:05:57 +08:00
houseme
ea4a225d70 cargo fmt 2025-05-25 17:46:59 +08:00
overtrue
142281e96a feat: translate Chinese comments to English across codebase 2025-05-25 15:24:34 +08:00
overtrue
b40b6a17f4 feat: enhance test coverage for IAM utils and OS utils modules 2025-05-25 14:09:40 +08:00
overtrue
9c90426032 feat: improve test coverage and fix critical crypto bug - Translate all Chinese comments to English in utils/ip.rs and config/constants/app.rs - Add comprehensive test suite for crypto/encdec/id.rs module (14 new tests) - Fix critical bug in Argon2 key generation that was returning all-zero keys - Improve test coverage for IP utilities and configuration constants - Ensure all test cases follow English naming conventions and meaningful descriptions 2025-05-25 13:53:59 +08:00
overtrue
a032c401d1 refactor: improve test code quality by replacing meaningless names and content 2025-05-25 13:40:54 +08:00
安正超
a608e0dd65 Merge pull request #404 from rustfs/feat/zip-features
feat: complete zip compression module implementation
2025-05-25 13:35:46 +08:00
overtrue
62d994c103 feat: add comprehensive tests for event-notifier error module 2025-05-25 13:26:25 +08:00
overtrue
41373960bc feat: complete zip compression module implementation 2025-05-25 13:14:38 +08:00
安正超
bf543f9628 Merge pull request #403 from rustfs/feat/zip-tests
feat: add comprehensive test coverage for zip compression module
2025-05-25 13:11:23 +08:00
overtrue
f81bef28df feat: add comprehensive test coverage for zip compression module 2025-05-25 13:05:11 +08:00
overtrue
7730e6cd3a feat: enhance test coverage and fix compilation errors 2025-05-25 12:56:43 +08:00
houseme
3592ffb791 fix 2025-05-14 19:04:52 +08:00
houseme
65be8145ff feat(obs): implement global OpenTelemetry guard management 2025-05-12 13:32:18 +08:00
houseme
4ac4b35c5e Feature/rustfs config (#396)
* init rustfs config

* improve code for rustfs-config crate

* add

* improve code for comment

* fix: modify rustfs-config crate name

* add default fn

* improve error logger

* fix: modify docker config yaml

* improve code for config

* feat: restrict kafka feature to Linux only

- Add target-specific feature configuration in Cargo.toml for obs and event-notifier crates
- Implement conditional compilation for kafka feature only on Linux systems
- Add appropriate error handling for non-Linux platforms
- Ensure backward compatibility with existing code

* refactor(ci): optimize build workflow for better efficiency

- Integrate GUI build steps into main build-rustfs job
- Add conditional GUI build execution based on tag releases
- Simplify workflow by removing redundant build-rustfs-gui job
- Copy binary directly to embedded-rustfs directory without downloading artifacts
- Update merge job dependency to only rely on build-rustfs
- Improve cross-platform compatibility for Windows binary naming (.exe)
- Streamline artifact uploading and OSS publishing process
- Maintain consistent conditional logic for release operations

* refactor(ci): optimize build workflow for better efficiency

- Integrate GUI build steps into main build-rustfs job
- Add conditional GUI build execution based on tag releases
- Simplify workflow by removing redundant build-rustfs-gui job
- Copy binary directly to embedded-rustfs directory without downloading artifacts
- Update merge job dependency to only rely on build-rustfs
- Improve cross-platform compatibility for Windows binary naming (.exe)
- Streamline artifact uploading and OSS publishing process
- Maintain consistent conditional logic for release operations

* fix(ci): add repo-token to setup-protoc action for authentication

- Add GITHUB_TOKEN parameter to arduino/setup-protoc@v3 action
- Ensure proper authentication for Protoc installation in CI workflow
- Maintain consistent setup across different CI environments

* modify config

* improve readme.md

* remove env config relation

* add allow(dead_code)
2025-05-12 01:17:31 +08:00
weisd
f2c9464eb0 Merge branch 'main' into feat/zip 2025-05-09 14:38:14 +08:00
junxiang Mu
0a0fb41037 fix zero size object bug
Signed-off-by: junxiang Mu <1948535941@qq.com>
2025-05-08 19:34:58 +08:00
weisd
143017f925 feat: auto-extract support 2025-05-08 17:42:20 +08:00
houseme
38377f81ed refactor: standardize constant management and fix typos (#387)
* init rustfs config

* init rustfs-utils crate

* improve code for rustfs-config crate

* add

* improve code for comment

* init rustfs config

* improve code for rustfs-config crate

* add

* improve code for comment

* Unified management of configurations and constants

* fix: modify rustfs-config crate name

* add default fn

* improve code for rustfs config

* refactor: standardize constant management and fix typos

- Create centralized constants module for global static constants
- Replace runtime format! expressions with compile-time constants
- Fix DEFAULT_PORT reference issues in configuration arguments
- Use const-str crate for compile-time string concatenation
- Update tokio dependency from 1.42.2 to 1.45.0
- Ensure consistent naming convention for configuration constants

* fix

* Update common/workers/src/workers.rs

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2025-05-07 17:23:22 +08:00
houseme
26d4726181 Feature/bucket event notification (#365)
* add tracing instrument

* fix rebalance/decom

* modify Telemetry filter order

* feat: improve address binding and port handling mechanism (#366)

* feat: improve address binding and port handling mechanism

1. Add support for ":port" format to enable dual-stack binding (IPv4/IPv6)
2. Implement automatic port allocation when port 0 is specified
3. Optimize server startup process with unified address resolution
4. Enhance error handling and logging for address resolution
5. Improve graceful shutdown with signal listening
6. Clean up commented code in console.rs

Files:
- ecstore/src/utils/net.rs
- rustfs/src/console.rs
- rustfs/src/main.rs

Branch: feature/server-and-console-port

* improve code for console

* improve code

* improve code for console and net.rs

* Update rustfs/src/main.rs

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update rustfs/src/utils/mod.rs

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* upgrade config file

* modify

* fix readme

Signed-off-by: junxiang Mu <1948535941@qq.com>

* improve readme.md

* improve code for readme.md
add chinese readme.md

* Implement Storage Service Event Notification System

Added event notification capability to the storage module, enabling the storage service to publish object operation events. Key changes include:

1. Created `event_notifier` module providing core functionality:
   - `create_metadata()` - Creates event metadata objects with default configuration ID
   - `send_event()` - Asynchronously sends event notifications with error handling

2. Integrated the `rustfs_event_notifier` library:
   - Supports object creation, deletion, and access events
   - Provides event metadata building and management
   - Includes proper error propagation

These changes enable the system to trigger notifications when storage operations occur, facilitating auditing, monitoring, and integration with other systems.

* fix

---------

Signed-off-by: junxiang Mu <1948535941@qq.com>
Co-authored-by: weisd <im@weisd.in>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: junxiang Mu <1948535941@qq.com>
2025-04-30 00:31:55 +08:00
houseme
022304d046 improve code for request and telemetry 2025-04-29 11:55:59 +08:00
junxiang Mu
6f50b69c5f tmp1
Signed-off-by: junxiang Mu <1948535941@qq.com>
2025-04-29 02:53:42 +00:00
houseme
89709184c2 modify 2025-04-28 12:50:56 +08:00
houseme
02cef27230 modify Telemetry filter order 2025-04-27 09:50:32 +08:00
houseme
922330ded2 Feature/upgrade obs docker (#364)
* upgrade docker config

* upgrade obs.toml

* modify dockerfile image from alpine to ubuntu
2025-04-26 22:36:38 +08:00
houseme
87c63193ae improve code for opentelemetry 2025-04-26 17:26:54 +08:00
houseme
b80f64d9b4 improve code 2025-04-25 13:35:03 +08:00
houseme
fc9d433038 feat: add TraceLayer for HTTP service and improve metrics (#361)
* improve code for opentelemetry and add system metrics

* feat: add TraceLayer for HTTP service and improve metrics

- Add TraceLayer to HTTP server for request tracing
- Implement system metrics for process monitoring
- Optimize init_telemetry method for better resource management
- Add graceful shutdown handling for telemetry components
- Fix GracefulShutdown ownership issues with Arc wrapper

* improve code for init_process_observer

* remove tomlfmt.toml

* Translation comment

* improve code for console CompressionLayer params
2025-04-24 19:04:57 +08:00