Commit Graph

973 Commits

Author SHA1 Message Date
weisd
7eefafaad5 merge fix/pools 2025-04-23 16:48:57 +08:00
weisd
6e17a91919 fix: #331 admin info version,uptime 2025-04-23 16:34:39 +08:00
guojidan
6f2b4b34b9 Merge pull request #348 from rustfs/fix-data-scan
fix
2025-04-23 16:28:12 +08:00
junxiang Mu
dbb5625199 fix
Signed-off-by: junxiang Mu <1948535941@qq.com>
2025-04-23 07:26:37 +00:00
weisd
89785dc06b pool select idx
fixs:#346, #339, #338, #337, #336, #334

test healbucket

test get_available_pool_idx

fix
2025-04-23 15:11:43 +08:00
houseme
46c3134487 Feature/bucket event notification (#347)
* init event notifer

* feat: implement event notification system

- Add core event notification interfaces
- Support multiple notification backends:
  - Webhook (default)
  - Kafka
  - MQTT
  - HTTP Producer
- Implement configurable event filtering
- Add async event dispatching with backpressure handling
- Provide serialization/deserialization for event payloads

This module enables system events to be published to various endpoints
with consistent delivery guarantees and failure handling.

* feat(event-notifier): improve notification system initialization safety

- Add READY atomic flag to track full initialization status
- Implement initialize_safe and start_safe methods with mutex protection
- Add wait_until_ready function with configurable timeout
- Create initialize_and_start_with_ready_check helper method
- Replace sleep-based waiting with proper readiness checks
- Add safety checks before sending events
- Replace chrono with std::time for time handling
- Update error handling to provide clear initialization status

This change reduces race conditions in multi-threaded environments
and ensures events are only processed when the system is fully ready.

* fix sql

Signed-off-by: junxiang Mu <1948535941@qq.com>

* move protobuf generate into bin crate

Signed-off-by: junxiang Mu <1948535941@qq.com>

* improve Cargo toml

* improve code

* improve code for global

* improve code

* feat(event-notifier): improve environment variable handling

- Fix deserialization error when parsing config from environment variables
- Add proper array format support for adapters configuration
- Update environment variable examples with correct format
- Improve documentation for configuration loading
- Implement helper functions for environment variable validation

This change fixes the "invalid type: map, expected a sequence" error
by ensuring proper formatting of array-type fields in environment variables.

* feat: integrate event-notifier system with rustfs

- Rename package from rustfs-event-notifier to event-notifier for consistency
- Add shutdown hooks for event notification system in main process
- Handle graceful termination of notification services on server shutdown
- Implement initialization and configuration loading for event notification
- Fix environment variable configuration to properly parse adapter arrays
- Update example code to demonstrate proper configuration usage

This change ensures proper integration between rustfs and the event
notification system, with clean startup and shutdown sequences to prevent
resource leaks during application lifecycle.

* feat: improve webhook server and run script integration

- Enhance webhook example with proper shutdown handling using tokio::select!
- Update run.sh to automatically start webhook server alongside main service
- Add event notification configuration to run.sh using environment variables
- Set proper port bindings to ensure webhook server starts on port 3000
- Improve console output for better debugging experience
- Fix race condition during service startup and shutdown

This change ensures proper integration between the webhook server and
the main rustfs service, providing a seamless development experience
with automatic service discovery and clean termination.

* improve for logger

* improve code for global.rs

* fix: modify webhook port

* fix

---------

Signed-off-by: junxiang Mu <1948535941@qq.com>
Co-authored-by: junxiang Mu <1948535941@qq.com>
2025-04-22 23:08:03 +08:00
houseme
b1a6da8d73 fix 2025-04-22 23:07:13 +08:00
houseme
dde551ef94 fix: modify webhook port 2025-04-22 23:06:43 +08:00
houseme
65f036442a improve code for global.rs 2025-04-22 22:59:31 +08:00
houseme
2167f5e728 improve for logger 2025-04-22 22:41:45 +08:00
houseme
9fc284bef3 feat: improve webhook server and run script integration
- Enhance webhook example with proper shutdown handling using tokio::select!
- Update run.sh to automatically start webhook server alongside main service
- Add event notification configuration to run.sh using environment variables
- Set proper port bindings to ensure webhook server starts on port 3000
- Improve console output for better debugging experience
- Fix race condition during service startup and shutdown

This change ensures proper integration between the webhook server and
the main rustfs service, providing a seamless development experience
with automatic service discovery and clean termination.
2025-04-22 21:40:20 +08:00
houseme
b57b92e382 feat: integrate event-notifier system with rustfs
- Rename package from rustfs-event-notifier to event-notifier for consistency
- Add shutdown hooks for event notification system in main process
- Handle graceful termination of notification services on server shutdown
- Implement initialization and configuration loading for event notification
- Fix environment variable configuration to properly parse adapter arrays
- Update example code to demonstrate proper configuration usage

This change ensures proper integration between rustfs and the event
notification system, with clean startup and shutdown sequences to prevent
resource leaks during application lifecycle.
2025-04-22 20:49:39 +08:00
houseme
0d28c1cbf8 feat(event-notifier): improve environment variable handling
- Fix deserialization error when parsing config from environment variables
- Add proper array format support for adapters configuration
- Update environment variable examples with correct format
- Improve documentation for configuration loading
- Implement helper functions for environment variable validation

This change fixes the "invalid type: map, expected a sequence" error
by ensuring proper formatting of array-type fields in environment variables.
2025-04-22 20:31:38 +08:00
houseme
e3af3d1b94 Merge branch 'main' of github.com:rustfs/s3-rustfs into feature/bucket-event-notification
# Conflicts:
#	Cargo.toml
2025-04-22 09:16:48 +08:00
houseme
7411283fc8 improve code 2025-04-22 09:14:09 +08:00
weisd
c331365ebf otel debug filter tower 2025-04-22 09:07:43 +08:00
weisd
0e9d1d63d3 fix pool select idx 2025-04-22 09:07:43 +08:00
DamonXue
4e9e63dff6 Merge pull request #345 from rustfs/dev_objectEncrypt_v1
feat: update launch configuration and clean unuseful docker-compose config
2025-04-21 22:08:02 +08:00
DamonXue
ad528935ad feat: update launch configuration and docker-compose for enhanced observability and logging 2025-04-21 22:05:31 +08:00
houseme
e634ffdd23 improve code for global 2025-04-21 21:39:57 +08:00
houseme
165660e106 improve code 2025-04-21 19:01:31 +08:00
houseme
14425be416 improve Cargo toml 2025-04-21 15:27:58 +08:00
junxiang Mu
47afb1a651 move protobuf generate into bin crate
Signed-off-by: junxiang Mu <1948535941@qq.com>
2025-04-21 13:28:32 +08:00
junxiang Mu
95d59d7206 fix sql
Signed-off-by: junxiang Mu <1948535941@qq.com>
2025-04-21 13:28:32 +08:00
houseme
e2b7a9772c feat(event-notifier): improve notification system initialization safety
- Add READY atomic flag to track full initialization status
- Implement initialize_safe and start_safe methods with mutex protection
- Add wait_until_ready function with configurable timeout
- Create initialize_and_start_with_ready_check helper method
- Replace sleep-based waiting with proper readiness checks
- Add safety checks before sending events
- Replace chrono with std::time for time handling
- Update error handling to provide clear initialization status

This change reduces race conditions in multi-threaded environments
and ensures events are only processed when the system is fully ready.
2025-04-21 13:28:01 +08:00
guojidan
5ccb08e73f Merge pull request #343 from rustfs/improve-proto
move protobuf generate into bin crate
2025-04-21 11:11:35 +08:00
junxiang Mu
778ca76607 move protobuf generate into bin crate
Signed-off-by: junxiang Mu <1948535941@qq.com>
2025-04-21 03:09:54 +00:00
guojidan
c78338df9f Merge pull request #342 from rustfs/fix-sql
fix sql
2025-04-21 09:47:02 +08:00
junxiang Mu
adf401e09a fix sql
Signed-off-by: junxiang Mu <1948535941@qq.com>
2025-04-21 01:43:13 +00:00
houseme
bb6324b4e5 feat: implement event notification system
- Add core event notification interfaces
- Support multiple notification backends:
  - Webhook (default)
  - Kafka
  - MQTT
  - HTTP Producer
- Implement configurable event filtering
- Add async event dispatching with backpressure handling
- Provide serialization/deserialization for event payloads

This module enables system events to be published to various endpoints
with consistent delivery guarantees and failure handling.
2025-04-21 00:17:27 +08:00
houseme
9b6170e94f init event notifer 2025-04-19 02:20:50 +08:00
weisd
8d865eb048 fix scstore::new bugs, stop ns_lock 2025-04-19 01:52:27 +08:00
weisd
8fefb8f7cb add tracing 2025-04-18 16:21:14 +08:00
weisd
443fbeee38 Merge branch 'main' into dada/decom 2025-04-18 16:02:05 +08:00
weisd
31addf01dd feat:decom,rebalance 2025-04-18 16:00:21 +08:00
houseme
2a6554a3c6 fix console http server 2025-04-16 17:01:25 +08:00
Nugine
691a0bed5d build(deps): upgrade s3s
resolves: #322
2025-04-16 15:16:19 +08:00
loverustfs
5088f51236 Merge pull request #330 from rustfs/nugine/performance/jemalloc
feat(rustfs/main): use jemalloc
2025-04-16 14:16:34 +08:00
Nugine
aeb1beab17 feat(rustfs/main): use jemalloc 2025-04-16 11:59:08 +08:00
houseme
d4b4ef1108 Feature/status Support IPV4 and IPV6 dual stack and 308 Permanent Redirect (#329)
* test 308 Permanent Redirect

* improve code and Support IPV4 and IPV6 dual stack

* remove code
2025-04-15 23:29:12 +08:00
loverustfs
16bbe517b4 Merge pull request #328 from rustfs/nugine/performance/tcp_nodelay
feat(rustfs/main): set TCP_NODELAY
2025-04-15 20:47:06 +08:00
loverustfs
f1add9dc58 Merge pull request #326 from rustfs/nugine/performance/v4
feat(ecstore/erasure): optimize `encode_data`
2025-04-15 20:46:51 +08:00
Nugine
2363a3706f feat(rustfs/main): set TCP_NODELAY 2025-04-15 19:36:34 +08:00
Nugine
08e8c7258c feat(ecstore/erasure): optimize encode_data 2025-04-15 18:07:51 +08:00
Nugine
5df2bdb063 build(deps): upgrade s3s 2025-04-15 17:37:08 +08:00
weisd
115d9ea780 merge main 2025-04-14 22:38:58 +08:00
weisd
70b65343ab todo 2025-04-14 17:32:49 +08:00
junxiangMu
77c7146677 Merge pull request #323 from rustfs/fix-data-scan
fix datascanner
2025-04-14 10:19:06 +08:00
junxiang Mu
980ecb8bd3 fix datascanner
Signed-off-by: junxiang Mu <1948535941@qq.com>
2025-04-14 02:18:40 +00:00
weisd
0408f29301 fix config_handler 2025-04-13 23:52:52 +08:00