Optimize concurrent GetObject performance with Moka cache, comprehensive metrics, complete test suite, cache writeback, and comprehensive documentation (#916)

* Initial plan

* feat: add concurrency-aware buffer sizing and hot object caching for GetObject

- Implement adaptive buffer sizing based on concurrent request load
- Add per-request tracking with automatic cleanup using RAII guards
- Implement hot object cache (LRU) for frequently accessed small files (<= 10MB)
- Add disk I/O semaphore to prevent saturation under extreme load
- Integrate concurrency module into GetObject implementation
- Buffer sizes now adapt: low concurrency uses large buffers for throughput,
  high concurrency uses smaller buffers for fairness and memory efficiency
- Add comprehensive metrics collection for monitoring performance

Co-authored-by: houseme <4829346+houseme@users.noreply.github.com>

* docs: add comprehensive documentation and tests for concurrent GetObject optimization

- Add detailed technical documentation explaining the solution
- Document root cause analysis and solution architecture
- Include performance expectations and testing recommendations
- Add integration tests for concurrency tracking and buffer sizing
- Add cache behavior tests
- Include benchmark tests for concurrent request handling

Co-authored-by: houseme <4829346+houseme@users.noreply.github.com>

* fix: address code review issues in concurrency module

- Fix race condition in cache size tracking by using consistent atomic operations within lock
- Correct buffer sizing logic: 1-2 requests use 100%, 3-4 use 75%, 5-8 use 50%, >8 use 40%
- Improve error message for semaphore acquire failure
- Document limitation of streaming cache implementation (not yet implemented)
- Add TODO for proper streaming cache with suggested approaches
- Update tests to match corrected buffer sizing thresholds

Co-authored-by: houseme <4829346+houseme@users.noreply.github.com>

* docs: add comprehensive implementation summary for concurrent GetObject optimization

- Executive summary of problem and solution
- Detailed architecture documentation for each optimization
- Integration points and code examples
- Complete testing strategy and performance validation steps
- Monitoring and observability guidelines with Prometheus queries
- Deployment considerations and rollout strategy
- Future enhancement roadmap
- Success criteria and quantitative metrics

Co-authored-by: houseme <4829346+houseme@users.noreply.github.com>

* fix

* fix

* refactor: optimize cache with lru 0.16.2 read-first pattern and add advanced features

- Implement optimized read-first cache access using peek() to reduce write lock contention
- Add batch cache operations: get_cached_batch() for efficient multi-object retrieval
- Add cache utility methods: is_cached(), remove_cached(), get_hot_keys()
- Implement warm_cache() for pre-populating cache on startup
- Add get_advanced_buffer_size() with file size and sequential read optimization
- Enhance test suite with 8 new comprehensive tests covering:
  - Batch operations and cache warming
  - Hot keys tracking and analysis
  - Cache removal and LRU behavior verification
  - Concurrent cache access performance
  - Advanced buffer sizing strategies
- Improve documentation and code comments in English throughout

Co-authored-by: houseme <4829346+houseme@users.noreply.github.com>

* docs: add final comprehensive optimization summary

- Complete technical documentation of all optimizations
- Detailed performance analysis and metrics
- Production deployment guide with examples
- Comprehensive API reference and usage patterns
- Migration guide and future enhancement roadmap
- All documentation in professional English

Co-authored-by: houseme <4829346+houseme@users.noreply.github.com>

* fix

* fix

* add moka crate for cache

* feat: replace LRU with Moka cache and add comprehensive metrics

- Replace lru crate with moka 0.12.11 for superior concurrent performance
- Implement lock-free cache with automatic TTL/TTI expiration
- Add size-based eviction using Moka's weigher function
- Integrate comprehensive metrics collection throughout GetObject flow:
  * Cache hit/miss tracking with per-key access counts
  * Request concurrency gauges
  * Disk permit wait time histograms
  * Total request duration tracking
  * Response size and buffer size histograms
- Deep integration with ecfs.rs GetObject operation
- Add hit rate calculation method
- Enhanced CacheStats with hit/miss counters
- Lock-free concurrent reads for better scalability

Moka advantages over LRU:
- True lock-free concurrent access
- Built-in TTL and TTI support
- Automatic size-based eviction
- Better performance under high concurrency
- Native async support

Co-authored-by: houseme <4829346+houseme@users.noreply.github.com>

* docs: add comprehensive Moka cache migration and metrics documentation

- Complete technical documentation of LRU to Moka migration
- Detailed performance comparison and benchmarks
- Comprehensive metrics catalog with 15+ Prometheus metrics
- Prometheus query examples for monitoring
- Dashboard and alerting recommendations
- Migration guide with code examples
- Troubleshooting guide for common issues
- Future enhancement roadmap

Co-authored-by: houseme <4829346+houseme@users.noreply.github.com>

* fix

* fix

* refactor: update tests for Moka cache implementation

- Completely refactor test suite to align with Moka-based concurrency.rs
- Add Clone derive to ConcurrencyManager for test convenience
- Update all tests to handle Moka's async behavior with proper delays
- Add new tests:
  * test_cache_hit_rate - validate hit rate calculation
  * test_ttl_expiration - verify TTL configuration
  * test_is_cached_no_side_effects - ensure contains doesn't affect LRU
  * bench_concurrent_cache_performance - benchmark concurrent access
- Updated existing tests:
  * test_moka_cache_operations - renamed and updated for Moka API
  * test_moka_cache_eviction - validate automatic eviction
  * test_hot_keys_tracking - improved assertions for sorted results
  * test_concurrent_cache_access - validate lock-free performance
- All tests now include appropriate sleep delays for Moka's async processing
- Enhanced documentation and assertions for better test clarity
- Total: 18 comprehensive integration tests

Co-authored-by: houseme <4829346+houseme@users.noreply.github.com>

* docs: add comprehensive Moka test suite documentation

- Complete test suite documentation for all 18 tests
- Detailed test patterns and best practices for Moka
- Debugging guide for common test failures
- Performance expectations and CI integration
- 100% API coverage validation
- Future enhancement roadmap

Co-authored-by: houseme <4829346+houseme@users.noreply.github.com>

* fix

* fmt

* docs: enhance code comments and test documentation throughout

- Significantly improve module-level documentation in concurrency.rs
- Add detailed explanations of performance characteristics and expected improvements
- Document the solution to issue #911 with before/after metrics
- Enhance constant documentation with purpose and impact explanations
- Add comprehensive RAII guard documentation with thread safety notes
- Improve test documentation with detailed test strategy and rationale
- Add Moka-specific test patterns and timing guidance
- Include performance expectations for each test category
- Document why each test matters for solving the original issue
- All documentation written in professional English
- Follow Rust documentation best practices with examples

Co-authored-by: houseme <4829346+houseme@users.noreply.github.com>

* remove lru crate

* upgrade version

* fix: resolve test errors by correcting module structure and test assertions

- Fix test import paths to use crate:: instead of rustfs:: (binary-only crate)
- Keep test file in src/storage/ instead of tests/ (no lib.rs exists)
- Add #[cfg(test)] guard to mod declaration in storage/mod.rs
- Fix Arc type annotations for Moka's ConcurrencyManager in concurrent tests
- Correct test_buffer_size_bounds assertions to match actual implementation:
  * Minimum buffer is 32KB for files <100KB, 64KB otherwise
  * Maximum buffer respects base_buffer_size when concurrency is low
  * Buffer sizing doesn't cap at file size, only at min/max constraints
- All 17 integration tests now pass successfully

Co-authored-by: houseme <4829346+houseme@users.noreply.github.com>

* fix: modify `TimeoutLayer::new` to `TimeoutLayer::with_status_code` and improve docker health check

* fix

* feat: implement cache writeback for small objects in GetObject

- Add cache writeback logic for objects meeting caching criteria:
  * No range/part request (full object retrieval)
  * Object size known and <= 10MB (max_object_size threshold)
  * Not encrypted (SSE-C or managed encryption)
- Read eligible objects into memory and cache via background task
- Serve response from in-memory data for immediate client response
- Add metrics counter for cache writeback operations
- Add 3 new tests for cache writeback functionality:
  * test_cache_writeback_flow - validates round-trip caching
  * test_cache_writeback_size_limit - ensures large objects aren't cached
  * test_cache_writeback_concurrent - validates thread-safe concurrent writes
- Update test suite documentation (now 20 comprehensive tests)

Co-authored-by: houseme <4829346+houseme@users.noreply.github.com>

* improve code for const

* cargo clippy

* feat: add cache enable/disable configuration via environment variable

- Add is_cache_enabled() method to ConcurrencyManager
- Read RUSTFS_OBJECT_CACHE_ENABLE env var (default: false) at startup
- Update ecfs.rs to check is_cache_enabled() before cache lookup and writeback
- Cache lookup and writeback now respect the enable flag
- Add test_cache_enable_configuration test
- Constants already exist in rustfs_config:
  * ENV_OBJECT_CACHE_ENABLE = "RUSTFS_OBJECT_CACHE_ENABLE"
  * DEFAULT_OBJECT_CACHE_ENABLE = false
- Total: 21 comprehensive tests passing

Co-authored-by: houseme <4829346+houseme@users.noreply.github.com>

* fix

* fmt

* fix

* fix

* feat: implement comprehensive CachedGetObject response cache with metadata

- Add CachedGetObject struct with full response metadata fields:
  * body, content_length, content_type, e_tag, last_modified
  * expires, cache_control, content_disposition, content_encoding
  * storage_class, version_id, delete_marker, tag_count, etc.
- Add dual cache architecture in HotObjectCache:
  * Legacy simple byte cache for backward compatibility
  * New response cache for complete GetObject responses
- Add ConcurrencyManager methods for response caching:
  * get_cached_object() - retrieve cached response with metadata
  * put_cached_object() - store complete response
  * invalidate_cache() - invalidate on write operations
  * invalidate_cache_versioned() - invalidate both version and latest
  * make_cache_key() - generate cache keys with version support
  * max_object_size() - get cache threshold
- Add builder pattern for CachedGetObject construction
- Add 6 new tests for response cache functionality (27 total):
  * test_cached_get_object_basic - basic operations
  * test_cached_get_object_versioned - version key handling
  * test_cache_invalidation - write operation invalidation
  * test_cache_invalidation_versioned - versioned invalidation
  * test_cached_get_object_size_limit - size enforcement
  * test_max_object_size - threshold accessor

All 27 tests pass successfully.

Co-authored-by: houseme <4829346+houseme@users.noreply.github.com>

* feat: integrate CachedGetObject cache in ecfs.rs with full metadata and cache invalidation

Integration of CachedGetObject response cache in ecfs.rs:
1. get_object: Cache lookup uses get_cached_object() with full metadata
   - Returns complete response with e_tag, last_modified, content_type, etc.
   - Parses last_modified from RFC3339 string
   - Supports versioned cache keys via make_cache_key()

2. get_object: Cache writeback uses put_cached_object() with metadata
   - Stores content_type, e_tag, last_modified in CachedGetObject
   - Background writeback via tokio::spawn()

3. Cache invalidation added to write operations:
   - put_object: invalidate_cache_versioned() after store.put_object()
   - put_object_extract: invalidate_cache_versioned() after each file extraction
   - copy_object: invalidate_cache_versioned() after store.copy_object()
   - delete_object: invalidate_cache_versioned() after store.delete_object()
   - delete_objects: invalidate_cache_versioned() for each deleted object
   - complete_multipart_upload: invalidate_cache_versioned() after completion

4. Fixed test_adaptive_buffer_sizing to be more robust with parallel tests

All 27 tests pass.

Co-authored-by: houseme <4829346+houseme@users.noreply.github.com>

* fix: add error logging for time parsing in cache lookup and writeback

- Add warning log when RFC3339 parsing fails in cache lookup
- Add warning log when time formatting fails in cache writeback
- Improves debugging for cache-related issues

Co-authored-by: houseme <4829346+houseme@users.noreply.github.com>

* fix

* fix

* upgrade version

* fmt

* add http: 4138 and add max_object_size key

* fix

* fix

* fix

* modify metrics key

* add

* upgrade crates version and improve docker observability

* feat: implement adaptive I/O strategy based on disk permit wait time

Implements adaptive I/O strategy for GetObject operations based on disk permit
wait times, as requested in issue #911.

Key changes:

1. New types in concurrency.rs:
   - IoLoadLevel enum: Low (<10ms), Medium (10-50ms), High (50-200ms), Critical (>200ms)
   - IoStrategy struct: contains optimized I/O parameters based on load level
   - IoLoadMetrics: rolling window metrics for load tracking

2. New ConcurrencyManager methods:
   - calculate_io_strategy(): calculates adaptive strategy from permit wait duration
   - record_permit_wait(): records observation for load tracking
   - smoothed_load_level(): returns averaged load level for stability
   - io_load_stats(): returns (avg_wait, p95_wait, max_wait, count) for monitoring
   - adaptive_buffer_size(): convenience method for buffer sizing

3. Integration in ecfs.rs get_object:
   - Calculate IoStrategy after acquiring disk permit
   - Use strategy buffer_size instead of static sizing
   - Consider strategy.cache_writeback_enabled in cache decision
   - Record new metrics: io.load.level gauge, io.buffer.multiplier gauge,
     io.strategy.selected counter by load level

4. New tests (4 added, 31 total):
   - test_io_load_level_classification: validates load level thresholds
   - test_io_strategy_buffer_sizing: validates buffer multipliers
   - test_calculate_io_strategy: validates manager strategy calculation
   - test_io_load_stats: validates stats tracking

All 31 tests pass. Clippy clean. Formatted.

Co-authored-by: houseme <4829346+houseme@users.noreply.github.com>

* fix

* fix

* docs: add comprehensive architecture design and fix cache hit event notification

1. Added CONCURRENCY_ARCHITECTURE.md with complete design documentation:
   - Executive summary and problem statement
   - Architecture overview with request flow diagram
   - Detailed module analysis for concurrency.rs and ecfs.rs
   - Critical analysis of helper.complete() for cache hits
   - Adaptive I/O strategy design with algorithm
   - Cache architecture with CachedGetObject structure
   - Metrics and monitoring with Prometheus queries
   - Performance characteristics and future enhancements

2. Fixed critical issue: Cache hit path now calls helper.complete()
   - S3 bucket notifications (s3:GetObject events) now trigger for cache hits
   - Event-driven workflows (Lambda, SNS) work correctly for all object access
   - Maintains audit trail for both cache hits and misses

All 31 tests pass.

Co-authored-by: houseme <4829346+houseme@users.noreply.github.com>

* fix: set object info and version_id on helper before complete() for cache hits

When serving from cache, properly configure the OperationHelper before
calling complete() to ensure S3 bucket notifications include complete
object metadata:

1. Build ObjectInfo from cached metadata:
   - bucket, name, size, actual_size
   - etag, mod_time, version_id, delete_marker
   - storage_class, content_type, content_encoding
   - user_metadata (user_defined)

2. Set helper.object(event_info).version_id(version_id_str) before complete()

3. Updated CONCURRENCY_ARCHITECTURE.md with:
   - Complete code example for cache hit event notification
   - Explanation of why ObjectInfo is required
   - Documentation of version_id handling

This ensures:
- Lambda triggers receive proper object metadata for cache hits
- SNS/SQS notifications include complete information
- Audit logs contain accurate object details
- Version-specific event routing works correctly

All 31 tests pass.

Co-authored-by: houseme <4829346+houseme@users.noreply.github.com>

* fix

* improve code

* fmt

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: houseme <4829346+houseme@users.noreply.github.com>
Co-authored-by: houseme <housemecn@gmail.com>
This commit is contained in:
Copilot
2025-11-30 01:16:55 +08:00
committed by GitHub
parent a6cf0740cb
commit fdcdb30d28
35 changed files with 6833 additions and 230 deletions

252
Cargo.lock generated
View File

@@ -237,9 +237,9 @@ checksum = "7c02d123df017efcdfbd739ef81735b36c5ba83ec3c59c80a9d7ecc718f92e50"
[[package]]
name = "arrow"
version = "57.0.0"
version = "57.1.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "4df8bb5b0bd64c0b9bc61317fcc480bad0f00e56d3bc32c69a4c8dada4786bae"
checksum = "cb372a7cbcac02a35d3fb7b3fc1f969ec078e871f9bb899bf00a2e1809bec8a3"
dependencies = [
"arrow-arith",
"arrow-array",
@@ -258,9 +258,9 @@ dependencies = [
[[package]]
name = "arrow-arith"
version = "57.0.0"
version = "57.1.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "a1a640186d3bd30a24cb42264c2dafb30e236a6f50d510e56d40b708c9582491"
checksum = "0f377dcd19e440174596d83deb49cd724886d91060c07fec4f67014ef9d54049"
dependencies = [
"arrow-array",
"arrow-buffer",
@@ -272,9 +272,9 @@ dependencies = [
[[package]]
name = "arrow-array"
version = "57.0.0"
version = "57.1.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "219fe420e6800979744c8393b687afb0252b3f8a89b91027d27887b72aa36d31"
checksum = "a23eaff85a44e9fa914660fb0d0bb00b79c4a3d888b5334adb3ea4330c84f002"
dependencies = [
"ahash",
"arrow-buffer",
@@ -291,9 +291,9 @@ dependencies = [
[[package]]
name = "arrow-buffer"
version = "57.0.0"
version = "57.1.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "76885a2697a7edf6b59577f568b456afc94ce0e2edc15b784ce3685b6c3c5c27"
checksum = "a2819d893750cb3380ab31ebdc8c68874dd4429f90fd09180f3c93538bd21626"
dependencies = [
"bytes",
"half",
@@ -303,13 +303,14 @@ dependencies = [
[[package]]
name = "arrow-cast"
version = "57.0.0"
version = "57.1.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9c9ebb4c987e6b3b236fb4a14b20b34835abfdd80acead3ccf1f9bf399e1f168"
checksum = "e3d131abb183f80c450d4591dc784f8d7750c50c6e2bc3fcaad148afc8361271"
dependencies = [
"arrow-array",
"arrow-buffer",
"arrow-data",
"arrow-ord",
"arrow-schema",
"arrow-select",
"atoi",
@@ -324,9 +325,9 @@ dependencies = [
[[package]]
name = "arrow-csv"
version = "57.0.0"
version = "57.1.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "92386159c8d4bce96f8bd396b0642a0d544d471bdc2ef34d631aec80db40a09c"
checksum = "2275877a0e5e7e7c76954669366c2aa1a829e340ab1f612e647507860906fb6b"
dependencies = [
"arrow-array",
"arrow-cast",
@@ -339,9 +340,9 @@ dependencies = [
[[package]]
name = "arrow-data"
version = "57.0.0"
version = "57.1.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "727681b95de313b600eddc2a37e736dcb21980a40f640314dcf360e2f36bc89b"
checksum = "05738f3d42cb922b9096f7786f606fcb8669260c2640df8490533bb2fa38c9d3"
dependencies = [
"arrow-buffer",
"arrow-schema",
@@ -352,9 +353,9 @@ dependencies = [
[[package]]
name = "arrow-ipc"
version = "57.0.0"
version = "57.1.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "da9ba92e3de170295c98a84e5af22e2b037f0c7b32449445e6c493b5fca27f27"
checksum = "3d09446e8076c4b3f235603d9ea7c5494e73d441b01cd61fb33d7254c11964b3"
dependencies = [
"arrow-array",
"arrow-buffer",
@@ -368,9 +369,9 @@ dependencies = [
[[package]]
name = "arrow-json"
version = "57.0.0"
version = "57.1.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b969b4a421ae83828591c6bf5450bd52e6d489584142845ad6a861f42fe35df8"
checksum = "371ffd66fa77f71d7628c63f209c9ca5341081051aa32f9c8020feb0def787c0"
dependencies = [
"arrow-array",
"arrow-buffer",
@@ -392,9 +393,9 @@ dependencies = [
[[package]]
name = "arrow-ord"
version = "57.0.0"
version = "57.1.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "141c05298b21d03e88062317a1f1a73f5ba7b6eb041b350015b1cd6aabc0519b"
checksum = "cbc94fc7adec5d1ba9e8cd1b1e8d6f72423b33fe978bf1f46d970fafab787521"
dependencies = [
"arrow-array",
"arrow-buffer",
@@ -405,9 +406,9 @@ dependencies = [
[[package]]
name = "arrow-row"
version = "57.0.0"
version = "57.1.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "c5f3c06a6abad6164508ed283c7a02151515cef3de4b4ff2cebbcaeb85533db2"
checksum = "169676f317157dc079cc5def6354d16db63d8861d61046d2f3883268ced6f99f"
dependencies = [
"arrow-array",
"arrow-buffer",
@@ -418,9 +419,9 @@ dependencies = [
[[package]]
name = "arrow-schema"
version = "57.0.0"
version = "57.1.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9cfa7a03d1eee2a4d061476e1840ad5c9867a544ca6c4c59256496af5d0a8be5"
checksum = "d27609cd7dd45f006abae27995c2729ef6f4b9361cde1ddd019dc31a5aa017e0"
dependencies = [
"serde_core",
"serde_json",
@@ -428,9 +429,9 @@ dependencies = [
[[package]]
name = "arrow-select"
version = "57.0.0"
version = "57.1.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "bafa595babaad59f2455f4957d0f26448fb472722c186739f4fac0823a1bdb47"
checksum = "ae980d021879ea119dd6e2a13912d81e64abed372d53163e804dfe84639d8010"
dependencies = [
"ahash",
"arrow-array",
@@ -442,9 +443,9 @@ dependencies = [
[[package]]
name = "arrow-string"
version = "57.0.0"
version = "57.1.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "32f46457dbbb99f2650ff3ac23e46a929e0ab81db809b02aa5511c258348bef2"
checksum = "cf35e8ef49dcf0c5f6d175edee6b8af7b45611805333129c541a8b89a0fc0534"
dependencies = [
"arrow-array",
"arrow-buffer",
@@ -611,7 +612,7 @@ dependencies = [
"bytes",
"fastrand",
"hex",
"http 1.3.1",
"http 1.4.0",
"ring",
"time",
"tokio",
@@ -703,7 +704,7 @@ dependencies = [
"hex",
"hmac 0.12.1",
"http 0.2.12",
"http 1.3.1",
"http 1.4.0",
"http-body 0.4.6",
"lru",
"percent-encoding",
@@ -797,7 +798,7 @@ dependencies = [
"hex",
"hmac 0.12.1",
"http 0.2.12",
"http 1.3.1",
"http 1.4.0",
"p256 0.11.1",
"percent-encoding",
"ring",
@@ -864,7 +865,7 @@ dependencies = [
"futures-core",
"futures-util",
"http 0.2.12",
"http 1.3.1",
"http 1.4.0",
"http-body 0.4.6",
"percent-encoding",
"pin-project-lite",
@@ -884,7 +885,7 @@ dependencies = [
"h2 0.3.27",
"h2 0.4.12",
"http 0.2.12",
"http 1.3.1",
"http 1.4.0",
"http-body 0.4.6",
"hyper 0.14.32",
"hyper 1.8.1",
@@ -945,7 +946,7 @@ dependencies = [
"bytes",
"fastrand",
"http 0.2.12",
"http 1.3.1",
"http 1.4.0",
"http-body 0.4.6",
"http-body 1.0.1",
"pin-project-lite",
@@ -964,7 +965,7 @@ dependencies = [
"aws-smithy-types",
"bytes",
"http 0.2.12",
"http 1.3.1",
"http 1.4.0",
"pin-project-lite",
"tokio",
"tracing",
@@ -982,7 +983,7 @@ dependencies = [
"bytes-utils",
"futures-core",
"http 0.2.12",
"http 1.3.1",
"http 1.4.0",
"http-body 0.4.6",
"http-body 1.0.1",
"http-body-util",
@@ -1030,7 +1031,7 @@ dependencies = [
"bytes",
"form_urlencoded",
"futures-util",
"http 1.3.1",
"http 1.4.0",
"http-body 1.0.1",
"http-body-util",
"hyper 1.8.1",
@@ -1061,7 +1062,7 @@ checksum = "59446ce19cd142f8833f856eb31f3eb097812d1479ab224f54d72428ca21ea22"
dependencies = [
"bytes",
"futures-core",
"http 1.3.1",
"http 1.4.0",
"http-body 1.0.1",
"http-body-util",
"mime",
@@ -1083,7 +1084,7 @@ dependencies = [
"bytes",
"futures-core",
"futures-util",
"http 1.3.1",
"http 1.4.0",
"http-body 1.0.1",
"http-body-util",
"mime",
@@ -1102,7 +1103,7 @@ dependencies = [
"arc-swap",
"bytes",
"fs-err",
"http 1.3.1",
"http 1.4.0",
"http-body 1.0.1",
"hyper 1.8.1",
"hyper-util",
@@ -1370,9 +1371,9 @@ dependencies = [
[[package]]
name = "bytesize"
version = "2.3.0"
version = "2.3.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "00f4369ba008f82b968b1acbe31715ec37bd45236fa0726605a36cc3060ea256"
checksum = "6bd91ee7b2422bcb158d90ef4d14f75ef67f340943fc4149891dcce8f8b972a3"
[[package]]
name = "bytestring"
@@ -1468,9 +1469,9 @@ checksum = "37b2a672a2cb129a2e41c10b1224bb368f9f37a2b16b612598138befd7b37eb5"
[[package]]
name = "cc"
version = "1.2.47"
version = "1.2.48"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "cd405d82c84ff7f35739f175f67d8b9fb7687a0e84ccdc78bd3568839827cf07"
checksum = "c481bdbf0ed3b892f6f806287d72acd515b352a4ec27a208489b8c1bc839633a"
dependencies = [
"find-msvc-tools",
"jobserver",
@@ -1793,9 +1794,9 @@ dependencies = [
[[package]]
name = "crc"
version = "3.3.0"
version = "3.4.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9710d3b3739c2e349eb44fe848ad0b7c8cb1e42bd87ee49371df2f7acaf3e675"
checksum = "5eb8a2a1cd12ab0d987a5d5e825195d372001a4094a0376319d5a0ad71c1ba0d"
dependencies = [
"crc-catalog",
]
@@ -3632,7 +3633,7 @@ dependencies = [
"base64 0.22.1",
"bon",
"google-cloud-gax",
"http 1.3.1",
"http 1.4.0",
"jsonwebtoken",
"reqwest",
"rustc_version",
@@ -3656,7 +3657,7 @@ dependencies = [
"futures",
"google-cloud-rpc",
"google-cloud-wkt",
"http 1.3.1",
"http 1.4.0",
"pin-project",
"rand 0.9.2",
"serde",
@@ -3676,7 +3677,7 @@ dependencies = [
"google-cloud-gax",
"google-cloud-rpc",
"google-cloud-wkt",
"http 1.3.1",
"http 1.4.0",
"http-body-util",
"hyper 1.8.1",
"opentelemetry-semantic-conventions",
@@ -3783,7 +3784,7 @@ dependencies = [
"google-cloud-rpc",
"google-cloud-type",
"google-cloud-wkt",
"http 1.3.1",
"http 1.4.0",
"http-body 1.0.1",
"hyper 1.8.1",
"lazy_static",
@@ -3887,7 +3888,7 @@ dependencies = [
"fnv",
"futures-core",
"futures-sink",
"http 1.3.1",
"http 1.4.0",
"indexmap 2.12.1",
"slab",
"tokio",
@@ -4088,12 +4089,11 @@ dependencies = [
[[package]]
name = "http"
version = "1.3.1"
version = "1.4.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f4a85d31aea989eead29a3aaf9e1115a180df8282431156e533de47660892565"
checksum = "e3ba2a386d7f85a81f119ad7498ebe444d2e22c2af0b86b069416ace48b3311a"
dependencies = [
"bytes",
"fnv",
"itoa",
]
@@ -4115,7 +4115,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "1efedce1fb8e6913f23e0c92de8e62cd5b772a67e7b3946df930a62566c93184"
dependencies = [
"bytes",
"http 1.3.1",
"http 1.4.0",
]
[[package]]
@@ -4126,7 +4126,7 @@ checksum = "b021d93e26becf5dc7e1b75b1bed1fd93124b374ceb73f43d4d4eafec896a64a"
dependencies = [
"bytes",
"futures-core",
"http 1.3.1",
"http 1.4.0",
"http-body 1.0.1",
"pin-project-lite",
]
@@ -4193,7 +4193,7 @@ dependencies = [
"futures-channel",
"futures-core",
"h2 0.4.12",
"http 1.3.1",
"http 1.4.0",
"http-body 1.0.1",
"httparse",
"httpdate",
@@ -4227,7 +4227,7 @@ version = "0.27.7"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e3c93eb611681b207e1fe55d5a71ecf91572ec8a6705cdb6857f7d8d5242cf58"
dependencies = [
"http 1.3.1",
"http 1.4.0",
"hyper 1.8.1",
"hyper-util",
"log",
@@ -4264,7 +4264,7 @@ dependencies = [
"futures-channel",
"futures-core",
"futures-util",
"http 1.3.1",
"http 1.4.0",
"http-body 1.0.1",
"hyper 1.8.1",
"ipnet",
@@ -4619,9 +4619,9 @@ dependencies = [
[[package]]
name = "js-sys"
version = "0.3.82"
version = "0.3.83"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b011eec8cc36da2aab2d5cff675ec18454fad408585853910a202391cf9f8e65"
checksum = "464a3709c7f55f1f721e5389aa6ea4e3bc6aba669353300af094b29ffbdde1d8"
dependencies = [
"once_cell",
"wasm-bindgen",
@@ -4908,9 +4908,9 @@ dependencies = [
[[package]]
name = "lz4_flex"
version = "0.11.5"
version = "0.12.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "08ab2867e3eeeca90e844d1940eab391c9dc5228783db2ed999acbc0a9ed375a"
checksum = "ab6473172471198271ff72e9379150e9dfd70d8e533e0752a27e515b48dd375e"
dependencies = [
"twox-hash",
]
@@ -5022,9 +5022,9 @@ dependencies = [
[[package]]
name = "metrics"
version = "0.24.2"
version = "0.24.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "25dea7ac8057892855ec285c440160265225438c3c45072613c25a4b26e98ef5"
checksum = "5d5312e9ba3771cfa961b585728215e3d972c950a3eed9252aa093d6301277e8"
dependencies = [
"ahash",
"portable-atomic",
@@ -5439,7 +5439,7 @@ dependencies = [
"bytes",
"chrono",
"futures",
"http 1.3.1",
"http 1.4.0",
"humantime",
"itertools 0.14.0",
"parking_lot",
@@ -5513,7 +5513,7 @@ checksum = "d7a6d09a73194e6b66df7c8f1b680f156d916a1a942abf2de06823dd02b7855d"
dependencies = [
"async-trait",
"bytes",
"http 1.3.1",
"http 1.4.0",
"opentelemetry",
"reqwest",
]
@@ -5524,7 +5524,8 @@ version = "0.31.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "7a2366db2dca4d2ad033cad11e6ee42844fd727007af5ad04a1730f4cb8163bf"
dependencies = [
"http 1.3.1",
"flate2",
"http 1.4.0",
"opentelemetry",
"opentelemetry-http",
"opentelemetry-proto",
@@ -5533,7 +5534,6 @@ dependencies = [
"reqwest",
"thiserror 2.0.17",
"tracing",
"zstd",
]
[[package]]
@@ -5680,9 +5680,9 @@ dependencies = [
[[package]]
name = "parquet"
version = "57.0.0"
version = "57.1.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "7a0f31027ef1af7549f7cec603a9a21dce706d3f8d7c2060a68f43c1773be95a"
checksum = "be3e4f6d320dd92bfa7d612e265d7d08bba0a240bab86af3425e1d255a511d89"
dependencies = [
"ahash",
"arrow-array",
@@ -6642,7 +6642,7 @@ dependencies = [
"futures-core",
"futures-util",
"h2 0.4.12",
"http 1.3.1",
"http 1.4.0",
"http-body 1.0.1",
"http-body-util",
"hyper 1.8.1",
@@ -6656,6 +6656,7 @@ dependencies = [
"pin-project-lite",
"quinn",
"rustls 0.23.35",
"rustls-native-certs 0.8.2",
"rustls-pki-types",
"serde",
"serde_json",
@@ -6721,9 +6722,9 @@ dependencies = [
[[package]]
name = "rmcp"
version = "0.9.0"
version = "0.9.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "acc36ea743d4bbc97e9f3c33bf0b97765a5cf338de3d9c3d2f321a6e38095615"
checksum = "eaa07b85b779d1e1df52dd79f6c6bffbe005b191f07290136cc42a142da3409a"
dependencies = [
"async-trait",
"base64 0.22.1",
@@ -6743,9 +6744,9 @@ dependencies = [
[[package]]
name = "rmcp-macros"
version = "0.9.0"
version = "0.9.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "263caba1c96f2941efca0fdcd97b03f42bcde52d2347d05e5d77c93ab18c5b58"
checksum = "0f6fa09933cac0d0204c8a5d647f558425538ed6a0134b1ebb1ae4dc00c96db3"
dependencies = [
"darling 0.21.3",
"proc-macro2",
@@ -6943,7 +6944,7 @@ dependencies = [
"futures",
"futures-util",
"hex-simd",
"http 1.3.1",
"http 1.4.0",
"http-body 1.0.1",
"hyper 1.8.1",
"hyper-util",
@@ -6954,6 +6955,7 @@ dependencies = [
"metrics",
"mimalloc",
"mime_guess",
"moka",
"pin-project-lite",
"pprof",
"reqwest",
@@ -7076,7 +7078,7 @@ dependencies = [
"base64-simd",
"bytes",
"crc-fast",
"http 1.3.1",
"http 1.4.0",
"md-5 0.11.0-rc.3",
"pretty_assertions",
"sha1 0.11.0-rc.3",
@@ -7152,7 +7154,7 @@ dependencies = [
"google-cloud-storage",
"hex-simd",
"hmac 0.13.0-rc.3",
"http 1.3.1",
"http 1.4.0",
"hyper 1.8.1",
"hyper-rustls 0.27.7",
"hyper-util",
@@ -7437,7 +7439,7 @@ dependencies = [
"faster-hex",
"futures",
"hex-simd",
"http 1.3.1",
"http 1.4.0",
"md-5 0.11.0-rc.3",
"pin-project-lite",
"rand 0.10.0-rc.5",
@@ -7465,7 +7467,7 @@ dependencies = [
"datafusion",
"futures",
"futures-core",
"http 1.3.1",
"http 1.4.0",
"object_store",
"pin-project-lite",
"rustfs-common",
@@ -7502,7 +7504,7 @@ version = "0.0.5"
dependencies = [
"base64-simd",
"bytes",
"http 1.3.1",
"http 1.4.0",
"hyper 1.8.1",
"rustfs-utils",
"s3s",
@@ -7547,7 +7549,7 @@ dependencies = [
"hex-simd",
"highway",
"hmac 0.13.0-rc.3",
"http 1.3.1",
"http 1.4.0",
"hyper 1.8.1",
"libc",
"local-ip-address",
@@ -7605,7 +7607,7 @@ dependencies = [
"anyhow",
"async-trait",
"bytes",
"http 1.3.1",
"http 1.4.0",
"reqwest",
"rustify_derive",
"serde",
@@ -7728,9 +7730,9 @@ dependencies = [
[[package]]
name = "rustls-pki-types"
version = "1.13.0"
version = "1.13.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "94182ad936a0c91c324cd46c6511b9510ed16af436d7b5bab34beab0afd55f7a"
checksum = "708c0f9d5f54ba0272468c1d306a52c495b31fa155e91bc25371e6df7996908c"
dependencies = [
"web-time",
"zeroize",
@@ -7800,7 +7802,7 @@ dependencies = [
"futures",
"hex-simd",
"hmac 0.13.0-rc.3",
"http 1.3.1",
"http 1.4.0",
"http-body 1.0.1",
"http-body-util",
"httparse",
@@ -8117,9 +8119,9 @@ dependencies = [
[[package]]
name = "serde_with"
version = "3.16.0"
version = "3.16.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "10574371d41b0d9b2cff89418eda27da52bcaff2cc8741db26382a77c29131f1"
checksum = "4fa237f2807440d238e0364a218270b98f767a00d3dada77b1c53ae88940e2e7"
dependencies = [
"base64 0.22.1",
"chrono",
@@ -8136,9 +8138,9 @@ dependencies = [
[[package]]
name = "serde_with_macros"
version = "3.16.0"
version = "3.16.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "08a72d8216842fdd57820dc78d840bef99248e35fb2554ff923319e60f2d686b"
checksum = "52a8e3ca0ca629121f70ab50f95249e5a6f925cc0f6ffe8256c45b728875706c"
dependencies = [
"darling 0.21.3",
"proc-macro2",
@@ -9181,7 +9183,7 @@ dependencies = [
"bytes",
"flate2",
"h2 0.4.12",
"http 1.3.1",
"http 1.4.0",
"http-body 1.0.1",
"http-body-util",
"hyper 1.8.1",
@@ -9261,16 +9263,16 @@ dependencies = [
[[package]]
name = "tower-http"
version = "0.6.6"
version = "0.6.7"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "adc82fd73de2a9722ac5da747f12383d2bfdb93591ee6c58486e0097890f05f2"
checksum = "9cf146f99d442e8e68e585f5d798ccd3cad9a7835b917e09728880a862706456"
dependencies = [
"async-compression",
"bitflags 2.10.0",
"bytes",
"futures-core",
"futures-util",
"http 1.3.1",
"http 1.4.0",
"http-body 1.0.1",
"http-body-util",
"iri-string",
@@ -9298,9 +9300,9 @@ checksum = "8df9b6e13f2d32c91b9bd719c00d1958837bc7dec474d94952798cc8e69eeec3"
[[package]]
name = "tracing"
version = "0.1.41"
version = "0.1.43"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "784e0ac535deb450455cbfa28a6f0df145ea1bb7ae51b821cf5e7927fdcfbdd0"
checksum = "2d15d90a0b5c19378952d479dc858407149d7bb45a14de0142f6c534b16fc647"
dependencies = [
"log",
"pin-project-lite",
@@ -9310,21 +9312,21 @@ dependencies = [
[[package]]
name = "tracing-appender"
version = "0.2.3"
version = "0.2.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "3566e8ce28cc0a3fe42519fc80e6b4c943cc4c8cef275620eb8dac2d3d4e06cf"
checksum = "786d480bce6247ab75f005b14ae1624ad978d3029d9113f0a22fa1ac773faeaf"
dependencies = [
"crossbeam-channel",
"thiserror 1.0.69",
"thiserror 2.0.17",
"time",
"tracing-subscriber",
]
[[package]]
name = "tracing-attributes"
version = "0.1.30"
version = "0.1.31"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "81383ab64e72a7a8b8e13130c49e3dab29def6d0c7d76a03087b3cf71c5c6903"
checksum = "7490cfa5ec963746568740651ac6781f701c9c5ea257c58e057f3ba8cf69e8da"
dependencies = [
"proc-macro2",
"quote",
@@ -9333,9 +9335,9 @@ dependencies = [
[[package]]
name = "tracing-core"
version = "0.1.34"
version = "0.1.35"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b9d12581f227e93f094d3af2ae690a574abb8a2b9b7a96e7cfe9647b2b617678"
checksum = "7a04e24fab5c89c6a36eb8558c9656f30d81de51dfa4d3b45f26b21d61fa0a6c"
dependencies = [
"once_cell",
"valuable",
@@ -9393,9 +9395,9 @@ dependencies = [
[[package]]
name = "tracing-subscriber"
version = "0.3.20"
version = "0.3.22"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "2054a14f5307d601f88daf0553e1cbf472acc4f2c51afab632431cdcd72124d5"
checksum = "2f30143827ddab0d256fd843b7a66d164e9f271cfa0dde49142c5ca0ca291f1e"
dependencies = [
"matchers",
"nu-ansi-term",
@@ -9624,7 +9626,7 @@ dependencies = [
"async-trait",
"bytes",
"derive_builder 0.12.0",
"http 1.3.1",
"http 1.4.0",
"reqwest",
"rustify",
"rustify_derive",
@@ -9683,9 +9685,9 @@ dependencies = [
[[package]]
name = "wasm-bindgen"
version = "0.2.105"
version = "0.2.106"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "da95793dfc411fbbd93f5be7715b0578ec61fe87cb1a42b12eb625caa5c5ea60"
checksum = "0d759f433fa64a2d763d1340820e46e111a7a5ab75f993d1852d70b03dbb80fd"
dependencies = [
"cfg-if",
"once_cell",
@@ -9696,9 +9698,9 @@ dependencies = [
[[package]]
name = "wasm-bindgen-futures"
version = "0.4.55"
version = "0.4.56"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "551f88106c6d5e7ccc7cd9a16f312dd3b5d36ea8b4954304657d5dfba115d4a0"
checksum = "836d9622d604feee9e5de25ac10e3ea5f2d65b41eac0d9ce72eb5deae707ce7c"
dependencies = [
"cfg-if",
"js-sys",
@@ -9709,9 +9711,9 @@ dependencies = [
[[package]]
name = "wasm-bindgen-macro"
version = "0.2.105"
version = "0.2.106"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "04264334509e04a7bf8690f2384ef5265f05143a4bff3889ab7a3269adab59c2"
checksum = "48cb0d2638f8baedbc542ed444afc0644a29166f1595371af4fecf8ce1e7eeb3"
dependencies = [
"quote",
"wasm-bindgen-macro-support",
@@ -9719,9 +9721,9 @@ dependencies = [
[[package]]
name = "wasm-bindgen-macro-support"
version = "0.2.105"
version = "0.2.106"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "420bc339d9f322e562942d52e115d57e950d12d88983a14c79b86859ee6c7ebc"
checksum = "cefb59d5cd5f92d9dcf80e4683949f15ca4b511f4ac0a6e14d4e1ac60c6ecd40"
dependencies = [
"bumpalo",
"proc-macro2",
@@ -9732,9 +9734,9 @@ dependencies = [
[[package]]
name = "wasm-bindgen-shared"
version = "0.2.105"
version = "0.2.106"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "76f218a38c84bcb33c25ec7059b07847d465ce0e0a76b995e134a45adcb6af76"
checksum = "cbc538057e648b67f72a982e708d485b2efa771e1ac05fec311f9f63e5800db4"
dependencies = [
"unicode-ident",
]
@@ -9754,9 +9756,9 @@ dependencies = [
[[package]]
name = "web-sys"
version = "0.3.82"
version = "0.3.83"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "3a1f95c0d03a47f4ae1f7a64643a6bb97465d9b740f0fa8f90ea33915c99a9a1"
checksum = "9b32828d774c412041098d182a8b38b16ea816958e07cf40eec2bc080ae137ac"
dependencies = [
"js-sys",
"wasm-bindgen",
@@ -10159,9 +10161,9 @@ checksum = "d6bbff5f0aada427a1e5a6da5f1f98158182f26556f345ac9e04d36d0ebed650"
[[package]]
name = "winnow"
version = "0.7.13"
version = "0.7.14"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "21a0236b59786fed61e2a80582dd500fe61f18b5dca67a4a067d0bc9039339cf"
checksum = "5a5364e9d77fcdeeaa6062ced926ee3381faa2ee02d3eb83a5c27a8825540829"
dependencies = [
"memchr",
]
@@ -10252,18 +10254,18 @@ dependencies = [
[[package]]
name = "zerocopy"
version = "0.8.28"
version = "0.8.30"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "43fa6694ed34d6e57407afbccdeecfa268c470a7d2a5b0cf49ce9fcc345afb90"
checksum = "4ea879c944afe8a2b25fef16bb4ba234f47c694565e97383b36f3a878219065c"
dependencies = [
"zerocopy-derive",
]
[[package]]
name = "zerocopy-derive"
version = "0.8.28"
version = "0.8.30"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "c640b22cd9817fae95be82f0d2f90b11f7605f6c319d16705c459b27ac2cbc26"
checksum = "cf955aa904d6040f70dc8e9384444cb1030aed272ba3cb09bbc4ab9e7c1f34f5"
dependencies = [
"proc-macro2",
"quote",