docs(.docker): add bilingual README for OpenObserve+OpenTelemetry setup

- Create English and Chinese README files for the openobserve-otel directory
- Document configuration details for both OpenObserve and OTel Collector
- Include setup instructions and application integration examples
- Add badges for both OpenObserve and OpenTelemetry projects
This commit is contained in:
houseme
2025-05-29 23:48:11 +08:00
parent 5e13fc3be2
commit 2f20fe9749
12 changed files with 186 additions and 204 deletions

View File

@@ -0,0 +1,75 @@
# OpenObserve + OpenTelemetry Collector
[![OpenObserve](https://img.shields.io/badge/OpenObserve-OpenSource-blue.svg)](https://openobserve.org)
[![OpenTelemetry](https://img.shields.io/badge/OpenTelemetry-Collector-green.svg)](https://opentelemetry.io/)
English | [中文](README_ZH.md)
This directory contains the configuration files for setting up an observability stack with OpenObserve and OpenTelemetry
Collector.
### Overview
This setup provides a complete observability solution for your applications:
- **OpenObserve**: A modern, open-source observability platform for logs, metrics, and traces.
- **OpenTelemetry Collector**: Collects and processes telemetry data before sending it to OpenObserve.
### Setup Instructions
1. **Prerequisites**:
- Docker and Docker Compose installed
- Sufficient memory resources (minimum 2GB recommended)
2. **Starting the Services**:
```bash
cd .docker/openobserve-otel
docker compose -f docker-compose.yml up -d
```
3. **Accessing the Dashboard**:
- OpenObserve UI: http://localhost:5080
- Default credentials:
- Username: root@rustfs.com
- Password: rustfs123
### Configuration
#### OpenObserve Configuration
The OpenObserve service is configured with:
- Root user credentials
- Data persistence through a volume mount
- Memory cache enabled
- Health checks
- Exposed ports:
- 5080: HTTP API and UI
- 5081: OTLP gRPC
#### OpenTelemetry Collector Configuration
The collector is configured to:
- Receive telemetry data via OTLP (HTTP and gRPC)
- Collect logs from files
- Process data in batches
- Export data to OpenObserve
- Manage memory usage
### Integration with Your Application
To send telemetry data from your application, configure your OpenTelemetry SDK to send data to:
- OTLP gRPC: `localhost:4317`
- OTLP HTTP: `localhost:4318`
For example, in a Rust application using the `rustfs-obs` library:
```bash
export RUSTFS_OBS_ENDPOINT=http://localhost:4317
export RUSTFS_OBS_SERVICE_NAME=yourservice
export RUSTFS_OBS_SERVICE_VERSION=1.0.0
export RUSTFS_OBS_ENVIRONMENT=development
```

View File

@@ -0,0 +1,75 @@
# OpenObserve + OpenTelemetry Collector
[![OpenObserve](https://img.shields.io/badge/OpenObserve-OpenSource-blue.svg)](https://openobserve.org)
[![OpenTelemetry](https://img.shields.io/badge/OpenTelemetry-Collector-green.svg)](https://opentelemetry.io/)
[English](README.md) | 中文
## 中文
本目录包含搭建 OpenObserve 和 OpenTelemetry Collector 可观测性栈的配置文件。
### 概述
此设置为应用程序提供了完整的可观测性解决方案:
- **OpenObserve**:现代化、开源的可观测性平台,用于日志、指标和追踪。
- **OpenTelemetry Collector**:收集和处理遥测数据,然后将其发送到 OpenObserve。
### 设置说明
1. **前提条件**
- 已安装 Docker 和 Docker Compose
- 足够的内存资源(建议至少 2GB
2. **启动服务**
```bash
cd .docker/openobserve-otel
docker compose -f docker-compose.yml up -d
```
3. **访问仪表板**
- OpenObserve UIhttp://localhost:5080
- 默认凭据:
- 用户名root@rustfs.com
- 密码rustfs123
### 配置
#### OpenObserve 配置
OpenObserve 服务配置:
- 根用户凭据
- 通过卷挂载实现数据持久化
- 启用内存缓存
- 健康检查
- 暴露端口:
- 5080HTTP API 和 UI
- 5081OTLP gRPC
#### OpenTelemetry Collector 配置
收集器配置为:
- 通过 OTLPHTTP 和 gRPC接收遥测数据
- 从文件中收集日志
- 批处理数据
- 将数据导出到 OpenObserve
- 管理内存使用
### 与应用程序集成
要从应用程序发送遥测数据,将 OpenTelemetry SDK 配置为发送数据到:
- OTLP gRPC:`localhost:4317`
- OTLP HTTP:`localhost:4318`
例如,在使用 `rustfs-obs` 库的 Rust 应用程序中:
```bash
export RUSTFS_OBS_ENDPOINT=http://localhost:4317
export RUSTFS_OBS_SERVICE_NAME=yourservice
export RUSTFS_OBS_SERVICE_VERSION=1.0.0
export RUSTFS_OBS_ENVIRONMENT=development
```

View File

@@ -60,7 +60,7 @@ export RUSTFS_CONSOLE_ENABLE=true
export RUSTFS_CONSOLE_ADDRESS="0.0.0.0:9001"
# Observability config
export RUSTFS_OBS_CONFIG="./deploy/config/obs.toml"
export RUSTFS_OBS_ENDPOINT="http://localhost:4317"
# Event message configuration
#export RUSTFS_EVENT_CONFIG="./deploy/config/event.toml"
@@ -92,25 +92,3 @@ export RUSTFS_OBS_CONFIG="./deploy/config/obs.toml"
- Grafana: `http://localhost:3000` (credentials: `admin`/`admin`)
- Jaeger: `http://localhost:16686`
- Prometheus: `http://localhost:9090`
#### Configuring Observability
1. Copy the example configuration:
```bash
cd deploy/config
cp obs.toml.example obs.toml
```
2. Edit `obs.toml` with the following parameters:
| Parameter | Description | Example |
|----------------------|-----------------------------------|-----------------------|
| endpoint | OpenTelemetry Collector address | http://localhost:4317 |
| service_name | Service name | rustfs |
| service_version | Service version | 1.0.0 |
| environment | Runtime environment | production |
| meter_interval | Metrics export interval (seconds) | 30 |
| sample_ratio | Sampling ratio | 1.0 |
| use_stdout | Output to console | true/false |
| logger_level | Log level | info |
| local_logging_enable | stdout | true/false |

View File

@@ -60,7 +60,7 @@ export RUSTFS_CONSOLE_ENABLE=true
export RUSTFS_CONSOLE_ADDRESS="0.0.0.0:9001"
# 可观测性配置
export RUSTFS_OBS_CONFIG="./deploy/config/obs.toml"
export RUSTFS_OBS_ENDPOINT="http://localhost:4317"
# 事件消息配置
#export RUSTFS_EVENT_CONFIG="./deploy/config/event.toml"
@@ -94,24 +94,6 @@ export RUSTFS_OBS_CONFIG="./deploy/config/obs.toml"
#### 配置可观测性
1. 复制示例配置:
```bash
cd deploy/config
cp obs.toml.example obs.toml
```
2. 编辑 `obs.toml` 配置文件,参数如下:
| 配置项 | 说明 | 示例值 |
|----------------------|----------------------------|-----------------------|
| endpoint | OpenTelemetry Collector 地址 | http://localhost:4317 |
| service_name | 服务名称 | rustfs |
| service_version | 服务版本 | 1.0.0 |
| environment | 运行环境 | production |
| meter_interval | 指标导出间隔 (秒) | 30 |
| sample_ratio | 采样率 | 1.0 |
| use_stdout | 是否输出到控制台 | true/false |
| logger_level | 日志级别 | info |
| local_logging_enable | 控制台是否答应日志 | true/false |
```
OpenTelemetry Collector 地址(endpoint): http://localhost:4317
```

View File

@@ -37,14 +37,14 @@ impl OtelConfig {
pub fn extract_otel_config_from_env(endpoint: Option<String>) -> OtelConfig {
let endpoint = if let Some(endpoint) = endpoint {
if endpoint.is_empty() {
env::var("RUSTFS_OBSERVABILITY_ENDPOINT").unwrap_or_else(|_| "".to_string())
env::var("RUSTFS_OBS_ENDPOINT").unwrap_or_else(|_| "".to_string())
} else {
endpoint
}
} else {
env::var("RUSTFS_OBSERVABILITY_ENDPOINT").unwrap_or_else(|_| "".to_string())
env::var("RUSTFS_OBS_ENDPOINT").unwrap_or_else(|_| "".to_string())
};
let mut use_stdout = env::var("RUSTFS_OBSERVABILITY_USE_STDOUT")
let mut use_stdout = env::var("RUSTFS_OBS_USE_STDOUT")
.ok()
.and_then(|v| v.parse().ok())
.or(Some(USE_STDOUT));
@@ -55,51 +55,51 @@ impl OtelConfig {
OtelConfig {
endpoint,
use_stdout,
sample_ratio: env::var("RUSTFS_OBSERVABILITY_SAMPLE_RATIO")
sample_ratio: env::var("RUSTFS_OBS_SAMPLE_RATIO")
.ok()
.and_then(|v| v.parse().ok())
.or(Some(SAMPLE_RATIO)),
meter_interval: env::var("RUSTFS_OBSERVABILITY_METER_INTERVAL")
meter_interval: env::var("RUSTFS_OBS_METER_INTERVAL")
.ok()
.and_then(|v| v.parse().ok())
.or(Some(METER_INTERVAL)),
service_name: env::var("RUSTFS_OBSERVABILITY_SERVICE_NAME")
service_name: env::var("RUSTFS_OBS_SERVICE_NAME")
.ok()
.and_then(|v| v.parse().ok())
.or(Some(APP_NAME.to_string())),
service_version: env::var("RUSTFS_OBSERVABILITY_SERVICE_VERSION")
service_version: env::var("RUSTFS_OBS_SERVICE_VERSION")
.ok()
.and_then(|v| v.parse().ok())
.or(Some(SERVICE_VERSION.to_string())),
environment: env::var("RUSTFS_OBSERVABILITY_ENVIRONMENT")
environment: env::var("RUSTFS_OBS_ENVIRONMENT")
.ok()
.and_then(|v| v.parse().ok())
.or(Some(ENVIRONMENT.to_string())),
logger_level: env::var("RUSTFS_OBSERVABILITY_LOGGER_LEVEL")
logger_level: env::var("RUSTFS_OBS_LOGGER_LEVEL")
.ok()
.and_then(|v| v.parse().ok())
.or(Some(DEFAULT_LOG_LEVEL.to_string())),
local_logging_enabled: env::var("RUSTFS_OBSERVABILITY_LOCAL_LOGGING_ENABLED")
local_logging_enabled: env::var("RUSTFS_OBS_LOCAL_LOGGING_ENABLED")
.ok()
.and_then(|v| v.parse().ok())
.or(Some(false)),
log_directory: env::var("RUSTFS_OBSERVABILITY_LOG_DIRECTORY")
log_directory: env::var("RUSTFS_OBS_LOG_DIRECTORY")
.ok()
.and_then(|v| v.parse().ok())
.or(Some(DEFAULT_LOG_DIR.to_string())),
log_filename: env::var("RUSTFS_OBSERVABILITY_LOG_FILENAME")
log_filename: env::var("RUSTFS_OBS_LOG_FILENAME")
.ok()
.and_then(|v| v.parse().ok())
.or(Some(DEFAULT_LOG_FILENAME.to_string())),
log_rotation_size_mb: env::var("RUSTFS_OBSERVABILITY_LOG_ROTATION_SIZE_MB")
log_rotation_size_mb: env::var("RUSTFS_OBS_LOG_ROTATION_SIZE_MB")
.ok()
.and_then(|v| v.parse().ok())
.or(Some(DEFAULT_LOG_ROTATION_SIZE_MB)), // Default to 100 MB
log_rotation_time: env::var("RUSTFS_OBSERVABILITY_LOG_ROTATION_TIME")
log_rotation_time: env::var("RUSTFS_OBS_LOG_ROTATION_TIME")
.ok()
.and_then(|v| v.parse().ok())
.or(Some(DEFAULT_LOG_ROTATION_TIME.to_string())), // Default to "Day"
log_keep_files: env::var("RUSTFS_OBSERVABILITY_LOG_KEEP_FILES")
log_keep_files: env::var("RUSTFS_OBS_LOG_KEEP_FILES")
.ok()
.and_then(|v| v.parse().ok())
.or(Some(DEFAULT_LOG_KEEP_FILES)), // Default to keeping 30 log files

View File

@@ -1,28 +0,0 @@
#RUSTFS__OBSERVABILITY__ENDPOINT=http://localhost:4317
#RUSTFS__OBSERVABILITY__USE_STDOUT=true
#RUSTFS__OBSERVABILITY__SAMPLE_RATIO=2.0
#RUSTFS__OBSERVABILITY__METER_INTERVAL=30
#RUSTFS__OBSERVABILITY__SERVICE_NAME=rustfs
#RUSTFS__OBSERVABILITY__SERVICE_VERSION=0.1.0
#RUSTFS__OBSERVABILITY__ENVIRONMENT=develop
#RUSTFS__OBSERVABILITY__LOGGER_LEVEL=debug
#
#RUSTFS__SINKS_0__type=Kakfa
#RUSTFS__SINKS_0__brokers=localhost:9092
#RUSTFS__SINKS_0__topic=logs
#RUSTFS__SINKS_0__batch_size=100
#RUSTFS__SINKS_0__batch_timeout_ms=1000
#
#RUSTFS__SINKS_1__type=Webhook
#RUSTFS__SINKS_1__endpoint=http://localhost:8080/webhook
#RUSTFS__SINKS_1__auth_token=you-auth-token
#RUSTFS__SINKS_1__batch_size=100
#RUSTFS__SINKS_1__batch_timeout_ms=1000
#
#RUSTFS__SINKS_2__type=File
#RUSTFS__SINKS_2__path=./deploy/logs/rustfs.log
#RUSTFS__SINKS_2__buffer_size=10
#RUSTFS__SINKS_2__flush_interval_ms=1000
#RUSTFS__SINKS_2__flush_threshold=100
#
#RUSTFS__LOGGER__QUEUE_CAPACITY=10

View File

@@ -1,29 +0,0 @@
# config.toml
store_path = "./deploy/logs/event_store"
channel_capacity = 5000
[[adapters]]
type = "Webhook"
endpoint = "http://127.0.0.1:3020/webhook"
auth_token = "your-auth-token"
max_retries = 3
timeout = 50
[adapters.custom_headers]
custom_server = "value_server"
custom_client = "value_client"
#[[adapters]]
#type = "Kafka"
#brokers = "localhost:9092"
#topic = "notifications"
#max_retries = 3
#timeout = 60
#
#[[adapters]]
#type = "Mqtt"
#broker = "mqtt.example.com"
#port = 1883
#client_id = "event-notifier"
#topic = "events"
#max_retries = 3

View File

@@ -1,34 +0,0 @@
[observability]
endpoint = "http://localhost:4317" # 可观测性数据上报的终端地址,默认为"http://localhost:4317"
use_stdout = true # 是否将日志输出到标准输出
sample_ratio = 2.0 # 采样率,表示每 2 条数据采样 1 条
meter_interval = 30 # 指标收集间隔,单位为秒
service_name = "rustfs" # 服务名称,用于标识当前服务
service_version = "0.1.0" # 服务版本号
environments = "develop" # 运行环境,如开发环境 (develop)
logger_level = "debug" # 日志级别,可选 debug/info/warn/error 等
local_logging_enabled = true # 是否启用本地 stdout 日志记录true 表示启用false 表示禁用
#[[sinks]] # Kafka 接收器配置
#type = "Kafka" # 是否启用 Kafka 接收器,默认禁用
#brokers = "localhost:9092" # Kafka 服务器地址
#topic = "logs" # Kafka 主题名称
#batch_size = 100 # 批处理大小,每次发送的消息数量
#batch_timeout_ms = 1000 # 批处理超时时间,单位为毫秒
#[[sinks]] # Webhook 接收器配置
#type = "Webhook" # 是否启用 Webhook 接收器
#endpoint = "http://localhost:8080/webhook" # Webhook 接收地址
#auth_token = "" # 认证令牌
#batch_size = 100 # 批处理大小
#batch_timeout_ms = 1000 # 批处理超时时间,单位为毫秒
[[sinks]] # 文件接收器配置
type = "File" # 是否启用文件接收器
path = "./deploy/logs/rustfs.log" # 日志文件路径
buffer_size = 10 # 缓冲区大小,表示每次写入的字节数
flush_interval_ms = 100 # 批处理超时时间,单位为毫秒
flush_threshold = 100 # 刷新阈值,表示在达到该数量时刷新日志
[logger] # 日志器配置
queue_capacity = 10 # 日志队列容量,表示可以缓存的日志条数

View File

@@ -1,34 +0,0 @@
[observability]
endpoint = "http://localhost:4317" # Default is "http://localhost:4317" if not specified
use_stdout = false # Output with stdout, true output, false no output
sample_ratio = 2.0
meter_interval = 30
service_name = "rustfs"
service_version = "0.0.1"
environment = "production" # Default is "production" if not specified
logger_level = "info"
local_logging_enabled = true
#[[sinks]]
#type = "Kafka"
#brokers = "localhost:9092"
#topic = "logs"
#batch_size = 100 # Default is 100 if not specified
#batch_timeout_ms = 100 # Default is 1000ms if not specified
#
#[[sinks]]
#type = "Webhook"
#endpoint = "http://localhost:8080/webhook"
#auth_token = ""
#batch_size = 100 # Default is 3 if not specified
#batch_timeout_ms = 100 # Default is 100ms if not specified
[[sinks]]
type = "File"
path = "deploy/logs/rustfs.log"
buffer_size = 101 # Default is 8192 bytes if not specified
flush_interval_ms = 1000
flush_threshold = 100
[logger]
queue_capacity = 10000

View File

@@ -20,8 +20,8 @@ RUSTFS_SERVER_ENDPOINT="http://127.0.0.1:9000"
RUSTFS_SERVER_DOMAINS=127.0.0.1:9002
# RustFS 许可证内容
RUSTFS_LICENSE="license content"
# 可观测性配置文件路径deploy/config/obs.example.toml
RUSTFS_OBS_CONFIG=/etc/default/obs.toml
# 可观测性配置Endpointhttp://localhost:4317
RUSTFS_OBS_ENDPOINT=http://localhost:4317
# TLS 证书目录路径deploy/certs
RUSTFS_TLS_PATH=/etc/default/tls
# 事件通知配置文件路径deploy/config/event.example.toml

View File

@@ -20,8 +20,8 @@ RUSTFS_SERVER_ENDPOINT="http://127.0.0.1:9000"
RUSTFS_SERVER_DOMAINS=127.0.0.1:9002
# RustFS license content
RUSTFS_LICENSE="license content"
# Observability configuration file path: deploy/config/obs.example.toml
RUSTFS_OBS_CONFIG=/etc/default/obs.toml
# Observability configuration endpoint: RUSTFS_OBS_ENDPOINT
RUSTFS_OBS_ENDPOINT=http://localhost:4317
# TLS certificates directory path: deploy/certs
RUSTFS_TLS_PATH=/etc/default/tls
# event notification configuration file path: deploy/config/event.example.toml

View File

@@ -36,22 +36,19 @@ export RUSTFS_CONSOLE_ADDRESS=":9001"
# HTTPS 证书目录
# export RUSTFS_TLS_PATH="./deploy/certs"
# 具体路径修改为配置文件真实路径obs.example.toml 仅供参考 其中 `RUSTFS_OBS_CONFIG` 和下面变量二选一
# export RUSTFS_OBS_ENDPOINT=http://localhost:4317
# 如下变量需要必须参数都有值才可以,以及会覆盖配置文件中的值
#export RUSTFS_OBSERVABILITY_ENDPOINT=http://localhost:4317 # OpenTelemetry Collector 的地址
#export RUSTFS_OBSERVABILITY_USE_STDOUT=false # 是否使用标准输出
#export RUSTFS_OBSERVABILITY_SAMPLE_RATIO=2.0 # 采样率0.0-1.0之间0.0表示不采样1.0表示全部采样
#export RUSTFS_OBSERVABILITY_METER_INTERVAL=1 # 采样间隔,单位为秒
#export RUSTFS_OBSERVABILITY_SERVICE_NAME=rustfs # 服务名称
#export RUSTFS_OBSERVABILITY_SERVICE_VERSION=0.1.0 # 服务版本
#export RUSTFS_OBSERVABILITY_ENVIRONMENT=develop # 环境名称
#export RUSTFS_OBSERVABILITY_LOGGER_LEVEL=debug # 日志级别,支持 trace, debug, info, warn, error
export RUSTFS_OBSERVABILITY_LOCAL_LOGGING_ENABLED=true # 是否启用本地日志记录
export RUSTFS_OBSERVABILITY_LOG_DIRECTORY="./deploy/logs" # Log directory
export RUSTFS_OBSERVABILITY_LOG_ROTATION_TIME="minute" # Log rotation time unit, can be "second", "minute", "hour", "day"
export RUSTFS_OBSERVABILITY_LOG_ROTATION_SIZE_MB=1 # Log rotation size in MB
# 可观测性 相关配置信息
#export RUSTFS_OBS_ENDPOINT=http://localhost:4317 # OpenTelemetry Collector 的地址
#export RUSTFS_OBS_USE_STDOUT=false # 是否使用标准输出
#export RUSTFS_OBS_SAMPLE_RATIO=2.0 # 采样率0.0-1.0之间0.0表示不采样1.0表示全部采样
#export RUSTFS_OBS_METER_INTERVAL=1 # 采样间隔,单位为秒
#export RUSTFS_OBS_SERVICE_NAME=rustfs # 服务名称
#export RUSTFS_OBS_SERVICE_VERSION=0.1.0 # 服务版本
#export RUSTFS_OBS_ENVIRONMENT=develop # 环境名称
#export RUSTFS_OBS_LOGGER_LEVEL=debug # 日志级别,支持 trace, debug, info, warn, error
export RUSTFS_OBS_LOCAL_LOGGING_ENABLED=true # 是否启用本地日志记录
export RUSTFS_OBS_LOG_DIRECTORY="./deploy/logs" # Log directory
export RUSTFS_OBS_LOG_ROTATION_TIME="minute" # Log rotation time unit, can be "second", "minute", "hour", "day"
export RUSTFS_OBS_LOG_ROTATION_SIZE_MB=1 # Log rotation size in MB
#
#export RUSTFS_SINKS_FILE_PATH=./deploy/logs/rustfs.log