564a02f344
This commit introduces two key enhancements: the integration of Grafana Tempo for distributed tracing and the implementation of a dual-stack TCP listener for improved network compatibility. - **Observability**: - Adds the `tempo` service to the `docker-compose.yml` observability stack. - Tempo is configured to collect and store traces, integrating with the existing OpenTelemetry setup. - A custom `tempo-entrypoint.sh` script is included to manage volume permissions on startup. - **Networking**: - Modifies `http.rs` to support dual-stack (IPv4/IPv6) connections on a single socket. - By setting the `IPV6_V6ONLY` socket option to `false`, the server can now accept both IPv6 and IPv4-mapped IPv6 traffic, enhancing cross-platform support.
Observability
This directory contains the observability stack for the application. The stack is composed of the following components:
- Prometheus v3.2.1
- Grafana 11.6.0
- Loki 3.4.2
- Jaeger 2.4.0
- Otel Collector 0.120.0 # 0.121.0 remove loki
Prometheus
Prometheus is a monitoring and alerting toolkit. It scrapes metrics from instrumented jobs, either directly or via an intermediary push gateway for short-lived jobs. It stores all scraped samples locally and runs rules over this data to either aggregate and record new time series from existing data or generate alerts. Grafana or other API consumers can be used to visualize the collected data.
Grafana
Grafana is a multi-platform open-source analytics and interactive visualization web application. It provides charts, graphs, and alerts for the web when connected to supported data sources.
Loki
Loki is a horizontally-scalable, highly-available, multi-tenant log aggregation system inspired by Prometheus. It is designed to be very cost-effective and easy to operate. It does not index the contents of the logs, but rather a set of labels for each log stream.
Jaeger
Jaeger is a distributed tracing system released as open source by Uber Technologies. It is used for monitoring and troubleshooting microservices-based distributed systems, including:
- Distributed context propagation
- Distributed transaction monitoring
- Root cause analysis
- Service dependency analysis
- Performance / latency optimization
Otel Collector
The OpenTelemetry Collector offers a vendor-agnostic implementation on how to receive, process, and export telemetry data. It removes the need to run, operate, and maintain multiple agents/collectors in order to support open-source observability data formats (e.g. Jaeger, Prometheus, etc.) sending to one or more open-source or commercial back-ends.
How to use
To deploy the observability stack, run the following command:
- docker latest version
docker compose -f docker-compose.yml -f docker-compose.override.yml up -d
- docker compose v2.0.0 or before
docke-compose -f docker-compose.yml -f docker-compose.override.yml up -d
To access the Grafana dashboard, navigate to http://localhost:3000 in your browser. The default username and password
are admin and admin, respectively.
To access the Jaeger dashboard, navigate to http://localhost:16686 in your browser.
To access the Prometheus dashboard, navigate to http://localhost:9090 in your browser.
How to stop
To stop the observability stack, run the following command:
docker compose -f docker-compose.yml -f docker-compose.override.yml down
How to remove data
To remove the data generated by the observability stack, run the following command:
docker compose -f docker-compose.yml -f docker-compose.override.yml down -v
How to configure
To configure the observability stack, modify the docker-compose.override.yml file. The file contains the following
services:
prometheus:
environment:
- PROMETHEUS_CONFIG_FILE=/etc/prometheus/prometheus.yml
volumes:
- ./prometheus.yml:/etc/prometheus/prometheus.yml
grafana:
environment:
- GF_SECURITY_ADMIN_PASSWORD=admin
volumes:
- ./grafana/provisioning:/etc/grafana/provisioning
The prometheus service mounts the prometheus.yml file to /etc/prometheus/prometheus.yml. The grafana service
mounts the grafana/provisioning directory to /etc/grafana/provisioning. You can modify these files to configure the
observability stack.