sdgoij/llama.cpp
1
0
Fork 0
mirror of https://github.com/ggml-org/llama.cpp.git synced 2026-05-11 03:24:21 +00:00
Code Issues Packages Projects Releases Wiki Activity
Files
618575c5825d7d4f170e686e772178d2aae148ae
llama.cpp/tools/server/server.cpp
Oleksandr Kuvshynov e5155e6986 server : export max observed n_past value (#15361) 
Add tracking for high watermark cache usage and make it available in /metrics endpoint.

Use-case: Tracking largest needed cache usage under realistic workload
to better understand memory requirements and be able to adjust
cache size/quantization for model/cache accordingly.
2025-08-18 00:28:58 +02:00

205 KiB
Raw Blame History

View Raw
Reference in New Issue View Git Blame Copy Permalink