sdgoij/llama.cpp
1
0
Fork 0
mirror of https://github.com/ggml-org/llama.cpp.git synced 2026-05-10 11:04:06 +00:00
Code Issues Packages Projects Releases Wiki Activity
Files
fa31c438e0e709242ab7334d26fc3be3dcda07a0
llama.cpp/examples/server/server.cpp
Clauszy 06a92a193a server : fix cache reuse logic (#12161) 
The first kv shift offsets the positions of all tokens after head_c.
When using llama_kv_cache_seq_rm next, using head_c will remove the valid tokens because their positions have already been offset.
2025-03-05 09:25:45 +02:00

176 KiB
Raw Blame History

View Raw
Reference in New Issue View Git Blame Copy Permalink