Oliver Simons 217469f07f Make backend's top_p sampler inclusive ...

In addition to match the algorithm proposed in the original
[paper](https://arxiv.org/abs/1904.09751), this resolves the edge-case
where `max_p is > top_p` for a single logit, where the mask would
otherwise be empty (and we thus sample from the whole vocabulary with
equal likelihood)

2025-12-01 15:28:06 +01:00

42 KiB

Raw Blame History

View Raw