Georgi Gerganov 0ca458d892 examples: implement flexible grader system for answer validation ...

- Add Grader class supporting regex and CLI-based grading
- Implement built-in regex patterns for AIME, GSM8K, MMLU, HellaSwag, ARC, WinoGrande
- Add CLI grader interface: python script.py --answer <pred> --expected <gold>
- Add HF telemetry disable to avoid warnings
- Support exact match requirement for regex patterns
- Add 30-second timeout for CLI grader
- Handle both boxed and plain text formats for AIME answers

2026-05-10 18:13:45 +03:00

..

batched

libs : rename libcommon -> libllama-common (#21936)

2026-04-17 11:11:46 +03:00

batched.swift

examples : remove references to make in examples [no ci] (#15457)

2025-08-21 06:12:28 +02:00

convert-llama2c-to-ggml

libs : rename libcommon -> libllama-common (#21936)

2026-04-17 11:11:46 +03:00

debug

common: fix missing exports in llama-common (#22340)

2026-04-27 08:06:39 +03:00

deprecation-warning

Fix locale-dependent float printing in GGUF metadata (#17331)

2026-03-04 09:30:40 +01:00

diffusion

examples: refactor diffusion generation (#22590)

2026-05-04 20:19:30 +08:00

embedding

libs : rename libcommon -> libllama-common (#21936)

2026-04-17 11:11:46 +03:00

eval-callback

common: fix missing exports in llama-common (#22340)

2026-04-27 08:06:39 +03:00

gen-docs

spec : refactor params (#22397)

2026-04-28 09:07:33 +03:00

gguf

Fix locale-dependent float printing in GGUF metadata (#17331)

2026-03-04 09:30:40 +01:00

gguf-hash

Fix locale-dependent float printing in GGUF metadata (#17331)

2026-03-04 09:30:40 +01:00

idle

libs : rename libcommon -> libllama-common (#21936)

2026-04-17 11:11:46 +03:00

llama-eval

examples: implement flexible grader system for answer validation

2026-05-10 18:13:45 +03:00

llama.android

android : libcommon -> libllama-common (#22076)

2026-04-18 11:19:40 +02:00

llama.swiftui

llama : deprecate llama_kv_self_ API (#14030)

2025-06-06 14:11:15 +03:00

lookahead

libs : rename libcommon -> libllama-common (#21936)

2026-04-17 11:11:46 +03:00

lookup

spec : refactor params (#22397)

2026-04-28 09:07:33 +03:00

model-conversion

model-conversion : fix mmproj output file name [no ci] (#22274)

2026-04-23 15:07:38 +02:00

parallel

libs : rename libcommon -> libllama-common (#21936)

2026-04-17 11:11:46 +03:00

passkey

libs : rename libcommon -> libllama-common (#21936)

2026-04-17 11:11:46 +03:00

retrieval

libs : rename libcommon -> libllama-common (#21936)

2026-04-17 11:11:46 +03:00

save-load-state

common : only load backends when required (#22290)

2026-05-05 09:23:50 +02:00

simple

Fix locale-dependent float printing in GGUF metadata (#17331)

2026-03-04 09:30:40 +01:00

simple-chat

Fix locale-dependent float printing in GGUF metadata (#17331)

2026-03-04 09:30:40 +01:00

simple-cmake-pkg

examples : add missing code block end marker [no ci] (#17756)

2025-12-04 14:17:30 +01:00

speculative

spec : fix vocab compat checks in spec example (#22426)

2026-04-30 08:18:25 +03:00

speculative-simple

spec : refactor params (#22397)

2026-04-28 09:07:33 +03:00

sycl

fix script error (#22795sycl : )

2026-05-08 06:54:57 +03:00

training

libs : rename libcommon -> libllama-common (#21936)

2026-04-17 11:11:46 +03:00

CMakeLists.txt

examples : add debug utility/example (#18464)

2026-01-07 10:42:19 +01:00

convert_legacy_llama.py

metadata: Detailed Dataset Authorship Metadata (#8875)

2024-11-13 21:10:38 +11:00

json_schema_pydantic_example.py

py : type-check all Python scripts with Pyright (#8341)

2024-07-07 15:04:39 -04:00

json_schema_to_grammar.py

ci : switch from pyright to ty (#20826)

2026-03-21 08:54:34 +01:00

llama.vim

chore : correct typos [no ci] (#20041)

2026-03-05 08:50:21 +01:00

pydantic_models_to_grammar_examples.py

llama : move end-user examples to tools directory (#13249)

2025-05-02 20:27:13 +02:00

pydantic_models_to_grammar.py

ci : switch from pyright to ty (#20826)

2026-03-21 08:54:34 +01:00

reason-act.sh

scripts : make the shell scripts cross-platform (#14341)

2025-06-30 10:17:18 +02:00

regex_to_grammar.py

py : switch to snake_case (#8305)

2024-07-05 07:53:33 +03:00

server_embd.py

llama : fix FA when KV cache is not used (i.e. embeddings) (#12825)

2025-04-08 19:54:51 +03:00

server-llama2-13B.sh

scripts : make the shell scripts cross-platform (#14341)

2025-06-30 10:17:18 +02:00

ts-type-to-grammar.sh

scripts : make the shell scripts cross-platform (#14341)

2025-06-30 10:17:18 +02:00