Georgi Gerganov 1b9b79dd14 convert : fix pre-tokenizer type writing

2024-04-26 20:55:14 +03:00

..

.editorconfig

gguf : new file format with flexible meta data (beta) (#2398)

2023-08-21 23:07:43 +03:00

ggml-vocab-aquila.gguf

Work on the BPE tokenizer (#3252)

2023-10-03 09:16:26 +02:00

ggml-vocab-baichuan.gguf

Add more tokenizer tests (#3742)

2023-10-24 09:17:17 +02:00

ggml-vocab-deepseek-coder.gguf

convert : fix pre-tokenizer type writing

2024-04-26 20:55:14 +03:00

ggml-vocab-deepseek-llm.gguf

convert : fix pre-tokenizer type writing

2024-04-26 20:55:14 +03:00

ggml-vocab-falcon.gguf

Work on the BPE tokenizer (#3252)

2023-10-03 09:16:26 +02:00

ggml-vocab-gpt2.gguf

gpt2 : Add gpt2 architecture integration (#4555)

2023-12-28 15:03:57 +01:00

ggml-vocab-gpt-neox.gguf

Add more tokenizer tests (#3742)

2023-10-24 09:17:17 +02:00

ggml-vocab-llama.gguf

llama : use new pre-tokenizer type

2024-04-26 20:08:57 +03:00

ggml-vocab-mpt.gguf

Add test for MPT tokenization (#3728)

2023-10-22 21:21:42 +02:00

ggml-vocab-refact.gguf

Add more tokenizer tests (#3742)

2023-10-24 09:17:17 +02:00

ggml-vocab-stablelm-3b-4e1t.gguf

stablelm : StableLM support (#3586)

2023-11-14 11:17:12 +01:00

ggml-vocab-starcoder.gguf

Add more tokenizer tests (#3742)

2023-10-24 09:17:17 +02:00