mirror of
https://github.com/ggml-org/llama.cpp.git
synced 2026-03-17 16:44:07 +00:00
9789c4ecdc
* Update build doc * Add cgraph tensor output name to OV op name * Update openvino build instructions * Add initial NPU support * draft NPU support version 2: prefill + kvcache * NPU support version 2: prefill + kvcache * Change due to ggml cgraph changes, not correct yet * Change due to ggml cgraph changes, llama-3.2 CPU work * Add AMD64 to CMakeLists * Change due to ggml cgraph changes, all device work * Refactor: clean, fix warning * Update clang-format * Statful transformation for CPU GPU * Add SwiGLU * Fuse to SDPA * Replace Concat with Broadcast in MulMat for GQA * Pull out indices creation for kv cache update * Refactor: remove past_token_len from extra_inputs * Fix Phi3 SwiGLU and SoftMax * Pull out sin cos from rope * Reduce memory: free ov weights node after graph conversion * Fix CPY due to cgraph change * Added OpenVINO CI/CD. Updated docs * Fix llama-cli * Fix Phi3 ROPE; Add test-backend-ops * Fix NPU * Fix llama-bench; Clang-format * Fix llama-perplexity * temp. changes for mark decomp * matmul in fp32 * mulmat input conversion fix * mulmat type conversion update * add mark decomp pass * Revert changes in fuse_to_sdpa * Update build.md * Fix test-backend-ops * Skip test-thread-safety; Run ctest only in ci/run.sh * Use CiD for NPU * Optimize tensor conversion, improve TTFT * Support op SET_ROWS * Fix NPU * Remove CPY * Fix test-backend-ops * Minor updates for raising PR * Perf: RMS fused to OV internal RMS op * Fix after rebasing - Layout of cache k and cache v are unified: [seq, n_head, head_size] - Add CPY and FLASH_ATTN_EXT, flash attn is not used yet - Skip test-backend-ops due to flash attn test crash - Add mutex around graph conversion to avoid test-thread-safety fali in the future - Update NPU config - Update GPU config to disable SDPA opt to make phi-3 run * Change openvino device_type to GPU; Enable flash_attn * Update supports_buft and supports_op for quantized models * Add quant weight conversion functions from genai gguf reader * Quant models run with accuracy issue * Fix accuracy: disable cpu_repack * Fix CI; Disable test-backend-ops * Fix Q4_1 * Fix test-backend-ops: Treat quantized tensors as weights * Add NPU Q4_0 support * NPU perf: eliminate zp * Dequantize q4_1 q4_k q6_k for NPU * Add custom quant type: q8_1_c, q4_0_128 * Set m_is_static=false as default in decoder * Simpilfy translation of get_rows * Fix after rebasing * Improve debug util; Eliminate nop ReshapeReshape * STYLE: make get_types_to_requant a function * Support BF16 model * Fix NPU compile * WA for npu 1st token acc issue * Apply EliminateZP only for npu * Add GeGLU * Fix Hunyuan * Support iSWA * Fix NPU accuracy * Fix ROPE accuracy when freq_scale != 1 * Minor: not add attention_size_swa for non-swa model * Minor refactor * Add Q5_K to support phi-3-q4_k_m * Requantize Q6_K (gs16) to gs32 on GPU * Fix after rebasing * Always apply Eliminate_ZP to fix GPU compile issue on some platforms * kvcachefusion support * env variable GGML_OPENVINO_DISABLE_SDPA_OPTIMIZATION added * Fix for Phi3 * Fix llama-cli (need to run with --no-warmup) * Fix add_sliced_mask; Revert mulmat, softmax; Remove input attention_size, iSWA model not working * fix after rebasing * Fix llama-3-8b and phi3-mini q4_0 NPU * Update to OV-2025.3 and CMakeLists.txt * Add OV CI cache * Apply CISC review and update CI to OV2025.3 * Update CI to run OV dep install before build * Update OV dockerfile to use OV2025.3 and update build docs * Style: use switch in supports_ops * Style: middle ptr and ref align, omit optional struct keyword * NPU Unify PD (#14) * Stateless. Fix llama-cli llama-server * Simplify broadcast op in attention * Replace get_output_tensor+memcpy with set_output_tensor * NPU unify PD. Unify dynamic and static dims * Clean placeholders in ggml-openvino.cpp * NPU unify PD (handled internally) * change graph to 4d, support multi sequences * Fix llama-bench * Fix NPU * Update ggml-decoder.cpp Hitting error while compiling on windows: error C3861: 'unsetenv': identifier not found Reason: unsetenv() is a POSIX function; it doesn’t exist on Windows. Visual Studio (MSVC) won’t recognize it. Proposed fix: Use _putenv_s() (Windows equivalent) This is supported by MSVC and achieves the same effect: it removes the environment variable from the process environment. This keeps cross-platform compatibility. * Update ggml-decoder.cpp * Update ggml-decoder.cpp * Update ggml-decoder.cpp * Update ggml-decoder.cpp * Update ggml-decoder.cpp * Remove the second decoder for node. Moving the function into the model decoder * Fix error for naive * NPU prefill chunking * NPU fix llama-bench * fallback naive run with accuracy issue * NPU support llma-perplexity -b 512 --no-warmup * Refactor: split ov_graph_compute for dynamic and static * remove unused API GgmlOvDecoder::get_output_stride(const std::string & name) * minor update due to ov 2025.4 * remove unused API GgmlOvDecoder::get_output_names() * remove unused API get_output_shape(const std::string & name) * Modified API GgmlOvDecoder::get_output_type(const std::string & name) * Removed API GgmlOvDecoder::get_output_op_params(const std::string & name) * Removed API get_output_ggml_tensor(const std::string & name) * Removed API m_outputs * Removed m_output_names * Removed API GgmlOvDecoder::get_input_names() * Removed API GgmlOvDecoder::get_input_stride(const std::string& name) * Removed API get_input_type * Removed API get_input_type * Removed API GgmlOvDecoder::get_input_shape(const std::string & name) * Removed API GgmlOvDecoder::get_input_op_params(const std::string & name) * Fix error for decoder cache * Reuse cached decoder * GPU remove Q6_K requantization * NPU fix wrong model output shape * NPU fix q4 perf regression * Remove unused variable nodes * Fix decoder can_reuse for llama-bench * Update build.md for Windows * backend buffer: allocate on host * Use shared_buffer for GPU NPU; Refactor * Add ov_backend_host_buffer; Use cached remote context * Put kvcache on GPU * Use ggml_aligned_malloc * only use remote tensor for kvcache * only use remote tensor for kvcache for GPU * FIX: use remote tensor from singleton * Update build.md to include OpenCL * NPU always requant to q4_0_128 * Optimize symmetric quant weight extraction: use single zp * Use Q8_0_C in token embd, lm_head, and for 5 and 6 bits quant * Update build.md * Support -ctk f32 * Initial stateful graph support * Update ggml/src/ggml-openvino/ggml-decoder.cpp Co-authored-by: Yamini Nimmagadda <yamini.nimmagadda@intel.com> * code cleanup * npu perf fix * requant to f16 for Q6 embed on NPU * Update ggml/src/ggml-openvino/ggml-decoder.cpp * Update ggml/src/ggml-openvino/ggml-openvino-extra.cpp * Create OPENVINO.md in llama.cpp backend docs * Update OPENVINO.md * Update OPENVINO.md * Update OPENVINO.md * Update build.md * Update OPENVINO.md * Update OPENVINO.md * Update OPENVINO.md * kq_mask naming fix * Syntax correction for workflows build file * Change ov backend buffer is_host to false * Fix llama-bench -p -n where p<=256 * Fix --direct-io 0 * Don't put kvcache on GPU in stateful mode * Remove hardcode names * Fix stateful shapes * Simplification for stateful and update output shape processing * Remove hardcode names * Avoid re-compilation in llama-bench * Extract zp directly instead of bias * Refactor weight tensor processing * create_weight_node accept non-ov backend buffer * remove changes in llama-graph.cpp * stateful masking fix (#38) Fix for stateful accuracy issues and cl_out_of_resources error in stateful GPU with larger context sizes. * Fix test-backend-ops crash glu, get_rows, scale, rms_norm, add * hardcoded name handling for rope_freqs.weight * Suppress logging and add error handling to allow test-backend-ops to complete * Fix MUL_MAT with broadcast; Add unsupported MUL_MAT FLASH_ATTN cases * Use bias instead of zp in test-backend-ops * Update OV in CI, Add OV CI Tests in GH Actions * Temp fix for multithreading bug * Update OV CI, fix review suggestions. * fix editorconfig-checker, update docs * Fix tabs to spaces for editorconfig-checker * fix editorconfig-checker * Update docs * updated model link to be GGUF model links * Remove GGML_CPU_REPACK=OFF * Skip permuted ADD and MUL * Removed static variables from utils.cpp * Removed initializing non-existing variable * Remove unused structs * Fix test-backend-ops for OV GPU * unify api calling * Update utils.cpp * When the dim is dynamic, throw an error, need to is stastic forst * Add interface compute_model_outputs(), which get the model output through computing the node use count & status in the cgraph to avoid the flag using * No need to return * Fix test-backend-ops for OV GPU LNL * Fix test-thread-safety * use the shape from infer request of output tensor create to avoid issue * fix dynamic output shape issue * fix issue for the unused node in tests * Remove unused lock * Add comment * Update openvino docs * update to OV release version 2026.0 * add ci ov-gpu self hosted runner * fix editorconfig * Fix perplexity * Rewrite the model inputs finding mechanism (#54) * Rewrite the model inputs finding logistic * Put stateful shape handle in get input shape * Put the iteration logistic in func * Added ggml-ci-intel-openvino-gpu and doc update * .hpp files converted to .h * fix ggml-ci-x64-intel-openvino-gpu * Fix for stateful execution bug in llama-bench * Minor updates after stateful llama-bench fix * Update ggml/src/ggml-openvino/utils.cpp Co-authored-by: Yamini Nimmagadda <yamini.nimmagadda@intel.com> * Remove multiple get_shape calls * Bring back mutex into compute * Fix VIEW op, which slice the input node * Added token_len_per_seq existence check before slicing masks and moved node retrieval inside guarded block to prevent missing-key access * Temp. fix for test requant errors * Update to OV ggml-ci to low-perf * ci : temporary disable "test-llama-archs" * ci : cache v4 -> v5, checkout v4 -> v6, fix runner tag * docs : update url * Fix OV link in docker and Update docs --------- Co-authored-by: Ravi Panchumarthy <ravi.panchumarthy@intel.com> Co-authored-by: Cavus Mustafa <mustafa.cavus@intel.com> Co-authored-by: Arshath <arshath.ramzan@intel.com> Co-authored-by: XuejunZhai <Xuejun.Zhai@intel.com> Co-authored-by: Yamini Nimmagadda <yamini.nimmagadda@intel.com> Co-authored-by: Xuejun Zhai <Xuejun.Zhai@intel> Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
117 lines
5.9 KiB
Plaintext
117 lines
5.9 KiB
Plaintext
# collaborators can optionally add themselves here to indicate their availability for reviewing related PRs
|
|
# multiplie collaborators per item can be specified
|
|
|
|
/.devops/*.Dockerfile @ngxson
|
|
/.github/actions/ @CISC
|
|
/.github/workflows/ @CISC
|
|
/ci/ @ggerganov
|
|
/cmake/ @ggerganov
|
|
/common/CMakeLists.txt @ggerganov
|
|
/common/arg.* @ggerganov
|
|
/common/base64.hpp.* @ggerganov
|
|
/common/build-info.* @ggerganov
|
|
/common/chat.* @pwilkin
|
|
/common/chat-auto*.* @pwilkin
|
|
/common/chat-diff-analyzer.* @pwilkin
|
|
/common/chat-peg-parser.* @aldehir
|
|
/common/common.* @ggerganov
|
|
/common/console.* @ggerganov
|
|
/common/http.* @angt
|
|
/common/jinja/ @ngxson @CISC @aldehir
|
|
/common/llguidance.* @ggerganov
|
|
/common/log.* @ggerganov
|
|
/common/ngram-map.* @srogmann
|
|
/common/peg-parser.* @aldehir
|
|
/common/sampling.* @ggerganov
|
|
/common/speculative.* @ggerganov
|
|
/common/unicode.* @aldehir
|
|
/convert_*.py @CISC
|
|
/examples/batched.swift/ @ggerganov
|
|
/examples/batched/ @ggerganov
|
|
/examples/convert-llama2c-to-ggml/ @ggerganov
|
|
/examples/debug/ @danbev @pwilkin
|
|
/examples/deprecation-warning/ @ggerganov
|
|
/examples/diffusion/ @am17an
|
|
/examples/embedding/ @ggerganov
|
|
/examples/eval-callback/ @ggerganov
|
|
/examples/export-docs/ @ggerganov
|
|
/examples/gen-docs/ @ggerganov
|
|
/examples/gguf/ @ggerganov
|
|
/examples/llama.android/ @ggerganov @hanyin-arm @naco-siren
|
|
/examples/llama.swiftui/ @ggerganov
|
|
/examples/llama.vim @ggerganov
|
|
/examples/lookahead/ @ggerganov
|
|
/examples/lookup/ @JohannesGaessler
|
|
/examples/model-conversion/ @danbev
|
|
/examples/parallel/ @ggerganov
|
|
/examples/passkey/ @ggerganov
|
|
/examples/retrieval/ @ggerganov
|
|
/examples/save-load-state/ @ggerganov
|
|
/examples/speculative-simple/ @ggerganov
|
|
/examples/speculative/ @ggerganov
|
|
/ggml/cmake/ @ggerganov
|
|
/ggml/include/ @ggerganov
|
|
/ggml/src/ggml-common.h @ggerganov
|
|
/ggml/src/ggml-cpu/ @ggerganov
|
|
/ggml/src/ggml-cpu/spacemit/ @alex-spacemit
|
|
/ggml/src/ggml-cuda/fattn* @JohannesGaessler
|
|
/ggml/src/ggml-cuda/mmf.* @JohannesGaessler @am17an
|
|
/ggml/src/ggml-cuda/mmq.* @JohannesGaessler
|
|
/ggml/src/ggml-cuda/mmvf.* @JohannesGaessler
|
|
/ggml/src/ggml-cuda/mmvq.* @JohannesGaessler
|
|
/ggml/src/ggml-cuda/fattn-wmma* @IMbackK
|
|
/ggml/src/ggml-hip/ @IMbackK
|
|
/ggml/src/ggml-cuda/vendors/hip.h @IMbackK
|
|
/ggml/src/ggml-impl.h @ggerganov
|
|
/ggml/src/ggml-metal/ @ggerganov
|
|
/ggml/src/ggml-opencl/ @lhez @max-krasnyansky
|
|
/ggml/src/ggml-hexagon/ @max-krasnyansky @lhez
|
|
/ggml/src/ggml-opt.cpp @JohannesGaessler
|
|
/ggml/src/ggml-quants.* @ggerganov
|
|
/ggml/src/ggml-rpc/ @rgerganov
|
|
/ggml/src/ggml-threading.* @ggerganov
|
|
/ggml/src/ggml-vulkan/ @0cc4m
|
|
/ggml/src/ggml-virtgpu/ @kpouget
|
|
/ggml/src/ggml-webgpu/ @reeselevine
|
|
/ggml/src/ggml-zdnn/ @taronaeo @Andreas-Krebbel @AlekseiNikiforovIBM
|
|
/ggml/src/ggml-openvino/ @cavusmustafa @wine99
|
|
/ggml/src/ggml.c @ggerganov
|
|
/ggml/src/ggml.cpp @ggerganov
|
|
/ggml/src/gguf.cpp @JohannesGaessler @Green-Sky
|
|
/gguf-py/ @CISC
|
|
/media/ @ggerganov
|
|
/scripts/gen* @ggerganov
|
|
/scripts/get* @ggerganov
|
|
/scripts/sync* @ggerganov
|
|
/src/ @ggerganov
|
|
/src/llama-adapter.* @CISC
|
|
/src/llama-arch.* @CISC
|
|
/src/llama-chat.* @ngxson
|
|
/src/llama-graph.* @CISC
|
|
/src/llama-model.* @CISC
|
|
/src/llama-vocab.* @CISC
|
|
/src/models/ @CISC
|
|
/tests/ @ggerganov
|
|
/tests/test-chat.* @pwilkin
|
|
/tools/batched-bench/ @ggerganov
|
|
/tools/cli/ @ngxson
|
|
/tools/completion/ @ggerganov
|
|
/tools/mtmd/ @ngxson
|
|
/tools/perplexity/ @ggerganov
|
|
/tools/parser/ @pwilkin
|
|
/tools/quantize/ @ggerganov
|
|
/tools/rpc/ @rgerganov
|
|
/tools/server/* @ngxson @ggerganov # no subdir
|
|
/tools/server/webui/ @allozaur
|
|
/tools/tokenize/ @ggerganov
|
|
/tools/tts/ @ggerganov
|
|
/vendor/ @ggerganov
|
|
/AUTHORS @ggerganov
|
|
/CMakeLists.txt @ggerganov
|
|
/CONTRIBUTING.md @ggerganov
|
|
/LICENSE @ggerganov
|
|
/README.md @ggerganov
|
|
/SECURITY.md @ggerganov
|
|
/build-xcframework.sh @danbev
|
|
requirements*.txt @CISC
|