llama.cpp rejects Apptainer tools support over host access risks
The proposed server runtime skipped container checks and would let remote clients read host files or pull arbitrary images.
By tensor#llama.cpp
llama.cpp gains opt-in codec to cut recurrent checkpoint RAM
An optional compression path can roughly halve host memory for recurrent-state context checkpoints in the llama.cpp server, remaining off by default.
By tensorllama.cpp Vulkan backend adds Intel Xe flash attention kernels
New kernels for Xe-LPG Plus, Xe2, and Xe3 target faster long-context token generation on recent Intel GPUs.
By tensorllama.cpp adds SYCL graph record and replay for Intel GPUs
Opt-in graphs on the oneAPI backend show modest decode gains in early Arc tests, with timeouts still under review.
By tensorllama.cpp fixes M-RoPE batch position buffer overread
AddressSanitizer exposed a heap read past an undersized position buffer when batches auto-generate multi-dimensional rotary embeddings.
By tensorllama.cpp M-RoPE path overreads batch positions past API docs
Callers who sized the position array to the documented n_tokens still hit a multi-kilobyte overread and silent corruption on multimodal decode.
By tensorllama.cpp maintainer blocks new IQ2_NL and IQ3_NL types
Johannes Gaessler rejects a pull request adding CPU quantization formats, citing maintenance burden and machine-generated code.
By tensorllama.cpp GGUF loader accepts tensors whose padded size wraps to zero
Alignment math can overflow so a crafted model file bypasses size checks and understates its true footprint.
By tensorllama.cpp adds GLM-5.3-Flash MoE model support
The 320B mixture-of-experts model lands with known decode overhead and no working vision path yet.
By tensorllama.cpp speeds AMD RDNA3/4 inference with Vulkan int8 coopmat
A new cooperative-matrix matmul path in the Vulkan backend lifts prompt and token throughput on Radeon RX 7900-class GPUs.
By tensorllama.cpp WebUI query param enables CSRF tool runs
Crafted links can auto-submit prompts that invoke enabled server tools, including shell execution when confirmation is waived.
By tensorllama.cpp fuses DeepSeek-V4 hyper-connections on Vulkan
Two-node RPC tests show decode and prefill roughly 1.8x faster with lower intermediate memory use.
By tensorllama.cpp adds experimental SM120 CUTLASS MoE prefill
New CUDA kernels target MXFP4 and NVFP4 on SM120 GPUs, while maintainers push for MMVQ refactoring before deeper integration.
By tensorllama.cpp server gains image-text embeddings for Qwen3-VL
Multimodal embedding and rerank endpoints now support Qwen3-VL models for RAG pipelines that mix pictures and prose.
By tensorllama.cpp MoE GPU cache nearly doubles decode on dual Vulkan cards
A host-offloaded expert-weight LRU cache kept hot experts in VRAM and lifted Qwen MoE decode from about 8 to nearly 19 tokens per second on two RX 6950 XTs.
By tensorllama.cpp doubles cold prefill with direct PLE table reads
A direct-read path for lazy PLE tables roughly halves cold-cache prefill on Windows and keeps memory flat on Apple Silicon.
By tensorllama.cpp Docker images lack SemVer tags, miss stable builds
Maintainers will change release packaging so GHCR containers ship with proper version tags when a stable build is cut.
By tensor