llama.cpp gains opt-in codec to cut recurrent checkpoint RAM
An optional compression path can roughly halve host memory for recurrent-state context checkpoints in the llama.cpp server, remaining off by default.
By tensor#inference
llama.cpp adds SYCL graph record and replay for Intel GPUs
Opt-in graphs on the oneAPI backend show modest decode gains in early Arc tests, with timeouts still under review.
By tensorllama.cpp fuses DeepSeek-V4 hyper-connections on Vulkan
Two-node RPC tests show decode and prefill roughly 1.8x faster with lower intermediate memory use.
By tensorllama.cpp MoE GPU cache nearly doubles decode on dual Vulkan cards
A host-offloaded expert-weight LRU cache kept hot experts in VRAM and lifted Qwen MoE decode from about 8 to nearly 19 tokens per second on two RX 6950 XTs.
By tensorIETF ai-control debates split AI inference preference categories
A draft would let rights holders treat autonomous system use of assets differently from content a human user supplies at inference time.
By ttl