kimi-k3-in-c

`kimi-k3-in-c` is a minimalistic, bare-metal C99 inference engine designed to execute the 2.78-trillion-parameter Kimi K3 model on a single CPU using as little as 8.24 GB of RAM. It operates entirely without BLAS libraries, GPU acceleration, or heavy ML frameworks like PyTorch or llama.cpp. The engine itself compiles down to a tiny 176 KB binary. ### How It Works Under the Hood Running a 2.78T parameter model normally requires an array of high-end GPUs to store hundreds of gigabytes of weights in VRAM. `kimi-k3-in-c` bypasses this hardware barrier by decoupling model execution from total system RAM capacity. The entire 1.56 TB model checkpoint resides on disk. On low-memory systems, the engine streams weights directly off disk sequentially during every single token generation step. Because parameters are read on demand rather than preloaded, the peak Resident Set Size (RSS) stays locked at 8.24 GB. A key technical highlight of this design is execution determinism: the generation output is **byte-identical** across every hardware configuration, whether running on a budget laptop or a multi-socket server. ### Performance & Memory Scaling System RAM determines how much of the 1.56 TB checkpoint can be cached, directly impacting token throughput: * **8 GB RAM (Ordinary Laptop):** ~26.5 seconds per token. The entire model streams from disk on every step, making performance heavily dependent on drive read speeds. * **32 GB RAM (High-End Laptop):** ~24.2 seconds per token. A portion of the model stays cached in system memory. * **64 GB RAM (Desktop):** ~19.8 seconds per token. More weights are retained in memory, reducing disk wait times. * **128 GB+ RAM (Workstation):** ~5.6 seconds per token. The active weights fit entirely in memory, eliminating disk I/O bottlenecks completely. According to the v1.0.0 performance metrics, optimizations made the core math per token roughly **8x lighter**. Follow-up chat queries run **3.9x faster**, and long prompt processing overhead is cut in half compared to earlier builds. ### System Requirements & Setup Because the project relies purely on standard C99 and targeting Linux x86-64, setting it up requires no external C++ frameworks, CUDA drivers, or BLAS dependencies. **Requirements:** * **OS/Architecture:** Linux x86-64 * **Compiler:** Standard C99-compliant compiler (e.g., `gcc` or `clang`) * **Storage:** At least 1.56 TB of disk space for the model checkpoint (preferably on a fast NVMe SSD to minimize streaming latency) * **RAM:** Minimum 8.24 GB free memory This engine demonstrates how far raw C99 and simple stream-based execution can go when stripped of modern framework abstraction layers.