Hi folks! I’m happy to announce SonicBoom 0.0.1, a native tensor runtime and AI framework written in C++/CUDA, with Guile as its first-class language:
https://github.com/hardenedlinux/SonicBoom ------------------------------cut----------------------------------------- (use-modules (sonicboom inference)) (define model (load-model "gemma-4-E4B-it-Q3_K_M.gguf")) (define session (make-session model 4096)) (display (generate model session "Hello" 64)) -------------------------------end------------------------------------------ >From Guile, it decodes at 52 tok/s on a laptop RTX 3050. That is 14% faster than llama.cpp with the same quantized kernels; the extra speed comes from a graph planner. Other Guile parts: - SonicCross is PyTorch’s torchgen rewritten in Guile. It generates SonicBoom’s operator library at build time: https://github.com/hardenedlinux/SonicCross - A tracing prototype turns eager tensor code into graphs. Tensor branches are rewritten with syntax-rules, and delimited continuations handle the rest, including resuming twice, once for each branch. A whole training step, autodiff included, becomes one graph. - A server based on GNU Artanis is in the works. The api-server is done, but I haven't bound it with the SonicBoom core. And it requires the latest GNU Artanis for the enhanced WebSocket support. You should be patient. For Guix folks: - The architecture is not tied to CUDA. Backends sit behind a hardware abstraction layer, and the planner emits plans that do not depend on a particular backend. CPU and CUDA are supported today, and a Vulkan backend scaffold is already in the tree (-DSONICBOOM_USE_VULKAN=ON), so the GPU path can be fully free too. - No Python is needed to build or run it. SonicCross replaces torchgen, which is the one Python step in PyTorch’s build. - All generation inputs are pinned by commit and sha256 in tools/generation/ SONICCROSS.lock. - The default build is CPU-only and needs no proprietary components. CUDA is optional. Guix packaging is on the roadmap. Help and advice are welcome! 0.0.1 supports one model (Gemma 4 E4B) on CUDA sm_86. Details are in the README. Happy hacking!
