Zero-config discovery
Laptops find each other over mDNS. Run baton app on each one and nearby peers appear in the browser dashboard.
·
·
·
Distributed AI system2026
Baton
OngoingOne big model. Many laptops.
Runs one large language model across several laptops on the same Wi-Fi, passing activations from laptop to laptop like a relay baton.
4Engineers on the team
¼Weight size with int4 vs bf16
1 lineInstall on Windows, macOS, Linux
/v1OpenAI-compatible API
The problem
Large language models need more memory than a single student laptop has. Cloud GPUs cost money and send data off the network, while the laptops already sitting in a classroom or lab spend most of their time idle.
The approach
Baton splits a model's layers across the laptops on the same Wi-Fi. Each laptop holds a few layers and hands its activations to the next, like a relay baton, so a group can run a model that none of them could run alone. One laptop hosts, the others accept an invite, and the cluster serves chat through a standard API.
What it does
Every part earns its place.
Host, invite, start
Pick a model on one laptop, invite the others, and start once everyone accepts. The first start downloads the model.
Layer-wise pipeline
The model is cut into contiguous layer ranges sized to each laptop, and activations stream from one stage to the next.
Quantized weights
Optional int8 and int4 modes hold layer weights at about a half and a quarter of bf16, so bigger models fit.
Agent-ready API
An OpenAI-compatible /v1 endpoint with tool calling. The model asks for a tool and the client runs it.
Heartbeats & recovery
Heartbeats catch a laptop that goes silent, so the cluster drops it and dials again instead of hanging.
My role
Networking & fault tolerance: moves activations between laptops and handles a laptop that stops
Under the hood
Pipeline parallelism suits a Wi-Fi cluster: only one activation tensor crosses the network per layer boundary, instead of the constant all-to-all traffic tensor parallelism needs. Keeping the API OpenAI-compatible means existing chat clients and agents work without changes, and shipping the dashboard inside the Python package means installs need no Node.
- Runtime
- Python 3.11+, PyTorch (CPU or CUDA)
- Discovery
- mDNS on the local network
- Transport
- Activation streaming between pipeline stages
- API
- OpenAI-compatible /v1 with tool calling
- Dashboard
- Browser UI bundled into the package
- Distribution
- pip, Homebrew, one-line shell and PowerShell installers
Outcomes
- One-line install on Windows, macOS and Linux, also available via pip and Homebrew
- OpenAI-compatible endpoint with tool calling for agent clients
- Zero-config peer discovery over mDNS with an in-browser host/accept flow
- Built the network layer: activation transport between laptops and recovery when a laptop drops out
Have something in mind?