Our scheduling engines run distributed compute jobs across mixed hardware fleets. By discovering and coordinating NVIDIA CUDA, AMD ROCm, Apple Metal, and Intel Arc GPUs over an encrypted peer-to-peer gossip protocol, fleets operate without a single centralized master node.
Business value
Lowers compute spending by 40% to 60% by using all available GPU hardware across on-premise servers and cloud instances.
Architecture & Implementation
- Peer-to-peer gossip discovery with encrypted node messaging and bidirectional gRPC streams
- Native hardware detection reading driver counters directly without third-party agent software
- Single-binary Go and Rust architecture with zero CGO and zero external runtime libraries
- Cross-compiled execution supporting Linux, macOS, and Windows across x86_64 and arm64
Critical Problems Solved
- Eliminates dependence on a single GPU vendor, utilizing idle AMD and Apple Silicon hardware
- Removes single points of failure by replacing fragile centralized coordinators with a gossip mesh
- Simplifies cluster administration to a single binary deployment that requires no setup scripts