paddler
Open-source LLM/VLM load balancer and serving platform for self-hosting LLMs (and VLMs) at scale 🏓🦙 Alternative to projects like llm-d, Docker Model Runner, etc but with less moving parts and simple deployments built a. paddler is open source, written primarily in Rust, released under Apache License 2.0. Its repository has 1.6K GitHub stars and 92 forks, and was last committed to within the last two weeks. Niched places it in Hosting & deployment.
Visit ↗