Show HN: Open-Source Load Balancer for Llama.cpp https://ift.tt/GuZPO67

posted by Thar Desert Times , No Comments

Show HN: Open-Source Load Balancer for Llama.cpp Stateful load balancer customized for llama.cpp (with a reverse proxy). https://ift.tt/xAJXT72 June 2, 2024 at 05:05AM