Llm Serving
Top open-source Llm Serving AI systems, agents, and MCP tools.
Discover and install verified Llm Serving AI systems, autonomous agent skills, and Model Context Protocol servers on AIStoreHub. Reusable, security-scanned, and ready for 1-click execution in Cursor, Claude Code, and terminal workflows.
Available AI Systems & Skills (5)
Open in Advanced FilterFrontier-class open models on a free Kaggle TPU v5e-8: GLM-5.3-Flash 320B MoE (~165 tok/s, our own JAX engine) and Qwen3.8-27B bf16 (~130 tok/s), 262k context, prefix caching. Works with Claude Code, Codex, opencode and pi.
High-throughput and low-latency LLM serving engine featuring PagedAttention, continuous batching, and CUDA kernel optimizations.
High-performance Python ASGI backend template for serving streaming LLM tokens, async agent tool endpoints, and OpenAPI schemas.
Higher performance OpenAI LLM service than vLLM serve: A pure C++ high-performance OpenAI LLM service implemented with GPRS+TensorRT-LLM+Tokenizers.cpp, supporting chat and function call, AI agents, distributed multi-GPU inference, multimodal capabilities, and a Gradio chat interface.
Build & Monetize for Llm Serving
Have you created an autonomous agent, MCP server, or Claude Code skill for Llm Serving? Publish it to AIStoreHub with dual-currency checkout (USD $ + INR ₹) and earn 85–90% net revenue.