About grps_trtllm on AIStoreHub
Description: Higher performance OpenAI LLM service than vLLM serve: A pure C++ high-performance OpenAI LLM service implemented with GPRS+TensorRT-LLM+Tokenizers.cpp, supporting chat and function call, AI agents, distributed multi-GPU inference, multimodal capabilities, and a Gradio chat interface.
Asset Type: agent (agents).
Compatible Foundation Models: Frontier LLMs (GPT-4o, Claude 3.5 Sonnet, Gemini 1.5 Pro).
How to install or reuse: Using the official AIStoreHub CLI:pip install aistorehub && aihub pull community/grps-trtllm
