vllm-prefix-cache-bench
๐ฏSkillfrom vllm-project/vllm-skills
|Benchmarks vLLM's automatic prefix caching performance, measuring throughput and latency improvements from shared prompt prefixes in LLM serving scenarios using the OpenAI-compatible API.
Same repository
vllm-project/vllm-skills(6 items)
Installation
npx vibeindex add vllm-project/vllm-skills --skill vllm-prefix-cache-benchnpx skills add vllm-project/vllm-skills --skill vllm-prefix-cache-bench~/.claude/skills/vllm-prefix-cache-bench/SKILL.mdSKILL.md
More from this repository5
A collection of modular, reusable agent skills for deploying and benchmarking vLLM, including Docker deployment, Kubernetes deployment, simple deployment, and various performance benchmarking skills. Each skill is a self-contained directory following the Anthropic SKILL.md specification, installable as a Claude Code plugin marketplace.
A skill for quickly installing and deploying vLLM, starting a serving instance with a simple LLM model, and testing the OpenAI-compatible API endpoint.
A collection of modular skills for deploying and benchmarking vLLM, covering Docker deployment, Kubernetes deployment, simple local serving, prefix cache benchmarking, and OpenAI-compatible API endpoints.
Runs vLLM performance benchmarks using synthetic random data to measure throughput, TTFT, TPOT, and other key metrics without downloading external datasets. Part of a collection of skills for deploying and benchmarking vLLM.
Deploys vLLM to Kubernetes with GPU support, health probes, and an OpenAI-compatible API endpoint. Part of the vLLM Skills collection that also includes Docker deployment, simple deployment, and performance benchmarking skills.