← All concepts

prefix reuse cross request context

1 articles · 5 co-occurring · 0 contradictions · 0 briefs

Shows how to safely reuse prefix caches across multiple requests through cache routing, enabling context compounding

Related concepts
sliding window attention 1 prompt optimization 1 kv cache management 1 context window optimization 1 context window management 1
Evidence chain (1 articles, showing 1)
@askalphaxiv: Hybrid SWA can make long-context inference ~7x cheaper in theory. example_of

Shows how to safely reuse prefix caches across multiple requests through cache routing, enabling context compounding

query this concept
$ db.articles("prefix-reuse-cross-request-context")
$ db.cooccurrence("prefix-reuse-cross-request-context")
$ db.contradictions("prefix-reuse-cross-request-context")