RoutIR: Fast Server for Hosting Retrieval Models for RAG
RoutIR is a Python package that provides a simple and efficient wrapper around arbitrary retrieval models, including first stage retrieval, reranking, query expansion, and result fusion, with efficient asynchronous query batching and serving.
Quick Start
Install:
pip install routir
pip install "routir[dense,gpu]"
Start the service:
routir config.json --port 5000
Or use uvx:
uvx --with transformers --with torch routir config.json
User Guide
API Reference
Developer Guide