Configuration

This section contains auto-generated documentation for configuration modules.

Config Module

Pydantic models for service configuration.

class routir.config.config.ServiceConfig(*, name, engine, config, processor='BatchQueryProcessor', cache=-1, batch_size=32, cache_ttl=600, max_wait_time=0.05, cache_key_fields=<factory>, cache_redis_url=None, cache_redis_kwargs=<factory>, scoring_disabled=False)[source]

Bases: BaseModel

Configuration for a single search or ranking service.

One ServiceConfig entry in the "services" list corresponds to one loaded engine, one search processor, and optionally one scoring processor registered in ProcessorRegistry.

Example JSON:

{
    "name": "my-retriever",
    "engine": "Qwen3",
    "config": {
        "index_path": "/data/qwen3-index",
        "embedding_model_name": "Qwen/Qwen3-Embedding-8B"
    },
    "cache": 4096,
    "cache_ttl": 600,
    "batch_size": 16,
    "max_wait_time": 0.05
}

name

Service identifier used in API requests (the "service" field) and as the key in ProcessorRegistry.

Type:: str

engine

Class name of the Engine subclass to instantiate. Must be importable at startup — either a built-in engine or one loaded via file_imports.

Type:: str

config

Engine-specific parameters passed as config= to the engine constructor. Content varies by engine; see the engine’s __init__ for accepted keys. The special key "index_path" with the hfds:<repo> prefix triggers automatic download from Hugging Face Datasets.

Type:: dict

processor

Class name of the BatchProcessor subclass for the search role. Default "BatchQueryProcessor" works for most engines. Override only if you need custom batching or request logic.

Type:: str

cache

In-memory LRU cache size for search results (number of entries). -1 (default) disables the cache entirely. Ignored when cache_redis_url is set.

Type:: int

cache_ttl

Cache entry time-to-live in seconds (default 600). Applies to both LRU and Redis caches.

Type:: int

batch_size

Maximum requests accumulated into one batch before the engine processes them (default 32).

Type:: int

max_wait_time

Maximum seconds to wait for a batch to fill before processing a partial batch (default 0.05 s). Lower values reduce latency; higher values improve GPU utilisation.

Type:: float

cache_key_fields

Request fields included in the cache key. Default ["query", "limit"]. Add extra fields (e.g. "subset") whenever they affect the results.

Type:: list[str]

cache_redis_url

Redis connection URL for distributed caching (e.g. "redis://localhost:6379"). When set, Redis replaces the in-memory LRU cache.

Type:: str, optional

cache_redis_kwargs

Additional keyword arguments forwarded to the Redis client (e.g. {"password": "…", "db": 1}).

Type:: dict, optional

scoring_disabled

When True, the scoring/reranking processor for this service is not registered even if the engine implements score_batch. Useful when you want search-only access to an engine that also supports reranking.

Type:: bool

name: str

engine: str

config: Dict[str, Any]

processor: str

cache: int

batch_size: int

cache_ttl: int

max_wait_time: float

cache_key_fields: List[str]

cache_redis_url: str | None

cache_redis_kwargs: Dict[str, Any] | None

scoring_disabled: bool

model_config = {}: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

class routir.config.config.CollectionConfig(*, name, doc_path=None, processor='ContentProcessor', offset_source='offsetfile', id_field='id', content_field='text', id_to_lang_mapping=None, cache_path=None, force_load_all_documents=False)[source]

Bases: BaseModel

Configuration for a document collection.

Collections expose the /content endpoint and are used by reranking pipeline stages to fetch document text by ID.

Note

The class name has a historical typo (three ls). It is kept as-is to avoid breaking existing imports.

Example JSON:

{
    "name": "my-corpus",
    "doc_path": "/data/corpus.jsonl",
    "id_field": "docid",
    "content_field": "text"
}

name

Collection identifier used in API requests (the "collection" field) and as the key in ProcessorRegistry.

Type:: str

doc_path

Path to the JSONL (or gzip-compressed) document file. Each line must be a JSON object. May be a local path or an hfds:<repo> URL for Hugging Face Datasets.

Type:: str, optional

processor

Class name of the content-processor to use. Default "ContentProcessor" provides offset-based random access. Use "IRDSProcessor" to load from an ir_datasets dataset ID.

Type:: str

offset_source

Strategy for random document access:

"offsetfile" (default) — builds a byte-offset map (.offsetmap sidecar file) for fast O(1) lookup in a JSONL file.
"msmarco_seg" — reads from sharded gzipped files in the MSMARCO v2.1 segmented document format, using embedded byte offsets from the document ID.

Type:: str

id_field

JSON key whose value is the document ID (default "id").

Type:: str

content_field

JSON key(s) whose values are concatenated (space-joined) to form the document text returned by /content. Accepts a single string or a list for multi-field concatenation (e.g. ["title", "body"]). Always stored as a list internally after validation.

Type:: str or list[str]

id_to_lang_mapping

Path to a pickle file mapping document IDs to language codes. Used by processors that serve multilingual corpora.

Type:: str, optional

cache_path

Directory for the .offsetmap cache file. Defaults to the same directory as doc_path.

Type:: str, optional

force_load_all_documents

When True, all documents are loaded into memory at startup for maximum throughput. Only suitable for small corpora; default False uses on-demand offset-based access.

Type:: bool

name: str

doc_path: str | None

processor: str

offset_source: Literal['msmarco_seg', 'offsetfile']

id_field: str

content_field: str | List[str]

id_to_lang_mapping: str | None

cache_path: str | None

force_load_all_documents: bool

model_post_init(_CollectionConfig__context)[source]: Ensure content_field is always a list.

model_config = {}: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

class routir.config.config.Config(*, services=<factory>, collections=<factory>, server_imports=<factory>, file_imports=<factory>, dynamic_pipeline=True, pipeline_aliases=<factory>)[source]

Bases: BaseModel

Top-level configuration for the RoutIR service.

Passed as a JSON file (or JSON string) to routir <config.json>. Parsed by load_config(), which initialises all collections and services and registers them with ProcessorRegistry.

Example JSON skeleton:

{
    "file_imports": ["./my_engine.py"],
    "collections": [
        {
            "name": "my-corpus",
            "doc_path": "/data/corpus.jsonl"
        }
    ],
    "services": [
        {
            "name": "my-retriever",
            "engine": "MyEngine",
            "config": {"index_path": "/data/index"}
        }
    ],
    "server_imports": ["http://other-host:5000"]
}

services

Search/ranking services to load and register. Each entry loads one engine and creates its processors.

Type:: list[ServiceConfig]

collections

Document collections to register as content services. Required for reranking pipeline stages.

Type:: list[CollectionConfig]

server_imports

URLs of remote RoutIR servers whose services are proxied locally via Relay. Discovered automatically from each server’s /avail endpoint at startup.

Type:: list[str]

file_imports

Paths to Python files loaded before any service is initialised. Use this to register custom Engine subclasses.

Type:: list[str]

dynamic_pipeline

When True (default), the /pipeline endpoint accepts arbitrary pipeline DSL strings at request time. Set to False to restrict the server to pre-defined services only.

Type:: bool

pipeline_aliases

Named shortcuts for pipeline DSL fragments. An alias is a single identifier that expands to a (possibly multi-stage) pipeline at request time. Aliases may reference other aliases; cycles raise an error at startup. Each alias name must not collide with any service or collection name.

Example:

{
    "ragtime2":    "{zho%100, rus%100, spa%100, eng%100}ScoreFusion",
    "ragtime2-rr": "ragtime2%100 >> nemotron%20"
}

With these aliases, ragtime2%20 expands to {zho%100, rus%100, spa%100, eng%100}ScoreFusion%20 (the call-site limit is applied to the last stage of the alias body).

Type:: dict[str, str]

model_config = {'extra': 'forbid'}: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

services: List[ServiceConfig]

collections: List[CollectionConfig]

server_imports: List[str]

file_imports: List[str]

dynamic_pipeline: bool

pipeline_aliases: Dict[str, str]

Config Loader

async routir.config.load.auto_add_relay_services(servers)[source]

Discover and register services from remote RoutIR servers as local proxies.

Queries each server’s /avail endpoint to list its available services, then creates Relay-backed processors for every service not already registered locally. This lets the local server transparently forward requests to the remote server.

Only "search" and "score" service types are proxied. Services already registered locally take precedence (remote services with the same name are skipped).

Parameters:: servers (list[str]) – Base URLs of remote RoutIR servers to import from, e.g. ["http://gpu-host-1:5000", "http://gpu-host-2:5000"]. A single string is also accepted.

routir.config.load.load_index_from_hfds(repo_id)[source]

Download an index from HuggingFace Datasets.

Parameters:: repo_id (str) – Repository ID (with optional ‘hfds:’ prefix)
Returns:: Path to the downloaded index directory

async routir.config.load.load_config(config)[source]

Parse the service configuration and register all collections and services.

This is the main initialization entry point called by the server at startup. It performs the following steps in order:

Parse the JSON config (file path or raw string) into a Config model.
Load any Python files listed in file_imports (custom engine classes).
For each collection, create and register a content processor.
For each service, instantiate the engine and register search (and optionally score) processors. Index paths with the hfds: prefix are downloaded from Hugging Face Datasets first.
Discover and proxy services from remote servers listed in server_imports.

Parameters:: config (str) – Either a file path to a JSON config file or a raw JSON string. File paths are read and parsed automatically.

Note

This function modifies the global ProcessorRegistry singleton in place. It is not safe to call concurrently.