Production ML infrastructure library for Elixir, published as a Hex package. Nx, Bumblebee, and Axon handle the compute. This library handles the infrastructure layer: routing, batching, caching, and resilience patterns.

The router classifies incoming prompts by semantic complexity and routes them to the appropriate model tier. Simple factual queries go to fast cheap models, complex analytical tasks go to capable expensive ones. No external API call needed for the routing decision itself.

Categories are defined by example prompts via CSV. On startup, examples are embedded using all-MiniLM-L6-v2 running locally through Bumblebee. Embeddings are averaged to create a centroid per category. Incoming prompts are embedded and compared via cosine similarity. Best matching category is returned with all scores. Results are cached for one hour.

Custom categories beyond the default quick/complex split are loaded from CSV at startup or reloaded at runtime. Scores for all categories are returned so callers can implement their own routing logic with confidence thresholds.

Planned components: request batching for throughput, semantic similarity caching, graceful load shedding under pressure, and model instance pooling. Early development, API may change before 1.0.