omlx_agents

omlx_agents is a Python playground for implementing different types of AI agents, from simple chat calls to structured output and beyond. All agents are tested against free, open-weight models served locally by oMLX on an Apple M5 Max with 128 GB RAM. oMLX exposes an OpenAI-compatible API, so the scripts use the official openai Python client pointed at http://localhost:8000/v1. The current model used is Qwen3.6-35B-A3B-MLX-8bit.

The repository includes a basic chat completion agent (main.py) and a structured output agent that turns text into validated Pydantic objects. A RAG citation agent is in progress, currently covering the retrieval half of RAG: finding the text that answers a question and remembering where it came from. The roadmap lists twelve agent patterns, all built with open models only, running locally, with no paid model APIs.

The structured output agent addresses six parts of reliable LLM output: reliable outputs via a Pydantic model contract, schema validation with model_validate_json(), separate error handling for API and validation failures, retry logic with backoff and semantic repair, logging of validation and API attempts, and safe continuation via an AgentResult object. The agent is generic over the output model, so agent.run(prompt, output_model) works with any Pydantic model.

The RAG citation agent uses a small set of policy Documents, chunking with overlapping word windows, keyword-based retrieval scoring, and a relevance cut-off to drop weak matches. Known limitations include keyword matching only, every word counting towards the score, and no answer generation yet. The project requires uv and a running oMLX server, and tests use a fake client with scripted replies.