desafio-cosmeticbot

desafio-cosmeticbot is a repository for a reproducible evaluation suite and Quality Assurance pipeline for a cosmetics chatbot. It was developed during Desafio do Mês 1, and it contains the architecture, test suite, and QA pipeline focused on user experience and systemic safety. The ecosystem runs locally with foundation models and uses DeepEval and Pytest for deterministic validation, response compliance auditing, and hallucination mitigation through an LLM-as-a-Judge strategy.

The local infrastructure is based on Ollama. The bot model is Llama 3.1 8B via local Ollama, and the judge model is also Llama 3.1 8B via local Ollama. The repository includes modules such as chatbot.py, juiz.py, golden_dataset.py, test_suite.py, and main.py. The golden dataset is a matrix of 16 isolated test cases in four critical validation categories.

Quality validation uses three central metrics with strict thresholds: Answer Relevancy (≥ 0.7), Faithfulness (≥ 0.8), and G-Eval de Conformidade de Claims (≥ 0.8). The project also reports a baseline with 100% failure in initial tests and a refactoring phase focused on system prompt principles, including clear persona delimitation, literal instructions for disease cases, and blocking external knowledge.