portable-usb-llm-agent

Portable USB LLM Agent is a portable, offline, local coding agent for a Windows USB drive. It runs any GGUF model of your choice through llama.cpp's local OpenAI-compatible server, driven by a small FastAPI agent that writes files, runs a narrow allowlist of commands, and packages finished projects into ZIP artifacts inside a workspace/ sandbox. An optional, opt-in backend can use a locally running Ollama install instead, selectable from the UI's Settings modal or the CLI.

Every model, backend, and tuning value is configured through a single .env file, and nothing is hardcoded to a specific model. The agent works through a sequential-role pipeline with five labeled roles shown live in the UI header: Planner, Implementer, Reviewer, Tester, and Packager. It provides sandboxed file operations, project import, optional command execution, overwrite protection, ZIP artifact packaging, a live streamed action timeline, file explorer and viewer, a CLI client, and a cross-platform launcher.

The project is Windows-portable only unless Linux/macOS llama-server binaries are added separately. Performance depends on host CPU, RAM, USB drive read speed, GPU/VRAM, and model/quant choice, and it is intended for small, well-scoped tasks rather than large codebases or long unattended sessions. The model and llama.cpp binary are excluded deliberately.