local-llm-env logo

local-llm-env

Free

Reproducible environment for running open-weight language models locally, combining llama.cpp (inference) and opencode (agent harness) behind a small set of make targets.

Visit Website
Related by shared tags and keywordsYOEAINamaste-AIgenai-learning-os
generative-aiagentsai-agentsllama-cppllmaiai assistantautomation

Overview

local-llm is a reproducible, low-configuration environment for running open-weight language models entirely on the user's own machine. It provisions a complete local inference stack with a single command, using llama.cpp as the inference runtime and opencode as the command-line agent interface, which provides tool use, context management, and a terminal UI. Once models are downloaded, no external API or network access is required during normal operation.

The project is intended as a convenient baseline for experimenting with locally hosted models and for users who would rather not assemble the toolchain by hand. It was tested on hardware with 6 GB of VRAM, so the default models are small, quantized variants of roughly 2B–4B parameters. Nothing in the setup is tied to this configuration, and resource use can be adjusted through Makefile variables and model choice.

Installation and startup are driven by make targets, and the default setup requires no manual editing of configuration files. Dependencies are pinned in uv.lock, and installation is a single scripted procedure. llama.cpp runs in router mode, serving every cached model on demand through one endpoint at http://127.0.0.1:8080. The connection between opencode and the local server is defined in opencode.json.

The project is described as a work in progress, created as a learning exercise to better understand large language models and the tooling used to run them locally. It may contain errors, omissions, or outdated information. Corrections, bug reports, and suggestions are welcome through the issue tracker.

Pricing

Free

Detailed plans have not been confirmed in our catalog. Check the official website for current limits and billing terms.

Visit Website

Prices and limits may change. Confirm the currency, billing period, seat minimum and usage caps on the official website.

Use Cases

  • Teams use local-llm to run open-weight language models entirely on their own machine without external API or network access.
  • Users use local-llm to provision a complete local inference stack with a single command.
  • Developers use local-llm as a convenient baseline for experimenting with locally hosted models.
  • Users use local-llm to serve multiple cached models on demand through one endpoint.
  • Users use local-llm to install and start the environment through make targets without manually editing configuration files.
  • Users use local-llm to check the state of installed binaries, the server process, and available models.
  • Users use local-llm to stop the llama.cpp server or clean up the environment, opencode, llama.cpp, and downloaded models.
  • Users use local-llm to extend the agent with optional Model Context Protocol servers for terminal operations or web search and fetch.

Features

Run open-weight language models locally

Set up reproducible model environments

Use llama.cpp for inference

Use opencode as agent harness

Manage workflows through make targets

Reviews

No reviews yet. Be the first to share your experience!