← Back to Blog
How to run a local LLM as a coding agent.
Run Ornith 1.0 locally with Ollama and OpenHands, then turn a 9B model into a private coding agent for bounded offline coding and private work.
Run Ornith 1.0 locally with Ollama and OpenHands, then turn a 9B model into a private coding agent for bounded offline coding and private work.
A local LLM is a language model that runs on your own computer instead of sending prompts to a cloud API.
I tested Ornith 1.0 9B on a MacBook Pro with an Apple M5 chip, 24 GB of unified memory, and macOS 26.2. The model ran locally through Ollama, then connected to OpenHands as a local coding agent.
The setup uses two commands to install the model. The extra work is configuring the model for an agent harness, because a local model in a chat window is not the same tool as a local coding agent.
This guide covers:
A local LLM is a language model that runs on hardware you control. The model weights are downloaded to your computer, and prompts stay on that machine while the model is running.
That changes the cost and privacy model. After downloading Ornith 1.0 9B, the setup has no per-token bill and does not require an account to run. The model can also work offline.
This is different from using Claude or another hosted model through a subscription or API. Cloud models give you access to larger frontier systems, while a local LLM gives you control over where the model runs.
The model in this guide is Ornith 1.0 9B from DeepReinforce. It was released on 25 June 2026 under an MIT license. The Ornith family includes 9B and 31B dense models, plus 35B and 397B mixture-of-experts models.
The 9B model is the practical starting point for this local LLM setup because its Q6_K GGUF file is 7.4 GB.
You need a Mac with enough unified memory to hold the model, macOS, your other applications, and the model's working memory.
I tested the setup on a MacBook Pro with:
The 7.4 GB model loads entirely on the Metal GPU on that machine. A public field test also confirmed usable performance on a 16 GB M1 Mac, although the source material does not include the original field-test URL.
The memory calculation is straightforward. The model takes 7.4 GB, macOS and other applications need roughly 6 to 7 GB, and the remaining memory is available for the model's context and working state.
A 16 GB Mac is the entry point for this model. A 24 GB Mac gives the setup more room for a large context window and other applications.
Local LLM hardware matters because the GPU and CPU are not separate from system memory on Apple Silicon. The model shares unified memory with the rest of the operating system.
A local coding agent combines a model with a harness that lets the model interact with files, commands, and a workspace.
The model generates the reasoning and instructions. The harness gives it the ability to inspect a folder, write files, run commands, see errors, and try again.
That distinction matters with Ornith 1.0 9B. In a plain chat window, the model behaves like a normal 9B model. The reason to use it for local LLM coding is the agent harness around it.
DeepReinforce reports a 69.4 score for the 9B model on SWE-Bench Verified inside its agent harness. The 397B model is reported at 82.4. Those are vendor figures, and the 69.4 result applies to the harness setup rather than a plain chat session.
A chat window only tests whether the model can answer a prompt. An agent loop tests whether it can plan a task, make a change, run the relevant command, inspect the result, and recover from an error.
The harness is what turns the model into a local AI coding assistant.
I installed Ollama, downloaded the Ornith 1.0 9B Q6_K GGUF, and connected it to OpenHands.
The model file is 7.4 GB. Ollama serves the model locally, and OpenHands provides the coding-agent environment.
The setup worked on the 24 GB M5 Mac with the model running at 100 percent on the Metal GPU. The final Ollama configuration used:
OpenHands needs at least a 22K context window. Ollama defaults to 4K, so the default configuration is too small for this workflow.
I used 128K with a q8_0 cache instead of pushing to 256K with a lossy q4 cache. The 128K configuration gives the better cache quality at the same stated memory cost, and real agent sessions rarely need to pass 128K.
The setup also needs to survive an Ollama restart. I placed the context and cache settings in a LaunchAgent at:
~/Library/LaunchAgents/com.ollama.serve.plistWithout that persistent configuration, a restart returns Ollama to its defaults and OpenHands loses the context window it needs.
The Ornith GGUF contains an embedded Jinja chat template that fails when a harness inserts system messages in the middle of a conversation.
The error is:
System message must be at the beginningPlain chat does not trigger the issue. That is why the model can look functional when you first test it with Ollama, then fail as soon as OpenHands starts an agent loop.
The fix is to export the template from the GGUF, remove the two raise_exception guards that reject the mid-conversation system messages, and rewrite the GGUF with the patched template using --chat-template-file.
The patched model is registered under a new name:
ornith-ohOpenHands then points to openai/ornith-oh rather than the original model name.
This is the part of the setup that is easy to miss. If you only test the model in a chat window, you can conclude that the model works. The failure appears when the harness begins managing the conversation.
The basic local LLM install uses Ollama and the Ornith 1.0 9B Q6_K model.
Install Ollama with Homebrew:
brew install ollamaOllama serves the model on your Mac and exposes the local endpoint that OpenHands uses later.
Pull the Q6_K GGUF from Hugging Face:
ollama pull hf.co/deepreinforce-ai/Ornith-1.0-9B-GGUF:Q6_KThe download is 7.4 GB. The source material notes that Hugging Face can throttle the download, so the pull may take time and can resume if interrupted.
Run the model with Ollama and send it a simple prompt:
ollama run hf.co/deepreinforce-ai/Ornith-1.0-9B-GGUF:Q6_KA think block appears before the answer. That is the model's reasoning output, not a stalled process.
This test confirms that Ollama can load the model. It does not confirm that the model is ready for OpenHands.
Export the GGUF's embedded template, remove the two guards that throw the system-message error, and rewrite the model using the patched template file.
Register the result as:
ornith-ohThe exact export and rewrite commands are not included in the supplied source material, so I am not adding commands that have not been tested here.
Set the Ollama configuration to:
Context: 131072
Flash attention: enabled
KV cache: q8_0Then place those settings in the LaunchAgent at:
~/Library/LaunchAgents/com.ollama.serve.plistThe LaunchAgent keeps the settings after a restart. OpenHands needs at least 22K, so the default 4K context is not enough.
OpenHands provides the harness for local LLM coding.
Install and launch the CLI with:
uvx --python 3.12 --from openhands-ai openhandsUse these three values in the OpenHands configuration:
Model: openai/ornith-oh
Base URL: http://localhost:11434/v1
API key: any non-empty stringThe API key does not authenticate against a paid provider in this setup. Ollama is serving the model locally.
If you use the Docker web UI instead of the local CLI, use:
http://host.docker.internal:11434/v1The model name and API key remain the same.
OpenHands is the layer that gives Ornith access to a workspace. It can inspect files, write changes, run commands, and respond to tool results. That workflow is what makes this a local coding agent rather than a model sitting in a chat box.
For broader background on agent workflows, my guides to Claude Code agents, how to use Claude Code, and Claude Code MCP cover the same general distinction between a model and the tools around it.
A 9B local model works best when the task is bounded and the workspace is small.
I use three rules for local LLM coding.
Keep the agent inside a bounded folder. A 9B model can lose track of the task when it has to search through thousands of unrelated files.
A small repository or a single project directory gives the model a clearer working set. It also makes the agent's actions easier to review.
Create a git commit before handing the folder to the agent.
That gives you a clean restore point if the model makes a poor change or edits more files than you expected. The local agent can work quickly, but the commit keeps the consequences reversible.
Keep difficult architectural decisions, ambiguous debugging, and high-risk work with a frontier model.
A local 9B model is a separate tool. It is useful for private data, offline work, and bounded coding tasks. It is not a replacement for Claude or another frontier system.
A local 9B model has clear limits.
It is not a Claude replacement. The model can handle bounded tasks, but it does not provide the same level of judgment as a frontier model on complex work.
The 69.4 SWE-Bench Verified score should not be read as proof that every local coding session will match a much larger hosted model. It is DeepReinforce's own figure, and it was achieved inside the agent harness.
The model also needs the right context configuration. With Ollama's default 4K context, OpenHands cannot function properly. A model can appear broken when the actual issue is a serving default.
The chat-template problem creates another failure mode. Plain chat works, while the agent loop fails. That means a normal installation test does not cover the complete workflow.
Local is also not automatically cheaper or smarter than a small cloud model. The reason to run this setup is privacy and control. Client files can remain on the machine, and offline work does not depend on a cloud endpoint.
A local LLM and a cloud coding agent solve different problems.
Use a local coding agent when the files should stay on your machine, the work needs to continue offline, or the task is bounded enough for a 9B model.
Use a cloud agent when the task needs frontier-level reasoning, a larger model, or broader judgment across a complex codebase.
The practical setup is not either-or. I keep heavy thinking with Claude and use the local model for work that fits its lane.
The local model handles privacy-sensitive and repetitive tasks. The frontier model handles the work where model capability matters more than local execution.
A local coding agent is worth setting up if privacy, offline access, or control over model execution matters to you.
This Ornith setup is a good entry point for technical builders who already work in a terminal and have a Mac with at least 16 GB of unified memory. It also gives consultants and agency owners a way to test local coding workflows with client data that should not leave the machine.
It is not worth treating as a universal replacement for cloud models. The 9B model needs a harness, careful context configuration, bounded folders, and realistic task selection.
My verdict is simple. Use Ornith locally for private, offline, bounded coding. Keep a frontier model available for difficult reasoning and work that needs stronger judgment.
There is no single best local LLM for every coding task. In this setup, I used Ornith 1.0 9B with Ollama and OpenHands because the model was tested inside an agent harness and ran on a 24 GB M5 Mac.
The 9B model is suited to bounded local coding tasks, not every software-engineering problem.
Ornith 1.0 9B can run on a 16 GB Mac according to a public field test referenced in the source brief. The exact field-test URL was not included in the supplied material.
The model file is 7.4 GB, so the remaining memory must also cover macOS, other applications, the context window, and the model's working memory.
This setup has no per-token bill after the model download and does not require an account to run. You still need compatible hardware and you pay the electricity cost of operating that hardware.
The model download is 7.4 GB, and the setup uses Ollama to serve it locally.
Yes. The model runs on the local machine, so the coding workflow can continue without sending prompts to a cloud model.
You still need internet access to install Ollama, download the model, and retrieve the required software before working offline.
Yes, but a plain chat installation is not enough. You need a harness such as OpenHands that can provide file access, command execution, and feedback from the workspace.
With Ornith, you also need to patch the GGUF chat template and register the patched model as ornith-oh.
Ollama defaults to a 4K context window, while OpenHands needs at least 22K to function.
The setup I tested used a 131072 context window, flash attention, and a q8_0 KV cache. Those settings were persisted in a LaunchAgent so they survived Ollama restarts.
No. A local 9B model is not a Claude replacement.
It has a useful lane for private data, offline work, and bounded coding tasks. More difficult reasoning should stay with a frontier model.
Model files and the Q6_K GGUF used in this setup.
Official Ornith repository and local OpenHands setup context.
Presentations/Local-Coding-Agent-Ornith/training-walkthrough.mdPrimary local setup walkthrough. Commands and configuration were tested on a MacBook Pro with an Apple M5 chip, 24 GB unified memory, and macOS 26.2 on 2 July 2026.
Content Creation/Organic Posts/Local Coding Agent Ornith - 2026-07-27/brief.mdVerified source brief containing the model, hardware, benchmark attribution, template bug, OpenHands wiring, context configuration, and usage limitations.
Install Ollama, pull Ornith 1.0 9B, patch the template, and connect the result to OpenHands.
The useful version of local LLM coding is not asking a small model to replace every cloud system. It is giving a private model a bounded folder, a clean git checkpoint, and tasks that fit the hardware.
That is enough to make local AI coding practical without pretending the 9B model is a frontier system.
Five interactive lessons. Install Claude Code, build your first automation, and deploy it live on the internet — all in under an hour. Free, no coding required.
Grab the Blueprint →