✦
Hermes Agent
Open navigation menu

How-To Guide

Run Hermes Agent Locally with Ollama

Connect Hermes Agent to Ollama with the current quick launch or manual OpenAI-compatible endpoint, then verify context, tool calls, privacy, and fallbacks.

Quick answer

Run ollama launch hermes for the shortest current setup. For manual control, start Ollama, pull a tool-capable model, run hermes model, choose a custom endpoint, and enter http://127.0.0.1:11434/v1. Leave the API key empty, confirm the exact detected model, then test a real file-and-terminal task before enabling gateways or cron.

Ollama handles local model serving; Hermes supplies memory, tools, skills, cron, browser automation, and messaging gateways. Local inference removes per-token API billing, but your hardware, context window, tool-call reliability, and network-enabled tools still determine whether the setup is private and production-ready. Compare the broader Hermes local LLM options before choosing Ollama over LM Studio, vLLM, SGLang, or llama.cpp.

Deploy Hermes faster with FlyHermes

Managed cloud · API costs included · Skill library · Cancel anytime

Before you start:

  • ☑Hermes Agent installed and current; check hermes --version and hermes doctor
  • ☑Ollama installed on the same machine or a private network host
  • ☑Enough RAM/VRAM and disk for the chosen model; stronger tool-capable models often need substantially more than 8 GB
  • ☑A decision about whether web, browser, speech, memory, MCP, and fallback services are allowed to use the network

Steps

  1. 1

    Install Ollama and verify the server

    Install Ollama for your OS, then run ollama --version and curl http://127.0.0.1:11434/api/tags. Fix the server before changing Hermes if the second command cannot return the local model list.

  2. 2

    Use the one-command Hermes launch when you want the fastest path

    Run ollama launch hermes. Choose a local model, let Ollama point Hermes at http://127.0.0.1:11434/v1, and skip messaging until the CLI tool-use test passes.

  3. 3

    Pull a tool-capable model for manual setup

    Use a current tool-capable model that fits your RAM or VRAM; the Hermes docs currently demonstrate ollama pull gemma4:31b. Do not choose a chat-only model for file, terminal, browser, or cron work.

  4. 4

    Configure the manual endpoint

    Run hermes model, choose the custom endpoint path, enter http://127.0.0.1:11434/v1, leave the API key empty, and select the exact model returned by Ollama. Use 127.0.0.1 for a same-machine server.

  5. 5

    Handle context and slow local inference

    Leave context blank only when detection is correct. Hermes agent work needs a large context for system instructions and tool schemas; use a 64K-capable model variant when smaller windows are rejected. For slow CPU inference, put HERMES_API_TIMEOUT=1800 in ~/.hermes/.env.

  6. 6

    Run an agent acceptance test

    Run hermes chat -q "List the files in this directory, read README.md, and report the project name." Confirm Hermes actually called file or terminal tools; a normal chat reply does not prove local agent compatibility.

  7. 7

    Add a fallback only if the data may leave the machine

    Use hermes fallback or configure fallback_providers after the local path works. A hosted fallback improves reliability but changes the privacy boundary, so do not enable it for air-gapped or sensitive workloads.

  8. 8

    Connect messaging only after the local CLI passes

    Run hermes gateway setup, then hermes gateway start. Test a real Telegram, Discord, Slack, WhatsApp, Signal, or Email message and watch local latency before calling the bot production-ready.

Pro Tips

  • 💡Use ollama ps to see whether the model is loaded and whether GPU layers are offloaded.
  • 💡For an always-on gateway, configure Ollama keep-alive so the first message does not wait for a cold model load.
  • 💡Do not expose port 11434 directly to the public internet; keep it loopback-only, private, or behind authenticated network controls.
  • 💡Use /compress, not /compact, when a long local session needs context reduction.
  • 💡Compare local and hosted Hermes costs using hardware, electricity, uptime, and operator time—not only API tokens.

Troubleshooting

❌ `Connection refused` at `127.0.0.1:11434`

✅ Start Ollama with ollama serve or its OS service, then prove curl http://127.0.0.1:11434/api/tags works before reopening Hermes.

❌ The model replies but Hermes does not use tools

✅ Run a forced file-and-terminal task, not a greeting. Choose a model with tool-call support, verify the context is large enough, and inspect whether the response contains structured tool calls rather than prose that pretends work happened.

❌ Hermes rejects the model or reports insufficient context

✅ Use a model/variant configured for at least a 64K context window, then reconnect through hermes model. Large context also increases RAM/VRAM use, so choose a model that still fits without swapping.

❌ Responses take 30–120 seconds or time out

✅ Run ollama ps, choose a smaller quantization or model, enable GPU offload, reduce stale session context with /compress, and set HERMES_API_TIMEOUT=1800 in ~/.hermes/.env for slow CPU inference.

❌ The local bot stops replying after gateway setup

✅ Prove the local model with hermes chat -q first, then inspect hermes gateway status. A connected platform can still time out while Ollama cold-loads or a local tool turn exceeds the gateway patience window.

FAQ

What is the fastest way to run Hermes Agent with Ollama?

Run ollama launch hermes. Ollama can install/configure Hermes, point it at the local OpenAI-compatible endpoint, set the chosen model, and optionally begin gateway setup.

What Ollama URL should Hermes Agent use?

For Ollama on the same machine, use http://127.0.0.1:11434/v1. Verify the server first through http://127.0.0.1:11434/api/tags.

Does Ollama make every Hermes tool offline?

No. It localizes model inference. Web search, browser automation, remote MCP, hosted memory, cloud speech, and fallbacks can still use the network unless you disable or isolate them.

Why can my Ollama model chat but not run Hermes tools?

Chat ability is not tool-call reliability. Use a model that supports structured tools, provide enough context for Hermes schemas, and verify a real file-and-terminal task.

Should I use a cloud fallback with local Hermes?

Use one when reliability matters and the workload is allowed to leave the machine. Do not add a hosted fallback to an air-gapped or sensitive lane merely for convenience.

Related setup and cost guides

Hermes local LLM guide

Choose between Ollama, LM Studio, vLLM, SGLang, and llama.cpp.

Fully offline Hermes setup

Audit every network-capable component, not only model inference.

Hermes vs Ollama

Understand the difference between an agent runtime and a model server.

Local model cost comparison

Count hardware, power, latency, API spend, and operator time.

Provider fallback guide

Add a tested hosted route for permitted workloads.

Related Guides

local llm supportollama hermesrun hermes offlineconfigure ollama local providerollama