Choose a local model for Hermes when keeping inference on your own machine matters and your hardware can finish the work. Choose a cloud API when its measured quality, speed or capacity is worth sending the permitted data to that provider. Neither choice answers where every tool or helper runs.
Quick answer#
A local main model removes cloud inference charges for requests handled locally. It does not remove hardware costs, electricity, queue delays or human repair. It also does not prove that vision, compression, fallbacks and connected tools remain local. Test the complete workflow, not only a text reply.
For setup, use the Ollama guide. For billing diagnosis, use Hermes provider costs and rate limits. This comparison helps decide which route deserves that setup effort.
Compare the same finished task#
Pick one representative task and define what a passing result looks like before running it. Use the same files, tool permissions and acceptance check on both routes. Start with one worker; parallel requests can turn a usable local model into a queue.
Record time to a verified result, failed attempts, human corrections and actual charges. Include the cost of cloud helpers or fallback calls in the local result when they ran. Do not compare a local task that failed with the token price of a cloud task that succeeded.
Where local inference makes sense#
Local inference is worth testing when the data must stay on controlled infrastructure, you already own suitable hardware, or you can accept slower completion in exchange for fewer metered cloud requests. Model size, quantization, context length and tool-call reliability all affect the result. A model loading into memory is only the first check.
Where a cloud API makes sense#
A cloud API is worth testing when your workload needs capabilities or response times the local route cannot deliver reliably. You trade hardware operation for provider billing, data-handling terms, network dependency and quota limits. A subscription login and a direct API key may draw from different allowances; verify the payer with the API-key guide.
The cost calculator includes infrastructure and maintenance. No benchmark score or universal monthly saving is claimed here.
Local text does not prove local vision#
An October 5 community report describes local text use followed by unexpected remote vision behavior and a 401. It is a report of confusion, not a reproduced defect or proof that a particular model build supports the user's requested path.
Current official Hermes provider documentation says auxiliary tasks on provider: auto start with the main chat model; task-specific overrides and fallback policy can change the route. Inspect those settings before declaring the installation offline or free of cloud charges.
Use a non-sensitive test image and check:
- Which provider, endpoint and model handled the text request?
- Did the image go natively to that model or through an auxiliary vision task?
- Does the serving endpoint support the image request format, not just text completion?
- Was an explicit helper or fallback configured to use a cloud provider?
- Which endpoint returned any authentication error?
Do not remove authentication or publish a local model server to fix an unexplained 401. A local server or proxy can require credentials too. Find the rejecting endpoint first.
Verify the privacy boundary#
A local model and a local-only workflow are different claims. Inventory model requests, auxiliary tasks, search/extraction services, browser backends, connected apps and fallbacks. Any of those can send data outside the machine.
Test text, an image and a longer task separately with harmless inputs. Inspect server access logs and relevant provider usage. For a strict offline requirement, deliberately test the workflow with external network access blocked in a controlled environment after saving your work. A task failing offline reveals a dependency to investigate; a model claiming it stayed local is not evidence.
Use local LLM support for supported connection options and the privacy guide for the wider data boundary. Do not quietly add a cloud fallback to a workflow whose requirement is local-only processing.
Docker changes where localhost points#
When Hermes runs in a container and Ollama runs on the host, localhost inside the container refers to that container. Use the host address appropriate to your platform and verify reachability from the Hermes runtime. Do not expose the model port to the public internet just to make the connection work.
Keep network reachability separate from model capability: a reachable endpoint can still reject the model ID, tool schema, image request or context length. The Docker troubleshooting guide covers the runtime boundary; switching model providers will not repair a wrong network address.
Start with a small verification record#
Run the text check from the profile that will own the work. Replace work with its name.
hermes -p work config get model
hermes -p work doctor
hermes -p work chat -q "Reply with exactly: provider ok"
Keep configuration output private until checked for sensitive endpoint details. Then run a harmless tool task and verify the resulting file or answer. Record the model, server, context setting, elapsed time, failures and any external calls. Only increase context or concurrency after this baseline passes.
If local inference meets the acceptance check and privacy requirement, keep it. If not, identify the missing capability before buying hardware or moving the whole workload to a cloud provider.
Hosting is a separate choice#
A cloud model does not keep your laptop awake. A local model does not make the gateway reachable while its host is stopped. In both cases, always-on work still needs a maintained runtime.
The self-hosted Dashboard provides configuration and monitoring, not the full hosted browser-chat experience. Compare FlyHermes pricing when you want managed browser/mobile access and less infrastructure maintenance. Check its current model and billing terms separately; managed hosting is not a promise of unlimited inference.
Source: official Hermes provider and configuration documentation, checked October 5, 2026. The community vision report motivates the diagnostic checklist; no local-model benchmark or universal compatibility claim is inferred from it.