{"id":23751,"date":"2010-02-26T00:51:24","date_gmt":"2010-02-26T00:51:24","guid":{"rendered":"https:\/\/scannn.com\/magnitudedev-magnitude-open-source-inference-server-that-runs-the-best-local-models-for-your-hardware-plugged-into-the-agent-you-already-use-works-with-pi-opencode-hermes-openclaw-codex-claude\/"},"modified":"2010-02-26T00:51:24","modified_gmt":"2010-02-26T00:51:24","slug":"magnitudedev-magnitude-open-source-inference-server-that-runs-the-best-local-models-for-your-hardware-plugged-into-the-agent-you-already-use-works-with-pi-opencode-hermes-openclaw-codex-claude","status":"publish","type":"post","link":"https:\/\/scannn.com\/lv\/magnitudedev-magnitude-open-source-inference-server-that-runs-the-best-local-models-for-your-hardware-plugged-into-the-agent-you-already-use-works-with-pi-opencode-hermes-openclaw-codex-claude\/","title":{"rendered":"magnitudedev\/magnitude: Open source inference server that runs the best local models for your hardware, plugged into the agent you already use. Works with Pi, OpenCode, Hermes, OpenClaw, Codex, Claude Code, Oh My Pi, and Cline. \u00b7 GitHub"},"content":{"rendered":"\n<div id=\"\">\n<p align=\"center\" dir=\"auto\">\n  <themed-picture data-catalyst-inline=\"true\"><picture><source media=\"(prefers-color-scheme: dark)\" srcset=\"https:\/\/github.com\/magnitudedev\/magnitude\/raw\/main\/assets\/brand\/icon-dark.svg\"><source media=\"(prefers-color-scheme: light)\" srcset=\"https:\/\/github.com\/magnitudedev\/magnitude\/raw\/main\/assets\/brand\/icon-light.svg\"><br \/>\n  <\/source><\/source><\/picture><\/themed-picture><\/p>\n<p align=\"center\" dir=\"auto\"><strong>Run your agent on local models. Free, private, and offline.<\/strong><\/p>\n<p align=\"center\" dir=\"auto\">\n  <a href=\"https:\/\/docs.magnitude.dev\" rel=\"nofollow\"><img decoding=\"async\" src=\"https:\/\/camo.githubusercontent.com\/755fdc537ec9c427889e3dc9dffec8273c41c519983a770ecdcb062fc4f39de2\/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f2546302539462539332539352d446f63732d3033363961313f7374796c653d666c61742d737175617265266c6162656c436f6c6f723d30333639613126636f6c6f723d67726179\" alt=\"Documentation\" data-canonical-src=\"https:\/\/img.shields.io\/badge\/%F0%9F%93%95-Docs-0369a1?style=flat-square&amp;labelColor=0369a1&amp;color=gray\" style=\"max-width: 100%;\"\/><\/a><br \/>\n  <a href=\"https:\/\/discord.gg\/EHt48pPWdC\" rel=\"nofollow\"><img decoding=\"async\" src=\"https:\/\/camo.githubusercontent.com\/de250f576fb329219910076dde09e810f1663c08d2d9edf4206f404c2e332f0e\/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f446973636f72642d4a6f696e2d3538363546323f7374796c653d666c61742d737175617265266c6f676f3d646973636f7264266c6f676f436f6c6f723d7768697465266c6162656c436f6c6f723d35383635463226636f6c6f723d67726179\" alt=\"Discord\" data-canonical-src=\"https:\/\/img.shields.io\/badge\/Discord-Join-5865F2?style=flat-square&amp;logo=discord&amp;logoColor=white&amp;labelColor=5865F2&amp;color=gray\" style=\"max-width: 100%;\"\/><\/a><br \/>\n  <a href=\"https:\/\/x.com\/usemagnitude\" rel=\"nofollow\"><img decoding=\"async\" src=\"https:\/\/camo.githubusercontent.com\/4a3f353097988cd6b8898934df554de9c24e537967dd3063ab267263fcaf77a4\/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f547769747465722d466f6c6c6f772d3030303030303f7374796c653d666c61742d737175617265266c6f676f3d78266c6f676f436f6c6f723d7768697465266c6162656c436f6c6f723d30303030303026636f6c6f723d67726179\" alt=\"Follow Magnitude on Twitter\" data-canonical-src=\"https:\/\/img.shields.io\/badge\/Twitter-Follow-000000?style=flat-square&amp;logo=x&amp;logoColor=white&amp;labelColor=000000&amp;color=gray\" style=\"max-width: 100%;\"\/><\/a><br \/>\n  <a href=\"https:\/\/github.com\/magnitudedev\/magnitude\/stargazers\"><img decoding=\"async\" src=\"https:\/\/camo.githubusercontent.com\/f46074392f1d3f080ae7ffa7427dca2654348aa06e3691b16d81dbc87270edb9\/68747470733a2f2f696d672e736869656c64732e696f2f6769746875622f73746172732f6d61676e69747564656465762f6d61676e6974756465\" alt=\"GitHub Repo stars\" data-canonical-src=\"https:\/\/img.shields.io\/github\/stars\/magnitudedev\/magnitude\" style=\"max-width: 100%;\"\/><\/a><br \/>\n  <a href=\"https:\/\/www.npmjs.com\/package\/@magnitudedev\/cli\" rel=\"nofollow\"><img decoding=\"async\" src=\"https:\/\/camo.githubusercontent.com\/243fe9911dab82744a93df6b8f86bf83375f1aa2855ae728db14e6f3ebd57e21\/68747470733a2f2f696d672e736869656c64732e696f2f6e706d2f762f2534306d61676e6974756465646576253246636c69\" alt=\"npm version\" data-canonical-src=\"https:\/\/img.shields.io\/npm\/v\/%40magnitudedev%2Fcli\" style=\"max-width: 100%;\"\/><\/a>\n<\/p>\n<p dir=\"auto\">Magnitude is an open source inference server that runs the best local models for your hardware, plugged into the agent you already use. It profiles your machine, recommends the models that fit, then downloads, tunes, and runs them. Works with Pi, OpenCode, Hermes, OpenClaw, Codex, Claude Code, Oh My Pi, and Cline, or use the built-in harness.<\/p>\n<p dir=\"auto\"> Help us reach more developers and grow the Magnitude community. Star this repo!<\/p>\n<themed-picture data-catalyst-inline=\"true\"><picture><source media=\"(prefers-color-scheme: dark)\" srcset=\"https:\/\/github.com\/magnitudedev\/magnitude\/raw\/main\/assets\/readme\/ecosystem-dark.png\"><source media=\"(prefers-color-scheme: light)\" srcset=\"https:\/\/github.com\/magnitudedev\/magnitude\/raw\/main\/assets\/readme\/ecosystem-light.png\"><img decoding=\"async\" alt=\"Pi, OpenCode, Hermes, Codex, Claude Code, and OpenClaw connected to Magnitude, which runs local models for your hardware.\" src=\"https:\/\/github.com\/magnitudedev\/magnitude\/raw\/main\/assets\/readme\/ecosystem-light.png\"\/><br \/>\n<\/source><\/source><\/picture><\/themed-picture>\n<p dir=\"auto\"><strong>Send this to your agent to walk through models and setup:<\/strong><\/p>\n<div class=\"snippet-clipboard-content notranslate position-relative overflow-auto\" data-snippet-clipboard-copy-content=\"Set up local models for me with the Magnitude CLI. Install it with `npm i -g @magnitudedev\/cli` (or my package manager), then run `magnitude docs onboarding` and follow the instructions.\">\n<pre lang=\"text\" class=\"notranslate\"><code>Set up local models for me with the Magnitude CLI. Install it with `npm i -g @magnitudedev\/cli` (or my package manager), then run `magnitude docs onboarding` and follow the instructions.\n<\/code><\/pre>\n<\/div>\n<p dir=\"auto\">Your agent will profile your hardware, walk you through the best local models for it, download the ones you pick, and switch itself over to them.<\/p>\n<p dir=\"auto\">Magnitude supports macOS and Linux. Windows is supported through WSL.<\/p>\n<details>\n<summary>Want to browse the models directly?<\/summary>\n<div class=\"highlight highlight-source-shell notranslate position-relative overflow-auto\" dir=\"auto\" data-snippet-clipboard-copy-content=\"npm i -g @magnitudedev\/cli&#10;magnitude setup\">\n<pre>npm i -g @magnitudedev\/cli\nmagnitude setup<\/pre>\n<\/div>\n<p dir=\"auto\">The interactive setup lets you browse the recommended models and choose one yourself.<\/p>\n<\/details>\n<ul dir=\"auto\">\n<li><strong>Free to run:<\/strong> no token costs, API keys, or rate limits<\/li>\n<li><strong>Fully private and offline:<\/strong> models, prompts, and files stay on your machine<\/li>\n<li><strong>Agent-first setup:<\/strong> one prompt and your agent walks you through the rest<\/li>\n<li><strong>Knows your hardware:<\/strong> profiles your chip, memory, and bandwidth<\/li>\n<li><strong>Recommends what fits:<\/strong> the best models for your machine, with estimated tok\/s<\/li>\n<li><strong>Tuned end to end:<\/strong> speculative decoding, concurrency, all set for your machine<\/li>\n<li><strong>Models on demand:<\/strong> loaded on request, unloaded when idle or memory fills<\/li>\n<li><strong>Open source:<\/strong> Apache 2.0, yours to modify<\/li>\n<\/ul>\n<p dir=\"auto\">An open source inference server that runs the best local models for your hardware, plugged into the agent you already use. It profiles your machine, recommends the models that fit, then downloads, tunes, and runs them.<\/p>\n<p dir=\"auto\">There&#8217;s no fixed minimum. Magnitude profiles your hardware and recommends the best models for your machine. More memory lets you run larger models.<\/p>\n<div class=\"markdown-heading\" dir=\"auto\">\n<h3 tabindex=\"-1\" class=\"heading-element\" dir=\"auto\">Why not just have my agent set up Ollama?<\/h3>\n<p><a id=\"user-content-why-not-just-have-my-agent-set-up-ollama\" class=\"anchor\" aria-label=\"Permalink: Why not just have my agent set up Ollama?\" href=\"#why-not-just-have-my-agent-set-up-ollama\"><svg data-component=\"Octicon\" class=\"octicon octicon-link\" viewbox=\"0 0 16 16\" version=\"1.1\" width=\"16\" height=\"16\" aria-hidden=\"true\"><path d=\"m7.775 3.275 1.25-1.25a3.5 3.5 0 1 1 4.95 4.95l-2.5 2.5a3.5 3.5 0 0 1-4.95 0 .751.751 0 0 1 .018-1.042.751.751 0 0 1 1.042-.018 1.998 1.998 0 0 0 2.83 0l2.5-2.5a2.002 2.002 0 0 0-2.83-2.83l-1.25 1.25a.751.751 0 0 1-1.042-.018.751.751 0 0 1-.018-1.042Zm-4.69 9.64a1.998 1.998 0 0 0 2.83 0l1.25-1.25a.751.751 0 0 1 1.042.018.751.751 0 0 1 .018 1.042l-1.25 1.25a3.5 3.5 0 1 1-4.95-4.95l2.5-2.5a3.5 3.5 0 0 1 4.95 0 .751.751 0 0 1-.018 1.042.751.751 0 0 1-1.042.018 1.998 1.998 0 0 0-2.83 0l-2.5 2.5a1.998 1.998 0 0 0 0 2.83Z\"\/><\/svg><\/a><\/div>\n<p dir=\"auto\">Your agent would be guessing. It doesn&#8217;t know your hardware, which quant fits, or how fast it&#8217;ll run. Magnitude gives it a catalog with recommendations computed for your machine, an onboarding flow that writes your harness config, and inference built for agent workloads. Models load just in time and unload when idle or memory gets tight.<\/p>\n<div class=\"markdown-heading\" dir=\"auto\">\n<h3 tabindex=\"-1\" class=\"heading-element\" dir=\"auto\">Which harnesses work with it?<\/h3>\n<p><a id=\"user-content-which-harnesses-work-with-it\" class=\"anchor\" aria-label=\"Permalink: Which harnesses work with it?\" href=\"#which-harnesses-work-with-it\"><svg data-component=\"Octicon\" class=\"octicon octicon-link\" viewbox=\"0 0 16 16\" version=\"1.1\" width=\"16\" height=\"16\" aria-hidden=\"true\"><path d=\"m7.775 3.275 1.25-1.25a3.5 3.5 0 1 1 4.95 4.95l-2.5 2.5a3.5 3.5 0 0 1-4.95 0 .751.751 0 0 1 .018-1.042.751.751 0 0 1 1.042-.018 1.998 1.998 0 0 0 2.83 0l2.5-2.5a2.002 2.002 0 0 0-2.83-2.83l-1.25 1.25a.751.751 0 0 1-1.042-.018.751.751 0 0 1-.018-1.042Zm-4.69 9.64a1.998 1.998 0 0 0 2.83 0l1.25-1.25a.751.751 0 0 1 1.042.018.751.751 0 0 1 .018 1.042l-1.25 1.25a3.5 3.5 0 1 1-4.95-4.95l2.5-2.5a3.5 3.5 0 0 1 4.95 0 .751.751 0 0 1-.018 1.042.751.751 0 0 1-1.042.018 1.998 1.998 0 0 0-2.83 0l-2.5 2.5a1.998 1.998 0 0 0 0 2.83Z\"\/><\/svg><\/a><\/div>\n<p dir=\"auto\">Pi, OpenCode, Hermes, OpenClaw, Codex, Claude Code, Oh My Pi, and Cline. During setup, your agent connects your harness to the model you pick. Or use Magnitude&#8217;s built-in harness.<\/p>\n<div class=\"markdown-heading\" dir=\"auto\">\n<h3 tabindex=\"-1\" class=\"heading-element\" dir=\"auto\">Do I need to manage Magnitude after setup?<\/h3>\n<p><a id=\"user-content-do-i-need-to-manage-magnitude-after-setup\" class=\"anchor\" aria-label=\"Permalink: Do I need to manage Magnitude after setup?\" href=\"#do-i-need-to-manage-magnitude-after-setup\"><svg data-component=\"Octicon\" class=\"octicon octicon-link\" viewbox=\"0 0 16 16\" version=\"1.1\" width=\"16\" height=\"16\" aria-hidden=\"true\"><path d=\"m7.775 3.275 1.25-1.25a3.5 3.5 0 1 1 4.95 4.95l-2.5 2.5a3.5 3.5 0 0 1-4.95 0 .751.751 0 0 1 .018-1.042.751.751 0 0 1 1.042-.018 1.998 1.998 0 0 0 2.83 0l2.5-2.5a2.002 2.002 0 0 0-2.83-2.83l-1.25 1.25a.751.751 0 0 1-1.042-.018.751.751 0 0 1-.018-1.042Zm-4.69 9.64a1.998 1.998 0 0 0 2.83 0l1.25-1.25a.751.751 0 0 1 1.042.018.751.751 0 0 1 .018 1.042l-1.25 1.25a3.5 3.5 0 1 1-4.95-4.95l2.5-2.5a3.5 3.5 0 0 1 4.95 0 .751.751 0 0 1-.018 1.042.751.751 0 0 1-1.042.018 1.998 1.998 0 0 0-2.83 0l-2.5 2.5a1.998 1.998 0 0 0 0 2.83Z\"\/><\/svg><\/a><\/div>\n<p dir=\"auto\">No. It runs in the background, loads models when your agent needs them, and unloads them when idle or memory gets tight. Your agent can install or switch models through the Magnitude CLI anytime.<\/p>\n<div class=\"markdown-heading\" dir=\"auto\">\n<h3 tabindex=\"-1\" class=\"heading-element\" dir=\"auto\">Does my data go to the cloud?<\/h3>\n<p><a id=\"user-content-does-my-data-go-to-the-cloud\" class=\"anchor\" aria-label=\"Permalink: Does my data go to the cloud?\" href=\"#does-my-data-go-to-the-cloud\"><svg data-component=\"Octicon\" class=\"octicon octicon-link\" viewbox=\"0 0 16 16\" version=\"1.1\" width=\"16\" height=\"16\" aria-hidden=\"true\"><path d=\"m7.775 3.275 1.25-1.25a3.5 3.5 0 1 1 4.95 4.95l-2.5 2.5a3.5 3.5 0 0 1-4.95 0 .751.751 0 0 1 .018-1.042.751.751 0 0 1 1.042-.018 1.998 1.998 0 0 0 2.83 0l2.5-2.5a2.002 2.002 0 0 0-2.83-2.83l-1.25 1.25a.751.751 0 0 1-1.042-.018.751.751 0 0 1-.018-1.042Zm-4.69 9.64a1.998 1.998 0 0 0 2.83 0l1.25-1.25a.751.751 0 0 1 1.042.018.751.751 0 0 1 .018 1.042l-1.25 1.25a3.5 3.5 0 1 1-4.95-4.95l2.5-2.5a3.5 3.5 0 0 1 4.95 0 .751.751 0 0 1-.018 1.042.751.751 0 0 1-1.042.018 1.998 1.998 0 0 0-2.83 0l-2.5 2.5a1.998 1.998 0 0 0 0 2.83Z\"\/><\/svg><\/a><\/div>\n<p dir=\"auto\">No. Prompts, files, and models stay on your machine.<\/p>\n<div class=\"markdown-heading\" dir=\"auto\">\n<h3 tabindex=\"-1\" class=\"heading-element\" dir=\"auto\">Can it run completely offline?<\/h3>\n<p><a id=\"user-content-can-it-run-completely-offline\" class=\"anchor\" aria-label=\"Permalink: Can it run completely offline?\" href=\"#can-it-run-completely-offline\"><svg data-component=\"Octicon\" class=\"octicon octicon-link\" viewbox=\"0 0 16 16\" version=\"1.1\" width=\"16\" height=\"16\" aria-hidden=\"true\"><path d=\"m7.775 3.275 1.25-1.25a3.5 3.5 0 1 1 4.95 4.95l-2.5 2.5a3.5 3.5 0 0 1-4.95 0 .751.751 0 0 1 .018-1.042.751.751 0 0 1 1.042-.018 1.998 1.998 0 0 0 2.83 0l2.5-2.5a2.002 2.002 0 0 0-2.83-2.83l-1.25 1.25a.751.751 0 0 1-1.042-.018.751.751 0 0 1-.018-1.042Zm-4.69 9.64a1.998 1.998 0 0 0 2.83 0l1.25-1.25a.751.751 0 0 1 1.042.018.751.751 0 0 1 .018 1.042l-1.25 1.25a3.5 3.5 0 1 1-4.95-4.95l2.5-2.5a3.5 3.5 0 0 1 4.95 0 .751.751 0 0 1-.018 1.042.751.751 0 0 1-1.042.018 1.998 1.998 0 0 0-2.83 0l-2.5 2.5a1.998 1.998 0 0 0 0 2.83Z\"\/><\/svg><\/a><\/div>\n<p dir=\"auto\">Yes. Once Magnitude and a model are downloaded, no internet connection needed.<\/p>\n<div class=\"markdown-heading\" dir=\"auto\">\n<h3 tabindex=\"-1\" class=\"heading-element\" dir=\"auto\">Can I use models outside the catalog?<\/h3>\n<p><a id=\"user-content-can-i-use-models-outside-the-catalog\" class=\"anchor\" aria-label=\"Permalink: Can I use models outside the catalog?\" href=\"#can-i-use-models-outside-the-catalog\"><svg data-component=\"Octicon\" class=\"octicon octicon-link\" viewbox=\"0 0 16 16\" version=\"1.1\" width=\"16\" height=\"16\" aria-hidden=\"true\"><path d=\"m7.775 3.275 1.25-1.25a3.5 3.5 0 1 1 4.95 4.95l-2.5 2.5a3.5 3.5 0 0 1-4.95 0 .751.751 0 0 1 .018-1.042.751.751 0 0 1 1.042-.018 1.998 1.998 0 0 0 2.83 0l2.5-2.5a2.002 2.002 0 0 0-2.83-2.83l-1.25 1.25a.751.751 0 0 1-1.042-.018.751.751 0 0 1-.018-1.042Zm-4.69 9.64a1.998 1.998 0 0 0 2.83 0l1.25-1.25a.751.751 0 0 1 1.042.018.751.751 0 0 1 .018 1.042l-1.25 1.25a3.5 3.5 0 1 1-4.95-4.95l2.5-2.5a3.5 3.5 0 0 1 4.95 0 .751.751 0 0 1-.018 1.042.751.751 0 0 1-1.042.018 1.998 1.998 0 0 0-2.83 0l-2.5 2.5a1.998 1.998 0 0 0 0 2.83Z\"\/><\/svg><\/a><\/div>\n<p dir=\"auto\">Yes. You can <a href=\"https:\/\/docs.magnitude.dev\/models#download-a-model-outside-the-catalog\" rel=\"nofollow\">download compatible GGUF models from Hugging Face<\/a> and use them in Magnitude.<\/p>\n<p dir=\"auto\">Magnitude is licensed under the <a href=\"https:\/\/github.com\/magnitudedev\/magnitude\/blob\/main\/LICENSE\">Apache License 2.0<\/a>.<\/p>\n<\/div>\n<p><a href=\"https:\/\/github.com\/magnitudedev\/magnitude?utm_source=tldrdevops\">Source link <\/a><\/p>\n","protected":false},"excerpt":{"rendered":"<p>Run your agent on local models. Free, private, and offline. Magnitude is an open source inference server that runs the best local models for your hardware, plugged into the agent you already use. It profiles your machine, recommends the models that fit, then downloads, tunes, and runs them. Works with Pi, OpenCode, Hermes, OpenClaw, Codex, [&hellip;]<\/p>\n","protected":false},"author":16,"featured_media":23752,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[143],"tags":[],"class_list":["post-23751","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-ai"],"_links":{"self":[{"href":"https:\/\/scannn.com\/lv\/wp-json\/wp\/v2\/posts\/23751","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/scannn.com\/lv\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/scannn.com\/lv\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/scannn.com\/lv\/wp-json\/wp\/v2\/users\/16"}],"replies":[{"embeddable":true,"href":"https:\/\/scannn.com\/lv\/wp-json\/wp\/v2\/comments?post=23751"}],"version-history":[{"count":0,"href":"https:\/\/scannn.com\/lv\/wp-json\/wp\/v2\/posts\/23751\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/scannn.com\/lv\/wp-json\/wp\/v2\/media\/23752"}],"wp:attachment":[{"href":"https:\/\/scannn.com\/lv\/wp-json\/wp\/v2\/media?parent=23751"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/scannn.com\/lv\/wp-json\/wp\/v2\/categories?post=23751"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/scannn.com\/lv\/wp-json\/wp\/v2\/tags?post=23751"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}