Funny, i did this myself some months ago - its actually not that tricky to run a llm in your browser. Tho, the problem is that the models fitting in there are well relativly "tiny" in my opinion.
HashAgent started with a simple question: can you run a useful AI agent with zero hosting costs? The answer I landed on: the user's own device is the server.
How it works:
- The entire agent definition (name, system prompt, greeting, generation config) is encoded as compressed base64url JSON in the URL hash fragment. The hash never hits a server — sharing a link is sharing the agent.
- Inference runs in-browser via WebGPU. Two runtimes: WebLLM (MLC) for Llama 3.2 / Phi / Mistral / Qwen, and Transformers.js for newer ONNX models like Gemma 4 E2B and LFM2 1.2B. First load downloads the weights, then they're cached and work offline.
- Tools are split into two tiers. Local tier (calculator, time, Wikipedia, weather via direct CORS calls) never touches my infrastructure. Gateway tier (web search, page reading) goes through open-source Cloudflare Pages Functions because search engines don't allow browser CORS — this can be switched off in settings, at which point nothing touches any server I run.
- Each turn runs a temperature-zero planning pass that can only emit a JSON tool call matching an allowlisted schema, separate from the answer pass — so 1B-class models don't accidentally "hallucinate" tool invocations mid-prose.
Honest limitations:
- Model ceiling is ~8B quantized on 16GB+ desktops. Phones are realistically 1B-class (LFM2 1.2B / Llama 3.2 1B). These are single-task agents, not GPT-4 replacements.
- iOS is the hard wall: WebKit caps a single tab at roughly 1.5GB regardless of how much RAM the iPhone has, so iPhones are locked to the smallest models. Mid-range Android often outruns the newest iPhone here.
- First load is a 700MB–2GB download depending on the model. After that it's cached.
- Optional short links (/s/<id>) store the agent payload in KV for social previews — the full hash URL remains self-contained and works even if this deployment disappears.
Cool idea tho to make it as accessible (:
HashAgent started with a simple question: can you run a useful AI agent with zero hosting costs? The answer I landed on: the user's own device is the server.
How it works:
- The entire agent definition (name, system prompt, greeting, generation config) is encoded as compressed base64url JSON in the URL hash fragment. The hash never hits a server — sharing a link is sharing the agent.
- Inference runs in-browser via WebGPU. Two runtimes: WebLLM (MLC) for Llama 3.2 / Phi / Mistral / Qwen, and Transformers.js for newer ONNX models like Gemma 4 E2B and LFM2 1.2B. First load downloads the weights, then they're cached and work offline.
- Tools are split into two tiers. Local tier (calculator, time, Wikipedia, weather via direct CORS calls) never touches my infrastructure. Gateway tier (web search, page reading) goes through open-source Cloudflare Pages Functions because search engines don't allow browser CORS — this can be switched off in settings, at which point nothing touches any server I run.
- Each turn runs a temperature-zero planning pass that can only emit a JSON tool call matching an allowlisted schema, separate from the answer pass — so 1B-class models don't accidentally "hallucinate" tool invocations mid-prose.
Honest limitations:
- Model ceiling is ~8B quantized on 16GB+ desktops. Phones are realistically 1B-class (LFM2 1.2B / Llama 3.2 1B). These are single-task agents, not GPT-4 replacements.
- iOS is the hard wall: WebKit caps a single tab at roughly 1.5GB regardless of how much RAM the iPhone has, so iPhones are locked to the smallest models. Mid-range Android often outruns the newest iPhone here.
- First load is a 700MB–2GB download depending on the model. After that it's cached.
- Optional short links (/s/<id>) store the agent payload in KV for social previews — the full hash URL remains self-contained and works even if this deployment disappears.
MIT licensed: https://github.com/mason131928/hashagent
Happy to answer anything about squeezing LLMs into browser memory limits — the iOS jetsam debugging alone was a journey.