POCKET-35B: A 35B Model That Runs on Your Phone — Without a GPU

The Dawn of Democratized AI

For years, generative AI has been synonymous with the cloud, monthly subscriptions, and data-center GPUs. In 2026, something is shifting: increasingly capable open-weight models can run on laptops, mini-PCs, and even smartphones — without a dedicated graphics card. This is not marketing magic: it is MoE engineering, aggressive quantization, and a mature ecosystem (llama.cpp, Ollama, MLX) that finally makes local inference a credible option for makers, researchers, and small businesses.

POCKET-35B is one of the strongest signals of this transition. A model with 34.66 billion total parameters that, thanks to Mixture-of-Experts sparsity, activates only about 3 billion parameters per token. The result: capacity close to a large model, computational footprint of a small one. For anyone building robots, home automation, or offline assistants, this changes the rules of the game.

The maker community does not need another cloud chatbot. It needs models that work in the garage, on a Raspberry Pi connected to a robot arm, or on a MacBook traveling without Wi-Fi. POCKET answers that need — with honest limits we will explore at the end of this article.

What Is POCKET

POCKET (Portable Open Compute Knowledge Transformer) is a family of open-weight language models developed by the VIDRAFT team as part of the FINAL-Bench research project. The main release, POCKET-35B, is licensed under Apache 2.0 and available on Hugging Face in the repository FINAL-Bench/POCKET-35B-GGUF.

Pay attention to attribution: many online articles incorrectly credit POCKET to Hugging Face as the author. In reality, HF hosted the weights and featured them on the platform blog — a distribution channel, not the training source. Credit belongs to VIDRAFT and FINAL-Bench.

The base model derives from Darwin-36B-Opus, itself linked to the Qwen3.6-35B-A3B architecture. VIDRAFT applied distillation, evolutionary merging, and quantization to optimize execution on consumer hardware. The GGUF version is what you will use in practice: ready for llama.cpp, Ollama, LM Studio, and MLX on Apple Silicon.

What distinguishes POCKET from classic dense models

• MoE (Mixture of Experts): not all parameters participate in every forward pass.
• Community quantization: Q2_K, Q4_K_M, and phone-oriented variants already available.
• Permissive license: Apache 2.0 allows integration into commercial products with attribution.
• FINAL-Bench benchmarks: positioned as a reference for measuring latency and quality on edge.

Technology: How MoE Works

A dense 35B transformer requires loading and multiplying all weights on every token. On consumer CPU or GPU this is impractical: you need tens of gigabytes of VRAM and acceptable throughput only with server-class hardware.

POCKET-35B uses a router that, for each token, selects a subset of "experts" (feed-forward sub-networks). Only about 3 billion parameters are active during inference — the rest stays in memory but is not computed. Memory and FLOPs scale with active parameters, not the total.

GGUF quantization further compresses weights into low-precision formats (mixed INT4, INT2). A Q2_K build weighs about 13 GB on disk; Q4_K_M about 21 GB. Experimental smartphone builds drop toward 5–8 GB, sacrificing quality for footprint.

MoE does not break the laws of physics: it breaks the assumption that "35B" always means "35B active." POCKET trades bragging rights on total parameters for real deployability — and that is exactly the trade-off the local-AI community needed. — Summary from the FINAL-Bench / Hugging Face blog

Supported execution stack

• llama.cpp: maximum control over threads, context, and partial GPU offload.
• Ollama: ollama pull from community manifests pointing to FINAL-Bench GGUF files.
• LM Studio: graphical interface for Windows, macOS, and Linux.
• MLX (Apple Silicon): optimized path on M-series chips.

Specifications and Hardware Requirements

Implications for Makers

For anyone building robots, home automation, or lab tools, POCKET opens concrete scenarios:

Offline coding assistant

On a 32 GB RAM laptop, POCKET can suggest fixes to Python scripts for ROS, parse serial logs, and generate YAML configurations — all without sending proprietary code to external APIs. Useful in air-gapped environments or under strict NDAs.

Glue layer for edge automation

A mini-PC next to the robot can use the model to interpret natural-language commands, translate them into API calls to microcontrollers (MQTT, REST), and summarize telemetry. It does not replace a motion planner — but it accelerates integration.

Research and reproducibility

FINAL-Bench provides standardized metrics. University labs can replicate latency and quality benchmarks without cloud budget, publishing comparable results.

Privacy by design

Medical notes, client data, or NDA projects stay on the local machine. For many maker use cases this is a non-negotiable requirement, not a nice-to-have.

• Travel and remote sites: inference without connectivity (ships, construction sites, rural areas).
• Voice/text prototypes: human-machine interface on modest hardware.
• Education: students can experiment with a "large" model without an OpenAI credit card.

Getting Started: Real Paths (Not pip install pocket-llm)

Warning: a PyPI package called pocket-llm circulates that is not an official VIDRAFT/FINAL-Bench distribution. Installing frontier model weights from unverified PyPI packages is a supply-chain risk. Use only the canonical channels below.

Option 1: Hugging Face + llama.cpp

# Download GGUF weights from the official repository
huggingface-cli download FINAL-Bench/POCKET-35B-GGUF \\
    --include "POCKET-35B-Q2_K.gguf" \\
    --local-dir ./models/pocket

# Run with llama.cpp
./llama-cli -m ./models/pocket/POCKET-35B-Q2_K.gguf \\
    -p "Explain the MQTT protocol in two paragraphs." -n 256

Option 2: Ollama

# After creating a Modelfile pointing to the downloaded GGUF:
# FROM ./models/pocket/POCKET-35B-Q2_K.gguf
ollama create pocket-35b -f Modelfile
ollama run pocket-35b

Alternatively, search for community tags on ollama.com that explicitly reference FINAL-Bench/POCKET-35B-GGUF — always verify provenance.

Option 3: MLX on Apple Silicon Mac

# Check the FINAL-Bench repository for conversion scripts
# or pre-converted weights for mlx_lm
pip install mlx-lm
python -m mlx_lm.generate --model FINAL-Bench/POCKET-35B-MLX \\
    --prompt "Hello, how are you?"

Comparison with Other Local Models

POCKET wins when you need capacity from a large model with a footprint of a small one and an absolute offline requirement. For pure CPU latency, a dense 7B–8B may beat POCKET on tokens per second — evaluate your use case.

Honest Limitations

• It is not GPT-4: multi-step reasoning, advanced math, and minor languages suffer with aggressive quantization.
• "Runs on your phone" means recent flagships with 8+ GB RAM and patience — not every handset in your drawer.
• MoE routing adds overhead: on some benchmarks a dense Llama 8B is faster.
• Fine-tuning and batch embedding on CPU remain impractical: POCKET is for inference, not training.
• Attribute correctly: VIDRAFT/FINAL-Bench, not Hugging Face as author.

The Future of Edge AI

Next 12 months

Expect more POCKET variants (multilingual, coding-specialized), even more aggressive quantizations for mobile NPUs, and native integration in Ollama/LM Studio without manual Modelfile.

3–5 years

MoE + dedicated smartphone NPUs could make "35B-class" inference routine. The gap between edge and cloud will shrink for moderately complex tasks; the cloud will remain necessary for training and closed frontier models.

Maker impact

Home robots, offline voice assistants, and field diagnostic tools will benefit from models like POCKET as a local "language brain" — paired with specialized control policies, not as a substitute for the entire stack.

Edge AI will not replace the cloud tomorrow. But for millions of maker projects that cannot — or will not — send data over the network, models like POCKET are already sufficient to build something real.

Conclusion

POCKET-35B is a credible milestone in the local-AI movement: frontier-scale capacity with MoE sparsity, an open license, and a practical GGUF ecosystem. Credit belongs to VIDRAFT and FINAL-Bench — Hugging Face amplified the release but did not author it.

If you have 16 GB of RAM, start with Q2_K via Ollama or llama.cpp. With 32 GB, move to Q4_K_M. Avoid mysterious pip packages; download from FINAL-Bench/POCKET-35B-GGUF. Treat phone builds as exciting experiments — not promises of data-center performance in your pocket.

Sources

• Hugging Face Blog — FINAL-Bench/pocket — POCKET: a 35B model that runs on iPhone and PC without GPU
• FINAL-Bench/POCKET-35B-GGUF — Official GGUF weights repository
• llama.cpp — GGUF inference runtime
• Ollama — Local model serving