On-device model

Compass VLD

A single checkpoint that reads screenshots and points at elements, writes text and tool calls, and answers decision questions such as “is the task done?”. Running all three in one on-device model replaces a captioner, a planner and a judge.

1.55Bparameters, Compass VLD Tiny v5
120/126d1 page-state test, zero false completions
0.94 sper decision in the browser (RTX 5090)
32/89compact routing, loose: top among newly tested

Why it matters

Every extra model is another download, another wait and another place for an agent to go wrong. One small model that can see, talk and decide is what makes a capable agent practical on an ordinary computer.

Two models

Compass VLD Tiny v5 is a fine-tune of OpenBMB MiniCPM-V-4.7-1B: 1.55B parameters in total (498M language layers, 508M embeddings and head, 522M vision encoder, 26M projector). Round 1 used 344,139 samples with LoRA rank 64 on every language layer; round 2 added d1 page-state decision data in 51,811 samples. It trained on one RTX 5090 in about 20 hours.

LFM3-VLD-2B applies the same idea to Liquid’s LFM3-VL-2B (1.92B parameters), where Liquid itself splits the roles between LFM3-VL-2B for chat and d1-2B for decisions.

How they compare

On general benchmarks (BenchKit, 4,857 tasks per model) the larger LFM3-VL-2B base wins 6 of 9 suites against Compass v5, and Compass wins none, while decoding faster: 157 tokens/s against 91. On WebBrain’s own compact routing test, both VLDs tie for the best loose score among the newly tested models at 32/89.

BenchKit: LFM3-VL-2B vs Compass VLD v5

SuiteLFM3-VL-2BCompass VLD v5
V*79.665.4
OCRBench81.572.6
ChartQA82.065.6
MMLU-Pro46.723.0
GSM8K68.270.6
Decode speed91 tok/s157 tok/s

General benchmarks, not WebBrain’s tasks, on an RTX A4000. GSM8K is a tie within noise. LFM3 ran thinking-off on a pre-release llama.cpp and had 52 MMLU-Pro timeouts; Compass hit repetition loops (60 on IFEval, 24 on GSM8K).

Models

Caveats

  • LFM3 has not yet been run on WebBrain’s decision and grounding tests.
  • An LFM3-based VLD becomes WebBrain’s default on-device model only if it holds up.

Read more on the blog