Local small models can help cloud-written agent programs keep user data on-device

Key points

• Proposed architecture: cloud model writes a program, which defers bounded questions over user data to a small local model via a function call.
• On AppWorld, success rate of a code-only agent improved in all six backbone and partition combinations tested, by up to 14.9 percentage points.
• Gains recover a substantial part of the gap to a cloud-hosted ReAct loop, according to the authors, while user data stays on the device.
• Paper was submitted October 9, 2026 and accepted at the SLM-Agents workshop at NeurIPS 2026.

Why it matters for marketers: Content and product teams building privacy-sensitive agents may be able to use small on-device models for run-time interpretation without sending user data to the cloud.