Skip to content
AA-022 In Development

Beelze: A Tiered Safety Policy for a Desk AI Agent

Beelze is a desk AI assistant whose actions on a Windows computer pass through a policy enforced in code. Tools are tiered by risk, an outward action runs only after a confirmation bound to its exact payload, and a pattern-based injection guard is treated as a filter, not as the defense.

What is Beelze?

Beelze is a desk AI companion I am building in two parts: a voice-driven body, and a server called hands that lets the assistant act on a Windows computer through the Model Context Protocol. This case study is about the hands server, because it is the part that carries risk. It reads mail, manages a calendar, and starts coding workers, and every one of those actions passes through a policy that is enforced in code.

The software is written and tested offline. The code and its tests were produced in a one-day build by six parallel AI builders and an integrating session, working from a build specification. A fresh clone of the repository recorded 862 tests passed. The physical body is designed and has not been built, and the live tests that need a real account have not been run. Both limits are detailed at the end.

Why does an assistant with real access need rules enforced in code?

Because an instruction in a prompt is a request, and the model that receives it also reads untrusted text. An assistant that listens to a room and reads an inbox will eventually read a message written to steer it. If the only thing between that message and a sent email is the model’s judgment, the control is the attacker’s to argue with.

The project’s safety document states the contract in one sentence: every rule is enforced in code by the hands server and the body, not just requested in a prompt. The model proposes an action, and deterministic code decides whether it runs.

How do you tier an agent’s tools by risk?

Each tool is assigned a tier in one policy file, and the tier decides what happens when the model calls it. The policy currently tiers 44 tools.

Observe

The 19 tools in the lowest tier read state or stop work that is already running. They run immediately and the call is logged.

Reversible and local

The 21 tools in the next tier change something on my own machine or in my own data that I am able to undo. They run, and the assistant says aloud what it did.

Outward

The 4 tools in the confirmation tier reach other people or other systems: sending mail, sending a calendar invitation, acting on a connected account, and operating the computer directly. A call to one of the first three is staged and does not run until I confirm it. The fourth, direct computer control, is disabled and refuses every call.

Refused

A tool with no assigned tier is refused, so a new tool is blocked until someone classifies it. Six categories are refused regardless of tool: payments and purchases, moving or trading money, entering passwords or payment details, changing security settings, disabling the assistant’s own safety rules, and messages to more than 10 recipients.

The policy file is itself a protected path. A coding worker started by the assistant has no permission to write it.

How do you stop an agent from sending an email the user did not approve?

By making the confirmation a property of the exact payload, decided by code the model has no handle on. The sequence has three steps.

First, the action is staged. The server stores the payload with its SHA-256 hash and an expiry of 60 seconds. It builds the spoken readback from the stored payload in code, so the model’s own wording has no opportunity to leave out the recipient.

Second, the confirmation arrives through a channel that is not in the model’s tool list. The server checks that the action is pending, unexpired, and unused, and that what I said matches a configured phrase. For those three tools a spoken confirmation also requires approval of a desktop notification, because anyone in the room is able to speak. A confirmation sent from my own allowlisted phone account counts as the second factor instead.

Third, the server mints a single-use token, an HMAC over the action id, the payload hash, and the expiry. The token never leaves the process. The server then re-hashes and executes the stored payload, not anything sent again after confirmation. The check and the claim happen under one lock, so two confirmations of the same action cannot both win.

What does a prompt-injection guard check, and where does it stop?

The guard is a pass of 17 patterns over text that arrives from outside: mail bodies, answers from connected accounts, and the output of the swarm of small models. It looks for instructions to ignore earlier instructions, attempts to assign a new role, text that addresses the assistant by name, internal tool names, tool-call markup, requests to say the confirmation phrase, requests to forward all mail, and destructive shell commands. A sentence that matches is dropped before the text is spoken, and the item is flagged as a suspected injection.

The guard is pattern matching and nothing more. An instruction that is paraphrased, written in another language, split across sentences, or encoded passes it. I do not treat it as the defense. The defense is the confirmation tier: an injected instruction that gets past the guard still cannot send, invite, or act outward without a staged payload, a readback built by code, and my approval.

What does the audit log record?

Every tool call appends one line to a log that is never rewritten. The line holds the time, the tool, its tier, a hash of the arguments beside a redacted copy of them, the action id, the channel that confirmed it, the status, and the cost. Secrets are masked twice, by the name of the field and by the shape of the value, so a token pasted into an unexpected field is still caught.

A spend meter sits beside the log. It sets a daily cap for cloud model calls, and when the cap is reached the cloud calls stop while local functions continue.

What is not built or not proven?

No hardware exists. The enclosure is modelled and its parts pass a fit check in software, and nothing has been printed or assembled. The wake word is a stock model.

The tests ran with stand-ins for the operating system, the agent SDK, and the desktop notification. The test that sends a real injected email and the test that sends a real confirmed one are written down as steps that need me, and they have not been run. The tier, confirmation, and audit logic is tested as logic. It is not yet tested against a live account.