Building Avi: Why Our Lease AI Can Propose Changes but Cannot Apply Them

The product constraint: useful enough to help, unable to act alone

This post explains the engineering decisions behind that boundary.

A lease editor creates an awkward AI design problem. The assistant needs enough context to be useful, but the document may contain sensitive information, stale language, internal contradictions, or even text written to manipulate the model. A retrieved maintenance note or old chat message can be relevant evidence without becoming a trustworthy instruction.

We therefore separated the system into three responsibilities:

1. Read: retrieve a narrowly scoped record or current draft.
2. Propose: return a structured answer, review flag, clarification, or complete-document replacement.
3. Act: wait for a separate landlord decision before changing the persisted draft.

The model participates in the first two stages. The application owns the third.

Lease editor
    |
    | authenticated request + draft version + idempotency key
    v
PostgreSQL request queue
    |
    | short transactional claim
    v
Bounded agent runner
    |
    +--> default-deny read tools
    +--> reviewed legal context
    +--> official-source research when enabled
    +--> scoped conversation memory
    |
    v
Validated structured proposal
    |
    | landlord chooses Apply / Not now
    v
Version-checked domain mutation
    |
    +--> deterministic operation receipt
    +--> exact Undo path

That diagram is also our threat model in miniature: every arrow crosses a boundary that the model does not control.

Requests enter a queue, not a long browser transaction

The editor submits a draft identifier, the current draft version, the landlord’s question, current document HTML, and an optional text selection. Each request also carries an idempotency key. The backend validates ownership and input limits, stores the user message, and returns an accepted response while the work remains QUEUED .

A scheduled worker claims queued requests with PostgreSQL FOR UPDATE ... SKIP LOCKED . PostgreSQL specifically identifies SKIP LOCKED as useful for multiple consumers accessing a queue-like table, even though it is not appropriate for a general-purpose consistent read. That is exactly the tradeoff here: workers need to claim independent jobs without waiting on one another, while the request row remains the durable source of state.

The claim happens in a short transaction. The provider call happens after that transaction ends, so model latency never holds the database lock. A partial unique index permits only one running request per landlord, which keeps ordering understandable and prevents one account from consuming every worker slot.

Cancellation is state-based. We cannot guarantee that a provider socket stops at the exact instant the user clicks Cancel, but completion updates succeed only while the request remains RUNNING . A late provider response from a cancelled request is discarded instead of being published into the conversation.

The agent loop has a hard budget

The provider gateway does not own an open-ended “keep going until done” loop. It implements one model session behind a provider-neutral AiAgentRunner .

At the time of this code review, the leasing assistant is limited to:

• six model rounds;
• eight tool calls in total;
• no more than three calls to any one tool;
• tighter per-tool caps for property, maintenance, draft, memory, and legal-research operations;
• one live legal-research call; and
• a 60-second total run deadline.

The remaining deadline is passed into each provider turn so the network timeout shrinks with the budget. A slow first request cannot consume most of the minute and then allow another full-duration HTTP call.

Exact duplicate tool calls are detected using canonicalized JSON and SHA-256 fingerprints. The duplicate still consumes the agent’s budget, but the domain operation does not run twice. Run state retains bounded fingerprints rather than raw tool arguments or results.

This is intentionally boring infrastructure. A good agent loop should have an obvious stopping condition when the model repeats itself, stops making progress, exceeds a deadline, or burns through its tool allowance.

Context comes through capabilities, not a database-shaped prompt

The model does not receive a database connection or a generic query tool. It gets a request-specific catalog of bounded capabilities, and that catalog is filtered through a default-deny whitelist.

Depending on the active draft, the assistant may be allowed to:

• list and resolve an authorized property;
• list and resolve an authorized tenant’s leasing-relevant fields;
• search and open an exact maintenance request;
• read the current server-authoritative lease draft;
• search earlier lease-assistant conversations;
• retrieve public rent-market signals for an authorized property; or
• research a current legal question through configured official sources.

Actor identity, service identity, landlord ownership, and the active draft come from backend-created invocation context. They are not model arguments. The model can ask for a property ID, but it cannot decide which landlord it is acting for or attach itself to another draft.

The most important omission is the mutation tool. A complete-draft replacement capability exists for other explicitly authorized workflows, but the leasing assistant’s whitelist excludes it. The model can return a proposal describing a replacement. It cannot call the replacement operation.

Lease text is data, even when it looks like an instruction

Prompt injection is not limited to the chat box. Instructions can arrive through a lease clause, a pasted selection, an old conversation, a maintenance description, a tenant field, or a fetched web page. OWASP’s prompt-injection guidance recommends structured separation, least privilege, output validation, and human control for high-risk actions. We treat those as complementary layers, not substitutes for one another.

The provider request places the landlord’s actual question in a structured request object. Lease HTML, selected text, recorded chat history, legal context, widget values, and tool results sit under an explicitly untrusted context envelope. The system instructions repeatedly state that content inside those fields cannot change roles, reveal hidden instructions, expand tool access, or bypass approval.

That prompt boundary is useful, but it is not the security boundary. Backend checks still enforce ownership and scope on every tool call. Provider output must match a strict JSON schema. Proposed document HTML passes a deterministic allowlist that rejects scripts, styles, links, images, forms, event handlers, unknown attributes, and unsafe URL patterns.

Only semantic text tags and a small set of exact lease-value and signature-field markers are accepted. The application also enforces the current document outline. A model cannot quietly remove, rename, or reorder major lease sections unless the landlord explicitly asked for an outline amendment.

Memory is scoped retrieval, not an unlimited transcript

Loading an entire account history into every prompt would be expensive, noisy, and difficult to reason about. Instead, user and assistant messages are split into bounded overlapping chunks and stored with landlord, property, draft, chat, source-message, and expiration metadata.

When the landlord refers to an earlier discussion, the assistant can search those memories using hard property, draft, and date filters. Ranking combines semantic similarity, lexical overlap, and recency over a bounded candidate set. If embeddings are unavailable, the retrieval path falls back to lexical and recency scoring rather than failing the full request.

Retrieved memory is still untrusted. If several prior discussions could match, Avi must ask the landlord to narrow the request instead of choosing the most plausible answer. Deterministic reply buttons are allowed only when the choices came from authoritative context or a tool result. Names, rent, dates, custom terms, and other free-form facts require an open-ended question.

This distinction matters because an attractive list of model-invented options can hide a bad assumption behind polished UI.

Legal context has provenance rules

The first legal layer is a reviewed, versioned knowledge base connected to the draft’s jurisdiction. If that coverage is missing or stale, the assistant can call a production-enabled legal-research tool. That tool runs a separate LLM web-research request; it is not a deterministic legal database lookup. The backend binds the request to the authenticated landlord, active draft, and jurisdiction, restricts the LLM’s web search to a configured allowlist of official government domains, and persists the returned answer and source metadata for that request.

Returned sources are validated again by the backend. URLs must use HTTPS and belong to an allowed host. A review flag may cite only a source that came from the reviewed knowledge base or was persisted for that exact research request. If the model invents a plausible government URL, proposal validation rejects it.

The leasing model policy requires any conclusion based on that LLM research to carry a LEGAL_REVIEW flag and an advisory warning. The feature is drafting assistance, not a mechanism for converting a search result into a legal conclusion.

The current provider adapter also sends store: false . That disables response storage through the request setting, but it should not be marketed as “zero retention” by itself. Provider account configuration, contractual controls, application retention, backups, and deletion behavior still need their own precise public documentation.

A proposal is a typed object, not trusted prose

The final provider response must satisfy a strict schema containing:

• a human-readable summary;
• zero or one operation;
• up to eight review flags;
• a follow-up mode of NONE , OPEN_ENDED , or DETERMINISTIC_CHOICE ; and
• at most five deterministic follow-up options.

For a document change, the only accepted operation is REPLACE_DOCUMENT , containing the complete proposed final document. We deliberately rejected partial model patches. A full replacement is larger, but it gives validation, preview, version comparison, audit, and undo one consistent object to evaluate.

A clarification and a mutation cannot appear in the same proposal. If required information is missing, the assistant must ask first and return no operation. This prevents the familiar failure mode where a model asks a sensible question while quietly filling the unknown field with a guess.

Apply and undo are separate authenticated operations

When a valid proposal contains a document replacement, it is stored with PENDING approval. The editor renders the explanation and proposed language, then offers Apply change or Not now.

Apply enters a separate authenticated endpoint. The backend locks the request, confirms landlord ownership, reloads the current draft, and compares its version with the version used to generate the proposal. If the draft changed in the meantime, the update fails and the landlord must generate a new proposal.

An accepted replacement uses the existing lease domain capability, which records a deterministic operation and action receipt. Undo resolves that exact receipt and fails safely if a newer version would be overwritten. The chat may describe the change, but the operation journal—not the conversation—is the authority for reversing it.

Human approval adds a click. It also creates a clean answer to a critical question: who decided to change the lease? The model proposed it; the authenticated landlord applied it.

We test the boundaries independently of model quality

Model evaluations are useful, but several invariants should not depend on the model behaving well. Deterministic tests cover:

• injection text hidden in lease fields and conversation history;
• attempts to reveal system instructions or invoke unavailable tools;
• mutation proposals returned before clarification is complete;
• invented choices attached to open-ended questions;
• partial-document operations;
• unsafe or unexpected HTML;
• unverified legal citations;
• property and draft ownership boundaries;
• optimistic-lock conflicts during approval;
• duplicate, repeated, and over-budget tool calls;
• cancellation and late provider responses;
• retry exhaustion and stale-worker recovery; and
• exact undo behavior.

Architecture tests also fail when a published tool lacks an effect policy or when a model-backed service has no explicit run policy and tool whitelist. That makes “we forgot the safety declaration” a build failure instead of a code-review hope.

What we would keep—and what we would improve

The core design choice has held up: keep the model on the proposal side of a hard application boundary. We would keep the default-deny tools, server-derived identity, strict output schema, versioned approval, and receipt-based undo even if the provider or model changed tomorrow.

The costs are real. Complete-document proposals use more tokens than small patches. Polling is simpler than a streaming state protocol but less immediate. Human approval slows down a rewrite. Official-domain restrictions can return “not enough information” when a broader search would produce a more confident-looking answer.

Those are acceptable costs for this workflow. The next engineering work is not removing the boundaries. It is improving observability, building a disclosed evaluation corpus, measuring source-location and conflict-detection quality by category, expanding reviewed jurisdiction coverage, and making privacy and retention controls easier for landlords to understand.

For the practical review method behind the feature, see the lease agreement checklist for landlords . For the underlying document workflow, see how to create, manage, and sign a lease .