Why AI Agents Fail Humans — and How User Alignment Specs Fix It

User modeling isn't a new idea. It's existed in HCI, dialogue systems, and adaptive agents for decades. What's been missing is the operational discipline to make it a first-class artifact in real agent deployments — and the live memory system to keep it current as the project evolves.

This article documents the 4-layer User Alignment Spec (UAS) system we built and tested across three production agents managing two platforms and 100+ autonomous sub-agents. The code is open source. The benchmarks don't exist yet — that's an honest limitation we'll address below.

The Four Layers

Layer 3 — The Human Model in Detail

Most frameworks handle Layer 1 well. MemGPT has user context sections. CrewAI supports agent personas. LangChain supports custom memory with user profiles. The difference here is treating the human profile as a hand-written, actively maintained first-class artifact — not something inferred from conversation history.

The most important addition from peer review: Operating Modes. Humans don't communicate with one fixed style. They shift by task urgency, cognitive load, and context. A non-technical founder at 2am with a broken production deploy communicates completely differently from the same person doing strategic planning on a Tuesday morning. Static profiles miss this entirely.

## Human Profile — Shawn
- Technical level: non-technical — never ask him to explain tech
- "ok [action]" = execute immediately, no discussion
- Short message + "?" = yes or no answer only
- Silence after a completed task = it worked, move on

## Operating Modes        ← NEW — added after peer review
- Normal: standard autonomy, report after completion
- Quick iteration ("just try it"): ship fast, minimal verification
- Review mode ("check this before"): verify with human first
- Deep work (long task): check in at natural breakpoints only

## What he dislikes
- Being asked questions he can't answer technically
- Over-explanation, filler phrases, suggested calls/video chats

## What he appreciates
- Handles it without hand-holding
- Honest pushback when something's wrong
- Remembers past decisions

Layer 4 — Living Memory (and Its Honest Limitations)

After every tool-using task, a fast LLM extracts facts learned and appends them to memory.md automatically.

def _auto_save_memory(message, response, tools_used):
    if not tools_used:
        return  # Pure conversation — nothing actionable

    extracted = fast_llm_call(
        system="""Extract ONLY new facts worth remembering long-term.
        Max 5 bullets, max 15 words each.
        Focus on: decisions, bugs fixed, configs changed, preferences.
        Output SKIP if nothing meaningful.""",
        user=f"TASK: {message[:300]}\nRESULT: {response[:500]}"
    )
    if extracted != "SKIP":
        append_to_memory_file(f"### {timestamp}\n{extracted}")

The Five Failure Modes

How This Relates to Existing Research

User modeling isn't new. What's new is the operational discipline and the production context. Here's honest prior art positioning:

What's Still Open