Blog

The Problem

Language models are everywhere now. People praise them, but also complain about responses—unreliable, hallucination, cannot let it work alone, and so on. These systems, capable of understanding and generating human-like text, are often called copilots—a term borrowed from aerospace or car racing. That term indicates their main expected role being support for the pilot.

But how do we actually classify what these models can do? And more importantly, how much can we trust them?

A Hybrid Classification Framework

Drawing inspiration from the SAE levels of driving automation and grounded in human-computer interaction research on trust in automation, we propose a two-dimensional framework for classifying language models:

1. Operational Autonomy – adapted from SAE Levels (0–5): What can the model do on its own?
2. Cognitive Trust and Delegation – how much mental effort does the user expend, and how much responsibility is delegated?

Each level in the chart below reflects both dimensions.

Level	Autonomy Description	Trust/Delegation Role
0 – Basic Support	Passive tools like spellcheckers; no real autonomy	No Trust: User must fully control and interpret everything
1 – Assisted Generation	Suggests words or phrases (autocomplete); constant oversight needed	Suggestive Aid: User supervises and approves each suggestion
2 – Semi-Autonomous Text Production	Generates coherent content from prompts (emails, outlines); needs close supervision	Co-Creator: User relies in low-stakes tasks but reviews all outputs
3 – Context-Aware Assistance	Can handle structured tasks (e.g., medical summaries); users remain alert	Delegate: User lets go during routine tasks but monitors for failure
4 – Fully Autonomous Within Domains	Works independently in narrow contexts (e.g., customer service bot)	Advisor: Trusted within scope; user rarely intervenes
5 – General Language Agent	Hypothetical general-purpose assistant capable across domains without oversight	Agent: Fully trusted to operate independently and responsibly

Why SAE Levels Make Sense

While not acting in the physical world, it makes perfect sense to compare language models to autonomous vehicles in terms of their capabilities and limitations. The SAE classification helps clarify expectations, safety considerations, and technological milestones.

Let’s first briefly revisit what each SAE level entails for automobiles:

• Level 0 (No Automation): The human driver does everything; no automation features assist with driving beyond basic warnings.
• Level 1 (Driver Assistance): The vehicle offers assistance with either steering or acceleration/deceleration but requires constant oversight.
• Level 2 (Partial Automation): The system can manage both steering and acceleration but still requires the human to monitor closely.
• Level 3 (Conditional Automation): The vehicle handles all aspects of driving under specific conditions; the human must be ready to intervene if necessary.
• Level 4 (High Automation): The car can operate independently within designated areas or conditions without human input.
• Level 5 (Full Automation): Complete autonomy in all environments—no human intervention needed.

Adapting Levels to Language Models

Level 0: Basic Support

At this foundational level, language models serve as simple tools—spell checkers or basic chatbots—that provide minimal assistance without any real understanding or autonomy. They do not generate original content on their own but act as aids for humans who make all decisions.

Example: Elementary grammar correction programs that flag mistakes but don’t suggest nuanced rewrites.

Level 1: Assisted Generation

Moving up one step, some language models begin offering suggestions based on partial input. For example, autocomplete functions in email clients that predict next words or phrases fall into this category—they assist but require constant supervision from users who must review outputs before accepting them.

Example: Gmail’s smart compose feature.

Level 2: Semi-Autonomous Text Production

At this stage, models can generate longer stretches of coherent text when given prompts—think about AI tools that draft emails or outline articles—but they still demand continuous oversight. Users need to supervise outputs actively because errors such as factual inaccuracies or inappropriate tone remain common pitfalls.

Example: ChatGPT generating email drafts or article outlines.

Level 3: Context-Aware Assistance

Now we reach an intriguing analogy with conditional automation—where AI systems can handle complex tasks within certain constraints yet require humans to step back temporarily while remaining alert for potential issues. Large language models operating at this level might manage summarization tasks under specific domains (e.g., medical summaries) but could falter outside their trained scope.

Example: Medical AI assistants that can summarize patient records but require doctor oversight.

Level 4: Fully Autonomous Within Domains

Imagine an AI-powered assistant capable of managing conversations entirely within predefined contexts—say customer service bots handling standard inquiries autonomously within specified industries—but unable beyond those limits without retraining or manual intervention.

Example: Customer service chatbots for specific industries like banking or retail.

Level 5: Fully Autonomous General Language Understanding

Envisioning true “full autonomy” for language models means creating systems that understand context deeply across countless topics and produce accurate responses seamlessly everywhere—all without prompting from humans if desired. While such systems remain theoretical today, research aims toward developing general-purpose AI assistants capable not only of conversing fluently across domains but doing so responsibly without oversight.

Example: Theoretical future AI systems that could operate across all domains without human oversight.

Current State and Implications

Now that we have a clear classification framework, let’s examine where we stand today and what this means for practical applications.

What does this classification tell us about our current standing? Most contemporary large-scale language models sit somewhere around Levels 2 or early-Level 3—they generate impressive content when given prompts yet still struggle with consistency outside narrow contexts and require vigilant supervision by humans who evaluate accuracy critically.

However, there’s an important limitation to the SAE analogy that we need to address.

The Trust Dimension

While the SAE levels offer a useful metaphor for understanding increasing autonomy, they aren’t a perfect fit for language models because:

• Language models don’t act in the physical world themselves—humans interpret and act on their outputs
• Risk and impact in NLP are mediated by human cognition and behavior, unlike the immediate physical risks of self-driving cars
• Autonomy in NLP often deals more with semantic understanding, trustworthiness, context handling, and ethical alignment than sensor-actuator loops

Therefore, I also propose a mapping of the SAE levels to trust levels taking into account cognitive load and responsibility:

• Level 0: No trust: tool offers isolated corrections, requires full user oversight (spellcheck)
• Level 1: Suggestive aid: user must review and approve every suggestion (autocomplete)
• Level 2: Co-creator: user maintains active oversight, only defers in low-stakes contexts (drafting emails)
• Level 3: Delegate: user maintains regular oversight with frequent spot checks and validation (10-20% review)
• Level 4: Advisor: user maintains strategic oversight with periodic reviews (5-10% audit), especially for high-stakes outputs
• Level 5: Agent: user maintains governance oversight with systematic audits (1-5% review) despite autonomous operation

Practical Implications

Classifying language models along SAE-like levels provides practical benefits:

1. Common vocabulary for developers, researchers, policymakers, and end-users
2. Realistic expectations about capabilities—the difference between tools assisting writing versus fully automating complex decision-making processes
3. Regulatory guidance for ensuring safe deployment at each stage
4. Effort per level is increasing, probably exponentially

Design Priorities

It’s vital not simply to categorize these technologies for academic interest but also because such clarity informs design priorities:

• Should future efforts focus on improving reliability before granting more independence?
• How do safety concerns evolve as we move up each level?
• What ethical considerations arise when deploying increasingly autonomous NLP systems?

Each incremental step toward higher levels demands careful consideration regarding:

• Transparency: Can users understand when they’re interacting with an assistant versus an agent?
• Accountability: Who bears responsibility if an AI-generated statement causes harm?

Conclusion

Applying SAE-level classifications offers more than just terminology—it provides a roadmap illustrating how far we’ve come and how much further we need to go in developing intelligent language systems capable not only of mimicking human conversation but doing so responsibly across diverse environments.

Recognizing where current technology resides on this spectrum enables us all—from engineers designing smarter assistants to regulators crafting informed policies—to make conscious choices grounded in realistic assessments rather than hype or fear.

As artificial intelligence continues its ascent along these levels—from rudimentary support towards full autonomy—the journey will demand ongoing collaboration among technologists, ethicists, policymakers, and ultimately society itself to ensure these powerful tools serve humanity’s best interests every step along the way.

References

SAE J3016™. “Taxonomy and Definitions for Terms Related to Driving Automation Systems for On-Road Motor Vehicles.” First published: 2014. Most recent version (as of 2024): SAE J3016_202104 (April 2021). 🔗 https://www.sae.org/standards/content/j3016_202104/

Hoffman, R. R., Johnson, M., Bradshaw, J. M., & Underbrink, A. (2013). “Trust in automation.” IEEE Intelligent Systems, 28(1), 84–88. DOI: 10.1109/MIS.2013.24