AI Architecture22 min read

JEV vs LLM: Choosing Between Deterministic Logic and Generative Intelligence

A practical guide to understanding when to use JEV-style rule engines, when to use Large Language Models, and how to combine them in production systems.

LLMEnterprise AIRule EnginesAgentic SystemsAI Architecture

As organizations accelerate AI adoption, one architectural mistake appears repeatedly:


Treating every problem as an LLM problem.


Large Language Models are remarkably good at understanding language, extracting information, reasoning over unstructured context and generating human-friendly responses.


But they are not the right component for every decision.


Many business processes still require something much less exciting, but far more predictable:


Deterministic logic.


Eligibility checks. Authorization. Policy enforcement. Transaction limits. Compliance rules. Approval thresholds.


These are areas where a traditional rule-driven execution and validation system can be a better fit than an LLM.


For simplicity, I will call this a JEV-style system throughout this article.


Terminology note: JEV is not a universally standardized industry term. In this article, JEV refers to a structured, deterministic execution and validation layer that evaluates explicit business rules and controls whether an action is permitted.


The interesting question, therefore, is not:


JEV or LLM?


It is:


Which component should own which responsibility?


In many production systems, the strongest answer is both.




The Production Problem


Imagine an enterprise customer support platform.


It needs to:


  • Understand customer requests
  • Respond naturally
  • Enforce company policies
  • Validate compliance requirements
  • Authorize actions
  • Maintain audit trails
  • Operate at scale
  • Keep costs under control

Now consider this request:


"I forgot my password, but I also changed my email address and no longer have access to the old one."


An LLM may be excellent at recognizing that the customer has an account recovery problem involving a changed identity attribute.


But should the LLM independently decide:


  • Whether identity verification succeeded?
  • Whether account ownership has been established?
  • Whether regulatory requirements have been satisfied?
  • Whether the account's email address may be changed?
  • Whether a privileged recovery action should execute?

Usually, no.


Those decisions should be based on explicit policy and trusted system state.


This gives us a useful initial separation:


| Responsibility | JEV | LLM | | --- | :---: | :---: | | Business rule execution | ✅ | ❌ | | Natural language understanding | ❌ | ✅ | | Policy enforcement | ✅ | ❌ | | Content generation | ❌ | ✅ | | Deterministic authorization | ✅ | ❌ | | Reasoning over unstructured text | Limited | ✅ | | Audit trail | ✅ | Possible, but not sufficient alone | | Conversational interaction | Limited | ✅ |


This distinction becomes particularly important when an AI system progresses from answering questions to taking actions.




What Is a JEV-Style System?


A JEV-style system executes explicitly defined rules against structured inputs.


A simplified rule might look like this:


IF:
  customer_age >= 18
  AND account_status == "ACTIVE"
  AND verification_passed == true

THEN:
  permit_transaction

Its important characteristics are:


  • Deterministic execution
  • Explicit rules
  • Repeatable outcomes
  • Testability
  • Traceability
  • Controlled change management

Given the same rules and the same input state, the system should produce the same decision.


The key advantage is not simply that these systems are traditional.


It is that predictability is sometimes a feature.


For systems responsible for authorization, eligibility or policy enforcement, predictability may be one of the most important features they have.




What Is an LLM?


A Large Language Model is a probabilistic model capable of generating output based on patterns learned during training and the context provided at inference time.


Ask an LLM:


Summarize this contract and identify its renewal,
termination and notice clauses.

and it can work through language that would be extremely difficult to represent as thousands of traditional rules.


LLMs are particularly effective when the input is:


  • Ambiguous
  • Conversational
  • Unstructured
  • Semantically rich
  • Variable in format

That makes them excellent for tasks such as:


  • Summarization
  • Information extraction
  • Classification
  • Intent detection
  • Conversational interfaces
  • Semantic search
  • Interpretation

But there is an important architectural distinction:


Understanding something is not the same as being authorized to decide what happens next.


This distinction is fundamental when designing production AI systems.




Two Different Execution Models


At a high level, deterministic systems and LLMs solve problems differently.


A deterministic execution flow looks approximately like this:


Structured Input
       |
       v
   Rule Engine
       |
       v
Policy Evaluation
       |
   +---+---+
   |       |
   v       v
 Allow    Deny
   |       |
   +---+---+
       |
       v
   Audit Log

The important characteristic is that the decision path is explicitly defined.


An LLM execution flow looks different:


User / Document
       |
       v
Context + Prompt
       |
       v
      LLM
       |
       v
Interpretation
       |
       v
Structured Output
or Generated Response

The LLM handles uncertainty and language far better.


But the generated interpretation should not automatically become permission to perform a sensitive action.


That brings us to the architecture I prefer.




The Hybrid Pattern


The simplest architecture might look like this:


User
  |
  v
LLM
  |
  v
Action

It is attractive because it is easy to build.


It is also where problems begin once the LLM receives access to powerful business operations.


A stronger architecture separates interpretation from decision-making and execution.


Hybrid architecture separating interpretation from decision-making and execution


A production architecture separates probabilistic interpretation from deterministic validation, authorization and business execution.


Conceptually, the flow becomes:


User
  |
  v
LLM
Interpret / Extract
  |
  v
Structured Request
  |
  v
Schema Validation
  |
  v
JEV / Policy Engine
Validate / Decide / Authorize
  |
  v
Business System
Execute
  |
  v
LLM
Explain Result
  |
  v
User

The central idea is:


The LLM proposes an interpretation. The deterministic layer decides whether the resulting action is allowed.


This separation lets each component perform the work it is best suited for.




Why JEV Still Matters


1. Deterministic Behavior


Suppose a policy states:


Order age <= 30 days
AND
Item status = damaged
AND
Refund eligibility = true

→ Refund permitted

The business does not normally want the result to change because a model interpreted the policy differently during another execution.


It wants:


Same facts
+
Same rule version
=
Same decision

That property is extremely valuable for policy-driven systems.




2. Auditability


Imagine investigating why a transaction was rejected.


A deterministic system can produce an execution record such as:


Decision: DENIED
Rule: ACCOUNT_AGE_027
Reason: Account age < 30 days
Rule version: 4.3
Timestamp: 2026-09-27T14:31:22Z

That is considerably more useful operationally than:


"The model determined that the transaction appeared risky."


For important decisions, we want the system to tell us:


  • Which policy was applied
  • Which rule was triggered
  • Which facts were evaluated
  • Which version of the rules was active
  • What decision was produced

That information matters for debugging, governance and auditability.




3. Efficient Execution


Simple rules can usually be evaluated without invoking a generative model.


If something can reliably be expressed as:


if customer.age >= 18:
    ...

we probably do not need an LLM to determine the answer.


This leads to a useful architecture principle:


Don't spend probabilistic compute on deterministic work.


LLMs should be used where their language and reasoning capabilities provide meaningful value.




4. Explicit Policy Control


Rules are visible.


Rules can be:


  • Reviewed
  • Versioned
  • Unit tested
  • Approved
  • Deployed
  • Monitored
  • Rolled back

This makes change management easier when the underlying business requirement is explicit.




Where JEV Falls Short


The strengths of rule engines are also their limitation.


They require the world to be represented in terms the rules understand.


Consider:


IF message contains "refund"
THEN route_to_refund_process

It works for:


"I want a refund."


But what about:


"The product arrived broken. Can I get my money back?"


Or:


"I don't want another one. Please return what I paid."


Maintaining every possible linguistic variation as a rule quickly becomes impractical.


This is precisely the kind of ambiguity where an LLM provides value.




Rule Explosion


Complex rule ecosystems can also become difficult to manage.


Over time, you may end up with:


Rule 17 depends on Rule 6
Rule 38 overrides Rule 17
Rule 92 applies only when Rule 41 is false
...

At that point, determinism alone does not guarantee maintainability.


Good rule architecture still requires:


  • Modularity
  • Versioning
  • Dependency management
  • Automated testing
  • Conflict detection
  • Policy ownership

Deterministic does not automatically mean simple.




Where LLMs Excel


1. Language Understanding


Consider:


"The screen was already cracked when I opened the package."


An LLM might translate that into structured information:


{
  "intent": "damage_claim",
  "damage_type": "physical",
  "reported_condition": "damaged_on_arrival"
}

That translation from human language to machine-understandable structure is extremely valuable.


The resulting structured data can then be passed into conventional validation and business logic.




2. Information Extraction


Enterprise data frequently hides inside:


  • PDFs
  • Emails
  • Support tickets
  • Contracts
  • Reports
  • Meeting notes
  • Free-text forms

An LLM can transform that information into structured fields.


For example:


{
  "invoice_number": "INV-2026-1842",
  "due_date": "2026-10-15",
  "currency": "EUR",
  "amount": 4750
}

The result can then be validated by conventional software before it enters a downstream workflow.




3. Natural Interaction


Traditional software often makes humans speak the language of machines:


Press 1 for orders
Press 2 for returns
Press 3 for billing

LLMs allow machines to understand the language of humans:


"I received the wrong product and want to replace it before Friday."


That is a significant improvement in user experience.




4. Handling Novel Variations


A deterministic system needs anticipated paths.


An LLM can work with requests such as:


"Explain the main risks in this financial report as if you were briefing a non-technical executive."


Trying to create explicit rules for every possible variation of that request would defeat the purpose.




Where LLMs Fall Short


Hallucination and Confabulation


An LLM might tell a customer:


"Your policy allows returns within 90 days."


while the authoritative policy actually says 30 days.


Retrieval, grounding, structured output, validation and better prompting can reduce this risk.


They do not transform a generative model into a deterministic policy engine.


This matters whenever generated information can trigger meaningful business consequences.




Probabilistic Behavior


LLMs generate rather than execute traditional business rules.


Even when models are configured for highly consistent output, I would avoid building a critical architecture around the assumption:


"The model will always make exactly the same policy decision."


If reproducibility is a hard requirement, enforce the relevant policy outside the LLM.




Cost and Latency


Every model invocation introduces some combination of:


  • Inference cost
  • Token usage
  • Network latency
  • Model latency
  • Capacity requirements

That does not mean LLMs are inherently too expensive.


It means they should be used where their capabilities add value.


Checking:


amount < credit_limit

is usually not one of those places.




Real-World Use Cases


Scenario 1: Customer Support


A customer says:


"My phone arrived with a broken screen and I want my money back."


The LLM interprets the request:


{
  "intent": "refund_request",
  "reason": "damaged_on_arrival"
}

The JEV or policy layer evaluates:


Order exists?
Customer owns order?
Return window open?
Item eligible?
Required verification complete?

It might then return:


{
  "decision": "approved",
  "rule": "DAMAGED_ITEM_REFUND_V3"
}

The LLM can communicate the result:


Your refund request has been approved. You will receive confirmation when processing begins.


The model handles interpretation and communication.


The policy engine owns the decision.




Scenario 2: Insurance Claims


The LLM extracts information from a claim and supporting documents:


  • Policy number
  • Incident date
  • Loss type
  • Claim description
  • Supporting evidence

The deterministic layer evaluates:


  • Was the policy active?
  • Is this loss covered?
  • Does the deductible apply?
  • Are mandatory fields present?
  • Does the claim require manual review?

The LLM can then help explain the result to the customer or claims handler.


The architecture combines the strengths of both technologies without requiring either component to perform every responsibility.




Scenario 3: Enterprise Search


A user asks:


"What changed in our travel policy for European employees?"


The LLM is useful for understanding the question, retrieving relevant information and synthesizing a response.


But another layer should determine:


Who is the user?
Which documents may they access?
Which regions may they query?
Which sensitivity labels apply?

This distinction matters:


The LLM can help determine what information the user wants. It should not be the ultimate authority on whether the user is allowed to see it.


Authorization belongs outside the generative layer.




From Chatbots to Agents


This separation becomes even more important with agentic systems.


A chatbot primarily generates information.


An agent may also propose or initiate actions through tools.


An agent might attempt to:


  • Issue a refund
  • Update a CRM record
  • Create a support ticket
  • Modify an account
  • Query internal systems
  • Send an email
  • Start a business workflow

That means the architecture must distinguish between:


What the model proposes

and:


What the system permits

Agent policy control layer deciding whether a proposed action is executed, denied or escalated


The agent proposes an action, while the policy layer determines whether the action may be executed, denied or escalated for human review.


A useful pattern is:


LLM / Agent
     |
     | Proposes action
     v
Policy Engine
     |
 +---+-----------+
 |       |       |
 v       v       v
ALLOW   DENY  ESCALATE
 |               |
 v               v
Tool/API       Human
 |
 v
Audit

Or put more simply:


The agent can propose. The policy layer disposes.


This separation becomes increasingly valuable as an agent gains access to more powerful tools.




Example: Refund Agent


Suppose an AI agent determines:


{
  "action": "issue_refund",
  "order_id": "12345",
  "reason": "damaged_item"
}

That should be treated as an action proposal, not an authorization.


The deterministic layer receives trusted system data:


{
  "order_age_days": 12,
  "item_status": "damaged",
  "payment_status": "paid",
  "customer_verified": true
}

It can then evaluate the actual policy:


approve_refund = (
    order_age_days <= 30
    and item_status == "damaged"
    and payment_status == "paid"
    and customer_verified
)

Only after approval does the application call the refund service.


The model can then communicate the result:


Your refund has been approved and submitted for processing.

Notice what happened.


The LLM interpreted the request.


The policy layer made the decision.


The business service performed the transaction.


The LLM communicated the outcome.


That separation is much safer than giving the model unrestricted control over all four responsibilities.




Production Hardening


A hybrid architecture does not automatically become production-ready.


Several additional controls matter.


Structured Interfaces


Prefer:


LLM
 |
 v
Structured JSON
 |
 v
Schema Validation
 |
 v
Policy Engine

over:


LLM
 |
 v
Free Text
 |
 v
Business Action

Contracts between probabilistic and deterministic components should be explicit.


For example:


{
  "intent": "refund_request",
  "order_id": "12345",
  "reason": "damaged_item"
}

Before this reaches the policy engine, validate:


  • Required fields
  • Allowed values
  • Data types
  • Identifiers
  • Permissions

An LLM output should be treated as untrusted input until validated.




Authorization


Never rely solely on an LLM to enforce:


  • User permissions
  • Authentication
  • Transaction limits
  • Sensitive-data access
  • Mandatory compliance controls

Use conventional security and policy mechanisms for these responsibilities.


The LLM can request an operation.


It should not automatically acquire permission to perform that operation simply because it requested it.




Observability


Track both sides of the architecture.


For the LLM layer, useful measurements include:


  • Latency
  • Token consumption
  • Extraction accuracy
  • Tool-call failures
  • Groundedness
  • Escalation rate

For the deterministic layer, useful measurements include:


  • Rules triggered
  • Allow/deny ratio
  • Execution latency
  • Validation failures
  • Rule conflicts
  • Policy version

End-to-end measurements should include:


  • Task success
  • Business outcome
  • Human escalation
  • Unauthorized action attempts
  • Cost per successful task

Observability should tell us not simply whether the model answered, but whether the overall system behaved correctly.




Version Everything


Production AI systems involve considerably more than model versions.


Version:


Model
Prompt
System Instructions
Tools
Schemas
Rules
Policies
Knowledge Sources
Evaluations

Then an audit trail can tell us:


Request: 78451
Model: model-v3
Prompt: support-agent-v12
Policy: returns-v7.2
Rule: damaged-item-14
Decision: APPROVED
Tool: refund-api-v3

That is much closer to production-grade AI engineering than simply logging a conversation.




Human Escalation Is Part of the Architecture


There is another possible outcome besides ALLOW and DENY:


ALLOW
DENY
ESCALATE

For example:


Confidence insufficient?
Policy conflict?
Missing documents?
Transaction above threshold?
Unusual account state?

→ HUMAN REVIEW

A good architecture does not force automation where uncertainty is too high.


Sometimes the safest automated decision is:


Don't decide. Escalate.


Human review is not necessarily a failure of automation.


In high-impact systems, human review can be an intentional part of the control architecture.




When Should You Use JEV?


Choose deterministic execution when the requirement sounds like:


"Decide whether this is permitted according to explicit rules."


Typical characteristics include:


  • ✅ Deterministic outcomes
  • ✅ Policy enforcement
  • ✅ Auditability
  • ✅ Explicit business logic
  • ✅ Reproducibility
  • ✅ Authorization controls

Examples include:


  • Eligibility checks
  • Transaction limits
  • Identity verification workflows
  • Policy enforcement
  • Approval routing
  • Tax calculations
  • Regulatory checks



When Should You Use an LLM?


Choose an LLM when the requirement sounds like:


"Understand, extract, summarize, reason about or communicate this information."


Typical characteristics include:


  • ✅ Natural language understanding
  • ✅ Summarization
  • ✅ Semantic search
  • ✅ Information extraction
  • ✅ Conversational interaction
  • ✅ Handling unstructured input

Examples include:


  • Customer support assistants
  • Enterprise search
  • Document intelligence
  • Contract analysis
  • Meeting summaries
  • Knowledge assistants
  • Intent classification



When Should You Use Both?


Very often.


The most interesting enterprise workloads contain both uncertainty and rules.


A practical separation of responsibility looks like this:


| Layer | Primary Responsibility | | --- | --- | | LLM | Understand | | Retrieval | Find relevant knowledge | | Structured Schema | Normalize | | JEV / Policy Engine | Decide | | Authorization | Permit | | Business API | Execute | | LLM | Explain | | Observability | Measure | | Audit System | Record | | Human | Resolve exceptions |


This is not about limiting AI.


It is about placing AI where it provides the greatest advantage.




A Simple Decision Framework


When designing a new capability, I find it useful to ask a series of questions.


1. Is the input primarily unstructured?


If yes, an LLM may be useful.


2. Does the task require interpretation or semantic understanding?


If yes, consider an LLM.


3. Is there an explicit business rule that determines the outcome?


If yes, implement that rule deterministically.


4. Must identical facts produce the same policy decision?


If yes, keep that decision outside the LLM.


5. Does the action affect money, permissions, sensitive information, compliance or another important business state?


If yes, introduce deterministic authorization before execution.


6. Is the situation ambiguous even after validation?


Escalate instead of guessing.


In shorthand:


Need to understand?  → LLM
Need to decide?      → Rules
Need to authorize?   → Policy
Need to execute?     → Business API
Need to explain?     → LLM
Not sure?            → Human

This framework is intentionally simple.


The goal is not to determine which technology is more sophisticated.


The goal is to assign each responsibility to the component best suited to perform it.




Final Thoughts


The JEV vs LLM debate is based on a false choice.


The two approaches solve fundamentally different parts of the problem.


LLMs are powerful interfaces to ambiguity.


They can turn human language, documents, conversations and incomplete context into useful structured understanding.


Deterministic systems are powerful interfaces to policy.


They turn trusted facts and explicit rules into repeatable decisions.


So if the requirement is:


"Tell me what this means."


Use an LLM.


If the requirement is:


"Determine whether this is allowed."


Use deterministic policy evaluation.


And if the requirement is:


"Understand what the user wants, determine whether it is allowed, execute it and explain what happened."


Use both.


For me, the production pattern can be summarized in one line:


LLMs interpret. Rules decide. Systems execute. Humans handle the exceptions.


That separation gives us something much more valuable than simply an "AI-powered" application.


It gives us an architecture in which generative intelligence and deterministic control reinforce each other instead of competing for the same responsibility.

Continue Reading

Related Articles