[
https://issues.apache.org/jira/browse/CAMEL-24977?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Luigi De Masi resolved CAMEL-24977.
-----------------------------------
Fix Version/s: 4.23.0
Resolution: Fixed
> Add provider-agnostic semantic evaluation across Camel EIPs and error handling
> ------------------------------------------------------------------------------
>
> Key: CAMEL-24977
> URL: https://issues.apache.org/jira/browse/CAMEL-24977
> Project: Camel
> Issue Type: New Feature
> Components: camel-ai, camel-core, camel-yaml-dsl, eip
> Reporter: Luigi De Masi
> Assignee: Luigi De Masi
> Priority: Major
> Fix For: 4.23.0
>
>
> h2. Motivation
> Provide a provider-agnostic semantic evaluation API and Camel language so
> route authors can ask named questions about message content and use the
> resulting boolean decisions, categories, or scores across EIPs. For example,
> classify a support message, assess urgency, decide whether another attempt is
> worthwhile after a failure, validate whether an output addresses a request,
> or check whether a permitted action serves an approved task. The reusable
> capability is semantic evaluation; routing, validation, error handling and
> contextual action checks are applications of that capability.
> CAMEL-24871 added the Jev integration. This proposal builds on that work by
> separating the common question/evaluation contract from provider-specific
> implementations and using existing Camel EIP expression and predicate APIs.
> Jev can implement the common SPI, and other hosted or local classifiers can
> implement it without requiring Jev wire compatibility.
> The desired authoring experience is declarative: named questions in the same
> YAML file as routes, automatic provider-adapter discovery, and direct use
> through Camel's language integration. Route authors should not need to
> instantiate a predicate or adapter bean for the normal case.
> This is a design proposal. The syntax below illustrates the intended
> behavior; public names, packaging and exact DSL/schema details require
> community review. It does not claim that the proposed semantic language is
> already available.
> Keep the artifact name {{camel-semantic}} and language identifier
> {{semantic}}. Use *Semantic Evaluation* as the public catalog title and
> describe it as: "Evaluate named questions about message content to produce
> boolean decisions, categories and scores through provider adapters."
> h2. Common semantic evaluation contract
> * Define provider-independent question and result types for categorical
> Choice, yes/no judgments (including probability-to-boolean decision policy),
> and scoring against descriptive ordered levels. Preserve the useful shape of
> Jev Choice/Noul/Score through its adapter; final common type names should not
> force other providers to adopt Jev terminology.
> * Separate the selected category/value, probabilities, provider confidence,
> uncertainty policy and operational errors. A probability, a confidence
> measure and a guarantee of correctness are not interchangeable. Preserve
> provider metadata where useful without requiring all providers to report the
> same optional fields.
> * A provider component advertises an adapter implementing the common SPI. The
> adapter maps questions/state to its backend, validates supported
> capabilities, and maps typed results back to the common contract.
> * Expose evaluations through Camel's existing {{Language}}, {{Expression}}
> and {{Predicate}} contracts. Predicate use requires a boolean decision or
> explicit comparison/threshold policy; a category string is not implicitly a
> predicate.
> * Keep provider credentials, model selection, transport/inference
> configuration, and provider-specific resources with the implementing
> component. Define thread safety, bounded evaluation, cancellation and
> lifecycle behavior. Synchronous Predicate/Expression evaluation must document
> any blocking inference call.
> h2. Named questions and state in YAML
> Support a dedicated top-level {{semantic.question}} declaration from the
> initial version, alongside route definitions. The following is proposed
> syntax:
> {code}
> - semantic:
> question:
> department:
> type: choice
> state: ${header.myState}
> instructions: Which department should handle this message?
> criteria:
> billing: Invoices, payments, and refunds
> technical: Bugs, outages, and technical problems
> other: Everything else
> - route:
> id: ticket-routing
> from:
> uri: direct:tickets
> steps:
> - setProperty:
> name: department
> expression:
> language:
> language: semantic
> expression: "ref:department"
> - choice:
> when:
> - expression:
> simple:
> expression: "${exchangeProperty.department} == 'billing'"
> steps:
> - to: direct:billing
> - expression:
> simple:
> expression: "${exchangeProperty.department} ==
> 'technical'"
> steps:
> - to: direct:technical
> otherwise:
> steps:
> - to: direct:general
> {code}
> Each question owns its input-state selector. An optional language default can
> be configured:
> {code}
> camel.language.semantic.default-state=${body}
> {code}
> Proposed precedence: question {{state}}, then configured {{default-state}},
> then built-in {{${body}}}. Use Camel Simple expressions for these selectors
> initially. Compile/initialize selectors during route setup and evaluate
> against the current exchange at runtime. Only an absent selector inherits the
> default: a malformed selector or an explicitly selected header that evaluates
> to null must not silently cause different data to be classified. Input values
> are data, not recursively evaluated expressions.
> Question declarations need loader, validation, schema and tooling support.
> Provide the YAML resolver from {{camel-semantic}} through the YAML DSL
> extension mechanism; the core YAML DSL must not depend on the semantic
> component. Keep YAML support optional for Java applications using semantic
> evaluation. Resolve references before traffic where possible; reject
> duplicate names, unknown references and unsupported question capabilities.
> Define behavior across multiple route resources and reload without retaining
> obsolete definitions. Loading/validating declarations must not perform
> inference.
> h2. Adapter discovery and explicit selection
> The language must not require a provider endpoint URI such as
> {{jev:decisions}}. With exactly one advertised adapter implementation
> available, discover and select it automatically. With none, report a missing
> provider; with multiple, report the candidates and require explicit
> selection. Do not choose by classpath order or construct every provider
> during discovery.
> Allow an explicit reference to an existing registered instance:
> {code}
> camel.language.semantic.adapter=mySemanticAdapter
> {code}
> Alternatively, accept a fully qualified class name:
> {code}
> camel.language.semantic.adapter=com.example.MySemanticAdapter
> {code}
> The example class is illustrative. Use plain bean or class names, without
> {{#bean:}} or {{#class:}} prefixes. Registry lookup takes precedence over
> class resolution; this uses normal property binding without extending the
> core property-configurer SPI. For a class-selected or discovered adapter, the
> semantic infrastructure should resolve and type-check the class through
> Camel, instantiate it using the context Injector, register it once for the
> language configuration, and integrate its lifecycle. Reuse referenced beans
> without duplicating their existing lifecycle ownership. Define registry
> ownership, collision handling and startup-failure cleanup; binding an object
> does not itself establish service lifecycle management.
> Provider configuration must remain effective even when the route contains no
> provider endpoint. For example, a Jev adapter must honor the configured Jev
> component's credentials and connection options. A manually declared adapter
> bean is an optional customization path, not a prerequisite for normal
> operation.
> h2. Choice integration through the existing model
> Keep Camel's existing Choice, When and expression models unchanged. Boolean
> questions can be used directly as ordinary when predicates. For category
> questions, evaluate once using Set Property or Set Header and compare the
> stored category using ordinary Choice predicates:
> {code:java}
> from("direct:tickets")
> .setProperty("department").language("semantic", "ref:department")
> .choice()
> .when(exchangeProperty("department").isEqualTo("billing"))
> .to("direct:billing")
> .when(exchangeProperty("department").isEqualTo("technical"))
> .to("direct:technical")
> .otherwise()
> .to("direct:general")
> .end();
> {code}
> * Perform one evaluation each time execution reaches the Set Property or Set
> Header step. Branch predicates reuse the stored result without invoking the
> provider again.
> * Place that evaluation step inside a loop or retry path when changed state
> must be reevaluated. Nested decisions can use distinct properties to retain
> independent results. There is no implicit exchange-wide semantic-result cache.
> * Valid categories without an explicit matching branch follow ordinary Choice
> otherwise behavior. Inference and invalid-response failures follow Camel
> error handling; uncertainty requires an explicit policy.
> * Keep Choice precondition behavior unchanged. Message-dependent semantic
> predicates belong in runtime routes, not in startup preconditions evaluated
> against a dummy exchange.
> * Use existing Java, XML and YAML expression/predicate forms. No Choice
> selector, literal when-value syntax, core EIP model changes or new routing
> EIP is included in this issue.
> h2. Evaluation across other EIPs
> The following are candidate uses, not a requirement to introduce
> semantic-specific changes to every EIP.
> || EIP / integration point || Example question or benefit || Integration ||
> | Filter | Is this message relevant to this workflow? | A boolean predicate.
> Existing regression tests already cover semantic filtering. |
> | Validate | Does this bug report describe an actionable problem? | A boolean
> decision supplies the validation predicate; negative decisions use normal
> validation failure handling. |
> | Set Header / Set Property | Attach department, sentiment or urgency to a
> message. | Store a category or score for explicit reuse by subsequent
> processors. |
> | Aggregate: grouping | Group feedback by delivery, pricing or usability. |
> Use a category as part of the correlation key, retaining tenant/case
> boundaries and bounded aggregation windows where needed. |
> | Aggregate: completion | Does the accumulated conversation contain enough
> information to proceed? | Evaluate a boolean completion predicate against
> selected aggregate state, alongside a size or timeout completion condition. |
> | Recipient List | Send a complaint to support and the relevant product team.
> | Map a category, or multiple independent decisions, to configured
> recipients. |
> | Routing Slip | Select a document-processing sequence. | Map a document
> category to a predefined sequence of endpoints. |
> | Enrich | Choose the appropriate knowledge source for a support request. |
> Map a topic to a configured enrichment endpoint; ordinary code retrieves and
> merges the additional data. |
> | Loop | Continue refinement while the updated output remains incomplete. |
> Reevaluate a semantic predicate against current state, with an explicit
> iteration/time budget. |
> | Sort | Order a batch of tickets by assessed urgency. | Score each item
> once, then compare stored scores with a deterministic comparator. |
> | On Exception / retryWhile | Given this failure, is another attempt
> worthwhile? | Use a boolean predicate with an explicit retry budget checked
> before inference. {{retryWhile}} replaces the normal {{maximumRedeliveries}}
> decision; the predicate must enforce the cap itself. A negative decision or
> exhausted budget follows the configured escalation path. |
> | Contextual action validation | Does this proposed action serve the approved
> task? | Apply existing identity, permission and tenant checks first, then
> Validate with a semantic predicate before the action. Negative, uncertain or
> failed evaluations must not execute the action. No dedicated semantic
> {{AuthorizationPolicy}} implementation is required. |
> | Intercept / Intercept Send To Endpoint | Does this outgoing message require
> review? | Use {{onWhen}} to conditionally intercept processing or divert a
> send across selected routes. When enforcing a gate, configure interception to
> prevent the original send on rejection. |
> | Dynamic Router | Given the updated state, which processing step should run
> next? | Map a category to a configured endpoint after each step. Enforce a
> hop budget and explicitly return {{null}} from the routing expression to
> finish. |
> | Split with Filter or Validate | Which individual records are relevant or
> acceptable? | Split an existing collection, then evaluate each item. Semantic
> evaluation does not itself extract or generate the collection. |
> | Poll Enrich / To Dynamic | Which configured source or service should handle
> this content? | Classify first, then map the label to an approved endpoint.
> This extends the destination-mapping pattern used for Enrich. |
> | Throttle | Is this request routine or expensive to process? | Classify into
> configured workload groups, then apply deterministic rate or concurrency
> limits. The model does not invent limits. |
> | On Completion | Does this completed interaction warrant follow-up? |
> Evaluate for auditing, review or follow-up processing. This cannot prevent
> actions already performed. |
> Validate, metadata assignment and Aggregate can consume Predicate/Expression
> results directly. Recipient List, Routing Slip and Enrich require a mapping
> from semantic labels to route-author-defined destinations. The classifier
> does not invent endpoint addresses or processing steps.
> For aggregate completion, the aggregation strategy must expose the relevant
> accumulated content; the classifier does not implicitly remember earlier
> exchanges. The default completion predicate sees the aggregated exchange,
> while {{eagerCheckCompletion}} uses the incoming exchange. State selection
> must match that behavior.
> For Sort, the EIP expression selects the collection and the comparator
> determines ordering. Inference belongs before comparison. For loops or
> changed input, explicit result reuse must not hide relevant state changes.
> Bounded retry and contextual action validation use existing Camel hooks and
> are part of the focused documentation and integration-test scope. Prepare
> current failure context in {{onExceptionOccurred}}, before the retry
> predicate; {{onRedelivery}} runs later. A retry-predicate evaluation error
> propagates as an error and does not automatically execute the normal
> escalation route. For contextual action validation, keep authentication,
> permissions and tenant boundaries authoritative; semantic evaluation is an
> additional check, and its failures must prevent execution.
> h2. Broader Camel applications
> * *AI tool and MCP routes:* check whether a proposed tool action fits the
> approved task before executing it, alongside existing permissions. Camel
> exposes tools through ordinary routes, so contextual validation can be
> applied there. See [Camel's tool authorization
> example|https://camel.apache.org/blog/2026/09/securing-ai-agent-tools/].
> * *Input and output quality checks:* ask "Is this request sufficiently
> specified?" or "Does this answer address the request?" Route-level validation
> can surround an AI component. Direct integration into LangChain4j's internal
> guardrail interfaces would require an additional bridge; the semantic
> language does not provide that bridge. See [LangChain4j Agent guardrail
> support|https://camel.apache.org/components/4.22.x/langchain4j-agent-component.html].
> * *Retrieval relevance and ranking:* evaluate retrieved documents against the
> question, filter irrelevant passages, and score candidates before
> deterministic sorting. This combines Split, Filter, Set Property and Sort; it
> complements retrieval rather than implementing a search engine.
> The additional combinations are integration candidates inferred from existing
> extension points, not a claim that all combinations have been tested.
> Evaluation quality depends on the provider and question. Keep candidate
> exploration distinct from the initial regression coverage below; no
> semantic-specific changes to the core EIP models are required.
> h2. Acceptance criteria and initial coverage
> # A common provider-independent question/result contract and adapter SPI
> support direct semantic Expression/Predicate use without requiring manual
> predicate instances.
> # A YAML file can declare named questions under {{semantic.question}} and
> reference them from routes. Loader, schema, validation and tooling support
> agree on the proposed configuration.
> # Adapter discovery, bean-reference override and class-name override are
> tested, including missing/ambiguous providers, wrong types, instance reuse,
> configuration and lifecycle behavior.
> # Question state overrides the language default; omitting both uses the body.
> Invalid selectors and missing selected state produce useful errors without
> silent input substitution.
> # Choice integration uses the unchanged EIP model: store a category once per
> evaluation step and route with ordinary predicates. Tests cover invocation
> counts, first/later/otherwise branches, preservation of message content and
> reevaluation after state changes in a loop. Boolean questions remain usable
> directly as predicates.
> # Demonstrate Choice, Filter, Validate, Set Header / Set Property, Aggregate,
> Recipient List, Routing Slip, Enrich, bounded Loop and Sort integration.
> Document the additional candidate integrations and their mapping/state
> requirements without assuming every EIP needs a core change.
> # Document and test bounded semantic retry: approval followed by success,
> negative decision and budget exhaustion leading to escalation, refreshed
> failure state, no inference after the budget is exhausted, and propagation of
> timeout, malformed-response and uncertainty errors without another attempt.
> # Document and test contextual action validation: ordinary permission denial
> prevents inference and action; a permitted action proceeds only after a
> positive semantic decision; negative, uncertain and failed evaluations
> prevent execution. Use existing Validate and authorization services without
> introducing a dedicated semantic authorization policy.
> # Provide a Jev adapter and validate the contract with deterministic test
> adapters, including a second implementation and capability mismatch cases. Do
> not claim interchangeability of model quality, probabilities or calibration.
> # Tests cover meaningful result mappings, invocation counts,
> threshold/uncertainty policy, explicit result reuse, changed-state
> reevaluation, timeouts/failures and preservation of message content. Routine
> tests require no live model credentials.
> # Document the supported property namespaces and runtime configuration paths.
> Coordinate with the generic-language starter work in CAMEL-24913 and
> CAMEL-24914 rather than assuming language configuration is generated
> automatically.
> h2. Boundaries and references
> Choosing a local inference runtime (pure JVM/WASM versus native inference),
> model weights, or a particular hosted service is outside this proposal's
> initial decision. A provider can implement Jev API compatibility, but the
> common SPI does not require it and does not imply equivalent proprietary
> model behavior or calibration. The SPI's final package, discovery metadata
> and public syntax remain open to design review.
> Related work:
> * CAMEL-24871 — Jev component and semantic EIP integration background.
> * CAMEL-24913 — Jev Spring Boot starter configuration.
> * CAMEL-24914 — Starter generation for languages backed by the generic model.
> * CAMEL-17755 — Existing Choice precondition design.
> * [Camel Language
> SPI|https://github.com/apache/camel/blob/main/core/camel-api/src/main/java/org/apache/camel/spi/Language.java]
> * [Choice
> model|https://github.com/apache/camel/blob/main/core/camel-core-model/src/main/java/org/apache/camel/model/ChoiceDefinition.java]
> * [Aggregate
> EIP|https://camel.apache.org/components/next/eips/aggregate-eip.html]
> * [Intercept|https://camel.apache.org/components/next/eips/intercept.html]
> * [Dynamic Router
> EIP|https://camel.apache.org/components/next/eips/dynamicRouter-eip.html]
> * [On Completion|https://camel.apache.org/manual/oncompletion.html]
> * [Retry predicates|https://camel.apache.org/manual/exception-advanced.html]
> * [System One Models|https://systemonemodels.org/]
> _AI-generated by Codex on behalf of
> [luigidemasi|https://github.com/luigidemasi]._
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
