[
https://issues.apache.org/jira/browse/CAMEL-26285?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Claus Ibsen updated CAMEL-26285:
--------------------------------
Description:
The vector database components milvus, qdrant, weaviate and pgvector, and
neo4j, take the action only from a header ({{CamelMilvusAction}},
{{CamelQdrantAction}}, {{CamelWeaviateAction}}, {{CamelPgVectorAction}},
{{CamelNeo4jOperation}}), and the producer fails with
{{NoSuchHeaderException("The action is a required header")}} without it. There
is no endpoint option, so a route cannot write {{milvus:docs?action=SEARCH}}:
it must set the header before every call. pinecone had the same design until
CAMEL-21796 added an {{action}} option, with the header still winning
({{in.getHeader(ACTION, getConfiguration().getAction(), ...)}}).
It also leaves the catalog body types (CAMEL-24961) without the Java type of
each action. {{byOption}} gives the carriers and the fully qualified
{{javaType}} per value of an option, but there is no option to key on, so
milvus is {{pojo}} without a {{javaType}} on both sides, and its description
names the SDK classes by their simple names only (CreateCollectionParam,
InsertParam, SearchSimpleParam ... in; RpcStatus, MutationResult,
SearchResponse, QueryResults out). The same for qdrant (VectorParams,
PointStruct, PointId ... in; RetrievedPoint, ScoredPoint, CollectionInfo out)
and weaviate (WeaviateObject, InsertManyResponse, QueryResponse ... out). The
carriers depend on the action too, for pgvector and neo4j as well, so the
catalog gives the union of all actions.
*Proposed change*:
# Add an {{action}} endpoint option to milvus, qdrant, weaviate and pgvector,
and an {{operation}} option to neo4j, as pinecone has: the header, when set,
wins over the option, and without both the producer fails as now.
# Give their body types per value of the option with {{byOption}}: the carriers
and the fully qualified {{javaType}} of each action, so tooling (validator,
TUI, MCP) knows which SDK class an action takes and gives.
h4. Check other components for the same design flaw
Other components may also need a header to be used normally: an operation, an
action or another value that is required, or reachable only, from a header and
not from an endpoint option. These need the same check and fix. A first scan of
the catalog, of the headers that select an operation only:
* *required, no option*: milvus, qdrant, weaviate, pgvector, neo4j (this issue);
* *a default, but the other operations only by header*: zookeeper
({{CamelZookeeperOperation}}: WRITE by default, DELETE by header only) and jcr
({{CamelJcrOperation}}: insert by default);
* *fine*: azure-servicebus ({{producerOperation}}) and hazelcast
({{defaultOperation}}) have an option beside the header.
The scan only looked at the headers that select an operation. Required headers
that carry other values, such as an id or a key, still need checking.
h4. Found while looking, not part of this issue
Of the 270 component sides with the {{pojo}} carrier, 206 have no {{javaType}}.
Apart from the components above:
* *The option exists, the per-operation types are missing* (data only,
{{byOption}} covers it today): aws-config, aws-security-hub, aws2-ecs,
aws2-eks, aws2-eventbridge, aws2-iam, aws2-kms, aws2-mq, aws2-msk,
aws2-redshift-data, aws2-rekognition, aws2-step-functions, aws2-sts,
aws2-textract, aws2-timestream, aws2-transcribe, hwcloud-frs, stripe,
ibm-watsonx-data, the openstack components, and some operations of docker, git,
consul, dapr, elasticsearch, opensearch and the kubernetes components.
* *The API components* select the method by two options, apiName and
methodName, which {{byOption}} cannot key: as2, box, braintree, dhis2, fhir,
google-calendar, google-drive, google-mail, google-sheets, olingo2, olingo4,
twilio and zendesk. The return types are known from the generated API method
classes, so the build could generate them.
* *A class the user configures* (jpa and jooq entityType, grpc, thrift, cxf,
mapstruct, optaplanner, stax, hazelcast, ignite, jgroups, vertx, salesforce and
servicenow models): no {{javaType}} is right for these.
Raised on the dev list in reply to the body types work: for a Java object, the
fully qualified name is worth having, such as for milvus.
_Claude Code on behalf of davsclaus_
was:
The catalog body types (CAMEL-24961) give the Java type of a body with
{{javaType}}, also per value of an endpoint option with {{byOption}}, such as
{{operation=putObject}}. Some components pick the action from a header only,
and have no option for it, so the catalog cannot say which class each action
takes and gives:
* *milvus* ({{CamelMilvusAction}}, 7 actions): producerIn and producerOut are
{{pojo}} without a {{javaType}}; the description names the SDK classes by their
simple names (CreateCollectionParam, InsertParam, SearchSimpleParam ... in;
RpcStatus, MutationResult, SearchResponse, QueryResults out).
* *qdrant* ({{CamelQdrantAction}}, 7 actions): VectorParams, PointStruct,
PointId ... in; a List of RetrievedPoint or ScoredPoint, CollectionInfo out.
* *weaviate* ({{CamelWeaviateAction}}, 11 actions): WeaviateObject,
InsertManyResponse, QueryResponse ... out.
The carriers depend on the header as well, for these and for pgvector
({{CamelPgVectorAction}}) and neo4j ({{CamelNeo4jOperation}}), so the catalog
now gives the union of the carriers of all actions.
*Proposed change*: let a body type be given per value of a header, as
{{byOption}} does for an option (a {{byHeader}} beside it, or a header in
{{@BodyType.When}}), with the carriers and the fully qualified {{javaType}} of
each value, and fill it in for milvus, qdrant, weaviate, pgvector and neo4j.
Tooling (validator, TUI, MCP) then reads it as it reads {{byOption}}. An
alternative for a component is an endpoint option for the action, as pinecone
has beside its {{CamelPineconeAction}} header, which {{byOption}} covers today.
h4. Found while looking, not part of this issue
Of the 270 component sides with the {{pojo}} carrier, 206 have no {{javaType}}.
Apart from the components above:
* *The option exists, the per-operation types are missing* (data only,
{{byOption}} covers it today): aws-config, aws-security-hub, aws2-ecs,
aws2-eks, aws2-eventbridge, aws2-iam, aws2-kms, aws2-mq, aws2-msk,
aws2-redshift-data, aws2-rekognition, aws2-step-functions, aws2-sts,
aws2-textract, aws2-timestream, aws2-transcribe, hwcloud-frs, stripe,
ibm-watsonx-data, the openstack components, and some operations of docker, git,
consul, dapr, elasticsearch, opensearch and the kubernetes components.
* *The API components* select the method by two options, apiName and
methodName, which {{byOption}} cannot key: as2, box, braintree, dhis2, fhir,
google-calendar, google-drive, google-mail, google-sheets, olingo2, olingo4,
twilio and zendesk. The return types are known from the generated API method
classes, so the build could generate them.
* *A class the user configures* (jpa and jooq entityType, grpc, thrift, cxf,
mapstruct, optaplanner, stax, hazelcast, ignite, jgroups, vertx, salesforce and
servicenow models): no {{javaType}} is right for these.
Raised on the dev list in reply to the body types work: for a Java object, the
fully qualified name is worth having, such as for milvus.
_Claude Code on behalf of davsclaus_
Summary: camel-milvus, camel-qdrant, camel-weaviate, camel-pgvector,
camel-neo4j - add an action option instead of requiring a header, and give the
Java type of each action in the catalog (was: camel-catalog: body types by the
value of a header, with the Java type of each, such as the action of milvus)
> camel-milvus, camel-qdrant, camel-weaviate, camel-pgvector, camel-neo4j - add
> an action option instead of requiring a header, and give the Java type of
> each action in the catalog
> ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: CAMEL-26285
> URL: https://issues.apache.org/jira/browse/CAMEL-26285
> Project: Camel
> Issue Type: Improvement
> Components: camel-catalog
> Reporter: Claus Ibsen
> Assignee: Claus Ibsen
> Priority: Major
> Fix For: 4.24.0
>
>
> The vector database components milvus, qdrant, weaviate and pgvector, and
> neo4j, take the action only from a header ({{CamelMilvusAction}},
> {{CamelQdrantAction}}, {{CamelWeaviateAction}}, {{CamelPgVectorAction}},
> {{CamelNeo4jOperation}}), and the producer fails with
> {{NoSuchHeaderException("The action is a required header")}} without it.
> There is no endpoint option, so a route cannot write
> {{milvus:docs?action=SEARCH}}: it must set the header before every call.
> pinecone had the same design until CAMEL-21796 added an {{action}} option,
> with the header still winning ({{in.getHeader(ACTION,
> getConfiguration().getAction(), ...)}}).
> It also leaves the catalog body types (CAMEL-24961) without the Java type of
> each action. {{byOption}} gives the carriers and the fully qualified
> {{javaType}} per value of an option, but there is no option to key on, so
> milvus is {{pojo}} without a {{javaType}} on both sides, and its description
> names the SDK classes by their simple names only (CreateCollectionParam,
> InsertParam, SearchSimpleParam ... in; RpcStatus, MutationResult,
> SearchResponse, QueryResults out). The same for qdrant (VectorParams,
> PointStruct, PointId ... in; RetrievedPoint, ScoredPoint, CollectionInfo out)
> and weaviate (WeaviateObject, InsertManyResponse, QueryResponse ... out). The
> carriers depend on the action too, for pgvector and neo4j as well, so the
> catalog gives the union of all actions.
> *Proposed change*:
> # Add an {{action}} endpoint option to milvus, qdrant, weaviate and pgvector,
> and an {{operation}} option to neo4j, as pinecone has: the header, when set,
> wins over the option, and without both the producer fails as now.
> # Give their body types per value of the option with {{byOption}}: the
> carriers and the fully qualified {{javaType}} of each action, so tooling
> (validator, TUI, MCP) knows which SDK class an action takes and gives.
> h4. Check other components for the same design flaw
> Other components may also need a header to be used normally: an operation, an
> action or another value that is required, or reachable only, from a header
> and not from an endpoint option. These need the same check and fix. A first
> scan of the catalog, of the headers that select an operation only:
> * *required, no option*: milvus, qdrant, weaviate, pgvector, neo4j (this
> issue);
> * *a default, but the other operations only by header*: zookeeper
> ({{CamelZookeeperOperation}}: WRITE by default, DELETE by header only) and
> jcr ({{CamelJcrOperation}}: insert by default);
> * *fine*: azure-servicebus ({{producerOperation}}) and hazelcast
> ({{defaultOperation}}) have an option beside the header.
> The scan only looked at the headers that select an operation. Required
> headers that carry other values, such as an id or a key, still need checking.
> h4. Found while looking, not part of this issue
> Of the 270 component sides with the {{pojo}} carrier, 206 have no
> {{javaType}}. Apart from the components above:
> * *The option exists, the per-operation types are missing* (data only,
> {{byOption}} covers it today): aws-config, aws-security-hub, aws2-ecs,
> aws2-eks, aws2-eventbridge, aws2-iam, aws2-kms, aws2-mq, aws2-msk,
> aws2-redshift-data, aws2-rekognition, aws2-step-functions, aws2-sts,
> aws2-textract, aws2-timestream, aws2-transcribe, hwcloud-frs, stripe,
> ibm-watsonx-data, the openstack components, and some operations of docker,
> git, consul, dapr, elasticsearch, opensearch and the kubernetes components.
> * *The API components* select the method by two options, apiName and
> methodName, which {{byOption}} cannot key: as2, box, braintree, dhis2, fhir,
> google-calendar, google-drive, google-mail, google-sheets, olingo2, olingo4,
> twilio and zendesk. The return types are known from the generated API method
> classes, so the build could generate them.
> * *A class the user configures* (jpa and jooq entityType, grpc, thrift, cxf,
> mapstruct, optaplanner, stax, hazelcast, ignite, jgroups, vertx, salesforce
> and servicenow models): no {{javaType}} is right for these.
> Raised on the dev list in reply to the body types work: for a Java object,
> the fully qualified name is worth having, such as for milvus.
> _Claude Code on behalf of davsclaus_
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
