[
https://issues.apache.org/jira/browse/CAMEL-26092?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Work on CAMEL-26092 started by Claus Ibsen.
-------------------------------------------
> camel-aws2-s3-vectors - the headers of an earlier step win over the endpoint,
> so a route from the consumer overwrites the consumed vectors
> ------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: CAMEL-26092
> URL: https://issues.apache.org/jira/browse/CAMEL-26092
> Project: Camel
> Issue Type: Bug
> Components: camel-aws2
> Affects Versions: 4.18.0, 4.22.0, 4.23.0
> Reporter: Claus Ibsen
> Assignee: Claus Ibsen
> Priority: Major
> Fix For: 4.24.0
>
>
> Found while adding the catalog body types of aws2-s3-vectors (CAMEL-24961).
> h4. 1. The headers of an earlier step win over the endpoint
> The producer reads the bucket and the index from the
> {{CamelAwsS3VectorsVectorBucketName}} and
> {{CamelAwsS3VectorsVectorIndexName}} headers first, and from its own options
> only when they are not set ({{AWS2S3VectorsProducer}}, main ed3d719b379c,
> lines 418-438). The key of putVectors comes from
> {{CamelAwsS3VectorsVectorId}} (line 140), and getVectors and deleteVectors
> read their keys from it before the body (lines 222 and 249). The consumer
> sets all three headers ({{AWS2S3VectorsConsumer}}, lines 123-125),
> createVectorBucket sets the bucket and putVectors sets the key (line 177).
> Checked on 4.24.0-SNAPSHOT with {{camel run}} against the AWS emulator of
> {{camel infra run aws}}, through a proxy that logs the requests:
> * a route from
> {{aws2-s3-vectors:bt-bucket?vectorIndexName=bt-index&consumerQueryVector=...}}
> to
> {{aws2-s3-vectors:bt-other?vectorIndexName=bt-other-index&operation=putVectors}}
> wrote its vector into bt-bucket/bt-index under the consumed key, so each
> consumed vector was overwritten;
> * a createVectorBucket on {{aws2-s3-vectors:bt-other}} after a
> createVectorBucket on bt-bucket created bt-bucket again ("The vector bucket
> bt-bucket already exists");
> * a getVectors after a putVectors in the same route ignored its body (the key
> "doc-list") and got the key of the putVectors.
> aws2-s3 had the same problem with the bucket header and has an explicit
> {{CamelAwsS3OverrideBucketName}} header for it.
> *Proposed change*: let the endpoint options win, or read only headers meant
> as input (an override header), and do not reuse the headers that the consumer
> and the producer set as outputs.
> h4. 2. The vectors come back without their data, metadata or distance
> getVectors, queryVectors and the consumer never set returnData,
> returnMetadata or returnDistance on the request (lines 257-261 and 200-205,
> {{AWS2S3VectorsConsumer}} lines 85-89), so each vector holds its key only.
> Checked on 4.24.0-SNAPSHOT with {{camel run}} against the AWS emulator of
> {{camel infra run aws}}, through a proxy that logs the requests: getVectors
> gives {{[GetOutputVector(Key=doc-list)]}}. So the
> {{CamelAwsS3VectorsSimilarityScore}} header is never set, and the
> similarityThreshold option, the {{CamelAwsS3VectorsSimilarityThreshold}} and
> {{CamelAwsS3VectorsMetadataFilter}} headers and the consumerMetadataFilter
> option do nothing (the consumer only logs the filter, lines 92-96).
> h4. 3. A null in a vector or a metadata value that is not a String fails with
> an unclear error
> A vector {{[0.1, null, 0.3]}} fails with {{NullPointerException: Cannot
> invoke "Object.getClass()" because "item" is null}} (line 454), and a
> {{CamelAwsS3VectorsVectorMetadata}} value such as the number 2026 fails with
> {{ClassCastException: class java.lang.Integer cannot be cast to class
> java.lang.String}} (line 504), although S3 Vectors metadata takes numbers and
> booleans. Checked on 4.24.0-SNAPSHOT with {{camel run}} against the AWS
> emulator of {{camel infra run aws}}, through a proxy that logs the requests.
> h4. Also read in the code, not run
> * The dataType option offers float16 and the distanceMetric option
> dot-product (AWS2S3VectorsConfiguration, lines 70-72), which the DataType and
> DistanceMetric enums of the AWS SDK do not have.
> * getVectors and deleteVectors pass a List body as the keys without checking
> its items (line 482): a List of numbers is sent as {{"keys":[1,2]}}.
> * listVectorBuckets and listVectorIndexes give the first page only.
> _Claude Code on behalf of davsclaus_
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
