davsclaus opened a new pull request, #28343:
URL: https://github.com/apache/camel/pull/28343
Stacked on #28340. This group covers the last eleven AWS components of the
catalog body types (CAMEL-24961): aws-config, aws-parameter-store,
aws-security-hub, aws2-athena, aws2-ecs, aws2-eks, aws2-mq, aws2-msk,
aws2-redshift-data, aws2-s3-vectors and aws2-step-functions. With it, every AWS
component has body types.
**What the catalog says now**
| Component | Consumer | Producer input | Producer output |
|---|---|---|---|
| aws-config | | `any`: the rule and the template are headers | the response
of the SDK |
| aws-parameter-store | | `any`; putParameter takes the value as text from
the body when the CamelAwsParameterStoreValue header is not set | the value as
`text` for getParameter, the response of the SDK for the others |
| aws-security-hub | | `any`: batchImportFindings, the default, takes a List
of `AwsSecurityFinding` or one finding from the header or the body | the
response of the SDK, with the counts of batchImportFindings in headers |
| aws2-athena | | `any`; text, bytes, a stream or a file for
startQueryExecution, the default: the SQL, or no body when the header or the
queryString option has it | no body after startQueryExecution, or the
`GetQueryExecutionResponse` with waitTimeout; for getQueryResults an iterable
of the pages of rows, one page, or the S3 location as `text`, by outputType;
the response of the SDK for the others |
| aws2-ecs, aws2-eks, aws2-mq, aws2-msk | | `any`: the inputs are headers |
the response of the SDK |
| aws2-redshift-data | | `any`: the SQL is a header | the response of the
SDK |
| aws2-s3-vectors | one `QueryOutputVector` per vector found | `any`: the
vector as a List of numbers, a float[] or a double[], or the keys, unless a
header has them | a List for queryVectors, getVectors and the two list
operations, the response of the SDK for the others |
| aws2-step-functions | | `any`: the input of an execution is a header | the
response of the SDK |
`pojoRequest=true` takes the request of the AWS SDK as the body, for every
component that has the option (not athena and s3-vectors).
**aws-parameter-store is not in the catalog.** Its module is missing from
the list of `camel-aws` modules in `MojoHelper` of camel-package-maven-plugin
since the component was added in 4.18.0 (CAMEL-22819). So the catalog, the
Endpoint DSL and Camel CLI do not know the component, and only its own JSON
gets the body types. The fix is CAMEL-26093, in its own PR.
**Checked at runtime.** Routes run with `camel run` logged the class of the
body after each step, with text, bytes, streams, files, Maps, POJOs and no
body. Every call went through a small fake of the AWS API that logs what Camel
sends:
- the fake passes Config, SSM, ECS, EKS, MQ, MSK, S3 Vectors and Step
Functions on to the AWS emulator of `camel infra run aws`;
- it answers Security Hub itself, as the emulator 404s the findings calls;
- it answers Athena and Redshift Data itself, as the emulator cannot run a
query.
The results:
- **aws-config, aws2-ecs, aws2-eks, aws2-mq, aws2-msk, aws2-step-functions:**
- text, a Map and no body are not read, and the body becomes the response
of the SDK;
- a body is never taken as the name or the id of a resource;
- a Map in the input header of startExecution is sent as `{a=1}`.
- **aws-parameter-store:**
- putParameter stores text, bytes, a stream and a file as their text, and
a Map or a POJO as its toString();
- the CamelAwsParameterStoreValue header wins over the body;
- getParameter gives a `String`, and the version as a `Long` header.
- **aws-security-hub:**
- batchImportFindings takes one `AwsSecurityFinding` or a List of them,
from the body or the header;
- text, JSON, a Map, an array and a Set fail with "At least one finding
must be specified".
- **aws2-athena:**
- startQueryExecution sends text, bytes, a stream and a file as the SQL,
and a Map or a POJO as its toString();
- it leaves no body, or the `GetQueryExecutionResponse` with waitTimeout;
- getQueryResults gives a `GetQueryResultsIterable` whose split gives
pages of `GetQueryResultsResponse`, a `GetQueryResultsResponse` with
SelectList, and a `String` with S3Pointer.
- **aws2-redshift-data:** SQL in the body is not sent; executeStatement
needs the CamelAwsRedshiftDataSqlStatement header.
- **aws2-s3-vectors:**
- putVectors and queryVectors take a List of numbers, a float[] or a
double[], and JSON text fails until it is unmarshalled;
- the List of queryVectors and getVectors holds the keys only;
- the consumer sends one `QueryOutputVector` per vector found.
**Choices**
- Where a header can stand in for the body, the input side is `any`: the
value of putParameter, the findings of batchImportFindings and the vectors of
s3-vectors.
- `pojoRequest` is named only where the input side is `any`, as in the
previous AWS groups.
- For aws2-athena, the operation option picks the sides, as waitTimeout and
outputType take any value. The output of startQueryExecution is "no body or a
POJO", and that of getQueryResults "an iterator, a POJO or text"; neither makes
the validator report anything.
- aws2-redshift-data: the descriptions are short, so that camel_catalog_doc
still lists its 20 options in full.
**Bugs found and filed, one per component:**
- CAMEL-26091 (Major), aws2-redshift-data: the producer never uses the
client the endpoint builds, so with accessKey and secretKey every operation
fails with a NullPointerException ("redshiftDataClient is null"); only a client
given as an option or a bean works. Also: no operation fails with a
NullPointerException; a CamelAwsRedshiftDataSqlParameterList of Strings fails
with a ClassCastException.
- CAMEL-26092 (Major), aws2-s3-vectors: the headers an earlier step sets win
over the endpoint:
- a `putVectors` to another bucket after the consumer wrote over each
vector just consumed, in the consumed bucket and under the consumed key;
- a `createVectorBucket` created the bucket of the step before;
- `getVectors` after `putVectors` ignored its body.
Also: getVectors and queryVectors never ask for the data, the metadata or
the distance; similarityThreshold, the SimilarityThreshold and MetadataFilter
headers and consumerMetadataFilter do nothing; a null in a vector fails with a
NullPointerException; a metadata value that is not a String fails with a
ClassCastException.
- CAMEL-26093 (Major), aws-parameter-store: not in the catalog (above).
Also: no operation fails with a NullPointerException; getParameters splits the
names without trimming them; the nine YAML examples of the page put `steps`
beside `from`.
- CAMEL-26094, aws-security-hub:
- a CamelAwsSecurityHubFilters header as JSON text is dropped, so
getFindings returns every finding;
- the CamelAwsSecurityHubNextToken of getFindings is sent by a later
listEnabledProductsForImport;
- an array or a Set of findings in the body fails, while the header takes
them;
- a List of anything else fails with a ClassCastException;
- the page misses getFindingAggregator.
- CAMEL-26095, aws2-mq:
- no operation fails with a NullPointerException;
- CamelAwsMQBrokerUsers as Strings fails with a ClassCastException;
- a CamelAwsMQConfigurationID as text sends updateBroker without the
configuration;
- with pojoRequest, describeBroker and createBroker set no headers;
- the component is described as "Send messages to AWS MQ".
- CAMEL-26096, aws2-msk:
- no operation fails with a NullPointerException;
- CamelAwsMSKBrokerNodesGroupInfo as text is dropped;
- with pojoRequest, describeCluster and createCluster set no headers;
- the deleteCluster example of the page sets the headers of createCluster.
- CAMEL-26097, aws2-eks:
- no operation fails with a NullPointerException;
- CamelAwsEKSVPCConfig as text is dropped;
- the CamelAwsEKSDescription header is never read.
- CAMEL-26098, aws2-ecs: no operation fails with a NullPointerException;
with pojoRequest, an empty body is reported as "null" (as in aws2-eks).
- CAMEL-26099, aws-config: no operation fails with a NullPointerException;
putConformancePack without a name says "Rule Name must be specified"; the page
speaks of "AWS ECS ... clusters instances".
- CAMEL-26100, aws2-athena:
- a second startQueryExecution in a route takes the result file of the
first as its output location, from the CamelAwsAthenaOutputLocation header;
- the next-token header is shared by listQueryExecutions and
getQueryResults;
- no null check before reading the output location of an answer.
- CAMEL-26101, aws2-step-functions:
- no operation fails with a NullPointerException;
- startSyncExecution with uriEndpointOverride fails, as the AWS SDK
prefixes the host with `sync-`;
- the operation option lists the Lambda operations.
**Docs fixed**
- **aws2-athena:** the body after getQueryExecution is a
`GetQueryExecutionResponse` and after listQueryExecutions a
`ListQueryExecutionsResponse`; S3Pointer gives the S3 location of the results;
startQueryExecution leaves no body unless it waits, and a query that fails does
not fail the exchange.
**Measured.**
- The validator over the 1204 human-written and 1209 model-written route
files gives the same reports as with #28340.
- Body types still cover 98.1% of the endpoint uses in the human-written
routes; these eleven have no uses there. 160 of the 413 component JSONs of the
catalog have them.
**Tests**
- `TextBodyFlowTest.testTheBodyOfAwsDataAndAdminServices`:
- reported: a Map sent to startQueryExecution of aws2-athena, and the List
of queryVectors of aws2-s3-vectors sent to aws2-translate;
- not reported: a Map sent to aws2-ecs, aws2-eks, aws2-mq, aws2-msk,
aws-config, aws-security-hub, aws2-step-functions, aws2-redshift-data and
putVectors of aws2-s3-vectors, and to aws2-athena without an operation.
- These pass:
- the 295 tests of camel-yaml-dsl-validator;
- the 265 validator and catalog doc tests of camel-jbang-core;
- the builds of camel-yaml-dsl-maven-plugin and
camel-yaml-dsl-deserializers.
- The answers of camel_catalog_doc stay under 18 000 characters:
aws2-redshift-data, the largest, is 17 953.
_Claude Code on behalf of davsclaus_
🤖 Generated with [Claude Code](https://claude.com/claude-code)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
