oscerd opened a new pull request, #1863:
URL: https://github.com/apache/camel-kafka-connector/pull/1863
Fixes #1861.
### Problem
`SchemaHelper.buildSchemaBuilderForType` always gives a `Map` or `List`
value a `MAP` / `ARRAY` schema with no key or value schema:
- When `Values.inferSchema` cannot type the entries, as with a JSON object
whose values have different types, it falls back to `new
SchemaBuilder(Schema.Type.MAP)`.
- When inference succeeds, `SchemaUtil.copySchemaBasics(schema)` copies the
inferred schema into a new builder and drops the key and value schemas.
`JsonConverter` then fails, with either `schemas.enable` setting:
```
java.lang.NullPointerException: Cannot invoke
"org.apache.kafka.connect.data.Schema.type()" because the return value of
"org.apache.kafka.connect.data.Schema.keySchema()" is null
```
This hits any source connector whose route ends with a `Map` or `List` body,
for example:
- form-encoded requests to the webhook source (raised in the review of
#1859);
- `camel.source.unmarshal=jackson` without a `marshal` back.
### Change
`CamelSourceTask` gives a `Map` or `List` key or value the schema
`Values.inferSchema` returns. That schema is complete, with key and value
schemas, or `null` when the types can't be inferred. Kafka Connect supports
schemaless values, and `JsonConverter` writes them as the JSON object or array.
Other types still get their schema from `SchemaHelper`.
`SchemaHelper` itself is unchanged, because it is public and the SMTs use
it. The fix stays in the source task, where the issue is.
### Tests
- `CamelSourceTaskTest#testSourcePollingWithMapAndListBody` covers:
- a mixed-type map and a mixed-type list, which are sent schemaless;
- a `Map<String, String>`, which gets a complete `MAP(STRING, STRING)`
schema;
- all three serialized with `JsonConverter` (`schemas.enable=false`). The
test fails on `main`.
- `./mvnw -pl core -Psourcecheck install` runs 128 tests with 0 failures.
The full reactor build `./mvnw clean verify -DskipTests` succeeds.
_Claude Code on behalf of Andrea Cosentino_
🤖 Generated with [Claude Code](https://claude.com/claude-code)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
