[
https://issues.apache.org/jira/browse/KAFKA-7052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17032325#comment-17032325
]
ASF GitHub Bot commented on KAFKA-7052:
---------------------------------------
gunnarmorling commented on pull request #8059: KAFKA-7052 Adding option to
ExtractField SMT for controlling behavior…
URL: https://github.com/apache/kafka/pull/8059
… in case of non-existent fields
https://issues.apache.org/jira/browse/KAFKA-7052
*More detailed description of your change
n/a
*Summary of testing strategy (including rationale)
Added JUnit test
### Committer Checklist (excluded from commit message)
- [ ] Verify design and implementation
- [ ] Verify test coverage and CI build status
- [ ] Verify documentation (including upgrade notes)
@rhauch, hey, that's a quick first attempt for making the `ExtractField` SMT
more flexible when it comes to encountering a record that doesn't contain the
specified field. It's a common situation for connectors like Debezium which
produce different "kinds" of records/topics; in our case e.g. actual change
event topics and meta-topics such as TX data or schema history. One might want
to apply the `ExtractField` SMT to the CDC records but not to those others.
Also see [KAFKA-7052](https://issues.apache.org/jira/browse/KAFKA-7052) for
some backgrounds.
The proposal is to add a new option `behavior.on.non.existent.field` to the
SMT which makes the behavior configurable. Its supported values are:
* fail: raise an exception (default for records with schema)
* return-null: return null (default for records without schema)
* pass-on: pass on the unmodified original record
I did a quick implementation of that proposal to foster feedback. Happy to
adjust and expand as needed, e.g. to adjust with existing naming patterns for
the option and/or its values as well as docs (not sure where that'd go). Thanks!
CC @rmoff, @big-andy-coates.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> ExtractField SMT throws NPE - needs clearer error message
> ---------------------------------------------------------
>
> Key: KAFKA-7052
> URL: https://issues.apache.org/jira/browse/KAFKA-7052
> Project: Kafka
> Issue Type: Improvement
> Components: KafkaConnect
> Reporter: Robin Moffatt
> Priority: Major
>
> With the following Single Message Transform:
> {code:java}
> "transforms.ExtractId.type":"org.apache.kafka.connect.transforms.ExtractField$Key",
> "transforms.ExtractId.field":"id"{code}
> Kafka Connect errors with :
> {code:java}
> java.lang.NullPointerException
> at
> org.apache.kafka.connect.transforms.ExtractField.apply(ExtractField.java:61)
> at
> org.apache.kafka.connect.runtime.TransformationChain.apply(TransformationChain.java:38){code}
> There should be a better error message here, identifying the reason for the
> NPE.
> Version: Confluent Platform 4.1.1
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
