[
https://issues.apache.org/jira/browse/KAFKA-12305?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chris Egerton updated KAFKA-12305:
----------------------------------
Description:
The {{Flatten}} SMT fails for array types. A sophisticated approach that tries
to flatten arrays might be desirable in some cases, and may have been punted
during the early design phase of the transform, but in the interim, it's
probably not worth it to make array data and the SMT mutually exclusive.
A naive approach that preserves arrays as-are and doesn't attempt to flatten
them seems fair for now, but one alternative could be to traverse array
elements and, if any are maps or structs, flatten those as well.
Adding behavior to fully flatten arrays by essentially transforming them into
maps whose elements are the elements of the array and whose keys are the
indices of each element is likely out of scope for a bug fix and, although
useful, might have to wait for a KIP.
was:
The {{Flatten}} SMT fails for array types. A sophisticated approach that tries
to flatten arrays might be desirable in some cases, and may have been punted
during the early design phase of the transform, but in the interim, it's
probably not worth it to make array data and the SMT mutually exclusive.
A naive approach that preserves arrays as-are and doesn't attempt to flatten
them seems fair for now, but one alternative could be to traverse array
elements and, if any are maps or structs, flatten those as well.
> Flatten SMT fails on arrays
> ---------------------------
>
> Key: KAFKA-12305
> URL: https://issues.apache.org/jira/browse/KAFKA-12305
> Project: Kafka
> Issue Type: Bug
> Components: KafkaConnect
> Affects Versions: 2.0.1, 2.1.1, 2.2.2, 2.3.1, 2.4.1, 2.5.1, 2.7.0, 2.6.1,
> 2.8.0
> Reporter: Chris Egerton
> Assignee: Chris Egerton
> Priority: Major
>
> The {{Flatten}} SMT fails for array types. A sophisticated approach that
> tries to flatten arrays might be desirable in some cases, and may have been
> punted during the early design phase of the transform, but in the interim,
> it's probably not worth it to make array data and the SMT mutually exclusive.
> A naive approach that preserves arrays as-are and doesn't attempt to flatten
> them seems fair for now, but one alternative could be to traverse array
> elements and, if any are maps or structs, flatten those as well.
> Adding behavior to fully flatten arrays by essentially transforming them into
> maps whose elements are the elements of the array and whose keys are the
> indices of each element is likely out of scope for a bug fix and, although
> useful, might have to wait for a KIP.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
