clintropolis opened a new pull request, #13294:
URL: https://github.com/apache/druid/pull/13294
### Description
This PR fixes a flaw in Parquet list conversion for nullable lists of
nullable complex elements (groups with many fields, etc), which is amusingly
similar to #5433, which I fixed for avro and thought I had addressed for direct
group conversion in #6360, but was missing this case. We did cover the other
common list form conversions, we just weren't unwrapping stuff in this form
```
optional group a2 (LIST) {
repeated group list {
optional group element {
optional int64 b1;
optional int64 b2;
}
}
}
```
which has been added in this PR.
I've also cleaned some stuff up around conversion, and made some changes in
preparation of adding support for Parquet to nested column indexers and
expression transforms, via some of the same flattener/conversion mechanisms we
use for flattenSpec.
#### Release note
Fixes an issue with Parquet list conversion, where lists of complex objects
could unexpectedly be wrapped in an extra object, appearing as
`[{"element":<actual_list_element>},{"element":<another_one>}...]` instead of
the direct list.
<hr>
This PR has:
- [ ] been self-reviewed.
- [x] a release note entry in the PR description.
- [x] added Javadocs for most classes and all non-trivial methods. Linked
related entities via Javadoc links.
- [x] added comments explaining the "why" and the intent of the code
wherever would not be obvious for an unfamiliar reader.
- [x] added unit tests or modified existing tests to cover new code paths,
ensuring the threshold for [code
coverage](https://github.com/apache/druid/blob/master/dev/code-review/code-coverage.md)
is met.
- [x] been tested in a test Druid cluster.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
