jinyius commented on PR #988:
URL: https://github.com/apache/parquet-mr/pull/988#issuecomment-1232470935
hmm... what timing. i actually have a pr for what i think is a more robust
approach that truncates at an arbitrary recursion depth by putting the
remaining recursion levels into a binary blob. this approach lets downstream
querying things query the non-truncated parts fine, and allows for udfs to be
defined to reinstantiate the truncated recursed fields.
i didn't submit the pr for merge quite yet b/c i'm busy trying to finish off
the overall project i needed this for at work, so it's just coded against
1.12.3 and not head.
ptal, and if everyone likes my proposal, i can spend a few cycles and move
it to head:
schema converter pr:
- https://github.com/promotedai/parquet-mr/pull/1
write support pr:
- https://github.com/promotedai/parquet-mr/pull/2
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
