[
https://issues.apache.org/jira/browse/PARQUET-7?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14039998#comment-14039998
]
Dmitriy V. Ryaboy commented on PARQUET-7:
-----------------------------------------
playing with the code, starting to think that:
1) my suggested fix won't work because events are accumulated as an individual
struct is read
2) slowness is likely do to the deep stack of protocols for nested structures.
For each record, testThriftOptionalFieldsWithReadProjectionUsingParquetSchema
applies 24 protocols!
3) perhaps the way forward is to look at generating a single protocol that can
apply all of the required transformations, rather than delegating through so
many protocols?
> [parquet-thrift] improve performance of thrift push-down code
> -------------------------------------------------------------
>
> Key: PARQUET-7
> URL: https://issues.apache.org/jira/browse/PARQUET-7
> Project: Parquet
> Issue Type: Wish
> Reporter: Dmitriy V. Ryaboy
>
> A user reported seeing slowness when projection push-down code is active,
> which seems to stem from ProtocolEventsAmender.
> Details can be found in https://github.com/Parquet/parquet-mr/issues/406
--
This message was sent by Atlassian JIRA
(v6.2#6252)
