[
https://issues.apache.org/jira/browse/BEAM-11913?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kenneth Knowles updated BEAM-11913:
-----------------------------------
Status: Open (was: Triage Needed)
> Add support for Hadoop configuration on ParquetIO
> -------------------------------------------------
>
> Key: BEAM-11913
> URL: https://issues.apache.org/jira/browse/BEAM-11913
> Project: Beam
> Issue Type: Improvement
> Components: io-java-parquet
> Reporter: Ismaël Mejía
> Assignee: Ismaël Mejía
> Priority: P2
>
> We have discussed this issue in the past and we tried to avoid Hadoop objects
> in Parquet public API however there are two valid reasons for this:
> 1. Many functionalities of Parquet are configurable via public helper methods
> on Parquet that prepare data inside of Hadoop's Configuration object, e.g.
> Column Projection via
> `{color:#000000}AvroReadSupport{color}.setRequestedProjection({color:#871094}conf{color},
> {color:#871094}projectionSchema{color});` or Predicate Filters via
> `P{color:#000000}arquetInputFormat{color}.setFilterPredicate({color:#871094}sc{color}.hadoopConfiguration(),
> {color:#871094}filterPredicate{color});`. Giving access to those would allow
> power users to do advanced stuff without any maintenance on the IO side.
> 2. The main reason to avoid the Hadoop Configuration object was to align with
> future non Hadoop required APIs on Parquet see PARQUET-1126 for details but
> this does not seem that will happen soon.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
