[
https://issues.apache.org/jira/browse/PARQUET-1826?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17089075#comment-17089075
]
ASF GitHub Bot commented on PARQUET-1826:
-----------------------------------------
garawalid commented on a change in pull request #781:
URL: https://github.com/apache/parquet-mr/pull/781#discussion_r412495171
##########
File path: parquet-hadoop/README.md
##########
@@ -230,23 +236,28 @@ conf.set("parquet.bloom.filter.expected.ndv#column.path",
200)
## Class: ParquetInputFormat
**Property:** `parquet.read.support.class`
-**Description:** The read support class.
+**Description:** The read support class that is used in
+ParquetInputFormat to materialize records. It should be a the descendant class
of `org.apache.parquet.hadoop.api.ReadSupport`
---
**Property:** `parquet.read.filter`
-**Description:** **Todo**
+**Description:** The filter class name that implements
`org.apache.parquet.filter.UnboundRecordFilter`. This class is for the old
filter API in the package `org.apache.parquet.filter`, it filters records
during record assembly.
---
-**Property:** `parquet.strict.typing`
-**Description:** Whether to enable type checking for conflicting schema.
-**Default value:** `true`
+ **Property:** `parquet.private.read.filter.predicate`
+ **Description:** The filter class used in the new filter API in the package
`org.apache.parquet.filter2.predicate`
+ Note that this class should implements
`org.apache.parquet.filter2..FilterPredicate` and the value of this property
should be a gzip compressed base64 encoded java serialized object.
+ The new filter API can filter records or filter entire row groups of records
without reading them at all.
+
+**Note:** User should either use the old filter API (`parquet.read.filter`) or
the new one (`parquet.private.read.filter.predicate`).
Review comment:
I agree!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
> Document hadoop configuration options
> -------------------------------------
>
> Key: PARQUET-1826
> URL: https://issues.apache.org/jira/browse/PARQUET-1826
> Project: Parquet
> Issue Type: Improvement
> Components: parquet-mr
> Reporter: Gabor Szadovszky
> Assignee: Walid Gara
> Priority: Major
> Labels: pull-request-available
>
> The currently available hadoop configuration options is not documented
> properly. The only documentation we have is the javadoc comment and the
> implementation ofÂ
> [ParquetOutputFormat|https://github.com/apache/parquet-mr/blob/master/parquet-hadoop/src/main/java/org/apache/parquet/hadoop/ParquetOutputFormat.java].
> We shall investigate all the possible options and their usage/default values
> and document them properly in a way that it is easily accessible by our users.
> I would suggest creating a `README.md` file in the sub-module
> [parquet-hadoop|https://github.com/apache/parquet-mr/tree/master/parquet-hadoop]
> that would describe the purpose of the module and would have a section that
> lists the possible hadoop configuration options. (Later on we shall extend
> this document with other descriptions about the purpose and usage of our
> library in the hadoop ecosystem. These efforts shall be covered by other
> jiras.)
> By adding the description to the source code it would be easy to extend it by
> the new features we implement so it will be up-to-date for every release.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
