[
https://issues.apache.org/jira/browse/PARQUET-409?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16515945#comment-16515945
]
ASF GitHub Bot commented on PARQUET-409:
----------------------------------------
rgruener commented on a change in pull request #495: PARQUET-409: Add a
configuration key that controls min/max row count for block size check
URL: https://github.com/apache/parquet-mr/pull/495#discussion_r196134461
##########
File path:
parquet-column/src/main/java/org/apache/parquet/column/ParquetProperties.java
##########
@@ -45,8 +45,10 @@
public static final boolean DEFAULT_IS_DICTIONARY_ENABLED = true;
public static final WriterVersion DEFAULT_WRITER_VERSION =
WriterVersion.PARQUET_1_0;
public static final boolean DEFAULT_ESTIMATE_ROW_COUNT_FOR_PAGE_SIZE_CHECK =
true;
- public static final int DEFAULT_MINIMUM_RECORD_COUNT_FOR_CHECK = 100;
Review comment:
Totally forgot about that, fixed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
> InternalParquetRecordWriter doesn't use min/max row counts
> ----------------------------------------------------------
>
> Key: PARQUET-409
> URL: https://issues.apache.org/jira/browse/PARQUET-409
> Project: Parquet
> Issue Type: Bug
> Components: parquet-mr
> Affects Versions: 1.8.1
> Reporter: Ryan Blue
> Priority: Major
> Fix For: 1.9.0
>
>
> PARQUET-99 added settings to control the min and max number of rows between
> size checks when flushing pages, and a setting to control whether to always
> use a static size (the min). The [InternalParquetRecordWriter has similar
> checks|https://github.com/apache/parquet-mr/blob/master/parquet-hadoop/src/main/java/org/apache/parquet/hadoop/InternalParquetRecordWriter.java#L143]
> that don't use those settings. We should determine if it should update it to
> use those settings or similar.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
