[
https://issues.apache.org/jira/browse/PARQUET-1474?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16720333#comment-16720333
]
ASF GitHub Bot commented on PARQUET-1474:
-----------------------------------------
gszadovszky closed pull request #563: PARQUET-1474: Less verbose and lower
level logging for missing column/offset indexes
URL: https://github.com/apache/parquet-mr/pull/563
This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:
As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):
diff --git
a/parquet-column/src/main/java/org/apache/parquet/internal/filter2/columnindex/ColumnIndexFilter.java
b/parquet-column/src/main/java/org/apache/parquet/internal/filter2/columnindex/ColumnIndexFilter.java
index fb3077e87..f3efff7ba 100644
---
a/parquet-column/src/main/java/org/apache/parquet/internal/filter2/columnindex/ColumnIndexFilter.java
+++
b/parquet-column/src/main/java/org/apache/parquet/internal/filter2/columnindex/ColumnIndexFilter.java
@@ -85,7 +85,7 @@ public RowRanges visit(FilterPredicateCompat
filterPredicateCompat) {
return filterPredicateCompat.getFilterPredicate()
.accept(new ColumnIndexFilter(columnIndexStore, paths,
rowCount));
} catch (MissingOffsetIndexException e) {
- LOGGER.warn("Unable to do filtering", e);
+ LOGGER.info(e.getMessage());
return RowRanges.createSingle(rowCount);
}
}
@@ -169,7 +169,7 @@ private RowRanges applyPredicate(Column<?> column,
Function<ColumnIndex, Primiti
OffsetIndex oi = columnIndexStore.getOffsetIndex(columnPath);
ColumnIndex ci = columnIndexStore.getColumnIndex(columnPath);
if (ci == null) {
- LOGGER.warn("No column index for column {} is available; Unable to
filter on this column", columnPath);
+ LOGGER.info("No column index for column {} is available; Unable to
filter on this column", columnPath);
return allRows();
}
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
> Less verbose and lower level logging for missing column/offset indexes
> ----------------------------------------------------------------------
>
> Key: PARQUET-1474
> URL: https://issues.apache.org/jira/browse/PARQUET-1474
> Project: Parquet
> Issue Type: Improvement
> Reporter: Gabor Szadovszky
> Priority: Major
> Labels: pull-request-available
>
> Currently, exception stacktrace is logged at warn level if an offset index is
> missing. Also a warn level log happens if a column index is missing which is
> required for column-index based filtering. Both cases are properly valid
> scenarios if the file is written by older libraries (where no column/offset
> indexes are written at all) or the sorting order is undefined for the related
> column type (e.g. INT96).
> These logs shall be kept at INFO level and no stacktrace shall be provided.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
