Zoltan Borok-Nagy has posted comments on this change. ( http://gerrit.cloudera.org:8080/17075 )
Change subject: IMPALA-10494: Making use of the min/max column stats to improve min/max filters ...................................................................... Patch Set 19: (2 comments) http://gerrit.cloudera.org:8080/#/c/17075/19//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/17075/19//COMMIT_MSG@19 PS19, Line 19: show_column_minmax_stats > Did we arrive at a consensus on this ? Adding new query options means we h I guess we have quite a bunch of .test files that are executed against all kinds of file formats, and the tests require to have the same result data in each case. The other option is to calculate min/max for every file format. Since we only do it for numeric values I don't think it would add too much overhead. We couldn't use them during query execution (only for Parquet and later ORC?), but might be still useful for a user if they want to see some basic stats of a table. http://gerrit.cloudera.org:8080/#/c/17075/27/fe/src/main/java/org/apache/impala/catalog/HdfsFileFormat.java File fe/src/main/java/org/apache/impala/catalog/HdfsFileFormat.java: http://gerrit.cloudera.org:8080/#/c/17075/27/fe/src/main/java/org/apache/impala/catalog/HdfsFileFormat.java@279 PS27, Line 279: public boolean isParquetBased() { > nit: There's an identical method in HdfsScanNode.java which is currently pr Yeah, HUDI_PARQUET means Hudi table format + Parquet data files, therefore the min/max filters will work fine on them. -- To view, visit http://gerrit.cloudera.org:8080/17075 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I08581b44419bb8da5940cbf98502132acd1c86df Gerrit-Change-Number: 17075 Gerrit-PatchSet: 19 Gerrit-Owner: Qifan Chen <[email protected]> Gerrit-Reviewer: Aman Sinha <[email protected]> Gerrit-Reviewer: Impala Public Jenkins <[email protected]> Gerrit-Reviewer: Qifan Chen <[email protected]> Gerrit-Reviewer: Zoltan Borok-Nagy <[email protected]> Gerrit-Comment-Date: Thu, 25 Mar 2021 16:26:07 +0000 Gerrit-HasComments: Yes
