Lars Volker has posted comments on this change. ( http://gerrit.cloudera.org:8080/11355 )
Change subject: IMPALA-5937: [DOCS] PARQUET_READ_STATISTICS and PARQUET_DICTIONARY_FILTERING ...................................................................... Patch Set 4: (3 comments) http://gerrit.cloudera.org:8080/#/c/11355/4/docs/topics/impala_parquet_dictionary_filtering.xml File docs/topics/impala_parquet_dictionary_filtering.xml: http://gerrit.cloudera.org:8080/#/c/11355/4/docs/topics/impala_parquet_dictionary_filtering.xml@87 PS4, Line 87: The <codeph>NumDictFilteredRowGroups</codeph> field in the query profile output shows the We should mention somewhere that row groups can be filtered out by parquet stats, and in such cases won't even be considered for dictionary filtering. http://gerrit.cloudera.org:8080/#/c/11355/4/docs/topics/impala_parquet_read_statistics.xml File docs/topics/impala_parquet_read_statistics.xml: http://gerrit.cloudera.org:8080/#/c/11355/4/docs/topics/impala_parquet_read_statistics.xml@49 PS4, Line 49: Parquet stores min/max stats which can be used to skip reading blocks if they don't The stats are (currently) used to skip row groups. "qualify a predicate" sounds unfamiliar to me. This blog post has a more detailed description: https://blog.cloudera.com/blog/2017/12/faster-performance-for-selective-queries/ http://gerrit.cloudera.org:8080/#/c/11355/4/docs/topics/impala_parquet_read_statistics.xml@61 PS4, Line 61: Of the numerical types: bool, integer, floating point Impala also supports other data types, but there are restrictions around the library and version that wrote them. I'll share some more docs with you momentarily. -- To view, visit http://gerrit.cloudera.org:8080/11355 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I88fa8c4a64560711251076c50e1695f7f032f9c0 Gerrit-Change-Number: 11355 Gerrit-PatchSet: 4 Gerrit-Owner: Alex Rodoni <[email protected]> Gerrit-Reviewer: Alex Rodoni <[email protected]> Gerrit-Reviewer: Fredy Wijaya <[email protected]> Gerrit-Reviewer: Impala Public Jenkins <[email protected]> Gerrit-Reviewer: Joe McDonnell <[email protected]> Gerrit-Reviewer: Lars Volker <[email protected]> Gerrit-Comment-Date: Thu, 06 Sep 2018 00:11:28 +0000 Gerrit-HasComments: Yes
