Zoltan Borok-Nagy has posted comments on this change. ( http://gerrit.cloudera.org:8080/16697 )
Change subject: IMPALA-10310 Fix couldn't skip rows in parquet file on NextRowGroup ...................................................................... Patch Set 1: (1 comment) Thank you very much for fixing this. It would be nice to have an automated test for it as well. I can happily write that test in a follow-up commit, but I can also help you to create such test in the context of this change request. Maybe the easiest way is to check-in a Parquet file that has multiple row groups, but it's only a couple hundred kilobytes. To create such file you can take a look at this example: (Alternatively you can create the table and write the file during the test, using the Hive client) Then add a test to https://github.com/apache/impala/blob/master/testdata/workloads/functional-query/queries/QueryTest/parquet-page-index.test This change request provides a complete example of how to do that: https://gerrit.cloudera.org/#/c/16503/ But as i said I happily write all these if you don't want to bother with it. http://gerrit.cloudera.org:8080/#/c/16697/1/be/src/exec/parquet/parquet-column-readers.cc File be/src/exec/parquet/parquet-column-readers.cc: http://gerrit.cloudera.org:8080/#/c/16697/1/be/src/exec/parquet/parquet-column-readers.cc@1058 PS1, Line 1058: current_row_range_ = 0; This member is only used during page filtering, so maybe you could move this line to ResetPageFiltering(). -- To view, visit http://gerrit.cloudera.org:8080/16697 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I964695cd53f5d5fdb6485a85cd82e7a72ca6092c Gerrit-Change-Number: 16697 Gerrit-PatchSet: 1 Gerrit-Owner: guojingfeng <[email protected]> Gerrit-Reviewer: Impala Public Jenkins <[email protected]> Gerrit-Reviewer: Tim Armstrong <[email protected]> Gerrit-Reviewer: Zoltan Borok-Nagy <[email protected]> Gerrit-Reviewer: guojingfeng <[email protected]> Gerrit-Comment-Date: Mon, 09 Nov 2020 10:33:59 +0000 Gerrit-HasComments: Yes
