Daniel Becker has posted comments on this change. ( http://gerrit.cloudera.org:8080/21605 )
Change subject: IMPALA-13247: Support Reading Puffin files for the current snapshot ...................................................................... Patch Set 13: (2 comments) http://gerrit.cloudera.org:8080/#/c/21605/12//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/21605/12//COMMIT_MSG@20 PS12, Line 20: 'disable_reading_puffin_stats' startup flag - when it is set to true, : Impala will never read Puffin stats > I think a query option would be more convenient. We usually use query optio As discussed offline, I kept the startup flag and added a table property also. If another writer writes incorrect stats for a specific table, reading them can be disabled with the table property. If many tables are affected, we can use the startup flag to disable reading Puffin globally. Query options are unfortunately not available because the code runs in catalogd, and not necessarily during a query. http://gerrit.cloudera.org:8080/#/c/21605/6/fe/src/main/java/org/apache/impala/catalog/IcebergTable.java File fe/src/main/java/org/apache/impala/catalog/IcebergTable.java: http://gerrit.cloudera.org:8080/#/c/21605/6/fe/src/main/java/org/apache/impala/catalog/IcebergTable.java@471 PS6, Line 471: r (Map.Entry<Integer, PuffinStatsLoader.P > However, I tried setting this table property, then setting NDV > manually (as an HMS stat), and that is not capped. In this case I > think it would be strange to cap the Puffin NDV value while not > doing the same with the HMS NDV value. Probably we should leave it > as it is in this patch and maybe open a Jira to cap the values in > both cases. What do you think? @Riza Suminto what do you think of this? -- To view, visit http://gerrit.cloudera.org:8080/21605 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I50c1228988960a686d08a9b2942e01e366678866 Gerrit-Change-Number: 21605 Gerrit-PatchSet: 13 Gerrit-Owner: Daniel Becker <daniel.bec...@cloudera.com> Gerrit-Reviewer: Csaba Ringhofer <csringho...@cloudera.com> Gerrit-Reviewer: Daniel Becker <daniel.bec...@cloudera.com> Gerrit-Reviewer: Gabor Kaszab <gaborkas...@cloudera.com> Gerrit-Reviewer: Impala Public Jenkins <impala-public-jenk...@cloudera.com> Gerrit-Reviewer: Noemi Pap-Takacs <npaptak...@cloudera.com> Gerrit-Reviewer: Peter Rozsa <pro...@cloudera.com> Gerrit-Reviewer: Riza Suminto <riza.sumi...@cloudera.com> Gerrit-Reviewer: Zoltan Borok-Nagy <borokna...@cloudera.com> Gerrit-Comment-Date: Mon, 23 Sep 2024 13:35:20 +0000 Gerrit-HasComments: Yes
