Zoltan Borok-Nagy has posted comments on this change. ( http://gerrit.cloudera.org:8080/16008 )
Change subject: IMPALA-9791: Support validWriteIdList in getPartialCatalogObject API ...................................................................... Patch Set 2: (2 comments) http://gerrit.cloudera.org:8080/#/c/16008/2/fe/src/main/java/org/apache/impala/catalog/CatalogServiceCatalog.java File fe/src/main/java/org/apache/impala/catalog/CatalogServiceCatalog.java: http://gerrit.cloudera.org:8080/#/c/16008/2/fe/src/main/java/org/apache/impala/catalog/CatalogServiceCatalog.java@3026 PS2, Line 3026: if (req.table_info_selector.valid_write_ids != null) { > hmm. My understanding is that in Impala Materialized views are treated as v Yeah, sorry, you are right. http://gerrit.cloudera.org:8080/#/c/16008/2/fe/src/main/java/org/apache/impala/util/AcidUtils.java File fe/src/main/java/org/apache/impala/util/AcidUtils.java: http://gerrit.cloudera.org:8080/#/c/16008/2/fe/src/main/java/org/apache/impala/util/AcidUtils.java@324 PS2, Line 324: filterFDsforAcidState > Thanks Zoltan for the feedback. Is it fair to say that in case of base file Yeah, I think what you wrote is correct, and yes, we can fetch the latest validTxnIdList during the fallback code. However, delta directories created by Hive Streaming should be checked in a non-strict way. Such delta directories also contain files corresponding to a write id range, but they don't have visibilityTxnId, so e.g. delta_0001_0020. Now my only concern is the Hive cleaner thread which deletes old transactional directories. Given that maybe it would be better to force the client to refresh its table metadata when we cannot serve the file descriptors from cache? Or we don't expect that the clients' lag to be too big, so the necessary files should still exist? -- To view, visit http://gerrit.cloudera.org:8080/16008 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ied2c7c3cb2009c407e8fbc3af4722b0d34f57c4a Gerrit-Change-Number: 16008 Gerrit-PatchSet: 2 Gerrit-Owner: Vihang Karajgaonkar <vih...@cloudera.com> Gerrit-Reviewer: Aman Sinha <amsi...@cloudera.com> Gerrit-Reviewer: Anurag Mantripragada <anu...@cloudera.com> Gerrit-Reviewer: Csaba Ringhofer <csringho...@cloudera.com> Gerrit-Reviewer: Impala Public Jenkins <impala-public-jenk...@cloudera.com> Gerrit-Reviewer: Quanlong Huang <huangquanl...@gmail.com> Gerrit-Reviewer: Vihang Karajgaonkar <vih...@cloudera.com> Gerrit-Reviewer: Zoltan Borok-Nagy <borokna...@cloudera.com> Gerrit-Comment-Date: Wed, 03 Jun 2020 14:09:06 +0000 Gerrit-HasComments: Yes
