[ https://issues.apache.org/jira/browse/IMPALA-9841?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Tim Armstrong updated IMPALA-9841: ---------------------------------- Component/s: Catalog > Table level writeIdList represent the filemetadata snapshot > ----------------------------------------------------------- > > Key: IMPALA-9841 > URL: https://issues.apache.org/jira/browse/IMPALA-9841 > Project: IMPALA > Issue Type: Improvement > Components: Catalog > Reporter: Vihang Karajgaonkar > Assignee: Vihang Karajgaonkar > Priority: Major > > Currently, in HdfsTable > https://github.com/apache/impala/blob/master/fe/src/main/java/org/apache/impala/catalog/HdfsTable.java#L1017 > we reload the ValidWriteIdList every time. However in case of incremental > refresh, it is possible that the file-metadata is not loaded at all or only > loaded for certain partitions. This would mean that the cached > ValidWriteIdList of the table is out-of-sync with the actual file-metadata > which is cached in catalog. > This is currently not a problem since we on rely on the ValidWriteIdList > during the loading of the file-metadata. However, for a consistent reads of > the transactional tables, we would need to make sure that this > ValidWriteIdList is guaranteed to represent the cached files. -- This message was sent by Atlassian Jira (v8.3.4#803005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org