Paul Rogers has posted comments on this change. ( http://gerrit.cloudera.org:8080/11688 )
Change subject: IMPALA-7501: Slim down metastore Partition objects in LocalCatalog ...................................................................... Patch Set 2: (2 comments) Updated with changes from code review comments. http://gerrit.cloudera.org:8080/#/c/11688/1/fe/src/main/java/org/apache/impala/catalog/local/CatalogdMetaProvider.java File fe/src/main/java/org/apache/impala/catalog/local/CatalogdMetaProvider.java: http://gerrit.cloudera.org:8080/#/c/11688/1/fe/src/main/java/org/apache/impala/catalog/local/CatalogdMetaProvider.java@1169 PS1, Line 1169: // The per-partition schema information is not used by Impala. > good idea, that would reduce network bandwidth noticeably considering we do Good idea. In looking into this, the catalogd caches Impala-specific objects, then recreates an HMS Partition object, passing in the table schema when creating the StorageDescriptor(). The fix is to simply not add the schema. When running in local catalog mode, we hit this line, with the original Partition object, so we still need this line. There was discussion on some list or other about the idea of caching the Impala catalog objects rather than the HMS objects. The Impala objects appear to be stripped down. By caching HMS objects, we will find ourselves going through the same process that likely led to those light-weight Impala objects. http://gerrit.cloudera.org:8080/#/c/11688/1/fe/src/main/java/org/apache/impala/catalog/local/LocalFsPartition.java File fe/src/main/java/org/apache/impala/catalog/local/LocalFsPartition.java: http://gerrit.cloudera.org:8080/#/c/11688/1/fe/src/main/java/org/apache/impala/catalog/local/LocalFsPartition.java@71 PS1, Line 71: > checkArgument Done -- To view, visit http://gerrit.cloudera.org:8080/11688 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I4c861452cf974970ab511406fe2e175ea3dc668d Gerrit-Change-Number: 11688 Gerrit-PatchSet: 2 Gerrit-Owner: Paul Rogers <par0...@yahoo.com> Gerrit-Reviewer: Bharath Vissapragada <bhara...@cloudera.com> Gerrit-Reviewer: Paul Rogers <par0...@yahoo.com> Gerrit-Reviewer: Todd Lipcon <t...@apache.org> Gerrit-Comment-Date: Fri, 26 Oct 2018 22:33:23 +0000 Gerrit-HasComments: Yes
