Todd Lipcon has posted comments on this change. ( http://gerrit.cloudera.org:8080/11182 )
Change subject: IMPALA-7436: initial fetch-from-catalogd implementation ...................................................................... Patch Set 5: (10 comments) http://gerrit.cloudera.org:8080/#/c/11182/5/be/src/service/fe-support.cc File be/src/service/fe-support.cc: http://gerrit.cloudera.org:8080/#/c/11182/5/be/src/service/fe-support.cc@536 PS5, Line 536: NULL > nit: use nullptr for these to be consistent within the function. ideally we Done http://gerrit.cloudera.org:8080/#/c/11182/5/common/thrift/CatalogService.thrift File common/thrift/CatalogService.thrift: http://gerrit.cloudera.org:8080/#/c/11182/5/common/thrift/CatalogService.thrift@283 PS5, Line 283: partitions > is this guaranteed to have info for each partition that was requested? if n yep, guaranteed. It would have thrown an NPE if you request a missing one. Added a check on the server side for this and will document in the thrift file. I also rejiggered the API for requesting partition info a bit so that "partition list" is just the same as "ask for the names of all partitions" instead of being handled differently. http://gerrit.cloudera.org:8080/#/c/11182/5/common/thrift/CatalogService.thrift@311 PS5, Line 311: 2: optional TCatalogServiceRequestHeader header > heh, I omitted these for the incremental stats pull and nothing complained. yea, this seems unused, i'll just remove it from this RPC. http://gerrit.cloudera.org:8080/#/c/11182/5/common/thrift/CatalogService.thrift@324 PS5, Line 324: // The status of the operation, OK if the operation was successful. > nit: indentation off Done http://gerrit.cloudera.org:8080/#/c/11182/5/common/thrift/CatalogService.thrift@325 PS5, Line 325: optional > make it required if it'll always be set? I think I went with not setting it I fiddled with this a bit but it ends up being a little messy since we have several places we construct these response objects on the Java side. I think a convention of "no status means OK" is fairly normal. I'll add a comment indicating it. http://gerrit.cloudera.org:8080/#/c/11182/5/common/thrift/CatalogService.thrift@417 PS5, Line 417: TGetPartialCatalogObjectResponse GetPartialCatalogObject( > pls add a comment Done http://gerrit.cloudera.org:8080/#/c/11182/5/fe/src/main/java/org/apache/impala/catalog/CatalogServiceCatalog.java File fe/src/main/java/org/apache/impala/catalog/CatalogServiceCatalog.java: http://gerrit.cloudera.org:8080/#/c/11182/5/fe/src/main/java/org/apache/impala/catalog/CatalogServiceCatalog.java@1861 PS5, Line 1861: table > I had a bit of trouble with this one for pulling incremental stats. For exa yea I was a bit confused by that comment as well. From what I can tell, it can return an IncompleteTable with a bad "cause". I'll try to add a test case that provokes this. http://gerrit.cloudera.org:8080/#/c/11182/5/fe/src/main/java/org/apache/impala/catalog/HdfsPartition.java File fe/src/main/java/org/apache/impala/catalog/HdfsPartition.java: http://gerrit.cloudera.org:8080/#/c/11182/5/fe/src/main/java/org/apache/impala/catalog/HdfsPartition.java@100 PS5, Line 100: * index 'dstIndex' instead of the original index 'origIndex'. > perhaps add why this is needed. there's a bit comment at the call site. Is that sufficient? Didn't want to duplicate the explanation. http://gerrit.cloudera.org:8080/#/c/11182/5/fe/src/main/java/org/apache/impala/catalog/Table.java File fe/src/main/java/org/apache/impala/catalog/Table.java: http://gerrit.cloudera.org:8080/#/c/11182/5/fe/src/main/java/org/apache/impala/catalog/Table.java@381 PS5, Line 381: public TGetPartialCatalogObjectResponse getPartialInfo( > pls add a comment for this one stating that its only called on the catalog, Done http://gerrit.cloudera.org:8080/#/c/11182/5/fe/src/main/java/org/apache/impala/service/Frontend.java File fe/src/main/java/org/apache/impala/service/Frontend.java: http://gerrit.cloudera.org:8080/#/c/11182/5/fe/src/main/java/org/apache/impala/service/Frontend.java@1051 PS5, Line 1051: new EventSequence("Query Compilation"); > agreed. there's enough going on here so a todo works for me. k, removed from this patch. -- To view, visit http://gerrit.cloudera.org:8080/11182 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: If49207fc592b1cc552fbcc7199568b6833f86901 Gerrit-Change-Number: 11182 Gerrit-PatchSet: 5 Gerrit-Owner: Todd Lipcon <[email protected]> Gerrit-Reviewer: Bharath Vissapragada <[email protected]> Gerrit-Reviewer: Impala Public Jenkins <[email protected]> Gerrit-Reviewer: Tianyi Wang <[email protected]> Gerrit-Reviewer: Todd Lipcon <[email protected]> Gerrit-Reviewer: Vuk Ercegovac <[email protected]> Gerrit-Comment-Date: Wed, 15 Aug 2018 23:22:08 +0000 Gerrit-HasComments: Yes
