Dimitris Tsirogiannis has posted comments on this change. Change subject: IMPALA-3499: Frontend cannot deserialize catalog-update topic more than 2GB. ......................................................................
Patch Set 2: (9 comments) http://gerrit.cloudera.org:8080/#/c/3067/2/be/src/service/impala-server.cc File be/src/service/impala-server.cc: Line 196: CATALOG_UPDATE_LIMIT Let's rename it to MAX_CATALOG_CACHE_UPDATE_SIZE Line 196: int64_t uint64_t Line 1247: Process any updates Update the function comment to indicate that large updates are performed in batches so that they do not exceed the specified threshold. If you want you can reference the JIRA number here. Line 1251: int64_t uint64_t Line 1282: // IMPALA-3499 batch sending catalog update. This also work around IMPALA-3494. : // The limit is selected such that topic_deletions can be appended without exceeding : // 2GB java byte array limit, which simplify the code a lot. Remove comment. Line 1286: INFO I think WARN is more appropriate. Line 1287: update_size I don't think that is correct. The catalog update size is update_size - len, since you haven't added the current catalog object to the update_req. Line 1291: break Don't just break, please also add an error message. Line 1293: update_req.__set_is_delta(true); I don't think delta indicates that the updates are performed in batches. -- To view, visit http://gerrit.cloudera.org:8080/3067 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: comment Gerrit-Change-Id: I176db25124a32944f2396ce8aafbed49cac95928 Gerrit-PatchSet: 2 Gerrit-Project: Impala Gerrit-Branch: cdh5-trunk Gerrit-Owner: Huaisi Xu <[email protected]> Gerrit-Reviewer: Dimitris Tsirogiannis <[email protected]> Gerrit-Reviewer: Huaisi Xu <[email protected]> Gerrit-Reviewer: Tim Armstrong <[email protected]> Gerrit-HasComments: Yes
