Huaisi Xu has posted comments on this change. Change subject: IMPALA-3499: Split catalog update ......................................................................
Patch Set 21: Code-Review+1 Update... I managed to test this and https://issues.cloudera.org/browse/IMPALA-3494 by hacking impalad backend.. I set the name of each table to be super long before backend serializes that to frontend. 1. I hit the exception added by https://issues.cloudera.org/browse/IMPALA-3494 with each table name to be 4MB (1000+ tables in the initial update). 2. I then changed the name size to 2MB, and as expected I received frontend "NegativeArraySizeException: null". --- But it is different from what the Jira title and the commit message say: the exception happened here http://github.mtv.cloudera.com/CDH/Impala/blob/cdh5-trunk/be/src/rpc/jni-thrift-util.h#L39 when backend tried to write java byte array instead of deserialize that. this 100% makes sense. I will update the commit message and Jira title accordingly. 3. I then applied this patch and it went through. --- I tested it by running "show tables", and I saw expected result. (I could not do other things such as "select * from table" since the table names are all super long, so I could not copy&paste). I think this is valid test. don't you think so?... So verifying this patch comes with two part: 1. Verify split---merge works. This was done by full dataloading with 10kb limit. 2. Verify split update works around Java's limit on byte array. <- just verified this one. Carry Henry and Dimitris previous +1... -- To view, visit http://gerrit.cloudera.org:8080/3067 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: comment Gerrit-Change-Id: I176db25124a32944f2396ce8aafbed49cac95928 Gerrit-PatchSet: 21 Gerrit-Project: Impala Gerrit-Branch: cdh5-trunk Gerrit-Owner: Huaisi Xu <[email protected]> Gerrit-Reviewer: Bharath Vissapragada <[email protected]> Gerrit-Reviewer: Dan Hecht <[email protected]> Gerrit-Reviewer: Dimitris Tsirogiannis <[email protected]> Gerrit-Reviewer: Henry Robinson <[email protected]> Gerrit-Reviewer: Huaisi Xu <[email protected]> Gerrit-HasComments: No
