Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/24515 )
Change subject: IMPALA-15118: Copy ByteBuffers for partitionList_ ...................................................................... IMPALA-15118: Copy ByteBuffers for partitionList_ IcebergContentFileStore.fromThrift() didn't copy the contents of ByteBuffers when filling partitionList_. Thrift's binary handling in Java is special as the ByteBuffer object in deserialized Thrift object can still reference the transport buffer. Keeping the reference to that can almost double the size of IcebergContentFileStore in the coordinator. Another issue fixed is that partitionMap_ and partitionList_ did not share the ByteBuffers after fromThrift() Improvement for test table with 1M files, 25K partitions: 665MB->389MB Change-Id: Ib39940dd64aadcb094b099e3863dd42c22afc2b8 Reviewed-on: http://gerrit.cloudera.org:8080/24515 Reviewed-by: Impala Public Jenkins <[email protected]> Tested-by: Impala Public Jenkins <[email protected]> --- M fe/src/main/java/org/apache/impala/catalog/IcebergContentFileStore.java 1 file changed, 7 insertions(+), 4 deletions(-) Approvals: Impala Public Jenkins: Looks good to me, approved; Verified -- To view, visit http://gerrit.cloudera.org:8080/24515 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: Ib39940dd64aadcb094b099e3863dd42c22afc2b8 Gerrit-Change-Number: 24515 Gerrit-PatchSet: 3 Gerrit-Owner: Csaba Ringhofer <[email protected]> Gerrit-Reviewer: Impala Public Jenkins <[email protected]> Gerrit-Reviewer: Noemi Pap-Takacs <[email protected]> Gerrit-Reviewer: Zoltan Borok-Nagy <[email protected]>
