Todd Lipcon has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/15679 )
Change subject: columnar_serialization: avoid preallocating 8MB per column ...................................................................... columnar_serialization: avoid preallocating 8MB per column Previously we would reserve an 8MB buffer for every column of data to be scanned. This wouldn't scale well for high number of concurrent queries with lots of columns. The new approach is to use the configured batch size and apportion that memory budget across the columns based on the size of those columns. It's not 100% accurate but at least shouldn't overshoot by hundreds of MB like the prior approach. Change-Id: I9b7ff78547792acbd975a606a02ec388dba3a8e8 Reviewed-on: http://gerrit.cloudera.org:8080/15679 Reviewed-by: Andrew Wong <[email protected]> Tested-by: Kudu Jenkins --- M src/kudu/common/columnar_serialization.cc M src/kudu/common/columnar_serialization.h M src/kudu/common/wire_protocol-test.cc M src/kudu/tserver/tablet_service.cc 4 files changed, 25 insertions(+), 11 deletions(-) Approvals: Andrew Wong: Looks good to me, approved Kudu Jenkins: Verified -- To view, visit http://gerrit.cloudera.org:8080/15679 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: I9b7ff78547792acbd975a606a02ec388dba3a8e8 Gerrit-Change-Number: 15679 Gerrit-PatchSet: 4 Gerrit-Owner: Todd Lipcon <[email protected]> Gerrit-Reviewer: Alexey Serbin <[email protected]> Gerrit-Reviewer: Andrew Wong <[email protected]> Gerrit-Reviewer: Andrew Wong <[email protected]> Gerrit-Reviewer: Grant Henke <[email protected]> Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Todd Lipcon <[email protected]>
