Zoltan Martonka has uploaded this change for review. ( http://gerrit.cloudera.org:8080/24731
Change subject: WIP KUDU-2874 Test ordered scan memory usage with large cells. ...................................................................... WIP KUDU-2874 Test ordered scan memory usage with large cells. Currently sorted scans has a burned in 1024 size buffer. With 1 MB of cells this can cause large, untracked memory usage. This test does the following: + Start a tserver with one tablet. + Disable compaction for worst case result (compaction falls behind). + Write rows with 1 MB cells into a tablet. + Optionally disable flushing to disk at a point, so memory usage remains large. + Start 10 parallel ordered full scans. They are started one-by-one with a gap between them, so they probably request other parts of the data at the same time. + Compare tmalloc reported memory usage with the trackers. tmalloc also counts the tests memory usage, but the "keep_unflushed" reports 10 GB of "untracked" memory, which is a lot. 20 GB of tracked memory is also unacceptable for a 8GB memory limit. 2 things should be addressed to fix the test: + Dynamically downsize batches when cells are really large. + Properly count memory at scans when copying values from memrowset. (It is counted properly when loaded from disk rowsets). Change-Id: I75ceaebac1793c2421302f8b205ba85ad32a141a --- M src/kudu/tserver/CMakeLists.txt A src/kudu/tserver/upsert_ordered_scan-test.cc 2 files changed, 566 insertions(+), 0 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/31/24731/1 -- To view, visit http://gerrit.cloudera.org:8080/24731 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: I75ceaebac1793c2421302f8b205ba85ad32a141a Gerrit-Change-Number: 24731 Gerrit-PatchSet: 1 Gerrit-Owner: Zoltan Martonka <[email protected]>
