Hello Todd Lipcon,
I'd like you to do a code review. Please visit
http://gerrit.cloudera.org:8080/6662
to review the following change.
Change subject: WIP: KUDU-1970: node density integration test
......................................................................
WIP: KUDU-1970: node density integration test
This patch introduces a new itest that models a storage-dense Kudu
deployment. The idea is simple: rather than actually generating and storing
lots of data (which is both time intensive and developer unfriendly), let's
produce a lot of metadata instead, as that's cheaper and can proxy for data.
The test itself isn't that interesting; most of the challenge was in running
it repeatedly to determine which flag values yielded the most metadata.
In a run of the test on a 48 core el6.6 machine (max_blocks_per_container=8
and num_seconds=240), I produced ~110K blocks across ~21k LBM containers,
which yielded a subsequent ~100s LBM startup time.
WIP because an itest isn't a great fit; we're not actually testing anything.
But, it didn't really make sense as part of the CLI tool either as that
talks to existing clusters/deployments and never spins up an
ExternalMiniCluster (and I'm not convinced it could either; there are some
Kudu testing assumptions baked into EMC). I'm open to other ideas.
Change-Id: Ie9b5d01557eb41d386ce92f576ed01ec658e8e7d
---
M src/kudu/integration-tests/CMakeLists.txt
A src/kudu/integration-tests/dense_node-itest.cc
M src/kudu/integration-tests/test_workload.cc
M src/kudu/integration-tests/test_workload.h
4 files changed, 261 insertions(+), 39 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/62/6662/1
--
To view, visit http://gerrit.cloudera.org:8080/6662
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-MessageType: newchange
Gerrit-Change-Id: Ie9b5d01557eb41d386ce92f576ed01ec658e8e7d
Gerrit-PatchSet: 1
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-Owner: Adar Dembo <[email protected]>
Gerrit-Reviewer: Todd Lipcon <[email protected]>
