Alexey Serbin has posted comments on this change. ( http://gerrit.cloudera.org:8080/19278 )
Change subject: [tools] Add test to generate heavy rowset compaction ...................................................................... Patch Set 5: (1 comment) http://gerrit.cloudera.org:8080/#/c/19278/5//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/19278/5//COMMIT_MSG@13 PS5, Line 13: The test may require a follow-up change to accomodate recently : added change for 'memory budgeting of CompactRowSetsOp' > The reason behind having this patch is to document steps to reproduce high Thanks for the explanation. Since these scenarios are clearly of a stress nature with such unpredictable outcome and long run time, it doesn't make sense to include it every run of pre-commit verification Jenkins jobs, etc. as-is, indeed. However, keeping them DISABLED isn't a good idea since they would become 'bit rotten' over time, making them useless eventually. I'd think that a better form factor for these tests would be: * Parameterizing the tests using dedicated glfags, with their defaults set to reasonable values at least to make sure the tests exercise the existing control paths, but don't put a lot of stress on the node they are run (limiting the runtime to a few seconds at most). It should be built and ready to run with customized gflag values as a part of micro-benchmarks or another custom test set (you could take a look at various tests run by the $KUDU_HOME/src/kudu/scripts/benchmarks.sh). The crucial set of parameter values to set (i.e. the settings for the workload and customer overrides for kudu-master's and kudu-tserver's flags) should be documented (preferably, in the code). * Having them in a separate binary that not considered a test, but some sort of benchmark. $KUDU_HOME/src/kudu/benchmarks/wal_hiccup.cc is an example. One more point: kudu-tool-test.cc doesn't seem to be the right place to keep these tests long-term. The kudu-tool-test.cc file is for tests covering functionality of the kudu CLI tools, not something specific to compaction. Consider adding these new tests into either compaction-test.cc, or compaction_policy-test.cc, or separate them into a new test binary. -- To view, visit http://gerrit.cloudera.org:8080/19278 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ief4ec03f1bf0f7936f8fb054948f87e71333f824 Gerrit-Change-Number: 19278 Gerrit-PatchSet: 5 Gerrit-Owner: Ashwani Raina <[email protected]> Gerrit-Reviewer: Abhishek Chennaka <[email protected]> Gerrit-Reviewer: Alexey Serbin <[email protected]> Gerrit-Reviewer: Ashwani Raina <[email protected]> Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Comment-Date: Wed, 01 Mar 2023 19:22:35 +0000 Gerrit-HasComments: Yes
