Todd Lipcon has submitted this change and it was merged. (
http://gerrit.cloudera.org:8080/9759 )
Change subject: perf: improve string data generation performance
......................................................................
perf: improve string data generation performance
This changes the 'perf loadgen' tool to not use stringstream for
formatting a string. Instead, it reuses a single string buffer and uses
some functions from gutil to achieve the same output.
Tested with:
build/latest/bin/kudu.orig perf loadgen -num-threads 16 \
-num-rows-per-thread $[10*1000*1000] \
-table-num-replicas 3 -table-num-buckets=100 \
my-master
Before this fix:
time total : 130030 ms
2204814.701333 task-clock (msec) # 16.535 CPUs utilized
Top profile lines:
14.17% kudu.orig libstdc++.so.6.0.19 [.] std::locale::locale
7.26% kudu.orig libstdc++.so.6.0.19 [.] std::locale::~locale
5.91% kudu.orig libstdc++.so.6.0.19 [.]
std::use_facet<std::num_put<char, std::ostreambuf_iterator<char,
std::char_traits<char> > > >
5.41% kudu.orig libstdc++.so.6.0.19 [.]
std::has_facet<std::num_put<char, std::ostreambuf_iterator<char,
std::char_traits<char> > > >
4.69% kudu.orig libstdc++.so.6.0.19 [.]
std::has_facet<std::num_get<char, std::istreambuf_iterator<char,
std::char_traits<char> > > >
After this the throughput and CPU consumption were both improved by
about 40%:
time total : 90676.3 ms
1458961.567442 task-clock (msec) # 15.954 CPUs utilized
The top of the profile is now all related to client code instead of the
loadgen tool itself.
Change-Id: I4a3ebe39588d3da6a0775b168c5350fb420e025e
Reviewed-on: http://gerrit.cloudera.org:8080/9759
Reviewed-by: Adar Dembo <[email protected]>
Tested-by: Kudu Jenkins
---
M src/kudu/tools/tool_action_perf.cc
1 file changed, 7 insertions(+), 9 deletions(-)
Approvals:
Adar Dembo: Looks good to me, approved
Kudu Jenkins: Verified
--
To view, visit http://gerrit.cloudera.org:8080/9759
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: I4a3ebe39588d3da6a0775b168c5350fb420e025e
Gerrit-Change-Number: 9759
Gerrit-PatchSet: 2
Gerrit-Owner: Todd Lipcon <[email protected]>
Gerrit-Reviewer: Adar Dembo <[email protected]>
Gerrit-Reviewer: Alexey Serbin <[email protected]>
Gerrit-Reviewer: Dan Burkert <[email protected]>
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Todd Lipcon <[email protected]>
