Ashwani Raina has uploaded this change for review. (
http://gerrit.cloudera.org:8080/24615
Change subject: [tablet] Skip stable_sort for already sorted batch
......................................................................
[tablet] Skip stable_sort for already sorted batch
Tablet::BulkCheckPresence unconditionally runs std::stable_sort over the
encoded-key/index pairs for every write batch before deduplicating and
issuing the RowSetTree lookup. When a client submits rows already in
increasing key order such as bulk-ingest tools (e.g. 'kudu perf loadgen
--use_random=false', Impala INSERT ... ORDER BY, Spark/Sqoop pipelines
that pre-sort by PK) and workloads with monotonic primary keys typically
do, this sort is pure overhead: the input is already a valid
stable-sorted output.
This patch tracks an 'is_sorted' flag while building 'keys_and_indexes'
in the loop that already scans every op, flipping it to false on the
first out-of-order pair. When the flag survives to the end of the loop,
the stable_sort is skipped entirely, improving performance by a slight
margin if not significant.
Measured with 'kudu perf loadgen --use_upsert --num_rows_per_thread=5M'
on a single-tablet cluster, median of N=5 runs:
Without this patch and sorted input batch:
for i in 1 2 3 4 5; do ./build/latest/bin/kudu perf loadgen \
127.0.0.1:8764 --use_upsert --use_random=false --num_threads=1 \
--num_rows_per_thread=5000000 --table_num_hash_partitions=1 \
--table_num_range_partitions=1 --buffer_flush_watermark_pct=1.0 \
--run_scan=false --keep_auto_table=false \
2>&1 | grep "time total"; done
time total: 59511.7 ms
time total: 62104.7 ms
time total: 62865 ms
time total: 62317.8 ms
time total: 62660.1 ms
Median: 61,891.86 ms
With this patch and sorted input batch:
for i in 1 2 3 4 5; do ./build/latest/bin/kudu perf loadgen \
127.0.0.1:8764 --use_upsert --use_random=false --num_threads=1
--num_rows_per_thread=5000000 --table_num_hash_partitions=1
--table_num_range_partitions=1 --buffer_flush_watermark_pct=1.0 \
--run_scan=false --keep_auto_table=false \
2>&1 | grep "time total"; done
time total: 58113.7 ms
time total: 59352.5 ms
time total: 59518.3 ms
time total: 60156.6 ms
time total: 60377.6 ms
Median: 59,503.74 ms
Change-Id: I28d3f5ebdf7e3f13f04ffe8d35417a98c370fcf7
---
M src/kudu/tablet/tablet.cc
M src/kudu/tablet/tablet_metrics.cc
M src/kudu/tablet/tablet_metrics.h
3 files changed, 102 insertions(+), 11 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/15/24615/1
--
To view, visit http://gerrit.cloudera.org:8080/24615
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: I28d3f5ebdf7e3f13f04ffe8d35417a98c370fcf7
Gerrit-Change-Number: 24615
Gerrit-PatchSet: 1
Gerrit-Owner: Ashwani Raina <[email protected]>
