To your and Brock’s questions, my answers are as below.
What client are you using to benchmark? You might also be bound by the client
performance.
My Answer: we are using many force testing machines to test the highest TPS on
kudu. Our testing client should have enough ability.
I'd verify that the new nodes are assigned tablets? Along with considering an
increase the number of partitions on the table being tested.
My Answer: Yes, with machines added each time, I created a new table for
testing so that tablets can be assigned to new machines. For the partition
strategy, I am using 2-level partitions: the first level is a range partition
by date(I use 3 partitions here, meaning 3-days data), and the second level is
a hash partition(I use 3, 6, and 9 respectively for the clusters with 3, 6, and
9 tservers).