Re: Why is it slow to write Kudu with 100+ threads?

2020-06-09 Thread Alexey Serbin
f7a07b32ff70d/flink-connector-kudu/src/main/java/org/apache/flink/connectors/kudu/connector/writer/KuduWriter.java#L89 > > Basically, it calls KuduSession.apply with each element needs to be > inserted. > > > > > > > > *From: *"Ray Liu (rayliu)" > *Reply

Re: Why is it slow to write Kudu with 100+ threads?

2020-06-09 Thread Ray Liu (rayliu)
ayliu)" Reply-To: "user@kudu.apache.org" Date: Tuesday, June 9, 2020 at 23:31 To: "user@kudu.apache.org" Subject: Why is it slow to write Kudu with 100+ threads? We have a Kudu cluster with 5 tablet servers each has 28 CPU cores, 160GB RAM and 2TB SSD. The RPC queue

Why is it slow to write Kudu with 100+ threads?

2020-06-09 Thread Ray Liu (rayliu)
We have a Kudu cluster with 5 tablet servers each has 28 CPU cores, 160GB RAM and 2TB SSD. The RPC queue length we set is 500. We now write to 10 tables at the same time. We’re using 10 threads each to write(simply insert) to 8 out of these 10 tables. We have 5 task (each task with 10 threads)