Hi Amit Answers below
On Sun, May 29, 2016 at 11:37 AM, Amit Adhau <amit.ad...@globant.com> wrote: > Hi, > > What is the significance of using below gflags, which can help in imroving > the insert peformance with INSERT INTO SELECT clause. > > --num_tablets_to_open_simultaneously=8 > This only affects the startup time of a tablet server, and should not affect the insert performance at all. > --scanner_batch_size_rows=1000 > This only affects the read performance. I've seen it have a noticeable affect at times, but can also cause some memory management issues with wider tables - that's why the default is 100. It won't affect write performance at all. > > and maintenance_manager_num_threads(Kudu Tablet Server Maintenance > Threads) in cloudera manager. > > This could improve write performance, since it increases the number of threads available to perform compaction and flushes. Assuming your hardware looks like typical Hadoop nodes (eg 10-12 disks, 8-16 cores), I would try setting it to 4 as a starting point. > As playing with these configs, gives most of the time errors like "Timed > out: Failed to write batch of ops to tablet " OR "Illegal state: Tablet not > RUNNING: NOT_STARTED" > > Those errors seem to indicate you are probably trying to perform reads/writes while the servers are still in the process of starting up. Maybe you are not giving the cluster enough time to fully restart before you are restarting the workload after changing the tuning? -Todd
