Hi chenxingyu, How many executors you are having? Can you check how many select tasks are fired to query from parquet? Also got can check number of tasks being created is you do CTAS to hive table
Regards Raghu On Tue, 19 Jun 2018, 5:45 pm 陈星宇, <[email protected]> wrote: > hi , > > > i wrote data into carbondata table from parquet table by spark_sql 'insert > into carbondata_table select * from parquet_table', the task number is > always only one. > it caused the insert job was very slow . > i tried increase spark.default.parallelism = 1000, but only increase query > task. > the parquet files are more than 500. > how can i get better performance when insert into carbondata table. > > > THANKS > ChenXingYu
