I am doing a simple count like:

sqlContext.read.parquet("path").count

I have only 5000 parquet files.  But generate over 20000 tasks.

Each parquet file is converted from one gz text file.

Please give some advice.

Thanks

Reply via email to