Hi, I'm reading a large tsv file, and creating parquet files using sparksql: insert overwrite table tbl partition(year, month, day).... Select .... from tbl_tsv;
This works nicely, but generates small parquet files (15MB). I wanted to generate larger files, any idea how to address this? Thanks, Younes Naguib Triton Digital | 1440 Ste-Catherine W., Suite 1200 | Montreal, QC H3G 1R8 Tel.: +1 514 448 4037 x2688 | Tel.: +1 866 448 4037 x2688 | [email protected] <mailto:[email protected]>
