Hi all, I'm using a ExecuteSQLRecord processor with a ParquetRecordSetWriter to fetch data in parquet format. I have configured the writer as follows.
[image: 3.png] I have set the row group size to 256MB as above. However when I checked a generated file from *parquet-tools*, there were two row groups and the size of the first row group was about 264MB. However the file size was 244MB [image: 1 (1).png] [image: 2.png] I got above results when I set the Writer version to PARQUET_2_0. With the PARQUET_1_0, the first row group size was about 415MB. When the row group size is set to 256MB, Is it possible for a row group to exceed that memory limit.? Could you please explain how the parquertRecorSetWriter behaves. Thanks & Regards *Vibhath Ileperuma*
