How does one write to more than just one file and output to multiple s3 files so that reading a single column does not read all s3 data how can this be done?
If its a single file I've figured out how to output using AvroParquetWriter. But I can't find any documentation or examples of how to partition by column to multiple files on s3. Does Parquet even support this?
