marcin-krystianc commented on issue #5770: URL: https://github.com/apache/arrow-rs/issues/5770#issuecomment-2117300694
> How big are these files? At 1MB per page (the recommended size) this would come to 5TB?! I think file size depends on the number of rows per row group. With 100 row groups, 50k columns and single row per row group the file has 700MB (no compression, default page size, no stats). We use files with tens of thousands rows per row group, and the files sizes about 1TB. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
