elithrar commented on issue #34363: URL: https://github.com/apache/arrow/issues/34363#issuecomment-1544274719
(For context: I work at Cloudflare) Here's the latest: - R2 _currently_ requires that: all parts are of equal size, a maximum of 10K parts, and per-part limit of 5GB, and a total file size of 5TB. - DuckDB updated their custom http-fs implementation to work with this - We don't yet have any concrete plans to relax this in the medium term: s3fs would need to support the ability (or a client using it) to work within R2's requirements This doesn't mean we're not open to relaxing this, but it's a non-trivial change. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
