orf commented on PR #41564: URL: https://github.com/apache/arrow/pull/41564#issuecomment-2110822421
Just to note, issuing `HeadBucket` doesn't guarantee that a write will succeed - there isn't really a good way to check without actually writing. I think an optimization flag is appropriate as the behaviour is technically changing. Does it make sense to make the flag a somewhat generic one, rather than specific to this case? There are a [few other optimizations](https://github.com/apache/arrow/issues/40589) that might also fall into the "more performant, slightly different semantics" category. If I was to contribute a PR to improve one of the linked areas, would we want to add a new specific flag for this case? The upside would be that it becomes more configurable, whereas the downside is that the testing and support matrix explodes. Perhaps it's better to just have a single `optimized=True` flag, vs reciving issues when specifically `optimize_put_object=True, optimize_delete_dir=False, optimize_move=True, optimize_delete=False, optimize_foobar=True` are set? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
