steveloughran commented on pull request #31565: URL: https://github.com/apache/spark/pull/31565#issuecomment-857152998
Interesting use case. + @mukund-thakur @mehakmeet I'm not sure that the s3a connector handles ? presigning, so I'm not worried there. If https URLs to an s3 bucket are given out with the ? args, then AWS S3 will either take the signature and allow or fail with 40-something. There's no harm in comparing this way for any of the stores, not that I can see, and it can only help with this specific use case. Not that efficient to be pulling in individual files though., not if you have a lot of them -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
