steveloughran commented on PR #5172: URL: https://github.com/apache/hadoop/pull/5172#issuecomment-1335114105
> I'd like to make oss also implement openFile() in this pr as s3a does which could also meet our needs. This is exactly what the API was designed for -to let people provide extra hints/options. to the object stores. Do read the filesystem.md and related docs on the topic, and look at the abfs implementation as well as the s3a one. If your implementation takes a FileStatus or length option, it can then skip the HEAD request on opening and save time and money. All the hadoop internal uses of openFile() do this. My own copies of the parquet and avro readers also do it for better cloud reading performance -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
