steveloughran commented on code in PR #1807:
URL: https://github.com/apache/avro/pull/1807#discussion_r2285407691
##########
lang/java/mapred/src/main/java/org/apache/avro/mapred/FsInput.java:
##########
@@ -40,8 +45,19 @@ public FsInput(Path path, Configuration conf) throws
IOException {
/** Construct given a path and a {@code FileSystem}. */
public FsInput(Path path, FileSystem fileSystem) throws IOException {
- this.len = fileSystem.getFileStatus(path).getLen();
- this.stream = fileSystem.open(path);
+ final FileStatus st = fileSystem.getFileStatus(path);
+ this.len = st.getLen();
+ // use the hadoop 3.3+ openFile API, passing in status
+ // and read policy. object stores can use these to
+ // optimize read performance and save on a HEAD request when opening
+ // a file.
+ final FutureDataInputStreamBuilder builder =
fileSystem.openFile(path).opt(FS_OPTION_OPENFILE_READ_POLICY,
+ "avro, sequential, adaptive");
Review Comment:
we do have them in hadoop, but that hard codes to specific versions (the
"avro" one is fairly recent. Better for me to put them into an avro class
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
