[
https://issues.apache.org/jira/browse/HADOOP-17682?focusedWorklogId=626776&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-626776
]
ASF GitHub Bot logged work on HADOOP-17682:
-------------------------------------------
Author: ASF GitHub Bot
Created on: 22/Jul/21 16:44
Start Date: 22/Jul/21 16:44
Worklog Time Spent: 10m
Work Description: anoopsjohn commented on a change in pull request #2975:
URL: https://github.com/apache/hadoop/pull/2975#discussion_r674979323
##########
File path:
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/AzureBlobFileSystem.java
##########
@@ -204,38 +204,51 @@ public void registerListener(Listener listener1) {
public FSDataInputStream open(final Path path, final int bufferSize) throws
IOException {
LOG.debug("AzureBlobFileSystem.open path: {} bufferSize: {}", path,
bufferSize);
// bufferSize is unused.
- return open(path, Optional.empty());
+ return open(path, new OpenFileParameters());
Review comment:
Concern on using this way and passing OpenFileParameters param.
When FS#openFile() API is been used, you can see how system create the
OpenFileParameters(). There may be some stuff coming in as default there.
eg: bufferSize = fileSystem.getConf().getInt(IO_FILE_BUFFER_SIZE_KEY,
IO_FILE_BUFFER_SIZE_DEFAULT);
So correct buffer size as per conf will be there in OpenFileParameters also
We might not be using buffer size from the param now. But later its
possible to use some new stuff like this.
So better dont create OpenFileParameters () here and pass.
Instead we can have the private method accept Optional<OpenFileParameters>.
In open(Path), we can Optional.empty() anyways.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 626776)
Time Spent: 3h 10m (was: 3h)
> ABFS: Support FileStatus input to OpenFileWithOptions() via OpenFileParameters
> ------------------------------------------------------------------------------
>
> Key: HADOOP-17682
> URL: https://issues.apache.org/jira/browse/HADOOP-17682
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/azure
> Reporter: Sumangala Patki
> Assignee: Sumangala Patki
> Priority: Major
> Labels: pull-request-available
> Time Spent: 3h 10m
> Remaining Estimate: 0h
>
> ABFS open methods require certain information (contentLength, eTag, etc) to
> to create an InputStream for the file at the given path. This information is
> retrieved via a GetFileStatus request to backend.
> However, client applications may often have access to the FileStatus prior to
> invoking the open API. Providing this FileStatus to the driver through the
> OpenFileParameters argument of openFileWithOptions() can help avoid the call
> to Store for FileStatus.
> This PR adds handling for the FileStatus instance (if any) provided via the
> OpenFileParameters argument.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
