[ https://issues.apache.org/jira/browse/HIVE-24225?focusedWorklogId=496489&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-496489 ]
ASF GitHub Bot logged work on HIVE-24225: ----------------------------------------- Author: ASF GitHub Bot Created on: 07/Oct/20 10:54 Start Date: 07/Oct/20 10:54 Worklog Time Spent: 10m Work Description: pgaref edited a comment on pull request #1547: URL: https://github.com/apache/hive/pull/1547#issuecomment-704856290 Hey @steveloughran --- the approach of the above patch was a bit off, one problem was that the Fs objects were lazility initalized and could end up throwing exceptions when setting the option eagerly. The most important issue was that the LLAP IO creates its own FS object (and the above where only used for output) so the option itself was not properly propagated. A solution for all this could be the S3A **openFileWithOptions** call that adds file options for the open File call instead of on the FS (still needs to add support for **fadvise** though) https://github.com/apache/hadoop/blob/trunk/hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/S3AFileSystem.java#L4828 Talking about this TODO: https://github.com/apache/hadoop/blob/trunk/hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/S3AFileSystem.java#L1136 ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking ------------------- Worklog Id: (was: 496489) Time Spent: 1h (was: 50m) > FIX S3A recordReader policy selection > ------------------------------------- > > Key: HIVE-24225 > URL: https://issues.apache.org/jira/browse/HIVE-24225 > Project: Hive > Issue Type: Bug > Reporter: Panagiotis Garefalakis > Assignee: Panagiotis Garefalakis > Priority: Major > Labels: pull-request-available > Fix For: 4.0.0 > > Time Spent: 1h > Remaining Estimate: 0h > > Dynamic S3A recordReader policy selection can cause issues on lazy > initialized FS objects -- This message was sent by Atlassian Jira (v8.3.4#803005)