[
https://issues.apache.org/jira/browse/HADOOP-16202?focusedWorklogId=752186&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-752186
]
ASF GitHub Bot logged work on HADOOP-16202:
-------------------------------------------
Author: ASF GitHub Bot
Created on: 04/Apr/22 10:44
Start Date: 04/Apr/22 10:44
Worklog Time Spent: 10m
Work Description: steveloughran commented on PR #2584:
URL: https://github.com/apache/hadoop/pull/2584#issuecomment-1087397803
```
./hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/ChecksumFileSystem.java:29:import
java.util.concurrent.CompletableFuture;:8: Unused import -
java.util.concurrent.CompletableFuture. [UnusedImports]
./hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/ChecksumFileSystem.java:35:import
org.apache.hadoop.fs.impl.AbstractFSBuilderImpl;:8: Unused import -
org.apache.hadoop.fs.impl.AbstractFSBuilderImpl. [UnusedImports]
./hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/ChecksumFileSystem.java:37:import
org.apache.hadoop.fs.impl.OpenFileParameters;:8: Unused import -
org.apache.hadoop.fs.impl.OpenFileParameters. [UnusedImports]
./hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/ChecksumFileSystem.java:44:import
org.apache.hadoop.util.LambdaUtils;:8: Unused import -
org.apache.hadoop.util.LambdaUtils. [UnusedImports]
./hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/ChecksumFileSystem.java:47:import
static
org.apache.hadoop.fs.Options.OpenFileOptions.FS_OPTION_OPENFILE_STANDARD_OPTIONS;:15:
Unused import -
org.apache.hadoop.fs.Options.OpenFileOptions.FS_OPTION_OPENFILE_STANDARD_OPTIONS.
[UnusedImports]
./hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/statistics/impl/IOStatisticsBinding.java:528:
/**: First sentence should end with a period. [JavadocStyle]
./hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/Constants.java:618:
public static int DEFAULT_ASYNC_DRAIN_THRESHOLD = 16_000;:21: Name
'DEFAULT_ASYNC_DRAIN_THRESHOLD' must match pattern '^[a-z][a-zA-Z0-9]*$'.
[StaticVariableName]
./hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/Constants.java:618:
public static int DEFAULT_ASYNC_DRAIN_THRESHOLD = 16_000;:21: Variable
'DEFAULT_ASYNC_DRAIN_THRESHOLD' must be private and have accessor methods.
[VisibilityModifier]
./hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/Invoker.java:117:
@Retries.OnceTranslated: 'method def modifier' has incorrect indentation
level 4, expected level should be 2. [Indentation]
./hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/Invoker.java:121:
try (DurationInfo ignored = new DurationInfo(LOG, false, "%s", action))
{: 'try' has incorrect indentation level 6, expected level should be 4.
[Indentation]
./hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/Invoker.java:122:
return operation.apply();: 'try' child has incorrect indentation level
8, expected level should be 6. [Indentation]
./hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/Invoker.java:123:
} catch (AmazonClientException e) {: 'try rcurly' has incorrect
indentation level 6, expected level should be 4. [Indentation]
./hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/Invoker.java:124:
throw S3AUtils.translateException(action, path, e);: 'catch' child has
incorrect indentation level 8, expected level should be 6. [Indentation]
./hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/Invoker.java:125:
}: 'catch rcurly' has incorrect indentation level 6, expected level
should be 4. [Indentation]
./hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/Invoker.java:126:
}: 'method def rcurly' has incorrect indentation level 4, expected level
should be 2. [Indentation]
./hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/Invoker.java:137:
@Retries.OnceTranslated: 'method def modifier' has incorrect indentation
level 4, expected level should be 2. [Indentation]
./hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/Invoker.java:144:
try {: 'try' has incorrect indentation level 6, expected level should be
4. [Indentation]
./hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/Invoker.java:145:
return invokeTrackingDuration(tracker, operation);: 'try' child has
incorrect indentation level 8, expected level should be 6. [Indentation]
./hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/Invoker.java:146:
} catch (AmazonClientException e) {: 'try rcurly' has incorrect
indentation level 6, expected level should be 4. [Indentation]
./hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/Invoker.java:147:
throw S3AUtils.translateException(action, path, e);: 'catch' child has
incorrect indentation level 8, expected level should be 6. [Indentation]
./hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/Invoker.java:148:
}: 'catch rcurly' has incorrect indentation level 6, expected level
should be 4. [Indentation]
./hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/Invoker.java:149:
}: 'method def rcurly' has incorrect indentation level 4, expected level
should be 2. [Indentation]
./hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/S3AInputStream.java:95:
private static int ASYNC_DRAIN_THRESHOLD = 16_000;:22: Name
'ASYNC_DRAIN_THRESHOLD' must match pattern '^[a-z][a-zA-Z0-9]*$'.
[StaticVariableName]
./hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/S3AInputStream.java:641:
() -> drainOrAbortHttpStream(shouldAbort, reason, remaining,
requestObject, inner)));: Line is longer than 100 characters (found 101).
[LineLength]
./hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/TestS3AInputStreamRetry.java:25:import
java.util.concurrent.Callable;:8: Unused import -
java.util.concurrent.Callable. [UnusedImports]
```
+ some minor javadoc errors due to changed arguments
Issue Time Tracking
-------------------
Worklog Id: (was: 752186)
Time Spent: 16h (was: 15h 50m)
> Stabilize openFile() and adopt internally
> -----------------------------------------
>
> Key: HADOOP-16202
> URL: https://issues.apache.org/jira/browse/HADOOP-16202
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs, fs/s3, tools/distcp
> Affects Versions: 3.3.0
> Reporter: Steve Loughran
> Assignee: Steve Loughran
> Priority: Major
> Labels: pull-request-available
> Time Spent: 16h
> Remaining Estimate: 0h
>
> The {{openFile()}} builder API lets us add new options when reading a file
> Add an option {{"fs.s3a.open.option.length"}} which takes a long and allows
> the length of the file to be declared. If set, *no check for the existence of
> the file is issued when opening the file*
> Also: withFileStatus() to take any FileStatus implementation, rather than
> only S3AFileStatus -and not check that the path matches the path being
> opened. Needed to support viewFS-style wrapping and mounting.
> and Adopt where appropriate to stop clusters with S3A reads switched to
> random IO from killing download/localization
> * fs shell copyToLocal
> * distcp
> * IOUtils.copy
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
