[
https://issues.apache.org/jira/browse/HADOOP-15229?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16718054#comment-16718054
]
Steve Loughran commented on HADOOP-15229:
-----------------------------------------
Checkstyle
{code}
./hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FutureDataInputStreamBuilder.java:21:import
java.io.InputStream;:8: Unused import - java.io.InputStream. [UnusedImports]
./hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FutureDataInputStreamBuilder.java:27:import
static com.google.common.base.Preconditions.checkNotNull;:15: Unused import -
com.google.common.base.Preconditions.checkNotNull. [UnusedImports]
./hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FutureDataInputStreamBuilder.java:47:
extends FSBuilder<CompletableFuture<FSDataInputStream>,
FutureDataInputStreamBuilder> {: Line is longer than 80 characters (found 91).
[LineLength]
./hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/impl/AbstractFSBuilderImpl.java:68:AbstractFSBuilderImpl<S,
B extends FSBuilder<S, B>>: 'AbstractFSBuilderImpl' has incorrect indentation
level 0, expected level should be 4. [Indentation]
./hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/impl/FutureDataInputStreamBuilderImpl.java:56:
extends AbstractFSBuilderImpl<CompletableFuture<FSDataInputStream>,
FutureDataInputStreamBuilder>: Line is longer than 80 characters (found 101).
[LineLength]
./hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/S3AReadOpContext.java:39:
final Path path;:14: Variable 'path' must be private and have accessor
methods. [VisibilityModifier]
./hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/S3AReadOpContext.java:63:
public S3AReadOpContext(:10: More than 7 parameters (found 9).
[ParameterNumber]
./hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/select/ITestS3Select.java:118:
public void createStandardCsvFile(:15: More than 7 parameters (found 8).
[ParameterNumber]
./hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/select/ITestS3SelectLandsat.java:56:
* <a href="https://docs.opendata.aws/landsat-pds/readme.html";>Landsat on
AWS</a>: Line is longer than 80 characters (found 81). [LineLength]
./hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/select/ITestS3SelectLandsat.java:69:
* download_url: HTTPS URL
https://s3-us-west-2.amazonaws.com/landsat-pds/L8/010/117/LC80101172015002LGN00/index.html:
Line is longer than 80 characters (found 119). [LineLength]
./hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/select/ITestS3SelectLandsat.java:75:
* <li>Standard Greenwich Meridian (not the french one which still
surfaces)</li>: Line is longer than 80 characters (found 83). [LineLength]
./hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/select/ITestS3SelectLandsat.java:76:
* <li>Cloud cover <i>Should</i> be 0-100, but there are some negative
ones.</li>: Line is longer than 80 characters (found 83). [LineLength]
./hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/select/ITestS3SelectLandsat.java:81:
entityId,acquisitionDate,cloudCover,processingLevel,path,row,min_lat,min_lon,max_lat,max_lon,download_url:
Line is longer than 80 characters (found 106). [LineLength]
./hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/select/ITestS3SelectLandsat.java:82:
* LC80101172015002LGN00,2015-01-02
15:49:05.571384,80.81,L1GT,10,117,-79.09923,-139.66082,-77.7544,-125.09297,https://s3-us-west-2.amazonaws.com/landsat-pds/L8/010/117/LC80101172015002LGN00/index.html:
Line is longer than 80 characters (found 201). [LineLength]
./hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/select/ITestS3SelectLandsat.java:83:
* LC80260392015002LGN00,2015-01-02
16:56:51.399666,90.84,L1GT,26,39,29.23106,-97.48576,31.36421,-95.16029,https://s3-us-west-2.amazonaws.com/landsat-pds/L8/026/039/LC80260392015002LGN00/index.html:
Line is longer than 80 characters (found 197). [LineLength]
./hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/select/ITestS3SelectLandsat.java:84:
* LC82270742015002LGN00,2015-01-02
13:53:02.047000,83.44,L1GT,227,74,-21.28598,-59.27736,-19.17398,-57.07423,https://s3-us-west-2.amazonaws.com/landsat-pds/L8/227/074/LC82270742015002LGN00/index.html:
Line is longer than 80 characters (found 200). [LineLength]
./hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/select/ITestS3SelectLandsat.java:85:
* LC82270732015002LGN00,2015-01-02
13:52:38.110317,52.29,L1T,227,73,-19.84365,-58.93258,-17.73324,-56.74692,https://s3-us-west-2.amazonaws.com/landsat-pds/L8/227/073/LC82270732015002LGN00/index.html:
Line is longer than 80 characters (found 199). [LineLength]
{code}
javadoc is an existing issue; maybe its time to fix as long as it doesn't
create merge conflict
{code}
[WARNING]
/testptch/hadoop/hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/s3guard/S3GuardTool.java:392:
warning - Tag @link: reference not found: ExitUtil.ExitException
{code}
> Add FileSystem builder-based openFile() API to match createFile()
> -----------------------------------------------------------------
>
> Key: HADOOP-15229
> URL: https://issues.apache.org/jira/browse/HADOOP-15229
> Project: Hadoop Common
> Issue Type: New Feature
> Components: fs, fs/azure, fs/s3
> Affects Versions: 3.0.0
> Reporter: Steve Loughran
> Assignee: Steve Loughran
> Priority: Major
> Attachments: HADOOP-15229-001.patch, HADOOP-15229-002.patch,
> HADOOP-15229-003.patch, HADOOP-15229-004.patch, HADOOP-15229-004.patch,
> HADOOP-15229-005.patch, HADOOP-15229-006.patch, HADOOP-15229-007.patch,
> HADOOP-15229-009.patch
>
>
> Replicate HDFS-1170 and HADOOP-14365 with an API to open files.
> A key requirement of this is not HDFS, it's to put in the fadvise policy for
> working with object stores, where getting the decision to do a full GET and
> TCP abort on seek vs smaller GETs is fundamentally different: the wrong
> option can cost you minutes. S3A and Azure both have adaptive policies now
> (first backward seek), but they still don't do it that well.
> Columnar formats (ORC, Parquet) should be able to say "fs.input.fadvise"
> "random" as an option when they open files; I can imagine other options too.
> The Builder model of [~eddyxu] is the one to mimic, method for method.
> Ideally with as much code reuse as possible
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
