wujinhu commented on PR #6904:
URL: https://github.com/apache/hadoop/pull/6904#issuecomment-2205923493
> reviewed, only some minor comments, as you have based this on the most
recent s3a work
>
> In ITestS3AContractVectoredRead we added a test to check what happens if
you call openFile(), pass in a length shorter than that of the object and see
what breaks. Do you need to worry about this?
AliyunOSSFileSystem does not override the `FileSystem::openFileWithOptions`
interface, so it does not read the length passed by openFile() call. And I got
errors below:
```
[INFO] Running
org.apache.hadoop.fs.aliyun.oss.contract.TestAliyunOSSContractVectoredRead
[ERROR] Tests run: 48, Failures: 0, Errors: 2, Skipped: 0, Time elapsed:
171.493 s <<< FAILURE! - in
org.apache.hadoop.fs.aliyun.oss.contract.TestAliyunOSSContractVectoredRead
[ERROR] testEOFRanges416Handling[Buffer type :
direct](org.apache.hadoop.fs.aliyun.oss.contract.TestAliyunOSSContractVectoredRead)
Time elapsed: 3.76 s <<< ERROR!
java.io.EOFException: Range starts beyond the file length (65536): range
[65536-65636], length=100, reference=null
at
org.apache.hadoop.fs.VectoredReadUtils.validateAndSortRanges(VectoredReadUtils.java:330)
at
org.apache.hadoop.fs.aliyun.oss.AliyunOSSInputStream.readVectored(AliyunOSSInputStream.java:566)
at
org.apache.hadoop.fs.FSDataInputStream.readVectored(FSDataInputStream.java:299)
at
org.apache.hadoop.fs.aliyun.oss.contract.TestAliyunOSSContractVectoredRead.testEOFRanges416Handling(TestAliyunOSSContractVectoredRead.java:85)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
at
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at
org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
at
org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:61)
at
org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:299)
at
org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:293)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.lang.Thread.run(Thread.java:748)
[ERROR] testEOFRanges416Handling[Buffer type :
array](org.apache.hadoop.fs.aliyun.oss.contract.TestAliyunOSSContractVectoredRead)
Time elapsed: 3.686 s <<< ERROR!
java.io.EOFException: Range starts beyond the file length (65536): range
[65536-65636], length=100, reference=null
at
org.apache.hadoop.fs.VectoredReadUtils.validateAndSortRanges(VectoredReadUtils.java:330)
at
org.apache.hadoop.fs.aliyun.oss.AliyunOSSInputStream.readVectored(AliyunOSSInputStream.java:566)
at
org.apache.hadoop.fs.FSDataInputStream.readVectored(FSDataInputStream.java:299)
at
org.apache.hadoop.fs.aliyun.oss.contract.TestAliyunOSSContractVectoredRead.testEOFRanges416Handling(TestAliyunOSSContractVectoredRead.java:85)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
at
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at
org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
at
org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:61)
at
org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:299)
at
org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:293)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.lang.Thread.run(Thread.java:748)
[INFO]
[INFO] Results:
[INFO]
[ERROR] Errors:
[ERROR] TestAliyunOSSContractVectoredRead.testEOFRanges416Handling:85 »
EOF Range star...
[ERROR] TestAliyunOSSContractVectoredRead.testEOFRanges416Handling:85 »
EOF Range star...
[INFO]
[ERROR] Tests run: 48, Failures: 0, Errors: 2, Skipped: 0
```
I'm not sure whether I should implement the interface just like s3a does in
this patch or next patch. Could you please give some suggestions. Thanks. :)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
