Steve Loughran created HADOOP-18738:
---------------------------------------
Summary: Error message when multipart disabled and buffering is
array not useful
Key: HADOOP-18738
URL: https://issues.apache.org/jira/browse/HADOOP-18738
Project: Hadoop Common
Issue Type: Improvement
Components: fs/s3
Affects Versions: 3.3.9
Reporter: Steve Loughran
Assignee: Harshit Gupta
testing branch-3.3 on a bucket with array buffering, I get a fairly useless
error message.
Problem here is that AbstractCommitITest forces fs.s3a.fast.upload.buffer to
array and somehow multipart is disabled.
# error text needs fixing (move to %s)
# s3a fs should fail in init if the threshold is isn't allowed for the store
type
# then determine why the test is failing, and fix.
The tests do work in trunk, so something in branch-3.3 is stopping this
Also
* correct the name of the constant
STORE_CAPABILITY_DIRECTORY_MARKER_MULTIPART_UPLOAD_ENABLED to match the actual
string "fs.s3a.capability.multipart.uploads.enabled"; accidental copy there
* capability FS_MULTIPART_UPLOADER must declare itself as disabled when
multipart is off
{code}
[ERROR] testRevertCommit(org.apache.hadoop.fs.s3a.commit.ITestCommitOperations)
Time elapsed: 0.688 s <<< ERROR!
java.lang.IllegalArgumentException: Invalid block size: %d [-1]
at
org.apache.hadoop.thirdparty.com.google.common.base.Preconditions.checkArgument(Preconditions.java:205)
at
org.apache.hadoop.fs.s3a.S3ADataBlocks$ArrayBlockFactory.create(S3ADataBlocks.java:397)
at
org.apache.hadoop.fs.s3a.S3ABlockOutputStream.createBlockIfNeeded(S3ABlockOutputStream.java:235)
at
org.apache.hadoop.fs.s3a.S3ABlockOutputStream.<init>(S3ABlockOutputStream.java:217)
at
org.apache.hadoop.fs.s3a.S3AFileSystem.innerCreateFile(S3AFileSystem.java:1887)
at
org.apache.hadoop.fs.s3a.S3AFileSystem.lambda$create$7(S3AFileSystem.java:1789)
at
org.apache.hadoop.fs.statistics.impl.IOStatisticsBinding.invokeTrackingDuration(IOStatisticsBinding.java:547)
at
org.apache.hadoop.fs.statistics.impl.IOStatisticsBinding.lambda$trackDurationOfOperation$5(IOStatisticsBinding.java:528)
at
org.apache.hadoop.fs.statistics.impl.IOStatisticsBinding.trackDuration(IOStatisticsBinding.java:449)
at
org.apache.hadoop.fs.s3a.S3AFileSystem.trackDurationAndSpan(S3AFileSystem.java:2479)
at
org.apache.hadoop.fs.s3a.S3AFileSystem.trackDurationAndSpan(S3AFileSystem.java:2498)
at
org.apache.hadoop.fs.s3a.S3AFileSystem.create(S3AFileSystem.java:1788)
at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:1233)
at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:1210)
at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:1091)
at
org.apache.hadoop.fs.contract.ContractTestUtils.createFile(ContractTestUtils.java:650)
at
org.apache.hadoop.fs.contract.ContractTestUtils.touch(ContractTestUtils.java:686)
at
org.apache.hadoop.fs.s3a.commit.ITestCommitOperations.testRevertCommit(ITestCommitOperations.java:543)
{code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
