[
https://issues.apache.org/jira/browse/HADOOP-18724?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17724852#comment-17724852
]
ASF subversion and git services commented on HADOOP-18724:
----------------------------------------------------------
Commit e76c09ac3bd3b01522c1d9b06b18c9452e774b33 in hadoop's branch
refs/heads/feature-HADOOP-18073-s3a-sdk-upgrade from Steve Loughran
[ https://gitbox.apache.org/repos/asf?p=hadoop.git;h=e76c09ac3bd ]
HADOOP-18724. Open file fails with NumberFormatException for S3AFileSystem
(#5611)
This:
1. Adds optLong, optDouble, mustLong and mustDouble
methods to the FSBuilder interface to let callers explicitly
passin long and double arguments.
2. The opt() and must() builder calls which take float/double values
now only set long values instead, so as to avoid problems
related to overloaded methods resulting in a ".0" being appended
to a long value.
3. All of the relevant opt/must calls in the hadoop codebase move to
the new methods
4. And the s3a code is resilient to parse errors in is numeric options
-it will downgrade to the default.
This is nominally incompatible, but the floating-point builder methods
were never used: nothing currently expects floating point numbers.
For anyone who wants to safely set numeric builder options across all compatible
releases, convert the number to a string and then use the opt(String, String)
and must(String, String) methods.
Contributed by Steve Loughran
> Open file fails with NumberFormatException for S3AFileSystem
> ------------------------------------------------------------
>
> Key: HADOOP-18724
> URL: https://issues.apache.org/jira/browse/HADOOP-18724
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs, fs/azure, fs/s3
> Affects Versions: 3.3.5
> Reporter: Ayush Saxena
> Assignee: Steve Loughran
> Priority: Critical
> Labels: pull-request-available
> Fix For: 3.4.0, 3.3.9
>
>
> Saw the trace for Hive-Iceberg, was using the old client, doesn't happen once
> I upgraded.
> {noformat}
> Caused by: java.lang.NumberFormatException: For input string: "5783.0"
> at
> java.base/java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
> at java.base/java.lang.Long.parseLong(Long.java:692)
> at java.base/java.lang.Long.parseLong(Long.java:817)
> at org.apache.hadoop.conf.Configuration.getLong(Configuration.java:1601)
> at
> org.apache.hadoop.fs.s3a.impl.OpenFileSupport.prepareToOpenFile(OpenFileSupport.java:262)
> at
> org.apache.hadoop.fs.s3a.S3AFileSystem.openFileWithOptions(S3AFileSystem.java:5219)
> at
> org.apache.hadoop.fs.FileSystem$FSDataInputStreamBuilder.build(FileSystem.java:4753)
> at
> org.apache.iceberg.hadoop.HadoopInputFile.newStream(HadoopInputFile.java:196)
> at
> org.apache.iceberg.avro.AvroIterable.newFileReader(AvroIterable.java:101)
> at
> org.apache.iceberg.avro.AvroIterable.getMetadata(AvroIterable.java:66){noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
