[
https://issues.apache.org/jira/browse/HADOOP-18724?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17718087#comment-17718087
]
Steve Loughran commented on HADOOP-18724:
-----------------------------------------
Ok, I see the problem, adding opt(str, double) alongside opt(string, long) is
causing the compiler to (how?) choose the more relaxed type over the one of the
type passed in. This must be some quirk of numeric type inference which is
possibly different from the rest of java overloading.
This is bad. I think it'll be from HADOOP-16202, and only going to hit code
compiled against 3.3.5, not 3.3.3-.3.3.4.
Immediate Workaround for apps: use opt(string, Long.toString(l)) to convert the
string.p
I have an idea for a fix for this which is
* add explicit optLong() optDouble() calls
* have all the existing opt(string, long), double and float calls *forward to
`optLong()`
this means existing code can't set doubles, but we can be confident that
whatever overloaded method is linked to, a long is always set.
----
what breaks?
* nothing can set opt(string, double) but AFAIK nothing does this, it was added
due to some perceived need for it, not an actual one at present
* I've been doing some passthrough code which collects this for some testing on
hbase-fs; not sure of its state
* my WiP fs api should go for the new setters only.
https://github.com/steveloughran/fs-api-shim/blob/main/fs-api-shim-library/src/main/java/org/apache/hadoop/fs/shim/api/FSBuilder.java
Shall I do this?
> Open file fails with NumberFormatException for S3AFileSystem
> ------------------------------------------------------------
>
> Key: HADOOP-18724
> URL: https://issues.apache.org/jira/browse/HADOOP-18724
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs/s3
> Affects Versions: 3.3.5
> Reporter: Ayush Saxena
> Assignee: Steve Loughran
> Priority: Major
>
> Saw the trace for Hive-Iceberg, was using the old client, doesn't happen once
> I upgraded.
> {noformat}
> Caused by: java.lang.NumberFormatException: For input string: "5783.0"
> at
> java.base/java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
> at java.base/java.lang.Long.parseLong(Long.java:692)
> at java.base/java.lang.Long.parseLong(Long.java:817)
> at org.apache.hadoop.conf.Configuration.getLong(Configuration.java:1601)
> at
> org.apache.hadoop.fs.s3a.impl.OpenFileSupport.prepareToOpenFile(OpenFileSupport.java:262)
> at
> org.apache.hadoop.fs.s3a.S3AFileSystem.openFileWithOptions(S3AFileSystem.java:5219)
> at
> org.apache.hadoop.fs.FileSystem$FSDataInputStreamBuilder.build(FileSystem.java:4753)
> at
> org.apache.iceberg.hadoop.HadoopInputFile.newStream(HadoopInputFile.java:196)
> at
> org.apache.iceberg.avro.AvroIterable.newFileReader(AvroIterable.java:101)
> at
> org.apache.iceberg.avro.AvroIterable.getMetadata(AvroIterable.java:66){noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
