M. Manna created SPARK-40282:
--------------------------------
Summary: IntegerType is missed in "ExternalDataTypeForInput"
function
Key: SPARK-40282
URL: https://issues.apache.org/jira/browse/SPARK-40282
Project: Spark
Issue Type: Bug
Components: Spark Core
Affects Versions: 3.3.0
Reporter: M. Manna
Attachments: SparkApplication.kt, retailstore.csv
*Problem Description*
as part of contract mentioned here, Spark should be able to support
{{IntegerType}} as an argument in StructType.add method. However, it complaints
with {{scala.MatchError}} today.
If we call the override version which access String value as Type e.g.
"Integer" - it works.
*How to Reproduce*
# Create a Kotlin Project - I have used Kotlin but Java will also work (needs
minor adjustment)
# Place the attached CSV file in {{src/main/resources}}
# Compile the project with Java 11
# Run - it will give you error.
# Now change line (commented as HERE) - to have a String value i.e. "Integer"
# It works
*Ask*
# Why does it not accept IntegerType, StringType as DataType as part of the
parameters supplied through {{add}} function in {{StructType}} ?
# If this is a bug, do we know when the fix can come?
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
