Punit Shah created SPARK-33445:
----------------------------------
Summary: Can't parse decimal type from csv file
Key: SPARK-33445
URL: https://issues.apache.org/jira/browse/SPARK-33445
Project: Spark
Issue Type: Bug
Components: PySpark, SQL
Affects Versions: 3.0.0, 2.4.7, 2.4.6
Reporter: Punit Shah
Attachments: tsd.csv
The attached file is a one column csv file containing decimals.
Execute: {color:#de350b}mydf2 = spark_session.read.csv("tsd.csv", header=True,
inferSchema=True){color}
Then invoking {color:#de350b}mydf2.schema{color} will result in error:
{color:#ff8b00}ValueError: Could not parse datatype: decimal(6,-7){color}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
