Haejoon Lee created SPARK-46820:
-----------------------------------
Summary: Improve error message when createDataFrame have illegal
nullable
Key: SPARK-46820
URL: https://issues.apache.org/jira/browse/SPARK-46820
Project: Spark
Issue Type: Sub-task
Components: PySpark
Affects Versions: 4.0.0
Reporter: Haejoon Lee
>>> from pyspark.sql.types import StructType, StructField, StringType,
>>> IntegerType
>>> schema = StructType([
... StructField("name", StringType(), nullable=True),
... StructField("age", IntegerType(), nullable=False)
... ])
>>> df = spark.createDataFrame([("asd", None])], schema)
pyspark.errors.exceptions.base.PySparkValueError: [CANNOT_BE_NONE] Argument
`obj` cannot be None.
The error message in the example above says "obj", but createDataFrame function
has not "obj" reference. We should fix this error message properly.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
