Kevin Cox created SPARK-11319:
---------------------------------
Summary: PySpark silently Accepts null values in non-nullable
DataFrame fields.
Key: SPARK-11319
URL: https://issues.apache.org/jira/browse/SPARK-11319
Project: Spark
Issue Type: Bug
Components: PySpark, SQL
Reporter: Kevin Cox
Running the following code with a null value in a non-nullable column silently
works. This makes the code incredibly hard to trust.
{code}
In [2]: from pyspark.sql.types import *
In [3]: sqlContext.createDataFrame([(None,)], StructType([StructField("a",
TimestampType(), False)])).collect()
Out[3]: [Row(a=None)]
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
