[
https://issues.apache.org/jira/browse/SPARK-21327?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Apache Spark reassigned SPARK-21327:
------------------------------------
Assignee: Apache Spark
> ArrayConstructor should handle an array of typecode 'l' as long rather than
> int in Python 2.
> --------------------------------------------------------------------------------------------
>
> Key: SPARK-21327
> URL: https://issues.apache.org/jira/browse/SPARK-21327
> Project: Spark
> Issue Type: Bug
> Components: PySpark, SQL
> Affects Versions: 2.2.0
> Reporter: Takuya Ueshin
> Assignee: Apache Spark
>
> Currently {{ArrayConstructor}} handles an array of typecode {{'l'}} as
> {{int}} when converting Python object in Python 2 into Java object, so if the
> value is larger than {{Integer.MAX_VALUE}} or smaller than
> {{Integer.MIN_VALUE}} then the overflow occurs.
> {code}
> import array
> data = [Row(l=array.array('l', [-9223372036854775808, 0,
> 9223372036854775807]))]
> df = spark.createDataFrame(data)
> df.show(truncate=False)
> {code}
> {code}
> +----------+
> |l |
> +----------+
> |[0, 0, -1]|
> +----------+
> {code}
> This should be:
> {code}
> +----------------------------------------------+
> |l |
> +----------------------------------------------+
> |[-9223372036854775808, 0, 9223372036854775807]|
> +----------------------------------------------+
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
