[ https://issues.apache.org/jira/browse/SPARK-21420?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Xiang Gao updated SPARK-21420: ------------------------------ Description: Python's array type {{q}} and {{Q}} are currently not supported by {{net.razorvine.pickle}}, and therefore not supported by spark yet. On some platform, {{q}} is a signed 64bit integer, which in theory should be supported because it has the same size as {{Long}} in Scala. See https://github.com/apache/spark/pull/18444#discussion_r126072461 This issue can be easily fixed by adding back deleted lines in https://github.com/apache/spark/pull/18444/commits/cafa5fdc22270847657a29b0470dfd0ca4ad4ad2 was: Python's array type {{q}} and {{Q}} are currently not supported by {{net.razorvine.pickle}}, and therefore not supported by spark yet. On some platform, {{q}} is a signed 64bit integer, which in theory should be supported because it has the same size as {{Long}} in Scala. See https://github.com/apache/spark/pull/18444#discussion_r126072461 This issue might be fixed by adding back deleted lines in https://github.com/apache/spark/pull/18444/commits/cafa5fdc22270847657a29b0470dfd0ca4ad4ad2 > Support array type code 'q' and 'Q' > ----------------------------------- > > Key: SPARK-21420 > URL: https://issues.apache.org/jira/browse/SPARK-21420 > Project: Spark > Issue Type: Improvement > Components: PySpark > Affects Versions: 2.1.1 > Reporter: Xiang Gao > Priority: Minor > > Python's array type {{q}} and {{Q}} are currently not supported by > {{net.razorvine.pickle}}, and therefore not supported by spark yet. > On some platform, {{q}} is a signed 64bit integer, which in theory should be > supported because it has the same size as {{Long}} in Scala. > See https://github.com/apache/spark/pull/18444#discussion_r126072461 > This issue can be easily fixed by adding back deleted lines in > https://github.com/apache/spark/pull/18444/commits/cafa5fdc22270847657a29b0470dfd0ca4ad4ad2 -- This message was sent by Atlassian JIRA (v6.4.14#64029) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org