[jira] [Commented] (SPARK-17808) BinaryType fails in Python 3 due to outdated Pyrolite
[ https://issues.apache.org/jira/browse/SPARK-17808?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15566107#comment-15566107 ] Sean Owen commented on SPARK-17808: --- I think it could be OK. It's a bug fix, and while it is a minor version bump to a dependency in a maintenance release, it looks like a small change. > BinaryType fails in Python 3 due to outdated Pyrolite > - > > Key: SPARK-17808 > URL: https://issues.apache.org/jira/browse/SPARK-17808 > Project: Spark > Issue Type: Bug > Components: PySpark >Affects Versions: 2.0.1 > Environment: spark-2.0.1-bin-hadoop2.7 with Python 3.4.3 on Ubuntu > 14.04.4 LTS >Reporter: Pete Fein >Assignee: Bryan Cutler > Fix For: 2.1.0 > > Attachments: demo.py, demo_output.txt > > > Attempting to create a DataFrame using a BinaryType field fails under Python > 3 because the underlying Pyrolite library is out of date. Spark appears to be > using Pyrolite 4.9; this issue was fixed in Pyrolite 4.12. See [original bug > report|https://github.com/irmen/Pyrolite/issues/36] and > [patch|https://github.com/irmen/Pyrolite/commit/eec11786746d933b9d2c3eaeb1e1486319ae436e] > Test case & output attached. I'm just a Python guy, not really sure how to > build Spark / do classpath magic to test if this works correctly with updated > Pyrolite. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-17808) BinaryType fails in Python 3 due to outdated Pyrolite
[ https://issues.apache.org/jira/browse/SPARK-17808?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15565470#comment-15565470 ] Pete Fein commented on SPARK-17808: --- Any reason this can't be included in the next 2.0.x bug fix release? > BinaryType fails in Python 3 due to outdated Pyrolite > - > > Key: SPARK-17808 > URL: https://issues.apache.org/jira/browse/SPARK-17808 > Project: Spark > Issue Type: Bug > Components: PySpark >Affects Versions: 2.0.1 > Environment: spark-2.0.1-bin-hadoop2.7 with Python 3.4.3 on Ubuntu > 14.04.4 LTS >Reporter: Pete Fein >Assignee: Bryan Cutler > Fix For: 2.1.0 > > Attachments: demo.py, demo_output.txt > > > Attempting to create a DataFrame using a BinaryType field fails under Python > 3 because the underlying Pyrolite library is out of date. Spark appears to be > using Pyrolite 4.9; this issue was fixed in Pyrolite 4.12. See [original bug > report|https://github.com/irmen/Pyrolite/issues/36] and > [patch|https://github.com/irmen/Pyrolite/commit/eec11786746d933b9d2c3eaeb1e1486319ae436e] > Test case & output attached. I'm just a Python guy, not really sure how to > build Spark / do classpath magic to test if this works correctly with updated > Pyrolite. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-17808) BinaryType fails in Python 3 due to outdated Pyrolite
[ https://issues.apache.org/jira/browse/SPARK-17808?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15553685#comment-15553685 ] Apache Spark commented on SPARK-17808: -- User 'BryanCutler' has created a pull request for this issue: https://github.com/apache/spark/pull/15386 > BinaryType fails in Python 3 due to outdated Pyrolite > - > > Key: SPARK-17808 > URL: https://issues.apache.org/jira/browse/SPARK-17808 > Project: Spark > Issue Type: Bug > Components: PySpark >Affects Versions: 2.0.1 > Environment: spark-2.0.1-bin-hadoop2.7 with Python 3.4.3 on Ubuntu > 14.04.4 LTS >Reporter: Pete Fein > Attachments: demo.py, demo_output.txt > > > Attempting to create a DataFrame using a BinaryType field fails under Python > 3 because the underlying Pyrolite library is out of date. Spark appears to be > using Pyrolite 4.9; this issue was fixed in Pyrolite 4.12. See [original bug > report|https://github.com/irmen/Pyrolite/issues/36] and > [patch|https://github.com/irmen/Pyrolite/commit/eec11786746d933b9d2c3eaeb1e1486319ae436e] > Test case & output attached. I'm just a Python guy, not really sure how to > build Spark / do classpath magic to test if this works correctly with updated > Pyrolite. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-17808) BinaryType fails in Python 3 due to outdated Pyrolite
[ https://issues.apache.org/jira/browse/SPARK-17808?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15552975#comment-15552975 ] Pete Fein commented on SPARK-17808: --- [~bryanc] go for it - thanks! > BinaryType fails in Python 3 due to outdated Pyrolite > - > > Key: SPARK-17808 > URL: https://issues.apache.org/jira/browse/SPARK-17808 > Project: Spark > Issue Type: Bug > Components: PySpark >Affects Versions: 2.0.1 > Environment: spark-2.0.1-bin-hadoop2.7 with Python 3.4.3 on Ubuntu > 14.04.4 LTS >Reporter: Pete Fein > Attachments: demo.py, demo_output.txt > > > Attempting to create a DataFrame using a BinaryType field fails under Python > 3 because the underlying Pyrolite library is out of date. Spark appears to be > using Pyrolite 4.9; this issue was fixed in Pyrolite 4.12. See [original bug > report|https://github.com/irmen/Pyrolite/issues/36] and > [patch|https://github.com/irmen/Pyrolite/commit/eec11786746d933b9d2c3eaeb1e1486319ae436e] > Test case & output attached. I'm just a Python guy, not really sure how to > build Spark / do classpath magic to test if this works correctly with updated > Pyrolite. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-17808) BinaryType fails in Python 3 due to outdated Pyrolite
[ https://issues.apache.org/jira/browse/SPARK-17808?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15552787#comment-15552787 ] Bryan Cutler commented on SPARK-17808: -- I was able to reproduce this [~pfein], I can make a PR to upgrade Pyrolite unless you are planning on doing so? > BinaryType fails in Python 3 due to outdated Pyrolite > - > > Key: SPARK-17808 > URL: https://issues.apache.org/jira/browse/SPARK-17808 > Project: Spark > Issue Type: Bug > Components: PySpark >Affects Versions: 2.0.1 > Environment: spark-2.0.1-bin-hadoop2.7 with Python 3.4.3 on Ubuntu > 14.04.4 LTS >Reporter: Pete Fein > Attachments: demo.py, demo_output.txt > > > Attempting to create a DataFrame using a BinaryType field fails under Python > 3 because the underlying Pyrolite library is out of date. Spark appears to be > using Pyrolite 4.9; this issue was fixed in Pyrolite 4.12. See [original bug > report|https://github.com/irmen/Pyrolite/issues/36] and > [patch|https://github.com/irmen/Pyrolite/commit/eec11786746d933b9d2c3eaeb1e1486319ae436e] > Test case & output attached. I'm just a Python guy, not really sure how to > build Spark / do classpath magic to test if this works correctly with updated > Pyrolite. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
