Jai Kumar Singh created SPARK-3008:
--------------------------------------
Summary: PySpark fails due to zipimport not able to load the
assembly jar (/usr/bin/python: No module named pyspark)
Key: SPARK-3008
URL: https://issues.apache.org/jira/browse/SPARK-3008
Project: Spark
Issue Type: Bug
Components: PySpark
Environment: Assemebly Jar
target/scala-2.10/spark-assembly-1.1.0-SNAPSHOT-hadoop2.2.0.jar
jar -tf
assembly/target/scala-2.10/spark-assembly-1.1.0-SNAPSHOT-hadoop2.2.0.jar | wc -l
70441
git sha commit ba28a8fcbc3ba432e7ea4d6f0b535450a6ec96c6
Reporter: Jai Kumar Singh
PySpark is not working. It fails because zipimport not able to import assembly
jar because that contain more than 65536 files.
Email chains in this regard are below
http://mail-archives.apache.org/mod_mbox/incubator-spark-user/201406.mbox/%3ccamjob8kcgk0pqiogju6uokceyswcusw3xwd5wrs8ikpmgd2...@mail.gmail.com%3E
https://mail.python.org/pipermail/python-list/2014-May/671353.html
Is there any work around to bypass the issue ?
--
This message was sent by Atlassian JIRA
(v6.2#6252)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
