Hi Nick, All the nodes of the cluster need to have the same Python setup (path and version). So if, e.g. you start running in 2.7.5 on the master and it ships code to nodes that have 2.6.x, you'll get invalid opcode errors.
Thanks, Bryn On Thu, Feb 27, 2014 at 3:48 PM, nicholas.chammas < [email protected]> wrote: > The provided Spark EC2 > scripts<https://spark.incubator.apache.org/docs/0.9.0/ec2-scripts.html>and > default AMI ship with Python 2.6.8. > > I would like to use Python 2.7.5 or later. I believe that among the 2.x > versions, 2.7 is the most popular. > > What's the easiest way to get my Spark cluster on Python 2.7 and working > correctly? > > For starters, I naively upgraded Python on the master node using the > instructions > here<http://www.lecloud.net/post/61401763496/install-update-to-python-2-7-and-latest-pip-on-ec2>. > The pyspark shell started up fine but a basic map on an RDD that used to > work fine failed. Perhaps I just need to upgrade all the nodes in my > cluster, and perhaps it's not that simple. I thought I'd check here first. > > Nick > > > ------------------------------ > View this message in context: Running Spark with Python > 2.7.5+<http://apache-spark-user-list.1001560.n3.nabble.com/Running-Spark-with-Python-2-7-5-tp2131.html> > Sent from the Apache Spark User List mailing list > archive<http://apache-spark-user-list.1001560.n3.nabble.com/>at Nabble.com. >
