Re: PySpark: preference for Python 2.7 or Python 3.5?

[Ian Stokes Rees]

On 9/2/16 3:47 AM, Felix Cheung wrote:

There is an Anaconda parcel one could readily install on CDH

https://docs.continuum.io/anaconda/cloudera

As Sean says it is Python 2.7.x.

Spark should work for both 2.7 and 3.5.


Yes, I'm actually an engineer at Continuum, so I know the Anaconda 
parcel pretty well.  It is more a question of whether CDH and Spark 
"work" better with PySpark on Python 2.7 or Python 3.5.  My sense was 
"you choose: both are fine", but I wanted to ask here before committing 
to going down one path or another.


Thanks,

Ian

Re: PySpark: preference for Python 2.7 or Python 3.5?

[Felix Cheung]

There is an Anaconda parcel one could readily install on CDH

https://docs.continuum.io/anaconda/cloudera

As Sean says it is Python 2.7.x.

Spark should work for both 2.7 and 3.5.

_
From: Sean Owen <so...@cloudera.com<mailto:so...@cloudera.com>>
Sent: Friday, September 2, 2016 12:41 AM
Subject: Re: PySpark: preference for Python 2.7 or Python 3.5?
To: Ian Stokes Rees <ijsto...@continuum.io<mailto:ijsto...@continuum.io>>
Cc: user @spark <user@spark.apache.org<mailto:user@spark.apache.org>>


Spark should work fine with Python 3. I'm not a Python person, but all else 
equal I'd use 3.5 too. I assume the issue could be libraries you want that 
don't support Python 3. I don't think that changes with CDH. It includes a 
version of Anaconda from Continuum, but that lays down Python 2.7.11. I don't 
believe there's any particular position on 2 vs 3.

On Fri, Sep 2, 2016 at 3:56 AM, Ian Stokes Rees 
<ijsto...@continuum.io<mailto:ijsto...@continuum.io>> wrote:
I have the option of running PySpark with Python 2.7 or Python 3.5.  I am 
fairly expert with Python and know the Python-side history of the differences.  
All else being the same, I have a preference for Python 3.5.  I'm using CDH 5.8 
and I'm wondering if that biases whether I should proceed with PySpark on top 
of Python 2.7 or 3.5.  Opinions?  Does Cloudera have an official (or 
unofficial) position on this?

Thanks,

Ian
___
Ian Stokes-Rees
Computational Scientist

[Continuum Analytics]<http://continuum.io>
@ijstokes [Twitter] <http://twitter.com/ijstokes>  [LinkedIn] 
<http://linkedin.com/in/ijstokes>  [Github] <http://github.com/ijstokes>  
617.942.0218

Re: PySpark: preference for Python 2.7 or Python 3.5?

[Sean Owen]

Spark should work fine with Python 3. I'm not a Python person, but all else
equal I'd use 3.5 too. I assume the issue could be libraries you want that
don't support Python 3. I don't think that changes with CDH. It includes a
version of Anaconda from Continuum, but that lays down Python 2.7.11. I
don't believe there's any particular position on 2 vs 3.

On Fri, Sep 2, 2016 at 3:56 AM, Ian Stokes Rees 
wrote:

> I have the option of running PySpark with Python 2.7 or Python 3.5.  I am
> fairly expert with Python and know the Python-side history of the
> differences.  All else being the same, I have a preference for Python 3.5.
> I'm using CDH 5.8 and I'm wondering if that biases whether I should proceed
> with PySpark on top of Python 2.7 or 3.5.  Opinions?  Does Cloudera have an
> official (or unofficial) position on this?
>
> Thanks,
>
> Ian
> ___
> Ian Stokes-Rees
> Computational Scientist
>
> [image: Continuum Analytics] 
> @ijstokes [image: Twitter]  [image: LinkedIn]
>  [image: Github]
>  617.942.0218
>

PySpark: preference for Python 2.7 or Python 3.5?

[Ian Stokes Rees]

I have the option of running PySpark with Python 2.7 or Python 3.5. I am 
fairly expert with Python and know the Python-side history of the 
differences.  All else being the same, I have a preference for Python 
3.5.  I'm using CDH 5.8 and I'm wondering if that biases whether I 
should proceed with PySpark on top of Python 2.7 or 3.5. Opinions?  Does 
Cloudera have an official (or unofficial) position on this?


Thanks,

Ian
___
Ian Stokes-Rees
Computational Scientist

Continuum Analytics 
@ijstokes Twitter  LinkedIn 
 Github 
617.942.0218