Thanks for looking into this Shane. If we can only have a single python 3 version, I agree 3.6 would be better than 3.5. Otherwise, ideally I think it would be nice to test all supported 3.x versions (latest micros should be fine).
On Mon, Aug 20, 2018 at 7:07 PM shane knapp <skn...@berkeley.edu> wrote: > initially, i'd like to just choose one version to have the primary tests > against, but i'm also not opposed to supporting more of a matrix. the > biggest problem i see w/this approach, however, is that of build monitoring > and long-term ownership. this is why we have a relatively restrictive > current deployment. > > another thing i've been noticing during this project is that we have a lot > of flaky tests... for instance, i'm literally having every other build > fail on my (relatively) up-to-date PRB fork: > https://amplab.cs.berkeley.edu/jenkins/job/ubuntuSparkPRB/ > > (i'm testing more than python here, otherwise i could just build a spark > distro and run the python tests against that) > > i'll also set up a 3.6 env tomorrow and start testing against that. i'm > pretty confident about 3.5, tho. > > shane > > On Mon, Aug 20, 2018 at 11:33 AM, Bryan Cutler <cutl...@gmail.com> wrote: > >> Thanks for looking into this Shane! If we are choosing a single python >> 3.x, I think 3.6 would be good. It might still be nice to test against >> other versions too, so we can catch any issues. Is it possible to have more >> exhaustive testing as part of a nightly or scheduled build? As a point of >> reference for Python 3.6, Arrow is using this version for CI. >> >> Bryan >> >> On Sun, Aug 19, 2018 at 9:49 PM Hyukjin Kwon <gurwls...@gmail.com> wrote: >> >>> Actually Python 3.7 is released ( >>> https://www.python.org/downloads/release/python-370/) too and I fixed >>> the compatibility issues accordingly - >>> https://github.com/apache/spark/pull/21714 >>> There has been an issue for 3.6 (comparing to lower versions of Python >>> including 3.5) - https://github.com/apache/spark/pull/16429 >>> >>> I am not yet sure what's the best matrix for it actually. In case of R, >>> we test lowest version in Jenkins and highest version via AppVeyor FWIW. >>> I don't have a strong preference opinion on this since we have been >>> having compatibility issues for each Python version. >>> >>> >>> 2018년 8월 14일 (화) 오전 4:15, shane knapp <skn...@berkeley.edu>님이 작성: >>> >>>> hey everyone! >>>> >>>> i was checking out the EOL/release cycle for python 3.5 and it looks >>>> like we'll have 3.5.6 released in early 2019. >>>> >>>> this got me to thinking: instead of 3.5, what about 3.6? >>>> >>>> i looked around, and according to the 'docs' and 'collective wisdom of >>>> the internets', 3.5 and 3.6 should be fully backwards-compatible w/3.4. >>>> >>>> of course, this needs to be taken w/a grain of salt, as we're mostly >>>> focused on actual python package requirements, rather than worrying about >>>> core python functionality. >>>> >>>> thoughts? comments? >>>> >>>> thanks in advance, >>>> >>>> shane >>>> -- >>>> Shane Knapp >>>> UC Berkeley EECS Research / RISELab Staff Technical Lead >>>> https://rise.cs.berkeley.edu >>>> >>> > > > -- > Shane Knapp > UC Berkeley EECS Research / RISELab Staff Technical Lead > https://rise.cs.berkeley.edu >
