Nice work Shane! That all sounds good to me. We might want to use pyarrow 0.12.1 though, there is a major bug that was fixed, but we can discuss in the PR. I will put up the code changes in the next few days. Felix, I think you're right about Python 3.5, they just list one upcoming release and that's not necessarily the last. Comparing the histories, it might still be soon though. I think using 3.6 will be fine, as a point of reference, pyarrow CI uses 2.7 and 3.6.
On Mon, Apr 1, 2019 at 3:09 PM shane knapp <skn...@berkeley.edu> wrote: > well now! color me completely surprised... i decided to whip up a fresh > python3.6.8 conda environment this morning to "see if things just worked". > > well, apparently they do! :) > > regardless, this is pretty awesome news as i will be able to easily update > the 'py3k' python3.4 environment to a fresh, less bloated, but still > package-complete python3.6.8 environment (including pyarrow 0.12.0, pandas > 0.24.2, scipy 1.2.1). > > i tested this pretty extensively today on both the ubuntu and centos > workers, and i think i'm ready to pull the trigger for a build-system-wide > upgrade... however, i'll be out wednesday through friday this week and > don't want to make a massive change before disappearing for a few days. > > so: how does early next week sound for the python upgrade? :) > > shane > > On Mon, Apr 1, 2019 at 8:58 AM shane knapp <skn...@berkeley.edu> wrote: > >> i'd much prefer that we minimize the number of python versions that we >> test against... would 2.7 and 3.6 be sufficient? >> >> On Fri, Mar 29, 2019 at 10:23 PM Felix Cheung <felixcheun...@hotmail.com> >> wrote: >> >>> I don’t take it as Sept 2019 is end of life for python 3.5 tho. It’s >>> just saying the next release. >>> >>> In any case I think in the next release it will be great to get more >>> Python 3.x release test coverage. >>> >>> >>> >>> ------------------------------ >>> *From:* shane knapp <skn...@berkeley.edu> >>> *Sent:* Friday, March 29, 2019 4:46 PM >>> *To:* Bryan Cutler >>> *Cc:* Felix Cheung; Hyukjin Kwon; dev >>> *Subject:* Re: Upgrading minimal PyArrow version to 0.12.x >>> [SPARK-27276] >>> >>> i'm not opposed to 3.6 at all. >>> >>> On Fri, Mar 29, 2019 at 4:16 PM Bryan Cutler <cutl...@gmail.com> wrote: >>> >>>> PyArrow dropping Python 3.4 was mainly due to support going away at >>>> Conda-Forge and other dependencies also dropping it. I think we better >>>> upgrade Jenkins Python while we are at it. Are you all against jumping to >>>> Python 3.6 so we are not in the same boat in September? >>>> >>>> On Thu, Mar 28, 2019 at 7:58 PM Felix Cheung <felixcheun...@hotmail.com> >>>> wrote: >>>> >>>>> 3.4 is end of life but 3.5 is not. From your link >>>>> >>>>> we expect to release Python 3.5.8 around September 2019. >>>>> >>>>> >>>>> >>>>> ------------------------------ >>>>> *From:* shane knapp <skn...@berkeley.edu> >>>>> *Sent:* Thursday, March 28, 2019 7:54 PM >>>>> *To:* Hyukjin Kwon >>>>> *Cc:* Bryan Cutler; dev; Felix Cheung >>>>> *Subject:* Re: Upgrading minimal PyArrow version to 0.12.x >>>>> [SPARK-27276] >>>>> >>>>> looks like the same for 3.5... >>>>> https://www.python.org/dev/peps/pep-0478/ >>>>> >>>>> let's pick a python version and start testing. >>>>> >>>>> On Thu, Mar 28, 2019 at 7:52 PM shane knapp <skn...@berkeley.edu> >>>>> wrote: >>>>> >>>>>> >>>>>>> If there was, it looks inevitable to upgrade Jenkins\s Python from >>>>>>> 3.4 to 3.5. >>>>>>> >>>>>>> this is inevitable. 3.4s final release was 10 days ago ( >>>>>> https://www.python.org/dev/peps/pep-0429/) so we're basically EOL. >>>>>> >>>>> >>>>> >>>>> -- >>>>> Shane Knapp >>>>> UC Berkeley EECS Research / RISELab Staff Technical Lead >>>>> https://rise.cs.berkeley.edu >>>>> >>>> >>> >>> -- >>> Shane Knapp >>> UC Berkeley EECS Research / RISELab Staff Technical Lead >>> https://rise.cs.berkeley.edu >>> >> >> >> -- >> Shane Knapp >> UC Berkeley EECS Research / RISELab Staff Technical Lead >> https://rise.cs.berkeley.edu >> > > > -- > Shane Knapp > UC Berkeley EECS Research / RISELab Staff Technical Lead > https://rise.cs.berkeley.edu >
