well... i've been running in to problems (aka dependency hell), and just hit a show-stopper:
UnsatisfiableError: The following specifications were found to be in conflict: - pyarrow 0.10.* -> arrow-cpp 0.10.0.* -> python >=2.7,<2.8.0a0 - python 3.4* Use "conda info <package>" to see the dependencies for each package. yep, we're testing against python 3.4, and pyarrow 0.10.0 needs 3.5+ putting the workers back on-line until i figure out what to do next. shane On Wed, Aug 8, 2018 at 10:31 AM, shane knapp <skn...@berkeley.edu> wrote: > pyarrow 0.10.0 has been released, and this is important to be tested > against for the 2.4 release (esp due to memory leak problems, etc) > > https://issues.apache.org/jira/browse/SPARK-23874 > https://github.com/apache/spark/pull/21939 > https://issues.apache.org/jira/browse/ARROW-1973 > > i will be putting the centos workers in to quiet mode now, prepping the > upgrade and once the majority of existing builds are done, i'll kill any > outliers (and retrigger them later) and perform the upgrade. > > expect this to take ~3 hours, max. > > shane > -- > Shane Knapp > UC Berkeley EECS Research / RISELab Staff Technical Lead > https://rise.cs.berkeley.edu > -- Shane Knapp UC Berkeley EECS Research / RISELab Staff Technical Lead https://rise.cs.berkeley.edu
