Re: [build system] jenkins temporarily unreachable, fixing now

Hi, I could not access Jenkins via url https://amplab.cs.berkeley.edu/ anymore. May I know if the reverse proxy is still having issue? Many thanks Best regards, William On Wed, 10 Apr 2019 at 2:34 AM, shane knapp wrote: > the host serving the reverse proxy to jenkins is wedged and is being >

Re: [SPARK-25079] moving from python 3.4 to python 3.6.8, impacts all active branches

My guess is that branch 2.3, 2.4 still work with Python 3.6 even if we're not testing that. If we're only testing against 3.6, I guess we wouldn't detect if something broke for 3.4 which is still 'supported' in 2.3.x, 2.4.x? For 3.x, no problem, let's raise the minimum anyway. This could be fine

Re: [SPARK-25079] moving from python 3.4 to python 3.6.8, impacts all active branches

ok, i'll create a PR for each branch so we can test things that way, rather than backporting. the bigger question i have is how far back, branch-wise, are we willing to support w/regards to tests? On Wed, Apr 10, 2019 at 12:16 PM Sean Owen wrote: > In theory Spark 2.4 supports Python 3.4;

[SPARK-25079] moving from python 3.4 to python 3.6.8, impacts all active branches

details here (see most recent comments for current state of things): https://issues.apache.org/jira/browse/SPARK-25079 my PR for these changes: https://github.com/apache/spark/pull/24266 we're doing this because we need to support arrow 0.12.1: https://issues.apache.org/jira/browse/SPARK-27276

Re: [SPARK-25079] moving from python 3.4 to python 3.6.8, impacts all active branches

In theory Spark 2.4 supports Python 3.4; would this mean it's now just tested vs 3.6? that's not out of the question, but can the older branches continue to test on older versions or is that super complex? On Wed, Apr 10, 2019 at 1:37 PM shane knapp wrote: > > details here (see most recent

Support for Avro datasets

Hi The PR #24299 proposes a simple solution to add support for Avro objects; your opinions matter, we are eager to have a working solution to create Avro datasets regards -- Taoufik Dachraoui

Joining of one stream against the most recent update of another stream

Hi, I just opened a Jira task (https://issues.apache.org/jira/browse/SPARK-27424) outlining a specific streaming join and would like to raise your attention on this. As this feature is a critical requirement for my current project, I am willing to work on that task. Before putting too much