Re: python test infrastructure

2018-09-06 Thread Imran Rashid
> On Wed, Sep 5, 2018 at 11:59 PM Hyukjin Kwon wrote: > > > > > 1. all of the output in target/test-reports & python/unit-tests.log should be included in the jenkins archived artifacts. > > > > Hmmm, I thought they are already archived ( https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequ

Re: python test infrastructure

2018-09-05 Thread Hyukjin Kwon
> 1. all of the output in target/test-reports & python/unit-tests.log should be included in the jenkins archived artifacts. Hmmm, I thought they are already archived ( https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95734/artifact/target/unit-tests.log ). FWIW, unit-tests.log ar

Re: python test infrastructure

2018-09-05 Thread Imran Rashid
one more: seems like python/run-tests should have an option at least to not bail at the first failure: https://github.com/apache/spark/blob/master/python/run-tests.py#L113-L132 this is particularly annoying with flaky tests -- since the rest of the tests aren't run, you don't know whether you *onl