Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/16774
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/16774
**[Test build #73021 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73021/testReport)**
for PR 16774 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/16774
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/73021/
Test PASSed.
---
Github user BryanCutler commented on the issue:
https://github.com/apache/spark/pull/16774
> I think setting the default to match current behavior is best (i.e. 1).
I agree, just wanted to bring it up in case others had differing view since
it was a concern in the JIRA.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/16774
**[Test build #73021 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73021/testReport)**
for PR 16774 at commit
Github user MLnick commented on the issue:
https://github.com/apache/spark/pull/16774
I'd say coming up with a heuristic or algorithm to automatically set the
parallel execution param is going to be pretty challenging, since it depends on
the details of the individual pipeline
Github user BryanCutler commented on the issue:
https://github.com/apache/spark/pull/16774
Regarding the default value of the new parameter, do you think `1`, so that
it is run in serial as it is currently doing, is the best option? It's
definitely safe, but I wonder if most people
Github user BryanCutler commented on the issue:
https://github.com/apache/spark/pull/16774
Thanks for the review @MLnick! All good suggestions, I'll work on an
update and addition of the unit test.
---
If your project is set up for it, you can reply to this email and have your
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/16774
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/72267/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/16774
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/16774
**[Test build #72267 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/72267/testReport)**
for PR 16774 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/16774
**[Test build #72267 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/72267/testReport)**
for PR 16774 at commit
Github user BryanCutler commented on the issue:
https://github.com/apache/spark/pull/16774
CC @mlnick, this doesn't yet address the issue of double caching, but I
wanted to post what I had so far to discuss the additional param and default
value. I'll be looking into the caching
13 matches
Mail list logo
