Github user srowen commented on the issue:
https://github.com/apache/spark/pull/19183
I thought this check also existed in the non-streaming code; the theory was
that if you have set a fixed number of executors but enabled dynamic
allocation, then that's probably a configuration
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19183
Can one of the admins verify this patch?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user skonto commented on the issue:
https://github.com/apache/spark/pull/19183
@karth295 I think you could validate the config
[here](https://github.com/apache/spark/blob/fa2ae9d2019f839647d17932d8fea769e7622777/core/src/main/scala/org/apache/spark/SparkContext.scala#L365),
in
Github user karth295 commented on the issue:
https://github.com/apache/spark/pull/19183
@sansagara go for it -- it'll be a few days until I'll have time to look at
this again. I'll close my PR if/when you make a new one :)
---
Github user koeninger commented on the issue:
https://github.com/apache/spark/pull/19183
@sansagara sounds reasonable to me
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands,
Github user sansagara commented on the issue:
https://github.com/apache/spark/pull/19183
I can add a test on ExecutorAllocationManagerSuite.scala to assert
spark.executor.instances correctly allocates that number of executors initially
when uaing Streaming DA
---
Github user koeninger commented on the issue:
https://github.com/apache/spark/pull/19183
I don't have personal experience with streaming dynamic allocation, but
this patch makes sense to me and I don't see anything obviously wrong.
I agree with Holden regarding tests.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19183
**[Test build #90232 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90232/testReport)**
for PR 19183 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19183
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/90232/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19183
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user holdenk commented on the issue:
https://github.com/apache/spark/pull/19183
Also one point, the lack of tests leaves me a little concerned about this
change, maybe look at
Github user holdenk commented on the issue:
https://github.com/apache/spark/pull/19183
@karth295 for validating that spark.executors is in a valid range could we
look at it in the config with ConfigBuilder checkValue ?
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19183
**[Test build #90232 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90232/testReport)**
for PR 19183 at commit
Github user holdenk commented on the issue:
https://github.com/apache/spark/pull/19183
Jenkins, ok to test.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user sansagara commented on the issue:
https://github.com/apache/spark/pull/19183
Just commenting to be subscribed!
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands,
Github user sansagara2 commented on the issue:
https://github.com/apache/spark/pull/19183
This is actually a good design solution. Right now, it is not very clear
(not even docs, examples, google searches) how to set an initial number of
executors for a Streaming Application that
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19183
Can one of the admins verify this patch?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19183
Can one of the admins verify this patch?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19183
Can one of the admins verify this patch?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19183
Can one of the admins verify this patch?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
20 matches
Mail list logo
