Ngone51 commented on pull request #30650:
URL: https://github.com/apache/spark/pull/30650#issuecomment-782547345
thanks for the help @tgravescs @mridulm
This is an automated message from the Apache Git Service.
To respond
Ngone51 commented on pull request #30650:
URL: https://github.com/apache/spark/pull/30650#issuecomment-781180966
Addressed the last comment from @tgravescs. Please take another look, thanks!
This is an automated message from
Ngone51 commented on pull request #30650:
URL: https://github.com/apache/spark/pull/30650#issuecomment-775643884
Oh I mean, for the behavior of disabling legacy delay scheduling for the
barrier stage.
This is an automated
Ngone51 commented on pull request #30650:
URL: https://github.com/apache/spark/pull/30650#issuecomment-774876158
cc @mridulm @tgravescs Please take another look when you're available:)
This is an automated message from the
Ngone51 commented on pull request #30650:
URL: https://github.com/apache/spark/pull/30650#issuecomment-773860013
That sounds great to me. I've addressed it in 555b3a8 with a unit test.
Please take another look.
BTW, do you think we need a conf to be able to restore the original
Ngone51 commented on pull request #30650:
URL: https://github.com/apache/spark/pull/30650#issuecomment-773860013
That sounds great to me. I've addressed it in 555b3a8 with a unit test.
Please take another look.
BTW, do you think we need a conf to be able to restore the original
Ngone51 commented on pull request #30650:
URL: https://github.com/apache/spark/pull/30650#issuecomment-771326460
> If there is insufficient resources to run the entire taskset at (some)
highest locality level, and there is atleast one task that can be launched at
that level - we will end
Ngone51 commented on pull request #30650:
URL: https://github.com/apache/spark/pull/30650#issuecomment-771326460
> If there is insufficient resources to run the entire taskset at (some)
highest locality level, and there is atleast one task that can be launched at
that level - we will end
Ngone51 commented on pull request #30650:
URL: https://github.com/apache/spark/pull/30650#issuecomment-769587926
retest this please
This is an automated message from the Apache Git Service.
To respond to the message, please
Ngone51 commented on pull request #30650:
URL: https://github.com/apache/spark/pull/30650#issuecomment-769587860
kindly ping @mridulm @tgravescs
This is an automated message from the Apache Git Service.
To respond to the
Ngone51 commented on pull request #30650:
URL: https://github.com/apache/spark/pull/30650#issuecomment-767433788
Hi @mridulm @tgravescs sorry for the delay.
After times thinking, I think we should just keep the original behavior for
the barrier taskset with the legacy delay
Ngone51 commented on pull request #30650:
URL: https://github.com/apache/spark/pull/30650#issuecomment-757960531
Wait, I just realize that the test "SPARK-24818: test resource revert of
barrier TaskSetManager" can not pass with legacy delay scheduling. It does have
the issue with legacy
Ngone51 commented on pull request #30650:
URL: https://github.com/apache/spark/pull/30650#issuecomment-757337359
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and
Ngone51 commented on pull request #30650:
URL: https://github.com/apache/spark/pull/30650#issuecomment-757337593
@tgravescs @mridulm I have added the log.
This is an automated message from the Apache Git Service.
To respond
Ngone51 commented on pull request #30650:
URL: https://github.com/apache/spark/pull/30650#issuecomment-757337359
> any subtle dynamic resource allocation interactions though ?
Barrier execution currently doesn't support dynamic resource allocation. So
it's not a problem.
>
Ngone51 commented on pull request #30650:
URL: https://github.com/apache/spark/pull/30650#issuecomment-755867630
Thanks for the review @tgravescs
> The only thing is now we could hang infinitely if we never fill the slots,
correct? I guess we could hang before if we didn't ever get
Ngone51 commented on pull request #30650:
URL: https://github.com/apache/spark/pull/30650#issuecomment-747884574
Also cc @tgravescs for more inputs.
This is an automated message from the Apache Git Service.
To respond to the
Ngone51 commented on pull request #30650:
URL: https://github.com/apache/spark/pull/30650#issuecomment-747884488
Thank you very much @mridulm
This is an automated message from the Apache Git Service.
To respond to the
Ngone51 commented on pull request #30650:
URL: https://github.com/apache/spark/pull/30650#issuecomment-743248529
@mridulm Sorry for the confusion. I was confused at the beginning too and
thought about creating a JIRA ticket. But I gave up since there's already a
TODO ticket commented.
Ngone51 commented on pull request #30650:
URL: https://github.com/apache/spark/pull/30650#issuecomment-739976799
@jiangxb1987 Please help review, thanks!
This is an automated message from the Apache Git Service.
To respond
20 matches
Mail list logo
