Github user zhonghaihua commented on the issue:
https://github.com/apache/spark/pull/12258
@vanzin my JIRA username is `iward`. Thanks a lot.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
Github user zhonghaihua commented on the issue:
https://github.com/apache/spark/pull/12258
@vanzin retest this please.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled
Github user zhonghaihua commented on a diff in the pull request:
https://github.com/apache/spark/pull/12258#discussion_r66000194
--- Diff:
core/src/main/scala/org/apache/spark/scheduler/TaskSchedulerImpl.scala ---
@@ -343,17 +343,31 @@ private[spark] class TaskSchedulerImpl
Github user zhonghaihua commented on a diff in the pull request:
https://github.com/apache/spark/pull/12258#discussion_r65997697
--- Diff:
core/src/main/scala/org/apache/spark/scheduler/TaskSchedulerImpl.scala ---
@@ -343,17 +343,31 @@ private[spark] class TaskSchedulerImpl
Github user zhonghaihua commented on the issue:
https://github.com/apache/spark/pull/12258
@vanzin @andrewor14 Could someone review it, or any thoughts or concerns
for this patch ? Thanks a lot.
---
If your project is set up for it, you can reply to this email and have your
reply
Github user zhonghaihua commented on the issue:
https://github.com/apache/spark/pull/12258
Hi, @vanzin , the code has changed, could you review it, please?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project
Github user zhonghaihua commented on a diff in the pull request:
https://github.com/apache/spark/pull/12258#discussion_r65797802
--- Diff:
core/src/main/scala/org/apache/spark/scheduler/TaskSchedulerImpl.scala ---
@@ -343,17 +343,31 @@ private[spark] class TaskSchedulerImpl
Github user zhonghaihua commented on a diff in the pull request:
https://github.com/apache/spark/pull/12258#discussion_r6579
--- Diff:
core/src/main/scala/org/apache/spark/scheduler/TaskSchedulerImpl.scala ---
@@ -343,17 +343,31 @@ private[spark] class TaskSchedulerImpl
Github user zhonghaihua commented on a diff in the pull request:
https://github.com/apache/spark/pull/12258#discussion_r65797773
--- Diff:
core/src/main/scala/org/apache/spark/scheduler/TaskSchedulerImpl.scala ---
@@ -343,17 +343,31 @@ private[spark] class TaskSchedulerImpl
Github user zhonghaihua commented on the issue:
https://github.com/apache/spark/pull/12258
@srowen @JoshRosen @andrewor14 Could someone verify this PR? Thanks a lot.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If
Github user zhonghaihua commented on the pull request:
https://github.com/apache/spark/pull/12258#issuecomment-208881577
@JoshRosen @andrewor14 Could you verify this PR? Thanks a lot.
---
If your project is set up for it, you can reply to this email and have your
reply appear on
Github user zhonghaihua commented on the pull request:
https://github.com/apache/spark/pull/12258#issuecomment-207296818
cc @andrewor14
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this
GitHub user zhonghaihua opened a pull request:
https://github.com/apache/spark/pull/12258
Ignore task finish for executor lost and removed by driver
## What changes were proposed in this pull request?
Now, when executor is removed by driver with heartbeats timeout, driver
Github user zhonghaihua commented on the pull request:
https://github.com/apache/spark/pull/10794#issuecomment-204715025
Hi, @tgravescs , my jira id is `Iward`. Thanks a lot.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as
Github user zhonghaihua commented on the pull request:
https://github.com/apache/spark/pull/10794#issuecomment-204288682
@andrewor14 @tgravescs @vanzin The code and the comment is optimized. And
the description of this PR and jira is also updated. Please review it again.
Thanks a lot
Github user zhonghaihua commented on a diff in the pull request:
https://github.com/apache/spark/pull/10794#discussion_r58042774
--- Diff:
yarn/src/main/scala/org/apache/spark/deploy/yarn/YarnAllocator.scala ---
@@ -81,8 +82,20 @@ private[yarn] class YarnAllocator(
new
Github user zhonghaihua commented on a diff in the pull request:
https://github.com/apache/spark/pull/10794#discussion_r58032934
--- Diff:
core/src/main/scala/org/apache/spark/scheduler/cluster/CoarseGrainedSchedulerBackend.scala
---
@@ -155,6 +158,9 @@ class
Github user zhonghaihua commented on a diff in the pull request:
https://github.com/apache/spark/pull/10794#discussion_r58009178
--- Diff:
core/src/main/scala/org/apache/spark/scheduler/cluster/CoarseGrainedClusterMessage.scala
---
@@ -30,6 +30,8 @@ private[spark] object
Github user zhonghaihua commented on a diff in the pull request:
https://github.com/apache/spark/pull/10794#discussion_r57838823
--- Diff:
yarn/src/main/scala/org/apache/spark/deploy/yarn/YarnAllocator.scala ---
@@ -81,8 +82,20 @@ private[yarn] class YarnAllocator(
new
Github user zhonghaihua commented on a diff in the pull request:
https://github.com/apache/spark/pull/10794#discussion_r57838667
--- Diff:
yarn/src/main/scala/org/apache/spark/deploy/yarn/YarnAllocator.scala ---
@@ -81,8 +82,20 @@ private[yarn] class YarnAllocator(
new
Github user zhonghaihua commented on a diff in the pull request:
https://github.com/apache/spark/pull/10794#discussion_r57837944
--- Diff:
core/src/main/scala/org/apache/spark/scheduler/cluster/CoarseGrainedClusterMessage.scala
---
@@ -30,6 +30,8 @@ private[spark] object
Github user zhonghaihua commented on the pull request:
https://github.com/apache/spark/pull/10794#issuecomment-201880646
@andrewor14 @tgravescs @vanzin Could you verify this PR, or any thoughts or
concerns for this ? Thanks a lot.
---
If your project is set up for it, you can reply
Github user zhonghaihua commented on the pull request:
https://github.com/apache/spark/pull/10794#issuecomment-191830328
Hi @andrewor14 , any thoughts or concerns for this patch ?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub
Github user zhonghaihua commented on the pull request:
https://github.com/apache/spark/pull/10794#issuecomment-189095532
Hi @andrewor14 , could you review this ? Thanks a lot.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as
Github user zhonghaihua commented on a diff in the pull request:
https://github.com/apache/spark/pull/10794#discussion_r54094456
--- Diff:
core/src/main/scala/org/apache/spark/scheduler/cluster/CoarseGrainedSchedulerBackend.scala
---
@@ -78,6 +78,9 @@ class
Github user zhonghaihua commented on the pull request:
https://github.com/apache/spark/pull/10794#issuecomment-188730628
retest this please.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
Github user zhonghaihua commented on a diff in the pull request:
https://github.com/apache/spark/pull/10794#discussion_r54063643
--- Diff:
yarn/src/main/scala/org/apache/spark/deploy/yarn/YarnAllocator.scala ---
@@ -81,8 +82,20 @@ private[yarn] class YarnAllocator(
new
Github user zhonghaihua commented on a diff in the pull request:
https://github.com/apache/spark/pull/10794#discussion_r54062244
--- Diff:
yarn/src/main/scala/org/apache/spark/deploy/yarn/YarnAllocator.scala ---
@@ -81,8 +82,20 @@ private[yarn] class YarnAllocator(
new
Github user zhonghaihua commented on a diff in the pull request:
https://github.com/apache/spark/pull/10794#discussion_r54060168
--- Diff:
yarn/src/main/scala/org/apache/spark/deploy/yarn/YarnAllocator.scala ---
@@ -81,8 +82,20 @@ private[yarn] class YarnAllocator(
new
Github user zhonghaihua commented on a diff in the pull request:
https://github.com/apache/spark/pull/10794#discussion_r54058736
--- Diff:
yarn/src/main/scala/org/apache/spark/deploy/yarn/YarnAllocator.scala ---
@@ -81,8 +82,20 @@ private[yarn] class YarnAllocator(
new
Github user zhonghaihua commented on a diff in the pull request:
https://github.com/apache/spark/pull/10794#discussion_r54058368
--- Diff:
core/src/main/scala/org/apache/spark/scheduler/cluster/CoarseGrainedSchedulerBackend.scala
---
@@ -78,6 +78,9 @@ class
Github user zhonghaihua commented on a diff in the pull request:
https://github.com/apache/spark/pull/10794#discussion_r53919614
--- Diff:
yarn/src/main/scala/org/apache/spark/deploy/yarn/YarnAllocator.scala ---
@@ -169,6 +172,24 @@ private[yarn] class YarnAllocator
Github user zhonghaihua commented on the pull request:
https://github.com/apache/spark/pull/10794#issuecomment-186993839
Hi @andrewor14 , the reason of test failed seems `GitException`. Could you
retest it ? Thanks a lot.
---
If your project is set up for it, you can reply to this
Github user zhonghaihua commented on a diff in the pull request:
https://github.com/apache/spark/pull/10794#discussion_r53579600
--- Diff:
yarn/src/main/scala/org/apache/spark/deploy/yarn/YarnAllocator.scala ---
@@ -169,6 +172,24 @@ private[yarn] class YarnAllocator
Github user zhonghaihua commented on a diff in the pull request:
https://github.com/apache/spark/pull/10794#discussion_r53562669
--- Diff:
core/src/main/scala/org/apache/spark/scheduler/cluster/CoarseGrainedSchedulerBackend.scala
---
@@ -155,6 +158,9 @@ class
Github user zhonghaihua commented on a diff in the pull request:
https://github.com/apache/spark/pull/10794#discussion_r53562543
--- Diff:
core/src/main/scala/org/apache/spark/scheduler/cluster/CoarseGrainedClusterMessage.scala
---
@@ -30,6 +30,8 @@ private[spark] object
Github user zhonghaihua commented on the pull request:
https://github.com/apache/spark/pull/10794#issuecomment-186016863
Hi, @andrewor14 , I agree with @jerryshao , I think that is not related to
it.
---
If your project is set up for it, you can reply to this email and have your
Github user zhonghaihua commented on the pull request:
https://github.com/apache/spark/pull/10794#issuecomment-185625228
@andrewor14 @marmbrus @rxin , any thoughts or concerns for this patch ?
---
If your project is set up for it, you can reply to this email and have your
reply
Github user zhonghaihua commented on the pull request:
https://github.com/apache/spark/pull/10794#issuecomment-179924619
@andrewor14 Thanks for review it. Could this path merge to master ?
---
If your project is set up for it, you can reply to this email and have your
reply appear on
Github user zhonghaihua commented on the pull request:
https://github.com/apache/spark/pull/10794#issuecomment-174830247
@marmbrus @liancheng @yhuai Could you verify this patch?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as
Github user zhonghaihua commented on the pull request:
https://github.com/apache/spark/pull/10794#issuecomment-172323185
cc @rxin @marmbrus @chenghao-intel @jeanlyn could you give some advice ï¼
---
If your project is set up for it, you can reply to this email and have your
reply
GitHub user zhonghaihua opened a pull request:
https://github.com/apache/spark/pull/10794
initialize executorIdCounter after ApplicationMaster killed for max nâ¦
Currently, when max number of executor failures reached the
`maxNumExecutorFailures`, `ApplicationMaster` will be
Github user zhonghaihua commented on the pull request:
https://github.com/apache/spark/pull/10128#issuecomment-169231327
@marmbrus Thanks for your suggestions. I think your idea can simply solve
problem. But in some situations, this seems not very suitable.
For example:
`Join
Github user zhonghaihua commented on the pull request:
https://github.com/apache/spark/pull/10128#issuecomment-168892937
@rxin @yhuai Could you verify this patch?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
Github user zhonghaihua commented on the pull request:
https://github.com/apache/spark/pull/10128#issuecomment-161969421
@cloud-fan Thanks for your advice. But, as @jeanlyn said,`Repartition` will
deal with all data, and this PR will deal with join keys cause data skew.
Because in
Github user zhonghaihua commented on the pull request:
https://github.com/apache/spark/pull/10128#issuecomment-161590938
@rxin @cloud-fan @chenghao-intel @jeanlyn Could you give some suggestions
on this PR?
---
If your project is set up for it, you can reply to this email and have
GitHub user zhonghaihua opened a pull request:
https://github.com/apache/spark/pull/10128
pull out nondeterministic expressions from Join
Currently,`nondeterministic expressions` are only allowed in `Project` or
`Filter`,And only when we use nondeterministic expressions in
Github user zhonghaihua commented on the pull request:
https://github.com/apache/spark/pull/10122#issuecomment-161533958
I am so sorry to create this pull request, this pr is not on the right
branch.I will close it right now.
This is my mistake, cause trouble, very sorry.
---
If
Github user zhonghaihua closed the pull request at:
https://github.com/apache/spark/pull/10122
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature
GitHub user zhonghaihua opened a pull request:
https://github.com/apache/spark/pull/10122
Join nondeterministic
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/zhonghaihua/spark join_nondeterministic
Alternatively you can
Github user zhonghaihua commented on the pull request:
https://github.com/apache/spark/pull/9483#issuecomment-156810714
Hi @zhichao-li ,thanks for doing this.I got a problem of scanning
partitions slowly,and I apply this patch to my spark version.In my case:
* Before I apply this
Github user zhonghaihua commented on the pull request:
https://github.com/apache/spark/pull/9483#issuecomment-156810307
Hi @zhichao-li ,thanks for doing this.I got a problem of scanning
partitions slowly,and I apply this patch to my spark version.In my case:
* Before I apply this
52 matches
Mail list logo
