[
https://issues.apache.org/jira/browse/SPARK-18085?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16084451#comment-16084451
]
Ye Zhou commented on SPARK-18085:
-
I want to add my own testing experience with the codes
[
https://issues.apache.org/jira/browse/SPARK-36772?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17415879#comment-17415879
]
Ye Zhou commented on SPARK-36772:
-
I will work on this one and post PR ASAP.
> Finalize
[
https://issues.apache.org/jira/browse/SPARK-36892?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17422425#comment-17422425
]
Ye Zhou commented on SPARK-36892:
-
I am working on this issue. We have a job which can r
[
https://issues.apache.org/jira/browse/SPARK-36892?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17422558#comment-17422558
]
Ye Zhou commented on SPARK-36892:
-
Raised PR [https://github.com/apache/spark/pull/34156
Ye Zhou created SPARK-37023:
---
Summary: Avoid fetching merge status when shuffleMergeEnabled is
false for a shuffleDependency during retry
Key: SPARK-37023
URL: https://issues.apache.org/jira/browse/SPARK-37023
[ https://issues.apache.org/jira/browse/SPARK-33236 ]
Ye Zhou deleted comment on SPARK-33236:
-
was (Author: zhouyejoe):
WIP PR posted [https://github.com/apache/spark/pull/35906.]
> Enable Push-based shuffle service to store state in NM level DB f
Ye Zhou created SPARK-38987:
---
Summary: Handle fallback when merged shuffle blocks are corrupted
and spark.shuffle.detectCorrupt is set to true
Key: SPARK-38987
URL: https://issues.apache.org/jira/browse/SPARK-38987
Ye Zhou created SPARK-35546:
---
Summary: Handling race condition and memory leak in
RemoteBlockPushResolver
Key: SPARK-35546
URL: https://issues.apache.org/jira/browse/SPARK-35546
Project: Spark
Iss
Ye Zhou created SPARK-35548:
---
Summary: Handling new attempt has started error message in
BlockPushErrorHandler in client
Key: SPARK-35548
URL: https://issues.apache.org/jira/browse/SPARK-35548
Project: Spar
[
https://issues.apache.org/jira/browse/SPARK-35546?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ye Zhou updated SPARK-35546:
Summary: Properly handle race conditions in RemoteBlockPushResolver for
access to the internal ConcurrentH
[
https://issues.apache.org/jira/browse/SPARK-35546?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ye Zhou updated SPARK-35546:
Summary: Properly handle race conditions in RemoteBlockPushResolver for
access to the internal ConcurrentH
[
https://issues.apache.org/jira/browse/SPARK-35546?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ye Zhou updated SPARK-35546:
Summary: Properly handle race conditions in RemoteBlockPushResolver to
support push based shuffle with mul
[
https://issues.apache.org/jira/browse/SPARK-35546?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ye Zhou updated SPARK-35546:
Parent: (was: SPARK-33235)
Issue Type: Bug (was: Sub-task)
> Enable push-based shuffle when m
[
https://issues.apache.org/jira/browse/SPARK-35546?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ye Zhou updated SPARK-35546:
Parent: SPARK-30602
Issue Type: Sub-task (was: Bug)
> Enable push-based shuffle when multiple app
[
https://issues.apache.org/jira/browse/SPARK-33236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17508498#comment-17508498
]
Ye Zhou commented on SPARK-33236:
-
WIP PR posted [https://github.com/apache/spark/pull/3
[
https://issues.apache.org/jira/browse/SPARK-21961?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ye Zhou resolved SPARK-21961.
-
Resolution: Won't Fix
> Filter out BlockStatuses Accumulators during replaying history logs in Spark
>
Ye Zhou created SPARK-25634:
---
Summary: New Metrics in External Shuffle Service to help identify
abusing application
Key: SPARK-25634
URL: https://issues.apache.org/jira/browse/SPARK-25634
Project: Spark
[
https://issues.apache.org/jira/browse/SPARK-25634?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16637436#comment-16637436
]
Ye Zhou commented on SPARK-25634:
-
[~felixcheung] [~vanzin] [~tgraves] [~irashid] [~
Ye Zhou created SPARK-23607:
---
Summary: Use HDFS extended attributes to store application summary
to improve the Spark History Server performance
Key: SPARK-23607
URL: https://issues.apache.org/jira/browse/SPARK-23607
[
https://issues.apache.org/jira/browse/SPARK-23607?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16387126#comment-16387126
]
Ye Zhou commented on SPARK-23607:
-
[~vanzin] Any comments? Thanks.
> Use HDFS extended a
[
https://issues.apache.org/jira/browse/SPARK-23607?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ye Zhou updated SPARK-23607:
Shepherd: (was: Marcelo Vanzin)
> Use HDFS extended attributes to store application summary to improve th
[
https://issues.apache.org/jira/browse/SPARK-23607?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16387126#comment-16387126
]
Ye Zhou edited comment on SPARK-23607 at 3/6/18 1:42 AM:
-
[~vanzi
Ye Zhou created SPARK-23608:
---
Summary: SHS needs synchronization between attachSparkUI and
detachSparkUI functions
Key: SPARK-23608
URL: https://issues.apache.org/jira/browse/SPARK-23608
Project: Spark
[
https://issues.apache.org/jira/browse/SPARK-23608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16387162#comment-16387162
]
Ye Zhou commented on SPARK-23608:
-
I will post a pull request for this minor change.
[~v
[
https://issues.apache.org/jira/browse/SPARK-23608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16387175#comment-16387175
]
Ye Zhou commented on SPARK-23608:
-
Pull Request: https://github.com/apache/spark/pull/207
[
https://issues.apache.org/jira/browse/SPARK-23607?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16396193#comment-16396193
]
Ye Zhou commented on SPARK-23607:
-
[~vanzin] Cool. I will post a PR soon. Thanks.
> Use
[
https://issues.apache.org/jira/browse/SPARK-23206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16338378#comment-16338378
]
Ye Zhou commented on SPARK-23206:
-
[~zsxwing] Hi, Can you help find some one who can help
Ye Zhou created SPARK-43100:
---
Summary: Mismatch of field name in log event writer and parser for
push shuffle metrics
Key: SPARK-43100
URL: https://issues.apache.org/jira/browse/SPARK-43100
Project: Spark
[
https://issues.apache.org/jira/browse/SPARK-43100?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17711148#comment-17711148
]
Ye Zhou commented on SPARK-43100:
-
Created PR to fix the issue https://github.com/apache
[
https://issues.apache.org/jira/browse/SPARK-35546?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ye Zhou resolved SPARK-35546.
-
Fix Version/s: 3.2.0
Target Version/s: 3.2.0
Resolution: Fixed
Issue resolved by pull r
[
https://issues.apache.org/jira/browse/SPARK-30602?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17390184#comment-17390184
]
Ye Zhou commented on SPARK-30602:
-
[~Gengliang.Wang] This is the last PR(Subtask 9) to b
[
https://issues.apache.org/jira/browse/SPARK-33573?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ye Zhou updated SPARK-33573:
Description: Shuffle Server side metrics for push based shuffle. (was:
Need to add metrics on both server
[
https://issues.apache.org/jira/browse/SPARK-33573?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ye Zhou updated SPARK-33573:
Summary: Server side metrics related to push-based shuffle (was: Server
and client side metrics related t
[
https://issues.apache.org/jira/browse/SPARK-36744?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ye Zhou updated SPARK-36744:
Description:
As a follow up to SPARK-36705, push-based shuffle is not compatible with IO
encryption. We n
[
https://issues.apache.org/jira/browse/SPARK-21715?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ye Zhou updated SPARK-21715:
Attachment: Performance.png
> History Server respondes history page html content multiple times for only
>
Ye Zhou created SPARK-21715:
---
Summary: History Server respondes history page html content
multiple times for only one http request
Key: SPARK-21715
URL: https://issues.apache.org/jira/browse/SPARK-21715
Pro
[
https://issues.apache.org/jira/browse/SPARK-21715?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ye Zhou updated SPARK-21715:
Description:
UI looks fine for the home page. But we check the performance for each
individual components,
[
https://issues.apache.org/jira/browse/SPARK-21715?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ye Zhou updated SPARK-21715:
Description:
UI looks fine for the home page. But we check the performance for each
individual components,
[
https://issues.apache.org/jira/browse/SPARK-21715?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ye Zhou updated SPARK-21715:
Description:
UI looks fine for the home page. But we check the performance for each
individual components,
[
https://issues.apache.org/jira/browse/SPARK-21715?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ye Zhou updated SPARK-21715:
Attachment: ResponseContent.png
> History Server respondes history page html content multiple times for onl
[
https://issues.apache.org/jira/browse/SPARK-21715?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ye Zhou updated SPARK-21715:
Summary: History Server should not respond history page html content
multiple times for only one http reque
[
https://issues.apache.org/jira/browse/SPARK-21715?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16126398#comment-16126398
]
Ye Zhou commented on SPARK-21715:
-
Pull Request: https://github.com/apache/spark/pull/189
Ye Zhou created SPARK-21961:
---
Summary: Filter out BlockStatuses Accumulators during replaying
history logs in Spark History Server
Key: SPARK-21961
URL: https://issues.apache.org/jira/browse/SPARK-21961
Pro
[
https://issues.apache.org/jira/browse/SPARK-21961?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ye Zhou updated SPARK-21961:
Attachment: Objects_Count_in_Heap.png
One_Thread_Took_24GB.png
> Filter out BlockStatuses A
[
https://issues.apache.org/jira/browse/SPARK-21961?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ye Zhou updated SPARK-21961:
Description:
As described in SPARK-20923, TaskMetrics._updatedBlockStatuses uses a lot of
memory in Driver
[
https://issues.apache.org/jira/browse/SPARK-21961?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ye Zhou updated SPARK-21961:
Description:
As described in SPARK-20923, TaskMetrics._updatedBlockStatuses uses a lot of
memory in Driver
[
https://issues.apache.org/jira/browse/SPARK-21961?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ye Zhou updated SPARK-21961:
Description:
As described in SPARK-20923, TaskMetrics._updatedBlockStatuses uses a lot of
memory in Driver
[
https://issues.apache.org/jira/browse/SPARK-21961?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ye Zhou updated SPARK-21961:
Description:
As described in SPARK-20923, TaskMetrics._updatedBlockStatuses uses a lot of
memory in Driver
[
https://issues.apache.org/jira/browse/SPARK-21961?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ye Zhou updated SPARK-21961:
Description:
As described in SPARK-20923, TaskMetrics._updatedBlockStatuses uses a lot of
memory in Driver
[
https://issues.apache.org/jira/browse/SPARK-21961?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16159514#comment-16159514
]
Ye Zhou commented on SPARK-21961:
-
Pull Request Added: https://github.com/apache/spark/pu
[
https://issues.apache.org/jira/browse/SPARK-21961?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16172162#comment-16172162
]
Ye Zhou commented on SPARK-21961:
-
[~zsxwing] Can you help to take a look? Thanks.
> Fil
51 matches
Mail list logo