[jira] [Commented] (SPARK-21867) Support async spilling in UnsafeShuffleWriter

[ https://issues.apache.org/jira/browse/SPARK-21867?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16288298#comment-16288298 ] Eric Vandenberg commented on SPARK-21867: - 1. The default would be 1 so does not change default

[jira] [Updated] (SPARK-21867) Support async spilling in UnsafeShuffleWriter

[ https://issues.apache.org/jira/browse/SPARK-21867?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eric Vandenberg updated SPARK-21867: Attachment: Async ShuffleExternalSorter.pdf Here is a design proposal to implement this

[jira] [Commented] (SPARK-22077) RpcEndpointAddress fails to parse spark URL if it is an ipv6 address.

[ https://issues.apache.org/jira/browse/SPARK-22077?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16183117#comment-16183117 ] Eric Vandenberg commented on SPARK-22077: - We are using a Facebook internal cluster scheduler. I

[jira] [Commented] (SPARK-22077) RpcEndpointAddress fails to parse spark URL if it is an ipv6 address.

[ https://issues.apache.org/jira/browse/SPARK-22077?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16175479#comment-16175479 ] Eric Vandenberg commented on SPARK-22077: - Yes, it worked when I overloaded with "localhost" so

[jira] [Created] (SPARK-22077) RpcEndpointAddress fails to parse spark URL if it is an ipv6 address.

Eric Vandenberg created SPARK-22077: --- Summary: RpcEndpointAddress fails to parse spark URL if it is an ipv6 address. Key: SPARK-22077 URL: https://issues.apache.org/jira/browse/SPARK-22077 Project:

[jira] [Commented] (SPARK-21598) Collect usability/events information from Spark History Server

[ https://issues.apache.org/jira/browse/SPARK-21598?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16113040#comment-16113040 ] Eric Vandenberg commented on SPARK-21598: - [~steve_l] Do you have any input / thoughts here?

[jira] [Created] (SPARK-21598) Collect usability/events information from Spark History Server

Eric Vandenberg created SPARK-21598: --- Summary: Collect usability/events information from Spark History Server Key: SPARK-21598 URL: https://issues.apache.org/jira/browse/SPARK-21598 Project: Spark

[jira] [Commented] (SPARK-21571) Spark history server leaves incomplete or unreadable history files around forever.

[ https://issues.apache.org/jira/browse/SPARK-21571?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16109282#comment-16109282 ] Eric Vandenberg commented on SPARK-21571: - Link to pull request

[jira] [Updated] (SPARK-21571) Spark history server leaves incomplete or unreadable history files around forever.

[ https://issues.apache.org/jira/browse/SPARK-21571?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eric Vandenberg updated SPARK-21571: Description: We have noticed that history server logs are sometimes never cleaned up. The

[jira] [Created] (SPARK-21571) Spark history server leaves incomplete or unreadable history files around forever.

Eric Vandenberg created SPARK-21571: --- Summary: Spark history server leaves incomplete or unreadable history files around forever. Key: SPARK-21571 URL: https://issues.apache.org/jira/browse/SPARK-21571

[jira] [Commented] (SPARK-11170) ​ EOFException on History server reading in progress lz4

[ https://issues.apache.org/jira/browse/SPARK-11170?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16100537#comment-16100537 ] Eric Vandenberg commented on SPARK-11170: - There's a fix for this, see

[jira] [Created] (SPARK-21447) Spark history server fails to render compressed inprogress history file in some cases.

Eric Vandenberg created SPARK-21447: --- Summary: Spark history server fails to render compressed inprogress history file in some cases. Key: SPARK-21447 URL: https://issues.apache.org/jira/browse/SPARK-21447

[jira] [Updated] (SPARK-21219) Task retry occurs on same executor due to race condition with blacklisting

[ https://issues.apache.org/jira/browse/SPARK-21219?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eric Vandenberg updated SPARK-21219: Description: When a task fails it is (1) added into the pending task list and then (2)

[jira] [Updated] (SPARK-21219) Task retry occurs on same executor due to race condition with blacklisting

[ https://issues.apache.org/jira/browse/SPARK-21219?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eric Vandenberg updated SPARK-21219: Description: When a task fails it is added into the pending task list and corresponding

[jira] [Updated] (SPARK-21219) Task retry occurs on same executor due to race condition with blacklisting

[ https://issues.apache.org/jira/browse/SPARK-21219?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eric Vandenberg updated SPARK-21219: Attachment: spark_executor.log.anon spark_driver.log.anon > Task retry

[jira] [Created] (SPARK-21219) Task retry occurs on same executor due to race condition with blacklisting

Eric Vandenberg created SPARK-21219: --- Summary: Task retry occurs on same executor due to race condition with blacklisting Key: SPARK-21219 URL: https://issues.apache.org/jira/browse/SPARK-21219

[jira] [Commented] (SPARK-21155) Add (? running tasks) into Spark UI progress

[ https://issues.apache.org/jira/browse/SPARK-21155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16061225#comment-16061225 ] Eric Vandenberg commented on SPARK-21155: - Added screen shot with skipped tasks for reference. >

[jira] [Updated] (SPARK-21155) Add (? running tasks) into Spark UI progress

[ https://issues.apache.org/jira/browse/SPARK-21155?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eric Vandenberg updated SPARK-21155: Attachment: Screen Shot 2017-06-22 at 9.58.08 AM.png > Add (? running tasks) into Spark UI

[jira] [Issue Comment Deleted] (SPARK-21155) Add (? running tasks) into Spark UI progress

[ https://issues.apache.org/jira/browse/SPARK-21155?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eric Vandenberg updated SPARK-21155: Comment: was deleted (was: Before ) > Add (? running tasks) into Spark UI progress >

[jira] [Updated] (SPARK-21155) Add (? running tasks) into Spark UI progress

[ https://issues.apache.org/jira/browse/SPARK-21155?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eric Vandenberg updated SPARK-21155: Attachment: Screen Shot 2017-06-20 at 3.40.39 PM.png Before > Add (? running tasks) into

[jira] [Commented] (SPARK-21155) Add (? running tasks) into Spark UI progress

[ https://issues.apache.org/jira/browse/SPARK-21155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16056500#comment-16056500 ] Eric Vandenberg commented on SPARK-21155: - The pr is @

[jira] [Updated] (SPARK-21155) Add (? running tasks) into Spark UI progress

[ https://issues.apache.org/jira/browse/SPARK-21155?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eric Vandenberg updated SPARK-21155: Attachment: Screen Shot 2017-06-20 at 12.32.58 PM.png > Add (? running tasks) into Spark

[jira] [Created] (SPARK-21155) Add (? running tasks) into Spark UI progress

Eric Vandenberg created SPARK-21155: --- Summary: Add (? running tasks) into Spark UI progress Key: SPARK-21155 URL: https://issues.apache.org/jira/browse/SPARK-21155 Project: Spark Issue

[jira] [Created] (SPARK-20778) Implement array_intersect function

Eric Vandenberg created SPARK-20778: --- Summary: Implement array_intersect function Key: SPARK-20778 URL: https://issues.apache.org/jira/browse/SPARK-20778 Project: Spark Issue Type: