[jira] [Updated] (SPARK-5268) CoarseGrainedExecutorBackend exits for irrelevant DisassociatedEvent

[ https://issues.apache.org/jira/browse/SPARK-5268?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-5268: --- Priority: Major (was: Blocker) CoarseGrainedExecutorBackend exits for irrelevant DisassociatedEvent

[jira] [Comment Edited] (SPARK-2645) Spark driver calls System.exit(50) after calling SparkContext.stop() the second time

[ https://issues.apache.org/jira/browse/SPARK-2645?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14291046#comment-14291046 ] Vlad Komarov edited comment on SPARK-2645 at 1/25/15 11:19 AM:

[jira] [Commented] (SPARK-2645) Spark driver calls System.exit(50) after calling SparkContext.stop() the second time

[ https://issues.apache.org/jira/browse/SPARK-2645?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14291046#comment-14291046 ] Vlad Komarov commented on SPARK-2645: - [~sowen], are you using the same spark master

[jira] [Commented] (SPARK-2688) Need a way to run multiple data pipeline concurrently

[ https://issues.apache.org/jira/browse/SPARK-2688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14291144#comment-14291144 ] Sean Owen commented on SPARK-2688: -- I am still not clear on what you are trying to do

[jira] [Commented] (SPARK-4986) Graceful shutdown for Spark Streaming does not work in Standalone cluster mode

[ https://issues.apache.org/jira/browse/SPARK-4986?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14291158#comment-14291158 ] Sean Owen commented on SPARK-4986: -- [~koudelka] This sounds pretty related to the issue I

[jira] [Commented] (SPARK-2688) Need a way to run multiple data pipeline concurrently

[ https://issues.apache.org/jira/browse/SPARK-2688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14291184#comment-14291184 ] Sandy Ryza commented on SPARK-2688: --- [~xuefuz] Spark already has transformations that

[jira] [Resolved] (SPARK-3622) Provide a custom transformation that can output multiple RDDs

[ https://issues.apache.org/jira/browse/SPARK-3622?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandy Ryza resolved SPARK-3622. --- Resolution: Not a Problem Provide a custom transformation that can output multiple RDDs

[jira] [Commented] (SPARK-2688) Need a way to run multiple data pipeline concurrently

[ https://issues.apache.org/jira/browse/SPARK-2688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14291193#comment-14291193 ] Sean Owen commented on SPARK-2688: -- (Heh, OK, well I would have closed this one instead

[jira] [Commented] (SPARK-3621) Provide a way to broadcast an RDD (instead of just a variable made of the RDD) so that a job can access

[ https://issues.apache.org/jira/browse/SPARK-3621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14291145#comment-14291145 ] Xuefu Zhang commented on SPARK-3621: I'm not sure if I agree that this is not a

[jira] [Reopened] (SPARK-3621) Provide a way to broadcast an RDD (instead of just a variable made of the RDD) so that a job can access

[ https://issues.apache.org/jira/browse/SPARK-3621?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xuefu Zhang reopened SPARK-3621: Provide a way to broadcast an RDD (instead of just a variable made of the RDD) so that a job can

[jira] [Commented] (SPARK-2688) Need a way to run multiple data pipeline concurrently

[ https://issues.apache.org/jira/browse/SPARK-2688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14291192#comment-14291192 ] Sean Owen commented on SPARK-2688: -- [~sandyr] Yes I can appreciate the difference between

[jira] [Commented] (SPARK-3621) Provide a way to broadcast an RDD (instead of just a variable made of the RDD) so that a job can access

[ https://issues.apache.org/jira/browse/SPARK-3621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14291152#comment-14291152 ] Sean Owen commented on SPARK-3621: -- Hm, what is an example? I think you mean collect an

[jira] [Updated] (SPARK-4267) Failing to launch jobs on Spark on YARN with Hadoop 2.5.0 or later

[ https://issues.apache.org/jira/browse/SPARK-4267?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-4267: - Priority: Blocker (was: Major) Failing to launch jobs on Spark on YARN with Hadoop 2.5.0 or later

[jira] [Updated] (SPARK-4267) Failing to launch jobs on Spark on YARN with Hadoop 2.5.0 or later

[ https://issues.apache.org/jira/browse/SPARK-4267?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-4267: - Component/s: YARN Target Version/s: 1.3.0 Failing to launch jobs on Spark on YARN with Hadoop

[jira] [Updated] (SPARK-3782) Direct use of log4j in AkkaUtils interferes with certain logging configurations

[ https://issues.apache.org/jira/browse/SPARK-3782?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-3782: - Affects Version/s: 1.1.0 Direct use of log4j in AkkaUtils interferes with certain logging

[jira] [Commented] (SPARK-4506) Update documentation to clarify whether standalone-cluster mode is now officially supported

[ https://issues.apache.org/jira/browse/SPARK-4506?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14291307#comment-14291307 ] Andrew Or commented on SPARK-4506: -- Thanks for pointing that out. I have merged the patch

[jira] [Updated] (SPARK-4430) Apache RAT Checks fail spuriously on test files

[ https://issues.apache.org/jira/browse/SPARK-4430?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-4430: - Affects Version/s: (was: 1.2.0) 1.1.0 Apache RAT Checks fail spuriously on

[jira] [Closed] (SPARK-5384) Vectors.sqdist return inconsistent result for sparse/dense vectors when the vectors have different lengths

[ https://issues.apache.org/jira/browse/SPARK-5384?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] yuhao yang closed SPARK-5384. - fixed Vectors.sqdist return inconsistent result for sparse/dense vectors when the vectors have different

[jira] [Updated] (SPARK-5384) Vectors.sqdist return inconsistent result for sparse/dense vectors when the vectors have different lengths

[ https://issues.apache.org/jira/browse/SPARK-5384?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-5384: - Assignee: yuhao yang Vectors.sqdist return inconsistent result for sparse/dense vectors when the

[jira] [Updated] (SPARK-5237) UDTF don't work with multi-alias of multi-columns as output on SparK SQL

[ https://issues.apache.org/jira/browse/SPARK-5237?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Zhou updated SPARK-5237: --- Summary: UDTF don't work with multi-alias of multi-columns as output on SparK SQL (was: UDTF don't work on

[jira] [Resolved] (SPARK-5384) Vectors.sqdist return inconsistent result for sparse/dense vectors when the vectors have different lengths

[ https://issues.apache.org/jira/browse/SPARK-5384?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng resolved SPARK-5384. -- Resolution: Fixed Fix Version/s: (was: 1.2.1) 1.3.0 Issue

[jira] [Created] (SPARK-5406) LocalLAPACK mode in RowMatrix.computeSVD should have much smaller upper bound

yuhao yang created SPARK-5406: - Summary: LocalLAPACK mode in RowMatrix.computeSVD should have much smaller upper bound Key: SPARK-5406 URL: https://issues.apache.org/jira/browse/SPARK-5406 Project: Spark

[jira] [Updated] (SPARK-5237) UDTF don't work with multi-alias of multi-columns as output on SparK SQL

[ https://issues.apache.org/jira/browse/SPARK-5237?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Zhou updated SPARK-5237: --- Description: Hive query with UDTF don't work on Spark SQL like below example SELECT

[jira] [Commented] (SPARK-2688) Need a way to run multiple data pipeline concurrently

[ https://issues.apache.org/jira/browse/SPARK-2688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14291214#comment-14291214 ] Imran Rashid commented on SPARK-2688: - [~airhorns] I completely agree with your use

[jira] [Comment Edited] (SPARK-5388) Provide a stable application submission gateway

[ https://issues.apache.org/jira/browse/SPARK-5388?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14290847#comment-14290847 ] Dale Richardson edited comment on SPARK-5388 at 1/26/15 5:39 AM:

[jira] [Created] (SPARK-5404) Statistic of Logical Plan is too aggresive

Cheng Hao created SPARK-5404: Summary: Statistic of Logical Plan is too aggresive Key: SPARK-5404 URL: https://issues.apache.org/jira/browse/SPARK-5404 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-5405) Spark clusterer should support high dimensional data

[ https://issues.apache.org/jira/browse/SPARK-5405?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Derrick Burns updated SPARK-5405: - Description: The MLLIB clusterer works well for low (200) dimensional data. However,

[jira] [Created] (SPARK-5407) No 1.2 AMI available for ec2

Håkan Jonsson created SPARK-5407: Summary: No 1.2 AMI available for ec2 Key: SPARK-5407 URL: https://issues.apache.org/jira/browse/SPARK-5407 Project: Spark Issue Type: Bug

[jira] [Closed] (SPARK-5268) CoarseGrainedExecutorBackend exits for irrelevant DisassociatedEvent

[ https://issues.apache.org/jira/browse/SPARK-5268?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or closed SPARK-5268. Resolution: Fixed Fix Version/s: 1.3.0 Assignee: Nan Zhu Target Version/s:

[jira] [Comment Edited] (SPARK-5388) Provide a stable application submission gateway

[ https://issues.apache.org/jira/browse/SPARK-5388?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14290847#comment-14290847 ] Dale Richardson edited comment on SPARK-5388 at 1/26/15 5:30 AM:

[jira] [Commented] (SPARK-5404) Statistic of Logical Plan is too aggresive

[ https://issues.apache.org/jira/browse/SPARK-5404?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14291488#comment-14291488 ] Apache Spark commented on SPARK-5404: - User 'chenghao-intel' has created a pull

[jira] [Created] (SPARK-5405) Spark clusterer should support high dimensional data

Derrick Burns created SPARK-5405: Summary: Spark clusterer should support high dimensional data Key: SPARK-5405 URL: https://issues.apache.org/jira/browse/SPARK-5405 Project: Spark Issue

[jira] [Resolved] (SPARK-5382) Scripts do not use SPARK_CONF_DIR where they should

[ https://issues.apache.org/jira/browse/SPARK-5382?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jacek Lewandowski resolved SPARK-5382. -- Resolution: Fixed Scripts do not use SPARK_CONF_DIR where they should

[jira] [Updated] (SPARK-4430) Apache RAT Checks fail spuriously on test files

[ https://issues.apache.org/jira/browse/SPARK-4430?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-4430: - Affects Version/s: 1.2.0 Apache RAT Checks fail spuriously on test files

[jira] [Commented] (SPARK-5406) LocalLAPACK mode in RowMatrix.computeSVD should have much smaller upper bound

[ https://issues.apache.org/jira/browse/SPARK-5406?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14291539#comment-14291539 ] Apache Spark commented on SPARK-5406: - User 'hhbyyh' has created a pull request for

[jira] [Commented] (SPARK-2645) Spark driver calls System.exit(50) after calling SparkContext.stop() the second time

[ https://issues.apache.org/jira/browse/SPARK-2645?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14291072#comment-14291072 ] Sean Owen commented on SPARK-2645: -- Just checking if you believe this is still an issue,

[jira] [Created] (SPARK-5403) Avoid spark-ec2 hanging on Waiting for (...) 'ssh-ready' state.

Grzegorz Dubicki created SPARK-5403: --- Summary: Avoid spark-ec2 hanging on Waiting for (...) 'ssh-ready' state. Key: SPARK-5403 URL: https://issues.apache.org/jira/browse/SPARK-5403 Project: Spark

[jira] [Commented] (SPARK-5403) Avoid spark-ec2 hanging on Waiting for (...) 'ssh-ready' state.

[ https://issues.apache.org/jira/browse/SPARK-5403?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14291096#comment-14291096 ] Apache Spark commented on SPARK-5403: - User 'grzegorz-dubicki' has created a pull

[jira] [Commented] (SPARK-2688) Need a way to run multiple data pipeline concurrently

[ https://issues.apache.org/jira/browse/SPARK-2688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14291134#comment-14291134 ] Xuefu Zhang commented on SPARK-2688: I think SPARK-3622 is related to this JIRA but

[jira] [Closed] (SPARK-5401) Executor ID should be set before MetricsSystem is created

[ https://issues.apache.org/jira/browse/SPARK-5401?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or closed SPARK-5401. Resolution: Fixed Fix Version/s: 1.2.1 1.3.0 Assignee: Ryan

[jira] [Closed] (SPARK-5402) Log executor ID at executor-construction time

[ https://issues.apache.org/jira/browse/SPARK-5402?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or closed SPARK-5402. Resolution: Fixed Fix Version/s: 1.2.1 1.3.0 Assignee: Ryan

[jira] [Commented] (SPARK-2285) Give various TaskEndReason subclass more descriptive names

[ https://issues.apache.org/jira/browse/SPARK-2285?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14291331#comment-14291331 ] Reynold Xin commented on SPARK-2285: Just in the we lose the PR in the future. This is

[jira] [Commented] (SPARK-4986) Graceful shutdown for Spark Streaming does not work in Standalone cluster mode

[ https://issues.apache.org/jira/browse/SPARK-4986?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14291349#comment-14291349 ] Jesper Lundgren commented on SPARK-4986: [~srowen] The current patch I have does

[jira] [Closed] (SPARK-5344) HistoryServer cannot recognize that inprogress file was renamed to completed file

[ https://issues.apache.org/jira/browse/SPARK-5344?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or closed SPARK-5344. Resolution: Fixed Fix Version/s: 1.3.0 Assignee: Kousuke Saruta HistoryServer cannot

[jira] [Resolved] (SPARK-2285) Give various TaskEndReason subclass more descriptive names

[ https://issues.apache.org/jira/browse/SPARK-2285?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-2285. -- Resolution: Won't Fix Closing as WontFix per the PR discussion. Give various TaskEndReason subclass

[jira] [Commented] (SPARK-822) defaultMinSplits can't be set higher than 2

[ https://issues.apache.org/jira/browse/SPARK-822?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14291235#comment-14291235 ] Josh Rosen commented on SPARK-822: -- I merged a pull request that adds a link to the