[jira] [Created] (SPARK-4659) Implement K-core decomposition algorithm

2014-11-29 Thread Xiaoming Li (JIRA)
Xiaoming Li created SPARK-4659: -- Summary: Implement K-core decomposition algorithm Key: SPARK-4659 URL: https://issues.apache.org/jira/browse/SPARK-4659 Project: Spark Issue Type: New Feature

[jira] [Issue Comment Deleted] (SPARK-4101) [MLLIB] Improve API in Word2Vec model

2014-11-29 Thread Ilya Ganelin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4101?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ilya Ganelin updated SPARK-4101: Comment: was deleted (was: If no-one is working on this I would be happy to knock this out. Thanks!

[jira] [Commented] (SPARK-4101) [MLLIB] Improve API in Word2Vec model

2014-11-29 Thread Ilya Ganelin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4101?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14229011#comment-14229011 ] Ilya Ganelin commented on SPARK-4101: - If no-one is working on this I would be happy t

[jira] [Resolved] (SPARK-4543) Javadoc failure for network-common causes publish-local to fail

2014-11-29 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4543?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell resolved SPARK-4543. Resolution: Duplicate This turned out to be an instance of SPARK-4193. > Javadoc failure fo

[jira] [Resolved] (SPARK-4507) PR merge script should support closing multiple JIRA tickets

2014-11-29 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4507?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell resolved SPARK-4507. Resolution: Fixed Fix Version/s: 1.3.0 Assignee: Takayuki Hasegawa > PR merg

[jira] [Commented] (SPARK-4658) Code documentation issue in DDL of datasource

2014-11-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14229005#comment-14229005 ] Apache Spark commented on SPARK-4658: - User 'ravipesala' has created a pull request fo

[jira] [Created] (SPARK-4658) Code documentation issue in DDL of datasource

2014-11-29 Thread Ravindra Pesala (JIRA)
Ravindra Pesala created SPARK-4658: -- Summary: Code documentation issue in DDL of datasource Key: SPARK-4658 URL: https://issues.apache.org/jira/browse/SPARK-4658 Project: Spark Issue Type: B

[jira] [Comment Edited] (SPARK-4630) Dynamically determine optimal number of partitions

2014-11-29 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4630?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14229001#comment-14229001 ] Patrick Wendell edited comment on SPARK-4630 at 11/30/14 3:29 AM: --

[jira] [Commented] (SPARK-4630) Dynamically determine optimal number of partitions

2014-11-29 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4630?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14229001#comment-14229001 ] Patrick Wendell commented on SPARK-4630: Hey Kos - before starting to work on the

[jira] [Updated] (SPARK-4657) RuntimeException: Unsupported datatype DecimalType()

2014-11-29 Thread pengyanhong (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4657?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] pengyanhong updated SPARK-4657: --- Description: execute a query statement on a Hive table which contains decimal data type field, than s

[jira] [Updated] (SPARK-4657) RuntimeException: Unsupported datatype DecimalType()

2014-11-29 Thread pengyanhong (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4657?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] pengyanhong updated SPARK-4657: --- Description: execute a query statement on a Hive table which contains decimal data type field, got er

[jira] [Created] (SPARK-4657) RuntimeException: Unsupported datatype DecimalType()

2014-11-29 Thread pengyanhong (JIRA)
pengyanhong created SPARK-4657: -- Summary: RuntimeException: Unsupported datatype DecimalType() Key: SPARK-4657 URL: https://issues.apache.org/jira/browse/SPARK-4657 Project: Spark Issue Type: Bu

[jira] [Commented] (SPARK-4656) Typo in Programming Guide markdown

2014-11-29 Thread Kai Sasaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14228998#comment-14228998 ] Kai Sasaki commented on SPARK-4656: --- Created the patch. Please review it. https://github

[jira] [Commented] (SPARK-4656) Typo in Programming Guide markdown

2014-11-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14228999#comment-14228999 ] Apache Spark commented on SPARK-4656: - User 'Lewuathe' has created a pull request for

[jira] [Created] (SPARK-4656) Typo in Programming Guide markdown

2014-11-29 Thread Kai Sasaki (JIRA)
Kai Sasaki created SPARK-4656: - Summary: Typo in Programming Guide markdown Key: SPARK-4656 URL: https://issues.apache.org/jira/browse/SPARK-4656 Project: Spark Issue Type: Bug Componen

[jira] [Updated] (SPARK-4628) Put external projects and examples behind a build flag

2014-11-29 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4628?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-4628: --- Summary: Put external projects and examples behind a build flag (was: Put all external projec

[jira] [Updated] (SPARK-4505) Reduce the memory usage of CompactBuffer[T] when T is a primitive type

2014-11-29 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4505?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-4505: --- Assignee: Shixiong Zhu > Reduce the memory usage of CompactBuffer[T] when T is a primitive typ

[jira] [Resolved] (SPARK-4505) Reduce the memory usage of CompactBuffer[T] when T is a primitive type

2014-11-29 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4505?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell resolved SPARK-4505. Resolution: Fixed Fix Version/s: 1.3.0 > Reduce the memory usage of CompactBuffer[T]

[jira] [Resolved] (SPARK-4057) Use -agentlib instead of -Xdebug in sbt-launch-lib.bash for debugging

2014-11-29 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4057?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell resolved SPARK-4057. Resolution: Fixed Fix Version/s: 1.3.0 Assignee: Kousuke Saruta > Use -agent

[jira] [Commented] (SPARK-4498) Standalone Master can fail to recognize completed/failed applications

2014-11-29 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4498?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14228966#comment-14228966 ] Josh Rosen commented on SPARK-4498: --- Here's an interesting pattern to grep for in all-ma

[jira] [Commented] (SPARK-4498) Standalone Master can fail to recognize completed/failed applications

2014-11-29 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4498?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14228952#comment-14228952 ] Josh Rosen commented on SPARK-4498: --- In addition to exploring the "missing Disassociated

[jira] [Commented] (SPARK-4498) Standalone Master can fail to recognize completed/failed applications

2014-11-29 Thread Mark Hamstra (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4498?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14228947#comment-14228947 ] Mark Hamstra commented on SPARK-4498: - On a quick look-through, your analysis looks li

[jira] [Commented] (SPARK-4498) Standalone Master can fail to recognize completed/failed applications

2014-11-29 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4498?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14228943#comment-14228943 ] Josh Rosen commented on SPARK-4498: --- Adding 1.1.1 as an affected version, too, since SPA

[jira] [Updated] (SPARK-4498) Standalone Master can fail to recognize completed/failed applications

2014-11-29 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4498?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-4498: -- Target Version/s: 1.2.0, 1.1.2 > Standalone Master can fail to recognize completed/failed applications >

[jira] [Updated] (SPARK-4498) Standalone Master can fail to recognize completed/failed applications

2014-11-29 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4498?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-4498: -- Affects Version/s: 1.1.1 > Standalone Master can fail to recognize completed/failed applications > -

[jira] [Updated] (SPARK-4498) Standalone Master can fail to recognize completed/failed applications

2014-11-29 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4498?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-4498: -- Priority: Blocker (was: Critical) > Standalone Master can fail to recognize completed/failed applicatio

[jira] [Commented] (SPARK-4498) Standalone Master can fail to recognize completed/failed applications

2014-11-29 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4498?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14228937#comment-14228937 ] Josh Rosen commented on SPARK-4498: --- Hi [~airhorns], I finally got a chance to look int

[jira] [Resolved] (SPARK-4622) Add the some error infomation if using spark-sql in yarn-cluster mode

2014-11-29 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4622?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen resolved SPARK-4622. --- Resolution: Duplicate > Add the some error infomation if using spark-sql in yarn-cluster mode > --

[jira] [Commented] (SPARK-4654) Clean up DAGScheduler's getMissingParentStages() and stageDependsOn() methods

2014-11-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14228919#comment-14228919 ] Apache Spark commented on SPARK-4654: - User 'JoshRosen' has created a pull request for

[jira] [Assigned] (SPARK-4653) DAGScheduler refactoring and cleanup

2014-11-29 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4653?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen reassigned SPARK-4653: - Assignee: Josh Rosen > DAGScheduler refactoring and cleanup > ---

[jira] [Created] (SPARK-4655) Split Stage into ShuffleMapStage and ResultStage subclasses

2014-11-29 Thread Josh Rosen (JIRA)
Josh Rosen created SPARK-4655: - Summary: Split Stage into ShuffleMapStage and ResultStage subclasses Key: SPARK-4655 URL: https://issues.apache.org/jira/browse/SPARK-4655 Project: Spark Issue Ty

[jira] [Created] (SPARK-4654) Clean up DAGScheduler's getMissingParentStages() and stageDependsOn() methods

2014-11-29 Thread Josh Rosen (JIRA)
Josh Rosen created SPARK-4654: - Summary: Clean up DAGScheduler's getMissingParentStages() and stageDependsOn() methods Key: SPARK-4654 URL: https://issues.apache.org/jira/browse/SPARK-4654 Project: Spark

[jira] [Created] (SPARK-4653) DAGScheduler refactoring and cleanup

2014-11-29 Thread Josh Rosen (JIRA)
Josh Rosen created SPARK-4653: - Summary: DAGScheduler refactoring and cleanup Key: SPARK-4653 URL: https://issues.apache.org/jira/browse/SPARK-4653 Project: Spark Issue Type: Improvement

[jira] [Updated] (SPARK-4648) Support COALESCE function in Spark SQL and HiveQL

2014-11-29 Thread Ravindra Pesala (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4648?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ravindra Pesala updated SPARK-4648: --- Summary: Support COALESCE function in Spark SQL and HiveQL (was: Support Coalesce in Spark SQ

[jira] [Updated] (SPARK-4648) Support Coalesce in Spark SQL.

2014-11-29 Thread Ravindra Pesala (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4648?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ravindra Pesala updated SPARK-4648: --- Description: Support Coalesce function in Spark SQL. Support type widening in Coalesce functio

[jira] [Commented] (SPARK-4644) Implement skewed join

2014-11-29 Thread Aaron Davidson (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4644?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14228839#comment-14228839 ] Aaron Davidson commented on SPARK-4644: --- [~zsxwing] I believe that this problem is r

[jira] [Commented] (SPARK-4082) Show Waiting/Queued Stages in Spark UI

2014-11-29 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4082?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14228820#comment-14228820 ] Patrick Wendell commented on SPARK-4082: IMO this is sufficiently addressed by the

[jira] [Commented] (SPARK-4652) Add docs about spark-git-repo option

2014-11-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4652?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14228789#comment-14228789 ] Apache Spark commented on SPARK-4652: - User 'Lewuathe' has created a pull request for

[jira] [Created] (SPARK-4652) Add docs about spark-git-repo option

2014-11-29 Thread Kai Sasaki (JIRA)
Kai Sasaki created SPARK-4652: - Summary: Add docs about spark-git-repo option Key: SPARK-4652 URL: https://issues.apache.org/jira/browse/SPARK-4652 Project: Spark Issue Type: Documentation

[jira] [Commented] (SPARK-4598) Paginate stage page to avoid OOM with > 100,000 tasks

2014-11-29 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4598?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14228684#comment-14228684 ] Josh Rosen commented on SPARK-4598: --- Actually, it might be pretty hard to trim down the

[jira] [Commented] (SPARK-4598) Paginate stage page to avoid OOM with > 100,000 tasks

2014-11-29 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4598?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14228677#comment-14228677 ] Josh Rosen commented on SPARK-4598: --- I was able to reproduce this issue using the SparkP

[jira] [Updated] (SPARK-4598) Paginate stage page to avoid OOM with > 100,000 tasks

2014-11-29 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4598?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-4598: -- Affects Version/s: 1.2.0 > Paginate stage page to avoid OOM with > 100,000 tasks > -