[jira] [Commented] (SPARK-554) Add foldByKey and aggregateByKey

[ https://issues.apache.org/jira/browse/SPARK-554?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13993069#comment-13993069 ] Sandy Ryza commented on SPARK-554: -- I've run into a couple situations where aggregateByKey

[jira] [Resolved] (SPARK-1690) RDD.saveAsTextFile throws scala.MatchError if RDD contains empty elements

[ https://issues.apache.org/jira/browse/SPARK-1690?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell resolved SPARK-1690. Resolution: Fixed Fix Version/s: 1.0.0 Fixed by: https://github.com/apache/spark/pul

[jira] [Assigned] (SPARK-1708) Add ClassTag parameter on accumulator and broadcast methods

[ https://issues.apache.org/jira/browse/SPARK-1708?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matei Zaharia reassigned SPARK-1708: Assignee: Matei Zaharia > Add ClassTag parameter on accumulator and broadcast methods > ---

[jira] [Commented] (SPARK-1796) spark-submit does not set driver memory correctly

[ https://issues.apache.org/jira/browse/SPARK-1796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13994442#comment-13994442 ] Patrick Wendell commented on SPARK-1796: https://github.com/apache/spark/pull/730

[jira] [Commented] (SPARK-1776) Have Spark's SBT build read dependencies from Maven

[ https://issues.apache.org/jira/browse/SPARK-1776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13993407#comment-13993407 ] Guoqiang Li commented on SPARK-1776: But I would think only use maven is better > Hav

[jira] [Created] (SPARK-1796) spark-submit does not set driver memory correctly

Patrick Wendell created SPARK-1796: -- Summary: spark-submit does not set driver memory correctly Key: SPARK-1796 URL: https://issues.apache.org/jira/browse/SPARK-1796 Project: Spark Issue Typ

[jira] [Created] (SPARK-1795) Add recursive directory file search to fileInputStream

Rick OToole created SPARK-1795: -- Summary: Add recursive directory file search to fileInputStream Key: SPARK-1795 URL: https://issues.apache.org/jira/browse/SPARK-1795 Project: Spark Issue Type:

[jira] [Commented] (SPARK-1756) Add missing description to spark-env.sh.template

[ https://issues.apache.org/jira/browse/SPARK-1756?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13994421#comment-13994421 ] Guoqiang Li commented on SPARK-1756: Hi [~pwendell] {{./bin/spark-shell --driver-memo

[jira] [Commented] (SPARK-1785) Streaming requires receivers to be serializable

[ https://issues.apache.org/jira/browse/SPARK-1785?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13994186#comment-13994186 ] Hari Shreedharan commented on SPARK-1785: - This really is a documentation jira. In

[jira] [Resolved] (SPARK-1756) Add missing description to spark-env.sh.template

[ https://issues.apache.org/jira/browse/SPARK-1756?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell resolved SPARK-1756. Resolution: Won't Fix This option is deprecated now and has been replaced by setting --driv

[jira] [Resolved] (SPARK-1774) SparkSubmit --jars not working for yarn-client

[ https://issues.apache.org/jira/browse/SPARK-1774?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell resolved SPARK-1774. Resolution: Fixed Issue resolved by pull request 710 [https://github.com/apache/spark/pull/

[jira] [Updated] (SPARK-1789) Multiple versions of Netty dependencies cause FlumeStreamSuite failure

[ https://issues.apache.org/jira/browse/SPARK-1789?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-1789: --- Assignee: Sean Owen > Multiple versions of Netty dependencies cause FlumeStreamSuite failure

[jira] [Resolved] (SPARK-1789) Multiple versions of Netty dependencies cause FlumeStreamSuite failure

[ https://issues.apache.org/jira/browse/SPARK-1789?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell resolved SPARK-1789. Resolution: Fixed Fix Version/s: 1.0.0 Issue resolved by pull request 723 [https://g

[jira] [Assigned] (SPARK-1770) repartition and coalesce(shuffle=true) put objects with the same key in the same bucket

[ https://issues.apache.org/jira/browse/SPARK-1770?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell reassigned SPARK-1770: -- Assignee: Patrick Wendell > repartition and coalesce(shuffle=true) put objects with the

[jira] [Commented] (SPARK-1758) failing test org.apache.spark.JavaAPISuite.wholeTextFiles

[ https://issues.apache.org/jira/browse/SPARK-1758?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13992556#comment-13992556 ] Nishkam Ravi commented on SPARK-1758: - Related with SPARK-1575 > failing test org.apa

[jira] [Commented] (SPARK-1763) SparkSubmit arguments do not propagate to python files on YARN

[ https://issues.apache.org/jira/browse/SPARK-1763?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13993157#comment-13993157 ] Matei Zaharia commented on SPARK-1763: -- Is this only for Python or for spark-submit i

[jira] [Commented] (SPARK-1778) Add 'limit' transformation to SchemaRDD.

[ https://issues.apache.org/jira/browse/SPARK-1778?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13993461#comment-13993461 ] Takuya Ueshin commented on SPARK-1778: -- Pull-requested: https://github.com/apache/spa

[jira] [Commented] (SPARK-1770) repartition and coalesce(shuffle=true) put objects with the same key in the same bucket

[ https://issues.apache.org/jira/browse/SPARK-1770?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13994352#comment-13994352 ] Patrick Wendell commented on SPARK-1770: It was a mistake on my part - I accidenta

[jira] [Comment Edited] (SPARK-1764) EOF reached before Python server acknowledged

[ https://issues.apache.org/jira/browse/SPARK-1764?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13992994#comment-13992994 ] Bouke van der Bijl edited comment on SPARK-1764 at 5/8/14 6:31 PM: -

[jira] [Commented] (SPARK-1758) failing test org.apache.spark.JavaAPISuite.wholeTextFiles

[ https://issues.apache.org/jira/browse/SPARK-1758?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13992561#comment-13992561 ] Nishkam Ravi commented on SPARK-1758: - Pull requests: https://github.com/apache/spark

[jira] [Created] (SPARK-1791) SVM implementation does not use threshold parameter

Andrew Tulloch created SPARK-1791: - Summary: SVM implementation does not use threshold parameter Key: SPARK-1791 URL: https://issues.apache.org/jira/browse/SPARK-1791 Project: Spark Issue Typ

[jira] [Updated] (SPARK-571) Forbid return statements when cleaning closures

[ https://issues.apache.org/jira/browse/SPARK-571?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-571: -- Assignee: William Benton > Forbid return statements when cleaning closures > ---

[jira] [Created] (SPARK-1792) Missing Spark-Shell Configure Options

Joseph E. Gonzalez created SPARK-1792: - Summary: Missing Spark-Shell Configure Options Key: SPARK-1792 URL: https://issues.apache.org/jira/browse/SPARK-1792 Project: Spark Issue Type: Bug

[jira] [Created] (SPARK-1794) Generic ADMM implementation for SVM, lasso, and L1-regularized logistic regression

Andrew Tulloch created SPARK-1794: - Summary: Generic ADMM implementation for SVM, lasso, and L1-regularized logistic regression Key: SPARK-1794 URL: https://issues.apache.org/jira/browse/SPARK-1794 Pr

[jira] [Updated] (SPARK-1755) Spark-submit --name does not resolve to application name on YARN

[ https://issues.apache.org/jira/browse/SPARK-1755?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-1755: - Affects Version/s: (was: 1.0.0) 0.9.1 > Spark-submit --name does not resolve t

[jira] [Resolved] (SPARK-1686) Master switches thread when ElectedLeader

[ https://issues.apache.org/jira/browse/SPARK-1686?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aaron Davidson resolved SPARK-1686. --- Resolution: Fixed > Master switches thread when ElectedLeader > -

[jira] [Resolved] (SPARK-1326) make-distribution.sh's Tachyon support relies on GNU sed

[ https://issues.apache.org/jira/browse/SPARK-1326?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandeep Singh resolved SPARK-1326. -- Resolution: Fixed > make-distribution.sh's Tachyon support relies on GNU sed >

[jira] [Created] (SPARK-1777) Pass "cached" blocks directly to disk if memory is not large enough

Patrick Wendell created SPARK-1777: -- Summary: Pass "cached" blocks directly to disk if memory is not large enough Key: SPARK-1777 URL: https://issues.apache.org/jira/browse/SPARK-1777 Project: Spark

[jira] [Updated] (SPARK-1777) Pass "cached" blocks directly to disk if memory is not large enough

[ https://issues.apache.org/jira/browse/SPARK-1777?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-1777: --- Issue Type: Improvement (was: New Feature) > Pass "cached" blocks directly to disk if memory

[jira] [Created] (SPARK-1793) Heavily duplicated test setup code in SVMSUite

Andrew Tulloch created SPARK-1793: - Summary: Heavily duplicated test setup code in SVMSUite Key: SPARK-1793 URL: https://issues.apache.org/jira/browse/SPARK-1793 Project: Spark Issue Type: Im

[jira] [Updated] (SPARK-1793) Heavily duplicated test setup code in SVMSuite

[ https://issues.apache.org/jira/browse/SPARK-1793?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Tulloch updated SPARK-1793: -- Summary: Heavily duplicated test setup code in SVMSuite (was: Heavily duplicated test setup co

[jira] [Commented] (SPARK-1760) mvn -Dsuites=* test throw an ClassNotFoundException

[ https://issues.apache.org/jira/browse/SPARK-1760?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13993505#comment-13993505 ] Guoqiang Li commented on SPARK-1760: Yes, [PR 712 |https://github.com/apache/spark/pul

[jira] [Updated] (SPARK-1775) Unneeded lock in ShuffleMapTask.deserializeInfo

[ https://issues.apache.org/jira/browse/SPARK-1775?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matei Zaharia updated SPARK-1775: - Priority: Critical (was: Major) > Unneeded lock in ShuffleMapTask.deserializeInfo >

[jira] [Updated] (SPARK-1752) Standardize input/output format for vectors and labeled points

[ https://issues.apache.org/jira/browse/SPARK-1752?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-1752: - Fix Version/s: (was: 1.0.0) > Standardize input/output format for vectors and labeled points

[jira] [Updated] (SPARK-1755) Spark-submit --name does not resolve to application name on YARN

[ https://issues.apache.org/jira/browse/SPARK-1755?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-1755: --- Priority: Blocker (was: Major) > Spark-submit --name does not resolve to application name on

[jira] [Commented] (SPARK-1758) failing test org.apache.spark.JavaAPISuite.wholeTextFiles

[ https://issues.apache.org/jira/browse/SPARK-1758?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13992555#comment-13992555 ] Nishkam Ravi commented on SPARK-1758: - The prefix "file:" is missing in the string ins

[jira] [Commented] (SPARK-1519) support minPartitions parameter of wholeTextFiles() in pyspark

[ https://issues.apache.org/jira/browse/SPARK-1519?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13992980#comment-13992980 ] Kan Zhang commented on SPARK-1519: -- PR: https://github.com/apache/spark/pull/697 > suppo

[jira] [Updated] (SPARK-1775) Unneeded lock in ShuffleMapTask.deserializeInfo

[ https://issues.apache.org/jira/browse/SPARK-1775?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-1775: --- Assignee: Sandeep Singh > Unneeded lock in ShuffleMapTask.deserializeInfo > -

[jira] [Commented] (SPARK-1768) History Server enhancements

[ https://issues.apache.org/jira/browse/SPARK-1768?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13993185#comment-13993185 ] Marcelo Vanzin commented on SPARK-1768: --- BTW I'm working on this (patch isn't ready

[jira] [Commented] (SPARK-1755) Spark-submit --name does not resolve to application name on YARN

[ https://issues.apache.org/jira/browse/SPARK-1755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13993196#comment-13993196 ] Andrew Or commented on SPARK-1755: -- You are correct. However, it also doesn't work for ya

[jira] [Resolved] (SPARK-1754) Add missing arithmetic DSL operations.

[ https://issues.apache.org/jira/browse/SPARK-1754?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell resolved SPARK-1754. Resolution: Fixed Fix Version/s: 1.0.0 Resolved in: https://github.com/apache/spark/

[jira] [Closed] (SPARK-1763) SparkSubmit arguments do not propagate to python files on YARN

[ https://issues.apache.org/jira/browse/SPARK-1763?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or closed SPARK-1763. Resolution: Cannot Reproduce > SparkSubmit arguments do not propagate to python files on YARN > ---

[jira] [Updated] (SPARK-1776) Have Spark's SBT build read dependencies from Maven

[ https://issues.apache.org/jira/browse/SPARK-1776?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-1776: --- Description: We've wanted to consolidate Spark's build for a while see [here|http://mail-arc

[jira] [Commented] (SPARK-1763) SparkSubmit arguments do not propagate to python files on YARN

[ https://issues.apache.org/jira/browse/SPARK-1763?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13993218#comment-13993218 ] Andrew Or commented on SPARK-1763: -- Wait, unable to reproduce with the latest mater... >

[jira] [Created] (SPARK-1773) Standalone cluster docs should be updated to reflect Spark Submit

Patrick Wendell created SPARK-1773: -- Summary: Standalone cluster docs should be updated to reflect Spark Submit Key: SPARK-1773 URL: https://issues.apache.org/jira/browse/SPARK-1773 Project: Spark

[jira] [Created] (SPARK-1768) History Server enhancements

Marcelo Vanzin created SPARK-1768: - Summary: History Server enhancements Key: SPARK-1768 URL: https://issues.apache.org/jira/browse/SPARK-1768 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-1394) calling system.platform on worker raises IOError

[ https://issues.apache.org/jira/browse/SPARK-1394?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13993339#comment-13993339 ] Vlad Frolov commented on SPARK-1394: [~sgottipa] spark/python/pyspark/daemon.py:75

[jira] [Commented] (SPARK-1763) SparkSubmit arguments do not propagate to python files on YARN

[ https://issues.apache.org/jira/browse/SPARK-1763?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13993198#comment-13993198 ] Andrew Or commented on SPARK-1763: -- This is only for python. There is a seemingly related

[jira] [Resolved] (SPARK-1725) Can't use broadcast variables in pyspark on Mesos because pyspark isn't added to PYTHONPATH

[ https://issues.apache.org/jira/browse/SPARK-1725?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell resolved SPARK-1725. Resolution: Fixed Fix Version/s: 1.0.0 > Can't use broadcast variables in pyspark on

[jira] [Updated] (SPARK-1754) Add missing arithmetic DSL operations.

[ https://issues.apache.org/jira/browse/SPARK-1754?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-1754: --- Assignee: Takuya Ueshin > Add missing arithmetic DSL operations. > --

[jira] [Commented] (SPARK-1755) Spark-submit --name does not resolve to application name on YARN

[ https://issues.apache.org/jira/browse/SPARK-1755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13993197#comment-13993197 ] Andrew Or commented on SPARK-1755: -- https://github.com/apache/spark/pull/699 > Spark-sub

[jira] [Comment Edited] (SPARK-1763) SparkSubmit arguments do not propagate to python files on YARN

[ https://issues.apache.org/jira/browse/SPARK-1763?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13993218#comment-13993218 ] Andrew Or edited comment on SPARK-1763 at 5/8/14 10:40 PM: --- Wait

[jira] [Updated] (SPARK-1775) Unneeded lock in ShuffleMapTask.deserializeInfo

[ https://issues.apache.org/jira/browse/SPARK-1775?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matei Zaharia updated SPARK-1775: - Labels: Starter (was: ) > Unneeded lock in ShuffleMapTask.deserializeInfo >

[jira] [Resolved] (SPARK-1755) Spark-submit --name does not resolve to application name on YARN

[ https://issues.apache.org/jira/browse/SPARK-1755?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell resolved SPARK-1755. Resolution: Fixed Issue resolved by pull request 699 [https://github.com/apache/spark/pull/

[jira] [Created] (SPARK-1776) Have Spark's SBT build read dependencies from Maven

Patrick Wendell created SPARK-1776: -- Summary: Have Spark's SBT build read dependencies from Maven Key: SPARK-1776 URL: https://issues.apache.org/jira/browse/SPARK-1776 Project: Spark Issue T

[jira] [Commented] (SPARK-1725) Can't use broadcast variables in pyspark on Mesos because pyspark isn't added to PYTHONPATH

[ https://issues.apache.org/jira/browse/SPARK-1725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13993358#comment-13993358 ] Patrick Wendell commented on SPARK-1725: Closed via: https://github.com/apache/spa

[jira] [Commented] (SPARK-554) Add foldByKey and aggregateByKey

[ https://issues.apache.org/jira/browse/SPARK-554?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13993294#comment-13993294 ] Sandy Ryza commented on SPARK-554: -- https://github.com/apache/spark/pull/705 > Add foldBy

[jira] [Created] (SPARK-1775) Unneeded lock in ShuffleMapTask.deserializeInfo

Matei Zaharia created SPARK-1775: Summary: Unneeded lock in ShuffleMapTask.deserializeInfo Key: SPARK-1775 URL: https://issues.apache.org/jira/browse/SPARK-1775 Project: Spark Issue Type: Bug

[jira] [Created] (SPARK-1774) SparkSubmit --jars not working for yarn-client

Andrew Or created SPARK-1774: Summary: SparkSubmit --jars not working for yarn-client Key: SPARK-1774 URL: https://issues.apache.org/jira/browse/SPARK-1774 Project: Spark Issue Type: Improvement

[jira] [Resolved] (SPARK-1631) App name set in SparkConf (not in JVM properties) not respected by Yarn backend

[ https://issues.apache.org/jira/browse/SPARK-1631?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell resolved SPARK-1631. Resolution: Fixed Fix Version/s: 1.0.0 Issue resolved by pull request 539 [https://g