[jira] [Updated] (SPARK-3526) Docs section on data locality

2014-09-15 Thread Andrew Ash (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3526?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Ash updated SPARK-3526: -- Summary: Docs section on data locality (was: Section on data locality) Docs section on data locality

[jira] [Commented] (SPARK-3526) Docs section on data locality

2014-09-15 Thread Andrew Ash (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3526?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14133662#comment-14133662 ] Andrew Ash commented on SPARK-3526: --- Note: reports from users that reading from

[jira] [Created] (SPARK-3527) Strip the physical plan message margin

2014-09-15 Thread Cheng Hao (JIRA)
Cheng Hao created SPARK-3527: Summary: Strip the physical plan message margin Key: SPARK-3527 URL: https://issues.apache.org/jira/browse/SPARK-3527 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-3527) Strip the physical plan message margin

2014-09-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3527?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14133669#comment-14133669 ] Apache Spark commented on SPARK-3527: - User 'chenghao-intel' has created a pull

[jira] [Created] (SPARK-3528) Reading data from file:/// should be called NODE_LOCAL not PROCESS_LOCAL

2014-09-15 Thread Andrew Ash (JIRA)
Andrew Ash created SPARK-3528: - Summary: Reading data from file:/// should be called NODE_LOCAL not PROCESS_LOCAL Key: SPARK-3528 URL: https://issues.apache.org/jira/browse/SPARK-3528 Project: Spark

[jira] [Comment Edited] (SPARK-3526) Docs section on data locality

2014-09-15 Thread Andrew Ash (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3526?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14133662#comment-14133662 ] Andrew Ash edited comment on SPARK-3526 at 9/15/14 8:14 AM:

[jira] [Updated] (SPARK-3528) Reading data from file:/// should be called NODE_LOCAL not PROCESS_LOCAL

2014-09-15 Thread Andrew Ash (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3528?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Ash updated SPARK-3528: -- Description: Note that reading from {{file:///.../pom.xml}} is called a PROCESS_LOCAL task {noformat}

[jira] [Created] (SPARK-3529) Delete the temporal files after test exit

2014-09-15 Thread Cheng Hao (JIRA)
Cheng Hao created SPARK-3529: Summary: Delete the temporal files after test exit Key: SPARK-3529 URL: https://issues.apache.org/jira/browse/SPARK-3529 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-3529) Delete the temporal files after test exit

2014-09-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3529?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14133683#comment-14133683 ] Apache Spark commented on SPARK-3529: - User 'chenghao-intel' has created a pull

[jira] [Created] (SPARK-3530) Pipeline and Parameters

2014-09-15 Thread Xiangrui Meng (JIRA)
Xiangrui Meng created SPARK-3530: Summary: Pipeline and Parameters Key: SPARK-3530 URL: https://issues.apache.org/jira/browse/SPARK-3530 Project: Spark Issue Type: Sub-task

[jira] [Commented] (SPARK-2926) Add MR-style (merge-sort) SortShuffleReader for sort-based shuffle

2014-09-15 Thread Saisai Shao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2926?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14133694#comment-14133694 ] Saisai Shao commented on SPARK-2926: Hey [~rxin], here is the branch rebased on your

[jira] [Closed] (SPARK-3521) Missing modules in 1.1.0 source distribution - cant be build with maven

2014-09-15 Thread Radim Kolar (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3521?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Radim Kolar closed SPARK-3521. -- Resolution: Not a Problem Fix Version/s: 1.1.1 Compile problem is fixed on github branch-1.1

[jira] [Created] (SPARK-3531) select null from table would throw a MatchError

2014-09-15 Thread Adrian Wang (JIRA)
Adrian Wang created SPARK-3531: -- Summary: select null from table would throw a MatchError Key: SPARK-3531 URL: https://issues.apache.org/jira/browse/SPARK-3531 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-3531) select null from table would throw a MatchError

2014-09-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3531?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14133728#comment-14133728 ] Apache Spark commented on SPARK-3531: - User 'adrian-wang' has created a pull request

[jira] [Commented] (SPARK-3530) Pipeline and Parameters

2014-09-15 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3530?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14133760#comment-14133760 ] Sean Owen commented on SPARK-3530: -- A few high-level questions: Is this a rewrite of

[jira] [Commented] (SPARK-2594) Add CACHE TABLE name AS SELECT ...

2014-09-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2594?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14133776#comment-14133776 ] Apache Spark commented on SPARK-2594: - User 'ravipesala' has created a pull request

[jira] [Created] (SPARK-3532) Spark On FreeBSD. Snappy used by torrent broadcast fails to load native libs.

2014-09-15 Thread Prashant Sharma (JIRA)
Prashant Sharma created SPARK-3532: -- Summary: Spark On FreeBSD. Snappy used by torrent broadcast fails to load native libs. Key: SPARK-3532 URL: https://issues.apache.org/jira/browse/SPARK-3532

[jira] [Updated] (SPARK-3532) Spark On FreeBSD. Snappy used by torrent broadcast fails to load native libs.

2014-09-15 Thread Prashant Sharma (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3532?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prashant Sharma updated SPARK-3532: --- Description: While trying out spark on freebsd, this seemed like first blocker. Workaround:

[jira] [Commented] (SPARK-3532) Spark On FreeBSD. Snappy used by torrent broadcast fails to load native libs.

2014-09-15 Thread Radim Kolar (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3532?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14133821#comment-14133821 ] Radim Kolar commented on SPARK-3532: you need to grab snappy native library from

[jira] [Resolved] (SPARK-3410) The priority of shutdownhook for ApplicationMaster should not be integer literal

2014-09-15 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3410?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-3410. -- Resolution: Fixed Fix Version/s: 1.2.0 Assignee: Kousuke Saruta The priority

[jira] [Commented] (SPARK-3396) Change LogistricRegressionWithSGD's default regType to L2

2014-09-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3396?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14133917#comment-14133917 ] Apache Spark commented on SPARK-3396: - User 'BigCrunsh' has created a pull request for

[jira] [Commented] (SPARK-3528) Reading data from file:/// should be called NODE_LOCAL not PROCESS_LOCAL

2014-09-15 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3528?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14133927#comment-14133927 ] Nicholas Chammas commented on SPARK-3528: - [~aash] - How about for data read from

[jira] [Commented] (SPARK-3526) Docs section on data locality

2014-09-15 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3526?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14133935#comment-14133935 ] Nicholas Chammas commented on SPARK-3526: - FYI: Looks like the valid localities

[jira] [Resolved] (SPARK-3470) Have JavaSparkContext implement Closeable/AutoCloseable

2014-09-15 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3470?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-3470. -- Resolution: Fixed Fix Version/s: 1.2.0 Have JavaSparkContext implement Closeable/AutoCloseable

[jira] [Commented] (SPARK-1895) Run tests on windows

2014-09-15 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1895?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14133964#comment-14133964 ] Sean Owen commented on SPARK-1895: -- Can anyone still reproduce this? I know test temp

[jira] [Resolved] (SPARK-1258) RDD.countByValue optimization

2014-09-15 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1258?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-1258. -- Resolution: Won't Fix I'm taking the liberty of closing this, since this refers to an optimization

[jira] [Commented] (SPARK-3506) 1.1.0-SNAPSHOT in docs for 1.1.0 under docs/latest

2014-09-15 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3506?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14133976#comment-14133976 ] Sean Owen commented on SPARK-3506: -- Yeah, I imagine that can be touched up right now. For

[jira] [Commented] (SPARK-2620) case class cannot be used as key for reduce

2014-09-15 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2620?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14134096#comment-14134096 ] Sean Owen commented on SPARK-2620: -- FWIW, here is a mailing list comment that suggests

[jira] [Commented] (SPARK-2620) case class cannot be used as key for reduce

2014-09-15 Thread Daniel Siegmann (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2620?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14134101#comment-14134101 ] Daniel Siegmann commented on SPARK-2620: I have tested the case in spark-shell on

[jira] [Comment Edited] (SPARK-2984) FileNotFoundException on _temporary directory

2014-09-15 Thread Gregory Phillips (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2984?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14134073#comment-14134073 ] Gregory Phillips edited comment on SPARK-2984 at 9/15/14 4:43 PM:

[jira] [Commented] (SPARK-2932) Move MasterFailureTest out of main source directory

2014-09-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2932?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14134116#comment-14134116 ] Apache Spark commented on SPARK-2932: - User 'srowen' has created a pull request for

[jira] [Updated] (SPARK-1895) Run tests on windows

2014-09-15 Thread Andrew Ash (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1895?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Ash updated SPARK-1895: -- Description: bin\pyspark python\pyspark\rdd.py Sometimes tests complete without error _. Last

[jira] [Updated] (SPARK-1764) EOF reached before Python server acknowledged

2014-09-15 Thread Andrew Ash (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1764?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Ash updated SPARK-1764: -- Description: I'm getting EOF reached before Python server acknowledged while using PySpark on Mesos.

[jira] [Updated] (SPARK-2586) Lack of information to figure out connection to Tachyon master is inactive/ down

2014-09-15 Thread Andrew Ash (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2586?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Ash updated SPARK-2586: -- Description: When you running Spark with Tachyon, when the connection to Tachyon master is down (due

[jira] [Updated] (SPARK-2586) Lack of information to figure out connection to Tachyon master is inactive/ down

2014-09-15 Thread Andrew Ash (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2586?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Ash updated SPARK-2586: -- Description: When you running Spark with Tachyon, when the connection to Tachyon master is down (due

[jira] [Commented] (SPARK-1239) Don't fetch all map output statuses at each reducer during shuffles

2014-09-15 Thread Andrew Ash (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1239?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14134160#comment-14134160 ] Andrew Ash commented on SPARK-1239: --- For large statuses, would we expect that to exceed

[jira] [Commented] (SPARK-1239) Don't fetch all map output statuses at each reducer during shuffles

2014-09-15 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1239?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14134198#comment-14134198 ] Patrick Wendell commented on SPARK-1239: Yes, the current state of the art is to

[jira] [Resolved] (SPARK-3425) OpenJDK - when run with jvm 1.8, should not set MaxPermSize

2014-09-15 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3425?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell resolved SPARK-3425. Resolution: Fixed Fixed by: https://github.com/apache/spark/pull/2301 OpenJDK - when run

[jira] [Comment Edited] (SPARK-2377) Create a Python API for Spark Streaming

2014-09-15 Thread Jyotiska NK (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2377?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14134210#comment-14134210 ] Jyotiska NK edited comment on SPARK-2377 at 9/15/14 6:00 PM: -

[jira] [Commented] (SPARK-2377) Create a Python API for Spark Streaming

2014-09-15 Thread Jyotiska NK (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2377?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14134210#comment-14134210 ] Jyotiska NK commented on SPARK-2377: I have been watching the work going on PR #11. Is

[jira] [Commented] (SPARK-2377) Create a Python API for Spark Streaming

2014-09-15 Thread Matthew Farrellee (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2377?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14134225#comment-14134225 ] Matthew Farrellee commented on SPARK-2377: -- it's a little tricky. you need to

[jira] [Commented] (SPARK-2308) Add KMeans MiniBatch clustering algorithm to MLlib

2014-09-15 Thread Derrick Burns (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2308?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14134246#comment-14134246 ] Derrick Burns commented on SPARK-2308: -- I have implemented MiniBatch KMeans in Spark.

[jira] [Commented] (SPARK-2308) Add KMeans MiniBatch clustering algorithm to MLlib

2014-09-15 Thread RJ Nowling (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2308?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14134264#comment-14134264 ] RJ Nowling commented on SPARK-2308: --- It is true that we will save on the distance

[jira] [Created] (SPARK-3534) Avoid running MLlib and Streaming tests when testing SQL PRs

2014-09-15 Thread Michael Armbrust (JIRA)
Michael Armbrust created SPARK-3534: --- Summary: Avoid running MLlib and Streaming tests when testing SQL PRs Key: SPARK-3534 URL: https://issues.apache.org/jira/browse/SPARK-3534 Project: Spark

[jira] [Commented] (SPARK-3534) Avoid running MLlib and Streaming tests when testing SQL PRs

2014-09-15 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3534?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14134290#comment-14134290 ] Josh Rosen commented on SPARK-3534: --- Looks like this has been proposed before:

[jira] [Commented] (SPARK-2308) Add KMeans MiniBatch clustering algorithm to MLlib

2014-09-15 Thread Derrick Burns (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2308?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14134291#comment-14134291 ] Derrick Burns commented on SPARK-2308: -- I have submitted several issues regarding the

[jira] [Updated] (SPARK-1517) Publish nightly snapshots of documentation, maven artifacts, and binary builds

2014-09-15 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1517?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-1517: -- Fix Version/s: (was: 1.1.0) 1.2.0 We should revisit this for the 1.2.0 release

[jira] [Resolved] (SPARK-3104) Jenkins failing to test some PRs when asked to

2014-09-15 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3104?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen resolved SPARK-3104. --- Resolution: Cannot Reproduce Resolving this as cannot reproduce for now, since Jenkins seems to have

[jira] [Commented] (SPARK-2308) Add KMeans MiniBatch clustering algorithm to MLlib

2014-09-15 Thread RJ Nowling (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2308?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14134301#comment-14134301 ] RJ Nowling commented on SPARK-2308: --- I'm not a committer but [~mengxr] is. That said,

[jira] [Resolved] (SPARK-2232) Fix Jenkins tests in Maven

2014-09-15 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2232?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen resolved SPARK-2232. --- Resolution: Fixed This has been fixed; the Maven builds have now been green for a few days. Fix

[jira] [Updated] (SPARK-3533) Add saveAsTextFileByKey() method to RDDs

2014-09-15 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3533?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicholas Chammas updated SPARK-3533: Affects Version/s: 1.1.0 Add saveAsTextFileByKey() method to RDDs

[jira] [Commented] (SPARK-3219) K-Means clusterer should support Bregman distance functions

2014-09-15 Thread Derrick Burns (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3219?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14134339#comment-14134339 ] Derrick Burns commented on SPARK-3219: -- The key abstractions that need to be added to

[jira] [Comment Edited] (SPARK-3219) K-Means clusterer should support Bregman distance functions

2014-09-15 Thread Derrick Burns (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3219?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14134339#comment-14134339 ] Derrick Burns edited comment on SPARK-3219 at 9/15/14 7:14 PM:

[jira] [Comment Edited] (SPARK-3219) K-Means clusterer should support Bregman distance functions

2014-09-15 Thread Derrick Burns (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3219?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14134339#comment-14134339 ] Derrick Burns edited comment on SPARK-3219 at 9/15/14 7:16 PM:

[jira] [Comment Edited] (SPARK-3219) K-Means clusterer should support Bregman distance functions

2014-09-15 Thread Derrick Burns (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3219?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14134339#comment-14134339 ] Derrick Burns edited comment on SPARK-3219 at 9/15/14 7:16 PM:

[jira] [Comment Edited] (SPARK-3219) K-Means clusterer should support Bregman distance functions

2014-09-15 Thread Derrick Burns (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3219?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14134339#comment-14134339 ] Derrick Burns edited comment on SPARK-3219 at 9/15/14 7:15 PM:

[jira] [Comment Edited] (SPARK-3219) K-Means clusterer should support Bregman distance functions

2014-09-15 Thread Derrick Burns (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3219?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14134339#comment-14134339 ] Derrick Burns edited comment on SPARK-3219 at 9/15/14 7:23 PM:

[jira] [Comment Edited] (SPARK-3219) K-Means clusterer should support Bregman distance functions

2014-09-15 Thread Derrick Burns (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3219?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14134339#comment-14134339 ] Derrick Burns edited comment on SPARK-3219 at 9/15/14 7:23 PM:

[jira] [Comment Edited] (SPARK-3219) K-Means clusterer should support Bregman distance functions

2014-09-15 Thread Derrick Burns (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3219?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14134339#comment-14134339 ] Derrick Burns edited comment on SPARK-3219 at 9/15/14 7:23 PM:

[jira] [Comment Edited] (SPARK-3219) K-Means clusterer should support Bregman distance functions

2014-09-15 Thread Derrick Burns (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3219?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14134339#comment-14134339 ] Derrick Burns edited comment on SPARK-3219 at 9/15/14 7:24 PM:

[jira] [Comment Edited] (SPARK-3219) K-Means clusterer should support Bregman distance functions

2014-09-15 Thread Derrick Burns (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3219?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14134339#comment-14134339 ] Derrick Burns edited comment on SPARK-3219 at 9/15/14 7:26 PM:

[jira] [Comment Edited] (SPARK-3219) K-Means clusterer should support Bregman distance functions

2014-09-15 Thread Derrick Burns (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3219?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14134339#comment-14134339 ] Derrick Burns edited comment on SPARK-3219 at 9/15/14 7:26 PM:

[jira] [Comment Edited] (SPARK-3219) K-Means clusterer should support Bregman distance functions

2014-09-15 Thread Derrick Burns (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3219?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14120325#comment-14120325 ] Derrick Burns edited comment on SPARK-3219 at 9/15/14 7:26 PM:

[jira] [Commented] (SPARK-3308) Ability to read JSON Arrays as tables

2014-09-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3308?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14134385#comment-14134385 ] Apache Spark commented on SPARK-3308: - User 'yhuai' has created a pull request for

[jira] [Commented] (SPARK-3534) Avoid running MLlib and Streaming tests when testing SQL PRs

2014-09-15 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3534?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14134399#comment-14134399 ] Michael Armbrust commented on SPARK-3534: - Ah, yeah. Good catch Josh. I'll

[jira] [Commented] (SPARK-1405) parallel Latent Dirichlet Allocation (LDA) atop of spark in MLlib

2014-09-15 Thread Pedro Rodriguez (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1405?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14134445#comment-14134445 ] Pedro Rodriguez commented on SPARK-1405: Hi All. Just wanted to quickly introduce

[jira] [Created] (SPARK-3535) Spark on Mesos not correctly setting heap overhead

2014-09-15 Thread Brenden Matthews (JIRA)
Brenden Matthews created SPARK-3535: --- Summary: Spark on Mesos not correctly setting heap overhead Key: SPARK-3535 URL: https://issues.apache.org/jira/browse/SPARK-3535 Project: Spark Issue

[jira] [Commented] (SPARK-3535) Spark on Mesos not correctly setting heap overhead

2014-09-15 Thread Timothy St. Clair (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3535?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14134499#comment-14134499 ] Timothy St. Clair commented on SPARK-3535: -- Are you seeing this under fine

[jira] [Commented] (SPARK-3535) Spark on Mesos not correctly setting heap overhead

2014-09-15 Thread Brenden Matthews (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3535?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14134524#comment-14134524 ] Brenden Matthews commented on SPARK-3535: - I'm seeing this in fine grained mode. I

[jira] [Created] (SPARK-3536) SELECT on empty parquet table throws exception

2014-09-15 Thread Michael Armbrust (JIRA)
Michael Armbrust created SPARK-3536: --- Summary: SELECT on empty parquet table throws exception Key: SPARK-3536 URL: https://issues.apache.org/jira/browse/SPARK-3536 Project: Spark Issue

[jira] [Commented] (SPARK-3535) Spark on Mesos not correctly setting heap overhead

2014-09-15 Thread Brenden Matthews (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3535?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14134533#comment-14134533 ] Brenden Matthews commented on SPARK-3535: - I wrote a patch:

[jira] [Created] (SPARK-3537) Statistics for cached RDDs

2014-09-15 Thread Michael Armbrust (JIRA)
Michael Armbrust created SPARK-3537: --- Summary: Statistics for cached RDDs Key: SPARK-3537 URL: https://issues.apache.org/jira/browse/SPARK-3537 Project: Spark Issue Type: New Feature

[jira] [Created] (SPARK-3538) Provide way for workers to log messages to driver's out/err

2014-09-15 Thread Matthew Farrellee (JIRA)
Matthew Farrellee created SPARK-3538: Summary: Provide way for workers to log messages to driver's out/err Key: SPARK-3538 URL: https://issues.apache.org/jira/browse/SPARK-3538 Project: Spark

[jira] [Commented] (SPARK-3535) Spark on Mesos not correctly setting heap overhead

2014-09-15 Thread Andrew Ash (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3535?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14134592#comment-14134592 ] Andrew Ash commented on SPARK-3535: --- Why does the task need extra memory if the heap

[jira] [Commented] (SPARK-3535) Spark on Mesos not correctly setting heap overhead

2014-09-15 Thread Brenden Matthews (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3535?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14134599#comment-14134599 ] Brenden Matthews commented on SPARK-3535: - The JVM heap size does not include the

[jira] [Resolved] (SPARK-3518) Remove useless statement in JsonProtocol

2014-09-15 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3518?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell resolved SPARK-3518. Resolution: Fixed Fix Version/s: 1.2.0 1.1.1 Assignee:

[jira] [Updated] (SPARK-2505) Weighted Regularizer

2014-09-15 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2505?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-2505: --- Fix Version/s: (was: 1.1.0) 1.2.0 Weighted Regularizer

[jira] [Updated] (SPARK-2314) RDD actions are only overridden in Scala, not java or python

2014-09-15 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2314?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-2314: --- Fix Version/s: (was: 1.1.0) 1.2.0 RDD actions are only overridden in

[jira] [Updated] (SPARK-3403) NaiveBayes crashes with blas/lapack native libraries for breeze (netlib-java)

2014-09-15 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3403?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-3403: --- Fix Version/s: (was: 1.1.0) 1.2.0 NaiveBayes crashes with blas/lapack

[jira] [Updated] (SPARK-2703) Make Tachyon related unit tests execute without deploying a Tachyon system locally.

2014-09-15 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2703?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-2703: --- Fix Version/s: (was: 1.1.0) 1.2.0 Make Tachyon related unit tests

[jira] [Updated] (SPARK-3038) delete history server logs when there are too many logs

2014-09-15 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3038?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-3038: --- Fix Version/s: (was: 1.1.0) 1.2.0 delete history server logs when

[jira] [Updated] (SPARK-1832) Executor UI improvement suggestions

2014-09-15 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1832?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-1832: --- Fix Version/s: (was: 1.1.0) 1.2.0 Executor UI improvement suggestions

[jira] [Updated] (SPARK-2167) spark-submit should return exit code based on failure/success

2014-09-15 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2167?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-2167: --- Fix Version/s: (was: 1.1.0) 1.2.0 spark-submit should return exit

[jira] [Updated] (SPARK-2754) Document standalone-cluster mode now that it's working

2014-09-15 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2754?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-2754: --- Fix Version/s: (was: 1.1.0) 1.2.0 Document standalone-cluster mode

[jira] [Updated] (SPARK-2947) DAGScheduler resubmit the stage into an infinite loop

2014-09-15 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2947?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-2947: --- Fix Version/s: (was: 1.1.0) 1.2.0 DAGScheduler resubmit the stage

[jira] [Updated] (SPARK-2638) Improve concurrency of fetching Map outputs

2014-09-15 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2638?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-2638: --- Fix Version/s: (was: 1.1.0) 1.2.0 Improve concurrency of fetching Map

[jira] [Updated] (SPARK-1911) Warn users if their assembly jars are not built with Java 6

2014-09-15 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1911?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-1911: --- Fix Version/s: (was: 1.1.0) 1.2.0 Warn users if their assembly jars

[jira] [Updated] (SPARK-2793) Correctly lock directory creation in DiskBlockManager.getFile

2014-09-15 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2793?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-2793: --- Fix Version/s: (was: 1.1.0) 1.2.0 Correctly lock directory creation

[jira] [Updated] (SPARK-2069) MIMA false positives (umbrella)

2014-09-15 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2069?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-2069: --- Fix Version/s: (was: 1.1.0) 1.2.0 MIMA false positives (umbrella)

[jira] [Updated] (SPARK-1830) Deploy failover, Make Persistence engine and LeaderAgent Pluggable.

2014-09-15 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1830?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-1830: --- Fix Version/s: (was: 1.1.0) 1.2.0 Deploy failover, Make Persistence

[jira] [Updated] (SPARK-2795) Improve DiskBlockObjectWriter API

2014-09-15 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2795?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-2795: --- Fix Version/s: (was: 1.1.0) 1.2.0 Improve DiskBlockObjectWriter API

[jira] [Updated] (SPARK-1706) Allow multiple executors per worker in Standalone mode

2014-09-15 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1706?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-1706: --- Fix Version/s: (was: 1.1.0) 1.2.0 Allow multiple executors per worker

[jira] [Updated] (SPARK-1989) Exit executors faster if they get into a cycle of heavy GC

2014-09-15 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1989?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-1989: --- Fix Version/s: (was: 1.1.0) 1.2.0 Exit executors faster if they get

[jira] [Updated] (SPARK-1860) Standalone Worker cleanup should not clean up running executors

2014-09-15 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1860?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-1860: --- Fix Version/s: (was: 1.1.0) 1.2.0 Standalone Worker cleanup should

[jira] [Updated] (SPARK-1924) Make local:/ scheme work in more deploy modes

2014-09-15 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1924?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-1924: --- Fix Version/s: (was: 1.1.0) 1.2.0 Make local:/ scheme work in more

[jira] [Updated] (SPARK-1379) Calling .cache() on a SchemaRDD should do something more efficient than caching the individual row objects.

2014-09-15 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1379?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-1379: --- Fix Version/s: (was: 1.1.0) 1.2.0 Calling .cache() on a SchemaRDD

[jira] [Updated] (SPARK-1684) Merge script should standardize SPARK-XXX prefix

2014-09-15 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1684?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-1684: --- Fix Version/s: (was: 1.1.0) 1.2.0 Merge script should standardize

[jira] [Updated] (SPARK-1853) Show Streaming application code context (file, line number) in Spark Stages UI

2014-09-15 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1853?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-1853: --- Fix Version/s: (was: 1.1.0) 1.2.0 Show Streaming application code

[jira] [Updated] (SPARK-1201) Do not materialize partitions whenever possible in BlockManager

2014-09-15 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1201?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-1201: --- Fix Version/s: (was: 1.1.0) 1.2.0 Do not materialize partitions

[jira] [Updated] (SPARK-2159) Spark shell exit() does not stop SparkContext

2014-09-15 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2159?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-2159: --- Fix Version/s: (was: 1.1.0) 1.2.0 Spark shell exit() does not stop

  1   2   >