[jira] [Commented] (SPARK-2228) onStageSubmitted does not properly called so NoSuchElement will be thrown in onStageCompleted

[ https://issues.apache.org/jira/browse/SPARK-2228?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14045626#comment-14045626 ] Patrick Wendell commented on SPARK-2228: [~rxin] unfortunately I think it's more

[jira] [Updated] (SPARK-2292) NullPointerException in JavaPairRDD.mapToPair

[ https://issues.apache.org/jira/browse/SPARK-2292?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bharath Ravi Kumar updated SPARK-2292: -- Summary: NullPointerException in JavaPairRDD.mapToPair (was: NullPointerException in

[jira] [Updated] (SPARK-2292) NullPointerException in JavaPairRDD.mapToPair

[ https://issues.apache.org/jira/browse/SPARK-2292?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bharath Ravi Kumar updated SPARK-2292: -- Description: Correction: Invoking JavaPairRDD.mapToPair results in an NPE: {noformat}

[jira] [Updated] (SPARK-2292) NullPointerException in JavaPairRDD.mapToPair

[ https://issues.apache.org/jira/browse/SPARK-2292?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bharath Ravi Kumar updated SPARK-2292: -- Environment: Spark 1.0.0, Standalone with the master single slave running on Ubuntu

[jira] [Commented] (SPARK-2292) NullPointerException in JavaPairRDD.mapToPair

[ https://issues.apache.org/jira/browse/SPARK-2292?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14045663#comment-14045663 ] Bharath Ravi Kumar commented on SPARK-2292: --- Assuming this bug can be

[jira] [Commented] (SPARK-2290) Worker should directly use its own sparkHome instead of appDesc.sparkHome when LaunchExecutor

[ https://issues.apache.org/jira/browse/SPARK-2290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14045691#comment-14045691 ] YanTang Zhai commented on SPARK-2290: - I've created PR:

[jira] [Resolved] (SPARK-2291) Update EC2 scripts to use instance storage on m3 instance types

[ https://issues.apache.org/jira/browse/SPARK-2291?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell resolved SPARK-2291. Resolution: Duplicate Was already fixed by this PR:

[jira] [Created] (SPARK-2305) pyspark - depend on py4j 0.8.1

Matthew Farrellee created SPARK-2305: Summary: pyspark - depend on py4j 0.8.1 Key: SPARK-2305 URL: https://issues.apache.org/jira/browse/SPARK-2305 Project: Spark Issue Type:

[jira] [Commented] (SPARK-2228) onStageSubmitted does not properly called so NoSuchElement will be thrown in onStageCompleted

[ https://issues.apache.org/jira/browse/SPARK-2228?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14046102#comment-14046102 ] Baoxu Shi commented on SPARK-2228: -- I think a workaround would be adding

[jira] [Commented] (SPARK-2003) SparkContext(SparkConf) doesn't work in pyspark

[ https://issues.apache.org/jira/browse/SPARK-2003?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14046152#comment-14046152 ] Matthew Farrellee commented on SPARK-2003: -- first up - reproducer should be

[jira] [Commented] (SPARK-2003) SparkContext(SparkConf) doesn't work in pyspark

[ https://issues.apache.org/jira/browse/SPARK-2003?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14046161#comment-14046161 ] Matthew Farrellee commented on SPARK-2003: -- actually, the attribute issue may be

[jira] [Commented] (SPARK-2003) SparkContext(SparkConf) doesn't work in pyspark

[ https://issues.apache.org/jira/browse/SPARK-2003?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14046168#comment-14046168 ] Matthew Farrellee commented on SPARK-2003: -- pull request -

[jira] [Commented] (SPARK-1394) calling system.platform on worker raises IOError

[ https://issues.apache.org/jira/browse/SPARK-1394?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14046216#comment-14046216 ] Matthew Farrellee commented on SPARK-1394: -- i'm taking a look at this calling

[jira] [Updated] (SPARK-2307) SparkUI Storage page cached statuses incorrect

[ https://issues.apache.org/jira/browse/SPARK-2307?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-2307: - Attachment: Screen Shot 2014-06-27 at 11.09.54 AM.png SparkUI Storage page cached statuses incorrect

[jira] [Updated] (SPARK-2307) SparkUI Storage page cached statuses incorrect

[ https://issues.apache.org/jira/browse/SPARK-2307?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-2307: - Description: See attached: the executor has 512MB, but somehow it has cached (279 + 27 + 279 + 27) =

[jira] [Commented] (SPARK-1394) calling system.platform on worker raises IOError

[ https://issues.apache.org/jira/browse/SPARK-1394?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14046349#comment-14046349 ] Matthew Farrellee commented on SPARK-1394: -- the python daemon does two levels of

[jira] [Updated] (SPARK-2309) Generalize the binary logistic regression into multinomial logistic regression

[ https://issues.apache.org/jira/browse/SPARK-2309?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] DB Tsai updated SPARK-2309: --- Description: Currently, there is no multi-class classifier in mllib. Logistic regression can be extended to

[jira] [Created] (SPARK-2309) Generalize the binary logistic regression into multinomial logistic regression

DB Tsai created SPARK-2309: -- Summary: Generalize the binary logistic regression into multinomial logistic regression Key: SPARK-2309 URL: https://issues.apache.org/jira/browse/SPARK-2309 Project: Spark

[jira] [Commented] (SPARK-1392) Local spark-shell Runs Out of Memory With Default Settings

[ https://issues.apache.org/jira/browse/SPARK-1392?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14046374#comment-14046374 ] Andrew Or commented on SPARK-1392: -- and by fixed by SPARK-1777 he means

[jira] [Comment Edited] (SPARK-2293) Replace RDD.zip usage by map with predict inside.

[ https://issues.apache.org/jira/browse/SPARK-2293?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14045874#comment-14045874 ] Sean Owen edited comment on SPARK-2293 at 6/27/14 8:53 PM: --- I

[jira] [Commented] (SPARK-2292) NullPointerException in JavaPairRDD.mapToPair

[ https://issues.apache.org/jira/browse/SPARK-2292?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14046426#comment-14046426 ] Reynold Xin commented on SPARK-2292: FYI I cannot reproduce the problem (using the

[jira] [Commented] (SPARK-2292) NullPointerException in JavaPairRDD.mapToPair

[ https://issues.apache.org/jira/browse/SPARK-2292?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14046429#comment-14046429 ] Reynold Xin commented on SPARK-2292: Strange - I can't reproduce this on 1.0 either.

[jira] [Commented] (SPARK-2111) pyspark errors when SPARK_PRINT_LAUNCH_COMMAND=1

[ https://issues.apache.org/jira/browse/SPARK-2111?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14046439#comment-14046439 ] Matthew Farrellee commented on SPARK-2111: -- i'll take a look at this pyspark

[jira] [Comment Edited] (SPARK-2292) NullPointerException in JavaPairRDD.mapToPair

[ https://issues.apache.org/jira/browse/SPARK-2292?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14046445#comment-14046445 ] Patrick Wendell edited comment on SPARK-2292 at 6/27/14 9:54 PM:

[jira] [Updated] (SPARK-2307) SparkUI Storage page cached statuses incorrect

[ https://issues.apache.org/jira/browse/SPARK-2307?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-2307: --- Assignee: Andrew Or SparkUI Storage page cached statuses incorrect

[jira] [Resolved] (SPARK-2307) SparkUI Storage page cached statuses incorrect

[ https://issues.apache.org/jira/browse/SPARK-2307?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell resolved SPARK-2307. Resolution: Fixed Fix Version/s: 1.0.1 Issue resolved by pull request 1249

[jira] [Updated] (SPARK-2310) Support arbitrary options on the command line with spark-submit

[ https://issues.apache.org/jira/browse/SPARK-2310?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandy Ryza updated SPARK-2310: -- Summary: Support arbitrary options on the command line with spark-submit (was: Allow giving arbitrary

[jira] [Updated] (SPARK-2311) Added additional GLMs (Poisson and Gamma) into MLlib

[ https://issues.apache.org/jira/browse/SPARK-2311?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaokai Wei updated SPARK-2311: --- Summary: Added additional GLMs (Poisson and Gamma) into MLlib (was: Added additional GLMs into

[jira] [Updated] (SPARK-2311) Added additional GLMs (Poisson and Gamma) into MLlib

[ https://issues.apache.org/jira/browse/SPARK-2311?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaokai Wei updated SPARK-2311: --- Description: Though GeneralizedLinearModel in MLlib 1.0.0 has some important GLMs such as Logistic

[jira] [Resolved] (SPARK-2259) Spark submit documentation for --deploy-mode is highly misleading

[ https://issues.apache.org/jira/browse/SPARK-2259?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell resolved SPARK-2259. Resolution: Fixed Fix Version/s: 1.1.0 1.0.1 Issue resolved by

[jira] [Updated] (SPARK-2243) Support multiple SparkContexts in the same JVM

[ https://issues.apache.org/jira/browse/SPARK-2243?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-2243: --- Summary: Support multiple SparkContexts in the same JVM (was: Using several Spark Contexts)

[jira] [Commented] (SPARK-2243) Support multiple SparkContexts in the same JVM

[ https://issues.apache.org/jira/browse/SPARK-2243?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14046574#comment-14046574 ] Patrick Wendell commented on SPARK-2243: This is not supported - but it's

[jira] [Commented] (SPARK-2138) The KMeans algorithm in the MLlib can lead to the Serialized Task size become bigger and bigger

[ https://issues.apache.org/jira/browse/SPARK-2138?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14046578#comment-14046578 ] Xiangrui Meng commented on SPARK-2138: -- [~piotrszul] The silent failure is due to a

[jira] [Commented] (SPARK-2111) pyspark errors when SPARK_PRINT_LAUNCH_COMMAND=1

[ https://issues.apache.org/jira/browse/SPARK-2111?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14046586#comment-14046586 ] Patrick Wendell commented on SPARK-2111: I was thinking that SPARK-2313 might be a

[jira] [Created] (SPARK-2313) PySpark should accept port via a command line argument rather than STDIN

Patrick Wendell created SPARK-2313: -- Summary: PySpark should accept port via a command line argument rather than STDIN Key: SPARK-2313 URL: https://issues.apache.org/jira/browse/SPARK-2313 Project:

[jira] [Commented] (SPARK-2138) The KMeans algorithm in the MLlib can lead to the Serialized Task size become bigger and bigger

[ https://issues.apache.org/jira/browse/SPARK-2138?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14046597#comment-14046597 ] DjvuLee commented on SPARK-2138: If this bug fixed, shall we closed this issue? [~mengxr]

[jira] [Commented] (SPARK-1945) Add full Java examples in MLlib docs

[ https://issues.apache.org/jira/browse/SPARK-1945?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14046599#comment-14046599 ] Michael Yannakopoulos commented on SPARK-1945: -- Hi guys, At the MLlib

[jira] [Created] (SPARK-2314) RDD actions are only overridden in Scala, not java or python

Michael Armbrust created SPARK-2314: --- Summary: RDD actions are only overridden in Scala, not java or python Key: SPARK-2314 URL: https://issues.apache.org/jira/browse/SPARK-2314 Project: Spark

[jira] [Created] (SPARK-2315) drop, dropRight and dropWhile which take RDD input and return RDD

Erik Erlandson created SPARK-2315: - Summary: drop, dropRight and dropWhile which take RDD input and return RDD Key: SPARK-2315 URL: https://issues.apache.org/jira/browse/SPARK-2315 Project: Spark

[jira] [Commented] (SPARK-2315) drop, dropRight and dropWhile which take RDD input and return RDD

[ https://issues.apache.org/jira/browse/SPARK-2315?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14046725#comment-14046725 ] Erik Erlandson commented on SPARK-2315: --- PR: