[jira] [Commented] (SPARK-3167) Port recent spark-submit changes to windows

2014-08-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3167?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14111912#comment-14111912 ] Apache Spark commented on SPARK-3167: - User 'andrewor14' has created a pull request

[jira] [Resolved] (SPARK-3167) Port recent spark-submit changes to windows

2014-08-27 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3167?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell resolved SPARK-3167. Resolution: Fixed Fix Version/s: 1.1.0 Issue resolved by pull request 2156

[jira] [Commented] (SPARK-3241) NumberFormat.getInstance() in SparkHiveHadoopWriter is not threadsafe

2014-08-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3241?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14111936#comment-14111936 ] Apache Spark commented on SPARK-3241: - User 'baishuo' has created a pull request for

[jira] [Created] (SPARK-3246) Support weighted SVMWithSGD for classification of unbalanced dataset

2014-08-27 Thread mahesh bhole (JIRA)
mahesh bhole created SPARK-3246: --- Summary: Support weighted SVMWithSGD for classification of unbalanced dataset Key: SPARK-3246 URL: https://issues.apache.org/jira/browse/SPARK-3246 Project: Spark

[jira] [Updated] (SPARK-3245) spark insert into hbase class not serialize

2014-08-27 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-3245?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] 刘勇 updated SPARK-3245: -- Description: val result: org.apache.spark.rdd.RDD[(String, Int)] result.foreach(res ={ var put = new

[jira] [Updated] (SPARK-3245) spark insert into hbase class not serialize

2014-08-27 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-3245?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] 刘勇 updated SPARK-3245: -- Description: val result: org.apache.spark.rdd.RDD[(String, Int)] result.foreach(res ={ var put = new

[jira] [Commented] (SPARK-3245) spark insert into hbase class not serialize

2014-08-27 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-3245?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14111948#comment-14111948 ] 刘勇 commented on SPARK-3245: --- before this NotSerializableException class was

[jira] [Resolved] (SPARK-3139) Akka timeouts from ContextCleaner when cleaning shuffles

2014-08-27 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3139?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell resolved SPARK-3139. Resolution: Fixed Fix Version/s: 1.1.0 Issue resolved by pull request 2143

[jira] [Resolved] (SPARK-2298) Show stage attempt in UI

2014-08-27 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2298?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell resolved SPARK-2298. Resolution: Fixed Assignee: Reynold Xin (was: Masayoshi TSUZUKI) Show stage

[jira] [Commented] (SPARK-2298) Show stage attempt in UI

2014-08-27 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2298?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14111981#comment-14111981 ] Patrick Wendell commented on SPARK-2298: fixed by:

[jira] [Resolved] (SPARK-2884) Create binary builds in parallel with release script

2014-08-27 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2884?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell resolved SPARK-2884. Resolution: Fixed This was fixed in:

[jira] [Updated] (SPARK-2608) Mesos scheduler backend create executor launch command not correctly

2014-08-27 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-2608: --- Priority: Critical (was: Minor) Mesos scheduler backend create executor launch command not

[jira] [Updated] (SPARK-2608) Mesos doesn't handle spark.executor.extraJavaOptions correctly (among other things)

2014-08-27 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-2608: --- Summary: Mesos doesn't handle spark.executor.extraJavaOptions correctly (among other things)

[jira] [Updated] (SPARK-2608) Mesos doesn't handle spark.executor.extraJavaOptions correctly (among other things)

2014-08-27 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-2608: --- Priority: Blocker (was: Critical) Mesos doesn't handle spark.executor.extraJavaOptions

[jira] [Resolved] (SPARK-2921) Mesos doesn't handle spark.executor.extraJavaOptions correctly (among other things)

2014-08-27 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2921?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell resolved SPARK-2921. Resolution: Duplicate Mesos doesn't handle spark.executor.extraJavaOptions correctly

[jira] [Resolved] (SPARK-3237) Push down of predicates with UDFS into parquet scan can result in serialization errors

2014-08-27 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3237?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-3237. - Resolution: Fixed Fix Version/s: 1.1.0 Push down of predicates with UDFS into

[jira] [Updated] (SPARK-2554) CountDistinct and SumDistinct should do partial aggregation

2014-08-27 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2554?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-2554: Target Version/s: 1.2.0 (was: 1.1.0) CountDistinct and SumDistinct should do partial

[jira] [Updated] (SPARK-2554) CountDistinct and SumDistinct should do partial aggregation

2014-08-27 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2554?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-2554: Assignee: (was: Michael Armbrust) CountDistinct and SumDistinct should do partial

[jira] [Commented] (SPARK-2554) CountDistinct and SumDistinct should do partial aggregation

2014-08-27 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2554?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14112005#comment-14112005 ] Michael Armbrust commented on SPARK-2554: - CountDistinct was done in the above PR.

[jira] [Updated] (SPARK-2594) Add CACHE TABLE name AS SELECT ...

2014-08-27 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2594?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-2594: Priority: Critical (was: Major) Add CACHE TABLE name AS SELECT ...

[jira] [Updated] (SPARK-2816) Type-safe SQL queries

2014-08-27 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2816?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-2816: Priority: Critical (was: Major) Type-safe SQL queries -

[jira] [Updated] (SPARK-2554) CountDistinct and SumDistinct should do partial aggregation

2014-08-27 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2554?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-2554: Priority: Minor (was: Blocker) CountDistinct and SumDistinct should do partial

[jira] [Updated] (SPARK-2360) CSV import to SchemaRDDs

2014-08-27 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2360?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-2360: Issue Type: Sub-task (was: New Feature) Parent: SPARK-3247 CSV import to

[jira] [Commented] (SPARK-2721) Fix MapType compatibility issues with reading Parquet datasets

2014-08-27 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14112011#comment-14112011 ] Michael Armbrust commented on SPARK-2721: - I think this was fixed by [SPARK-3036].

[jira] [Resolved] (SPARK-2721) Fix MapType compatibility issues with reading Parquet datasets

2014-08-27 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2721?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-2721. - Resolution: Fixed Fix Version/s: 1.1.0 Fix MapType compatibility issues with

[jira] [Updated] (SPARK-3236) Reading Parquet tables from Metastore mangles location

2014-08-27 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3236?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-3236: Priority: Blocker (was: Major) Target Version/s: 1.1.0 Reading Parquet

[jira] [Resolved] (SPARK-3227) Add MLlib migration guide (1.0 - 1.1)

2014-08-27 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3227?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng resolved SPARK-3227. -- Resolution: Fixed Fix Version/s: 1.1.0 Issue resolved by pull request 2146

[jira] [Commented] (SPARK-3220) K-Means clusterer should perform K-Means initialization in parallel

2014-08-27 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3220?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14112055#comment-14112055 ] Xiangrui Meng commented on SPARK-3220: -- By `parallel`, do you mean multi-threading on

[jira] [Updated] (SPARK-3218) K-Means clusterer can fail on degenerate data

2014-08-27 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3218?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-3218: - Target Version/s: 1.2.0 K-Means clusterer can fail on degenerate data

[jira] [Updated] (SPARK-3218) K-Means clusterer can fail on degenerate data

2014-08-27 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3218?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-3218: - Assignee: Derrick Burns K-Means clusterer can fail on degenerate data

[jira] [Commented] (SPARK-3218) K-Means clusterer can fail on degenerate data

2014-08-27 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3218?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14112059#comment-14112059 ] Xiangrui Meng commented on SPARK-3218: -- [~derrickburns] Please send a PR and ping me

[jira] [Resolved] (SPARK-3154) Make FlumePollingInputDStream shutdown cleaner

2014-08-27 Thread Tathagata Das (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3154?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tathagata Das resolved SPARK-3154. -- Resolution: Fixed Fix Version/s: 1.1.0 Make FlumePollingInputDStream shutdown cleaner

[jira] [Updated] (SPARK-2377) Create a Python API for Spark Streaming

2014-08-27 Thread Tathagata Das (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2377?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tathagata Das updated SPARK-2377: - Assignee: Kenichi Takagiwa (was: Tathagata Das) Create a Python API for Spark Streaming

[jira] [Updated] (SPARK-1405) parallel Latent Dirichlet Allocation (LDA) atop of spark in MLlib

2014-08-27 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1405?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-1405: - Target Version/s: 1.2.0 Affects Version/s: (was: 1.1.0) Fix Version/s: (was:

[jira] [Updated] (SPARK-1473) Feature selection for high dimensional datasets

2014-08-27 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1473?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-1473: - Target Version/s: 1.2.0 Fix Version/s: (was: 1.1.0) Feature selection for high

[jira] [Updated] (SPARK-1473) Feature selection for high dimensional datasets

2014-08-27 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1473?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-1473: - Assignee: Alexander Ulanov Feature selection for high dimensional datasets

[jira] [Commented] (SPARK-2308) Add KMeans MiniBatch clustering algorithm to MLlib

2014-08-27 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2308?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14112103#comment-14112103 ] Xiangrui Meng commented on SPARK-2308: -- Yes, it is O(n), unfortunately. To have more

[jira] [Comment Edited] (SPARK-1647) Prevent data loss when Streaming driver goes down

2014-08-27 Thread Giulio De Vecchi (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1647?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14110942#comment-14110942 ] Giulio De Vecchi edited comment on SPARK-1647 at 8/27/14 10:33 AM:

[jira] [Comment Edited] (SPARK-1647) Prevent data loss when Streaming driver goes down

2014-08-27 Thread Giulio De Vecchi (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1647?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14110942#comment-14110942 ] Giulio De Vecchi edited comment on SPARK-1647 at 8/27/14 10:34 AM:

[jira] [Created] (SPARK-3250) More Efficient Sampling

2014-08-27 Thread RJ Nowling (JIRA)
RJ Nowling created SPARK-3250: - Summary: More Efficient Sampling Key: SPARK-3250 URL: https://issues.apache.org/jira/browse/SPARK-3250 Project: Spark Issue Type: Improvement

[jira] [Updated] (SPARK-3250) More Efficient Sampling

2014-08-27 Thread RJ Nowling (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3250?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] RJ Nowling updated SPARK-3250: -- Description: Sampling, as currently implemented in Spark, is an O(n) operation. A number of

[jira] [Updated] (SPARK-3250) More Efficient Sampling

2014-08-27 Thread RJ Nowling (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3250?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] RJ Nowling updated SPARK-3250: -- Description: Sampling, as currently implemented in Spark, is an O\(n\) operation. A number of

[jira] [Commented] (SPARK-2429) Hierarchical Implementation of KMeans

2014-08-27 Thread RJ Nowling (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2429?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=1411#comment-1411 ] RJ Nowling commented on SPARK-2429: --- Discussion on the dev list mentioned a community

[jira] [Commented] (SPARK-2966) Add an approximation algorithm for hierarchical clustering to MLlib

2014-08-27 Thread RJ Nowling (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2966?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14112223#comment-14112223 ] RJ Nowling commented on SPARK-2966: --- This is a duplicate of SPARK-2429. Please see the

[jira] [Created] (SPARK-3251) Clarify learning interfaces

2014-08-27 Thread Christoph Sawade (JIRA)
Christoph Sawade created SPARK-3251: --- Summary: Clarify learning interfaces Key: SPARK-3251 URL: https://issues.apache.org/jira/browse/SPARK-3251 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-3251) Clarify learning interfaces

2014-08-27 Thread Christoph Sawade (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3251?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14112231#comment-14112231 ] Christoph Sawade commented on SPARK-3251: -

[jira] [Created] (SPARK-3252) Add missing condition in one SQL test

2014-08-27 Thread Liang-Chi Hsieh (JIRA)
Liang-Chi Hsieh created SPARK-3252: -- Summary: Add missing condition in one SQL test Key: SPARK-3252 URL: https://issues.apache.org/jira/browse/SPARK-3252 Project: Spark Issue Type: Test

[jira] [Commented] (SPARK-3252) Add missing condition in one SQL test

2014-08-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3252?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14112292#comment-14112292 ] Apache Spark commented on SPARK-3252: - User 'viirya' has created a pull request for

[jira] [Created] (SPARK-3253) KMeans cluster will fail on large number of clusters/high dimensional data

2014-08-27 Thread Derrick Burns (JIRA)
Derrick Burns created SPARK-3253: Summary: KMeans cluster will fail on large number of clusters/high dimensional data Key: SPARK-3253 URL: https://issues.apache.org/jira/browse/SPARK-3253 Project:

[jira] [Resolved] (SPARK-2933) Cleanup unnecessary and duplicated code in Yarn module

2014-08-27 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2933?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-2933. -- Resolution: Fixed Fix Version/s: 1.2.0 Cleanup unnecessary and duplicated code in Yarn

[jira] [Resolved] (SPARK-3152) Yarn AM cluster mode doesn't cleanup staging directory when it exits cleanly

2014-08-27 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3152?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-3152. -- Resolution: Duplicate Yarn AM cluster mode doesn't cleanup staging directory when it exits

[jira] [Commented] (SPARK-3152) Yarn AM cluster mode doesn't cleanup staging directory when it exits cleanly

2014-08-27 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3152?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14112377#comment-14112377 ] Thomas Graves commented on SPARK-3152: -- this was fixed by SPARK-2933 in master. I

[jira] [Updated] (SPARK-3219) K-Means clusterer should support Bregman distance functions

2014-08-27 Thread Derrick Burns (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3219?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Derrick Burns updated SPARK-3219: - Summary: K-Means clusterer should support Bregman distance functions (was: K-Means clusterer

[jira] [Resolved] (SPARK-3170) Bug Fix in Storage UI

2014-08-27 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3170?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or resolved SPARK-3170. -- Resolution: Fixed Bug Fix in Storage UI - Key: SPARK-3170

[jira] [Updated] (SPARK-1545) Add Random Forest algorithm to MLlib

2014-08-27 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1545?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-1545: - Target Version/s: 1.2.0 Affects Version/s: (was: 1.0.0) Add Random Forest algorithm to

[jira] [Resolved] (SPARK-953) Latent Dirichlet Association (LDA model)

2014-08-27 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-953?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng resolved SPARK-953. - Resolution: Duplicate Latent Dirichlet Association (LDA model)

[jira] [Resolved] (SPARK-3239) Choose disks for spilling randomly

2014-08-27 Thread Matei Zaharia (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3239?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matei Zaharia resolved SPARK-3239. -- Resolution: Fixed Fix Version/s: 1.1.0 Choose disks for spilling randomly

[jira] [Updated] (SPARK-3239) Choose disks for spilling randomly in PySpark

2014-08-27 Thread Matei Zaharia (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3239?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matei Zaharia updated SPARK-3239: - Summary: Choose disks for spilling randomly in PySpark (was: Choose disks for spilling

[jira] [Commented] (SPARK-2608) Mesos doesn't handle spark.executor.extraJavaOptions correctly (among other things)

2014-08-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14112536#comment-14112536 ] Apache Spark commented on SPARK-2608: - User 'liancheng' has created a pull request for

[jira] [Updated] (SPARK-3256) Enable :cp to add JARs in spark-shell (Scala 2.10)

2014-08-27 Thread Matei Zaharia (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3256?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matei Zaharia updated SPARK-3256: - Summary: Enable :cp to add JARs in spark-shell (Scala 2.10) (was: Enable :cp to add JARs in

[jira] [Created] (SPARK-3257) Enable :cp to add JARs in spark-shell (Scala 2.11)

2014-08-27 Thread Matei Zaharia (JIRA)
Matei Zaharia created SPARK-3257: Summary: Enable :cp to add JARs in spark-shell (Scala 2.11) Key: SPARK-3257 URL: https://issues.apache.org/jira/browse/SPARK-3257 Project: Spark Issue Type:

[jira] [Created] (SPARK-3254) Streaming K-Means

2014-08-27 Thread Xiangrui Meng (JIRA)
Xiangrui Meng created SPARK-3254: Summary: Streaming K-Means Key: SPARK-3254 URL: https://issues.apache.org/jira/browse/SPARK-3254 Project: Spark Issue Type: New Feature

[jira] [Created] (SPARK-3258) Python API for streaming MLlib algorithms

2014-08-27 Thread Xiangrui Meng (JIRA)
Xiangrui Meng created SPARK-3258: Summary: Python API for streaming MLlib algorithms Key: SPARK-3258 URL: https://issues.apache.org/jira/browse/SPARK-3258 Project: Spark Issue Type: Umbrella

[jira] [Updated] (SPARK-3256) Enable :cp to add JARs in spark-shell (Scala 2.10)

2014-08-27 Thread Matei Zaharia (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3256?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matei Zaharia updated SPARK-3256: - Fix Version/s: (was: 1.2.0) Enable :cp to add JARs in spark-shell (Scala 2.10)

[jira] [Created] (SPARK-3259) User data should be given to the master

2014-08-27 Thread Allan Douglas R. de Oliveira (JIRA)
Allan Douglas R. de Oliveira created SPARK-3259: --- Summary: User data should be given to the master Key: SPARK-3259 URL: https://issues.apache.org/jira/browse/SPARK-3259 Project: Spark

[jira] [Commented] (SPARK-3259) User data should be given to the master

2014-08-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3259?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14112616#comment-14112616 ] Apache Spark commented on SPARK-3259: - User 'douglaz' has created a pull request for

[jira] [Resolved] (SPARK-2501) Handle stage re-submissions properly in the UI

2014-08-27 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2501?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell resolved SPARK-2501. Resolution: Fixed Fix Version/s: 1.1.0 Fixed as a result of SPARK-3020 Handle

[jira] [Comment Edited] (SPARK-2501) Handle stage re-submissions properly in the UI

2014-08-27 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2501?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14112624#comment-14112624 ] Patrick Wendell edited comment on SPARK-2501 at 8/27/14 6:45 PM:

[jira] [Assigned] (SPARK-2501) Handle stage re-submissions properly in the UI

2014-08-27 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2501?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell reassigned SPARK-2501: -- Assignee: Patrick Wendell (was: Masayoshi TSUZUKI) Handle stage re-submissions

[jira] [Comment Edited] (SPARK-2501) Handle stage re-submissions properly in the UI

2014-08-27 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2501?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14112624#comment-14112624 ] Patrick Wendell edited comment on SPARK-2501 at 8/27/14 6:45 PM:

[jira] [Updated] (SPARK-3260) Yarn - pass acls along with executor launch

2014-08-27 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3260?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves updated SPARK-3260: - Target Version/s: 1.1.1, 1.2.0 (was: 1.2.0) Yarn - pass acls along with executor launch

[jira] [Created] (SPARK-3260) Yarn - pass acls along with executor launch

2014-08-27 Thread Thomas Graves (JIRA)
Thomas Graves created SPARK-3260: Summary: Yarn - pass acls along with executor launch Key: SPARK-3260 URL: https://issues.apache.org/jira/browse/SPARK-3260 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-3213) spark_ec2.py cannot find slave instances launched with Launch More Like This

2014-08-27 Thread Vida Ha (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14112652#comment-14112652 ] Vida Ha commented on SPARK-3213: I have a pull request that fixes the issue by copying the

[jira] [Created] (SPARK-3261) KMeans clusterer can return duplicate cluster centers

2014-08-27 Thread Derrick Burns (JIRA)
Derrick Burns created SPARK-3261: Summary: KMeans clusterer can return duplicate cluster centers Key: SPARK-3261 URL: https://issues.apache.org/jira/browse/SPARK-3261 Project: Spark Issue

[jira] [Updated] (SPARK-3213) spark_ec2.py cannot find slave instances launched with Launch More Like This

2014-08-27 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-3213: --- Priority: Critical (was: Major) spark_ec2.py cannot find slave instances launched with

[jira] [Updated] (SPARK-3213) spark_ec2.py cannot find slave instances launched with Launch More Like This

2014-08-27 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-3213: --- Priority: Major (was: Blocker) spark_ec2.py cannot find slave instances launched with

[jira] [Created] (SPARK-3262) CREATE VIEW is not supported but the error message is not clear

2014-08-27 Thread Xiangrui Meng (JIRA)
Xiangrui Meng created SPARK-3262: Summary: CREATE VIEW is not supported but the error message is not clear Key: SPARK-3262 URL: https://issues.apache.org/jira/browse/SPARK-3262 Project: Spark

[jira] [Resolved] (SPARK-2608) Mesos doesn't handle spark.executor.extraJavaOptions correctly (among other things)

2014-08-27 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell resolved SPARK-2608. Resolution: Fixed Resolved by: https://github.com/apache/spark/pull/2161 However, it

[jira] [Updated] (SPARK-3259) User data should be given to the master

2014-08-27 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3259?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-3259: --- Assignee: Allan Douglas R. de Oliveira User data should be given to the master

[jira] [Resolved] (SPARK-3259) User data should be given to the master

2014-08-27 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3259?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell resolved SPARK-3259. Resolution: Fixed Fix Version/s: 1.1.0 Issue resolved by pull request 2162

[jira] [Resolved] (SPARK-3118) add SHOW TBLPROPERTIES tblname; and SHOW COLUMNS (FROM|IN) table_name [(FROM|IN) db_name] support

2014-08-27 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3118?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-3118. - Resolution: Fixed Fix Version/s: 1.1.0 add SHOW TBLPROPERTIES tblname; and SHOW

[jira] [Resolved] (SPARK-3197) Reduce the expression tree object creation from the aggregation functions (min/max)

2014-08-27 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3197?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-3197. - Resolution: Fixed Fix Version/s: 1.1.0 Reduce the expression tree object

[jira] [Commented] (SPARK-3155) Support DecisionTree pruning

2014-08-27 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14112735#comment-14112735 ] Joseph K. Bradley commented on SPARK-3155: -- That sounds good---thank you!

[jira] [Updated] (SPARK-2706) Enable Spark to support Hive 0.13

2014-08-27 Thread Zhan Zhang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2706?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhan Zhang updated SPARK-2706: -- Attachment: v1.0.2.diff This is the patch against v1.0.2. I didn't fix the test cases. The regular

[jira] [Updated] (SPARK-3256) Enable :cp to add JARs in spark-shell (Scala 2.10)

2014-08-27 Thread Matei Zaharia (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3256?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matei Zaharia updated SPARK-3256: - Assignee: Chip Senkbeil Enable :cp to add JARs in spark-shell (Scala 2.10)

[jira] [Resolved] (SPARK-3256) Enable :cp to add JARs in spark-shell (Scala 2.10)

2014-08-27 Thread Matei Zaharia (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3256?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matei Zaharia resolved SPARK-3256. -- Resolution: Fixed Fix Version/s: 1.2.0 Enable :cp to add JARs in spark-shell (Scala

[jira] [Commented] (SPARK-3213) spark_ec2.py cannot find slave instances launched with Launch More Like This

2014-08-27 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14112748#comment-14112748 ] Joseph K. Bradley commented on SPARK-3213: -- Testing now... spark_ec2.py cannot

[jira] [Commented] (SPARK-2895) Support mapPartitionsWithContext in Spark Java API

2014-08-27 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2895?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14112756#comment-14112756 ] Reynold Xin commented on SPARK-2895: BTW feel free to submit a pull request on this.

[jira] [Commented] (SPARK-3155) Support DecisionTree pruning

2014-08-27 Thread Manish Amde (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14112752#comment-14112752 ] Manish Amde commented on SPARK-3155: [~chouqin] Thanks! I would suggest doing the

[jira] [Resolved] (SPARK-3138) sqlContext.parquetFile should be able to take a single file as parameter

2014-08-27 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3138?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-3138. - Resolution: Fixed Fix Version/s: 1.1.0 sqlContext.parquetFile should be able to

[jira] [Resolved] (SPARK-2871) Missing API in PySpark

2014-08-27 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2871?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen resolved SPARK-2871. --- Resolution: Fixed Fix Version/s: 1.2.0 Issue resolved by pull request 2093

[jira] [Updated] (SPARK-3213) spark_ec2.py cannot find slave instances launched with Launch More Like This

2014-08-27 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-3213: -- Target Version/s: 1.1.0 spark_ec2.py cannot find slave instances launched with Launch More Like This

[jira] [Commented] (SPARK-3215) Add remote interface for SparkContext

2014-08-27 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14112769#comment-14112769 ] Reynold Xin commented on SPARK-3215: I looked at the document. The high level proposal

[jira] [Created] (SPARK-3263) PR #720 broke GraphGenerator.logNormal

2014-08-27 Thread RJ Nowling (JIRA)
RJ Nowling created SPARK-3263: - Summary: PR #720 broke GraphGenerator.logNormal Key: SPARK-3263 URL: https://issues.apache.org/jira/browse/SPARK-3263 Project: Spark Issue Type: Bug

[jira] [Comment Edited] (SPARK-3215) Add remote interface for SparkContext

2014-08-27 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14112769#comment-14112769 ] Reynold Xin edited comment on SPARK-3215 at 8/27/14 8:27 PM: -

[jira] [Commented] (SPARK-3215) Add remote interface for SparkContext

2014-08-27 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14112780#comment-14112780 ] Marcelo Vanzin commented on SPARK-3215: --- Hi Reynold, thanks for the comments. This

[jira] [Commented] (SPARK-3061) Maven build fails in Windows OS

2014-08-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3061?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14112784#comment-14112784 ] Apache Spark commented on SPARK-3061: - User 'JoshRosen' has created a pull request for

[jira] [Commented] (SPARK-3215) Add remote interface for SparkContext

2014-08-27 Thread Matei Zaharia (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14112791#comment-14112791 ] Matei Zaharia commented on SPARK-3215: -- Hey Marcelo, while this could be useful for

[jira] [Commented] (SPARK-3215) Add remote interface for SparkContext

2014-08-27 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14112803#comment-14112803 ] Marcelo Vanzin commented on SPARK-3215: --- Hi Matei, Both suggestions came up during

[jira] [Created] (SPARK-3264) Allow users to set executor Spark home in Mesos

2014-08-27 Thread Andrew Or (JIRA)
Andrew Or created SPARK-3264: Summary: Allow users to set executor Spark home in Mesos Key: SPARK-3264 URL: https://issues.apache.org/jira/browse/SPARK-3264 Project: Spark Issue Type: Bug

  1   2   >