[jira] [Commented] (SPARK-6022) GraphX `diff` test incorrectly operating on values (not VertexId's)

2015-03-02 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6022?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14344699#comment-14344699 ] Takeshi Yamamuro commented on SPARK-6022: - Is the test correct? According to the c

[jira] [Issue Comment Deleted] (SPARK-6131) Spark 1.3.0 (RC1) missing some source files in sql.api.java

2015-03-02 Thread Littlestar (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6131?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Littlestar updated SPARK-6131: -- Comment: was deleted (was: spark-1.2.1 is ok. spark-1.2.1-src.jar\sql\core\src\main\scala\org\apache\spa

[jira] [Commented] (SPARK-6132) Context cleaner thread lives across SparkContexts

2015-03-02 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6132?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14344630#comment-14344630 ] Apache Spark commented on SPARK-6132: - User 'andrewor14' has created a pull request fo

[jira] [Commented] (SPARK-6020) Flaky test: o.a.s.sql.columnar.PartitionBatchPruningSuite

2015-03-02 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6020?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14344628#comment-14344628 ] Andrew Or commented on SPARK-6020: -- Ok, I will close this as resolved for now. We can alw

[jira] [Closed] (SPARK-6020) Flaky test: o.a.s.sql.columnar.PartitionBatchPruningSuite

2015-03-02 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6020?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or closed SPARK-6020. Resolution: Fixed > Flaky test: o.a.s.sql.columnar.PartitionBatchPruningSuite >

[jira] [Updated] (SPARK-6132) Context cleaner thread lives across SparkContexts

2015-03-02 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6132?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-6132: - Description: The context cleaner thread is not stopped properly. If a SparkContext is started immediately

[jira] [Updated] (SPARK-6132) Context cleaner thread lives across SparkContexts

2015-03-02 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6132?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-6132: - Description: The context cleaner thread is not stopped properly. If a SparkContext is started immediately

[jira] [Updated] (SPARK-6132) Context cleaner thread lives across SparkContexts

2015-03-02 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6132?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-6132: - Description: The context cleaner thread is not stopped properly. If a SparkContext is started immediately

[jira] [Resolved] (SPARK-6120) DecisionTree.save uses too much Java heap space for default spark shell settings

2015-03-02 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6120?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng resolved SPARK-6120. -- Resolution: Fixed Fix Version/s: 1.3.0 Issue resolved by pull request 4864 [https://githu

[jira] [Updated] (SPARK-6131) Spark 1.3.0 (RC1) missing some source files in sql.api.java

2015-03-02 Thread Littlestar (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6131?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Littlestar updated SPARK-6131: -- Priority: Minor (was: Critical) > Spark 1.3.0 (RC1) missing some source files in sql.api.java > ---

[jira] [Resolved] (SPARK-6097) Support model save/load in Python's tree models

2015-03-02 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6097?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng resolved SPARK-6097. -- Resolution: Fixed Fix Version/s: 1.3.0 Issue resolved by pull request 4854 [https://githu

[jira] [Created] (SPARK-6133) SparkContext#stop is not idempotent

2015-03-02 Thread Andrew Or (JIRA)
Andrew Or created SPARK-6133: Summary: SparkContext#stop is not idempotent Key: SPARK-6133 URL: https://issues.apache.org/jira/browse/SPARK-6133 Project: Spark Issue Type: Bug Component

[jira] [Created] (SPARK-6132) Context cleaner thread lives across SparkContexts

2015-03-02 Thread Andrew Or (JIRA)
Andrew Or created SPARK-6132: Summary: Context cleaner thread lives across SparkContexts Key: SPARK-6132 URL: https://issues.apache.org/jira/browse/SPARK-6132 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-6131) Spark 1.3.0 (RC1) missing some source files in sql.api.java

2015-03-02 Thread Littlestar (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6131?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14344604#comment-14344604 ] Littlestar commented on SPARK-6131: --- spark 1.2.1: sql\core\src\main\scala\org\apache\sp

[jira] [Commented] (SPARK-6020) Flaky test: o.a.s.sql.columnar.PartitionBatchPruningSuite

2015-03-02 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6020?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14344602#comment-14344602 ] Cheng Lian commented on SPARK-6020: --- Hey [~andrewor14], I think [PR #4835|https://github

[jira] [Updated] (SPARK-6116) Making DataFrame API non-experimental

2015-03-02 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6116?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-6116: --- Issue Type: Umbrella (was: Task) > Making DataFrame API non-experimental > --

[jira] [Commented] (SPARK-6131) Spark 1.3.0 (RC1) missing some source files in sql.api.java

2015-03-02 Thread Littlestar (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6131?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14344584#comment-14344584 ] Littlestar commented on SPARK-6131: --- spark-1.2.1 is ok. spark-1.2.1-src.jar\sql\core\src

[jira] [Updated] (SPARK-6131) Spark 1.3.0 (RC1) missing some source files in sql.api.java

2015-03-02 Thread Littlestar (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6131?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Littlestar updated SPARK-6131: -- Description: I notice that [VOTE] Release Apache Spark 1.3.0 (RC1) http://apache-spark-developers-list.

[jira] [Updated] (SPARK-6131) Spark 1.3.0 (RC1) missing some source files in sql.api.java

2015-03-02 Thread Littlestar (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6131?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Littlestar updated SPARK-6131: -- Description: I notice that [VOTE] Release Apache Spark 1.3.0 (RC1) http://apache-spark-developers-list.

[jira] [Updated] (SPARK-6131) Spark 1.3.0 (RC1) missing some source files in sql.api.java

2015-03-02 Thread Littlestar (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6131?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Littlestar updated SPARK-6131: -- Description: I notice that [VOTE] Release Apache Spark 1.3.0 (RC1) http://apache-spark-developers-list.

[jira] [Created] (SPARK-6131) Spark 1.3.0 (RC1) missing some source files in sql.api.java

2015-03-02 Thread Littlestar (JIRA)
Littlestar created SPARK-6131: - Summary: Spark 1.3.0 (RC1) missing some source files in sql.api.java Key: SPARK-6131 URL: https://issues.apache.org/jira/browse/SPARK-6131 Project: Spark Issue Ty

[jira] [Updated] (SPARK-6117) describe function for summary statistics

2015-03-02 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6117?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-6117: --- Labels: starter (was: ) > describe function for summary statistics >

[jira] [Comment Edited] (SPARK-5791) [Spark SQL] show poor performance when multiple table do join operation

2015-03-02 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5791?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14319364#comment-14319364 ] Reynold Xin edited comment on SPARK-5791 at 3/3/15 5:05 AM: {c

[jira] [Commented] (SPARK-2545) Add a diagnosis mode for closures to figure out what they're bringing in

2015-03-02 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2545?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14344504#comment-14344504 ] Nicholas Chammas commented on SPARK-2545: - cc [~tobias.schlatter] > Add a diagnos

[jira] [Commented] (SPARK-2545) Add a diagnosis mode for closures to figure out what they're bringing in

2015-03-02 Thread Aaron Davidson (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2545?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14344502#comment-14344502 ] Aaron Davidson commented on SPARK-2545: --- Probably especially useful there, yes. > A

[jira] [Commented] (SPARK-5791) [Spark SQL] show poor performance when multiple table do join operation

2015-03-02 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5791?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14344486#comment-14344486 ] Yin Huai commented on SPARK-5791: - [~jameszhouyi] Can you also add the plan generated by H

[jira] [Commented] (SPARK-2545) Add a diagnosis mode for closures to figure out what they're bringing in

2015-03-02 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2545?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14344482#comment-14344482 ] Nicholas Chammas commented on SPARK-2545: - [~adav] - Would this potentially also b

[jira] [Commented] (SPARK-2095) sc.getExecutorCPUCounts()

2015-03-02 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2095?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14344480#comment-14344480 ] Nicholas Chammas commented on SPARK-2095: - cc [~pwendell], [~joshrosen] This seem

[jira] [Commented] (SPARK-882) Have link for feedback/suggestions in docs

2015-03-02 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-882?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14344475#comment-14344475 ] Nicholas Chammas commented on SPARK-882: Is the intended use here that users could

[jira] [Commented] (SPARK-5310) Update SQL programming guide for 1.3

2015-03-02 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5310?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14344470#comment-14344470 ] Apache Spark commented on SPARK-5310: - User 'marmbrus' has created a pull request for

[jira] [Commented] (SPARK-6122) Upgrade Tachyon dependency to 0.6.0

2015-03-02 Thread Calvin Jia (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6122?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14344462#comment-14344462 ] Calvin Jia commented on SPARK-6122: --- New dependencies in Tachyon 0.6.0 include commons-

[jira] [Commented] (SPARK-6122) Upgrade Tachyon dependency to 0.6.0

2015-03-02 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6122?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14344460#comment-14344460 ] Apache Spark commented on SPARK-6122: - User 'calvinjia' has created a pull request for

[jira] [Resolved] (SPARK-5950) Insert array into a metastore table saved as parquet should work when using datasource api

2015-03-02 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5950?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-5950. - Resolution: Fixed Fix Version/s: 1.3.0 Issue resolved by pull request 4826 [https:/

[jira] [Updated] (SPARK-6125) Custom Loss function

2015-03-02 Thread Joanne Shin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6125?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joanne Shin updated SPARK-6125: --- Description: Currently, there are only a small selection of loss functions available for the SGD solve

[jira] [Commented] (SPARK-5537) Expand user guide for multinomial logistic regression

2015-03-02 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5537?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14344429#comment-14344429 ] Apache Spark commented on SPARK-5537: - User 'dbtsai' has created a pull request for th

[jira] [Commented] (SPARK-6130) support if not exists for insert overwrite into partition in hiveQl

2015-03-02 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14344419#comment-14344419 ] Apache Spark commented on SPARK-6130: - User 'adrian-wang' has created a pull request f

[jira] [Created] (SPARK-6130) support if not exists for insert overwrite into partition in hiveQl

2015-03-02 Thread Adrian Wang (JIRA)
Adrian Wang created SPARK-6130: -- Summary: support if not exists for insert overwrite into partition in hiveQl Key: SPARK-6130 URL: https://issues.apache.org/jira/browse/SPARK-6130 Project: Spark

[jira] [Updated] (SPARK-6122) Upgrade Tachyon dependency to 0.6.0

2015-03-02 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6122?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-6122: --- Assignee: Calvin Jia > Upgrade Tachyon dependency to 0.6.0 > -

[jira] [Updated] (SPARK-6122) Upgrade Tachyon dependency to 0.6.0

2015-03-02 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6122?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-6122: --- Fix Version/s: (was: 1.3.0) > Upgrade Tachyon dependency to 0.6.0 > --

[jira] [Comment Edited] (SPARK-3203) ClassNotFoundException in spark-shell with Cassandra

2015-03-02 Thread Hung Duong (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3203?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14344396#comment-14344396 ] Hung Duong edited comment on SPARK-3203 at 3/3/15 2:58 AM: --- I ra

[jira] [Updated] (SPARK-6122) Upgrade Tachyon dependency to 0.6.0

2015-03-02 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6122?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-6122: --- Assignee: Patrick Wendell > Upgrade Tachyon dependency to 0.6.0 >

[jira] [Updated] (SPARK-6122) Upgrade Tachyon dependency to 0.6.0

2015-03-02 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6122?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-6122: --- Target Version/s: 1.4.0 > Upgrade Tachyon dependency to 0.6.0 > --

[jira] [Updated] (SPARK-6122) Upgrade Tachyon dependency to 0.6.0

2015-03-02 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6122?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-6122: --- Assignee: (was: Patrick Wendell) > Upgrade Tachyon dependency to 0.6.0 > -

[jira] [Commented] (SPARK-3203) ClassNotFoundException in spark-shell with Cassandra

2015-03-02 Thread Hung Duong (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3203?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14344396#comment-14344396 ] Hung Duong commented on SPARK-3203: --- I ran into the same exception in cluster mode (it w

[jira] [Resolved] (SPARK-6127) Kafka API not visible in Python API docs

2015-03-02 Thread Tathagata Das (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6127?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tathagata Das resolved SPARK-6127. -- Resolution: Fixed Fix Version/s: 1.3.0 > Kafka API not visible in Python API docs > -

[jira] [Resolved] (SPARK-5537) Expand user guide for multinomial logistic regression

2015-03-02 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5537?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng resolved SPARK-5537. -- Resolution: Fixed Fix Version/s: 1.3.0 Issue resolved by pull request 4861 [https://githu

[jira] [Commented] (SPARK-6120) DecisionTree.save uses too much Java heap space for default spark shell settings

2015-03-02 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14344323#comment-14344323 ] Apache Spark commented on SPARK-6120: - User 'jkbradley' has created a pull request for

[jira] [Commented] (SPARK-1391) BlockManager cannot transfer blocks larger than 2G in size

2015-03-02 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1391?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14344320#comment-14344320 ] Reynold Xin commented on SPARK-1391: - I absolutely agree that better error messages i

[jira] [Updated] (SPARK-6129) Add a section in user guide for model evaluation

2015-03-02 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6129?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-6129: - Summary: Add a section in user guide for model evaluation (was: Add section in user guide for mod

[jira] [Commented] (SPARK-6090) Add BinaryClassificationMetrics in PySpark/MLlib

2015-03-02 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6090?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14344317#comment-14344317 ] Apache Spark commented on SPARK-6090: - User 'mengxr' has created a pull request for th

[jira] [Commented] (SPARK-3530) Pipeline and Parameters

2015-03-02 Thread Evan Sparks (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3530?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14344291#comment-14344291 ] Evan Sparks commented on SPARK-3530: We have looked at integrating Caffe with spark -

[jira] [Commented] (SPARK-1391) BlockManager cannot transfer blocks larger than 2G in size

2015-03-02 Thread Imran Rashid (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1391?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14344286#comment-14344286 ] Imran Rashid commented on SPARK-1391: - [~rxin] Sure thing, I can break it into multipl

[jira] [Commented] (SPARK-6123) Parquet reader should use the schema of every file to create converter

2015-03-02 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6123?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14344278#comment-14344278 ] Yin Huai commented on SPARK-6123: - To workaround this issue, users need to load the existi

[jira] [Commented] (SPARK-5310) Update SQL programming guide for 1.3

2015-03-02 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5310?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14344259#comment-14344259 ] Apache Spark commented on SPARK-5310: - User 'rxin' has created a pull request for this

[jira] [Updated] (SPARK-6090) Add BinaryClassificationMetrics in PySpark/MLlib

2015-03-02 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6090?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-6090: - Target Version/s: 1.4.0 > Add BinaryClassificationMetrics in PySpark/MLlib > -

[jira] [Updated] (SPARK-6090) Add BinaryClassificationMetrics in PySpark/MLlib

2015-03-02 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6090?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-6090: - Assignee: Xiangrui Meng > Add BinaryClassificationMetrics in PySpark/MLlib > -

[jira] [Commented] (SPARK-6120) DecisionTree.save uses too much Java heap space for default spark shell settings

2015-03-02 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14344252#comment-14344252 ] Joseph K. Bradley commented on SPARK-6120: -- Rather than one of the difficult opti

[jira] [Assigned] (SPARK-6120) DecisionTree.save uses too much Java heap space for default spark shell settings

2015-03-02 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6120?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley reassigned SPARK-6120: Assignee: Joseph K. Bradley > DecisionTree.save uses too much Java heap space for d

[jira] [Updated] (SPARK-6120) DecisionTree.save uses too much Java heap space for default spark shell settings

2015-03-02 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6120?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-6120: - Target Version/s: 1.3.0 (was: 1.4.0) > DecisionTree.save uses too much Java heap space fo

[jira] [Commented] (SPARK-6120) DecisionTree.save uses too much Java heap space for default spark shell settings

2015-03-02 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14344243#comment-14344243 ] Joseph K. Bradley commented on SPARK-6120: -- This also affects tree ensembles, of

[jira] [Commented] (SPARK-6120) DecisionTree.save uses too much Java heap space for default spark shell settings

2015-03-02 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14344234#comment-14344234 ] Joseph K. Bradley commented on SPARK-6120: -- I checked, and this only happens in s

[jira] [Created] (SPARK-6129) Add section in user guide for model evaluation

2015-03-02 Thread Xiangrui Meng (JIRA)
Xiangrui Meng created SPARK-6129: Summary: Add section in user guide for model evaluation Key: SPARK-6129 URL: https://issues.apache.org/jira/browse/SPARK-6129 Project: Spark Issue Type: New

[jira] [Resolved] (SPARK-6121) Python DataFrame type inference for LabeledPoint gets wrong type

2015-03-02 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6121?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng resolved SPARK-6121. -- Resolution: Fixed Fix Version/s: 1.3.0 Issue resolved by pull request 4858 [https://githu

[jira] [Created] (SPARK-6128) Update Spark Streaming Guide for Spark 1.3

2015-03-02 Thread Tathagata Das (JIRA)
Tathagata Das created SPARK-6128: Summary: Update Spark Streaming Guide for Spark 1.3 Key: SPARK-6128 URL: https://issues.apache.org/jira/browse/SPARK-6128 Project: Spark Issue Type: Improvem

[jira] [Commented] (SPARK-5537) Expand user guide for multinomial logistic regression

2015-03-02 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5537?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14344221#comment-14344221 ] Apache Spark commented on SPARK-5537: - User 'mengxr' has created a pull request for th

[jira] [Commented] (SPARK-6127) Kafka API not visible in Python API docs

2015-03-02 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6127?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14344210#comment-14344210 ] Apache Spark commented on SPARK-6127: - User 'tdas' has created a pull request for this

[jira] [Updated] (SPARK-6127) Kafka API not visible in Python API docs

2015-03-02 Thread Tathagata Das (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6127?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tathagata Das updated SPARK-6127: - Summary: Kafka API not visible in Python API docs (was: Kafka API not visible in Python docs) >

[jira] [Created] (SPARK-6127) Kafka API not visible in Python docs

2015-03-02 Thread Tathagata Das (JIRA)
Tathagata Das created SPARK-6127: Summary: Kafka API not visible in Python docs Key: SPARK-6127 URL: https://issues.apache.org/jira/browse/SPARK-6127 Project: Spark Issue Type: Bug

[jira] [Comment Edited] (SPARK-3071) Increase default driver memory

2015-03-02 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3071?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14344186#comment-14344186 ] Joseph K. Bradley edited comment on SPARK-3071 at 3/3/15 1:00 AM: --

[jira] [Commented] (SPARK-3071) Increase default driver memory

2015-03-02 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3071?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14344186#comment-14344186 ] Joseph K. Bradley commented on SPARK-3071: -- +1000 for increasing default driver m

[jira] [Closed] (SPARK-4777) Some block memory after unrollSafely not count into used memory(memoryStore.entrys or unrollMemory)

2015-03-02 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4777?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or closed SPARK-4777. Resolution: Fixed Fix Version/s: 1.4.0 Assignee: SuYan Target Version/s: 1.4.0

[jira] [Closed] (SPARK-5977) PySpark SPARK_CLASSPATH doesn't distribute jars to executors

2015-03-02 Thread Michael Nazario (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5977?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Nazario closed SPARK-5977. -- Resolution: Not a Problem This was my misunderstanding of how setting in spark are supposed to b

[jira] [Updated] (SPARK-5522) Accelerate the History Server start

2015-03-02 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5522?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-5522: - Fix Version/s: 1.4.0 > Accelerate the History Server start > --- > >

[jira] [Updated] (SPARK-5522) Accelerate the History Server start

2015-03-02 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5522?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-5522: - Assignee: Liangliang Gu > Accelerate the History Server start > --- > >

[jira] [Updated] (SPARK-5522) Accelerate the History Server start

2015-03-02 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5522?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-5522: - Affects Version/s: 1.0.0 > Accelerate the History Server start > --- > >

[jira] [Commented] (SPARK-6126) Support UDTs in JSON

2015-03-02 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6126?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14344170#comment-14344170 ] Joseph K. Bradley commented on SPARK-6126: -- There is not an immediately need AFAI

[jira] [Updated] (SPARK-6126) Support UDTs in JSON

2015-03-02 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6126?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-6126: - Description: This would be nice to do (in Python): {code} from pyspark.mllib.util import M

[jira] [Updated] (SPARK-6126) Support UDTs in JSON

2015-03-02 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6126?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-6126: - Summary: Support UDTs in JSON (was: RDD[LabeledPoint].toDF().toJSON() fails) > Support U

[jira] [Updated] (SPARK-6126) Support UDTs in JSON

2015-03-02 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6126?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-6126: - Issue Type: Improvement (was: Bug) > Support UDTs in JSON > > >

[jira] [Updated] (SPARK-6126) RDD[LabeledPoint].toDF().toJSON() fails

2015-03-02 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6126?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-6126: - Priority: Major (was: Blocker) > RDD[LabeledPoint].toDF().toJSON() fails > --

[jira] [Updated] (SPARK-6126) RDD[LabeledPoint].toDF().toJSON() fails

2015-03-02 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6126?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-6126: - Target Version/s: 1.4.0 (was: 1.3.0) > RDD[LabeledPoint].toDF().toJSON() fails >

[jira] [Updated] (SPARK-6112) Leverage HDFS RAM_DISK capacity to provide off_heap feature similar to Tachyon

2015-03-02 Thread Zhan Zhang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6112?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhan Zhang updated SPARK-6112: -- Component/s: (was: Spark Core) Block Manager > Leverage HDFS RAM_DISK capacity to p

[jira] [Resolved] (SPARK-6048) SparkConf.translateConfKey should not translate on set

2015-03-02 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6048?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell resolved SPARK-6048. Resolution: Fixed Fix Version/s: 1.3.0 > SparkConf.translateConfKey should not transl

[jira] [Resolved] (SPARK-6066) Metadata in event log makes it very difficult for external libraries to parse event log

2015-03-02 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6066?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell resolved SPARK-6066. Resolution: Fixed Fix Version/s: 1.3.0 Thanks Andrew and Marcelo for your work on thi

[jira] [Created] (SPARK-6126) RDD[LabeledPoint].toDF().toJSON() fails

2015-03-02 Thread Joseph K. Bradley (JIRA)
Joseph K. Bradley created SPARK-6126: Summary: RDD[LabeledPoint].toDF().toJSON() fails Key: SPARK-6126 URL: https://issues.apache.org/jira/browse/SPARK-6126 Project: Spark Issue Type: Bug

[jira] [Created] (SPARK-6125) Custom Loss function

2015-03-02 Thread Joanne Shin (JIRA)
Joanne Shin created SPARK-6125: -- Summary: Custom Loss function Key: SPARK-6125 URL: https://issues.apache.org/jira/browse/SPARK-6125 Project: Spark Issue Type: Question Components: MLl

[jira] [Resolved] (SPARK-6082) SparkSQL should fail gracefully when input data format doesn't match expectations

2015-03-02 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6082?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-6082. - Resolution: Fixed Fix Version/s: 1.3.0 Issue resolved by pull request 4842 [https:/

[jira] [Resolved] (SPARK-3915) backport 'spark.localExecution.enabled' to 1.0

2015-03-02 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3915?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-3915. -- Resolution: Not a Problem Resolving per comment from Patrick > backport 'spark.localExecution.enabled'

[jira] [Resolved] (SPARK-6114) Explode on nested field fails

2015-03-02 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6114?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-6114. - Resolution: Fixed Fix Version/s: 1.3.0 Issue resolved by pull request 4855 [https:/

[jira] [Commented] (SPARK-3684) Can't configure local dirs in Yarn mode

2015-03-02 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3684?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14344088#comment-14344088 ] Sean Owen commented on SPARK-3684: -- Wouldn't this amount to just configuring YARN to use

[jira] [Resolved] (SPARK-3551) Remove redundant putting FetchResult which means Fetch Fail when Remote fetching

2015-03-02 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3551?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-3551. -- Resolution: Not a Problem It looks like this code was subsequently changed anyway, and no longer returns

[jira] [Commented] (SPARK-6124) Support jdbc connection properties in OPTIONS part of the query

2015-03-02 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14344071#comment-14344071 ] Apache Spark commented on SPARK-6124: - User 'vlyubin' has created a pull request for t

[jira] [Created] (SPARK-6124) Support jdbc connection properties in OPTIONS part of the query

2015-03-02 Thread Volodymyr Lyubinets (JIRA)
Volodymyr Lyubinets created SPARK-6124: -- Summary: Support jdbc connection properties in OPTIONS part of the query Key: SPARK-6124 URL: https://issues.apache.org/jira/browse/SPARK-6124 Project: Sp

[jira] [Commented] (SPARK-2769) Ganglia Support Broken / Not working

2015-03-02 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2769?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14344039#comment-14344039 ] Sean Owen commented on SPARK-2769: -- [~DarkSlice] do you believe this is still an issue? t

[jira] [Commented] (SPARK-1548) Add Partial Random Forest algorithm to MLlib

2015-03-02 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1548?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14344037#comment-14344037 ] Joseph K. Bradley commented on SPARK-1548: -- I think the idea is to train a differ

[jira] [Created] (SPARK-6123) Parquet reader should use the schema of every file to create converter

2015-03-02 Thread Yin Huai (JIRA)
Yin Huai created SPARK-6123: --- Summary: Parquet reader should use the schema of every file to create converter Key: SPARK-6123 URL: https://issues.apache.org/jira/browse/SPARK-6123 Project: Spark I

[jira] [Commented] (SPARK-1548) Add Partial Random Forest algorithm to MLlib

2015-03-02 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1548?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14343980#comment-14343980 ] Sean Owen commented on SPARK-1548: -- Same question, curious about the status as there hasn

[jira] [Commented] (SPARK-3621) Provide a way to broadcast an RDD (instead of just a variable made of the RDD) so that a job can access

2015-03-02 Thread Xuefu Zhang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14343977#comment-14343977 ] Xuefu Zhang commented on SPARK-3621: {quote} you can go a step further if you wanted

[jira] [Commented] (SPARK-1546) Add AdaBoost algorithm to Spark MLlib

2015-03-02 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1546?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14343974#comment-14343974 ] Sean Owen commented on SPARK-1546: -- I'm curious about the status of this and other subtas

[jira] [Commented] (SPARK-6121) Python DataFrame type inference for LabeledPoint gets wrong type

2015-03-02 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6121?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14343944#comment-14343944 ] Apache Spark commented on SPARK-6121: - User 'mengxr' has created a pull request for th

  1   2   3   >