[jira] [Commented] (SPARK-4243) Spark SQL SELECT COUNT DISTINCT optimization

2015-11-10 Thread Piotr Niemcunowicz (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4243?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14998799#comment-14998799 ] Piotr Niemcunowicz commented on SPARK-4243: --- Same happens when one uses HiveContext. > Spark

[jira] [Commented] (SPARK-4243) Spark SQL SELECT COUNT DISTINCT optimization

2015-11-10 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4243?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14998813#comment-14998813 ] Yin Huai commented on SPARK-4243: - The optimization of {{SELECT COUNT(DISTINCT f2) FROM parquetFile}} will

[jira] [Updated] (SPARK-4243) Spark SQL SELECT COUNT DISTINCT optimization

2015-11-10 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4243?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai updated SPARK-4243: Target Version/s: 1.6.0 > Spark SQL SELECT COUNT DISTINCT optimization >

[jira] [Updated] (SPARK-10388) Public dataset loader interface

2015-11-10 Thread Jeff Zhang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10388?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jeff Zhang updated SPARK-10388: --- Attachment: SPARK-10388PublicDataSetLoaderInterface.pdf > Public dataset loader interface >

[jira] [Commented] (SPARK-10388) Public dataset loader interface

2015-11-10 Thread Jeff Zhang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10388?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14998759#comment-14998759 ] Jeff Zhang commented on SPARK-10388: [~mengxr] I talked with [~rams] offline, and would love to

[jira] [Updated] (SPARK-4243) Spark SQL SELECT COUNT DISTINCT optimization

2015-11-10 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4243?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai updated SPARK-4243: Issue Type: Sub-task (was: Improvement) Parent: SPARK-4366 > Spark SQL SELECT COUNT DISTINCT

[jira] [Commented] (SPARK-11343) Regression Imposes doubles on prediction/label columns

2015-11-10 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11343?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14998732#comment-14998732 ] Apache Spark commented on SPARK-11343: -- User 'dahlem' has created a pull request for this issue:

[jira] [Commented] (SPARK-10978) Allow PrunedFilterScan to eliminate predicates from further evaluation

2015-11-10 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10978?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14998883#comment-14998883 ] Yin Huai commented on SPARK-10978: -- I am not sure {{partiallyHandledFilters}} is really useful. I mean a

[jira] [Commented] (SPARK-11578) User facing api for typed aggregation

2015-11-10 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11578?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14998754#comment-14998754 ] Apache Spark commented on SPARK-11578: -- User 'cloud-fan' has created a pull request for this issue:

[jira] [Resolved] (SPARK-4243) Spark SQL SELECT COUNT DISTINCT optimization

2015-11-10 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4243?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai resolved SPARK-4243. - Resolution: Fixed Assignee: Yin Huai Fix Version/s: 1.6.0 > Spark SQL SELECT COUNT

[jira] [Resolved] (SPARK-11590) use native json_tuple in lateral view

2015-11-10 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11590?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai resolved SPARK-11590. -- Resolution: Fixed Fix Version/s: 1.6.0 Issue resolved by pull request 9562

[jira] [Commented] (SPARK-4243) Spark SQL SELECT COUNT DISTINCT optimization

2015-11-10 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4243?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14999138#comment-14999138 ] Yin Huai commented on SPARK-4243: - https://github.com/apache/spark/pull/9556 has been merged. I am

[jira] [Resolved] (SPARK-10863) Method coltypes() to return the R column types of a DataFrame

2015-11-10 Thread Shivaram Venkataraman (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10863?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shivaram Venkataraman resolved SPARK-10863. --- Resolution: Fixed Fix Version/s: 1.6.0 Issue resolved by pull request

[jira] [Updated] (SPARK-10863) Method coltypes() to return the R column types of a DataFrame

2015-11-10 Thread Shivaram Venkataraman (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10863?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shivaram Venkataraman updated SPARK-10863: -- Assignee: Oscar D. Lara Yejas > Method coltypes() to return the R column types

[jira] [Updated] (SPARK-10892) Join with Data Frame returns wrong results

2015-11-10 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10892?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai updated SPARK-10892: - Fix Version/s: (was: 1.6.0) > Join with Data Frame returns wrong results >

[jira] [Commented] (SPARK-10892) Join with Data Frame returns wrong results

2015-11-10 Thread Ofer Mendelevitch (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10892?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14999171#comment-14999171 ] Ofer Mendelevitch commented on SPARK-10892: --- Sure, sorry. Any idea when this might be

[jira] [Updated] (SPARK-11626) ml.feature.Word2Vec.transform() should not recompute word-vector map each time

2015-11-10 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11626?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-11626: -- Issue Type: Bug (was: Improvement) > ml.feature.Word2Vec.transform() should not recompute

[jira] [Commented] (SPARK-11631) DAGScheduler prints "Stopping DAGScheduler" at INFO to the logs with no corresponding "Starting"

2015-11-10 Thread Xiu(Joe) Guo (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11631?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14999101#comment-14999101 ] Xiu(Joe) Guo commented on SPARK-11631: -- I am looking at it, will submit a PR shortly. >

[jira] [Updated] (SPARK-10444) Remove duplication in Mesos schedulers

2015-11-10 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10444?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-10444: -- Target Version/s: 1.7.0 (was: 1.6.0) > Remove duplication in Mesos schedulers >

[jira] [Commented] (SPARK-11583) Make MapStatus use less memory uage

2015-11-10 Thread Imran Rashid (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14999009#comment-14999009 ] Imran Rashid commented on SPARK-11583: -- I think [~srowen] is right that we are going in circles a

[jira] [Updated] (SPARK-11622) Make LibSVMRelation extends HadoopFsRelation and Add LibSVMOutputWriter

2015-11-10 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11622?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-11622: -- Shepherd: Kai Sasaki > Make LibSVMRelation extends HadoopFsRelation and Add LibSVMOutputWriter

[jira] [Updated] (SPARK-11622) Make LibSVMRelation extends HadoopFsRelation and Add LibSVMOutputWriter

2015-11-10 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11622?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-11622: -- Assignee: Jeff Zhang > Make LibSVMRelation extends HadoopFsRelation and Add LibSVMOutputWriter

[jira] [Updated] (SPARK-11626) ml.feature.Word2Vec.transform() should not recompute word-vector map each time

2015-11-10 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11626?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-11626: -- Assignee: q79969786 > ml.feature.Word2Vec.transform() should not recompute word-vector map

[jira] [Updated] (SPARK-11626) ml.feature.Word2Vec.transform() should not recompute word-vector map each time

2015-11-10 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11626?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-11626: -- Shepherd: Xiangrui Meng Target Version/s: 1.6.0 Priority: Major (was:

[jira] [Updated] (SPARK-10892) Join with Data Frame returns wrong results

2015-11-10 Thread Ofer Mendelevitch (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10892?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ofer Mendelevitch updated SPARK-10892: -- Fix Version/s: 1.6.0 > Join with Data Frame returns wrong results >

[jira] [Resolved] (SPARK-10371) Optimize sequential projections

2015-11-10 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10371?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-10371. -- Resolution: Fixed Fix Version/s: 1.6.0 Issue resolved by pull request 9480

[jira] [Commented] (SPARK-10892) Join with Data Frame returns wrong results

2015-11-10 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10892?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14999178#comment-14999178 ] Yin Huai commented on SPARK-10892: -- We are still working on it. It is a pretty hard one. For not, can

[jira] [Updated] (SPARK-11382) Replace example code in mllib-decision-tree.md using include_example

2015-11-10 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11382?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-11382: -- Shepherd: Xiangrui Meng (was: Xusen Yin) Assignee: Xusen Yin > Replace example code in

[jira] [Commented] (SPARK-10892) Join with Data Frame returns wrong results

2015-11-10 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10892?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14999156#comment-14999156 ] Yin Huai commented on SPARK-10892: -- [~ofermend] (we only assign fix version after a jira is resolved.)

[jira] [Created] (SPARK-11634) Make simple transformers and estimators implement default read/write

2015-11-10 Thread Xiangrui Meng (JIRA)
Xiangrui Meng created SPARK-11634: - Summary: Make simple transformers and estimators implement default read/write Key: SPARK-11634 URL: https://issues.apache.org/jira/browse/SPARK-11634 Project:

[jira] [Comment Edited] (SPARK-10892) Join with Data Frame returns wrong results

2015-11-10 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10892?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14999178#comment-14999178 ] Yin Huai edited comment on SPARK-10892 at 11/10/15 7:32 PM: We are still

[jira] [Resolved] (SPARK-7841) Spark build should not use lib_managed for dependencies

2015-11-10 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7841?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-7841. - Resolution: Fixed Fix Version/s: 1.6.0 Issue resolved by pull request 9575

[jira] [Resolved] (SPARK-9830) Remove AggregateExpression1 and Aggregate Operator used to evaluate AggregateExpression1s

2015-11-10 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9830?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-9830. - Resolution: Fixed Fix Version/s: 1.6.0 Issue resolved by pull request 9556

[jira] [Commented] (SPARK-11583) Make MapStatus use less memory uage

2015-11-10 Thread Daniel Lemire (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14999130#comment-14999130 ] Daniel Lemire commented on SPARK-11583: --- > So, this is about compressing the in-memory

[jira] [Assigned] (SPARK-6990) Add Java linting script

2015-11-10 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6990?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-6990: --- Assignee: (was: Apache Spark) > Add Java linting script > --- > >

[jira] [Assigned] (SPARK-6990) Add Java linting script

2015-11-10 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6990?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-6990: --- Assignee: Apache Spark > Add Java linting script > --- > >

[jira] [Commented] (SPARK-6990) Add Java linting script

2015-11-10 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14998961#comment-14998961 ] Apache Spark commented on SPARK-6990: - User 'dskrvk' has created a pull request for this issue:

[jira] [Created] (SPARK-11633) HiveContext throws TreeNode Exception : Failed to Copy Node

2015-11-10 Thread Saurabh Santhosh (JIRA)
Saurabh Santhosh created SPARK-11633: Summary: HiveContext throws TreeNode Exception : Failed to Copy Node Key: SPARK-11633 URL: https://issues.apache.org/jira/browse/SPARK-11633 Project: Spark

[jira] [Resolved] (SPARK-11382) Replace example code in mllib-decision-tree.md using include_example

2015-11-10 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11382?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng resolved SPARK-11382. --- Resolution: Fixed Fix Version/s: 1.6.0 Issue resolved by pull request 9596

[jira] [Commented] (SPARK-11583) Make MapStatus use less memory uage

2015-11-10 Thread Daniel Lemire (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14999117#comment-14999117 ] Daniel Lemire commented on SPARK-11583: --- > When a set is nearly full, RoaringBitmap does not

[jira] [Commented] (SPARK-11583) Make MapStatus use less memory uage

2015-11-10 Thread Daniel Lemire (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14999142#comment-14999142 ] Daniel Lemire commented on SPARK-11583: --- [~Qin Yao] wrote: "Roaringbitmap is same as the BitSet

[jira] [Updated] (SPARK-11590) use native json_tuple in lateral view

2015-11-10 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11590?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai updated SPARK-11590: - Assignee: Wenchen Fan > use native json_tuple in lateral view > - >

[jira] [Comment Edited] (SPARK-6152) Spark does not support Java 8 compiled Scala classes

2015-11-10 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6152?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14999190#comment-14999190 ] Josh Rosen edited comment on SPARK-6152 at 11/10/15 7:44 PM: - Does anyone have

[jira] [Assigned] (SPARK-11636) Support as on Classes defined in the REPL

2015-11-10 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11636?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-11636: Assignee: Apache Spark (was: Michael Armbrust) > Support as on Classes defined in the

[jira] [Updated] (SPARK-11637) Alias do not work with udf with * parameter

2015-11-10 Thread Pierre Borckmans (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11637?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Borckmans updated SPARK-11637: - Description: In Spark < 1.5.0, this used to work : {code:java|title=Spark

[jira] [Assigned] (SPARK-11631) DAGScheduler prints "Stopping DAGScheduler" at INFO to the logs with no corresponding "Starting"

2015-11-10 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11631?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-11631: Assignee: (was: Apache Spark) > DAGScheduler prints "Stopping DAGScheduler" at INFO

[jira] [Updated] (SPARK-11638) Apache Spark on Docker with Bridge networking / run Spark in Mesos on Docker with Bridge networking

2015-11-10 Thread Radoslaw Gruchalski (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11638?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Radoslaw Gruchalski updated SPARK-11638: Attachment: 2.3.11.patch 2.3.4.patch

[jira] [Commented] (SPARK-6519) Add spark.ml API for bisecting k-means

2015-11-10 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6519?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14999358#comment-14999358 ] Apache Spark commented on SPARK-6519: - User 'yu-iskw' has created a pull request for this issue:

[jira] [Assigned] (SPARK-6519) Add spark.ml API for bisecting k-means

2015-11-10 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6519?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-6519: --- Assignee: Apache Spark > Add spark.ml API for bisecting k-means >

[jira] [Updated] (SPARK-11640) shading packages in spark-assembly jar

2015-11-10 Thread PJ Fanning (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11640?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] PJ Fanning updated SPARK-11640: --- Issue Type: Wish (was: Bug) > shading packages in spark-assembly jar >

[jira] [Created] (SPARK-11640) shading packages in spark-assembly jar

2015-11-10 Thread PJ Fanning (JIRA)
PJ Fanning created SPARK-11640: -- Summary: shading packages in spark-assembly jar Key: SPARK-11640 URL: https://issues.apache.org/jira/browse/SPARK-11640 Project: Spark Issue Type: Bug

[jira] [Reopened] (SPARK-11640) shading packages in spark-assembly jar

2015-11-10 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11640?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen reopened SPARK-11640: --- > shading packages in spark-assembly jar > -- > >

[jira] [Resolved] (SPARK-11640) shading packages in spark-assembly jar

2015-11-10 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11640?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-11640. --- Resolution: Fixed I'm going to take a pretty good guess that you want to build with the

[jira] [Commented] (SPARK-6152) Spark does not support Java 8 compiled Scala classes

2015-11-10 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6152?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14999190#comment-14999190 ] Josh Rosen commented on SPARK-6152: --- Does anyone have a standalone reproduction of this issue that I can

[jira] [Assigned] (SPARK-11635) Avoid Guava version dependency for Hashcodes

2015-11-10 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11635?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-11635: Assignee: Apache Spark > Avoid Guava version dependency for Hashcodes >

[jira] [Created] (SPARK-11637) Alias do not work with udf with * parameter

2015-11-10 Thread Pierre Borckmans (JIRA)
Pierre Borckmans created SPARK-11637: Summary: Alias do not work with udf with * parameter Key: SPARK-11637 URL: https://issues.apache.org/jira/browse/SPARK-11637 Project: Spark Issue

[jira] [Resolved] (SPARK-11618) Refactoring of basic ML import/export

2015-11-10 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11618?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng resolved SPARK-11618. --- Resolution: Fixed Fix Version/s: 1.6.0 Issue resolved by pull request 9587

[jira] [Commented] (SPARK-6152) Spark does not support Java 8 compiled Scala classes

2015-11-10 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6152?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14999222#comment-14999222 ] Josh Rosen commented on SPARK-6152: --- Yep, was able to reproduce trivially by running Spark's existing

[jira] [Updated] (SPARK-11637) Alias do not work with udf with * parameter

2015-11-10 Thread Pierre Borckmans (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11637?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Borckmans updated SPARK-11637: - Description: In Spark < 1.5.0, this used to work : {code:java|title=Spark

[jira] [Comment Edited] (SPARK-11583) Make MapStatus use less memory uage

2015-11-10 Thread Daniel Lemire (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14999309#comment-14999309 ] Daniel Lemire edited comment on SPARK-11583 at 11/10/15 8:40 PM: -

[jira] [Comment Edited] (SPARK-11583) Make MapStatus use less memory uage

2015-11-10 Thread Daniel Lemire (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14999142#comment-14999142 ] Daniel Lemire edited comment on SPARK-11583 at 11/10/15 8:42 PM: - [~Qin

[jira] [Created] (SPARK-11638) Apache Spark on Docker with Bridge networking / run Spark in Mesos on Docker with Bridge networking

2015-11-10 Thread Radoslaw Gruchalski (JIRA)
Radoslaw Gruchalski created SPARK-11638: --- Summary: Apache Spark on Docker with Bridge networking / run Spark in Mesos on Docker with Bridge networking Key: SPARK-11638 URL:

[jira] [Commented] (SPARK-11583) Make MapStatus use less memory uage

2015-11-10 Thread Imran Rashid (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14999185#comment-14999185 ] Imran Rashid commented on SPARK-11583: -- thanks [~lemire]. So I guess that means that option 2,

[jira] [Assigned] (SPARK-11636) Support as on Classes defined in the REPL

2015-11-10 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11636?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-11636: Assignee: Michael Armbrust (was: Apache Spark) > Support as on Classes defined in the

[jira] [Commented] (SPARK-11636) Support as on Classes defined in the REPL

2015-11-10 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11636?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14999219#comment-14999219 ] Apache Spark commented on SPARK-11636: -- User 'marmbrus' has created a pull request for this issue:

[jira] [Updated] (SPARK-11637) Alias do not work with udf with * parameter

2015-11-10 Thread Pierre Borckmans (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11637?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Borckmans updated SPARK-11637: - Description: In Spark < 1.5.0, this used to work : {code:java|title=Spark

[jira] [Commented] (SPARK-11631) DAGScheduler prints "Stopping DAGScheduler" at INFO to the logs with no corresponding "Starting"

2015-11-10 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11631?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14999305#comment-14999305 ] Apache Spark commented on SPARK-11631: -- User 'xguo27' has created a pull request for this issue:

[jira] [Assigned] (SPARK-11631) DAGScheduler prints "Stopping DAGScheduler" at INFO to the logs with no corresponding "Starting"

2015-11-10 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11631?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-11631: Assignee: Apache Spark > DAGScheduler prints "Stopping DAGScheduler" at INFO to the logs

[jira] [Closed] (SPARK-11635) Avoid Guava version dependency for Hashcodes

2015-11-10 Thread Nishkam Ravi (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11635?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nishkam Ravi closed SPARK-11635. > Avoid Guava version dependency for Hashcodes > > >

[jira] [Commented] (SPARK-11638) Apache Spark on Docker with Bridge networking / run Spark in Mesos on Docker with Bridge networking

2015-11-10 Thread Radoslaw Gruchalski (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11638?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14999363#comment-14999363 ] Radoslaw Gruchalski commented on SPARK-11638: - Working on providing full description. I will

[jira] [Created] (SPARK-11635) Avoid Guava version dependency for Hashcodes

2015-11-10 Thread Nishkam Ravi (JIRA)
Nishkam Ravi created SPARK-11635: Summary: Avoid Guava version dependency for Hashcodes Key: SPARK-11635 URL: https://issues.apache.org/jira/browse/SPARK-11635 Project: Spark Issue Type:

[jira] [Updated] (SPARK-11077) Join elimination in Catalyst

2015-11-10 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11077?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-11077: - Target Version/s: (was: 1.6.0) > Join elimination in Catalyst >

[jira] [Updated] (SPARK-11633) HiveContext throws TreeNode Exception : Failed to Copy Node

2015-11-10 Thread Saurabh Santhosh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11633?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saurabh Santhosh updated SPARK-11633: - Affects Version/s: 1.5.0 > HiveContext throws TreeNode Exception : Failed to Copy Node >

[jira] [Resolved] (SPARK-11635) Avoid Guava version dependency for Hashcodes

2015-11-10 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11635?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-11635. Resolution: Invalid Believe me, I'd have used the non-deprecated version if it were

[jira] [Updated] (SPARK-11633) HiveContext throws TreeNode Exception : Failed to Copy Node

2015-11-10 Thread Saurabh Santhosh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11633?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saurabh Santhosh updated SPARK-11633: - Affects Version/s: 1.5.1 > HiveContext throws TreeNode Exception : Failed to Copy Node >

[jira] [Commented] (SPARK-11635) Avoid Guava version dependency for Hashcodes

2015-11-10 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11635?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14999319#comment-14999319 ] Marcelo Vanzin commented on SPARK-11635: bq. why do we use Guava 14 instead of a newer version?

[jira] [Assigned] (SPARK-11635) Avoid Guava version dependency for Hashcodes

2015-11-10 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11635?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-11635: Assignee: (was: Apache Spark) > Avoid Guava version dependency for Hashcodes >

[jira] [Commented] (SPARK-11635) Avoid Guava version dependency for Hashcodes

2015-11-10 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11635?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14999213#comment-14999213 ] Apache Spark commented on SPARK-11635: -- User 'nishkamravi2' has created a pull request for this

[jira] [Updated] (SPARK-11637) Alias do not work with udf with * parameter

2015-11-10 Thread Pierre Borckmans (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11637?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Borckmans updated SPARK-11637: - Description: In Spark < 1.5.0, this used to work : {code:scala|title=Spark

[jira] [Commented] (SPARK-11635) Avoid Guava version dependency for Hashcodes

2015-11-10 Thread Nishkam Ravi (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11635?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14999308#comment-14999308 ] Nishkam Ravi commented on SPARK-11635: -- I see. So we end up with a dependency either way. HashCode

[jira] [Commented] (SPARK-11583) Make MapStatus use less memory uage

2015-11-10 Thread Daniel Lemire (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14999309#comment-14999309 ] Daniel Lemire commented on SPARK-11583: --- [~irashid] What I would suggest is a quantified

[jira] [Assigned] (SPARK-11639) Flaky test: BatchedWriteAheadLog - name log with aggregated entries with the timestamp of last entry

2015-11-10 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11639?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-11639: Assignee: (was: Apache Spark) > Flaky test: BatchedWriteAheadLog - name log with

[jira] [Created] (SPARK-11639) Flaky test: BatchedWriteAheadLog - name log with aggregated entries with the timestamp of last entry

2015-11-10 Thread Burak Yavuz (JIRA)
Burak Yavuz created SPARK-11639: --- Summary: Flaky test: BatchedWriteAheadLog - name log with aggregated entries with the timestamp of last entry Key: SPARK-11639 URL:

[jira] [Assigned] (SPARK-11639) Flaky test: BatchedWriteAheadLog - name log with aggregated entries with the timestamp of last entry

2015-11-10 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11639?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-11639: Assignee: Apache Spark > Flaky test: BatchedWriteAheadLog - name log with aggregated

[jira] [Commented] (SPARK-11639) Flaky test: BatchedWriteAheadLog - name log with aggregated entries with the timestamp of last entry

2015-11-10 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11639?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14999376#comment-14999376 ] Apache Spark commented on SPARK-11639: -- User 'brkyvz' has created a pull request for this issue:

[jira] [Resolved] (SPARK-11640) shading packages in spark-assembly jar

2015-11-10 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11640?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-11640. --- Resolution: Not A Problem > shading packages in spark-assembly jar >

[jira] [Created] (SPARK-11636) Support as on Classes defined in the REPL

2015-11-10 Thread Michael Armbrust (JIRA)
Michael Armbrust created SPARK-11636: Summary: Support as on Classes defined in the REPL Key: SPARK-11636 URL: https://issues.apache.org/jira/browse/SPARK-11636 Project: Spark Issue

[jira] [Updated] (SPARK-11633) HiveContext throws TreeNode Exception : Failed to Copy Node

2015-11-10 Thread Saurabh Santhosh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11633?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saurabh Santhosh updated SPARK-11633: - Description: h2. HiveContext#sql is throwing the following exception in a specific

[jira] [Comment Edited] (SPARK-11583) Make MapStatus use less memory uage

2015-11-10 Thread Daniel Lemire (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14999117#comment-14999117 ] Daniel Lemire edited comment on SPARK-11583 at 11/10/15 8:40 PM: - > When

[jira] [Updated] (SPARK-11638) Apache Spark in Docker with Bridge networking / run Spark on Mesos, in Docker with Bridge networking

2015-11-10 Thread Radoslaw Gruchalski (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11638?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Radoslaw Gruchalski updated SPARK-11638: Summary: Apache Spark in Docker with Bridge networking / run Spark on Mesos, in

[jira] [Commented] (SPARK-11638) Apache Spark in Docker with Bridge networking / run Spark on Mesos, in Docker with Bridge networking

2015-11-10 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11638?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14999399#comment-14999399 ] Sean Owen commented on SPARK-11638: --- We don't use patches in Spark. Why not just get the text ready

[jira] [Updated] (SPARK-8459) Add import/export to spark.mllib bisecting k-means

2015-11-10 Thread Yu Ishikawa (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8459?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yu Ishikawa updated SPARK-8459: --- Labels: 1.7.0 (was: ) > Add import/export to spark.mllib bisecting k-means >

[jira] [Commented] (SPARK-9830) Remove AggregateExpression1 and Aggregate Operator used to evaluate AggregateExpression1s

2015-11-10 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9830?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14999441#comment-14999441 ] Apache Spark commented on SPARK-9830: - User 'yhuai' has created a pull request for this issue:

[jira] [Assigned] (SPARK-11641) Exchange plan string is too verbose

2015-11-10 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11641?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-11641: Assignee: Apache Spark (was: Reynold Xin) > Exchange plan string is too verbose >

[jira] [Resolved] (SPARK-11550) Replace example code in mllib-optimization.md using include_example

2015-11-10 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11550?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng resolved SPARK-11550. --- Resolution: Fixed Fix Version/s: 1.6.0 Issue resolved by pull request 9516

[jira] [Updated] (SPARK-11550) Replace example code in mllib-optimization.md using include_example

2015-11-10 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11550?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-11550: -- Assignee: Pravin Gadakh > Replace example code in mllib-optimization.md using include_example

[jira] [Updated] (SPARK-11550) Replace example code in mllib-optimization.md using include_example

2015-11-10 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11550?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-11550: -- Target Version/s: 1.6.0 > Replace example code in mllib-optimization.md using include_example

[jira] [Resolved] (SPARK-11641) Exchange plan string is too verbose

2015-11-10 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11641?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-11641. - Resolution: Fixed Assignee: Yin Huai (was: Reynold Xin) Fix Version/s: 1.6.0 >

[jira] [Updated] (SPARK-6519) Add spark.ml API for bisecting k-means

2015-11-10 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6519?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-6519: - Shepherd: Xiangrui Meng Assignee: Yu Ishikawa > Add spark.ml API for bisecting k-means >

[jira] [Resolved] (SPARK-10726) Using dynamic-executor-allocation,When jobs are submitted parallelly, executors will be removed before tasks finish

2015-11-10 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10726?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or resolved SPARK-10726. --- Resolution: Duplicate > Using dynamic-executor-allocation,When jobs are submitted parallelly, >

  1   2   3   >