[jira] [Updated] (SPARK-13850) TimSort Comparison method violates its general contract

[ https://issues.apache.org/jira/browse/SPARK-13850?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sital Kedia updated SPARK-13850: Description: While running a query which does a group by on a large dataset, the query fails with

[jira] [Created] (SPARK-13850) TimSort Comparison method violates its general contract

Sital Kedia created SPARK-13850: --- Summary: TimSort Comparison method violates its general contract Key: SPARK-13850 URL: https://issues.apache.org/jira/browse/SPARK-13850 Project: Spark Issue

[jira] [Updated] (SPARK-13764) Parse modes in JSON data source

[ https://issues.apache.org/jira/browse/SPARK-13764?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-13764: - Description: Currently, JSON data source just fails to read if some JSON documents are

[jira] [Commented] (SPARK-13249) Filter null keys for inner join

[ https://issues.apache.org/jira/browse/SPARK-13249?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15192723#comment-15192723 ] Liang-Chi Hsieh commented on SPARK-13249: - I think this can be closed? > Filter null keys for

[jira] [Resolved] (SPARK-13823) Always specify Charset in String <-> byte[] conversions (and remaining Coverity items)

[ https://issues.apache.org/jira/browse/SPARK-13823?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-13823. - Resolution: Fixed Fix Version/s: 2.0.0 > Always specify Charset in String <-> byte[]

[jira] [Commented] (SPARK-13724) Parameter maxMemoryInMB has gone missing in MlLib 1.6.0 DecisionTree.trainClassifier()

[ https://issues.apache.org/jira/browse/SPARK-13724?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15192718#comment-15192718 ] senthil gandhi commented on SPARK-13724: I was able to replicate this, here is the exception, how

[jira] [Updated] (SPARK-13845) BlockStatus and StreamBlockId keep on growing result driver OOM

[ https://issues.apache.org/jira/browse/SPARK-13845?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jeanlyn updated SPARK-13845: Summary: BlockStatus and StreamBlockId keep on growing result driver OOM (was: Driver OOM after few days

[jira] [Resolved] (SPARK-13834) Update sbt and sbt plugins for 2.x.

[ https://issues.apache.org/jira/browse/SPARK-13834?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-13834. - Resolution: Fixed Assignee: Dongjoon Hyun (was: Apache Spark) Fix Version/s:

[jira] [Created] (SPARK-13849) REGEX Column Specification

Ruslan Dautkhanov created SPARK-13849: - Summary: REGEX Column Specification Key: SPARK-13849 URL: https://issues.apache.org/jira/browse/SPARK-13849 Project: Spark Issue Type: Wish

[jira] [Commented] (SPARK-13335) Optimize Data Frames collect_list and collect_set with declarative aggregates

[ https://issues.apache.org/jira/browse/SPARK-13335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15192588#comment-15192588 ] Ruslan Dautkhanov commented on SPARK-13335: --- It would be great to have this optimization in. In

[jira] [Assigned] (SPARK-13335) Optimize Data Frames collect_list and collect_set with declarative aggregates

[ https://issues.apache.org/jira/browse/SPARK-13335?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13335: Assignee: Apache Spark > Optimize Data Frames collect_list and collect_set with

[jira] [Commented] (SPARK-13335) Optimize Data Frames collect_list and collect_set with declarative aggregates

[ https://issues.apache.org/jira/browse/SPARK-13335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15192570#comment-15192570 ] Apache Spark commented on SPARK-13335: -- User 'mccheah' has created a pull request for this issue:

[jira] [Assigned] (SPARK-13335) Optimize Data Frames collect_list and collect_set with declarative aggregates

[ https://issues.apache.org/jira/browse/SPARK-13335?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13335: Assignee: (was: Apache Spark) > Optimize Data Frames collect_list and collect_set

[jira] [Commented] (SPARK-5185) pyspark --jars does not add classes to driver class path

[ https://issues.apache.org/jira/browse/SPARK-5185?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15192544#comment-15192544 ] Apache Spark commented on SPARK-5185: - User 'JoshRosen' has created a pull request for this issue:

[jira] [Commented] (SPARK-13848) Upgrade to Py4J 0.9.2

[ https://issues.apache.org/jira/browse/SPARK-13848?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15192543#comment-15192543 ] Apache Spark commented on SPARK-13848: -- User 'JoshRosen' has created a pull request for this issue:

[jira] [Updated] (SPARK-5185) pyspark --jars does not add classes to driver class path

[ https://issues.apache.org/jira/browse/SPARK-5185?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-5185: -- Target Version/s: 2.0.0 > pyspark --jars does not add classes to driver class path >

[jira] [Resolved] (SPARK-6047) pyspark - class loading on driver failing with --jars and --packages

[ https://issues.apache.org/jira/browse/SPARK-6047?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen resolved SPARK-6047. --- Resolution: Duplicate > pyspark - class loading on driver failing with --jars and --packages >

[jira] [Assigned] (SPARK-5185) pyspark --jars does not add classes to driver class path

[ https://issues.apache.org/jira/browse/SPARK-5185?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen reassigned SPARK-5185: - Assignee: Josh Rosen (was: Andrew Or) > pyspark --jars does not add classes to driver class

[jira] [Commented] (SPARK-6047) pyspark - class loading on driver failing with --jars and --packages

[ https://issues.apache.org/jira/browse/SPARK-6047?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15192540#comment-15192540 ] Josh Rosen commented on SPARK-6047: --- Resolving this as a duplicate of SPARK-5185, since I'm about to

[jira] [Resolved] (SPARK-13812) Fix SparkR lint-r test errors

[ https://issues.apache.org/jira/browse/SPARK-13812?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shivaram Venkataraman resolved SPARK-13812. --- Resolution: Fixed Assignee: Sun Rui Fix Version/s: 2.0.0

[jira] [Commented] (SPARK-13784) Model export/import for spark.ml: RandomForests

[ https://issues.apache.org/jira/browse/SPARK-13784?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15192537#comment-15192537 ] Gayathri Murali commented on SPARK-13784: - I can work on this, if no one else has started >

[jira] [Commented] (SPARK-6722) Model import/export for StreamingKMeansModel

[ https://issues.apache.org/jira/browse/SPARK-6722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15192533#comment-15192533 ] Furkan KAMACI commented on SPARK-6722: -- Is there any road map suggested about how to implement this?

[jira] [Created] (SPARK-13848) Upgrade to Py4J 0.9.2

Josh Rosen created SPARK-13848: -- Summary: Upgrade to Py4J 0.9.2 Key: SPARK-13848 URL: https://issues.apache.org/jira/browse/SPARK-13848 Project: Spark Issue Type: Bug Components:

[jira] [Commented] (SPARK-13825) Upgrade to Scala 2.11.8

[ https://issues.apache.org/jira/browse/SPARK-13825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15192498#comment-15192498 ] Sean Owen commented on SPARK-13825: --- I don't think it's confusing to bump maintenance releases of two

[jira] [Commented] (SPARK-13825) Upgrade to Scala 2.11.8

[ https://issues.apache.org/jira/browse/SPARK-13825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15192496#comment-15192496 ] Jacek Laskowski commented on SPARK-13825: - I can do it in a separate task. The title would not

[jira] [Commented] (SPARK-13825) Upgrade to Scala 2.11.8

[ https://issues.apache.org/jira/browse/SPARK-13825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15192389#comment-15192389 ] Sean Owen commented on SPARK-13825: --- While you're at it, I think it's fine to update the 2.10 line to

[jira] [Closed] (SPARK-13847) Defer the variable evaluation for Limit codegen

[ https://issues.apache.org/jira/browse/SPARK-13847?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liang-Chi Hsieh closed SPARK-13847. --- Resolution: Not A Problem > Defer the variable evaluation for Limit codegen >

[jira] [Commented] (SPARK-13847) Defer the variable evaluation for Limit codegen

[ https://issues.apache.org/jira/browse/SPARK-13847?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15192371#comment-15192371 ] Apache Spark commented on SPARK-13847: -- User 'viirya' has created a pull request for this issue:

[jira] [Assigned] (SPARK-13847) Defer the variable evaluation for Limit codegen

[ https://issues.apache.org/jira/browse/SPARK-13847?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13847: Assignee: (was: Apache Spark) > Defer the variable evaluation for Limit codegen >

[jira] [Assigned] (SPARK-13847) Defer the variable evaluation for Limit codegen

[ https://issues.apache.org/jira/browse/SPARK-13847?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13847: Assignee: Apache Spark > Defer the variable evaluation for Limit codegen >

[jira] [Created] (SPARK-13847) Defer the variable evaluation for Limit codegen

Liang-Chi Hsieh created SPARK-13847: --- Summary: Defer the variable evaluation for Limit codegen Key: SPARK-13847 URL: https://issues.apache.org/jira/browse/SPARK-13847 Project: Spark Issue

[jira] [Comment Edited] (SPARK-6378) srcAttr in graph.triplets don't update when the size of graph is huge

[ https://issues.apache.org/jira/browse/SPARK-6378?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15192332#comment-15192332 ] Zhaokang Wang edited comment on SPARK-6378 at 3/13/16 1:08 PM: --- I have met a

[jira] [Resolved] (SPARK-12216) Spark failed to delete temp directory

[ https://issues.apache.org/jira/browse/SPARK-12216?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-12216. --- Resolution: Invalid > Spark failed to delete temp directory >

[jira] [Commented] (SPARK-6378) srcAttr in graph.triplets don't update when the size of graph is huge

[ https://issues.apache.org/jira/browse/SPARK-6378?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15192333#comment-15192333 ] Zhaokang Wang commented on SPARK-6378: -- I have reproduced this problem in a small toy graph demo with

[jira] [Updated] (SPARK-6378) srcAttr in graph.triplets don't update when the size of graph is huge

[ https://issues.apache.org/jira/browse/SPARK-6378?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhaokang Wang updated SPARK-6378: - Attachment: TripletsViewDonotUpdate.scala I have met a similar problem with triplets update in

[jira] [Commented] (SPARK-13298) DAG visualization does not render correctly for jobs

[ https://issues.apache.org/jira/browse/SPARK-13298?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15192324#comment-15192324 ] Todd Leo commented on SPARK-13298: -- Pls also see SPARK-13645: DAG Diagram not shown properly in Chrome

[jira] [Commented] (SPARK-13718) Scheduler "creating" straggler node

[ https://issues.apache.org/jira/browse/SPARK-13718?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15192309#comment-15192309 ] Sean Owen commented on SPARK-13718: --- What do you mean that it tries to assign without an available core

[jira] [Comment Edited] (SPARK-13073) creating R like summary for logistic Regression in Spark - Scala

[ https://issues.apache.org/jira/browse/SPARK-13073?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15192290#comment-15192290 ] Mohamed Baddar edited comment on SPARK-13073 at 3/13/16 11:03 AM: --

[jira] [Commented] (SPARK-13073) creating R like summary for logistic Regression in Spark - Scala

[ https://issues.apache.org/jira/browse/SPARK-13073?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15192290#comment-15192290 ] Mohamed Baddar commented on SPARK-13073: [~josephkb] After more investigation in the code , and

[jira] [Comment Edited] (SPARK-12216) Spark failed to delete temp directory

[ https://issues.apache.org/jira/browse/SPARK-12216?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15192281#comment-15192281 ] Guram Savinov edited comment on SPARK-12216 at 3/13/16 10:49 AM: - I have

[jira] [Commented] (SPARK-12216) Spark failed to delete temp directory

[ https://issues.apache.org/jira/browse/SPARK-12216?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15192281#comment-15192281 ] Guram Savinov commented on SPARK-12216: --- I have the same problem when exit from spark-shell on

[jira] [Resolved] (SPARK-13810) Add Port Configuration Suggestions on Bind Exceptions

[ https://issues.apache.org/jira/browse/SPARK-13810?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-13810. --- Resolution: Fixed Fix Version/s: 1.6.2 2.0.0 Issue resolved by pull

[jira] [Updated] (SPARK-13810) Add Port Configuration Suggestions on Bind Exceptions

[ https://issues.apache.org/jira/browse/SPARK-13810?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-13810: -- Assignee: Bjorn Jonsson > Add Port Configuration Suggestions on Bind Exceptions >

[jira] [Updated] (SPARK-13846) VectorIndexer output on unknown feature should be more descriptive

[ https://issues.apache.org/jira/browse/SPARK-13846?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-13846: -- Issue Type: Improvement (was: Bug) > VectorIndexer output on unknown feature should be more

[jira] [Updated] (SPARK-13846) VectorIndexer output on unknown feature should be more descriptive

[ https://issues.apache.org/jira/browse/SPARK-13846?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dmitry Spikhalskiy updated SPARK-13846: --- Description: I got the exception and looks like it's related to unknown categorical

[jira] [Created] (SPARK-13846) VectorIndexer output on unknown feature should be more descriptive

Dmitry Spikhalskiy created SPARK-13846: -- Summary: VectorIndexer output on unknown feature should be more descriptive Key: SPARK-13846 URL: https://issues.apache.org/jira/browse/SPARK-13846

[jira] [Commented] (SPARK-13845) Driver OOM after few days when running streaming

[ https://issues.apache.org/jira/browse/SPARK-13845?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15192259#comment-15192259 ] Sean Owen commented on SPARK-13845: --- [~jeanlyn] You didn't write a description, and the title does not

[jira] [Resolved] (SPARK-10775) add search keywords in history page ui

[ https://issues.apache.org/jira/browse/SPARK-10775?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta resolved SPARK-10775. Resolution: Duplicate Fix Version/s: 2.0.0 This issue is resolved by SPARK-10873.

[jira] [Assigned] (SPARK-13845) Driver OOM after few days when running streaming

[ https://issues.apache.org/jira/browse/SPARK-13845?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13845: Assignee: (was: Apache Spark) > Driver OOM after few days when running streaming >

[jira] [Commented] (SPARK-13845) Driver OOM after few days when running streaming

[ https://issues.apache.org/jira/browse/SPARK-13845?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15192227#comment-15192227 ] Apache Spark commented on SPARK-13845: -- User 'jeanlyn' has created a pull request for this issue:

[jira] [Assigned] (SPARK-13845) Driver OOM after few days when running streaming

[ https://issues.apache.org/jira/browse/SPARK-13845?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13845: Assignee: Apache Spark > Driver OOM after few days when running streaming >

[jira] [Created] (SPARK-13845) Driver OOM after few days when running streaming

jeanlyn created SPARK-13845: --- Summary: Driver OOM after few days when running streaming Key: SPARK-13845 URL: https://issues.apache.org/jira/browse/SPARK-13845 Project: Spark Issue Type: Bug

[jira] [Assigned] (SPARK-13844) Generate better code for filters with a non-nullable column

[ https://issues.apache.org/jira/browse/SPARK-13844?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13844: Assignee: Apache Spark > Generate better code for filters with a non-nullable column >

[jira] [Commented] (SPARK-13844) Generate better code for filters with a non-nullable column

[ https://issues.apache.org/jira/browse/SPARK-13844?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15192223#comment-15192223 ] Apache Spark commented on SPARK-13844: -- User 'kiszk' has created a pull request for this issue:

[jira] [Assigned] (SPARK-13844) Generate better code for filters with a non-nullable column

[ https://issues.apache.org/jira/browse/SPARK-13844?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13844: Assignee: (was: Apache Spark) > Generate better code for filters with a non-nullable