[jira] [Created] (SPARK-18847) PageRank gives incorrect results for graphs with sinks

Andrew Ray created SPARK-18847: -- Summary: PageRank gives incorrect results for graphs with sinks Key: SPARK-18847 URL: https://issues.apache.org/jira/browse/SPARK-18847 Project: Spark Issue

[jira] [Updated] (SPARK-18471) In treeAggregate, generate (big) zeros instead of sending them.

[ https://issues.apache.org/jira/browse/SPARK-18471?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-18471: -- Assignee: Anthony Truchet > In treeAggregate, generate (big) zeros instead of sending them. >

[jira] [Resolved] (SPARK-18471) In treeAggregate, generate (big) zeros instead of sending them.

[ https://issues.apache.org/jira/browse/SPARK-18471?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-18471. --- Resolution: Fixed Fix Version/s: 2.2.0 Issue resolved by pull request 16037

[jira] [Updated] (SPARK-18715) Fix wrong AIC calculation in Binomial GLM

[ https://issues.apache.org/jira/browse/SPARK-18715?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-18715: -- Assignee: Wayne Zhang Priority: Major (was: Critical) > Fix wrong AIC calculation in Binomial GLM

[jira] [Resolved] (SPARK-18715) Fix wrong AIC calculation in Binomial GLM

[ https://issues.apache.org/jira/browse/SPARK-18715?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-18715. --- Resolution: Fixed Issue resolved by pull request 16149 [https://github.com/apache/spark/pull/16149]

[jira] [Commented] (SPARK-18845) PageRank has incorrect initialization value that leads to slow convergence

[ https://issues.apache.org/jira/browse/SPARK-18845?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15746316#comment-15746316 ] Sean Owen commented on SPARK-18845: --- See https://issues.apache.org/jira/browse/SPARK-7005 ? > PageRank

[jira] [Created] (SPARK-18846) Fix flakiness in SchedulerIntegrationSuite

Imran Rashid created SPARK-18846: Summary: Fix flakiness in SchedulerIntegrationSuite Key: SPARK-18846 URL: https://issues.apache.org/jira/browse/SPARK-18846 Project: Spark Issue Type: Test

[jira] [Created] (SPARK-18845) PageRank has incorrect initialization value that leads to slow convergence

Andrew Ray created SPARK-18845: -- Summary: PageRank has incorrect initialization value that leads to slow convergence Key: SPARK-18845 URL: https://issues.apache.org/jira/browse/SPARK-18845 Project:

[jira] [Commented] (SPARK-18844) Add more binary classification metrics to BinaryClassificationMetrics

[ https://issues.apache.org/jira/browse/SPARK-18844?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15746293#comment-15746293 ] Zak Patterson commented on SPARK-18844: --- I'm not familiar with the python API much, but it seems to

[jira] [Commented] (SPARK-18838) High latency of event processing for large jobs

[ https://issues.apache.org/jira/browse/SPARK-18838?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15746217#comment-15746217 ] Shixiong Zhu commented on SPARK-18838: -- [~sitalke...@gmail.com] Instead of your proposal, I prefer

[jira] [Commented] (SPARK-18844) Add more binary classification metrics to BinaryClassificationMetrics

[ https://issues.apache.org/jira/browse/SPARK-18844?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15746212#comment-15746212 ] Sean Owen commented on SPARK-18844: --- Yeah I think we discussed something like this before, and the

[jira] [Comment Edited] (SPARK-18281) toLocalIterator yields time out error on pyspark2

[ https://issues.apache.org/jira/browse/SPARK-18281?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15746023#comment-15746023 ] Mike Dusenberry edited comment on SPARK-18281 at 12/13/16 7:56 PM: ---

[jira] [Commented] (SPARK-18281) toLocalIterator yields time out error on pyspark2

[ https://issues.apache.org/jira/browse/SPARK-18281?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15746023#comment-15746023 ] Mike Dusenberry commented on SPARK-18281: - Here's another interesting finding. The first

[jira] [Commented] (SPARK-18281) toLocalIterator yields time out error on pyspark2

[ https://issues.apache.org/jira/browse/SPARK-18281?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15746002#comment-15746002 ] Mike Dusenberry commented on SPARK-18281: - [~viirya] Thanks for taking on this bug! I tried out

[jira] [Updated] (SPARK-18844) Add more binary classification metrics to BinaryClassificationMetrics

[ https://issues.apache.org/jira/browse/SPARK-18844?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zak Patterson updated SPARK-18844: -- Description: BinaryClassificationMetrics only implements Precision (positive predictive

[jira] [Updated] (SPARK-18844) Add more binary classification metrics to BinaryClassificationMetrics

[ https://issues.apache.org/jira/browse/SPARK-18844?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zak Patterson updated SPARK-18844: -- Remaining Estimate: 5h (was: 5m) Original Estimate: 5h (was: 5m) > Add more binary

[jira] [Created] (SPARK-18844) Add more binary classification metrics to BinaryClassificationMetrics

Zak Patterson created SPARK-18844: - Summary: Add more binary classification metrics to BinaryClassificationMetrics Key: SPARK-18844 URL: https://issues.apache.org/jira/browse/SPARK-18844 Project:

[jira] [Commented] (SPARK-18278) Support native submission of spark jobs to a kubernetes cluster

[ https://issues.apache.org/jira/browse/SPARK-18278?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15745944#comment-15745944 ] Matt Cheah commented on SPARK-18278: [~rxin] - thanks for thinking about this! The concerns around

[jira] [Commented] (SPARK-18840) HDFSCredentialProvider throws exception in non-HDFS security environment

[ https://issues.apache.org/jira/browse/SPARK-18840?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15745879#comment-15745879 ] Marcelo Vanzin commented on SPARK-18840: [~jerryshao] are you planning to fix this for 2.0/1.6

[jira] [Updated] (SPARK-18835) Do not expose shaded types in JavaTypeInference API

[ https://issues.apache.org/jira/browse/SPARK-18835?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin updated SPARK-18835: --- Fix Version/s: 2.2.0 > Do not expose shaded types in JavaTypeInference API >

[jira] [Updated] (SPARK-18840) HDFSCredentialProvider throws exception in non-HDFS security environment

[ https://issues.apache.org/jira/browse/SPARK-18840?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin updated SPARK-18840: --- Fix Version/s: 2.2.0 2.1.1 > HDFSCredentialProvider throws exception in

[jira] [Updated] (SPARK-18840) HDFSCredentialProvider throws exception in non-HDFS security environment

[ https://issues.apache.org/jira/browse/SPARK-18840?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin updated SPARK-18840: --- Assignee: Saisai Shao > HDFSCredentialProvider throws exception in non-HDFS security

[jira] [Assigned] (SPARK-18843) Fix timeout in awaitResultInForkJoinSafely

[ https://issues.apache.org/jira/browse/SPARK-18843?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-18843: Assignee: Shixiong Zhu (was: Apache Spark) > Fix timeout in awaitResultInForkJoinSafely

[jira] [Commented] (SPARK-18843) Fix timeout in awaitResultInForkJoinSafely

[ https://issues.apache.org/jira/browse/SPARK-18843?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15745871#comment-15745871 ] Apache Spark commented on SPARK-18843: -- User 'zsxwing' has created a pull request for this issue:

[jira] [Assigned] (SPARK-18843) Fix timeout in awaitResultInForkJoinSafely

[ https://issues.apache.org/jira/browse/SPARK-18843?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-18843: Assignee: Apache Spark (was: Shixiong Zhu) > Fix timeout in awaitResultInForkJoinSafely

[jira] [Commented] (SPARK-18786) pySpark SQLContext.getOrCreate(sc) take stopped sparkContext

[ https://issues.apache.org/jira/browse/SPARK-18786?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15745867#comment-15745867 ] Bryan Cutler commented on SPARK-18786: -- The problem is that {{SQLContext.getOrCreate(sc)}} will not

[jira] [Created] (SPARK-18843) Fix timeout in awaitResultInForkJoinSafely

Shixiong Zhu created SPARK-18843: Summary: Fix timeout in awaitResultInForkJoinSafely Key: SPARK-18843 URL: https://issues.apache.org/jira/browse/SPARK-18843 Project: Spark Issue Type: Bug

[jira] [Resolved] (SPARK-18835) Do not expose shaded types in JavaTypeInference API

[ https://issues.apache.org/jira/browse/SPARK-18835?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-18835. Resolution: Fixed Assignee: Marcelo Vanzin Fix Version/s: 2.1.1 > Do not

[jira] [Resolved] (SPARK-13747) Concurrent execution in SQL doesn't work with Scala ForkJoinPool

[ https://issues.apache.org/jira/browse/SPARK-13747?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai resolved SPARK-13747. -- Resolution: Fixed Fix Version/s: (was: 2.0.2) (was: 2.1.0)

[jira] [Resolved] (SPARK-18675) CTAS for hive serde table should work for all hive versions

[ https://issues.apache.org/jira/browse/SPARK-18675?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai resolved SPARK-18675. -- Resolution: Fixed Fix Version/s: 2.2.0 Issue resolved by pull request 16104

[jira] [Commented] (SPARK-18823) Assignation by column name variable not available or bug?

[ https://issues.apache.org/jira/browse/SPARK-18823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15745633#comment-15745633 ] Shivaram Venkataraman commented on SPARK-18823: --- Thanks [~masip85] for verifying this. I

[jira] [Updated] (SPARK-18842) De-duplicate paths in classpaths in processes for local-cluster mode to work around the length limitation on Windows

[ https://issues.apache.org/jira/browse/SPARK-18842?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-18842: - Priority: Major (was: Minor) > De-duplicate paths in classpaths in processes for local-cluster

[jira] [Commented] (SPARK-3012) Standardized Distance Functions between two Vectors for MLlib

[ https://issues.apache.org/jira/browse/SPARK-3012?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15745388#comment-15745388 ] Dhaval Modi commented on SPARK-3012: I have implemented Mahalanobis Distance in Spark 1.6.2+ using

[jira] [Commented] (SPARK-18842) De-duplicate paths in classpaths in processes for local-cluster mode to work around the length limitation on Windows

[ https://issues.apache.org/jira/browse/SPARK-18842?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15745350#comment-15745350 ] Hyukjin Kwon commented on SPARK-18842: -- This completes the parent task and I can proceed far more.

[jira] [Assigned] (SPARK-18841) PushProjectionThroughUnion exception when there are same column

[ https://issues.apache.org/jira/browse/SPARK-18841?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-18841: Assignee: (was: Apache Spark) > PushProjectionThroughUnion exception when there are

[jira] [Commented] (SPARK-18841) PushProjectionThroughUnion exception when there are same column

[ https://issues.apache.org/jira/browse/SPARK-18841?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15745339#comment-15745339 ] Apache Spark commented on SPARK-18841: -- User 'windpiger' has created a pull request for this issue:

[jira] [Assigned] (SPARK-18841) PushProjectionThroughUnion exception when there are same column

[ https://issues.apache.org/jira/browse/SPARK-18841?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-18841: Assignee: Apache Spark > PushProjectionThroughUnion exception when there are same column

[jira] [Updated] (SPARK-18841) PushProjectionThroughUnion exception when there are same column

[ https://issues.apache.org/jira/browse/SPARK-18841?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell updated SPARK-18841: -- Description: {noformat} DROP TABLE IF EXISTS p1 ; DROP TABLE IF EXISTS p2 ; DROP TABLE

[jira] [Assigned] (SPARK-18842) De-duplicate paths in classpaths in processes for local-cluster mode to work around the length limitation on Windows

[ https://issues.apache.org/jira/browse/SPARK-18842?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-18842: Assignee: Apache Spark > De-duplicate paths in classpaths in processes for local-cluster

[jira] [Assigned] (SPARK-18842) De-duplicate paths in classpaths in processes for local-cluster mode to work around the length limitation on Windows

[ https://issues.apache.org/jira/browse/SPARK-18842?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-18842: Assignee: (was: Apache Spark) > De-duplicate paths in classpaths in processes for

[jira] [Commented] (SPARK-18842) De-duplicate paths in classpaths in processes for local-cluster mode to work around the length limitation on Windows

[ https://issues.apache.org/jira/browse/SPARK-18842?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15745319#comment-15745319 ] Apache Spark commented on SPARK-18842: -- User 'HyukjinKwon' has created a pull request for this

[jira] [Commented] (SPARK-18609) [SQL] column mixup with CROSS JOIN

[ https://issues.apache.org/jira/browse/SPARK-18609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15745309#comment-15745309 ] Song Jun commented on SPARK-18609: -- open another jira SPARK-18841 > [SQL] column mixup with CROSS JOIN

[jira] [Created] (SPARK-18842) De-duplicate paths in classpaths in processes for local-cluster mode to work around the length limitation on Windows

Hyukjin Kwon created SPARK-18842: Summary: De-duplicate paths in classpaths in processes for local-cluster mode to work around the length limitation on Windows Key: SPARK-18842 URL:

[jira] [Created] (SPARK-18841) PushProjectionThroughUnion exception when there are same column

Song Jun created SPARK-18841: Summary: PushProjectionThroughUnion exception when there are same column Key: SPARK-18841 URL: https://issues.apache.org/jira/browse/SPARK-18841 Project: Spark

[jira] [Commented] (SPARK-17890) scala.ScalaReflectionException

[ https://issues.apache.org/jira/browse/SPARK-17890?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15744964#comment-15744964 ] Arkadiusz Komarzewski commented on SPARK-17890: --- I hit this error recently, checked 2.1

[jira] [Updated] (SPARK-18796) StreamingQueryManager should not hold a lock when starting a query

[ https://issues.apache.org/jira/browse/SPARK-18796?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-18796: -- Assignee: Shixiong Zhu > StreamingQueryManager should not hold a lock when starting a query >

[jira] [Updated] (SPARK-18797) Update spark.logit in sparkr-vignettes

[ https://issues.apache.org/jira/browse/SPARK-18797?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-18797: -- Assignee: Miao Wang > Update spark.logit in sparkr-vignettes > --

[jira] [Updated] (SPARK-18642) Spark SQL: Catalyst is scanning undesired columns

[ https://issues.apache.org/jira/browse/SPARK-18642?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-18642: -- Assignee: Dongjoon Hyun > Spark SQL: Catalyst is scanning undesired columns >

[jira] [Updated] (SPARK-18752) "isSrcLocal" parameter to Hive loadTable / loadPartition should come from user

[ https://issues.apache.org/jira/browse/SPARK-18752?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-18752: -- Assignee: Marcelo Vanzin > "isSrcLocal" parameter to Hive loadTable / loadPartition should come from

[jira] [Commented] (SPARK-18823) Assignation by column name variable not available or bug?

[ https://issues.apache.org/jira/browse/SPARK-18823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15744740#comment-15744740 ] Vicente Masip commented on SPARK-18823: --- Yes. I've been able to do it with your suggestion. In my

[jira] [Comment Edited] (SPARK-18650) race condition in FileScanRDD.scala

[ https://issues.apache.org/jira/browse/SPARK-18650?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15744695#comment-15744695 ] Soumabrata Chakraborty edited comment on SPARK-18650 at 12/13/16 9:43 AM:

[jira] [Commented] (SPARK-18650) race condition in FileScanRDD.scala

[ https://issues.apache.org/jira/browse/SPARK-18650?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15744695#comment-15744695 ] Soumabrata Chakraborty commented on SPARK-18650: I am facing the same issue while trying

[jira] [Comment Edited] (SPARK-18840) HDFSCredentialProvider throws exception in non-HDFS security environment

[ https://issues.apache.org/jira/browse/SPARK-18840?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15744370#comment-15744370 ] Saisai Shao edited comment on SPARK-18840 at 12/13/16 9:30 AM: --- This

[jira] [Assigned] (SPARK-18840) HDFSCredentialProvider throws exception in non-HDFS security environment

[ https://issues.apache.org/jira/browse/SPARK-18840?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-18840: Assignee: (was: Apache Spark) > HDFSCredentialProvider throws exception in non-HDFS

[jira] [Commented] (SPARK-18840) HDFSCredentialProvider throws exception in non-HDFS security environment

[ https://issues.apache.org/jira/browse/SPARK-18840?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15744670#comment-15744670 ] Apache Spark commented on SPARK-18840: -- User 'jerryshao' has created a pull request for this issue:

[jira] [Assigned] (SPARK-18840) HDFSCredentialProvider throws exception in non-HDFS security environment

[ https://issues.apache.org/jira/browse/SPARK-18840?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-18840: Assignee: Apache Spark > HDFSCredentialProvider throws exception in non-HDFS security

[jira] [Updated] (SPARK-18837) Very long stage descriptions do not wrap in the UI

[ https://issues.apache.org/jira/browse/SPARK-18837?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-18837: -- Summary: Very long stage descriptions do not wrap in the UI (was: It will not hidden if job or stage

[jira] [Updated] (SPARK-18837) It will not hidden if job or stage description too long

[ https://issues.apache.org/jira/browse/SPARK-18837?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-18837: -- Priority: Minor (was: Major) Issue Type: Improvement (was: Bug) I agree that at least this

[jira] [Commented] (SPARK-18835) Do not expose shaded types in JavaTypeInference API

[ https://issues.apache.org/jira/browse/SPARK-18835?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15744600#comment-15744600 ] Sean Owen commented on SPARK-18835: --- (Ideally we should fix this for 2.1.0 because it causes the tests

[jira] [Closed] (SPARK-18804) Join doesn't work in Spark on Bigger tables

[ https://issues.apache.org/jira/browse/SPARK-18804?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen closed SPARK-18804. - > Join doesn't work in Spark on Bigger tables > --- > >

[jira] [Resolved] (SPARK-18804) Join doesn't work in Spark on Bigger tables

[ https://issues.apache.org/jira/browse/SPARK-18804?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-18804. --- Resolution: Not A Problem (Please don't reopen if the discussion has not meaningfully changed. JIRA

[jira] [Commented] (SPARK-18823) Assignation by column name variable not available or bug?

[ https://issues.apache.org/jira/browse/SPARK-18823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15744519#comment-15744519 ] Shivaram Venkataraman commented on SPARK-18823: --- Ah I see your point - `withColumn` does

<    1   2