[jira] [Resolved] (SPARK-1021) sortByKey() launches a cluster job when it shouldn't

[ https://issues.apache.org/jira/browse/SPARK-1021?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-1021. Resolution: Fixed Fix Version/s: 1.2.0 sortByKey() launches a cluster job when it shouldn't

[jira] [Commented] (SPARK-3420) Using Sphinx to generate API docs for PySpark

[ https://issues.apache.org/jira/browse/SPARK-3420?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14150592#comment-14150592 ] cocoatomo commented on SPARK-3420: -- Do you mean to generate API documents using

[jira] [Created] (SPARK-3707) Type Coercion for DIV doesn't work for non-numeric argument

Cheng Hao created SPARK-3707: Summary: Type Coercion for DIV doesn't work for non-numeric argument Key: SPARK-3707 URL: https://issues.apache.org/jira/browse/SPARK-3707 Project: Spark Issue

[jira] [Commented] (SPARK-3707) Type Coercion for DIV doesn't work for non-numeric argument

[ https://issues.apache.org/jira/browse/SPARK-3707?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14150622#comment-14150622 ] Apache Spark commented on SPARK-3707: - User 'chenghao-intel' has created a pull

[jira] [Commented] (SPARK-3420) Using Sphinx to generate API docs for PySpark

[ https://issues.apache.org/jira/browse/SPARK-3420?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14150654#comment-14150654 ] Davies Liu commented on SPARK-3420: --- This is partially fixed by

[jira] [Created] (SPARK-3708) Backticks aren't handled correctly is aliases

Michael Armbrust created SPARK-3708: --- Summary: Backticks aren't handled correctly is aliases Key: SPARK-3708 URL: https://issues.apache.org/jira/browse/SPARK-3708 Project: Spark Issue

[jira] [Commented] (SPARK-922) Update Spark AMI to Python 2.7

[ https://issues.apache.org/jira/browse/SPARK-922?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14150695#comment-14150695 ] Andrew Davidson commented on SPARK-922: --- I must have missed something. I am running

[jira] [Updated] (SPARK-3586) Support nested directories in Spark Streaming

[ https://issues.apache.org/jira/browse/SPARK-3586?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-3586: - Issue Type: Improvement (was: Bug) Summary: Support nested directories in Spark Streaming (was:

[jira] [Commented] (SPARK-3603) InvalidClassException on a Linux VM - probably problem with serialization

[ https://issues.apache.org/jira/browse/SPARK-3603?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14150700#comment-14150700 ] Sean Owen commented on SPARK-3603: -- Can you clarify what works and doesn't work -- works

[jira] [Resolved] (SPARK-1279) Stage.name return apply at Option.scala:120

[ https://issues.apache.org/jira/browse/SPARK-1279?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-1279. -- Resolution: Duplicate Obvious accidental dupe of SPARK-1280 Stage.name return apply at

[jira] [Resolved] (SPARK-2960) Spark executables fail to start via symlinks

[ https://issues.apache.org/jira/browse/SPARK-2960?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-2960. -- Resolution: Duplicate Fix Version/s: (was: 1.0.2) I suggest this be marked a dupe of

[jira] [Commented] (SPARK-3504) KMeans clusterer is slow, can be sped up by 75%

[ https://issues.apache.org/jira/browse/SPARK-3504?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14150709#comment-14150709 ] Sean Owen commented on SPARK-3504: -- Is this the same as

[jira] [Resolved] (SPARK-3324) YARN module has nonstandard structure which cause compile error In IntelliJ

[ https://issues.apache.org/jira/browse/SPARK-3324?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-3324. -- Resolution: Won't Fix I propose considering this WontFix, as it will go away when yarn-alpha goes away

[jira] [Updated] (SPARK-3195) Can you add some statistics to do logistic regression better in mllib?

[ https://issues.apache.org/jira/browse/SPARK-3195?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-3195: - Priority: Minor (was: Major) Target Version/s: (was: 1.3.0) Fix Version/s: (was:

[jira] [Resolved] (SPARK-1046) Enable to build behind a proxy.

[ https://issues.apache.org/jira/browse/SPARK-1046?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-1046. -- Resolution: Fixed Enable to build behind a proxy. ---

[jira] [Resolved] (SPARK-3676) jdk version lead to spark sql test suite error

[ https://issues.apache.org/jira/browse/SPARK-3676?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-3676. - Resolution: Fixed Issue resolved by pull request 2517

[jira] [Resolved] (SPARK-3101) Missing volatile annotation in ApplicationMaster

[ https://issues.apache.org/jira/browse/SPARK-3101?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-3101. -- Resolution: Fixed This was actually subsumed by the commit for SPARK-2933 Missing volatile annotation

[jira] [Resolved] (SPARK-3680) java.lang.Exception: makeCopy when using HiveGeneric UDFs on Converted Parquet Metastore tables

[ https://issues.apache.org/jira/browse/SPARK-3680?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-3680. - Resolution: Fixed Fix Version/s: 1.2.0 Issue resolved by pull request 2525

[jira] [Resolved] (SPARK-2815) Compilation failed upon the hadoop version 2.0.0-cdh4.5.0

[ https://issues.apache.org/jira/browse/SPARK-2815?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-2815. -- Resolution: Won't Fix Target Version/s: (was: 1.1.0) This looks like a WontFix, given the

[jira] [Resolved] (SPARK-3681) Failed to serialized ArrayType or MapType after accessing them in Python

[ https://issues.apache.org/jira/browse/SPARK-3681?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-3681. - Resolution: Fixed Fix Version/s: 1.2.0 Issue resolved by pull request 2526

[jira] [Commented] (SPARK-3652) upgrade spark sql hive version to 0.13.1

[ https://issues.apache.org/jira/browse/SPARK-3652?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14150719#comment-14150719 ] Sean Owen commented on SPARK-3652: -- Yes, there is a lot to updating to 0.13 as you can

[jira] [Commented] (SPARK-2655) Change the default logging level to WARN

[ https://issues.apache.org/jira/browse/SPARK-2655?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14150740#comment-14150740 ] Sean Owen commented on SPARK-2655: -- Given the PR discussion, this is a WontFix? Change

[jira] [Resolved] (SPARK-2655) Change the default logging level to WARN

[ https://issues.apache.org/jira/browse/SPARK-2655?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell resolved SPARK-2655. Resolution: Won't Fix Change the default logging level to WARN

[jira] [Commented] (SPARK-2331) SparkContext.emptyRDD has wrong return type

[ https://issues.apache.org/jira/browse/SPARK-2331?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14150746#comment-14150746 ] Sean Owen commented on SPARK-2331: -- Yes your analysis is right-er, I think. I imagine

[jira] [Commented] (SPARK-2247) Data frame (or Pandas) like API for structured data

[ https://issues.apache.org/jira/browse/SPARK-2247?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14150748#comment-14150748 ] Sean Owen commented on SPARK-2247: -- For pandas, is this what Sparkling Pandas provides?

[jira] [Updated] (SPARK-2153) CassandraTest fails for newer Cassandra due to case insensitive key space

[ https://issues.apache.org/jira/browse/SPARK-2153?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-2153: - Summary: CassandraTest fails for newer Cassandra due to case insensitive key space (was: Spark Examples)

[jira] [Resolved] (SPARK-1895) Run tests on windows

[ https://issues.apache.org/jira/browse/SPARK-1895?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-1895. -- Resolution: Cannot Reproduce Run tests on windows Key:

[jira] [Commented] (SPARK-3504) KMeans clusterer is slow, can be sped up by 75%

[ https://issues.apache.org/jira/browse/SPARK-3504?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14150750#comment-14150750 ] Derrick Burns commented on SPARK-3504: -- No, these are two separate issues. One

[jira] [Commented] (SPARK-2517) Remove as many compilation warning messages as possible

[ https://issues.apache.org/jira/browse/SPARK-2517?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14150753#comment-14150753 ] Sean Owen commented on SPARK-2517: -- [~rxin] I think you resolved this? I don't see these

[jira] [Commented] (SPARK-3359) `sbt/sbt unidoc` doesn't work with Java 8

[ https://issues.apache.org/jira/browse/SPARK-3359?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14150778#comment-14150778 ] Sean Owen commented on SPARK-3359: -- Yeah I noticed this. The problem is that

[jira] [Created] (SPARK-3709) BroadcastSuite.Unpersisting rg.apache.spark.broadcast.BroadcastSuite.Unpersisting TorrentBroadcast is flaky

Patrick Wendell created SPARK-3709: -- Summary: BroadcastSuite.Unpersisting rg.apache.spark.broadcast.BroadcastSuite.Unpersisting TorrentBroadcast is flaky Key: SPARK-3709 URL:

[jira] [Created] (SPARK-3710) YARN integration test is flaky

Patrick Wendell created SPARK-3710: -- Summary: YARN integration test is flaky Key: SPARK-3710 URL: https://issues.apache.org/jira/browse/SPARK-3710 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-3710) YARN integration test is flaky

[ https://issues.apache.org/jira/browse/SPARK-3710?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-3710: --- Description: This has been regularly failing the master build: Example failure:

[jira] [Commented] (SPARK-3324) YARN module has nonstandard structure which cause compile error In IntelliJ

[ https://issues.apache.org/jira/browse/SPARK-3324?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14150873#comment-14150873 ] Yi Tian commented on SPARK-3324: I think the deeper problem is the codes in common and

[jira] [Resolved] (SPARK-2517) Remove as many compilation warning messages as possible

[ https://issues.apache.org/jira/browse/SPARK-2517?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-2517. Resolution: Fixed Fix Version/s: 1.2.0 Assignee: Reynold Xin (was: Yin Huai)

[jira] [Resolved] (SPARK-3389) Add converter class to make reading Parquet files easy with PySpark

[ https://issues.apache.org/jira/browse/SPARK-3389?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matei Zaharia resolved SPARK-3389. -- Resolution: Fixed Fix Version/s: 1.2.0 Add converter class to make reading Parquet

[jira] [Commented] (SPARK-1547) Add gradient boosting algorithm to MLlib

[ https://issues.apache.org/jira/browse/SPARK-1547?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14150964#comment-14150964 ] Manish Amde commented on SPARK-1547: Given the interesting in boosting algorithms for

[jira] [Resolved] (SPARK-3651) Consolidate executor maps in CoarseGrainedSchedulerBackend

[ https://issues.apache.org/jira/browse/SPARK-3651?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-3651. Resolution: Fixed Fix Version/s: 1.2.0 Consolidate executor maps in

[jira] [Commented] (SPARK-3543) Write TaskContext in Java and expose it through a static accessor

[ https://issues.apache.org/jira/browse/SPARK-3543?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14150981#comment-14150981 ] Apache Spark commented on SPARK-3543: - User 'rxin' has created a pull request for this

[jira] [Created] (SPARK-3711) Optimize where in clause filter queries

Yash Datta created SPARK-3711: - Summary: Optimize where in clause filter queries Key: SPARK-3711 URL: https://issues.apache.org/jira/browse/SPARK-3711 Project: Spark Issue Type: Improvement