[jira] [Commented] (SPARK-24760) Pandas UDF does not handle NaN correctly

[ https://issues.apache.org/jira/browse/SPARK-24760?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16541209#comment-16541209 ] Mortada Mehyar commented on SPARK-24760: [~bryanc] thanks for the example. It lo

[jira] [Commented] (SPARK-20202) Remove references to org.spark-project.hive

[ https://issues.apache.org/jira/browse/SPARK-20202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16541191#comment-16541191 ] Felix Cheung commented on SPARK-20202: -- How like will there be a hive release? HIVE

[jira] [Created] (SPARK-24792) Add API `.avro` in DataFrameReader/DataFrameWriter

Gengliang Wang created SPARK-24792: -- Summary: Add API `.avro` in DataFrameReader/DataFrameWriter Key: SPARK-24792 URL: https://issues.apache.org/jira/browse/SPARK-24792 Project: Spark Issue

[jira] [Created] (SPARK-24791) Spark Structured Streaming randomly does not process batch

Arvind Ramachandran created SPARK-24791: --- Summary: Spark Structured Streaming randomly does not process batch Key: SPARK-24791 URL: https://issues.apache.org/jira/browse/SPARK-24791 Project: Spar

[jira] [Commented] (SPARK-24705) Spark.sql.adaptive.enabled=true is enabled and self-join query

[ https://issues.apache.org/jira/browse/SPARK-24705?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16541063#comment-16541063 ] Apache Spark commented on SPARK-24705: -- User 'maropu' has created a pull request fo

[jira] [Assigned] (SPARK-24705) Spark.sql.adaptive.enabled=true is enabled and self-join query

[ https://issues.apache.org/jira/browse/SPARK-24705?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24705: Assignee: Apache Spark > Spark.sql.adaptive.enabled=true is enabled and self-join query >

[jira] [Assigned] (SPARK-24705) Spark.sql.adaptive.enabled=true is enabled and self-join query

[ https://issues.apache.org/jira/browse/SPARK-24705?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24705: Assignee: (was: Apache Spark) > Spark.sql.adaptive.enabled=true is enabled and self-j

[jira] [Commented] (SPARK-24705) Spark.sql.adaptive.enabled=true is enabled and self-join query

[ https://issues.apache.org/jira/browse/SPARK-24705?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16541065#comment-16541065 ] Takeshi Yamamuro commented on SPARK-24705: -- Sorry for my late response. IIUC I

[jira] [Commented] (SPARK-20202) Remove references to org.spark-project.hive

[ https://issues.apache.org/jira/browse/SPARK-20202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16541034#comment-16541034 ] Hyukjin Kwon commented on SPARK-20202: -- I am asking this to set the goal for this J

[jira] [Commented] (SPARK-24432) Add support for dynamic resource allocation

[ https://issues.apache.org/jira/browse/SPARK-24432?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16541024#comment-16541024 ] Mark Hamilton commented on SPARK-24432: --- Hey [~liyinan926], and [~foxish], I would

[jira] [Resolved] (SPARK-24529) Add spotbugs into maven build process

[ https://issues.apache.org/jira/browse/SPARK-24529?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-24529. -- Resolution: Fixed Fix Version/s: 2.4.0 Issue resolved by pull request 21542 [https://gi

[jira] [Assigned] (SPARK-24529) Add spotbugs into maven build process

[ https://issues.apache.org/jira/browse/SPARK-24529?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-24529: Assignee: Kazuaki Ishizaki > Add spotbugs into maven build process >

[jira] [Resolved] (SPARK-24761) Check modifiability of config parameters

[ https://issues.apache.org/jira/browse/SPARK-24761?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-24761. - Resolution: Fixed Assignee: Maxim Gekk Fix Version/s: 2.4.0 > Check modifiability of con

[jira] [Resolved] (SPARK-24782) Simplify conf access in expressions

[ https://issues.apache.org/jira/browse/SPARK-24782?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-24782. - Resolution: Fixed Assignee: Marco Gaido Fix Version/s: 2.4.0 > Simplify conf access in e

[jira] [Assigned] (SPARK-24790) Allow complex aggregate expressions in Pivot

[ https://issues.apache.org/jira/browse/SPARK-24790?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24790: Assignee: Apache Spark > Allow complex aggregate expressions in Pivot > -

[jira] [Commented] (SPARK-24790) Allow complex aggregate expressions in Pivot

[ https://issues.apache.org/jira/browse/SPARK-24790?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16540809#comment-16540809 ] Apache Spark commented on SPARK-24790: -- User 'maryannxue' has created a pull reques

[jira] [Assigned] (SPARK-24790) Allow complex aggregate expressions in Pivot

[ https://issues.apache.org/jira/browse/SPARK-24790?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24790: Assignee: (was: Apache Spark) > Allow complex aggregate expressions in Pivot > --

[jira] [Assigned] (SPARK-24764) Add ServiceLoader implementation for SparkHadoopUtil

[ https://issues.apache.org/jira/browse/SPARK-24764?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] DB Tsai reassigned SPARK-24764: --- Assignee: Shruti Gumma > Add ServiceLoader implementation for SparkHadoopUtil > ---

[jira] [Updated] (SPARK-24790) Allow complex aggregate expressions in Pivot

[ https://issues.apache.org/jira/browse/SPARK-24790?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maryann Xue updated SPARK-24790: Description: In SPARK-24035, to conform with Oracle PIVOT syntax, a strict check of PIVOT's aggreg

[jira] [Created] (SPARK-24790) Allow complex aggregate expressions in Pivot

Maryann Xue created SPARK-24790: --- Summary: Allow complex aggregate expressions in Pivot Key: SPARK-24790 URL: https://issues.apache.org/jira/browse/SPARK-24790 Project: Spark Issue Type: Improv

[jira] [Created] (SPARK-24789) SparkThriftServer - support for hive.server2.authentication.ldap.userFilter

t oo created SPARK-24789: Summary: SparkThriftServer - support for hive.server2.authentication.ldap.userFilter Key: SPARK-24789 URL: https://issues.apache.org/jira/browse/SPARK-24789 Project: Spark

[jira] [Assigned] (SPARK-24724) Discuss necessary info and access in barrier mode + Kubernetes

[ https://issues.apache.org/jira/browse/SPARK-24724?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li reassigned SPARK-24724: --- Assignee: Yinan Li (was: Xiangrui Meng) > Discuss necessary info and access in barrier mode + Kube

[jira] [Assigned] (SPARK-24724) Discuss necessary info and access in barrier mode + Kubernetes

[ https://issues.apache.org/jira/browse/SPARK-24724?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li reassigned SPARK-24724: --- Assignee: Xiangrui Meng > Discuss necessary info and access in barrier mode + Kubernetes >

[jira] [Assigned] (SPARK-24788) RelationalGroupedDataset.toString throws errors when grouping by UnresolvedAttribute

[ https://issues.apache.org/jira/browse/SPARK-24788?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24788: Assignee: (was: Apache Spark) > RelationalGroupedDataset.toString throws errors when

[jira] [Commented] (SPARK-24788) RelationalGroupedDataset.toString throws errors when grouping by UnresolvedAttribute

[ https://issues.apache.org/jira/browse/SPARK-24788?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16540699#comment-16540699 ] Apache Spark commented on SPARK-24788: -- User 'c-horn' has created a pull request fo

[jira] [Assigned] (SPARK-24788) RelationalGroupedDataset.toString throws errors when grouping by UnresolvedAttribute

[ https://issues.apache.org/jira/browse/SPARK-24788?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24788: Assignee: Apache Spark > RelationalGroupedDataset.toString throws errors when grouping by

[jira] [Created] (SPARK-24788) RelationalGroupedDataset.toString throws errors when grouping by UnresolvedAttribute

Chris Horn created SPARK-24788: -- Summary: RelationalGroupedDataset.toString throws errors when grouping by UnresolvedAttribute Key: SPARK-24788 URL: https://issues.apache.org/jira/browse/SPARK-24788 Proj

[jira] [Commented] (SPARK-14220) Build and test Spark against Scala 2.12

[ https://issues.apache.org/jira/browse/SPARK-14220?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16540676#comment-16540676 ] Stavros Kontopoulos commented on SPARK-14220: - If you make it more specific

[jira] [Comment Edited] (SPARK-14220) Build and test Spark against Scala 2.12

[ https://issues.apache.org/jira/browse/SPARK-14220?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16540676#comment-16540676 ] Stavros Kontopoulos edited comment on SPARK-14220 at 7/11/18 9:17 PM:

[jira] [Commented] (SPARK-24787) Events being dropped at an alarming rate due to hsync being slow for eventLogging

[ https://issues.apache.org/jira/browse/SPARK-24787?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16540638#comment-16540638 ] Sanket Reddy commented on SPARK-24787: -- I am happy to work on this... will have a p

[jira] [Created] (SPARK-24787) Events being dropped at an alarming rate due to hsync being slow for eventLogging

Sanket Reddy created SPARK-24787: Summary: Events being dropped at an alarming rate due to hsync being slow for eventLogging Key: SPARK-24787 URL: https://issues.apache.org/jira/browse/SPARK-24787 Pro

[jira] [Commented] (SPARK-24208) Cannot resolve column in self join after applying Pandas UDF

[ https://issues.apache.org/jira/browse/SPARK-24208?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16540612#comment-16540612 ] Apache Spark commented on SPARK-24208: -- User 'mgaido91' has created a pull request

[jira] [Resolved] (SPARK-24697) Fix the reported start offsets in streaming query progress

[ https://issues.apache.org/jira/browse/SPARK-24697?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tathagata Das resolved SPARK-24697. --- Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 21744 [https://

[jira] [Assigned] (SPARK-24697) Fix the reported start offsets in streaming query progress

[ https://issues.apache.org/jira/browse/SPARK-24697?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tathagata Das reassigned SPARK-24697: - Assignee: Tathagata Das > Fix the reported start offsets in streaming query progress >

[jira] [Updated] (SPARK-24786) Executors not being released after all cached data is unpersisted

[ https://issues.apache.org/jira/browse/SPARK-24786?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jeffrey Charles updated SPARK-24786: Affects Version/s: (was: 2.2.1) 2.3.0 > Executors not being rel

[jira] [Comment Edited] (SPARK-14220) Build and test Spark against Scala 2.12

[ https://issues.apache.org/jira/browse/SPARK-14220?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16540543#comment-16540543 ] Stavros Kontopoulos edited comment on SPARK-14220 at 7/11/18 7:16 PM:

[jira] [Comment Edited] (SPARK-14220) Build and test Spark against Scala 2.12

[ https://issues.apache.org/jira/browse/SPARK-14220?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16540543#comment-16540543 ] Stavros Kontopoulos edited comment on SPARK-14220 at 7/11/18 7:14 PM:

[jira] [Commented] (SPARK-14220) Build and test Spark against Scala 2.12

[ https://issues.apache.org/jira/browse/SPARK-14220?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16540543#comment-16540543 ] Stavros Kontopoulos commented on SPARK-14220: - [~easel] I (resumed) working

[jira] [Commented] (SPARK-24754) Minhash integer overflow

[ https://issues.apache.org/jira/browse/SPARK-24754?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16540539#comment-16540539 ] Apache Spark commented on SPARK-24754: -- User 'srowen' has created a pull request fo

[jira] [Assigned] (SPARK-24754) Minhash integer overflow

[ https://issues.apache.org/jira/browse/SPARK-24754?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24754: Assignee: (was: Apache Spark) > Minhash integer overflow > >

[jira] [Assigned] (SPARK-24754) Minhash integer overflow

[ https://issues.apache.org/jira/browse/SPARK-24754?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24754: Assignee: Apache Spark > Minhash integer overflow > > >

[jira] [Resolved] (SPARK-23254) Add user guide entry for DataFrame multivariate summary

[ https://issues.apache.org/jira/browse/SPARK-23254?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-23254. --- Resolution: Fixed Fix Version/s: 2.4.0 Issue resolved by pull request 20446 [https://github.c

[jira] [Assigned] (SPARK-23254) Add user guide entry for DataFrame multivariate summary

[ https://issues.apache.org/jira/browse/SPARK-23254?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen reassigned SPARK-23254: - Assignee: Weichen Xu > Add user guide entry for DataFrame multivariate summary > --

[jira] [Resolved] (SPARK-24470) RestSubmissionClient to be robust against 404 & non json responses

[ https://issues.apache.org/jira/browse/SPARK-24470?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-24470. --- Resolution: Fixed Fix Version/s: 2.4.0 Issue resolved by pull request 21684 [https://github.c

[jira] [Assigned] (SPARK-24470) RestSubmissionClient to be robust against 404 & non json responses

[ https://issues.apache.org/jira/browse/SPARK-24470?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen reassigned SPARK-24470: - Assignee: Rekha Joshi > RestSubmissionClient to be robust against 404 & non json responses > --

[jira] [Commented] (SPARK-14220) Build and test Spark against Scala 2.12

[ https://issues.apache.org/jira/browse/SPARK-14220?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16540509#comment-16540509 ] Anthony Cros commented on SPARK-14220: -- I also recall having to cheat a bit and bas

[jira] [Commented] (SPARK-14220) Build and test Spark against Scala 2.12

[ https://issues.apache.org/jira/browse/SPARK-14220?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16540495#comment-16540495 ] Erik LaBianca commented on SPARK-14220: --- Using `dev/change-scala-version 2.12` and

[jira] [Commented] (SPARK-14220) Build and test Spark against Scala 2.12

[ https://issues.apache.org/jira/browse/SPARK-14220?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16540491#comment-16540491 ] Anthony Cros commented on SPARK-14220: -- I used `build/sbt -Pscala-2.12 assembly` th

[jira] [Comment Edited] (SPARK-20202) Remove references to org.spark-project.hive

[ https://issues.apache.org/jira/browse/SPARK-20202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16540483#comment-16540483 ] Hyukjin Kwon edited comment on SPARK-20202 at 7/11/18 6:24 PM: ---

[jira] [Created] (SPARK-24786) Executors not being released after all cached data is unpersisted

Jeffrey Charles created SPARK-24786: --- Summary: Executors not being released after all cached data is unpersisted Key: SPARK-24786 URL: https://issues.apache.org/jira/browse/SPARK-24786 Project: Spar

[jira] [Assigned] (SPARK-24785) Making sure REPL prints Spark UI info and then Welcome message

[ https://issues.apache.org/jira/browse/SPARK-24785?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24785: Assignee: Apache Spark > Making sure REPL prints Spark UI info and then Welcome message >

[jira] [Commented] (SPARK-20202) Remove references to org.spark-project.hive

[ https://issues.apache.org/jira/browse/SPARK-20202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16540483#comment-16540483 ] Hyukjin Kwon commented on SPARK-20202: -- [~rxin], there was an initial try above alr

[jira] [Commented] (SPARK-24785) Making sure REPL prints Spark UI info and then Welcome message

[ https://issues.apache.org/jira/browse/SPARK-24785?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16540482#comment-16540482 ] Apache Spark commented on SPARK-24785: -- User 'dbtsai' has created a pull request fo

[jira] [Assigned] (SPARK-24785) Making sure REPL prints Spark UI info and then Welcome message

[ https://issues.apache.org/jira/browse/SPARK-24785?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24785: Assignee: (was: Apache Spark) > Making sure REPL prints Spark UI info and then Welcom

[jira] [Comment Edited] (SPARK-14220) Build and test Spark against Scala 2.12

[ https://issues.apache.org/jira/browse/SPARK-14220?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16540478#comment-16540478 ] Erik LaBianca edited comment on SPARK-14220 at 7/11/18 6:07 PM: --

[jira] [Commented] (SPARK-14220) Build and test Spark against Scala 2.12

[ https://issues.apache.org/jira/browse/SPARK-14220?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16540478#comment-16540478 ] Erik LaBianca commented on SPARK-14220: --- Sorry for the newbie question, but I'd li

[jira] [Created] (SPARK-24785) Making sure REPL prints Spark UI info and then Welcome message

DB Tsai created SPARK-24785: --- Summary: Making sure REPL prints Spark UI info and then Welcome message Key: SPARK-24785 URL: https://issues.apache.org/jira/browse/SPARK-24785 Project: Spark Issue T

[jira] [Commented] (SPARK-23146) Support client mode for Kubernetes cluster backend

[ https://issues.apache.org/jira/browse/SPARK-23146?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16540470#comment-16540470 ] Apache Spark commented on SPARK-23146: -- User 'mccheah' has created a pull request f

[jira] [Commented] (SPARK-20202) Remove references to org.spark-project.hive

[ https://issues.apache.org/jira/browse/SPARK-20202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16540468#comment-16540468 ] Reynold Xin commented on SPARK-20202: - Yea you can try and see how difficult it is.

[jira] [Commented] (SPARK-20202) Remove references to org.spark-project.hive

[ https://issues.apache.org/jira/browse/SPARK-20202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16540467#comment-16540467 ] Hyukjin Kwon commented on SPARK-20202: -- I was thinking we target it for 3.0.0 (othe

[jira] [Commented] (SPARK-20202) Remove references to org.spark-project.hive

[ https://issues.apache.org/jira/browse/SPARK-20202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16540454#comment-16540454 ] Reynold Xin commented on SPARK-20202: - If you want to try and put together a PR that

[jira] [Resolved] (SPARK-24562) Allow running same tests with multiple configs in SQLQueryTestSuite

[ https://issues.apache.org/jira/browse/SPARK-24562?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-24562. - Resolution: Fixed Fix Version/s: 2.4.0 Issue resolved by pull request 21568 [https://gith

[jira] [Assigned] (SPARK-24562) Allow running same tests with multiple configs in SQLQueryTestSuite

[ https://issues.apache.org/jira/browse/SPARK-24562?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-24562: --- Assignee: Marco Gaido > Allow running same tests with multiple configs in SQLQueryTestSuite

[jira] [Commented] (SPARK-24432) Add support for dynamic resource allocation

[ https://issues.apache.org/jira/browse/SPARK-24432?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16540252#comment-16540252 ] Yinan Li commented on SPARK-24432: -- No one is working on this right now, but I think fo

[jira] [Commented] (SPARK-21097) Dynamic allocation will preserve cached data

[ https://issues.apache.org/jira/browse/SPARK-21097?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16540195#comment-16540195 ] Brad commented on SPARK-21097: -- Hi [~jothor], Thanks for the interest, but to be honest I

[jira] [Comment Edited] (SPARK-17901) NettyRpcEndpointRef: Error sending message and Caused by: java.util.ConcurrentModificationException

[ https://issues.apache.org/jira/browse/SPARK-17901?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16540155#comment-16540155 ] Harel Ben Attia edited comment on SPARK-17901 at 7/11/18 2:36 PM:

[jira] [Commented] (SPARK-24432) Add support for dynamic resource allocation

[ https://issues.apache.org/jira/browse/SPARK-24432?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16540159#comment-16540159 ] Stavros Kontopoulos commented on SPARK-24432: - [~liyinan926] is anyone worki

[jira] [Commented] (SPARK-17901) NettyRpcEndpointRef: Error sending message and Caused by: java.util.ConcurrentModificationException

[ https://issues.apache.org/jira/browse/SPARK-17901?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16540155#comment-16540155 ] Harel Ben Attia commented on SPARK-17901: - We're encountering the same issue, wi

[jira] [Commented] (SPARK-19680) Offsets out of range with no configured reset policy for partitions

[ https://issues.apache.org/jira/browse/SPARK-19680?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16540138#comment-16540138 ] Cody Koeninger commented on SPARK-19680: A new consumer group is the easiest thi

[jira] [Created] (SPARK-24784) Retraining (each document as separate file) creates OOME

Jatin Puri created SPARK-24784: -- Summary: Retraining (each document as separate file) creates OOME Key: SPARK-24784 URL: https://issues.apache.org/jira/browse/SPARK-24784 Project: Spark Issue Ty

[jira] [Commented] (SPARK-19680) Offsets out of range with no configured reset policy for partitions

[ https://issues.apache.org/jira/browse/SPARK-19680?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16539906#comment-16539906 ] Biplob Biswas commented on SPARK-19680: --- This essentially poisons the Consumer Gro

[jira] [Commented] (SPARK-24165) UDF within when().otherwise() raises NullPointerException

[ https://issues.apache.org/jira/browse/SPARK-24165?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16539894#comment-16539894 ] Apache Spark commented on SPARK-24165: -- User 'mn-mikke' has created a pull request

[jira] [Commented] (SPARK-24666) Word2Vec generate infinity vectors when numIterations are large

[ https://issues.apache.org/jira/browse/SPARK-24666?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16539865#comment-16539865 ] ZhongYu commented on SPARK-24666: - It is very easy to reproduce. We have about 600,000 w

[jira] [Created] (SPARK-24783) spark.sql.shuffle.partitions=0 should throw exception

Avi minsky created SPARK-24783: -- Summary: spark.sql.shuffle.partitions=0 should throw exception Key: SPARK-24783 URL: https://issues.apache.org/jira/browse/SPARK-24783 Project: Spark Issue Type:

[jira] [Commented] (SPARK-24699) Watermark / Append mode should work with Trigger.Once

[ https://issues.apache.org/jira/browse/SPARK-24699?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16539784#comment-16539784 ] Apache Spark commented on SPARK-24699: -- User 'tdas' has created a pull request for

[jira] [Commented] (SPARK-24781) Using a reference from Dataset in Filter/Sort might not work.

[ https://issues.apache.org/jira/browse/SPARK-24781?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16539757#comment-16539757 ] Apache Spark commented on SPARK-24781: -- User 'viirya' has created a pull request fo

[jira] [Assigned] (SPARK-24781) Using a reference from Dataset in Filter/Sort might not work.

[ https://issues.apache.org/jira/browse/SPARK-24781?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24781: Assignee: (was: Apache Spark) > Using a reference from Dataset in Filter/Sort might n

[jira] [Commented] (SPARK-23125) Offset commit failed when spark-streaming batch time is more than kafkaParams session timeout.

[ https://issues.apache.org/jira/browse/SPARK-23125?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16539758#comment-16539758 ] Tim Van Wassenhove commented on SPARK-23125: We were also impacted by this.

[jira] [Assigned] (SPARK-24781) Using a reference from Dataset in Filter/Sort might not work.

[ https://issues.apache.org/jira/browse/SPARK-24781?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24781: Assignee: Apache Spark > Using a reference from Dataset in Filter/Sort might not work. >

[jira] [Commented] (SPARK-24782) Simplify conf access in expressions

[ https://issues.apache.org/jira/browse/SPARK-24782?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16539755#comment-16539755 ] Apache Spark commented on SPARK-24782: -- User 'mgaido91' has created a pull request

[jira] [Assigned] (SPARK-24782) Simplify conf access in expressions

[ https://issues.apache.org/jira/browse/SPARK-24782?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24782: Assignee: (was: Apache Spark) > Simplify conf access in expressions > ---

[jira] [Assigned] (SPARK-24782) Simplify conf access in expressions

[ https://issues.apache.org/jira/browse/SPARK-24782?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24782: Assignee: Apache Spark > Simplify conf access in expressions > --

[jira] [Created] (SPARK-24782) Simplify conf access in expressions

Marco Gaido created SPARK-24782: --- Summary: Simplify conf access in expressions Key: SPARK-24782 URL: https://issues.apache.org/jira/browse/SPARK-24782 Project: Spark Issue Type: Improvement

[jira] [Resolved] (SPARK-24370) spark checkpoint creates many 0 byte empty files(partitions) in checkpoint directory

[ https://issues.apache.org/jira/browse/SPARK-24370?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-24370. -- Resolution: Cannot Reproduce It would be nicer if you able to test this in an upper version or

[jira] [Resolved] (SPARK-24409) exception when sending large list in filter(col(x).isin(list))

[ https://issues.apache.org/jira/browse/SPARK-24409?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-24409. -- Resolution: Not A Problem > exception when sending large list in filter(col(x).isin(list)) > -