[jira] [Commented] (SPARK-14220) Build and test Spark against Scala 2.12

[ https://issues.apache.org/jira/browse/SPARK-14220?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16539622#comment-16539622 ] Felix Cheung commented on SPARK-14220: -- this shouldn't block 2.3.2 right? > Build and test Spark

[jira] [Updated] (SPARK-24781) Using a reference from Dataset in Filter/Sort might not work.

[ https://issues.apache.org/jira/browse/SPARK-24781?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin updated SPARK-24781: -- Description: When we use a reference from {{Dataset}} in {{filter}} or {{sort}}, which was

[jira] [Updated] (SPARK-24781) Using a reference from Dataset in Filter/Sort might not work.

[ https://issues.apache.org/jira/browse/SPARK-24781?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin updated SPARK-24781: -- Summary: Using a reference from Dataset in Filter/Sort might not work. (was: Using a

[jira] [Commented] (SPARK-24781) Using a reference from Dataset in Filter/Sort might not work.

[ https://issues.apache.org/jira/browse/SPARK-24781?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16539630#comment-16539630 ] Saisai Shao commented on SPARK-24781: - Thanks Felix. Does this have to be in 2.3.2? [~ueshin] >

[jira] [Commented] (SPARK-24781) Using a reference from Dataset in Filter/Sort might not work.

[ https://issues.apache.org/jira/browse/SPARK-24781?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16539633#comment-16539633 ] Takuya Ueshin commented on SPARK-24781: --- Yes, I think so. This is a regression from 2.2 to 2.3. >

[jira] [Commented] (SPARK-23858) Need to apply pyarrow adjustments to complex types with DateType/TimestampType

[ https://issues.apache.org/jira/browse/SPARK-23858?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16539634#comment-16539634 ] Hyukjin Kwon commented on SPARK-23858: -- [~semanticbeeng], create a pandas udf that takes nested

[jira] [Assigned] (SPARK-23461) vignettes should include model predictions for some ML models

[ https://issues.apache.org/jira/browse/SPARK-23461?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung reassigned SPARK-23461: Assignee: Huaxin Gao > vignettes should include model predictions for some ML models >

[jira] [Resolved] (SPARK-23461) vignettes should include model predictions for some ML models

[ https://issues.apache.org/jira/browse/SPARK-23461?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung resolved SPARK-23461. -- Resolution: Fixed Fix Version/s: 2.4.0 > vignettes should include model predictions

[jira] [Updated] (SPARK-23410) Unable to read jsons in charset different from UTF-8

[ https://issues.apache.org/jira/browse/SPARK-23410?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-23410: - Shepherd: Hyukjin Kwon (was: Herman van Hovell) > Unable to read jsons in charset different

[jira] [Resolved] (SPARK-24667) If folders managed by DiskBlockManager are deleted manually, shell throws FileNotFoundException

[ https://issues.apache.org/jira/browse/SPARK-24667?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-24667. -- Resolution: Not A Problem > If folders managed by DiskBlockManager are deleted manually,

[jira] [Commented] (SPARK-24781) Using a reference from Dataset in Filter/Sort might not work.

[ https://issues.apache.org/jira/browse/SPARK-24781?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16539635#comment-16539635 ] Saisai Shao commented on SPARK-24781: - I see. I will wait for this before cutting a new 2.3.2 RC

[jira] [Resolved] (SPARK-24409) exception when sending large list in filter(col(x).isin(list))

[ https://issues.apache.org/jira/browse/SPARK-24409?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-24409. -- Resolution: Not A Problem > exception when sending large list in filter(col(x).isin(list)) >

[jira] [Updated] (SPARK-24644) Pyarrow exception while running pandas_udf on pyspark 2.3.1

[ https://issues.apache.org/jira/browse/SPARK-24644?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-24644: - Component/s: (was: Block Manager) PySpark > Pyarrow exception while

[jira] [Commented] (SPARK-24781) Using a reference from Dataset in Filter/Sort might not work.

[ https://issues.apache.org/jira/browse/SPARK-24781?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16539627#comment-16539627 ] Felix Cheung commented on SPARK-24781: -- [~jerryshao] > Using a reference from Dataset in

[jira] [Commented] (SPARK-24644) Pyarrow exception while running pandas_udf on pyspark 2.3.1

[ https://issues.apache.org/jira/browse/SPARK-24644?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16539626#comment-16539626 ] Hyukjin Kwon commented on SPARK-24644: -- Thanks, [~helkhalfi]. mind if I ask to post the codes you

[jira] [Resolved] (SPARK-24370) spark checkpoint creates many 0 byte empty files(partitions) in checkpoint directory

[ https://issues.apache.org/jira/browse/SPARK-24370?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-24370. -- Resolution: Cannot Reproduce It would be nicer if you able to test this in an upper version

[jira] [Updated] (SPARK-24781) Using a reference from Dataset in Filter/Sort might not work.

[ https://issues.apache.org/jira/browse/SPARK-24781?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin updated SPARK-24781: -- Description: When we use a reference from {{Dataset}} in {{filter}} or {{sort}}, which was

[jira] [Resolved] (SPARK-23529) Specify hostpath volume and mount the volume in Spark driver and executor pods in Kubernetes

[ https://issues.apache.org/jira/browse/SPARK-23529?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung resolved SPARK-23529. -- Resolution: Fixed Assignee: Andrew Korzhuev (was: Anirudh Ramanathan) Fix

[jira] [Commented] (SPARK-24781) Using a reference from Dataset in Filter/Sort might not work.

[ https://issues.apache.org/jira/browse/SPARK-24781?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16539757#comment-16539757 ] Apache Spark commented on SPARK-24781: -- User 'viirya' has created a pull request for this issue:

[jira] [Assigned] (SPARK-24781) Using a reference from Dataset in Filter/Sort might not work.

[ https://issues.apache.org/jira/browse/SPARK-24781?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24781: Assignee: (was: Apache Spark) > Using a reference from Dataset in Filter/Sort might

[jira] [Assigned] (SPARK-24781) Using a reference from Dataset in Filter/Sort might not work.

[ https://issues.apache.org/jira/browse/SPARK-24781?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24781: Assignee: Apache Spark > Using a reference from Dataset in Filter/Sort might not work. >

[jira] [Commented] (SPARK-23125) Offset commit failed when spark-streaming batch time is more than kafkaParams session timeout.

[ https://issues.apache.org/jira/browse/SPARK-23125?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16539758#comment-16539758 ] Tim Van Wassenhove commented on SPARK-23125: We were also impacted by this. Worked around

[jira] [Commented] (SPARK-24699) Watermark / Append mode should work with Trigger.Once

[ https://issues.apache.org/jira/browse/SPARK-24699?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16539784#comment-16539784 ] Apache Spark commented on SPARK-24699: -- User 'tdas' has created a pull request for this issue:

[jira] [Created] (SPARK-24782) Simplify conf access in expressions

Marco Gaido created SPARK-24782: --- Summary: Simplify conf access in expressions Key: SPARK-24782 URL: https://issues.apache.org/jira/browse/SPARK-24782 Project: Spark Issue Type: Improvement

[jira] [Created] (SPARK-24783) spark.sql.shuffle.partitions=0 should throw exception

Avi minsky created SPARK-24783: -- Summary: spark.sql.shuffle.partitions=0 should throw exception Key: SPARK-24783 URL: https://issues.apache.org/jira/browse/SPARK-24783 Project: Spark Issue

[jira] [Commented] (SPARK-24165) UDF within when().otherwise() raises NullPointerException

[ https://issues.apache.org/jira/browse/SPARK-24165?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16539894#comment-16539894 ] Apache Spark commented on SPARK-24165: -- User 'mn-mikke' has created a pull request for this issue:

[jira] [Commented] (SPARK-19680) Offsets out of range with no configured reset policy for partitions

[ https://issues.apache.org/jira/browse/SPARK-19680?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16539906#comment-16539906 ] Biplob Biswas commented on SPARK-19680: --- This essentially poisons the Consumer Group, and the only

[jira] [Assigned] (SPARK-24782) Simplify conf access in expressions

[ https://issues.apache.org/jira/browse/SPARK-24782?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24782: Assignee: (was: Apache Spark) > Simplify conf access in expressions >

[jira] [Assigned] (SPARK-24782) Simplify conf access in expressions

[ https://issues.apache.org/jira/browse/SPARK-24782?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24782: Assignee: Apache Spark > Simplify conf access in expressions >

[jira] [Commented] (SPARK-24782) Simplify conf access in expressions

[ https://issues.apache.org/jira/browse/SPARK-24782?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16539755#comment-16539755 ] Apache Spark commented on SPARK-24782: -- User 'mgaido91' has created a pull request for this issue:

[jira] [Commented] (SPARK-24666) Word2Vec generate infinity vectors when numIterations are large

[ https://issues.apache.org/jira/browse/SPARK-24666?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16539865#comment-16539865 ] ZhongYu commented on SPARK-24666: - It is very easy to reproduce. We have about 600,000 words and

[jira] [Commented] (SPARK-24432) Add support for dynamic resource allocation

[ https://issues.apache.org/jira/browse/SPARK-24432?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16540252#comment-16540252 ] Yinan Li commented on SPARK-24432: -- No one is working on this right now, but I think foxish planned to

[jira] [Commented] (SPARK-17901) NettyRpcEndpointRef: Error sending message and Caused by: java.util.ConcurrentModificationException

[ https://issues.apache.org/jira/browse/SPARK-17901?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16540155#comment-16540155 ] Harel Ben Attia commented on SPARK-17901: - We're encountering the same issue, with spark 

[jira] [Commented] (SPARK-24432) Add support for dynamic resource allocation

[ https://issues.apache.org/jira/browse/SPARK-24432?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16540159#comment-16540159 ] Stavros Kontopoulos commented on SPARK-24432: - [~liyinan926] is anyone working on this? Is

[jira] [Comment Edited] (SPARK-17901) NettyRpcEndpointRef: Error sending message and Caused by: java.util.ConcurrentModificationException

[ https://issues.apache.org/jira/browse/SPARK-17901?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16540155#comment-16540155 ] Harel Ben Attia edited comment on SPARK-17901 at 7/11/18 2:36 PM: --

[jira] [Commented] (SPARK-21097) Dynamic allocation will preserve cached data

[ https://issues.apache.org/jira/browse/SPARK-21097?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16540195#comment-16540195 ] Brad commented on SPARK-21097: -- Hi [~jothor], Thanks for the interest, but to be honest I got a new job

[jira] [Commented] (SPARK-20202) Remove references to org.spark-project.hive

[ https://issues.apache.org/jira/browse/SPARK-20202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16540454#comment-16540454 ] Reynold Xin commented on SPARK-20202: - If you want to try and put together a PR that actually does

[jira] [Commented] (SPARK-20202) Remove references to org.spark-project.hive

[ https://issues.apache.org/jira/browse/SPARK-20202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16540467#comment-16540467 ] Hyukjin Kwon commented on SPARK-20202: -- I was thinking we target it for 3.0.0 (otherwise 4.0.0

[jira] [Commented] (SPARK-20202) Remove references to org.spark-project.hive

[ https://issues.apache.org/jira/browse/SPARK-20202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16540468#comment-16540468 ] Reynold Xin commented on SPARK-20202: - Yea you can try and see how difficult it is.   > Remove

[jira] [Commented] (SPARK-23146) Support client mode for Kubernetes cluster backend

[ https://issues.apache.org/jira/browse/SPARK-23146?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16540470#comment-16540470 ] Apache Spark commented on SPARK-23146: -- User 'mccheah' has created a pull request for this issue:

[jira] [Created] (SPARK-24785) Making sure REPL prints Spark UI info and then Welcome message

DB Tsai created SPARK-24785: --- Summary: Making sure REPL prints Spark UI info and then Welcome message Key: SPARK-24785 URL: https://issues.apache.org/jira/browse/SPARK-24785 Project: Spark Issue

[jira] [Commented] (SPARK-14220) Build and test Spark against Scala 2.12

[ https://issues.apache.org/jira/browse/SPARK-14220?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16540478#comment-16540478 ] Erik LaBianca commented on SPARK-14220: --- Sorry for the newbie question, but I'd like to try a full

[jira] [Comment Edited] (SPARK-14220) Build and test Spark against Scala 2.12

[ https://issues.apache.org/jira/browse/SPARK-14220?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16540478#comment-16540478 ] Erik LaBianca edited comment on SPARK-14220 at 7/11/18 6:07 PM: Sorry

[jira] [Assigned] (SPARK-24785) Making sure REPL prints Spark UI info and then Welcome message

[ https://issues.apache.org/jira/browse/SPARK-24785?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24785: Assignee: Apache Spark > Making sure REPL prints Spark UI info and then Welcome message

[jira] [Commented] (SPARK-20202) Remove references to org.spark-project.hive

[ https://issues.apache.org/jira/browse/SPARK-20202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16540483#comment-16540483 ] Hyukjin Kwon commented on SPARK-20202: -- [~rxin], there was an initial try above already though

[jira] [Commented] (SPARK-24785) Making sure REPL prints Spark UI info and then Welcome message

[ https://issues.apache.org/jira/browse/SPARK-24785?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16540482#comment-16540482 ] Apache Spark commented on SPARK-24785: -- User 'dbtsai' has created a pull request for this issue:

[jira] [Assigned] (SPARK-24785) Making sure REPL prints Spark UI info and then Welcome message

[ https://issues.apache.org/jira/browse/SPARK-24785?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24785: Assignee: (was: Apache Spark) > Making sure REPL prints Spark UI info and then

[jira] [Created] (SPARK-24786) Executors not being released after all cached data is unpersisted

Jeffrey Charles created SPARK-24786: --- Summary: Executors not being released after all cached data is unpersisted Key: SPARK-24786 URL: https://issues.apache.org/jira/browse/SPARK-24786 Project:

[jira] [Comment Edited] (SPARK-20202) Remove references to org.spark-project.hive

[ https://issues.apache.org/jira/browse/SPARK-20202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16540483#comment-16540483 ] Hyukjin Kwon edited comment on SPARK-20202 at 7/11/18 6:24 PM: --- [~rxin],

[jira] [Commented] (SPARK-14220) Build and test Spark against Scala 2.12

[ https://issues.apache.org/jira/browse/SPARK-14220?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16540491#comment-16540491 ] Anthony Cros commented on SPARK-14220: -- I used `build/sbt -Pscala-2.12 assembly` though I still had

[jira] [Commented] (SPARK-14220) Build and test Spark against Scala 2.12

[ https://issues.apache.org/jira/browse/SPARK-14220?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16540495#comment-16540495 ] Erik LaBianca commented on SPARK-14220: --- Using `dev/change-scala-version 2.12` and then `build/mvn

[jira] [Commented] (SPARK-14220) Build and test Spark against Scala 2.12

[ https://issues.apache.org/jira/browse/SPARK-14220?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16540509#comment-16540509 ] Anthony Cros commented on SPARK-14220: -- I also recall having to cheat a bit and basically add

[jira] [Resolved] (SPARK-24470) RestSubmissionClient to be robust against 404 & non json responses

[ https://issues.apache.org/jira/browse/SPARK-24470?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-24470. --- Resolution: Fixed Fix Version/s: 2.4.0 Issue resolved by pull request 21684

[jira] [Assigned] (SPARK-24470) RestSubmissionClient to be robust against 404 & non json responses

[ https://issues.apache.org/jira/browse/SPARK-24470?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen reassigned SPARK-24470: - Assignee: Rekha Joshi > RestSubmissionClient to be robust against 404 & non json responses >

[jira] [Assigned] (SPARK-23254) Add user guide entry for DataFrame multivariate summary

[ https://issues.apache.org/jira/browse/SPARK-23254?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen reassigned SPARK-23254: - Assignee: Weichen Xu > Add user guide entry for DataFrame multivariate summary >

[jira] [Resolved] (SPARK-23254) Add user guide entry for DataFrame multivariate summary

[ https://issues.apache.org/jira/browse/SPARK-23254?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-23254. --- Resolution: Fixed Fix Version/s: 2.4.0 Issue resolved by pull request 20446

[jira] [Commented] (SPARK-24754) Minhash integer overflow

[ https://issues.apache.org/jira/browse/SPARK-24754?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16540539#comment-16540539 ] Apache Spark commented on SPARK-24754: -- User 'srowen' has created a pull request for this issue:

[jira] [Commented] (SPARK-14220) Build and test Spark against Scala 2.12

[ https://issues.apache.org/jira/browse/SPARK-14220?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16540543#comment-16540543 ] Stavros Kontopoulos commented on SPARK-14220: - [~easel] I (resumed) working on scala 2.12

[jira] [Assigned] (SPARK-24754) Minhash integer overflow

[ https://issues.apache.org/jira/browse/SPARK-24754?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24754: Assignee: Apache Spark > Minhash integer overflow > > >

[jira] [Assigned] (SPARK-24754) Minhash integer overflow

[ https://issues.apache.org/jira/browse/SPARK-24754?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24754: Assignee: (was: Apache Spark) > Minhash integer overflow >

[jira] [Comment Edited] (SPARK-14220) Build and test Spark against Scala 2.12

[ https://issues.apache.org/jira/browse/SPARK-14220?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16540543#comment-16540543 ] Stavros Kontopoulos edited comment on SPARK-14220 at 7/11/18 7:14 PM:

[jira] [Comment Edited] (SPARK-14220) Build and test Spark against Scala 2.12

[ https://issues.apache.org/jira/browse/SPARK-14220?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16540543#comment-16540543 ] Stavros Kontopoulos edited comment on SPARK-14220 at 7/11/18 7:16 PM:

[jira] [Commented] (SPARK-19680) Offsets out of range with no configured reset policy for partitions

[ https://issues.apache.org/jira/browse/SPARK-19680?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16540138#comment-16540138 ] Cody Koeninger commented on SPARK-19680: A new consumer group is the easiest thing to do, but if

[jira] [Created] (SPARK-24784) Retraining (each document as separate file) creates OOME

Jatin Puri created SPARK-24784: -- Summary: Retraining (each document as separate file) creates OOME Key: SPARK-24784 URL: https://issues.apache.org/jira/browse/SPARK-24784 Project: Spark Issue

[jira] [Resolved] (SPARK-24562) Allow running same tests with multiple configs in SQLQueryTestSuite

[ https://issues.apache.org/jira/browse/SPARK-24562?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-24562. - Resolution: Fixed Fix Version/s: 2.4.0 Issue resolved by pull request 21568

[jira] [Assigned] (SPARK-24562) Allow running same tests with multiple configs in SQLQueryTestSuite

[ https://issues.apache.org/jira/browse/SPARK-24562?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-24562: --- Assignee: Marco Gaido > Allow running same tests with multiple configs in

[jira] [Updated] (SPARK-24786) Executors not being released after all cached data is unpersisted

[ https://issues.apache.org/jira/browse/SPARK-24786?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jeffrey Charles updated SPARK-24786: Affects Version/s: (was: 2.2.1) 2.3.0 > Executors not being

[jira] [Resolved] (SPARK-24697) Fix the reported start offsets in streaming query progress

[ https://issues.apache.org/jira/browse/SPARK-24697?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tathagata Das resolved SPARK-24697. --- Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 21744

[jira] [Assigned] (SPARK-24697) Fix the reported start offsets in streaming query progress

[ https://issues.apache.org/jira/browse/SPARK-24697?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tathagata Das reassigned SPARK-24697: - Assignee: Tathagata Das > Fix the reported start offsets in streaming query progress >

[jira] [Commented] (SPARK-24208) Cannot resolve column in self join after applying Pandas UDF

[ https://issues.apache.org/jira/browse/SPARK-24208?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16540612#comment-16540612 ] Apache Spark commented on SPARK-24208: -- User 'mgaido91' has created a pull request for this issue:

[jira] [Commented] (SPARK-20202) Remove references to org.spark-project.hive

[ https://issues.apache.org/jira/browse/SPARK-20202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16541034#comment-16541034 ] Hyukjin Kwon commented on SPARK-20202: -- I am asking this to set the goal for this JIRA as of the

[jira] [Created] (SPARK-24792) Add API `.avro` in DataFrameReader/DataFrameWriter

Gengliang Wang created SPARK-24792: -- Summary: Add API `.avro` in DataFrameReader/DataFrameWriter Key: SPARK-24792 URL: https://issues.apache.org/jira/browse/SPARK-24792 Project: Spark Issue

[jira] [Created] (SPARK-24791) Spark Structured Streaming randomly does not process batch

Arvind Ramachandran created SPARK-24791: --- Summary: Spark Structured Streaming randomly does not process batch Key: SPARK-24791 URL: https://issues.apache.org/jira/browse/SPARK-24791 Project:

[jira] [Assigned] (SPARK-24788) RelationalGroupedDataset.toString throws errors when grouping by UnresolvedAttribute

[ https://issues.apache.org/jira/browse/SPARK-24788?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24788: Assignee: Apache Spark > RelationalGroupedDataset.toString throws errors when grouping

[jira] [Assigned] (SPARK-24788) RelationalGroupedDataset.toString throws errors when grouping by UnresolvedAttribute

[ https://issues.apache.org/jira/browse/SPARK-24788?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24788: Assignee: (was: Apache Spark) > RelationalGroupedDataset.toString throws errors when

[jira] [Commented] (SPARK-24788) RelationalGroupedDataset.toString throws errors when grouping by UnresolvedAttribute

[ https://issues.apache.org/jira/browse/SPARK-24788?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16540699#comment-16540699 ] Apache Spark commented on SPARK-24788: -- User 'c-horn' has created a pull request for this issue:

[jira] [Resolved] (SPARK-24529) Add spotbugs into maven build process

[ https://issues.apache.org/jira/browse/SPARK-24529?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-24529. -- Resolution: Fixed Fix Version/s: 2.4.0 Issue resolved by pull request 21542

[jira] [Assigned] (SPARK-24529) Add spotbugs into maven build process

[ https://issues.apache.org/jira/browse/SPARK-24529?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-24529: Assignee: Kazuaki Ishizaki > Add spotbugs into maven build process >

[jira] [Commented] (SPARK-24705) Spark.sql.adaptive.enabled=true is enabled and self-join query

[ https://issues.apache.org/jira/browse/SPARK-24705?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16541063#comment-16541063 ] Apache Spark commented on SPARK-24705: -- User 'maropu' has created a pull request for this issue:

[jira] [Commented] (SPARK-24705) Spark.sql.adaptive.enabled=true is enabled and self-join query

[ https://issues.apache.org/jira/browse/SPARK-24705?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16541065#comment-16541065 ] Takeshi Yamamuro commented on SPARK-24705: -- Sorry for my late response. IIUC I misunderstood

[jira] [Assigned] (SPARK-24705) Spark.sql.adaptive.enabled=true is enabled and self-join query

[ https://issues.apache.org/jira/browse/SPARK-24705?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24705: Assignee: (was: Apache Spark) > Spark.sql.adaptive.enabled=true is enabled and

[jira] [Assigned] (SPARK-24705) Spark.sql.adaptive.enabled=true is enabled and self-join query

[ https://issues.apache.org/jira/browse/SPARK-24705?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24705: Assignee: Apache Spark > Spark.sql.adaptive.enabled=true is enabled and self-join query

[jira] [Commented] (SPARK-24432) Add support for dynamic resource allocation

[ https://issues.apache.org/jira/browse/SPARK-24432?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16541024#comment-16541024 ] Mark Hamilton commented on SPARK-24432: --- Hey [~liyinan926], and [~foxish], I would like to try my

[jira] [Created] (SPARK-24790) Allow complex aggregate expressions in Pivot

Maryann Xue created SPARK-24790: --- Summary: Allow complex aggregate expressions in Pivot Key: SPARK-24790 URL: https://issues.apache.org/jira/browse/SPARK-24790 Project: Spark Issue Type:

[jira] [Updated] (SPARK-24790) Allow complex aggregate expressions in Pivot

[ https://issues.apache.org/jira/browse/SPARK-24790?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maryann Xue updated SPARK-24790: Description: In SPARK-24035, to conform with Oracle PIVOT syntax, a strict check of PIVOT's

[jira] [Assigned] (SPARK-24790) Allow complex aggregate expressions in Pivot

[ https://issues.apache.org/jira/browse/SPARK-24790?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24790: Assignee: Apache Spark > Allow complex aggregate expressions in Pivot >

[jira] [Commented] (SPARK-24787) Events being dropped at an alarming rate due to hsync being slow for eventLogging

[ https://issues.apache.org/jira/browse/SPARK-24787?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16540638#comment-16540638 ] Sanket Reddy commented on SPARK-24787: -- I am happy to work on this... will have a potential

[jira] [Commented] (SPARK-24790) Allow complex aggregate expressions in Pivot

[ https://issues.apache.org/jira/browse/SPARK-24790?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16540809#comment-16540809 ] Apache Spark commented on SPARK-24790: -- User 'maryannxue' has created a pull request for this

[jira] [Assigned] (SPARK-24790) Allow complex aggregate expressions in Pivot

[ https://issues.apache.org/jira/browse/SPARK-24790?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24790: Assignee: (was: Apache Spark) > Allow complex aggregate expressions in Pivot >

[jira] [Resolved] (SPARK-24782) Simplify conf access in expressions

[ https://issues.apache.org/jira/browse/SPARK-24782?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-24782. - Resolution: Fixed Assignee: Marco Gaido Fix Version/s: 2.4.0 > Simplify conf access in

[jira] [Resolved] (SPARK-24761) Check modifiability of config parameters

[ https://issues.apache.org/jira/browse/SPARK-24761?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-24761. - Resolution: Fixed Assignee: Maxim Gekk Fix Version/s: 2.4.0 > Check modifiability of

[jira] [Commented] (SPARK-14220) Build and test Spark against Scala 2.12

[ https://issues.apache.org/jira/browse/SPARK-14220?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16540676#comment-16540676 ] Stavros Kontopoulos commented on SPARK-14220: - If you make it more specific it works:

[jira] [Comment Edited] (SPARK-14220) Build and test Spark against Scala 2.12

[ https://issues.apache.org/jira/browse/SPARK-14220?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16540676#comment-16540676 ] Stavros Kontopoulos edited comment on SPARK-14220 at 7/11/18 9:17 PM:

[jira] [Assigned] (SPARK-24764) Add ServiceLoader implementation for SparkHadoopUtil

[ https://issues.apache.org/jira/browse/SPARK-24764?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] DB Tsai reassigned SPARK-24764: --- Assignee: Shruti Gumma > Add ServiceLoader implementation for SparkHadoopUtil >

[jira] [Created] (SPARK-24787) Events being dropped at an alarming rate due to hsync being slow for eventLogging

Sanket Reddy created SPARK-24787: Summary: Events being dropped at an alarming rate due to hsync being slow for eventLogging Key: SPARK-24787 URL: https://issues.apache.org/jira/browse/SPARK-24787

[jira] [Created] (SPARK-24788) RelationalGroupedDataset.toString throws errors when grouping by UnresolvedAttribute

Chris Horn created SPARK-24788: -- Summary: RelationalGroupedDataset.toString throws errors when grouping by UnresolvedAttribute Key: SPARK-24788 URL: https://issues.apache.org/jira/browse/SPARK-24788

[jira] [Created] (SPARK-24789) SparkThriftServer - support for hive.server2.authentication.ldap.userFilter

t oo created SPARK-24789: Summary: SparkThriftServer - support for hive.server2.authentication.ldap.userFilter Key: SPARK-24789 URL: https://issues.apache.org/jira/browse/SPARK-24789 Project: Spark

[jira] [Assigned] (SPARK-24724) Discuss necessary info and access in barrier mode + Kubernetes

[ https://issues.apache.org/jira/browse/SPARK-24724?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li reassigned SPARK-24724: --- Assignee: Yinan Li (was: Xiangrui Meng) > Discuss necessary info and access in barrier mode +

[jira] [Assigned] (SPARK-24724) Discuss necessary info and access in barrier mode + Kubernetes

[ https://issues.apache.org/jira/browse/SPARK-24724?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li reassigned SPARK-24724: --- Assignee: Xiangrui Meng > Discuss necessary info and access in barrier mode + Kubernetes >