[jira] [Assigned] (SPARK-34471) Document DataStreamReader/Writer table APIs in Structured Streaming Programming Guide

2021-02-19 Thread Jungtaek Lim (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34471?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim reassigned SPARK-34471: Assignee: Bo Zhang > Document DataStreamReader/Writer table APIs in Structured Streaming

[jira] [Resolved] (SPARK-34471) Document DataStreamReader/Writer table APIs in Structured Streaming Programming Guide

2021-02-19 Thread Jungtaek Lim (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34471?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim resolved SPARK-34471. -- Fix Version/s: 3.1.1 Resolution: Fixed Issue resolved by pull request 31590

[jira] [Commented] (SPARK-33602) Group exception messages in execution/datasources

2021-02-19 Thread jiaan.geng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33602?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17287485#comment-17287485 ] jiaan.geng commented on SPARK-33602: I'm working on. > Group exception messages in

[jira] [Issue Comment Deleted] (SPARK-33542) Group exception messages in catalyst/catalog

2021-02-19 Thread jiaan.geng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33542?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jiaan.geng updated SPARK-33542: --- Comment: was deleted (was: I'm working on.) > Group exception messages in catalyst/catalog >

[jira] [Issue Comment Deleted] (SPARK-33599) Group exception messages in catalyst/analysis

2021-02-19 Thread jiaan.geng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33599?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jiaan.geng updated SPARK-33599: --- Comment: was deleted (was: I'm working on.) > Group exception messages in catalyst/analysis >

[jira] [Created] (SPARK-34480) Module launcher build failed with profile hadoop-3.2 activated

2021-02-19 Thread Lichuanliang (Jira)
Lichuanliang created SPARK-34480: Summary: Module launcher build failed with profile hadoop-3.2 activated Key: SPARK-34480 URL: https://issues.apache.org/jira/browse/SPARK-34480 Project: Spark

[jira] [Commented] (SPARK-34478) Ignore or reject wrong config when start sparksession

2021-02-19 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34478?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17287479#comment-17287479 ] Apache Spark commented on SPARK-34478: -- User 'AngersZh' has created a pull request for this

[jira] [Assigned] (SPARK-34478) Ignore or reject wrong config when start sparksession

2021-02-19 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34478?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34478: Assignee: (was: Apache Spark) > Ignore or reject wrong config when start

[jira] [Commented] (SPARK-33602) Group exception messages in execution/datasources

2021-02-19 Thread jiaan.geng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33602?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17287492#comment-17287492 ] jiaan.geng commented on SPARK-33602: ping [~allisonwang-db] Should we put AnalysisException in

[jira] [Created] (SPARK-34481) Refactor dataframe reader/writer path option logic

2021-02-19 Thread Yuchen Huo (Jira)
Yuchen Huo created SPARK-34481: -- Summary: Refactor dataframe reader/writer path option logic Key: SPARK-34481 URL: https://issues.apache.org/jira/browse/SPARK-34481 Project: Spark Issue Type:

[jira] [Commented] (SPARK-34481) Refactor dataframe reader/writer path option logic

2021-02-19 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34481?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17287583#comment-17287583 ] Apache Spark commented on SPARK-34481: -- User 'yuchenhuo' has created a pull request for this issue:

[jira] [Assigned] (SPARK-34481) Refactor dataframe reader/writer path option logic

2021-02-19 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34481?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34481: Assignee: Apache Spark > Refactor dataframe reader/writer path option logic >

[jira] [Assigned] (SPARK-34481) Refactor dataframe reader/writer path option logic

2021-02-19 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34481?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34481: Assignee: (was: Apache Spark) > Refactor dataframe reader/writer path option logic >

[jira] [Resolved] (SPARK-34449) Upgrade Jetty to fix CVE-2020-27218

2021-02-19 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34449?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-34449. -- Fix Version/s: 3.1.2 3.0.3 2.4.8 Resolution:

[jira] [Issue Comment Deleted] (SPARK-33541) Group exception messages in catalyst/expressions

2021-02-19 Thread jiaan.geng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33541?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jiaan.geng updated SPARK-33541: --- Comment: was deleted (was: I'm working on.) > Group exception messages in catalyst/expressions >

[jira] [Issue Comment Deleted] (SPARK-33601) Group exception messages in catalyst/parser

2021-02-19 Thread jiaan.geng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33601?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jiaan.geng updated SPARK-33601: --- Comment: was deleted (was: I'm working on.) > Group exception messages in catalyst/parser >

[jira] [Commented] (SPARK-10816) EventTime based sessionization (session window)

2021-02-19 Thread Yuanjian Li (Jira)
[ https://issues.apache.org/jira/browse/SPARK-10816?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17287585#comment-17287585 ] Yuanjian Li commented on SPARK-10816: - Great thanks for your heads up! [~viirya] [~kabhwan]

[jira] [Assigned] (SPARK-34478) Ignore or reject wrong config when start sparksession

2021-02-19 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34478?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34478: Assignee: Apache Spark > Ignore or reject wrong config when start sparksession >

[jira] [Closed] (SPARK-34480) Module launcher build failed with profile hadoop-3.2 activated

2021-02-19 Thread Lichuanliang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34480?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lichuanliang closed SPARK-34480. Please ignore this issue > Module launcher build failed with profile hadoop-3.2 activated >

[jira] [Resolved] (SPARK-34480) Module launcher build failed with profile hadoop-3.2 activated

2021-02-19 Thread Lichuanliang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34480?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lichuanliang resolved SPARK-34480. -- Resolution: Fixed > Module launcher build failed with profile hadoop-3.2 activated >

[jira] [Commented] (SPARK-34478) Ignore or reject wrong config when start sparksession

2021-02-19 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34478?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17287478#comment-17287478 ] Apache Spark commented on SPARK-34478: -- User 'AngersZh' has created a pull request for this

[jira] [Updated] (SPARK-34481) Refactor dataframe reader/writer path option logic

2021-02-19 Thread Yuchen Huo (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34481?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuchen Huo updated SPARK-34481: --- Priority: Trivial (was: Major) > Refactor dataframe reader/writer path option logic >

[jira] [Created] (SPARK-34477) Kryo NPEs when serializing Avro GenericData objects (except GenericRecord)

2021-02-19 Thread Shardul Mahadik (Jira)
Shardul Mahadik created SPARK-34477: --- Summary: Kryo NPEs when serializing Avro GenericData objects (except GenericRecord) Key: SPARK-34477 URL: https://issues.apache.org/jira/browse/SPARK-34477

[jira] [Commented] (SPARK-34478) Ignore or reject wrong config when start sparksession

2021-02-19 Thread angerszhu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34478?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17287459#comment-17287459 ] angerszhu commented on SPARK-34478: --- Raise a pr soon > Ignore or reject wrong config when start

[jira] [Created] (SPARK-34478) Ignore or reject wrong config when start sparksession

2021-02-19 Thread angerszhu (Jira)
angerszhu created SPARK-34478: - Summary: Ignore or reject wrong config when start sparksession Key: SPARK-34478 URL: https://issues.apache.org/jira/browse/SPARK-34478 Project: Spark Issue Type:

[jira] [Created] (SPARK-34479) Add zstandard codec to spark.sql.avro.compression.codec

2021-02-19 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-34479: --- Summary: Add zstandard codec to spark.sql.avro.compression.codec Key: SPARK-34479 URL: https://issues.apache.org/jira/browse/SPARK-34479 Project: Spark Issue

[jira] [Updated] (SPARK-34479) Add zstandard codec to spark.sql.avro.compression.codec

2021-02-19 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34479?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-34479: Description: Avro add AVRO-2195 > Add zstandard codec to spark.sql.avro.compression.codec >

[jira] [Updated] (SPARK-34479) Add zstandard codec to spark.sql.avro.compression.codec

2021-02-19 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34479?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-34479: Description: Avro add zstandard codec since AVRO-2195. (was: Avro add AVRO-2195) > Add

[jira] [Commented] (SPARK-34479) Add zstandard codec to spark.sql.avro.compression.codec

2021-02-19 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34479?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17287463#comment-17287463 ] Yuming Wang commented on SPARK-34479: - But zstd 1.4.5-12 is not compatible with 1.4.8-4.

[jira] [Comment Edited] (SPARK-34479) Add zstandard codec to spark.sql.avro.compression.codec

2021-02-19 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34479?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17287463#comment-17287463 ] Yuming Wang edited comment on SPARK-34479 at 2/20/21, 3:02 AM: --- But zstd

[jira] [Comment Edited] (SPARK-34479) Add zstandard codec to spark.sql.avro.compression.codec

2021-02-19 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34479?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17287463#comment-17287463 ] Yuming Wang edited comment on SPARK-34479 at 2/20/21, 3:02 AM: --- But zstd

[jira] [Commented] (SPARK-34477) Kryo NPEs when serializing Avro GenericData objects (except GenericRecord)

2021-02-19 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34477?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17287474#comment-17287474 ] Apache Spark commented on SPARK-34477: -- User 'shardulm94' has created a pull request for this

[jira] [Assigned] (SPARK-34477) Kryo NPEs when serializing Avro GenericData objects (except GenericRecord)

2021-02-19 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34477?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34477: Assignee: Apache Spark > Kryo NPEs when serializing Avro GenericData objects (except

[jira] [Assigned] (SPARK-34477) Kryo NPEs when serializing Avro GenericData objects (except GenericRecord)

2021-02-19 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34477?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34477: Assignee: (was: Apache Spark) > Kryo NPEs when serializing Avro GenericData objects

[jira] [Resolved] (SPARK-34424) HiveOrcHadoopFsRelationSuite fails with seed 610710213676

2021-02-19 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34424?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-34424. - Fix Version/s: 3.1.1 3.0.2 Assignee: Maxim Gekk Resolution:

[jira] [Created] (SPARK-34473) avoid NPE in DataFrameReader.schema(StructType)

2021-02-19 Thread Wenchen Fan (Jira)
Wenchen Fan created SPARK-34473: --- Summary: avoid NPE in DataFrameReader.schema(StructType) Key: SPARK-34473 URL: https://issues.apache.org/jira/browse/SPARK-34473 Project: Spark Issue Type:

[jira] [Commented] (SPARK-25075) Build and test Spark against Scala 2.13

2021-02-19 Thread Guillaume Martres (Jira)
[ https://issues.apache.org/jira/browse/SPARK-25075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17287109#comment-17287109 ] Guillaume Martres commented on SPARK-25075: --- [~dongjoon] I think something is wrong with the

[jira] [Commented] (SPARK-34421) Custom functions can't be used in temporary views with CTEs

2021-02-19 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17287057#comment-17287057 ] Apache Spark commented on SPARK-34421: -- User 'peter-toth' has created a pull request for this

[jira] [Resolved] (SPARK-34283) Combines all adjacent 'Union' operators into a single 'Union' when using 'Dataset.union.distinct.union.distinct'

2021-02-19 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34283?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-34283. - Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 31404

[jira] [Assigned] (SPARK-28123) String Functions: Add support btrim

2021-02-19 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28123?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-28123: --- Assignee: jiaan.geng > String Functions: Add support btrim >

[jira] [Resolved] (SPARK-28123) String Functions: Add support btrim

2021-02-19 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28123?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-28123. - Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 31390

[jira] [Assigned] (SPARK-34473) avoid NPE in DataFrameReader.schema(StructType)

2021-02-19 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34473?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34473: Assignee: (was: Apache Spark) > avoid NPE in DataFrameReader.schema(StructType) >

[jira] [Commented] (SPARK-34473) avoid NPE in DataFrameReader.schema(StructType)

2021-02-19 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34473?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17287096#comment-17287096 ] Apache Spark commented on SPARK-34473: -- User 'cloud-fan' has created a pull request for this issue:

[jira] [Assigned] (SPARK-34473) avoid NPE in DataFrameReader.schema(StructType)

2021-02-19 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34473?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34473: Assignee: Apache Spark > avoid NPE in DataFrameReader.schema(StructType) >

[jira] [Assigned] (SPARK-34283) Combines all adjacent 'Union' operators into a single 'Union' when using 'Dataset.union.distinct.union.distinct'

2021-02-19 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34283?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-34283: --- Assignee: Zhichao Zhang > Combines all adjacent 'Union' operators into a single 'Union'

[jira] [Assigned] (SPARK-34421) Custom functions can't be used in temporary views with CTEs

2021-02-19 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34421?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-34421: --- Assignee: Peter Toth > Custom functions can't be used in temporary views with CTEs >

[jira] [Created] (SPARK-34471) Document DataStreamReader/Writer table APIs in Structured Streaming Programming Guide

2021-02-19 Thread Bo Zhang (Jira)
Bo Zhang created SPARK-34471: Summary: Document DataStreamReader/Writer table APIs in Structured Streaming Programming Guide Key: SPARK-34471 URL: https://issues.apache.org/jira/browse/SPARK-34471

[jira] [Assigned] (SPARK-34472) SparkContext.addJar with an ivy path fails in cluster mode with a custom ivySettings file

2021-02-19 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34472?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34472: Assignee: (was: Apache Spark) > SparkContext.addJar with an ivy path fails in

[jira] [Commented] (SPARK-34472) SparkContext.addJar with an ivy path fails in cluster mode with a custom ivySettings file

2021-02-19 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34472?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17287009#comment-17287009 ] Apache Spark commented on SPARK-34472: -- User 'shardulm94' has created a pull request for this

[jira] [Assigned] (SPARK-34472) SparkContext.addJar with an ivy path fails in cluster mode with a custom ivySettings file

2021-02-19 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34472?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34472: Assignee: Apache Spark > SparkContext.addJar with an ivy path fails in cluster mode with

[jira] [Commented] (SPARK-34472) SparkContext.addJar with an ivy path fails in cluster mode with a custom ivySettings file

2021-02-19 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34472?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17287008#comment-17287008 ] Apache Spark commented on SPARK-34472: -- User 'shardulm94' has created a pull request for this

[jira] [Commented] (SPARK-34424) HiveOrcHadoopFsRelationSuite fails with seed 610710213676

2021-02-19 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34424?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17286926#comment-17286926 ] Apache Spark commented on SPARK-34424: -- User 'MaxGekk' has created a pull request for this issue:

[jira] [Assigned] (SPARK-34471) Document DataStreamReader/Writer table APIs in Structured Streaming Programming Guide

2021-02-19 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34471?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34471: Assignee: Apache Spark > Document DataStreamReader/Writer table APIs in Structured

[jira] [Commented] (SPARK-34471) Document DataStreamReader/Writer table APIs in Structured Streaming Programming Guide

2021-02-19 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34471?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17286951#comment-17286951 ] Apache Spark commented on SPARK-34471: -- User 'bozhang2820' has created a pull request for this

[jira] [Assigned] (SPARK-34471) Document DataStreamReader/Writer table APIs in Structured Streaming Programming Guide

2021-02-19 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34471?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34471: Assignee: (was: Apache Spark) > Document DataStreamReader/Writer table APIs in

[jira] [Commented] (SPARK-34471) Document DataStreamReader/Writer table APIs in Structured Streaming Programming Guide

2021-02-19 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34471?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17286953#comment-17286953 ] Apache Spark commented on SPARK-34471: -- User 'bozhang2820' has created a pull request for this

[jira] [Created] (SPARK-34472) SparkContext.addJar with an ivy path fails in cluster mode with a custom ivySettings file

2021-02-19 Thread Shardul Mahadik (Jira)
Shardul Mahadik created SPARK-34472: --- Summary: SparkContext.addJar with an ivy path fails in cluster mode with a custom ivySettings file Key: SPARK-34472 URL: https://issues.apache.org/jira/browse/SPARK-34472

[jira] [Updated] (SPARK-34421) Custom functions can't be used in temporary views with CTEs

2021-02-19 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34421?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan updated SPARK-34421: Description: The following query works in Spark 3.0 not Spark 3.1.   Start with:

[jira] [Commented] (SPARK-21770) ProbabilisticClassificationModel: Improve normalization of all-zero raw predictions

2021-02-19 Thread Weichen Xu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-21770?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17286916#comment-17286916 ] Weichen Xu commented on SPARK-21770: [~rishi-aga] Could you create a new ticket for this with

[jira] [Assigned] (SPARK-34314) Wrong discovered partition value

2021-02-19 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34314?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-34314: --- Assignee: Maxim Gekk > Wrong discovered partition value >

[jira] [Resolved] (SPARK-34314) Wrong discovered partition value

2021-02-19 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34314?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-34314. - Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 31549

[jira] [Resolved] (SPARK-34421) Custom functions can't be used in temporary views with CTEs

2021-02-19 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34421?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-34421. - Fix Version/s: 3.1.1 Resolution: Fixed Issue resolved by pull request 31550

[jira] [Commented] (SPARK-34472) SparkContext.addJar with an ivy path fails in cluster mode with a custom ivySettings file

2021-02-19 Thread Shardul Mahadik (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34472?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17286977#comment-17286977 ] Shardul Mahadik commented on SPARK-34472: - I will be sending a PR for this soon. >

[jira] [Assigned] (SPARK-34469) Ignore RegisterExecutor when SparkContext is stopped

2021-02-19 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34469?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-34469: - Assignee: Dongjoon Hyun > Ignore RegisterExecutor when SparkContext is stopped >

[jira] [Resolved] (SPARK-34469) Ignore RegisterExecutor when SparkContext is stopped

2021-02-19 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34469?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-34469. --- Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 31587

[jira] [Updated] (SPARK-34475) Rename v2 logical nodes

2021-02-19 Thread Maxim Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34475?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maxim Gekk updated SPARK-34475: --- Description: Rename v2 logical nodes for simplicity in the form: +   (was: To be consistent with

[jira] [Commented] (SPARK-34468) Fix v2 ALTER TABLE .. RENAME TO

2021-02-19 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17287274#comment-17287274 ] Apache Spark commented on SPARK-34468: -- User 'MaxGekk' has created a pull request for this issue:

[jira] [Assigned] (SPARK-34468) Fix v2 ALTER TABLE .. RENAME TO

2021-02-19 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34468?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34468: Assignee: (was: Apache Spark) > Fix v2 ALTER TABLE .. RENAME TO >

[jira] [Assigned] (SPARK-34468) Fix v2 ALTER TABLE .. RENAME TO

2021-02-19 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34468?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34468: Assignee: Apache Spark > Fix v2 ALTER TABLE .. RENAME TO >

[jira] [Created] (SPARK-34474) Remove unnecessary Union under Distinct like operators

2021-02-19 Thread L. C. Hsieh (Jira)
L. C. Hsieh created SPARK-34474: --- Summary: Remove unnecessary Union under Distinct like operators Key: SPARK-34474 URL: https://issues.apache.org/jira/browse/SPARK-34474 Project: Spark Issue

[jira] [Updated] (SPARK-34424) HiveOrcHadoopFsRelationSuite fails with seed 610710213676

2021-02-19 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34424?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-34424: -- Fix Version/s: (was: 3.0.2) 3.0.3 > HiveOrcHadoopFsRelationSuite fails

[jira] [Commented] (SPARK-34475) Rename v2 logical nodes

2021-02-19 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34475?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17287340#comment-17287340 ] Apache Spark commented on SPARK-34475: -- User 'MaxGekk' has created a pull request for this issue:

[jira] [Assigned] (SPARK-34475) Rename v2 logical nodes

2021-02-19 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34475?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34475: Assignee: Apache Spark (was: Maxim Gekk) > Rename v2 logical nodes >

[jira] [Assigned] (SPARK-34475) Rename v2 logical nodes

2021-02-19 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34475?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34475: Assignee: Maxim Gekk (was: Apache Spark) > Rename v2 logical nodes >

[jira] [Assigned] (SPARK-34474) Remove unnecessary Union under Distinct like operators

2021-02-19 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34474?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34474: Assignee: Apache Spark (was: L. C. Hsieh) > Remove unnecessary Union under Distinct

[jira] [Commented] (SPARK-34474) Remove unnecessary Union under Distinct like operators

2021-02-19 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34474?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17287320#comment-17287320 ] Apache Spark commented on SPARK-34474: -- User 'viirya' has created a pull request for this issue:

[jira] [Assigned] (SPARK-34474) Remove unnecessary Union under Distinct like operators

2021-02-19 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34474?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34474: Assignee: L. C. Hsieh (was: Apache Spark) > Remove unnecessary Union under Distinct

[jira] [Created] (SPARK-34475) Rename v2 logical nodes

2021-02-19 Thread Maxim Gekk (Jira)
Maxim Gekk created SPARK-34475: -- Summary: Rename v2 logical nodes Key: SPARK-34475 URL: https://issues.apache.org/jira/browse/SPARK-34475 Project: Spark Issue Type: Sub-task

[jira] [Commented] (SPARK-25075) Build and test Spark against Scala 2.13

2021-02-19 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-25075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17287437#comment-17287437 ] Dongjoon Hyun commented on SPARK-25075: --- [~MasseGuillaume]. Feel free to create a new independent

[jira] [Updated] (SPARK-34476) Duplicate referenceNames are given for ambiguousReferences

2021-02-19 Thread Ted Yu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34476?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated SPARK-34476: --- Description: When running test with Spark extension that converts custom function to json path expression,

[jira] [Resolved] (SPARK-24818) Ensure all the barrier tasks in the same stage are launched together

2021-02-19 Thread Mridul Muralidharan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-24818?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan resolved SPARK-24818. - Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request

[jira] [Created] (SPARK-34476) Duplicate referenceNames are given for ambiguousReferences

2021-02-19 Thread Ted Yu (Jira)
Ted Yu created SPARK-34476: -- Summary: Duplicate referenceNames are given for ambiguousReferences Key: SPARK-34476 URL: https://issues.apache.org/jira/browse/SPARK-34476 Project: Spark Issue Type:

[jira] [Assigned] (SPARK-24818) Ensure all the barrier tasks in the same stage are launched together

2021-02-19 Thread Mridul Muralidharan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-24818?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan reassigned SPARK-24818: --- Assignee: wuyi > Ensure all the barrier tasks in the same stage are

[jira] [Commented] (SPARK-34476) Duplicate referenceNames are given for ambiguousReferences

2021-02-19 Thread Ted Yu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34476?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17287403#comment-17287403 ] Ted Yu commented on SPARK-34476: The basic jsonb test is here: