[jira] [Resolved] (SPARK-46905) Add dedicated class to keep column definition instead of StructField in Create/ReplaceTable command

2024-01-29 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46905?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-46905. - Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 44935 [https

[jira] [Assigned] (SPARK-46905) Add dedicated class to keep column definition instead of StructField in Create/ReplaceTable command

2024-01-29 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46905?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-46905: --- Assignee: Wenchen Fan > Add dedicated class to keep column definition inst

(spark) branch master updated: [SPARK-46905][SQL] Add dedicated class to keep column definition instead of StructField in Create/ReplaceTable command

2024-01-29 Thread wenchen
This is an automated email from the ASF dual-hosted git repository. wenchen pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new 4da7c3d316e3 [SPARK-46905][SQL] Add dedicated

[jira] [Created] (SPARK-46905) Add dedicated class to keep column definition instead of StructField in Create/ReplaceTable command

2024-01-29 Thread Wenchen Fan (Jira)
Wenchen Fan created SPARK-46905: --- Summary: Add dedicated class to keep column definition instead of StructField in Create/ReplaceTable command Key: SPARK-46905 URL: https://issues.apache.org/jira/browse/SPARK-46905

(spark) branch branch-3.5 updated: [SPARK-46590][SQL][FOLLOWUP] Update CoalesceShufflePartitions comments

2024-01-23 Thread wenchen
This is an automated email from the ASF dual-hosted git repository. wenchen pushed a commit to branch branch-3.5 in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/branch-3.5 by this push: new 0956db6901bf [SPARK-46590][SQL

(spark) branch master updated: [SPARK-46590][SQL][FOLLOWUP] Update CoalesceShufflePartitions comments

2024-01-23 Thread wenchen
This is an automated email from the ASF dual-hosted git repository. wenchen pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new fe4f8eac3efe [SPARK-46590][SQL][FOLLOWUP

[jira] [Resolved] (SPARK-46683) Write a subquery generator that generates subqueries of different variations to increase testing coverage in this area

2024-01-23 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46683?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-46683. - Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 44599 [https

(spark) branch master updated (2aed25b15dba -> bc889c8c2adb)

2024-01-23 Thread wenchen
This is an automated email from the ASF dual-hosted git repository. wenchen pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/spark.git from 2aed25b15dba Revert "[SPARK-46417][SQL] Do not fail when calling hive.getTable and throwException is false"

[jira] [Resolved] (SPARK-46763) ReplaceDeduplicateWithAggregate fails when non-grouping keys have duplicate attributes

2024-01-23 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46763?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-46763. - Fix Version/s: 3.4.3 3.5.1 4.0.0 Resolution: Fixed

[jira] [Assigned] (SPARK-46763) ReplaceDeduplicateWithAggregate fails when non-grouping keys have duplicate attributes

2024-01-23 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46763?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-46763: --- Assignee: Nikhil Sheoran > ReplaceDeduplicateWithAggregate fails when non-grouping k

(spark) branch branch-3.4 updated: [SPARK-46763] Fix assertion failure in ReplaceDeduplicateWithAggregate for duplicate attributes

2024-01-23 Thread wenchen
This is an automated email from the ASF dual-hosted git repository. wenchen pushed a commit to branch branch-3.4 in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/branch-3.4 by this push: new 34f81d647552 [SPARK-46763] Fix

(spark) branch branch-3.5 updated: [SPARK-46763] Fix assertion failure in ReplaceDeduplicateWithAggregate for duplicate attributes

2024-01-23 Thread wenchen
This is an automated email from the ASF dual-hosted git repository. wenchen pushed a commit to branch branch-3.5 in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/branch-3.5 by this push: new a559ff7bb9d3 [SPARK-46763] Fix

(spark) branch master updated: [SPARK-46763] Fix assertion failure in ReplaceDeduplicateWithAggregate for duplicate attributes

2024-01-23 Thread wenchen
This is an automated email from the ASF dual-hosted git repository. wenchen pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new 715b43428913 [SPARK-46763] Fix assertion

[jira] [Resolved] (SPARK-46590) Coalesce partiton assert error after skew join optimization

2024-01-23 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46590?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-46590. - Fix Version/s: 3.5.1 4.0.0 Resolution: Fixed Issue resolved by pull

[jira] [Assigned] (SPARK-46590) Coalesce partiton assert error after skew join optimization

2024-01-23 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46590?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-46590: --- Assignee: Jackey Lee > Coalesce partiton assert error after skew join optimizat

(spark) branch branch-3.5 updated: [SPARK-46590][SQL] Fix coalesce failed with unexpected partition indeces

2024-01-23 Thread wenchen
This is an automated email from the ASF dual-hosted git repository. wenchen pushed a commit to branch branch-3.5 in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/branch-3.5 by this push: new 6403a84b6854 [SPARK-46590][SQL] Fix

(spark) branch master updated: [SPARK-46590][SQL] Fix coalesce failed with unexpected partition indeces

2024-01-23 Thread wenchen
This is an automated email from the ASF dual-hosted git repository. wenchen pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new de0c4ad3947f [SPARK-46590][SQL] Fix coalesce

(spark) branch master updated: Revert "[SPARK-46219][SQL] Unwrap cast in join predicates"

2024-01-21 Thread wenchen
This is an automated email from the ASF dual-hosted git repository. wenchen pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new 498519ee6bb4 Revert "[SPARK-46219][SQL] U

[jira] [Updated] (SPARK-46769) Refine timestamp related schema inference

2024-01-20 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46769?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan updated SPARK-46769: Summary: Refine timestamp related schema inference (was: Fix inferring of TIMESTAMP_NTZ in CSV

[jira] [Updated] (SPARK-46769) Refine timestamp related schema inference

2024-01-20 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46769?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan updated SPARK-46769: Description: (was: After the PR https://github.com/apache/spark/pull/43243, the TIMESTAMP_NTZ

[jira] [Assigned] (SPARK-46769) Refine timestamp related schema inference

2024-01-20 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46769?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-46769: --- Assignee: Wenchen Fan (was: Max Gekk) > Refine timestamp related schema infere

[jira] [Resolved] (SPARK-46769) Fix inferring of TIMESTAMP_NTZ in CSV/JSON

2024-01-20 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46769?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-46769. - Fix Version/s: 3.5.1 4.0.0 Resolution: Fixed Issue resolved by pull

(spark) branch branch-3.5 updated: [SPARK-46769][SQL] Refine timestamp related schema inference

2024-01-20 Thread wenchen
This is an automated email from the ASF dual-hosted git repository. wenchen pushed a commit to branch branch-3.5 in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/branch-3.5 by this push: new c19bf01b5208 [SPARK-46769][SQL] Refine

(spark) branch master updated: [SPARK-46769][SQL] Refine timestamp related schema inference

2024-01-20 Thread wenchen
This is an automated email from the ASF dual-hosted git repository. wenchen pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new e4e40762ca41 [SPARK-46769][SQL] Refine

(spark) branch master updated: [SPARK-45827] Disallow partitioning on Variant column

2024-01-19 Thread wenchen
This is an automated email from the ASF dual-hosted git repository. wenchen pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new 689ab0ee8ca8 [SPARK-45827] Disallow

(spark) branch master updated: [SPARK-46707][SQL] Added throwable field to expressions to improve predicate pushdown

2024-01-18 Thread wenchen
This is an automated email from the ASF dual-hosted git repository. wenchen pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new 62956c92cfc7 [SPARK-46707][SQL] Added throwable

[jira] [Resolved] (SPARK-46644) Fix add in SQLMetric

2024-01-17 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46644?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-46644. - Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 44649 [https

(spark) branch master updated: [SPARK-46644] Change add and merge in SQLMetric to use isZero

2024-01-17 Thread wenchen
This is an automated email from the ASF dual-hosted git repository. wenchen pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new 89727bfa7529 [SPARK-46644] Change add and merge

[jira] [Assigned] (SPARK-46274) Range operator computeStats() proper long conversions

2024-01-14 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46274?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-46274: --- Assignee: Nick Young (was: Kelvin Jiang) > Range operator computeStats() proper l

[jira] [Assigned] (SPARK-45435) Document that lazy checkpoint may not be a consistent

2024-01-14 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45435?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-45435: --- Assignee: Juliusz Sompolski > Document that lazy checkpoint may not be a consist

[jira] [Resolved] (SPARK-45435) Document that lazy checkpoint may not be a consistent

2024-01-14 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45435?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-45435. - Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 43247 [https

(spark) branch master updated: [SPARK-45435][DOC] Document that lazy checkpoint may not be a consistent snapshot

2024-01-14 Thread wenchen
This is an automated email from the ASF dual-hosted git repository. wenchen pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new 65d822c44d93 [SPARK-45435][DOC] Document

[jira] [Created] (SPARK-46700) count the last spilling for the shuffle disk spilling bytes metric

2024-01-12 Thread Wenchen Fan (Jira)
Wenchen Fan created SPARK-46700: --- Summary: count the last spilling for the shuffle disk spilling bytes metric Key: SPARK-46700 URL: https://issues.apache.org/jira/browse/SPARK-46700 Project: Spark

(spark) branch master updated: [SPARK-46052][CORE] Remove function TaskScheduler.killAllTaskAttempts

2024-01-12 Thread wenchen
This is an automated email from the ASF dual-hosted git repository. wenchen pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new 96f34bbeb40c [SPARK-46052][CORE] Remove

[jira] [Assigned] (SPARK-46052) Remove unnecessary TaskScheduler.killAllTaskAttempts

2024-01-12 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46052?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-46052: --- Assignee: wuyi > Remove unnecessary TaskScheduler.killAllTaskAttem

[jira] [Resolved] (SPARK-46052) Remove unnecessary TaskScheduler.killAllTaskAttempts

2024-01-12 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46052?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-46052. - Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 43954 [https

[jira] [Assigned] (SPARK-46383) Reduce Driver Heap Usage by Reducing the Lifespan of `TaskInfo.accumulables()`

2024-01-11 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46383?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-46383: --- Assignee: Utkarsh Agarwal > Reduce Driver Heap Usage by Reducing the Lifes

[jira] [Resolved] (SPARK-46383) Reduce Driver Heap Usage by Reducing the Lifespan of `TaskInfo.accumulables()`

2024-01-11 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46383?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-46383. - Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 44321 [https

(spark) branch master updated: [SPARK-46383] Reduce Driver Heap Usage by Reducing the Lifespan of `TaskInfo.accumulables()`

2024-01-11 Thread wenchen
This is an automated email from the ASF dual-hosted git repository. wenchen pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new 28da1d853477 [SPARK-46383] Reduce Driver Heap

[jira] [Resolved] (SPARK-46640) RemoveRedundantAliases does not account for SubqueryExpression when removing aliases

2024-01-11 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46640?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-46640. - Fix Version/s: 3.5.1 4.0.0 Resolution: Fixed Issue resolved by pull

[jira] [Assigned] (SPARK-46640) RemoveRedundantAliases does not account for SubqueryExpression when removing aliases

2024-01-11 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46640?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-46640: --- Assignee: Nikhil Sheoran > RemoveRedundantAliases does not account for SubqueryExpress

(spark) branch branch-3.5 updated: [SPARK-46640][SQL] Fix RemoveRedundantAlias by excluding subquery attributes

2024-01-11 Thread wenchen
This is an automated email from the ASF dual-hosted git repository. wenchen pushed a commit to branch branch-3.5 in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/branch-3.5 by this push: new 8a0f64274f44 [SPARK-46640][SQL] Fix

(spark) branch master updated: [SPARK-46640][SQL] Fix RemoveRedundantAlias by excluding subquery attributes

2024-01-11 Thread wenchen
This is an automated email from the ASF dual-hosted git repository. wenchen pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new bbeb8d7417ba [SPARK-46640][SQL] Fix

Re: [VOTE] SPIP: Structured Streaming - Arbitrary State API v2

2024-01-10 Thread Wenchen Fan
+1 On Thu, Jan 11, 2024 at 9:32 AM L. C. Hsieh wrote: > +1 > > On Wed, Jan 10, 2024 at 9:06 AM Bhuwan Sahni > wrote: > >> +1. This is a good addition. >> >> >> *Bhuwan Sahni* >> Staff Software Engineer >> >> bhuwan.sa...@databricks.com >> 500 108th Ave. NE >>

(spark) branch master updated: [SPARK-46442][SQL] DS V2 supports push down PERCENTILE_CONT and PERCENTILE_DISC

2024-01-09 Thread wenchen
This is an automated email from the ASF dual-hosted git repository. wenchen pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new 85b504d64701 [SPARK-46442][SQL] DS V2 supports

[jira] [Created] (SPARK-46634) literal validation should not drill down to null fields

2024-01-09 Thread Wenchen Fan (Jira)
Wenchen Fan created SPARK-46634: --- Summary: literal validation should not drill down to null fields Key: SPARK-46634 URL: https://issues.apache.org/jira/browse/SPARK-46634 Project: Spark Issue

[jira] [Assigned] (SPARK-46331) Removing CodeGenFallback trait from subset of datetime and spark version functions

2024-01-09 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46331?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-46331: --- Assignee: Aleksandar Tomic > Removing CodeGenFallback trait from subset of datet

[jira] [Resolved] (SPARK-46331) Removing CodeGenFallback trait from subset of datetime and spark version functions

2024-01-09 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46331?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-46331. - Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 44261 [https

(spark) branch master updated: [SPARK-46331][SQL] Removing CodegenFallback from subset of DateTime expressions and version() expression

2024-01-09 Thread wenchen
This is an automated email from the ASF dual-hosted git repository. wenchen pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new ee1fd926802b [SPARK-46331][SQL] Removing

[jira] [Resolved] (SPARK-46610) Create table should throw exception when no value for a key in options

2024-01-08 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46610?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-46610. - Fix Version/s: 3.5.1 4.0.0 Resolution: Fixed Issue resolved by pull

(spark) branch branch-3.5 updated: [SPARK-46610][SQL] Create table should throw exception when no value for a key in options

2024-01-08 Thread wenchen
This is an automated email from the ASF dual-hosted git repository. wenchen pushed a commit to branch branch-3.5 in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/branch-3.5 by this push: new 2b0c3e12636b [SPARK-46610][SQL] Create

(spark) branch master updated: [SPARK-46610][SQL] Create table should throw exception when no value for a key in options

2024-01-08 Thread wenchen
This is an automated email from the ASF dual-hosted git repository. wenchen pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new e7536f2484af [SPARK-46610][SQL] Create table

[jira] [Assigned] (SPARK-46581) AccumulatorV2 isZero doesn't do what its name implies

2024-01-08 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46581?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-46581: --- Assignee: Davin Tjong > AccumulatorV2 isZero doesn't do what its name impl

[jira] [Resolved] (SPARK-46581) AccumulatorV2 isZero doesn't do what its name implies

2024-01-08 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46581?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-46581. - Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 44583 [https

(spark) branch master updated: [SPARK-46581][CORE] Update comment on isZero in AccumulatorV2

2024-01-08 Thread wenchen
This is an automated email from the ASF dual-hosted git repository. wenchen pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new 40078a4c3726 [SPARK-46581][CORE] Update comment

[jira] [Resolved] (SPARK-45352) Eliminate foldable window partitions

2024-01-08 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45352?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-45352. - Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 43144 [https

[jira] [Assigned] (SPARK-45352) Eliminate foldable window partitions

2024-01-08 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45352?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-45352: --- Assignee: Mingliang Zhu > Eliminate foldable window partiti

(spark) branch master updated: [SPARK-45352][SQL] Eliminate foldable window partitions

2024-01-08 Thread wenchen
This is an automated email from the ASF dual-hosted git repository. wenchen pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new 8facc992fe46 [SPARK-45352][SQL] Eliminate

(spark) branch master updated: [SPARK-40876][SQL] Widening type promotion for decimals with larger scale in Parquet readers

2024-01-08 Thread wenchen
This is an automated email from the ASF dual-hosted git repository. wenchen pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new d439e34d6bda [SPARK-40876][SQL] Widening type

[jira] [Created] (SPARK-46609) avoid exponential explosion in PartitioningPreservingUnaryExecNode

2024-01-05 Thread Wenchen Fan (Jira)
Wenchen Fan created SPARK-46609: --- Summary: avoid exponential explosion in PartitioningPreservingUnaryExecNode Key: SPARK-46609 URL: https://issues.apache.org/jira/browse/SPARK-46609 Project: Spark

[jira] [Assigned] (SPARK-46602) CREATE VIEW IF NOT EXISTS should never throw `TABLE_OR_VIEW_ALREADY_EXISTS` exception

2024-01-05 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46602?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-46602: --- Assignee: Xinyi Yu > CREATE VIEW IF NOT EXISTS should never th

[jira] [Resolved] (SPARK-46602) CREATE VIEW IF NOT EXISTS should never throw `TABLE_OR_VIEW_ALREADY_EXISTS` exception

2024-01-05 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46602?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-46602. - Fix Version/s: 3.5.1 4.0.0 Resolution: Fixed Issue resolved by pull

(spark) branch branch-3.5 updated: [SPARK-46602][SQL] Propagate `allowExisting` in view creation when the view/table does not exists

2024-01-05 Thread wenchen
This is an automated email from the ASF dual-hosted git repository. wenchen pushed a commit to branch branch-3.5 in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/branch-3.5 by this push: new 1b7ee9e56b30 [SPARK-46602][SQL

(spark) branch master updated: [SPARK-46602][SQL] Propagate `allowExisting` in view creation when the view/table does not exists

2024-01-05 Thread wenchen
This is an automated email from the ASF dual-hosted git repository. wenchen pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new 9b3c70f6094c [SPARK-46602][SQL] Propagate

[jira] [Resolved] (SPARK-46179) Generate golden files for SQLQueryTestSuites with Postgres

2024-01-04 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46179?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-46179. - Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 44084 [https

[jira] [Assigned] (SPARK-46179) Generate golden files for SQLQueryTestSuites with Postgres

2024-01-04 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46179?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-46179: --- Assignee: Andy Lam > Generate golden files for SQLQueryTestSuites with Postg

(spark) branch master updated: [SPARK-46179][SQL] Add CrossDbmsQueryTestSuites, which runs other DBMS against golden files with other DBMS, starting with Postgres

2024-01-04 Thread wenchen
This is an automated email from the ASF dual-hosted git repository. wenchen pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new f9ca519897d5 [SPARK-46179][SQL] Add

[jira] [Created] (SPARK-46598) OrcColumnarBatchReader should respect the memory mode when creating column vectors for the missing column

2024-01-04 Thread Wenchen Fan (Jira)
Wenchen Fan created SPARK-46598: --- Summary: OrcColumnarBatchReader should respect the memory mode when creating column vectors for the missing column Key: SPARK-46598 URL: https://issues.apache.org/jira/browse/SPARK

[jira] [Created] (SPARK-46536) Support GROUP BY calendar_interval_type

2023-12-28 Thread Wenchen Fan (Jira)
Wenchen Fan created SPARK-46536: --- Summary: Support GROUP BY calendar_interval_type Key: SPARK-46536 URL: https://issues.apache.org/jira/browse/SPARK-46536 Project: Spark Issue Type

(spark) branch master updated: [SPARK-46366][SQL] Use WITH expression in BETWEEN to avoid duplicate expressions

2023-12-27 Thread wenchen
This is an automated email from the ASF dual-hosted git repository. wenchen pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new 93a2526528c7 [SPARK-46366][SQL] Use

[jira] [Resolved] (SPARK-46366) Use with expression to avoid duplicating expressions in BETWEEN operation

2023-12-27 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46366?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-46366. - Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 44299 [https

[jira] [Assigned] (SPARK-46444) V2SessionCatalog#createTable should not load the table

2023-12-25 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46444?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-46444: --- Assignee: Wenchen Fan > V2SessionCatalog#createTable should not load the ta

[jira] [Resolved] (SPARK-46444) V2SessionCatalog#createTable should not load the table

2023-12-25 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46444?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-46444. - Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 44377 [https

(spark) branch master updated: [SPARK-46444][SQL] V2SessionCatalog#createTable should not load the table

2023-12-25 Thread wenchen
This is an automated email from the ASF dual-hosted git repository. wenchen pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new d8fb91e61352 [SPARK-46444][SQL

(spark) branch master updated: [SPARK-46481] Execute immediate VariableReference foldable

2023-12-25 Thread wenchen
This is an automated email from the ASF dual-hosted git repository. wenchen pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new c1d3fc1de30e [SPARK-46481] Execute immediate

[jira] [Resolved] (SPARK-46481) EXECUTE IMMEDIATE does not fold variables when given as parameters

2023-12-25 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46481?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-46481. - Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 44450 [https

[jira] [Resolved] (SPARK-46485) V1Write should not add Sort when not needed

2023-12-22 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46485?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-46485. - Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 44458 [https

(spark) branch master updated: [SPARK-46485][SQL] V1Write should not add Sort when not needed

2023-12-22 Thread wenchen
This is an automated email from the ASF dual-hosted git repository. wenchen pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new cb2f47bf7ec1 [SPARK-46485][SQL] V1Write should

[jira] [Updated] (SPARK-46466) vectorized parquet reader should never do rebase for timestamp ntz

2023-12-22 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46466?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan updated SPARK-46466: Fix Version/s: 3.5.1 3.4.3 > vectorized parquet reader should never do reb

(spark) branch branch-3.4 updated: [SPARK-46466][SQL][3.5] Vectorized parquet reader should never do rebase for timestamp ntz

2023-12-22 Thread wenchen
This is an automated email from the ASF dual-hosted git repository. wenchen pushed a commit to branch branch-3.4 in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/branch-3.4 by this push: new f8eb533774a8 [SPARK-46466][SQL][3.5

(spark) branch branch-3.5 updated: [SPARK-46466][SQL][3.5] Vectorized parquet reader should never do rebase for timestamp ntz

2023-12-22 Thread wenchen
This is an automated email from the ASF dual-hosted git repository. wenchen pushed a commit to branch branch-3.5 in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/branch-3.5 by this push: new 0948e24c30f6 [SPARK-46466][SQL][3.5

[jira] [Resolved] (SPARK-40876) Spark's Vectorized ParquetReader should support type promotions

2023-12-22 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40876?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-40876. - Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 44368 [https

[jira] [Assigned] (SPARK-40876) Spark's Vectorized ParquetReader should support type promotions

2023-12-22 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40876?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-40876: --- Assignee: Johan Lasperas > Spark's Vectorized ParquetReader should support type promoti

(spark) branch master updated: [SPARK-40876][SQL] Widening type promotions in Parquet readers

2023-12-22 Thread wenchen
This is an automated email from the ASF dual-hosted git repository. wenchen pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new 3361f25dc0ff [SPARK-40876][SQL] Widening type

[jira] [Created] (SPARK-46485) V1Write should not add Sort when not needed

2023-12-21 Thread Wenchen Fan (Jira)
Wenchen Fan created SPARK-46485: --- Summary: V1Write should not add Sort when not needed Key: SPARK-46485 URL: https://issues.apache.org/jira/browse/SPARK-46485 Project: Spark Issue Type: Bug

[jira] [Resolved] (SPARK-46443) Decimal precision and scale should decided by JDBC dialect.

2023-12-21 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46443?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-46443. - Fix Version/s: 3.5.1 4.0.0 Resolution: Fixed Issue resolved by pull

(spark) branch branch-3.5 updated: [SPARK-46443][SQL] Decimal precision and scale should decided by H2 dialect

2023-12-21 Thread wenchen
This is an automated email from the ASF dual-hosted git repository. wenchen pushed a commit to branch branch-3.5 in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/branch-3.5 by this push: new 286c469ad13 [SPARK-46443][SQL] Decimal

(spark) branch master updated: [SPARK-46443][SQL] Decimal precision and scale should decided by H2 dialect

2023-12-21 Thread wenchen
This is an automated email from the ASF dual-hosted git repository. wenchen pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new a921da8509a [SPARK-46443][SQL] Decimal

(spark) branch master updated: [SPARK-46380][SQL][FOLLOWUP] Simplify the code for ResolveInlineTables and ResolveInlineTablesSuite

2023-12-21 Thread wenchen
This is an automated email from the ASF dual-hosted git repository. wenchen pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new 8c63485189c [SPARK-46380][SQL][FOLLOWUP

[jira] [Resolved] (SPARK-46468) COUNT bug in lateral/exists subqueries

2023-12-21 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46468?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-46468. - Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 44451 [https

[jira] [Assigned] (SPARK-46468) COUNT bug in lateral/exists subqueries

2023-12-21 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46468?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-46468: --- Assignee: Andrey Gubichev > COUNT bug in lateral/exists subquer

(spark) branch master updated: [SPARK-46468][SQL] Handle COUNT bug for EXISTS subqueries with Aggregate without grouping keys

2023-12-21 Thread wenchen
This is an automated email from the ASF dual-hosted git repository. wenchen pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new 3432fd8dba5 [SPARK-46468][SQL] Handle COUNT bug

[jira] [Resolved] (SPARK-45525) Initial support for Python data source write API

2023-12-21 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45525?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-45525. - Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 43791 [https

[jira] [Assigned] (SPARK-45525) Initial support for Python data source write API

2023-12-21 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45525?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-45525: --- Assignee: Allison Wang > Initial support for Python data source write

(spark) branch master updated: [SPARK-45525][SQL][PYTHON] Support for Python data source write using DSv2

2023-12-21 Thread wenchen
This is an automated email from the ASF dual-hosted git repository. wenchen pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new 4fcd5bfe003 [SPARK-45525][SQL][PYTHON] Support

(spark) branch master updated: [SPARK-46378][SQL][FOLLOWUP] Do not rely on TreeNodeTag in Project

2023-12-21 Thread wenchen
This is an automated email from the ASF dual-hosted git repository. wenchen pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new 0e94f340a63a [SPARK-46378][SQL][FOLLOWUP] Do

[jira] [Resolved] (SPARK-46380) Replacing current time prior to inline table eval

2023-12-21 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46380?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-46380. - Fix Version/s: 3.5.1 4.0.0 Resolution: Fixed Issue resolved by pull

(spark) branch branch-3.5 updated: [SPARK-46380][SQL] Replace current time/date prior to evaluating inline table expressions

2023-12-21 Thread wenchen
This is an automated email from the ASF dual-hosted git repository. wenchen pushed a commit to branch branch-3.5 in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/branch-3.5 by this push: new d7534a3ec1ea [SPARK-46380][SQL] Replace

(spark) branch master updated: [SPARK-46380][SQL] Replace current time/date prior to evaluating inline table expressions

2023-12-21 Thread wenchen
This is an automated email from the ASF dual-hosted git repository. wenchen pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new 5fe963f8560e [SPARK-46380][SQL] Replace current

(spark) branch master updated: [SPARK-46179][SQL] Pull out code into reusable functions in SQLQueryTestSuite

2023-12-20 Thread wenchen
This is an automated email from the ASF dual-hosted git repository. wenchen pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new e804f5360144 [SPARK-46179][SQL] Pull out code

<    3   4   5   6   7   8   9   10   11   12   >