[jira] [Updated] (SPARK-43107) Coalesce buckets in join applied on broadcast join stream side

2023-04-12 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43107?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-43107: Summary: Coalesce buckets in join applied on broadcast join stream side (was: Coalesce applied

[jira] [Commented] (SPARK-43107) Coalesce applied on broadcast join stream side

2023-04-12 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43107?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17711605#comment-17711605 ] Yuming Wang commented on SPARK-43107: - https://github.com/apache/spark/pull/40756 > Coalesce

[jira] [Commented] (SPARK-43114) Add interval types to TypeCoercionSuite

2023-04-12 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43114?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17711604#comment-17711604 ] Yuming Wang commented on SPARK-43114: - https://github.com/apache/spark/pull/40763 > Add interval

[jira] [Created] (SPARK-43114) Add interval types to TypeCoercionSuite

2023-04-12 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-43114: --- Summary: Add interval types to TypeCoercionSuite Key: SPARK-43114 URL: https://issues.apache.org/jira/browse/SPARK-43114 Project: Spark Issue Type:

[jira] [Created] (SPARK-43107) Coalesce applied on broadcast join stream side

2023-04-12 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-43107: --- Summary: Coalesce applied on broadcast join stream side Key: SPARK-43107 URL: https://issues.apache.org/jira/browse/SPARK-43107 Project: Spark Issue Type:

[jira] [Resolved] (SPARK-43087) Support coalesce buckets in join in AQE

2023-04-11 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43087?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-43087. - Resolution: Duplicate > Support coalesce buckets in join in AQE >

[jira] [Created] (SPARK-43095) Avoid Once strategy's idempotence is broken for batch: Infer Filters

2023-04-11 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-43095: --- Summary: Avoid Once strategy's idempotence is broken for batch: Infer Filters Key: SPARK-43095 URL: https://issues.apache.org/jira/browse/SPARK-43095 Project: Spark

[jira] [Commented] (SPARK-43087) Support coalesce buckets in join in AQE

2023-04-11 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43087?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17710952#comment-17710952 ] Yuming Wang commented on SPARK-43087: - https://github.com/apache/spark/pull/40731 > Support

[jira] [Created] (SPARK-43087) Support coalesce buckets in join in AQE

2023-04-10 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-43087: --- Summary: Support coalesce buckets in join in AQE Key: SPARK-43087 URL: https://issues.apache.org/jira/browse/SPARK-43087 Project: Spark Issue Type:

[jira] [Commented] (SPARK-40609) Casts types according to bucket info for Equality expression

2023-04-10 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17710080#comment-17710080 ] Yuming Wang commented on SPARK-40609: - {code:scala} import org.apache.spark.benchmark.Benchmark val

[jira] [Updated] (SPARK-43050) Fix construct aggregate expressions by replacing grouping functions

2023-04-06 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43050?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-43050: Description: {code:sql} CREATE TEMPORARY VIEW grouping AS SELECT * FROM VALUES ("1", "2", "3",

[jira] [Commented] (SPARK-43053) Possible logic issue

2023-04-06 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43053?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17709536#comment-17709536 ] Yuming Wang commented on SPARK-43053: - It's not a issue. > Possible logic issue >

[jira] [Created] (SPARK-43050) Fix construct aggregate expressions by replacing grouping functions

2023-04-06 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-43050: --- Summary: Fix construct aggregate expressions by replacing grouping functions Key: SPARK-43050 URL: https://issues.apache.org/jira/browse/SPARK-43050 Project: Spark

[jira] [Created] (PARQUET-2263) Upgrade maven-shade-plugin to 3.4.1

2023-04-04 Thread Yuming Wang (Jira)
Yuming Wang created PARQUET-2263: Summary: Upgrade maven-shade-plugin to 3.4.1 Key: PARQUET-2263 URL: https://issues.apache.org/jira/browse/PARQUET-2263 Project: Parquet Issue Type:

[jira] [Updated] (SPARK-42926) Upgrade Parquet to 1.13.0

2023-04-02 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42926?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-42926: Summary: Upgrade Parquet to 1.13.0 (was: Upgrade Parquet to 1.12.4) > Upgrade Parquet to 1.13.0

[jira] [Created] (SPARK-43000) Do not cast to double type if one side is AnsiIntervalType in BinaryArithmetic

2023-04-01 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-43000: --- Summary: Do not cast to double type if one side is AnsiIntervalType in BinaryArithmetic Key: SPARK-43000 URL: https://issues.apache.org/jira/browse/SPARK-43000

[jira] [Updated] (SPARK-42991) Disable string type +/- interval in ANSI mode

2023-03-31 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42991?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-42991: Summary: Disable string type +/- interval in ANSI mode (was: Disable string +/- interval in ANSI

[jira] [Commented] (SPARK-42991) Disable string +/- interval in ANSI mode

2023-03-31 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42991?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17707158#comment-17707158 ] Yuming Wang commented on SPARK-42991: - https://github.com/apache/spark/pull/40616 > Disable string

[jira] [Created] (SPARK-42991) Disable string +/- interval in ANSI mode

2023-03-31 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-42991: --- Summary: Disable string +/- interval in ANSI mode Key: SPARK-42991 URL: https://issues.apache.org/jira/browse/SPARK-42991 Project: Spark Issue Type:

[jira] [Created] (SPARK-42975) Cast result type to timestamp type for string +/- interval

2023-03-30 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-42975: --- Summary: Cast result type to timestamp type for string +/- interval Key: SPARK-42975 URL: https://issues.apache.org/jira/browse/SPARK-42975 Project: Spark

[jira] [Commented] (SPARK-42932) Spark 3.3.2, with hadoop3, Error with java.io.IOException: Mkdirs failed to create file

2023-03-29 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42932?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17706702#comment-17706702 ] Yuming Wang commented on SPARK-42932: - I can't reproduce this issue. Maybe it's your disk issue? >

[jira] [Resolved] (SPARK-40610) Spark fall back to use getPartitions instead of getPartitionsByFilter when date_add functions used in where clause

2023-03-29 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40610?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-40610. - Resolution: Won't Fix > Spark fall back to use getPartitions instead of getPartitionsByFilter

[jira] [Commented] (SPARK-40610) Spark fall back to use getPartitions instead of getPartitionsByFilter when date_add functions used in where clause

2023-03-29 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40610?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17706665#comment-17706665 ] Yuming Wang commented on SPARK-40610: - Users can {{set

[jira] [Commented] (HIVE-27176) EXPLAIN SKEW

2023-03-29 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/HIVE-27176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17706227#comment-17706227 ] Yuming Wang commented on HIVE-27176: +1. Our internal Spark also supports similar feature:

[jira] [Reopened] (SPARK-42832) Remove repartition if it is the child of LocalLimit

2023-03-29 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42832?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang reopened SPARK-42832: - Assignee: (was: Yuming Wang) > Remove repartition if it is the child of LocalLimit >

[jira] [Updated] (SPARK-42832) Remove repartition if it is the child of LocalLimit

2023-03-29 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42832?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-42832: Fix Version/s: (was: 3.5.0) > Remove repartition if it is the child of LocalLimit >

[jira] [Resolved] (SPARK-42832) Remove repartition if it is the child of LocalLimit

2023-03-29 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42832?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-42832. - Resolution: Won't Do > Remove repartition if it is the child of LocalLimit >

[jira] [Updated] (SPARK-42935) Optimze shuffle for union spark plan

2023-03-27 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42935?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-42935: Fix Version/s: (was: 3.5.0) > Optimze shuffle for union spark plan >

[jira] [Created] (SPARK-42926) Upgrade Parquet to 1.12.4

2023-03-26 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-42926: --- Summary: Upgrade Parquet to 1.12.4 Key: SPARK-42926 URL: https://issues.apache.org/jira/browse/SPARK-42926 Project: Spark Issue Type: Improvement

[jira] [Resolved] (SPARK-33307) Refactor GROUPING ANALYTICS

2023-03-21 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33307?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-33307. - Fix Version/s: 3.2.0 Assignee: angerszhu Resolution: Fixed > Refactor GROUPING

[jira] [Created] (SPARK-42868) Support eliminate sorts in AQE Optimizer

2023-03-20 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-42868: --- Summary: Support eliminate sorts in AQE Optimizer Key: SPARK-42868 URL: https://issues.apache.org/jira/browse/SPARK-42868 Project: Spark Issue Type:

[jira] (SPARK-42760) The partition of result data frame of join is always 1

2023-03-17 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42760 ] Yuming Wang deleted comment on SPARK-42760: - was (Author: apachespark): User '1511351836' has created a pull request for this issue: https://github.com/apache/spark/pull/40380 > The

[jira] [Created] (SPARK-42832) Remove repartition if it is the child of LocalLimit

2023-03-16 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-42832: --- Summary: Remove repartition if it is the child of LocalLimit Key: SPARK-42832 URL: https://issues.apache.org/jira/browse/SPARK-42832 Project: Spark Issue

[jira] [Commented] (SPARK-42804) when target table format is textfile using `insert into select` will got error

2023-03-16 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42804?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17701006#comment-17701006 ] Yuming Wang commented on SPARK-42804: - I can't reproduce it. Did you set any configs? > when target

[jira] [Commented] (SPARK-42804) when target table format is textfile using `insert into select` will got error

2023-03-15 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42804?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17700984#comment-17700984 ] Yuming Wang commented on SPARK-42804: - How to reproduce? > when target table format is textfile

[jira] [Comment Edited] (SPARK-42513) Push down topK through join

2023-03-15 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42513?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17700945#comment-17700945 ] Yuming Wang edited comment on SPARK-42513 at 3/16/23 3:12 AM: -- TiDB({{mysql

[jira] [Commented] (SPARK-42513) Push down topK through join

2023-03-15 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42513?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17700945#comment-17700945 ] Yuming Wang commented on SPARK-42513: - TiDB: {code:sql} create table t1(a int, b int) ; create

[jira] [Resolved] (SPARK-42804) when target table format is textfile using `insert into select` will got error

2023-03-15 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42804?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-42804. - Resolution: Invalid It looks like this is a problem of your metastore. > when target table

[jira] [Updated] (SPARK-42789) Rewrite multiple GetJsonObjects to a JsonTuple if their json expression is the same

2023-03-14 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42789?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-42789: Description: Benchmark result: {noformat} Running benchmark: Benchmark rewrite GetJsonObjects

[jira] [Updated] (SPARK-42789) Rewrite multiple GetJsonObjects to a JsonTuple if their json expression is the same

2023-03-14 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42789?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-42789: Summary: Rewrite multiple GetJsonObjects to a JsonTuple if their json expression is the same

[jira] [Updated] (SPARK-42789) rewrites multiple GetJsonObjects to a JsonTuple if their json expression is the same

2023-03-14 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42789?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-42789: Description: Benchmark result: {noformat} Running benchmark: Benchmark rewrite GetJsonObjects

[jira] [Created] (SPARK-42789) rewrites multiple GetJsonObjects to a JsonTuple if their json expression is the same

2023-03-14 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-42789: --- Summary: rewrites multiple GetJsonObjects to a JsonTuple if their json expression is the same Key: SPARK-42789 URL: https://issues.apache.org/jira/browse/SPARK-42789

[jira] [Assigned] (SPARK-42741) Do not unwrap casts in binary comparison when literal is null

2023-03-14 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42741?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang reassigned SPARK-42741: --- Assignee: Yuming Wang > Do not unwrap casts in binary comparison when literal is null >

[jira] [Resolved] (SPARK-42741) Do not unwrap casts in binary comparison when literal is null

2023-03-14 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42741?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-42741. - Fix Version/s: 3.5.0 Resolution: Fixed Issue resolved by pull request 40360

[jira] [Updated] (SPARK-42782) Port the tests for get_json_object from the Apache Hive project

2023-03-14 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42782?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-42782: Summary: Port the tests for get_json_object from the Apache Hive project (was: Port TestUDFJson

[jira] [Created] (SPARK-42782) Port TestUDFJson from Hive

2023-03-14 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-42782: --- Summary: Port TestUDFJson from Hive Key: SPARK-42782 URL: https://issues.apache.org/jira/browse/SPARK-42782 Project: Spark Issue Type: Improvement

[jira] [Updated] (SPARK-42597) Support unwrap date type to timestamp type

2023-03-13 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42597?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-42597: Summary: Support unwrap date type to timestamp type (was: UnwrapCastInBinaryComparison support

[jira] [Resolved] (SPARK-42597) UnwrapCastInBinaryComparison support unwrap timestamp type

2023-03-13 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42597?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-42597. - Fix Version/s: 3.5.0 Resolution: Fixed Issue resolved by pull request 40190

[jira] [Assigned] (SPARK-42597) UnwrapCastInBinaryComparison support unwrap timestamp type

2023-03-13 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42597?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang reassigned SPARK-42597: --- Assignee: Yuming Wang > UnwrapCastInBinaryComparison support unwrap timestamp type >

[jira] [Commented] (SPARK-42760) The partition of result data frame of join is always 1

2023-03-11 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42760?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17699291#comment-17699291 ] Yuming Wang commented on SPARK-42760: - Could you try to disable AQE(set spark.sql.adaptive.enabled =

[jira] [Commented] (SPARK-42749) CAST(x as int) does not generate error with overflow

2023-03-10 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42749?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17699137#comment-17699137 ] Yuming Wang commented on SPARK-42749: - Please enable ansi: {code:sql} spark-sql (default)> set

[jira] [Created] (SPARK-42741) Do not unwrap casts in binary comparison when literal is null

2023-03-09 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-42741: --- Summary: Do not unwrap casts in binary comparison when literal is null Key: SPARK-42741 URL: https://issues.apache.org/jira/browse/SPARK-42741 Project: Spark

[jira] [Assigned] (SPARK-42696) Speed up parquet reading with Java Vector API

2023-03-07 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42696?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang reassigned SPARK-42696: --- Assignee: jiangjiguang0719 (was: Yuming Wang) > Speed up parquet reading with Java Vector

[jira] [Assigned] (SPARK-42696) Speed up parquet reading with Java Vector API

2023-03-07 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42696?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang reassigned SPARK-42696: --- Assignee: Yuming Wang > Speed up parquet reading with Java Vector API >

[jira] [Commented] (SPARK-42694) Data duplication and loss occur after executing 'insert overwrite...' in Spark 3.1.1

2023-03-07 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42694?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17697316#comment-17697316 ] Yuming Wang commented on SPARK-42694: - Could you upgrade to Spark 3.1.3 or Spark 3.3.2? > Data

[jira] [Resolved] (SPARK-42682) When I execute the spark-shell command, "WARN ui.JettyUtils: GET /jobs/ failed: java.util.NoSuchElementException: Failed to get the application information. If you are

2023-03-06 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42682?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-42682. - Resolution: Invalid This is because the spark context is not fully started. > When I execute

[jira] [Resolved] (SPARK-34145) Combine scalar subqueries

2023-03-05 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34145?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-34145. - Resolution: Not A Problem Issue fixed through SPARK-34079. > Combine scalar subqueries >

[jira] [Resolved] (SPARK-34117) Disable LeftSemi/LeftAnti push down over Aggregate

2023-03-05 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34117?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-34117. - Resolution: Not A Problem Issue fixed through SPARK-34081. > Disable LeftSemi/LeftAnti push

[jira] [Commented] (SPARK-42650) link issue SPARK-42550

2023-03-05 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42650?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17696665#comment-17696665 ] Yuming Wang commented on SPARK-42650: - It seems to be caused by

[jira] [Resolved] (SPARK-42675) Should clean up temp view after test

2023-03-05 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42675?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-42675. - Fix Version/s: 3.4.0 Assignee: Yang Jie (was: Apache Spark) Resolution: Fixed

[jira] [Commented] (SPARK-40610) Spark fall back to use getPartitions instead of getPartitionsByFilter when date_add functions used in where clause

2023-03-05 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40610?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17696634#comment-17696634 ] Yuming Wang commented on SPARK-40610: - This is because of

[jira] [Resolved] (SPARK-42473) An explicit cast will be needed when INSERT OVERWRITE SELECT UNION ALL

2023-03-02 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42473?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-42473. - Fix Version/s: 3.3.3 Assignee: Runyao.Chen Resolution: Fixed > An explicit cast

[jira] [Commented] (SPARK-42650) link issue SPARK-42550

2023-03-02 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42650?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17695989#comment-17695989 ] Yuming Wang commented on SPARK-42650: - It seems like a Kyuubi bug? > link issue SPARK-42550 >

[jira] [Updated] (SPARK-42597) UnwrapCastInBinaryComparison support unwrap timestamp type

2023-02-27 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42597?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-42597: Summary: UnwrapCastInBinaryComparison support unwrap timestamp type (was:

[jira] [Created] (SPARK-42597) UnwrapCastInBinaryComparison also support timestamp type

2023-02-27 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-42597: --- Summary: UnwrapCastInBinaryComparison also support timestamp type Key: SPARK-42597 URL: https://issues.apache.org/jira/browse/SPARK-42597 Project: Spark Issue

[jira] [Resolved] (SPARK-42594) spark can not read lastest view sql when run `create or replace view` by hive

2023-02-26 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42594?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-42594. - Resolution: Not A Bug > spark can not read lastest view sql when run `create or replace view`

[jira] [Reopened] (SPARK-42594) spark can not read lastest view sql when run `create or replace view` by hive

2023-02-26 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42594?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang reopened SPARK-42594: - > spark can not read lastest view sql when run `create or replace view` by hive >

[jira] [Commented] (SPARK-42594) spark can not read lastest view sql when run `create or replace view` by hive

2023-02-26 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42594?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17693799#comment-17693799 ] Yuming Wang commented on SPARK-42594: - Spark saves information to table properties, Hive does not

[jira] [Updated] (SPARK-42583) Remove outer join if all aggregate functions are distinct

2023-02-26 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42583?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-42583: Description: To support more cases:

[jira] [Created] (SPARK-42583) Remove outer join if all aggregate functions are distinct

2023-02-26 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-42583: --- Summary: Remove outer join if all aggregate functions are distinct Key: SPARK-42583 URL: https://issues.apache.org/jira/browse/SPARK-42583 Project: Spark

[jira] [Resolved] (SPARK-42525) collapse two adjacent windows with the same partition/order in subquery

2023-02-25 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42525?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-42525. - Fix Version/s: 3.5.0 Assignee: zhuml Resolution: Fixed > collapse two adjacent

[jira] [Resolved] (SPARK-42550) table directory will lost on hdfs when `INSERT OVERWRITE` faild

2023-02-25 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42550?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-42550. - Resolution: Cannot Reproduce > table directory will lost on hdfs when `INSERT OVERWRITE` faild

[jira] [Commented] (SPARK-42550) table directory will lost on hdfs when `INSERT OVERWRITE` faild

2023-02-24 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42550?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17693103#comment-17693103 ] Yuming Wang commented on SPARK-42550: - I can't reproduce this issue: {noformat} spark-sql> select

[jira] [Commented] (SPARK-42550) table directory will lost on hdfs when `INSERT OVERWRITE` faild

2023-02-23 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42550?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17693016#comment-17693016 ] Yuming Wang commented on SPARK-42550: - Please refresh table: {code:sql} refresh table

[jira] [Updated] (SPARK-42550) table directory will lost on hdfs when `INSERT OVERWRITE` faild

2023-02-23 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42550?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-42550: Description: {color:#4c9aff}when a  `{*}INSERT{*} OVERWRITE *TABLE`* statment faild during

[jira] [Updated] (SPARK-40158) Remove useless configuration & extract common code for parquet read

2023-02-21 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40158?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-40158: Fix Version/s: (was: 3.4.0) > Remove useless configuration & extract common code for parquet

[jira] [Updated] (SPARK-40355) Improve pushdown for orc & parquet when cast scenario

2023-02-21 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40355?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-40355: Fix Version/s: (was: 3.4.0) > Improve pushdown for orc & parquet when cast scenario >

[jira] [Updated] (SPARK-42513) Push down topK through join

2023-02-21 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42513?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-42513: Summary: Push down topK through join (was: Push topK through join) > Push down topK through join

[jira] [Updated] (SPARK-42513) Push topK through join

2023-02-21 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42513?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-42513: Attachment: before-UI.png > Push topK through join > -- > >

[jira] [Updated] (SPARK-42513) Push topK through join

2023-02-21 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42513?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-42513: Attachment: after-UI.png > Push topK through join > -- > >

[jira] [Created] (SPARK-42513) Push topK through join

2023-02-21 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-42513: --- Summary: Push topK through join Key: SPARK-42513 URL: https://issues.apache.org/jira/browse/SPARK-42513 Project: Spark Issue Type: Improvement

[jira] [Resolved] (SPARK-42506) Fix Sort's maxRowsPerPartition if maxRows does not exist

2023-02-21 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42506?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-42506. - Resolution: Not A Problem > Fix Sort's maxRowsPerPartition if maxRows does not exist >

[jira] [Created] (SPARK-42506) Fix Sort's maxRowsPerPartition if maxRows does not exist

2023-02-20 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-42506: --- Summary: Fix Sort's maxRowsPerPartition if maxRows does not exist Key: SPARK-42506 URL: https://issues.apache.org/jira/browse/SPARK-42506 Project: Spark Issue

[jira] [Commented] (SPARK-40278) Used databricks spark-sql-pref with Spark 3.3 to run 3TB tpcds test failed

2023-02-20 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40278?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17691388#comment-17691388 ] Yuming Wang commented on SPARK-40278: - [~LuciferYang] Is this issue still exist? > Used databricks

[jira] [Commented] (SPARK-40610) Spark fall back to use getPartitions instead of getPartitionsByFilter when date_add functions used in where clause

2023-02-20 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40610?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17691387#comment-17691387 ] Yuming Wang commented on SPARK-40610: - [~icyjhl] What's your dt data type? date, string or

[jira] [Commented] (SPARK-42503) Spark SQL should do further validation on join condition fields

2023-02-20 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42503?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17691321#comment-17691321 ] Yuming Wang commented on SPARK-42503: - Do other databases also have this validation? > Spark SQL

[jira] [Updated] (SPARK-42503) Spark SQL should do further validation on join condition fields

2023-02-20 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42503?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-42503: Fix Version/s: (was: 3.4.0) > Spark SQL should do further validation on join condition fields

[jira] [Updated] (SPARK-42503) Spark SQL should do further validation on join condition fields

2023-02-20 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42503?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-42503: Target Version/s: (was: 3.4.0) > Spark SQL should do further validation on join condition

[jira] [Resolved] (SPARK-41741) [SQL] ParquetFilters StringStartsWith push down matching string do not use UTF-8

2023-02-20 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41741?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-41741. - Resolution: Fixed > [SQL] ParquetFilters StringStartsWith push down matching string do not use

[jira] [Assigned] (SPARK-41741) [SQL] ParquetFilters StringStartsWith push down matching string do not use UTF-8

2023-02-20 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41741?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang reassigned SPARK-41741: --- Fix Version/s: 3.4.0 3.3.3 Assignee: Yuming Wang > [SQL]

[jira] [Created] (SPARK-42500) ConstantPropagation support more cases

2023-02-20 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-42500: --- Summary: ConstantPropagation support more cases Key: SPARK-42500 URL: https://issues.apache.org/jira/browse/SPARK-42500 Project: Spark Issue Type: Improvement

[jira] [Assigned] (SPARK-42286) Fix internal error for valid CASE WHEN expression with CAST when inserting into a table

2023-02-19 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42286?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang reassigned SPARK-42286: --- Fix Version/s: 3.4.0 Assignee: Runyao.Chen > Fix internal error for valid CASE

[jira] [Commented] (SPARK-42473) An explicit cast will be needed when INSERT OVERWRITE SELECT UNION ALL

2023-02-19 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42473?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17691011#comment-17691011 ] Yuming Wang commented on SPARK-42473: - It seems we should backport

[jira] [Commented] (SPARK-42473) An explicit cast will be needed when INSERT OVERWRITE SELECT UNION ALL

2023-02-18 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42473?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17690833#comment-17690833 ] Yuming Wang commented on SPARK-42473: - What is your test.spark33_decimal_orc column type? > An

[jira] [Assigned] (SPARK-39851) Improve join stats estimation if one side can keep uniqueness

2023-02-17 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39851?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang reassigned SPARK-39851: --- Assignee: Wan Kun > Improve join stats estimation if one side can keep uniqueness >

[jira] [Updated] (SPARK-42450) dataset.where() omit quotes if where IN clause has more than 10 operands

2023-02-16 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42450?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-42450: Fix Version/s: (was: 3.4.0) (was: 3.3.2) > dataset.where() omit quotes

[jira] [Updated] (SPARK-42450) dataset.where() omit quotes if where IN clause has more than 10 operands

2023-02-16 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42450?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-42450: Target Version/s: (was: 3.3.0) > dataset.where() omit quotes if where IN clause has more than

[jira] [Assigned] (SPARK-42451) Remove 3.1 and Java 17 check from filter condition of `testingVersions` in `HiveExternalCatalogVersionsSuite`

2023-02-15 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42451?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang reassigned SPARK-42451: --- Assignee: Yang Jie > Remove 3.1 and Java 17 check from filter condition of

[jira] [Resolved] (SPARK-42451) Remove 3.1 and Java 17 check from filter condition of `testingVersions` in `HiveExternalCatalogVersionsSuite`

2023-02-15 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42451?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-42451. - Fix Version/s: 3.5.0 Resolution: Fixed Issue resolved by pull request 40039

[jira] [Resolved] (SPARK-42346) distinct(count colname) with UNION ALL causes query analyzer bug

2023-02-06 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42346?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-42346. - Fix Version/s: 3.3.2 3.4.0 3.5.0 Assignee: Peter

<    1   2   3   4   5   6   7   8   9   10   >