[jira] [Assigned] (SPARK-39511) Push limit 1 to right side if join type is LeftSemiOrAnti and join condition is empty

2022-06-23 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39511?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang reassigned SPARK-39511: --- Assignee: Yuming Wang > Push limit 1 to right side if join type is LeftSemiOrAnti and join

[jira] [Resolved] (SPARK-39511) Push limit 1 to right side if join type is LeftSemiOrAnti and join condition is empty

2022-06-23 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39511?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-39511. - Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 36909 [https://gith

[jira] [Resolved] (SPARK-38614) After Spark update, df.show() shows incorrect F.percent_rank results

2022-06-22 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38614?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-38614. - Fix Version/s: 3.2.2 3.4.0 3.3.1 Assignee: Bruce R

[jira] [Resolved] (SPARK-39556) Propagate empty relation if maxRows is 0

2022-06-22 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39556?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-39556. - Resolution: Not A Problem It already supported through OptimizeLimitZero and PropagateEmptyRelat

[jira] [Created] (SPARK-39556) Propagate empty relation if maxRows is 0

2022-06-22 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-39556: --- Summary: Propagate empty relation if maxRows is 0 Key: SPARK-39556 URL: https://issues.apache.org/jira/browse/SPARK-39556 Project: Spark Issue Type: Improvemen

[jira] [Commented] (SPARK-38614) After Spark update, df.show() shows incorrect F.percent_rank results

2022-06-21 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38614?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17557179#comment-17557179 ] Yuming Wang commented on SPARK-38614: - Thank you for reporting this issue. Workaroun

[jira] [Updated] (SPARK-39539) Overflow on converting valid Milliseconds to Microseconds

2022-06-20 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39539?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-39539: Fix Version/s: (was: 3.2.0) > Overflow on converting valid Milliseconds to Microseconds >

[jira] [Created] (SPARK-39535) Make SchemaPruning only pruning if it contains nested column

2022-06-20 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-39535: --- Summary: Make SchemaPruning only pruning if it contains nested column Key: SPARK-39535 URL: https://issues.apache.org/jira/browse/SPARK-39535 Project: Spark

[jira] [Assigned] (SPARK-39445) Remove the window if windowExpressions is empty in column pruning

2022-06-20 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39445?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang reassigned SPARK-39445: --- Assignee: Yuming Wang > Remove the window if windowExpressions is empty in column pruning >

[jira] [Resolved] (SPARK-39445) Remove the window if windowExpressions is empty in column pruning

2022-06-20 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39445?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-39445. - Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 36842 [https://gith

[jira] [Resolved] (SPARK-39530) Fix KafkaTestUtils to support IPv6

2022-06-20 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39530?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-39530. - Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 36923 [https://gith

[jira] [Assigned] (SPARK-39530) Fix KafkaTestUtils to support IPv6

2022-06-20 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39530?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang reassigned SPARK-39530: --- Assignee: Dongjoon Hyun > Fix KafkaTestUtils to support IPv6 >

[jira] [Commented] (SPARK-39481) Pandas UDF executed twice if used in projection followed by filter

2022-06-19 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39481?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17556219#comment-17556219 ] Yuming Wang commented on SPARK-39481: - Copy the code from description: {code:python}

[jira] [Commented] (SPARK-39513) HiveMetastore serializable exception with Spark 3.3.0

2022-06-19 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39513?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17556050#comment-17556050 ] Yuming Wang commented on SPARK-39513: - [~tcarland] What is your java version? > Hiv

[jira] [Resolved] (SPARK-39514) LauncherBackendSuite should add java.net.preferIPv6Addresses conf

2022-06-19 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39514?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-39514. - Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 36911 [https://gith

[jira] [Assigned] (SPARK-39514) LauncherBackendSuite should add java.net.preferIPv6Addresses conf

2022-06-19 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39514?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang reassigned SPARK-39514: --- Assignee: Dongjoon Hyun > LauncherBackendSuite should add java.net.preferIPv6Addresses conf

[jira] [Created] (SPARK-39511) Push limit 1 to right side if join type is LeftSemiOrAnti and join condition is empty

2022-06-18 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-39511: --- Summary: Push limit 1 to right side if join type is LeftSemiOrAnti and join condition is empty Key: SPARK-39511 URL: https://issues.apache.org/jira/browse/SPARK-39511 P

[jira] [Created] (SPARK-39449) Propagate empty relation through Window

2022-06-11 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-39449: --- Summary: Propagate empty relation through Window Key: SPARK-39449 URL: https://issues.apache.org/jira/browse/SPARK-39449 Project: Spark Issue Type: Improvement

[jira] [Updated] (SPARK-39448) Add ReplaceCTERefWithRepartition into nonExcludableRules list

2022-06-11 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39448?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-39448: Description: A unit test to test excluded rules: {code:scala} /* * Licensed to the Apache Softwar

[jira] [Updated] (SPARK-39448) Add ReplaceCTERefWithRepartition into nonExcludableRules list

2022-06-11 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39448?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-39448: Description: A unit test to test excluded rules: {code:scala} /* * Licensed to the Apache Softwar

[jira] [Created] (SPARK-39448) Add ReplaceCTERefWithRepartition into nonExcludableRules list

2022-06-11 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-39448: --- Summary: Add ReplaceCTERefWithRepartition into nonExcludableRules list Key: SPARK-39448 URL: https://issues.apache.org/jira/browse/SPARK-39448 Project: Spark

[jira] [Updated] (SPARK-39447) Only non-broadcast query stage can propagate empty relation

2022-06-11 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39447?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-39447: Summary: Only non-broadcast query stage can propagate empty relation (was: Only non-broadcast can

[jira] [Created] (SPARK-39447) Only non-broadcast can propagate empty relation

2022-06-11 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-39447: --- Summary: Only non-broadcast can propagate empty relation Key: SPARK-39447 URL: https://issues.apache.org/jira/browse/SPARK-39447 Project: Spark Issue Type: Bug

[jira] [Assigned] (SPARK-39325) Improve MapOutputTracker convertMapStatuses performance

2022-06-11 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39325?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang reassigned SPARK-39325: --- Assignee: Wan Kun > Improve MapOutputTracker convertMapStatuses performance > -

[jira] [Resolved] (SPARK-39325) Improve MapOutputTracker convertMapStatuses performance

2022-06-11 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39325?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-39325. - Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 36709 [https://gith

[jira] [Resolved] (SPARK-39436) graph.connectedComponents(maxIterations) get ArrayIndexOutOfBoundsException: -1

2022-06-11 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39436?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-39436. - Resolution: Duplicate > graph.connectedComponents(maxIterations) get ArrayIndexOutOfBoundsExcept

[jira] [Created] (SPARK-39445) Remove the window if windowExpressions is empty in column pruning

2022-06-10 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-39445: --- Summary: Remove the window if windowExpressions is empty in column pruning Key: SPARK-39445 URL: https://issues.apache.org/jira/browse/SPARK-39445 Project: Spark

[jira] [Updated] (SPARK-39444) Add OptimizeSubqueries into nonExcludableRules list

2022-06-10 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39444?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-39444: Issue Type: Bug (was: Improvement) > Add OptimizeSubqueries into nonExcludableRules list > --

[jira] [Created] (SPARK-39444) Add OptimizeSubqueries into nonExcludableRules list

2022-06-10 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-39444: --- Summary: Add OptimizeSubqueries into nonExcludableRules list Key: SPARK-39444 URL: https://issues.apache.org/jira/browse/SPARK-39444 Project: Spark Issue Type:

[jira] [Created] (SPARK-39440) Add a config to disable event timeline

2022-06-10 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-39440: --- Summary: Add a config to disable event timeline Key: SPARK-39440 URL: https://issues.apache.org/jira/browse/SPARK-39440 Project: Spark Issue Type: Improvement

[jira] [Assigned] (SPARK-39428) use code block for `Coalesce Hints for SQL Queries`

2022-06-10 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39428?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang reassigned SPARK-39428: --- Assignee: qian > use code block for `Coalesce Hints for SQL Queries` >

[jira] [Resolved] (SPARK-39428) use code block for `Coalesce Hints for SQL Queries`

2022-06-10 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39428?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-39428. - Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 36820 [https://gith

[jira] [Created] (SPARK-39438) Add a threshold to not in line CTE

2022-06-09 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-39438: --- Summary: Add a threshold to not in line CTE Key: SPARK-39438 URL: https://issues.apache.org/jira/browse/SPARK-39438 Project: Spark Issue Type: Improvement

[jira] [Resolved] (SPARK-39400) spark-sql remain hive resource download dir after exit

2022-06-08 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39400?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-39400. - Resolution: Fixed Issue resolved by pull request 36786 [https://github.com/apache/spark/pull/367

[jira] [Assigned] (SPARK-39400) spark-sql remain hive resource download dir after exit

2022-06-08 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39400?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang reassigned SPARK-39400: --- Assignee: angerszhu > spark-sql remain hive resource download dir after exit >

[jira] [Created] (SPARK-39408) Update the buildKeys for DynamicPruningSubquery.withNewPlan

2022-06-07 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-39408: --- Summary: Update the buildKeys for DynamicPruningSubquery.withNewPlan Key: SPARK-39408 URL: https://issues.apache.org/jira/browse/SPARK-39408 Project: Spark I

[jira] [Created] (SPARK-39402) Optimize ReplaceCTERefWithRepartition

2022-06-07 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-39402: --- Summary: Optimize ReplaceCTERefWithRepartition Key: SPARK-39402 URL: https://issues.apache.org/jira/browse/SPARK-39402 Project: Spark Issue Type: Improvement

[jira] [Created] (SPARK-39401) Replace withView with withTempView in CTEInlineSuite

2022-06-07 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-39401: --- Summary: Replace withView with withTempView in CTEInlineSuite Key: SPARK-39401 URL: https://issues.apache.org/jira/browse/SPARK-39401 Project: Spark Issue Type

[jira] [Resolved] (SPARK-39395) Spark Thriftserver enabled LDAP,Error using beeline connection: error code 49 - invalid credentials

2022-06-07 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39395?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-39395. - Resolution: Duplicate > Spark Thriftserver enabled LDAP,Error using beeline connection: error co

[jira] [Commented] (SPARK-39386) Flaky Test: BloomFilterAggregateQuerySuite

2022-06-06 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39386?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17550334#comment-17550334 ] Yuming Wang commented on SPARK-39386: - [~dongjoon] Could you provide your environmen

[jira] [Commented] (SPARK-34631) Caught Hive MetaException when query by partition (partition col start with ‘$’)

2022-06-05 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34631?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17550298#comment-17550298 ] Yuming Wang commented on SPARK-34631: - Please backport SPARK-36137 to workaround thi

[jira] [Assigned] (SPARK-39377) Normalize expr ids in ListQuery and Exists expressions

2022-06-05 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39377?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang reassigned SPARK-39377: --- Assignee: Yuming Wang > Normalize expr ids in ListQuery and Exists expressions > --

[jira] [Resolved] (SPARK-39377) Normalize expr ids in ListQuery and Exists expressions

2022-06-05 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39377?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-39377. - Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 36764 [https://gith

[jira] [Created] (SPARK-39377) Normalize expr ids in ListQuery and Exists expressions

2022-06-03 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-39377: --- Summary: Normalize expr ids in ListQuery and Exists expressions Key: SPARK-39377 URL: https://issues.apache.org/jira/browse/SPARK-39377 Project: Spark Issue Ty

[jira] [Resolved] (SPARK-39368) Move RewritePredicateSubquery into InjectRuntimeFilter

2022-06-03 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39368?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-39368. - Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 36755 [https://gith

[jira] [Assigned] (SPARK-39368) Move RewritePredicateSubquery into InjectRuntimeFilter

2022-06-03 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39368?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang reassigned SPARK-39368: --- Assignee: Yuming Wang > Move RewritePredicateSubquery into InjectRuntimeFilter > --

[jira] [Created] (SPARK-39374) Improve error message for user specified column list

2022-06-03 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-39374: --- Summary: Improve error message for user specified column list Key: SPARK-39374 URL: https://issues.apache.org/jira/browse/SPARK-39374 Project: Spark Issue Type

[jira] [Commented] (SPARK-29260) Enable supported Hive metastore versions once it support altering database location

2022-06-02 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29260?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17545726#comment-17545726 ] Yuming Wang commented on SPARK-29260: - Please backport HIVE-8472 to your Hive metast

[jira] [Resolved] (SPARK-29260) Enable supported Hive metastore versions once it support altering database location

2022-06-02 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29260?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-29260. - Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 36750 [https://gith

[jira] [Assigned] (SPARK-29260) Enable supported Hive metastore versions once it support altering database location

2022-06-02 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29260?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang reassigned SPARK-29260: --- Assignee: Chao Sun > Enable supported Hive metastore versions once it support altering data

[jira] [Commented] (SPARK-39368) Move RewritePredicateSubquery into InjectRuntimeFilter

2022-06-02 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39368?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17545719#comment-17545719 ] Yuming Wang commented on SPARK-39368: - https://github.com/apache/spark/pull/36755 >

[jira] [Created] (SPARK-39368) Move RewritePredicateSubquery into InjectRuntimeFilter

2022-06-02 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-39368: --- Summary: Move RewritePredicateSubquery into InjectRuntimeFilter Key: SPARK-39368 URL: https://issues.apache.org/jira/browse/SPARK-39368 Project: Spark Issue Ty

[jira] [Commented] (SPARK-29260) Enable supported Hive metastore versions once it support altering database location

2022-06-01 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29260?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17545264#comment-17545264 ] Yuming Wang commented on SPARK-29260: - The syntax supports it, but it doesn't change

[jira] [Commented] (SPARK-39354) The analysis exception is incorrect

2022-06-01 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39354?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17544914#comment-17544914 ] Yuming Wang commented on SPARK-39354: - Yes. It is only exception message related. >

[jira] [Commented] (SPARK-39354) The analysis exception is incorrect

2022-06-01 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39354?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17544894#comment-17544894 ] Yuming Wang commented on SPARK-39354: - [~maxgekk], I think this is a blocker issue f

[jira] [Created] (SPARK-39354) The analysis exception is incorrect

2022-06-01 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-39354: --- Summary: The analysis exception is incorrect Key: SPARK-39354 URL: https://issues.apache.org/jira/browse/SPARK-39354 Project: Spark Issue Type: Bug C

[jira] [Resolved] (SPARK-39338) Remove dynamic pruning subquery if pruningKey's references is empty

2022-05-31 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39338?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-39338. - Fix Version/s: 3.4.0 Assignee: Yuming Wang Resolution: Fixed > Remove dynamic pr

[jira] [Updated] (SPARK-39203) Fix remote table location based on database location

2022-05-31 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39203?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-39203: Affects Version/s: 3.2.0 3.1.1 3.1.0

[jira] [Resolved] (SPARK-39203) Fix remote table location based on database location

2022-05-31 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39203?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-39203. - Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 36625 [https://gith

[jira] [Assigned] (SPARK-39203) Fix remote table location based on database location

2022-05-31 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39203?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang reassigned SPARK-39203: --- Assignee: Yuming Wang > Fix remote table location based on database location >

[jira] [Commented] (SPARK-37609) Transient StackOverflowError on DataFrame from Catalyst QueryPlan

2022-05-31 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17544204#comment-17544204 ] Yuming Wang commented on SPARK-37609: - [~angerszhuuu] How to reproduce this issue?

[jira] [Updated] (SPARK-39338) Remove dynamic pruning subquery if pruningKey's references is empty

2022-05-30 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39338?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-39338: Issue Type: Bug (was: Improvement) > Remove dynamic pruning subquery if pruningKey's references i

[jira] [Updated] (SPARK-39338) Remove dynamic pruning subquery if pruningKey's references is empty

2022-05-30 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39338?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-39338: Affects Version/s: 3.3.0 > Remove dynamic pruning subquery if pruningKey's references is empty > -

[jira] [Created] (SPARK-39338) Remove dynamic pruning subquery if pruningKey's references is empty

2022-05-29 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-39338: --- Summary: Remove dynamic pruning subquery if pruningKey's references is empty Key: SPARK-39338 URL: https://issues.apache.org/jira/browse/SPARK-39338 Project: Spark

[jira] [Resolved] (SPARK-39308) Upgrade parquet to 1.12.3

2022-05-26 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39308?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-39308. - Fix Version/s: 3.4.0 Assignee: Yuming Wang Resolution: Fixed Issue resolved by p

[jira] [Created] (SPARK-39308) Upgrade parquet to 1.12.3

2022-05-26 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-39308: --- Summary: Upgrade parquet to 1.12.3 Key: SPARK-39308 URL: https://issues.apache.org/jira/browse/SPARK-39308 Project: Spark Issue Type: Improvement Com

[jira] [Created] (SPARK-39277) Make Optimizer extends SQLConfHelper

2022-05-24 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-39277: --- Summary: Make Optimizer extends SQLConfHelper Key: SPARK-39277 URL: https://issues.apache.org/jira/browse/SPARK-39277 Project: Spark Issue Type: Improvement

[jira] [Comment Edited] (SPARK-38506) Push partial aggregation through join

2022-05-24 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38506?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17541486#comment-17541486 ] Yuming Wang edited comment on SPARK-38506 at 5/24/22 2:40 PM:

[jira] [Commented] (SPARK-38506) Push partial aggregation through join

2022-05-24 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38506?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17541486#comment-17541486 ] Yuming Wang commented on SPARK-38506: - Benchmark result: |SQL|Before(ms)|With Parti

[jira] [Created] (SPARK-39251) Simplify MultiLike if remainPatterns is empty

2022-05-22 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-39251: --- Summary: Simplify MultiLike if remainPatterns is empty Key: SPARK-39251 URL: https://issues.apache.org/jira/browse/SPARK-39251 Project: Spark Issue Type: Impro

[jira] [Commented] (SPARK-39241) Spark SQL 'Like' operator behaves wrongly while filtering on partitioned column after Spark 3.1

2022-05-22 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39241?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17540601#comment-17540601 ] Yuming Wang commented on SPARK-39241: - I can't reproduce this issue: {code:scala}

[jira] [Updated] (SPARK-39248) Decimal divide much slower than multiply

2022-05-21 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39248?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-39248: Description: How to reproduce this issue: {code:scala} import org.apache.spark.benchmark.Benc

[jira] [Created] (SPARK-39248) Decimal divide much slower than multiply

2022-05-21 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-39248: --- Summary: Decimal divide much slower than multiply Key: SPARK-39248 URL: https://issues.apache.org/jira/browse/SPARK-39248 Project: Spark Issue Type: Improvemen

[jira] [Commented] (SPARK-19609) Broadcast joins should pushdown join constraints as Filter to the larger relation

2022-05-19 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-19609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17539884#comment-17539884 ] Yuming Wang commented on SPARK-19609: - [~devict] Do you have benchmark number if pus

[jira] [Created] (SPARK-39217) Makes DPP support the pruning side has Union

2022-05-17 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-39217: --- Summary: Makes DPP support the pruning side has Union Key: SPARK-39217 URL: https://issues.apache.org/jira/browse/SPARK-39217 Project: Spark Issue Type: Improv

[jira] [Created] (SPARK-39203) Fix remote table location based on database location

2022-05-16 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-39203: --- Summary: Fix remote table location based on database location Key: SPARK-39203 URL: https://issues.apache.org/jira/browse/SPARK-39203 Project: Spark Issue Type

[jira] [Resolved] (SPARK-39120) Prunes the duplicate columns from child of UnaryNode

2022-05-16 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39120?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-39120. - Resolution: Won't Do > Prunes the duplicate columns from child of UnaryNode > --

[jira] [Updated] (SPARK-39173) The error message is different if disable broadcast join

2022-05-13 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39173?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-39173: Description: How to reproduce this issue: {code:scala} sql( """ |CREATE TABLE sto

[jira] [Updated] (SPARK-39173) The error message is different if disable broadcast join

2022-05-12 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39173?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-39173: Description: How to reproduce this issue: {code:scala} Seq(-1, 10L).foreach { broadcastThr

[jira] [Updated] (SPARK-39173) The error message is different if disable broadcast join

2022-05-12 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39173?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-39173: Description: How to reproduce this issue: {code:scala} Seq(-1, 10L).foreach { broadcastThr

[jira] [Created] (SPARK-39173) The error message is different if disable broadcast join

2022-05-12 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-39173: --- Summary: The error message is different if disable broadcast join Key: SPARK-39173 URL: https://issues.apache.org/jira/browse/SPARK-39173 Project: Spark Issue

[jira] [Updated] (SPARK-39120) Prunes the duplicate columns from child of UnaryNode

2022-05-06 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39120?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-39120: Description: {code:scala} sql("create table t1(a int, b int, c int) using parquet") sql("select a

[jira] [Created] (SPARK-39120) Prunes the duplicate columns from child of UnaryNode

2022-05-06 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-39120: --- Summary: Prunes the duplicate columns from child of UnaryNode Key: SPARK-39120 URL: https://issues.apache.org/jira/browse/SPARK-39120 Project: Spark Issue Type

[jira] [Updated] (SPARK-39112) UnsupportedOperationException if spark.sql.ui.explainMode is set to cost

2022-05-06 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39112?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-39112: Summary: UnsupportedOperationException if spark.sql.ui.explainMode is set to cost (was: Unsupport

[jira] [Created] (SPARK-39112) UnsupportedOperationException if set spark.sql.ui.explainMode to cost

2022-05-05 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-39112: --- Summary: UnsupportedOperationException if set spark.sql.ui.explainMode to cost Key: SPARK-39112 URL: https://issues.apache.org/jira/browse/SPARK-39112 Project: Spark

[jira] [Created] (SPARK-39069) Simplify another conditionals case in predicate

2022-04-29 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-39069: --- Summary: Simplify another conditionals case in predicate Key: SPARK-39069 URL: https://issues.apache.org/jira/browse/SPARK-39069 Project: Spark Issue Type: Imp

[jira] [Resolved] (SPARK-38797) Runtime Filter support pushdown through window

2022-04-13 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38797?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-38797. - Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 36080 [https://gith

[jira] [Assigned] (SPARK-38797) Runtime Filter support pushdown through window

2022-04-13 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38797?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang reassigned SPARK-38797: --- Assignee: Yuming Wang > Runtime Filter support pushdown through window > --

[jira] [Resolved] (SPARK-38565) Support Left Semi join in row level runtime filters

2022-04-11 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38565?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-38565. - Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 36131 [https://gith

[jira] [Assigned] (SPARK-38565) Support Left Semi join in row level runtime filters

2022-04-11 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38565?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang reassigned SPARK-38565: --- Assignee: Yuming Wang > Support Left Semi join in row level runtime filters > -

[jira] [Commented] (SPARK-38853) optimizeSkewsInRebalancePartitions has performance issue

2022-04-11 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38853?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17520345#comment-17520345 ] Yuming Wang commented on SPARK-38853: - Config: {noformat} spark.master

[jira] [Updated] (SPARK-38853) optimizeSkewsInRebalancePartitions has performance issue

2022-04-11 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38853?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-38853: Description: How to reproduce this issue: {code:sql} CREATE TABLE t USING PARQUET AS SELECT /*

[jira] [Updated] (SPARK-38853) optimizeSkewsInRebalancePartitions has performance issue

2022-04-11 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38853?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-38853: Description: How to reproduce this issue: {code:sql} CREATE TABLE t USING PARQUET AS SELECT /*

[jira] [Updated] (SPARK-38853) optimizeSkewsInRebalancePartitions has performance issue

2022-04-11 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38853?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-38853: Attachment: enable.png > optimizeSkewsInRebalancePartitions has performance issue > --

[jira] [Updated] (SPARK-38853) optimizeSkewsInRebalancePartitions has performance issue

2022-04-11 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38853?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-38853: Attachment: Disable.png > optimizeSkewsInRebalancePartitions has performance issue > -

[jira] [Updated] (SPARK-38853) optimizeSkewsInRebalancePartitions has performance issue

2022-04-11 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38853?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-38853: Description: How to reproduce this issue: {code:sql} CREATE TABLE t USING PARQUET AS SELECT /*

[jira] [Updated] (SPARK-38853) optimizeSkewsInRebalancePartitions has performance issue

2022-04-11 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38853?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-38853: Description: How to reproduce this issue: {code:sql} CREATE TABLE t USING PARQUET AS SELECT /*

[jira] [Created] (SPARK-38853) optimizeSkewsInRebalancePartitions has performance issue

2022-04-11 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-38853: --- Summary: optimizeSkewsInRebalancePartitions has performance issue Key: SPARK-38853 URL: https://issues.apache.org/jira/browse/SPARK-38853 Project: Spark Issue

[jira] [Reopened] (SPARK-34444) Pushdown scalar-subquery filter to FileSourceScan

2022-04-09 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-3?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang reopened SPARK-3: - > Pushdown scalar-subquery filter to FileSourceScan > --

[jira] [Created] (SPARK-38843) Fix translate metadata col filters

2022-04-09 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-38843: --- Summary: Fix translate metadata col filters Key: SPARK-38843 URL: https://issues.apache.org/jira/browse/SPARK-38843 Project: Spark Issue Type: Bug Co

<    3   4   5   6   7   8   9   10   11   12   >