[jira] [Commented] (SPARK-36429) JacksonParser should throw exception when data type unsupported.

2021-08-05 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36429?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17394557#comment-17394557 ] Apache Spark commented on SPARK-36429: -- User 'sarutak' has created a pull request f

[jira] [Commented] (SPARK-36440) Spark3 fails to read hive table with mixed format

2021-08-05 Thread Chao Sun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36440?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17394529#comment-17394529 ] Chao Sun commented on SPARK-36440: -- Hmm really? Spark 2.x support this? I'm not sure wh

[jira] [Commented] (SPARK-36162) extractJoinKeysWithColStats support EqualNullSafe

2021-08-05 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17394495#comment-17394495 ] Apache Spark commented on SPARK-36162: -- User 'changvvb' has created a pull request

[jira] [Assigned] (SPARK-36162) extractJoinKeysWithColStats support EqualNullSafe

2021-08-05 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36162?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36162: Assignee: Apache Spark > extractJoinKeysWithColStats support EqualNullSafe >

[jira] [Assigned] (SPARK-36162) extractJoinKeysWithColStats support EqualNullSafe

2021-08-05 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36162?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36162: Assignee: (was: Apache Spark) > extractJoinKeysWithColStats support EqualNullSafe > -

[jira] [Commented] (SPARK-36162) extractJoinKeysWithColStats support EqualNullSafe

2021-08-05 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17394494#comment-17394494 ] Apache Spark commented on SPARK-36162: -- User 'changvvb' has created a pull request

[jira] [Resolved] (SPARK-36415) Add docs for try_cast/try_add/try_divide

2021-08-05 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36415?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-36415. -- Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 33638 [https://gi

[jira] [Updated] (SPARK-36443) Demote BroadcastJoin causes performance regression and increases OOM risks

2021-08-05 Thread Kent Yao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36443?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao updated SPARK-36443: - Description:     !image-2021-08-06-11-24-34-122.png! was:   !image-2021-08-06-11-24-34-122.png!

[jira] [Updated] (SPARK-36443) Demote BroadcastJoin causes performance regression and increases OOM risks

2021-08-05 Thread Kent Yao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36443?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao updated SPARK-36443: - Description:   !image-2021-08-06-11-24-34-122.png! was:!image-2021-08-06-11-24-34-122.png! > Demot

[jira] [Updated] (SPARK-36443) Demote BroadcastJoin causes performance regression and increases OOM risks

2021-08-05 Thread Kent Yao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36443?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao updated SPARK-36443: - Attachment: image-2021-08-06-11-24-34-122.png > Demote BroadcastJoin causes performance regression and i

[jira] [Updated] (SPARK-36443) Demote BroadcastJoin causes performance regression and increases OOM risks

2021-08-05 Thread Kent Yao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36443?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao updated SPARK-36443: - Description: !image-2021-08-06-11-24-34-122.png! (was: !image-2021-08-06-11-19-00-105.png!) > Demote B

[jira] [Created] (SPARK-36443) Demote BroadcastJoin causes performance regression and increases OOM risks

2021-08-05 Thread Kent Yao (Jira)
Kent Yao created SPARK-36443: Summary: Demote BroadcastJoin causes performance regression and increases OOM risks Key: SPARK-36443 URL: https://issues.apache.org/jira/browse/SPARK-36443 Project: Spark

[jira] [Resolved] (SPARK-36420) Use `isEmpty` to improve performance in Pregel's superstep

2021-08-05 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36420?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-36420. -- Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 33648 [https://gi

[jira] [Assigned] (SPARK-36420) Use `isEmpty` to improve performance in Pregel's superstep

2021-08-05 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36420?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-36420: Assignee: xiepengjie > Use `isEmpty` to improve performance in Pregel's superstep > -

[jira] [Created] (SPARK-36442) Do not reference the deprecated UserDefinedAggregateFunction

2021-08-05 Thread Wenchen Fan (Jira)
Wenchen Fan created SPARK-36442: --- Summary: Do not reference the deprecated UserDefinedAggregateFunction Key: SPARK-36442 URL: https://issues.apache.org/jira/browse/SPARK-36442 Project: Spark I

[jira] [Issue Comment Deleted] (SPARK-36429) JacksonParser should throw exception when data type unsupported.

2021-08-05 Thread jiaan.geng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36429?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jiaan.geng updated SPARK-36429: --- Comment: was deleted (was: I'm working on.) > JacksonParser should throw exception when data type u

[jira] [Assigned] (SPARK-36431) Support comparison of ANSI intervals with different fields

2021-08-05 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36431?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36431: Assignee: (was: Apache Spark) > Support comparison of ANSI intervals with different f

[jira] [Assigned] (SPARK-36431) Support comparison of ANSI intervals with different fields

2021-08-05 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36431?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36431: Assignee: Apache Spark > Support comparison of ANSI intervals with different fields > ---

[jira] [Commented] (SPARK-36431) Support comparison of ANSI intervals with different fields

2021-08-05 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36431?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17394434#comment-17394434 ] Apache Spark commented on SPARK-36431: -- User 'AngersZh' has created a pull requ

[jira] [Updated] (SPARK-36420) Use `isEmpty` to improve performance in Pregel's superstep

2021-08-05 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36420?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-36420: - Fix Version/s: (was: 3.3.0) > Use `isEmpty` to improve performance in Pregel's superstep > -

[jira] [Commented] (SPARK-36386) Fix DataFrame groupby-expanding to follow pandas 1.3

2021-08-05 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36386?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17394430#comment-17394430 ] Apache Spark commented on SPARK-36386: -- User 'itholic' has created a pull request f

[jira] [Commented] (SPARK-36386) Fix DataFrame groupby-expanding to follow pandas 1.3

2021-08-05 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36386?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17394429#comment-17394429 ] Apache Spark commented on SPARK-36386: -- User 'itholic' has created a pull request f

[jira] [Assigned] (SPARK-36386) Fix DataFrame groupby-expanding to follow pandas 1.3

2021-08-05 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36386?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36386: Assignee: (was: Apache Spark) > Fix DataFrame groupby-expanding to follow pandas 1.3

[jira] [Assigned] (SPARK-36386) Fix DataFrame groupby-expanding to follow pandas 1.3

2021-08-05 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36386?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36386: Assignee: Apache Spark > Fix DataFrame groupby-expanding to follow pandas 1.3 > -

[jira] [Assigned] (SPARK-36421) Validate all SQL configs to prevent from wrong use for ConfigEntry

2021-08-05 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36421?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-36421: Assignee: Kent Yao > Validate all SQL configs to prevent from wrong use for ConfigEntry >

[jira] [Updated] (SPARK-36421) Validate all SQL configs to prevent from wrong use for ConfigEntry

2021-08-05 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36421?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-36421: - Fix Version/s: 3.3.0 3.2.0 > Validate all SQL configs to prevent from wrong u

[jira] [Resolved] (SPARK-36421) Validate all SQL configs to prevent from wrong use for ConfigEntry

2021-08-05 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36421?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-36421. -- Resolution: Fixed Fixed in https://github.com/apache/spark/pull/33647 > Validate all SQL conf

[jira] [Resolved] (SPARK-36441) Downloading lintr dependencies fail on GA

2021-08-05 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36441?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-36441. -- Fix Version/s: 3.1.3 3.2.0 3.0.4 Resolution: Fixed

[jira] [Assigned] (SPARK-36441) Downloading lintr dependencies fail on GA

2021-08-05 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36441?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-36441: Assignee: Kousuke Saruta > Downloading lintr dependencies fail on GA > --

[jira] [Assigned] (SPARK-36441) Downloading lintr dependencies fail on GA

2021-08-05 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36441?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36441: Assignee: Apache Spark > Downloading lintr dependencies fail on GA >

[jira] [Commented] (SPARK-36441) Downloading lintr dependencies fail on GA

2021-08-05 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36441?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17394413#comment-17394413 ] Apache Spark commented on SPARK-36441: -- User 'sarutak' has created a pull request f

[jira] [Assigned] (SPARK-36441) Downloading lintr dependencies fail on GA

2021-08-05 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36441?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36441: Assignee: (was: Apache Spark) > Downloading lintr dependencies fail on GA > -

[jira] [Created] (SPARK-36441) Downloading lintr dependencies fail on GA

2021-08-05 Thread Kousuke Saruta (Jira)
Kousuke Saruta created SPARK-36441: -- Summary: Downloading lintr dependencies fail on GA Key: SPARK-36441 URL: https://issues.apache.org/jira/browse/SPARK-36441 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-36440) Spark3 fails to read hive table with mixed format

2021-08-05 Thread Jason Xu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36440?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Xu updated SPARK-36440: - Description: Spark3 fails to read hive table with mixed format with hive Serde, this is a regression co

[jira] [Updated] (SPARK-36440) Spark3 fails to read hive table with mixed format

2021-08-05 Thread Jason Xu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36440?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Xu updated SPARK-36440: - Description: Spark3 fails to read hive table with mixed format with hive Serde, this is a regression co

[jira] [Created] (SPARK-36440) Spark3 fails to read hive table with mixed format

2021-08-05 Thread Jason Xu (Jira)
Jason Xu created SPARK-36440: Summary: Spark3 fails to read hive table with mixed format Key: SPARK-36440 URL: https://issues.apache.org/jira/browse/SPARK-36440 Project: Spark Issue Type: Bug

[jira] [Created] (SPARK-36439) Implement DataFrame.join on key column

2021-08-05 Thread Xinrong Meng (Jira)
Xinrong Meng created SPARK-36439: Summary: Implement DataFrame.join on key column Key: SPARK-36439 URL: https://issues.apache.org/jira/browse/SPARK-36439 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-36438) Support list-like Python objects for Series comparison

2021-08-05 Thread Xinrong Meng (Jira)
Xinrong Meng created SPARK-36438: Summary: Support list-like Python objects for Series comparison Key: SPARK-36438 URL: https://issues.apache.org/jira/browse/SPARK-36438 Project: Spark Issue

[jira] [Created] (SPARK-36437) Enable binary operations with list-like Python objects

2021-08-05 Thread Xinrong Meng (Jira)
Xinrong Meng created SPARK-36437: Summary: Enable binary operations with list-like Python objects Key: SPARK-36437 URL: https://issues.apache.org/jira/browse/SPARK-36437 Project: Spark Issue

[jira] [Created] (SPARK-36436) Implement 'weights' and 'axis' in sample at DataFrame and Series

2021-08-05 Thread Xinrong Meng (Jira)
Xinrong Meng created SPARK-36436: Summary: Implement 'weights' and 'axis' in sample at DataFrame and Series Key: SPARK-36436 URL: https://issues.apache.org/jira/browse/SPARK-36436 Project: Spark

[jira] [Created] (SPARK-36435) Implement MultIndex.equal_levels

2021-08-05 Thread Xinrong Meng (Jira)
Xinrong Meng created SPARK-36435: Summary: Implement MultIndex.equal_levels Key: SPARK-36435 URL: https://issues.apache.org/jira/browse/SPARK-36435 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-36434) Implement DataFrame.lookup

2021-08-05 Thread Xinrong Meng (Jira)
Xinrong Meng created SPARK-36434: Summary: Implement DataFrame.lookup Key: SPARK-36434 URL: https://issues.apache.org/jira/browse/SPARK-36434 Project: Spark Issue Type: Sub-task Com

[jira] [Assigned] (SPARK-36433) Logs should show correct URL of where HistoryServer is started

2021-08-05 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36433?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36433: Assignee: (was: Apache Spark) > Logs should show correct URL of where HistoryServer i

[jira] [Assigned] (SPARK-36433) Logs should show correct URL of where HistoryServer is started

2021-08-05 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36433?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36433: Assignee: Apache Spark > Logs should show correct URL of where HistoryServer is started >

[jira] [Commented] (SPARK-36433) Logs should show correct URL of where HistoryServer is started

2021-08-05 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17394316#comment-17394316 ] Apache Spark commented on SPARK-36433: -- User 'thejdeep' has created a pull request

[jira] [Created] (SPARK-36433) Logs should show correct URL of where HistoryServer is started

2021-08-05 Thread Thejdeep Gudivada (Jira)
Thejdeep Gudivada created SPARK-36433: - Summary: Logs should show correct URL of where HistoryServer is started Key: SPARK-36433 URL: https://issues.apache.org/jira/browse/SPARK-36433 Project: Spa

[jira] [Commented] (SPARK-36393) Try to raise memory and parallelism again for GA

2021-08-05 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36393?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17394311#comment-17394311 ] Apache Spark commented on SPARK-36393: -- User 'viirya' has created a pull request fo

[jira] [Commented] (SPARK-36393) Try to raise memory and parallelism again for GA

2021-08-05 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36393?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17394308#comment-17394308 ] Apache Spark commented on SPARK-36393: -- User 'viirya' has created a pull request fo

[jira] [Commented] (SPARK-36393) Try to raise memory and parallelism again for GA

2021-08-05 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36393?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17394306#comment-17394306 ] Apache Spark commented on SPARK-36393: -- User 'viirya' has created a pull request fo

[jira] [Commented] (SPARK-36393) Try to raise memory and parallelism again for GA

2021-08-05 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36393?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17394305#comment-17394305 ] Apache Spark commented on SPARK-36393: -- User 'viirya' has created a pull request fo

[jira] [Commented] (SPARK-36432) Upgrade Jetty version to 9.4.43

2021-08-05 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36432?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17394290#comment-17394290 ] Apache Spark commented on SPARK-36432: -- User 'this' has created a pull request for

[jira] [Assigned] (SPARK-36432) Upgrade Jetty version to 9.4.43

2021-08-05 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36432?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36432: Assignee: Apache Spark > Upgrade Jetty version to 9.4.43 > --

[jira] [Commented] (SPARK-36432) Upgrade Jetty version to 9.4.43

2021-08-05 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36432?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17394289#comment-17394289 ] Apache Spark commented on SPARK-36432: -- User 'this' has created a pull request for

[jira] [Assigned] (SPARK-36432) Upgrade Jetty version to 9.4.43

2021-08-05 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36432?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36432: Assignee: (was: Apache Spark) > Upgrade Jetty version to 9.4.43 > ---

[jira] [Created] (SPARK-36432) Upgrade Jetty version to 9.4.43

2021-08-05 Thread Sajith A (Jira)
Sajith A created SPARK-36432: Summary: Upgrade Jetty version to 9.4.43 Key: SPARK-36432 URL: https://issues.apache.org/jira/browse/SPARK-36432 Project: Spark Issue Type: Bug Components:

[jira] [Commented] (SPARK-24815) Structured Streaming should support dynamic allocation

2021-08-05 Thread Holden Karau (Jira)
[ https://issues.apache.org/jira/browse/SPARK-24815?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17394265#comment-17394265 ] Holden Karau commented on SPARK-24815: -- cc [~tdas] for thoughts? > Structured Stre

[jira] [Commented] (SPARK-36431) Support comparison of ANSI intervals with different fields

2021-08-05 Thread angerszhu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36431?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17394235#comment-17394235 ] angerszhu commented on SPARK-36431: --- Working on this > Support comparison of ANSI in

[jira] [Updated] (SPARK-36431) Support comparison of ANSI intervals with different fields

2021-08-05 Thread Max Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36431?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk updated SPARK-36431: - Summary: Support comparison of ANSI intervals with different fields (was: Support comparison of ANSI in

[jira] [Commented] (SPARK-36431) Support comparison of ANSI interval with different fields

2021-08-05 Thread Max Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36431?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17394230#comment-17394230 ] Max Gekk commented on SPARK-36431: -- FYI [~cloud_fan] and [~sarutak] [~angerszhuuu] [~be

[jira] [Created] (SPARK-36431) Support comparison of ANSI interval with different fields

2021-08-05 Thread Max Gekk (Jira)
Max Gekk created SPARK-36431: Summary: Support comparison of ANSI interval with different fields Key: SPARK-36431 URL: https://issues.apache.org/jira/browse/SPARK-36431 Project: Spark Issue Type:

[jira] [Commented] (SPARK-36414) Disable timeout for BroadcastQueryStageExec in AQE

2021-08-05 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36414?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17394140#comment-17394140 ] Dongjoon Hyun commented on SPARK-36414: --- Thank you, [~Qin Yao] and [~cloud_fan]. I

[jira] [Updated] (SPARK-36414) Disable timeout for BroadcastQueryStageExec in AQE

2021-08-05 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36414?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-36414: -- Parent: SPARK-33828 Issue Type: Sub-task (was: Improvement) > Disable timeout for Bro

[jira] [Assigned] (SPARK-36430) Adaptively calculate the target size when coalescing shuffle partitions in AQE

2021-08-05 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36430?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36430: Assignee: (was: Apache Spark) > Adaptively calculate the target size when coalescing

[jira] [Commented] (SPARK-36430) Adaptively calculate the target size when coalescing shuffle partitions in AQE

2021-08-05 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36430?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17394136#comment-17394136 ] Apache Spark commented on SPARK-36430: -- User 'cloud-fan' has created a pull request

[jira] [Assigned] (SPARK-36430) Adaptively calculate the target size when coalescing shuffle partitions in AQE

2021-08-05 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36430?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36430: Assignee: Apache Spark > Adaptively calculate the target size when coalescing shuffle par

[jira] [Commented] (SPARK-36430) Adaptively calculate the target size when coalescing shuffle partitions in AQE

2021-08-05 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36430?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17394134#comment-17394134 ] Apache Spark commented on SPARK-36430: -- User 'cloud-fan' has created a pull request

[jira] [Created] (SPARK-36430) Adaptively calculate the target size when coalescing shuffle partitions in AQE

2021-08-05 Thread Wenchen Fan (Jira)
Wenchen Fan created SPARK-36430: --- Summary: Adaptively calculate the target size when coalescing shuffle partitions in AQE Key: SPARK-36430 URL: https://issues.apache.org/jira/browse/SPARK-36430 Project:

[jira] [Resolved] (SPARK-36426) Pin pyzmq to 2.22.0

2021-08-05 Thread Kousuke Saruta (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36426?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta resolved SPARK-36426. Resolution: Invalid I re-triggered the failed GA job and it finally succeeded so I'll clos

[jira] [Resolved] (SPARK-36414) Disable timeout for BroadcastQueryStageExec in AQE

2021-08-05 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36414?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-36414. - Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 33636 [https://gith

[jira] [Assigned] (SPARK-36414) Disable timeout for BroadcastQueryStageExec in AQE

2021-08-05 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36414?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-36414: --- Assignee: Kent Yao > Disable timeout for BroadcastQueryStageExec in AQE > -

[jira] [Resolved] (SPARK-36409) Splitting test cases from datetime.sql

2021-08-05 Thread Gengliang Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36409?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang resolved SPARK-36409. Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 33640 [https:

[jira] [Assigned] (SPARK-36409) Splitting test cases from datetime.sql

2021-08-05 Thread Gengliang Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36409?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang reassigned SPARK-36409: -- Assignee: Wenchen Fan (was: Gengliang Wang) > Splitting test cases from datetime.sql

[jira] [Assigned] (SPARK-36353) RemoveNoopOperators should keep output schema

2021-08-05 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36353?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-36353: --- Assignee: angerszhu > RemoveNoopOperators should keep output schema > -

[jira] [Resolved] (SPARK-36353) RemoveNoopOperators should keep output schema

2021-08-05 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36353?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-36353. - Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 33587 [https://gith

[jira] [Assigned] (SPARK-36429) JacksonParser should throw exception when data type unsupported.

2021-08-05 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36429?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36429: Assignee: (was: Apache Spark) > JacksonParser should throw exception when data type u

[jira] [Assigned] (SPARK-36429) JacksonParser should throw exception when data type unsupported.

2021-08-05 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36429?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36429: Assignee: Apache Spark > JacksonParser should throw exception when data type unsupported.

[jira] [Commented] (SPARK-36429) JacksonParser should throw exception when data type unsupported.

2021-08-05 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36429?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17393891#comment-17393891 ] Apache Spark commented on SPARK-36429: -- User 'beliefer' has created a pull request

[jira] [Commented] (SPARK-36429) JacksonParser should throw exception when data type unsupported.

2021-08-05 Thread jiaan.geng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36429?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17393828#comment-17393828 ] jiaan.geng commented on SPARK-36429: I'm working on. > JacksonParser should throw e

[jira] [Updated] (SPARK-36429) JacksonParser should throw exception when data type unsupported.

2021-08-05 Thread jiaan.geng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36429?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jiaan.geng updated SPARK-36429: --- Description: Currently, when set spark.sql.timestampType=TIMESTAMP_NTZ, the behavior is different b

[jira] [Updated] (SPARK-36429) JacksonParser should throw exception when data type unsupported.

2021-08-05 Thread jiaan.geng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36429?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jiaan.geng updated SPARK-36429: --- Description: Currently, when set spark.sql.timestampType=TIMESTAMP_NTZ, the behavior is different b

[jira] [Created] (SPARK-36429) JacksonParser should throw exception when data type unsupported.

2021-08-05 Thread jiaan.geng (Jira)
jiaan.geng created SPARK-36429: -- Summary: JacksonParser should throw exception when data type unsupported. Key: SPARK-36429 URL: https://issues.apache.org/jira/browse/SPARK-36429 Project: Spark

[jira] [Resolved] (SPARK-36355) NamedExpression add method `withName(newName: String)`

2021-08-05 Thread angerszhu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36355?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] angerszhu resolved SPARK-36355. --- Resolution: Not A Problem > NamedExpression add method `withName(newName: String)` > ---

[jira] [Commented] (SPARK-36428) the 'seconds' parameter of 'make_timestamp' should accept integer type

2021-08-05 Thread jiaan.geng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36428?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17393818#comment-17393818 ] jiaan.geng commented on SPARK-36428: I will take a look. > the 'seconds' parameter

[jira] [Updated] (SPARK-36428) the 'second' parameter of 'make_timestamp' should accept integer type

2021-08-05 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36428?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan updated SPARK-36428: Description: With ANSI mode, {{SELECT make_timestamp(1, 1, 1, 1, 1, 1)}} fails, because the 'seco

[jira] [Updated] (SPARK-36428) the 'seconds' parameter of 'make_timestamp' should accept integer type

2021-08-05 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36428?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan updated SPARK-36428: Summary: the 'seconds' parameter of 'make_timestamp' should accept integer type (was: the 'second

[jira] [Updated] (SPARK-36428) the 'seconds' parameter of 'make_timestamp' should accept integer type

2021-08-05 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36428?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan updated SPARK-36428: Description: With ANSI mode, {{SELECT make_timestamp(1, 1, 1, 1, 1, 1)}} fails, because the 'seco

[jira] [Created] (SPARK-36428) the 'second' parameter of 'make_timestamp' should accept integer type

2021-08-05 Thread Wenchen Fan (Jira)
Wenchen Fan created SPARK-36428: --- Summary: the 'second' parameter of 'make_timestamp' should accept integer type Key: SPARK-36428 URL: https://issues.apache.org/jira/browse/SPARK-36428 Project: Spark

[jira] [Updated] (SPARK-36426) Pin pyzmq to 2.22.0

2021-08-05 Thread Kousuke Saruta (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36426?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta updated SPARK-36426: --- Description: GA is failing and the root cause seems the latest release of pyzmq, which is r

[jira] [Updated] (SPARK-36426) Pin pyzmq to 2.22.0

2021-08-05 Thread Kousuke Saruta (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36426?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta updated SPARK-36426: --- Description: GA is failing and the root cause seems the latest release of pyzmq, which is r

[jira] [Updated] (SPARK-36426) Pin pyzmq to 2.22.0

2021-08-05 Thread Kousuke Saruta (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36426?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta updated SPARK-36426: --- Description: GA is failing and the root cause seems the latest release of pyzmq. https://pyp

[jira] [Updated] (SPARK-36426) Pin pyzmq to 2.22.0

2021-08-05 Thread Kousuke Saruta (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36426?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta updated SPARK-36426: --- Affects Version/s: 3.2.0 > Pin pyzmq to 2.22.0 > --- > > Key

[jira] [Assigned] (SPARK-36425) PySpark: support CrossValidatorModel get standard deviation of metrics for each paramMap

2021-08-05 Thread Weichen Xu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36425?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Weichen Xu reassigned SPARK-36425: -- Assignee: Weichen Xu > PySpark: support CrossValidatorModel get standard deviation of metrics

[jira] [Commented] (SPARK-36426) Pin pyzmq to 2.22.0

2021-08-05 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36426?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17393743#comment-17393743 ] Apache Spark commented on SPARK-36426: -- User 'sarutak' has created a pull request f

[jira] [Created] (SPARK-36427) Scala API: support CrossValidatorModel get standard deviation of metrics for each paramMap

2021-08-05 Thread Weichen Xu (Jira)
Weichen Xu created SPARK-36427: -- Summary: Scala API: support CrossValidatorModel get standard deviation of metrics for each paramMap Key: SPARK-36427 URL: https://issues.apache.org/jira/browse/SPARK-36427

[jira] [Assigned] (SPARK-36426) Pin pyzmq to 2.22.0

2021-08-05 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36426?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36426: Assignee: Kousuke Saruta (was: Apache Spark) > Pin pyzmq to 2.22.0 > ---

[jira] [Assigned] (SPARK-36426) Pin pyzmq to 2.22.0

2021-08-05 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36426?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36426: Assignee: Apache Spark (was: Kousuke Saruta) > Pin pyzmq to 2.22.0 > ---

[jira] [Created] (SPARK-36426) Pin pyzmq to 2.22.0

2021-08-05 Thread Kousuke Saruta (Jira)
Kousuke Saruta created SPARK-36426: -- Summary: Pin pyzmq to 2.22.0 Key: SPARK-36426 URL: https://issues.apache.org/jira/browse/SPARK-36426 Project: Spark Issue Type: Bug Components:

[jira] [Commented] (SPARK-36425) PySpark: support CrossValidatorModel get standard deviation of metrics for each paramMap

2021-08-05 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36425?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17393740#comment-17393740 ] Apache Spark commented on SPARK-36425: -- User 'WeichenXu123' has created a pull requ

[jira] [Assigned] (SPARK-36425) PySpark: support CrossValidatorModel get standard deviation of metrics for each paramMap

2021-08-05 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36425?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36425: Assignee: (was: Apache Spark) > PySpark: support CrossValidatorModel get standard dev

[jira] [Assigned] (SPARK-36425) PySpark: support CrossValidatorModel get standard deviation of metrics for each paramMap

2021-08-05 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36425?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36425: Assignee: Apache Spark > PySpark: support CrossValidatorModel get standard deviation of m

  1   2   >