[jira] [Assigned] (SPARK-37343) Implement createIndex and IndexExists in JDBC (Postgres dialect)

[ https://issues.apache.org/jira/browse/SPARK-37343?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37343: Assignee: Apache Spark > Implement createIndex and IndexExists in JDBC (Postgres

[jira] [Commented] (SPARK-37343) Implement createIndex and IndexExists in JDBC (Postgres dialect)

[ https://issues.apache.org/jira/browse/SPARK-37343?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17446754#comment-17446754 ] Apache Spark commented on SPARK-37343: -- User 'dchvn' has created a pull request for this issue:

[jira] [Assigned] (SPARK-37343) Implement createIndex and IndexExists in JDBC (Postgres dialect)

[ https://issues.apache.org/jira/browse/SPARK-37343?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37343: Assignee: (was: Apache Spark) > Implement createIndex and IndexExists in JDBC

[jira] [Resolved] (SPARK-37379) Add tree pattern pruning to CTESubstitution rule

[ https://issues.apache.org/jira/browse/SPARK-37379?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen resolved SPARK-37379. Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 34658

[jira] [Updated] (SPARK-37394) Skip registering with ESS if a customized shuffle manager is configured

[ https://issues.apache.org/jira/browse/SPARK-37394?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Weiwei Yang updated SPARK-37394: Summary: Skip registering with ESS if a customized shuffle manager is configured (was: Skip

[jira] [Assigned] (SPARK-37394) Skip registering to ESS if a customized shuffle manager is configured

[ https://issues.apache.org/jira/browse/SPARK-37394?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37394: Assignee: Apache Spark > Skip registering to ESS if a customized shuffle manager is

[jira] [Updated] (SPARK-36900) "SPARK-36464: size returns correct positive number even with over 2GB data" will oom with JDK17

[ https://issues.apache.org/jira/browse/SPARK-36900?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-36900: -- Fix Version/s: 3.2.1 > "SPARK-36464: size returns correct positive number even with over 2GB

[jira] [Assigned] (SPARK-37394) Skip registering to ESS if a customized shuffle manager is configured

[ https://issues.apache.org/jira/browse/SPARK-37394?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37394: Assignee: (was: Apache Spark) > Skip registering to ESS if a customized shuffle

[jira] [Commented] (SPARK-37394) Skip registering to ESS if a customized shuffle manager is configured

[ https://issues.apache.org/jira/browse/SPARK-37394?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17446717#comment-17446717 ] Apache Spark commented on SPARK-37394: -- User 'yangwwei' has created a pull request for this issue:

[jira] [Created] (SPARK-37394) Skip registering to ESS if a customized shuffle manager is configured

Weiwei Yang created SPARK-37394: --- Summary: Skip registering to ESS if a customized shuffle manager is configured Key: SPARK-37394 URL: https://issues.apache.org/jira/browse/SPARK-37394 Project: Spark

[jira] [Assigned] (SPARK-37393) Inline annotations for {ml, mllib}/common.py

[ https://issues.apache.org/jira/browse/SPARK-37393?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37393: Assignee: Apache Spark > Inline annotations for {ml, mllib}/common.py >

[jira] [Commented] (SPARK-37393) Inline annotations for {ml, mllib}/common.py

[ https://issues.apache.org/jira/browse/SPARK-37393?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17446673#comment-17446673 ] Apache Spark commented on SPARK-37393: -- User 'nchammas' has created a pull request for this issue:

[jira] [Assigned] (SPARK-37393) Inline annotations for {ml, mllib}/common.py

[ https://issues.apache.org/jira/browse/SPARK-37393?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37393: Assignee: (was: Apache Spark) > Inline annotations for {ml, mllib}/common.py >

[jira] [Resolved] (SPARK-36997) Test type hints against examples

[ https://issues.apache.org/jira/browse/SPARK-36997?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maciej Szymkiewicz resolved SPARK-36997. Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request

[jira] [Assigned] (SPARK-36997) Test type hints against examples

[ https://issues.apache.org/jira/browse/SPARK-36997?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maciej Szymkiewicz reassigned SPARK-36997: -- Assignee: Maciej Szymkiewicz > Test type hints against examples >

[jira] [Updated] (SPARK-37393) Inline annotations for {ml, mllib}/common.py

[ https://issues.apache.org/jira/browse/SPARK-37393?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicholas Chammas updated SPARK-37393: - Description: This will allow us to run type checks against those files themselves. >

[jira] [Created] (SPARK-37393) Inline annotations for {ml, mllib}/common.py

Nicholas Chammas created SPARK-37393: Summary: Inline annotations for {ml, mllib}/common.py Key: SPARK-37393 URL: https://issues.apache.org/jira/browse/SPARK-37393 Project: Spark Issue

[jira] [Updated] (SPARK-37392) Catalyst optimizer very time-consuming and memory-intensive with some "explode(array)"

[ https://issues.apache.org/jira/browse/SPARK-37392?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Francois MARTIN updated SPARK-37392: Description: The problem occurs with the simple code below: {code:java} import

[jira] [Updated] (SPARK-37392) Catalyst optimizer very time-consuming and memory-intensive with some "explode(array)"

[ https://issues.apache.org/jira/browse/SPARK-37392?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Francois MARTIN updated SPARK-37392: Priority: Major (was: Minor) > Catalyst optimizer very time-consuming and

[jira] [Created] (SPARK-37392) Catalyst optimizer very time-consuming and memory-intensive with some "explode(array)"

Francois MARTIN created SPARK-37392: --- Summary: Catalyst optimizer very time-consuming and memory-intensive with some "explode(array)" Key: SPARK-37392 URL: https://issues.apache.org/jira/browse/SPARK-37392

[jira] [Updated] (SPARK-37391) SIGNIFICANT bottleneck introduced by fix for SPARK-34497

[ https://issues.apache.org/jira/browse/SPARK-37391?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Guinther updated SPARK-37391: --- Description: The fix for SPARK-34497 ( [https://github.com/apache/spark/pull/31622] ) does

[jira] [Updated] (SPARK-37391) SIGNIFICANT bottleneck introduced by fix for SPARK-34497

[ https://issues.apache.org/jira/browse/SPARK-37391?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Guinther updated SPARK-37391: --- Description: The fix for SPARK-34497 ([https://github.com/apache/spark/pull/31622)] does

[jira] [Updated] (SPARK-37349) Improve SQL Rest API Parsing

[ https://issues.apache.org/jira/browse/SPARK-37349?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yian Liou updated SPARK-37349: -- Description: https://issues.apache.org/jira/browse/SPARK-31440 added improvements for SQL Rest API.

[jira] [Created] (SPARK-37391) SIGNIFICANT bottleneck introduced by fix for SPARK-34497

Danny Guinther created SPARK-37391: -- Summary: SIGNIFICANT bottleneck introduced by fix for SPARK-34497 Key: SPARK-37391 URL: https://issues.apache.org/jira/browse/SPARK-37391 Project: Spark

[jira] [Updated] (SPARK-37349) Improve SQL Rest API Parsing

[ https://issues.apache.org/jira/browse/SPARK-37349?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yian Liou updated SPARK-37349: -- Description: https://issues.apache.org/jira/browse/SPARK-31440 added improvements for SQL Rest API.

[jira] [Commented] (SPARK-25642) Add new Metrics in External Shuffle Service to help determine Network performance and Connection Handling capabilities of the Shuffle Service

[ https://issues.apache.org/jira/browse/SPARK-25642?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17446592#comment-17446592 ] Parth Gandhi commented on SPARK-25642: -- Hello [~yzhangal] , ideally you are correct,

[jira] [Resolved] (SPARK-37385) Add tests for TimestampNTZ and TimestampLTZ for Parquet data source

[ https://issues.apache.org/jira/browse/SPARK-37385?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-37385. - Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 34663

[jira] [Assigned] (SPARK-37385) Add tests for TimestampNTZ and TimestampLTZ for Parquet data source

[ https://issues.apache.org/jira/browse/SPARK-37385?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-37385: --- Assignee: Ivan Sadikov > Add tests for TimestampNTZ and TimestampLTZ for Parquet data

[jira] [Commented] (SPARK-37259) JDBC read is always going to wrap the query in a select statement

[ https://issues.apache.org/jira/browse/SPARK-37259?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17446554#comment-17446554 ] Peter Toth commented on SPARK-37259: [~KevinAppelBofa], how about adding a new `withClause` to the

[jira] [Assigned] (SPARK-37388) WidthBucket throws NullPointerException in WholeStageCodegenExec

[ https://issues.apache.org/jira/browse/SPARK-37388?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37388: Assignee: Apache Spark > WidthBucket throws NullPointerException in

[jira] [Assigned] (SPARK-37388) WidthBucket throws NullPointerException in WholeStageCodegenExec

[ https://issues.apache.org/jira/browse/SPARK-37388?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37388: Assignee: Apache Spark > WidthBucket throws NullPointerException in

[jira] [Commented] (SPARK-37388) WidthBucket throws NullPointerException in WholeStageCodegenExec

[ https://issues.apache.org/jira/browse/SPARK-37388?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17446524#comment-17446524 ] Apache Spark commented on SPARK-37388: -- User 'tomvanbussel' has created a pull request for this

[jira] [Assigned] (SPARK-37388) WidthBucket throws NullPointerException in WholeStageCodegenExec

[ https://issues.apache.org/jira/browse/SPARK-37388?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37388: Assignee: (was: Apache Spark) > WidthBucket throws NullPointerException in

[jira] [Commented] (SPARK-37388) WidthBucket throws NullPointerException in WholeStageCodegenExec

[ https://issues.apache.org/jira/browse/SPARK-37388?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17446525#comment-17446525 ] Apache Spark commented on SPARK-37388: -- User 'tomvanbussel' has created a pull request for this

[jira] [Assigned] (SPARK-37390) Buggy method retrival in pyspark.docs.conf.setup

[ https://issues.apache.org/jira/browse/SPARK-37390?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37390: Assignee: Apache Spark > Buggy method retrival in pyspark.docs.conf.setup >

[jira] [Commented] (SPARK-37390) Buggy method retrival in pyspark.docs.conf.setup

[ https://issues.apache.org/jira/browse/SPARK-37390?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17446504#comment-17446504 ] Apache Spark commented on SPARK-37390: -- User 'zero323' has created a pull request for this issue:

[jira] [Assigned] (SPARK-37390) Buggy method retrival in pyspark.docs.conf.setup

[ https://issues.apache.org/jira/browse/SPARK-37390?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37390: Assignee: (was: Apache Spark) > Buggy method retrival in pyspark.docs.conf.setup >

[jira] [Created] (SPARK-37390) Buggy method retrival in pyspark.docs.conf.setup

Maciej Szymkiewicz created SPARK-37390: -- Summary: Buggy method retrival in pyspark.docs.conf.setup Key: SPARK-37390 URL: https://issues.apache.org/jira/browse/SPARK-37390 Project: Spark

[jira] [Assigned] (SPARK-37389) Check unclosed bracketed comments

[ https://issues.apache.org/jira/browse/SPARK-37389?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37389: Assignee: (was: Apache Spark) > Check unclosed bracketed comments >

[jira] [Commented] (SPARK-37389) Check unclosed bracketed comments

[ https://issues.apache.org/jira/browse/SPARK-37389?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17446495#comment-17446495 ] Apache Spark commented on SPARK-37389: -- User 'beliefer' has created a pull request for this issue:

[jira] [Assigned] (SPARK-37389) Check unclosed bracketed comments

[ https://issues.apache.org/jira/browse/SPARK-37389?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37389: Assignee: Apache Spark > Check unclosed bracketed comments >

[jira] [Created] (SPARK-37389) Check unclosed bracketed comments

jiaan.geng created SPARK-37389: -- Summary: Check unclosed bracketed comments Key: SPARK-37389 URL: https://issues.apache.org/jira/browse/SPARK-37389 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-37388) WidthBucket throws NullPointerException in WholeStageCodegenExec

Tom van Bussel created SPARK-37388: -- Summary: WidthBucket throws NullPointerException in WholeStageCodegenExec Key: SPARK-37388 URL: https://issues.apache.org/jira/browse/SPARK-37388 Project: Spark

[jira] [Commented] (SPARK-37387) Allow nondeterministic expression in aggregate function

[ https://issues.apache.org/jira/browse/SPARK-37387?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17446457#comment-17446457 ] Leona Yoda commented on SPARK-37387: [~zhenw] I found this behavior when I tried to find the

[jira] [Commented] (SPARK-36902) Migrate CreateTableAsSelectStatement to v2 command

[ https://issues.apache.org/jira/browse/SPARK-36902?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17446398#comment-17446398 ] Apache Spark commented on SPARK-36902: -- User 'dchvn' has created a pull request for this issue:

[jira] [Commented] (SPARK-36902) Migrate CreateTableAsSelectStatement to v2 command

[ https://issues.apache.org/jira/browse/SPARK-36902?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17446397#comment-17446397 ] Apache Spark commented on SPARK-36902: -- User 'dchvn' has created a pull request for this issue:

[jira] [Assigned] (SPARK-37192) Migrate SHOW TBLPROPERTIES to use V2 command by default

[ https://issues.apache.org/jira/browse/SPARK-37192?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37192: Assignee: Apache Spark > Migrate SHOW TBLPROPERTIES to use V2 command by default >

[jira] [Assigned] (SPARK-37192) Migrate SHOW TBLPROPERTIES to use V2 command by default

[ https://issues.apache.org/jira/browse/SPARK-37192?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37192: Assignee: (was: Apache Spark) > Migrate SHOW TBLPROPERTIES to use V2 command by

[jira] [Commented] (SPARK-37192) Migrate SHOW TBLPROPERTIES to use V2 command by default

[ https://issues.apache.org/jira/browse/SPARK-37192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17446371#comment-17446371 ] Apache Spark commented on SPARK-37192: -- User 'Peng-Lei' has created a pull request for this issue:

[jira] [Commented] (SPARK-37382) `with as` clause got inconsistent results

[ https://issues.apache.org/jira/browse/SPARK-37382?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17446367#comment-17446367 ] Zhen Wang commented on SPARK-37382: --- https://issues.apache.org/jira/browse/SPARK-36447 related? >

[jira] [Commented] (SPARK-37387) Allow nondeterministic expression in aggregate function

[ https://issues.apache.org/jira/browse/SPARK-37387?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17446365#comment-17446365 ] Zhen Wang commented on SPARK-37387: --- [~yoda-mon] what is the use case behind, random doesn't seem

[jira] [Commented] (SPARK-37382) `with as` clause got inconsistent results

[ https://issues.apache.org/jira/browse/SPARK-37382?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17446344#comment-17446344 ] jiasheng55 commented on SPARK-37382: The imsages are broken, please have a look:) > `with as`

[jira] [Comment Edited] (SPARK-37382) `with as` clause got inconsistent results

[ https://issues.apache.org/jira/browse/SPARK-37382?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17446344#comment-17446344 ] jiasheng55 edited comment on SPARK-37382 at 11/19/21, 7:59 AM: --- The images