[jira] [Updated] (SPARK-45621) Add feature to evaluate subquery before push down filter Optimizer rule

2023-10-20 Thread Maytas Monsereenusorn (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45621?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maytas Monsereenusorn updated SPARK-45621: -- Summary: Add feature to evaluate subquery before push down filter Optimizer

[jira] [Created] (SPARK-45621) Add feature to evaluate subquery before Optimizer rule to push down filter

2023-10-20 Thread Maytas Monsereenusorn (Jira)
Maytas Monsereenusorn created SPARK-45621: - Summary: Add feature to evaluate subquery before Optimizer rule to push down filter Key: SPARK-45621 URL: https://issues.apache.org/jira/browse/SPARK-45621

[jira] [Updated] (SPARK-43778) RewriteCorrelatedScalarSubquery should handle duplicate attributes

2023-10-20 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43778?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-43778: --- Labels: pull-request-available (was: ) > RewriteCorrelatedScalarSubquery should handle

[jira] [Updated] (SPARK-45620) Fix user-facing APIs related to Python UDTF to use camelCase.

2023-10-20 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45620?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-45620: --- Labels: pull-request-available (was: ) > Fix user-facing APIs related to Python UDTF to

[jira] [Created] (SPARK-45620) Fix user-facing APIs related to Python UDTF to use camelCase.

2023-10-20 Thread Takuya Ueshin (Jira)
Takuya Ueshin created SPARK-45620: - Summary: Fix user-facing APIs related to Python UDTF to use camelCase. Key: SPARK-45620 URL: https://issues.apache.org/jira/browse/SPARK-45620 Project: Spark

[jira] [Resolved] (SPARK-45523) Return useful error message if UDTF returns None for non-nullable column

2023-10-20 Thread Takuya Ueshin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45523?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin resolved SPARK-45523. --- Fix Version/s: 4.0.0 Assignee: Daniel Resolution: Fixed Issue resolved by

[jira] [Updated] (SPARK-45619) Apply the observed metrics to Observation object.

2023-10-20 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45619?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-45619: --- Labels: pull-request-available (was: ) > Apply the observed metrics to Observation object.

[jira] [Created] (SPARK-45619) Apply the observed metrics to Observation object.

2023-10-20 Thread Takuya Ueshin (Jira)
Takuya Ueshin created SPARK-45619: - Summary: Apply the observed metrics to Observation object. Key: SPARK-45619 URL: https://issues.apache.org/jira/browse/SPARK-45619 Project: Spark Issue

[jira] [Updated] (SPARK-45617) Upgrade Apache Commons Crypto 1.2.0

2023-10-20 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45617?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-45617: --- Labels: pull-request-available (was: ) > Upgrade Apache Commons Crypto 1.2.0 >

[jira] [Updated] (SPARK-45618) Remove BaseErrorHandler

2023-10-20 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45618?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-45618: --- Labels: pull-request-available (was: ) > Remove BaseErrorHandler > ---

[jira] [Created] (SPARK-45618) Remove BaseErrorHandler

2023-10-20 Thread L. C. Hsieh (Jira)
L. C. Hsieh created SPARK-45618: --- Summary: Remove BaseErrorHandler Key: SPARK-45618 URL: https://issues.apache.org/jira/browse/SPARK-45618 Project: Spark Issue Type: Improvement

[jira] [Created] (SPARK-45617) Upgrade Apache Commons Crypto 1.2.0

2023-10-20 Thread L. C. Hsieh (Jira)
L. C. Hsieh created SPARK-45617: --- Summary: Upgrade Apache Commons Crypto 1.2.0 Key: SPARK-45617 URL: https://issues.apache.org/jira/browse/SPARK-45617 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-45023) SPIP: Python Stored Procedures

2023-10-20 Thread Allison Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45023?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1954#comment-1954 ] Allison Wang commented on SPARK-45023: -- [~abhinavofficial] this proposal is on hold, given the

[jira] [Resolved] (SPARK-45023) SPIP: Python Stored Procedures

2023-10-20 Thread Allison Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45023?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allison Wang resolved SPARK-45023. -- Resolution: Won't Do > SPIP: Python Stored Procedures > -- > >

[jira] [Updated] (SPARK-45616) Usages of ParVector are unsafe because it does not propagate ThreadLocals or SparkSession

2023-10-20 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45616?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-45616: --- Labels: pull-request-available (was: ) > Usages of ParVector are unsafe because it does

[jira] [Created] (SPARK-45616) Usages of ParVector are unsafe because it does not propagate ThreadLocals or SparkSession

2023-10-20 Thread Ankur Dave (Jira)
Ankur Dave created SPARK-45616: -- Summary: Usages of ParVector are unsafe because it does not propagate ThreadLocals or SparkSession Key: SPARK-45616 URL: https://issues.apache.org/jira/browse/SPARK-45616

[jira] [Resolved] (SPARK-30848) Remove manual backport of Murmur3 MurmurHash3.productHash fix from Scala 2.13

2023-10-20 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30848?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie resolved SPARK-30848. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 43161

[jira] [Assigned] (SPARK-30848) Remove manual backport of Murmur3 MurmurHash3.productHash fix from Scala 2.13

2023-10-20 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30848?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie reassigned SPARK-30848: Assignee: BingKun Pan > Remove manual backport of Murmur3 MurmurHash3.productHash fix from Scala

[jira] [Updated] (SPARK-30848) Remove manual backport of Murmur3 MurmurHash3.productHash fix from Scala 2.13

2023-10-20 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30848?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-30848: --- Labels: pull-request-available (was: ) > Remove manual backport of Murmur3

[jira] [Resolved] (SPARK-45583) Spark SQL returning incorrect values for full outer join on keys with the same name.

2023-10-20 Thread Bruce Robbins (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45583?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bruce Robbins resolved SPARK-45583. --- Resolution: Fixed > Spark SQL returning incorrect values for full outer join on keys with

[jira] [Resolved] (SPARK-45602) Replace `s.c.MapOps.filterKeys` with `s.c.MapOps.view.filterKeys`

2023-10-20 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45602?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie resolved SPARK-45602. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 43445

[jira] [Resolved] (SPARK-45595) Expose SQLSTATE in error message

2023-10-20 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45595?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-45595. - Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 43438

[jira] [Assigned] (SPARK-45595) Expose SQLSTATE in error message

2023-10-20 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45595?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-45595: --- Assignee: Serge Rielau > Expose SQLSTATE in error message >

[jira] [Updated] (SPARK-45509) Investigate the behavior difference in self-join

2023-10-20 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45509?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-45509: --- Labels: pull-request-available (was: ) > Investigate the behavior difference in self-join

[jira] [Updated] (SPARK-45610) Fix "Auto-application to `()` is deprecated."

2023-10-20 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45610?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie updated SPARK-45610: - Summary: Fix "Auto-application to `()` is deprecated." (was: Handle "Auto-application to `()` is

[jira] [Created] (SPARK-45615) Remove redundant"Auto-application to `()` is deprecated" compile suppression rules.

2023-10-20 Thread Yang Jie (Jira)
Yang Jie created SPARK-45615: Summary: Remove redundant"Auto-application to `()` is deprecated" compile suppression rules. Key: SPARK-45615 URL: https://issues.apache.org/jira/browse/SPARK-45615 Project:

[jira] [Updated] (SPARK-39910) DataFrameReader API cannot read files from hadoop archives (.har)

2023-10-20 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39910?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-39910: --- Labels: DataFrameReader pull-request-available (was: DataFrameReader) > DataFrameReader

[jira] [Assigned] (SPARK-45592) AQE and InMemoryTableScanExec correctness bug

2023-10-20 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45592?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-45592: -- Assignee: (was: Apache Spark) > AQE and InMemoryTableScanExec correctness bug >

[jira] [Assigned] (SPARK-45592) AQE and InMemoryTableScanExec correctness bug

2023-10-20 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45592?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-45592: -- Assignee: Apache Spark > AQE and InMemoryTableScanExec correctness bug >

[jira] [Resolved] (SPARK-45609) Include SqlState in SparkThrowable proto message

2023-10-20 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45609?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-45609. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 43457

[jira] [Assigned] (SPARK-45609) Include SqlState in SparkThrowable proto message

2023-10-20 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45609?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-45609: Assignee: Yihong He > Include SqlState in SparkThrowable proto message >

[jira] [Commented] (SPARK-43851) Support LCA in grouping expressions

2023-10-20 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43851?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1591#comment-1591 ] Yuming Wang commented on SPARK-43851: - The resolution should be unresolved. > Support LCA in

[jira] [Reopened] (SPARK-43851) Support LCA in grouping expressions

2023-10-20 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43851?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang reopened SPARK-43851: - Assignee: (was: Jia Fan) > Support LCA in grouping expressions >

[jira] [Updated] (SPARK-43851) Support LCA in grouping expressions

2023-10-20 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43851?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-43851: Fix Version/s: (was: 3.5.0) > Support LCA in grouping expressions >

[jira] [Resolved] (SPARK-45613) Expose DeterministicLevel as a DeveloperApi

2023-10-20 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45613?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-45613. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 43461

[jira] [Assigned] (SPARK-45613) Expose DeterministicLevel as a DeveloperApi

2023-10-20 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45613?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-45613: Assignee: Mridul Muralidharan > Expose DeterministicLevel as a DeveloperApi >