[jira] [Created] (SPARK-48557) Support scalar subquery with group-by on column equal to constant

2024-06-06 Thread Jack Chen (Jira)
Jack Chen created SPARK-48557: - Summary: Support scalar subquery with group-by on column equal to constant Key: SPARK-48557 URL: https://issues.apache.org/jira/browse/SPARK-48557 Project: Spark

[jira] [Updated] (SPARK-48503) Scalar subquery with group-by and non-equality predicate incorrectly allowed, wrong results

2024-06-02 Thread Jack Chen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48503?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jack Chen updated SPARK-48503: -- Parent: SPARK-35553 Issue Type: Sub-task (was: Bug) > Scalar subquery with group-by and

[jira] [Updated] (SPARK-48501) Loosen `correlated scalar subqueries must be aggregated` error by doing runtime check for scalar subqueries output rowcount

2024-06-02 Thread Jack Chen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48501?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jack Chen updated SPARK-48501: -- Description: Currently if a scalar subquery’s result isn’t aggregated or limit 1, we throw an error

[jira] [Created] (SPARK-48503) Scalar subquery with group-by and non-equality predicate incorrectly allowed, wrong results

2024-06-02 Thread Jack Chen (Jira)
Jack Chen created SPARK-48503: - Summary: Scalar subquery with group-by and non-equality predicate incorrectly allowed, wrong results Key: SPARK-48503 URL: https://issues.apache.org/jira/browse/SPARK-48503

[jira] [Created] (SPARK-48501) Loosen `correlated scalar subqueries must be aggregated` error by doing runtime check for scalar subqueries output rowcount

2024-06-02 Thread Jack Chen (Jira)
Jack Chen created SPARK-48501: - Summary: Loosen `correlated scalar subqueries must be aggregated` error by doing runtime check for scalar subqueries output rowcount Key: SPARK-48501 URL:

[jira] [Resolved] (SPARK-36115) Handle the COUNT bug for correlated IN/EXISTS subquery

2024-06-02 Thread Jack Chen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36115?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jack Chen resolved SPARK-36115. --- Resolution: Fixed This was fixed by https://github.com/apache/spark/pull/43111 > Handle the COUNT

[jira] [Updated] (SPARK-47716) SQLQueryTestSuite flaky case due to view name conflict

2024-04-03 Thread Jack Chen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47716?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jack Chen updated SPARK-47716: -- Description: In SQLQueryTestSuite, the test case "Test logic for determining whether a query is

[jira] [Created] (SPARK-47716) SQLQueryTestSuite flaky case due to view name conflict

2024-04-03 Thread Jack Chen (Jira)
Jack Chen created SPARK-47716: - Summary: SQLQueryTestSuite flaky case due to view name conflict Key: SPARK-47716 URL: https://issues.apache.org/jira/browse/SPARK-47716 Project: Spark Issue Type:

[jira] [Created] (SPARK-47700) Fix formatting of error messages with treeNode

2024-04-02 Thread Jack Chen (Jira)
Jack Chen created SPARK-47700: - Summary: Fix formatting of error messages with treeNode Key: SPARK-47700 URL: https://issues.apache.org/jira/browse/SPARK-47700 Project: Spark Issue Type:

[jira] [Created] (SPARK-47525) Support subquery correlation joining on map attributes

2024-03-22 Thread Jack Chen (Jira)
Jack Chen created SPARK-47525: - Summary: Support subquery correlation joining on map attributes Key: SPARK-47525 URL: https://issues.apache.org/jira/browse/SPARK-47525 Project: Spark Issue Type:

[jira] [Updated] (SPARK-47525) Support subquery correlation joining on map attributes

2024-03-22 Thread Jack Chen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47525?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jack Chen updated SPARK-47525: -- Description: Currently, when a subquery is correlated on a condition like `outer_map[1] =

[jira] [Comment Edited] (SPARK-36113) Unify the logic to handle COUNT bug for scalar and lateral subqueries

2024-03-07 Thread Jack Chen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36113?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17824521#comment-17824521 ] Jack Chen edited comment on SPARK-36113 at 3/7/24 8:52 PM: --- The tricky aspect

[jira] [Comment Edited] (SPARK-36113) Unify the logic to handle COUNT bug for scalar and lateral subqueries

2024-03-07 Thread Jack Chen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36113?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17824521#comment-17824521 ] Jack Chen edited comment on SPARK-36113 at 3/7/24 8:47 PM: --- The tricky aspect

[jira] [Comment Edited] (SPARK-36113) Unify the logic to handle COUNT bug for scalar and lateral subqueries

2024-03-07 Thread Jack Chen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36113?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17824521#comment-17824521 ] Jack Chen edited comment on SPARK-36113 at 3/7/24 8:26 PM: --- The tricky aspect

[jira] [Comment Edited] (SPARK-36113) Unify the logic to handle COUNT bug for scalar and lateral subqueries

2024-03-07 Thread Jack Chen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36113?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17824521#comment-17824521 ] Jack Chen edited comment on SPARK-36113 at 3/7/24 8:09 PM: --- The tricky aspect

[jira] [Commented] (SPARK-36113) Unify the logic to handle COUNT bug for scalar and lateral subqueries

2024-03-07 Thread Jack Chen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36113?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17824521#comment-17824521 ] Jack Chen commented on SPARK-36113: --- The tricky aspect of this is that if we do the count bug handling

[jira] [Updated] (SPARK-46446) Correctness bug in correlated subquery with OFFSET

2023-12-18 Thread Jack Chen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46446?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jack Chen updated SPARK-46446: -- Description: Subqueries with correlation under LIMIT with OFFSET have a correctness bug, introduced

[jira] [Updated] (SPARK-46446) Correctness bug in correlated subquery with OFFSET

2023-12-18 Thread Jack Chen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46446?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jack Chen updated SPARK-46446: -- Description: Subqueries with correlation under LIMIT with OFFSET have a correctness bug, introduced

[jira] [Created] (SPARK-46446) Correctness bug in correlated subquery with OFFSET

2023-12-18 Thread Jack Chen (Jira)
Jack Chen created SPARK-46446: - Summary: Correctness bug in correlated subquery with OFFSET Key: SPARK-46446 URL: https://issues.apache.org/jira/browse/SPARK-46446 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-45009) Correlated EXISTS subqueries in join ON condition unsupported and fail with internal error

2023-08-31 Thread Jack Chen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45009?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17761003#comment-17761003 ] Jack Chen commented on SPARK-45009: --- https://github.com/apache/spark/pull/42725 > Correlated EXISTS

[jira] [Created] (SPARK-45009) Correlated EXISTS subqueries in join ON condition unsupported and fail with internal error

2023-08-29 Thread Jack Chen (Jira)
Jack Chen created SPARK-45009: - Summary: Correlated EXISTS subqueries in join ON condition unsupported and fail with internal error Key: SPARK-45009 URL: https://issues.apache.org/jira/browse/SPARK-45009

[jira] [Updated] (SPARK-44550) Wrong semantics for null IN (empty list)

2023-07-25 Thread Jack Chen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44550?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jack Chen updated SPARK-44550: -- Fix Version/s: (was: 3.5.0) > Wrong semantics for null IN (empty list) >

[jira] [Created] (SPARK-44551) Wrong semantics for null IN (empty list) - IN expression execution

2023-07-25 Thread Jack Chen (Jira)
Jack Chen created SPARK-44551: - Summary: Wrong semantics for null IN (empty list) - IN expression execution Key: SPARK-44551 URL: https://issues.apache.org/jira/browse/SPARK-44551 Project: Spark

[jira] [Updated] (SPARK-44431) Wrong semantics for null IN (empty list) - optimization rules

2023-07-25 Thread Jack Chen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44431?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jack Chen updated SPARK-44431: -- Parent: SPARK-44550 Issue Type: Sub-task (was: Bug) > Wrong semantics for null IN (empty

[jira] [Updated] (SPARK-44431) Wrong semantics for null IN (empty list) - optimization rules

2023-07-25 Thread Jack Chen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44431?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jack Chen updated SPARK-44431: -- Summary: Wrong semantics for null IN (empty list) - optimization rules (was: Wrong semantics for

[jira] [Created] (SPARK-44550) Wrong semantics for null IN (empty list)

2023-07-25 Thread Jack Chen (Jira)
Jack Chen created SPARK-44550: - Summary: Wrong semantics for null IN (empty list) Key: SPARK-44550 URL: https://issues.apache.org/jira/browse/SPARK-44550 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-44448) Wrong results for dense_rank() <= k from InferWindowGroupLimit and DenseRankLimitIterator

2023-07-18 Thread Jack Chen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-8?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jack Chen updated SPARK-8: -- Affects Version/s: 3.5.0 (was: 3.4.0) > Wrong results for dense_rank() <=

[jira] [Commented] (SPARK-44448) Wrong results for dense_rank() <= k from InferWindowGroupLimit and DenseRankLimitIterator

2023-07-16 Thread Jack Chen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-8?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17743594#comment-17743594 ] Jack Chen commented on SPARK-8: --- Fix PR: https://github.com/apache/spark/pull/42026 > Wrong

[jira] [Updated] (SPARK-44448) Wrong results for dense_rank() <= k from InferWindowGroupLimit and DenseRankLimitIterator

2023-07-16 Thread Jack Chen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-8?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jack Chen updated SPARK-8: -- Description: Top-k filters on a dense_rank() window function return wrong results, due to a bug in

[jira] [Created] (SPARK-44448) Wrong results for dense_rank() <= k from InferWindowGroupLimit and DenseRankLimitIterator

2023-07-16 Thread Jack Chen (Jira)
Jack Chen created SPARK-8: - Summary: Wrong results for dense_rank() <= k from InferWindowGroupLimit and DenseRankLimitIterator Key: SPARK-8 URL: https://issues.apache.org/jira/browse/SPARK-8

[jira] [Updated] (SPARK-44431) Wrong semantics for null IN (empty list)

2023-07-14 Thread Jack Chen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44431?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jack Chen updated SPARK-44431: -- Description: {{null IN (empty list)}} incorrectly evaluates to null, when it should evaluate to

[jira] [Created] (SPARK-44431) Wrong semantics for null IN (empty list)

2023-07-14 Thread Jack Chen (Jira)
Jack Chen created SPARK-44431: - Summary: Wrong semantics for null IN (empty list) Key: SPARK-44431 URL: https://issues.apache.org/jira/browse/SPARK-44431 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-43596) Subquery decorrelation rewriteDomainJoins failure from ConstantFolding to isnull

2023-05-19 Thread Jack Chen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43596?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jack Chen updated SPARK-43596: -- Description: We can get a decorrelation error because of rewrites that run in between

[jira] [Updated] (SPARK-43596) Subquery decorrelation rewriteDomainJoins failure from ConstantFolding to isnull

2023-05-19 Thread Jack Chen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43596?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jack Chen updated SPARK-43596: -- Description: We can get a decorrelation error because of rewrites that run in between

[jira] [Updated] (SPARK-43596) Subquery decorrelation rewriteDomainJoins failure from ConstantFolding to isnull

2023-05-19 Thread Jack Chen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43596?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jack Chen updated SPARK-43596: -- Description: We can get a decorrelation error because of rewrites that run in between

[jira] [Created] (SPARK-43596) Subquery decorrelation rewriteDomainJoins failure from ConstantFolding to isnull

2023-05-19 Thread Jack Chen (Jira)
Jack Chen created SPARK-43596: - Summary: Subquery decorrelation rewriteDomainJoins failure from ConstantFolding to isnull Key: SPARK-43596 URL: https://issues.apache.org/jira/browse/SPARK-43596 Project:

[jira] [Updated] (SPARK-43596) Subquery decorrelation rewriteDomainJoins failure from ConstantFolding to isnull

2023-05-19 Thread Jack Chen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43596?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jack Chen updated SPARK-43596: -- Description: We can get a decorrelation error because of rewrites that run in between

[jira] [Updated] (SPARK-43413) IN subquery ListQuery has wrong nullability

2023-05-08 Thread Jack Chen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43413?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jack Chen updated SPARK-43413: -- Description: IN subquery expressions are incorrectly marked as non-nullable, even when they are

[jira] [Updated] (SPARK-43413) IN subquery ListQuery has wrong nullability

2023-05-08 Thread Jack Chen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43413?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jack Chen updated SPARK-43413: -- Description: IN subquery expressions are incorrectly always marked as non-nullable, even when they

[jira] [Updated] (SPARK-43413) IN subquery ListQuery has wrong nullability

2023-05-08 Thread Jack Chen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43413?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jack Chen updated SPARK-43413: -- Description: IN subquery expressions currently are marked as nullable if and only if the

[jira] [Created] (SPARK-43413) IN subquery ListQuery has wrong nullability

2023-05-08 Thread Jack Chen (Jira)
Jack Chen created SPARK-43413: - Summary: IN subquery ListQuery has wrong nullability Key: SPARK-43413 URL: https://issues.apache.org/jira/browse/SPARK-43413 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-43156) Correctness COUNT bug in correlated scalar subselect with `COUNT(*) is null`

2023-04-26 Thread Jack Chen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43156?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17716952#comment-17716952 ] Jack Chen commented on SPARK-43156: --- My above pull request only added a related test. Hisoka-X's PR

[jira] [Reopened] (SPARK-43156) Correctness COUNT bug in correlated scalar subselect with `COUNT(*) is null`

2023-04-26 Thread Jack Chen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43156?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jack Chen reopened SPARK-43156: --- > Correctness COUNT bug in correlated scalar subselect with `COUNT(*) is null` >

[jira] [Updated] (SPARK-43156) Correctness COUNT bug in correlated scalar subselect with `COUNT(*) is null`

2023-04-16 Thread Jack Chen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43156?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jack Chen updated SPARK-43156: -- Description: Example query: {code:java} spark.sql("select *, (select (count(1)) is null from t1 where

[jira] [Updated] (SPARK-43156) Correctness COUNT bug in correlated scalar subselect with `COUNT(*) is null`

2023-04-16 Thread Jack Chen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43156?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jack Chen updated SPARK-43156: -- Description: Example query: {code:java} spark.sql("select *, (select (count(1)) is null from t1 where

[jira] [Created] (SPARK-43156) Correctness COUNT bug in correlated scalar subselect with `COUNT(*) is null`

2023-04-16 Thread Jack Chen (Jira)
Jack Chen created SPARK-43156: - Summary: Correctness COUNT bug in correlated scalar subselect with `COUNT(*) is null` Key: SPARK-43156 URL: https://issues.apache.org/jira/browse/SPARK-43156 Project:

[jira] [Created] (SPARK-43098) Should not handle the COUNT bug when the GROUP BY clause of a correlated scalar subquery is non-empty

2023-04-11 Thread Jack Chen (Jira)
Jack Chen created SPARK-43098: - Summary: Should not handle the COUNT bug when the GROUP BY clause of a correlated scalar subquery is non-empty Key: SPARK-43098 URL: https://issues.apache.org/jira/browse/SPARK-43098

[jira] [Created] (SPARK-2901) it's a test

2014-08-07 Thread Jack chen (JIRA)
Jack chen created SPARK-2901: Summary: it's a test Key: SPARK-2901 URL: https://issues.apache.org/jira/browse/SPARK-2901 Project: Spark Issue Type: Bug Reporter: Jack chen