[jira] [Updated] (SPARK-47633) Cache miss for queries using JOIN LATERAL with join condition

2024-03-28 Thread Bruce Robbins (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47633?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bruce Robbins updated SPARK-47633: -- Affects Version/s: 3.4.2 > Cache miss for queries using JOIN LATERAL with join condition >

[jira] [Updated] (SPARK-47633) Cache miss for queries using JOIN LATERAL with join condition

2024-03-28 Thread Bruce Robbins (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47633?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bruce Robbins updated SPARK-47633: -- Affects Version/s: 3.5.1 > Cache miss for queries using JOIN LATERAL with join condition >

[jira] [Created] (SPARK-47633) Cache miss for queries using JOIN LATERAL with join condition

2024-03-28 Thread Bruce Robbins (Jira)
Bruce Robbins created SPARK-47633: - Summary: Cache miss for queries using JOIN LATERAL with join condition Key: SPARK-47633 URL: https://issues.apache.org/jira/browse/SPARK-47633 Project: Spark

[jira] [Resolved] (SPARK-47527) Cache miss for queries using With expressions

2024-03-24 Thread Bruce Robbins (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47527?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bruce Robbins resolved SPARK-47527. --- Resolution: Duplicate > Cache miss for queries using With expressions >

[jira] [Created] (SPARK-47527) Cache misses for queries using With expressions

2024-03-23 Thread Bruce Robbins (Jira)
Bruce Robbins created SPARK-47527: - Summary: Cache misses for queries using With expressions Key: SPARK-47527 URL: https://issues.apache.org/jira/browse/SPARK-47527 Project: Spark Issue

[jira] [Updated] (SPARK-47527) Cache miss for queries using With expressions

2024-03-23 Thread Bruce Robbins (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47527?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bruce Robbins updated SPARK-47527: -- Description: For example: {noformat} create or replace temp view v1 as select id from

[jira] [Updated] (SPARK-47527) Cache miss for queries using With expressions

2024-03-23 Thread Bruce Robbins (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47527?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bruce Robbins updated SPARK-47527: -- Summary: Cache miss for queries using With expressions (was: Cache misses for queries using

[jira] [Comment Edited] (SPARK-47193) Converting dataframe to rdd results in data loss

2024-02-27 Thread Bruce Robbins (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47193?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17821393#comment-17821393 ] Bruce Robbins edited comment on SPARK-47193 at 2/27/24 8:48 PM: Running

[jira] [Commented] (SPARK-47193) Converting dataframe to rdd results in data loss

2024-02-27 Thread Bruce Robbins (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47193?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17821393#comment-17821393 ] Bruce Robbins commented on SPARK-47193: --- Running this in Spark 3.5.0 in local mode on my laptop, I

[jira] [Commented] (SPARK-47134) Unexpected nulls when casting decimal values in specific cases

2024-02-22 Thread Bruce Robbins (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17819789#comment-17819789 ] Bruce Robbins commented on SPARK-47134: --- Oddly, I cannot reproduce on either 3.4.1 or 3.5.0.

[jira] [Updated] (SPARK-47104) Spark SQL query fails with NullPointerException

2024-02-21 Thread Bruce Robbins (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47104?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bruce Robbins updated SPARK-47104: -- Affects Version/s: 3.5.0 3.4.2 > Spark SQL query fails with

[jira] [Commented] (SPARK-47104) Spark SQL query fails with NullPointerException

2024-02-20 Thread Bruce Robbins (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47104?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17818934#comment-17818934 ] Bruce Robbins commented on SPARK-47104: --- It's not a CSV specific issue. You can reproduce with a

[jira] [Commented] (SPARK-47034) join between cached temp tables result in missing entries

2024-02-13 Thread Bruce Robbins (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47034?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17817123#comment-17817123 ] Bruce Robbins commented on SPARK-47034: --- I wonder if this is SPARK-45592 (and, relatedly,

[jira] [Commented] (SPARK-47019) AQE dynamic cache partitioning causes SortMergeJoin to result in data loss

2024-02-10 Thread Bruce Robbins (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47019?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17816321#comment-17816321 ] Bruce Robbins commented on SPARK-47019: --- I can reproduce on my laptop using Spark 3.5.0 and

[jira] [Updated] (SPARK-46779) Grouping by subquery with a cached relation can fail

2024-01-19 Thread Bruce Robbins (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46779?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bruce Robbins updated SPARK-46779: -- Description: Example: {noformat} create or replace temp view data(c1, c2) as values (1, 2),

[jira] [Updated] (SPARK-46779) Grouping by subquery with a cached relation can fail

2024-01-19 Thread Bruce Robbins (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46779?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bruce Robbins updated SPARK-46779: -- Affects Version/s: 3.5.0 3.4.2 > Grouping by subquery with a cached

[jira] [Created] (SPARK-46779) Grouping by subquery with a cached relation can fail

2024-01-19 Thread Bruce Robbins (Jira)
Bruce Robbins created SPARK-46779: - Summary: Grouping by subquery with a cached relation can fail Key: SPARK-46779 URL: https://issues.apache.org/jira/browse/SPARK-46779 Project: Spark Issue

[jira] [Commented] (SPARK-46373) Create DataFrame Bug

2023-12-13 Thread Bruce Robbins (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46373?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17796385#comment-17796385 ] Bruce Robbins commented on SPARK-46373: --- Maybe due to this (from [the

[jira] [Updated] (SPARK-46289) Exception when ordering by UDT in interpreted mode

2023-12-08 Thread Bruce Robbins (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46289?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bruce Robbins updated SPARK-46289: -- Priority: Minor (was: Major) > Exception when ordering by UDT in interpreted mode >

[jira] [Updated] (SPARK-46289) Exception when ordering by UDT in interpreted mode

2023-12-06 Thread Bruce Robbins (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46289?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bruce Robbins updated SPARK-46289: -- Affects Version/s: 3.3.3 > Exception when ordering by UDT in interpreted mode >

[jira] [Created] (SPARK-46289) Exception when ordering by UDT in interpreted mode

2023-12-06 Thread Bruce Robbins (Jira)
Bruce Robbins created SPARK-46289: - Summary: Exception when ordering by UDT in interpreted mode Key: SPARK-46289 URL: https://issues.apache.org/jira/browse/SPARK-46289 Project: Spark Issue

[jira] [Commented] (SPARK-45644) After upgrading to Spark 3.4.1 and 3.5.0 we receive RuntimeException "scala.Some is not a valid external type for schema of array"

2023-12-04 Thread Bruce Robbins (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45644?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17792942#comment-17792942 ] Bruce Robbins commented on SPARK-45644: --- Even though this is the original issue, I closed it as a

[jira] [Resolved] (SPARK-45644) After upgrading to Spark 3.4.1 and 3.5.0 we receive RuntimeException "scala.Some is not a valid external type for schema of array"

2023-12-04 Thread Bruce Robbins (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45644?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bruce Robbins resolved SPARK-45644. --- Resolution: Duplicate > After upgrading to Spark 3.4.1 and 3.5.0 we receive

[jira] [Updated] (SPARK-46189) Various Pandas functions fail in interpreted mode

2023-11-30 Thread Bruce Robbins (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46189?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bruce Robbins updated SPARK-46189: -- Description: Various Pandas functions ({{kurt}}, {{var}}, {{skew}}, {{cov}}, and {{stddev}})

[jira] [Created] (SPARK-46189) Various Pandas functions fail in interpreted mode

2023-11-30 Thread Bruce Robbins (Jira)
Bruce Robbins created SPARK-46189: - Summary: Various Pandas functions fail in interpreted mode Key: SPARK-46189 URL: https://issues.apache.org/jira/browse/SPARK-46189 Project: Spark Issue

[jira] [Commented] (SPARK-45896) Expression encoding fails for Seq/Map of Option[Seq/Date/Timestamp/BigDecimal]

2023-11-11 Thread Bruce Robbins (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45896?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17785234#comment-17785234 ] Bruce Robbins commented on SPARK-45896: --- I think I have a handle on this and will make a PR

[jira] [Updated] (SPARK-45896) Expression encoding fails for Seq/Map of Option[Seq/Date/Timestamp/BigDecimal]

2023-11-11 Thread Bruce Robbins (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45896?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bruce Robbins updated SPARK-45896: -- Description: The following action fails on 3.4.1, 3.5.0, and master: {noformat} scala> val df

[jira] [Updated] (SPARK-45896) Expression encoding fails for Seq/Map of Option[Seq/Date/Timestamp/BigDecimal]

2023-11-11 Thread Bruce Robbins (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45896?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bruce Robbins updated SPARK-45896: -- Summary: Expression encoding fails for Seq/Map of Option[Seq/Date/Timestamp/BigDecimal]

[jira] [Updated] (SPARK-45896) Expression encoding fails for Seq/Map of Option[Seq]

2023-11-11 Thread Bruce Robbins (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45896?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bruce Robbins updated SPARK-45896: -- Description: The following action fails on 3.4.1, 3.5.0, and master: {noformat} scala> val df

[jira] [Created] (SPARK-45896) Expression encoding fails for Seq/Map of Option[Seq]

2023-11-11 Thread Bruce Robbins (Jira)
Bruce Robbins created SPARK-45896: - Summary: Expression encoding fails for Seq/Map of Option[Seq] Key: SPARK-45896 URL: https://issues.apache.org/jira/browse/SPARK-45896 Project: Spark Issue

[jira] [Commented] (SPARK-45797) Discrepancies in PySpark DataFrame Results When Using Window Functions and Filters

2023-11-05 Thread Bruce Robbins (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45797?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17783015#comment-17783015 ] Bruce Robbins commented on SPARK-45797: --- I wonder if this is the same as SPARK-45543, which had

[jira] [Commented] (SPARK-45644) After upgrading to Spark 3.4.1 and 3.5.0 we receive RuntimeException "scala.Some is not a valid external type for schema of array"

2023-10-31 Thread Bruce Robbins (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45644?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17781531#comment-17781531 ] Bruce Robbins commented on SPARK-45644: --- I will look into it and try to submit a fix. If I can't,

[jira] [Commented] (SPARK-45644) After upgrading to Spark 3.4.1 and 3.5.0 we receive RuntimeException "scala.Some is not a valid external type for schema of array"

2023-10-31 Thread Bruce Robbins (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45644?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17781494#comment-17781494 ] Bruce Robbins commented on SPARK-45644: --- OK, I can reproduce. I will take a look. I will also try

[jira] [Commented] (SPARK-45644) After upgrading to Spark 3.4.1 and 3.5.0 we receive RuntimeException "scala.Some is not a valid external type for schema of array"

2023-10-30 Thread Bruce Robbins (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45644?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17781091#comment-17781091 ] Bruce Robbins commented on SPARK-45644: --- You can turn on display of the generated code by adding

[jira] [Updated] (SPARK-45580) Subquery changes the output schema of outer query

2023-10-21 Thread Bruce Robbins (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45580?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bruce Robbins updated SPARK-45580: -- Summary: Subquery changes the output schema of outer query (was: RewritePredicateSubquery

[jira] [Updated] (SPARK-45580) Subquery changes the output schema of the outer query

2023-10-21 Thread Bruce Robbins (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45580?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bruce Robbins updated SPARK-45580: -- Summary: Subquery changes the output schema of the outer query (was: Subquery changes the

[jira] [Resolved] (SPARK-45583) Spark SQL returning incorrect values for full outer join on keys with the same name.

2023-10-20 Thread Bruce Robbins (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45583?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bruce Robbins resolved SPARK-45583. --- Resolution: Fixed > Spark SQL returning incorrect values for full outer join on keys with

[jira] [Commented] (SPARK-45601) stackoverflow when executing rule ExtractWindowExpressions

2023-10-19 Thread Bruce Robbins (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45601?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1304#comment-1304 ] Bruce Robbins commented on SPARK-45601: --- Possibly SPARK-38666 > stackoverflow when executing rule

[jira] [Commented] (SPARK-45583) Spark SQL returning incorrect values for full outer join on keys with the same name.

2023-10-18 Thread Bruce Robbins (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17776783#comment-17776783 ] Bruce Robbins commented on SPARK-45583: --- Strangely, I cannot reproduce. Is some setting required?

[jira] [Commented] (SPARK-45580) RewritePredicateSubquery unexpectedly changes the output schema of certain queries

2023-10-17 Thread Bruce Robbins (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45580?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17776401#comment-17776401 ] Bruce Robbins commented on SPARK-45580: --- I'll make a PR in the coming days. >

[jira] [Updated] (SPARK-45580) RewritePredicateSubquery unexpectedly changes the output schema of certain queries

2023-10-17 Thread Bruce Robbins (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45580?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bruce Robbins updated SPARK-45580: -- Description: A query can have an incorrect output schema because of a subquery. Assume this

[jira] [Created] (SPARK-45580) RewritePredicateSubquery unexpectedly changes the output schema of certain queries

2023-10-17 Thread Bruce Robbins (Jira)
Bruce Robbins created SPARK-45580: - Summary: RewritePredicateSubquery unexpectedly changes the output schema of certain queries Key: SPARK-45580 URL: https://issues.apache.org/jira/browse/SPARK-45580

[jira] [Commented] (SPARK-45440) Incorrect summary counts from a CSV file

2023-10-06 Thread Bruce Robbins (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45440?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17772724#comment-17772724 ] Bruce Robbins commented on SPARK-45440: --- I added {{inferSchema=true}} as a datasource option in

[jira] [Created] (SPARK-45171) GenerateExec fails to initialize non-deterministic expressions before use

2023-09-14 Thread Bruce Robbins (Jira)
Bruce Robbins created SPARK-45171: - Summary: GenerateExec fails to initialize non-deterministic expressions before use Key: SPARK-45171 URL: https://issues.apache.org/jira/browse/SPARK-45171 Project:

[jira] [Commented] (SPARK-44912) Spark 3.4 multi-column sum slows with many columns

2023-09-10 Thread Bruce Robbins (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44912?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17763455#comment-17763455 ] Bruce Robbins commented on SPARK-44912: --- It looks like this was fixed with SPARK-45071. Your issue

[jira] [Updated] (SPARK-45106) percentile_cont gets internal error when user input fails runtime replacement's input type check

2023-09-08 Thread Bruce Robbins (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45106?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bruce Robbins updated SPARK-45106: -- Affects Version/s: 3.3.2 > percentile_cont gets internal error when user input fails runtime

[jira] [Created] (SPARK-45106) percentile_cont gets internal error when user input fails runtime replacement's input type check

2023-09-08 Thread Bruce Robbins (Jira)
Bruce Robbins created SPARK-45106: - Summary: percentile_cont gets internal error when user input fails runtime replacement's input type check Key: SPARK-45106 URL:

[jira] [Updated] (SPARK-44805) Data lost after union using spark.sql.parquet.enableNestedColumnVectorizedReader=true

2023-09-07 Thread Bruce Robbins (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44805?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bruce Robbins updated SPARK-44805: -- Affects Version/s: 3.4.1 > Data lost after union using >

[jira] [Commented] (SPARK-44805) Data lost after union using spark.sql.parquet.enableNestedColumnVectorizedReader=true

2023-09-07 Thread Bruce Robbins (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17762792#comment-17762792 ] Bruce Robbins commented on SPARK-44805: --- PR here: https://github.com/apache/spark/pull/42850 >

[jira] [Commented] (SPARK-44805) Data lost after union using spark.sql.parquet.enableNestedColumnVectorizedReader=true

2023-09-05 Thread Bruce Robbins (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17762234#comment-17762234 ] Bruce Robbins commented on SPARK-44805: --- I looked at this yesterday and I think I have a handle on

[jira] [Updated] (SPARK-44805) Data lost after union using spark.sql.parquet.enableNestedColumnVectorizedReader=true

2023-09-04 Thread Bruce Robbins (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44805?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bruce Robbins updated SPARK-44805: -- Labels: correctness (was: ) > Data lost after union using >

[jira] [Comment Edited] (SPARK-44805) Data lost after union using spark.sql.parquet.enableNestedColumnVectorizedReader=true

2023-08-14 Thread Bruce Robbins (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17754344#comment-17754344 ] Bruce Robbins edited comment on SPARK-44805 at 8/15/23 12:26 AM: -

[jira] [Commented] (SPARK-44805) Data lost after union using spark.sql.parquet.enableNestedColumnVectorizedReader=true

2023-08-14 Thread Bruce Robbins (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17754344#comment-17754344 ] Bruce Robbins commented on SPARK-44805: --- It seems to be some weird interaction between Parquet and

[jira] [Commented] (SPARK-44477) CheckAnalysis uses error subclass as an error class

2023-07-18 Thread Bruce Robbins (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44477?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17744314#comment-17744314 ] Bruce Robbins commented on SPARK-44477: --- PR here: https://github.com/apache/spark/pull/42064 >

[jira] [Created] (SPARK-44477) CheckAnalysis uses error subclass as an error class

2023-07-18 Thread Bruce Robbins (Jira)
Bruce Robbins created SPARK-44477: - Summary: CheckAnalysis uses error subclass as an error class Key: SPARK-44477 URL: https://issues.apache.org/jira/browse/SPARK-44477 Project: Spark Issue

[jira] [Updated] (SPARK-44251) Potential for incorrect results or NPE when full outer USING join has null key value

2023-07-01 Thread Bruce Robbins (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44251?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bruce Robbins updated SPARK-44251: -- Labels: correctness (was: ) > Potential for incorrect results or NPE when full outer USING

[jira] [Updated] (SPARK-44251) Potential for incorrect results or NPE when full outer USING join has null key value

2023-06-30 Thread Bruce Robbins (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44251?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bruce Robbins updated SPARK-44251: -- Affects Version/s: 3.3.2 > Potential for incorrect results or NPE when full outer USING join

[jira] [Updated] (SPARK-44251) Potential for incorrect results or NPE when full outer USING join has null key value

2023-06-30 Thread Bruce Robbins (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44251?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bruce Robbins updated SPARK-44251: -- Affects Version/s: 3.4.1 > Potential for incorrect results or NPE when full outer USING join

[jira] [Commented] (SPARK-44251) Potential for incorrect results or NPE when full outer USING join has null key value

2023-06-30 Thread Bruce Robbins (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44251?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17739180#comment-17739180 ] Bruce Robbins commented on SPARK-44251: --- PR can be found here:

[jira] [Commented] (SPARK-44251) Potential for incorrect results or NPE when full outer USING join has null key value

2023-06-29 Thread Bruce Robbins (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44251?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17738762#comment-17738762 ] Bruce Robbins commented on SPARK-44251: --- This is similar to, but not quite the same as

[jira] [Updated] (SPARK-44251) Potential for incorrect results or NPE when full outer USING join has null key value

2023-06-29 Thread Bruce Robbins (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44251?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bruce Robbins updated SPARK-44251: -- Summary: Potential for incorrect results or NPE when full outer USING join has null key value

[jira] [Created] (SPARK-44251) Potentially incorrect results or NPE when full outer USING join has null key value

2023-06-29 Thread Bruce Robbins (Jira)
Bruce Robbins created SPARK-44251: - Summary: Potentially incorrect results or NPE when full outer USING join has null key value Key: SPARK-44251 URL: https://issues.apache.org/jira/browse/SPARK-44251

[jira] [Commented] (SPARK-44132) nesting full outer joins confuses code generator

2023-06-21 Thread Bruce Robbins (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44132?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17735976#comment-17735976 ] Bruce Robbins commented on SPARK-44132: --- [~steven.aerts] Go for it! > nesting full outer joins

[jira] [Comment Edited] (SPARK-44132) nesting full outer joins confuses code generator

2023-06-21 Thread Bruce Robbins (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44132?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17735944#comment-17735944 ] Bruce Robbins edited comment on SPARK-44132 at 6/22/23 1:51 AM: You may

[jira] [Commented] (SPARK-44132) nesting full outer joins confuses code generator

2023-06-21 Thread Bruce Robbins (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44132?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17735944#comment-17735944 ] Bruce Robbins commented on SPARK-44132: --- You may have this figured out already, but in case not,

[jira] [Commented] (SPARK-44040) Incorrect result after count distinct

2023-06-13 Thread Bruce Robbins (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44040?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17732163#comment-17732163 ] Bruce Robbins commented on SPARK-44040: --- It seems this can be reproduced in {{spark-sql}} as well.

[jira] [Resolved] (SPARK-43843) Saving an AVRO file with Scala 2.13 results in NoClassDefFoundError

2023-05-28 Thread Bruce Robbins (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43843?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bruce Robbins resolved SPARK-43843. --- Resolution: Invalid > Saving an AVRO file with Scala 2.13 results in NoClassDefFoundError >

[jira] [Commented] (SPARK-43843) Saving an AVRO file with Scala 2.13 results in NoClassDefFoundError

2023-05-28 Thread Bruce Robbins (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43843?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17726988#comment-17726988 ] Bruce Robbins commented on SPARK-43843: --- Nevermind, I had an old

[jira] [Commented] (SPARK-43841) Non-existent column in projection of full outer join with USING results in StringIndexOutOfBoundsException

2023-05-28 Thread Bruce Robbins (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43841?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17726980#comment-17726980 ] Bruce Robbins commented on SPARK-43841: --- PR at https://github.com/apache/spark/pull/41353 >

[jira] [Updated] (SPARK-43843) Saving an AVRO file with Scala 2.13 results in NoClassDefFoundError

2023-05-28 Thread Bruce Robbins (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43843?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bruce Robbins updated SPARK-43843: -- Environment: Scala version 2.13.8 (Java HotSpot(TM) 64-Bit Server VM, Java 11.0.12) > Saving

[jira] [Created] (SPARK-43843) Saving an AVRO file with Scala 2.13 results in NoClassDefFoundError

2023-05-28 Thread Bruce Robbins (Jira)
Bruce Robbins created SPARK-43843: - Summary: Saving an AVRO file with Scala 2.13 results in NoClassDefFoundError Key: SPARK-43843 URL: https://issues.apache.org/jira/browse/SPARK-43843 Project: Spark

[jira] [Created] (SPARK-43841) Non-existent column in projection of full outer join with USING results in StringIndexOutOfBoundsException

2023-05-28 Thread Bruce Robbins (Jira)
Bruce Robbins created SPARK-43841: - Summary: Non-existent column in projection of full outer join with USING results in StringIndexOutOfBoundsException Key: SPARK-43841 URL:

[jira] [Commented] (SPARK-43718) References to a specific side's key in a USING join can have wrong nullability

2023-05-22 Thread Bruce Robbins (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43718?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17725143#comment-17725143 ] Bruce Robbins commented on SPARK-43718: --- PR here: https://github.com/apache/spark/pull/41267 >

[jira] [Updated] (SPARK-43718) References to a specific side's key in a USING join can have wrong nullability

2023-05-22 Thread Bruce Robbins (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43718?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bruce Robbins updated SPARK-43718: -- Description: Assume this data: {noformat} create or replace temp view t1 as values (1), (2),

[jira] [Updated] (SPARK-43718) References to a specific side's key in a USING join can have wrong nullability

2023-05-22 Thread Bruce Robbins (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43718?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bruce Robbins updated SPARK-43718: -- Affects Version/s: 3.3.2 > References to a specific side's key in a USING join can have wrong

[jira] [Updated] (SPARK-43718) References to a specific side's key in a USING join can have wrong nullability

2023-05-22 Thread Bruce Robbins (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43718?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bruce Robbins updated SPARK-43718: -- Affects Version/s: 3.4.0 > References to a specific side's key in a USING join can have wrong

[jira] [Commented] (SPARK-43718) References to a specific side's key in a USING join can have wrong nullability

2023-05-22 Thread Bruce Robbins (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43718?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17725122#comment-17725122 ] Bruce Robbins commented on SPARK-43718: --- I think I have a handle on this. I will submit in a PR in

[jira] [Updated] (SPARK-43718) References to a specific side's key in a USING join can have wrong nullability

2023-05-22 Thread Bruce Robbins (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43718?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bruce Robbins updated SPARK-43718: -- Labels: correctness (was: ) > References to a specific side's key in a USING join can have

[jira] [Updated] (SPARK-43718) References to a specific side's key in a USING join can have wrong nullability

2023-05-22 Thread Bruce Robbins (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43718?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bruce Robbins updated SPARK-43718: -- Description: Assume this data: {noformat} create or replace temp view t1 as values (1), (2),

[jira] [Updated] (SPARK-43718) References to a specific side's key in a USING join can have wrong nullability

2023-05-22 Thread Bruce Robbins (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43718?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bruce Robbins updated SPARK-43718: -- Description: Assume this data: {noformat} create or replace temp view t1 as values (1), (2),

[jira] [Created] (SPARK-43718) References to a specific side's key in a USING join can have wrong nullability

2023-05-22 Thread Bruce Robbins (Jira)
Bruce Robbins created SPARK-43718: - Summary: References to a specific side's key in a USING join can have wrong nullability Key: SPARK-43718 URL: https://issues.apache.org/jira/browse/SPARK-43718

[jira] [Created] (SPARK-43149) When CREATE USING fails to store metadata in metastore, data gets left around

2023-04-14 Thread Bruce Robbins (Jira)
Bruce Robbins created SPARK-43149: - Summary: When CREATE USING fails to store metadata in metastore, data gets left around Key: SPARK-43149 URL: https://issues.apache.org/jira/browse/SPARK-43149

[jira] [Updated] (SPARK-43149) When CTAS with USING fails to store metadata in metastore, data gets left around

2023-04-14 Thread Bruce Robbins (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43149?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bruce Robbins updated SPARK-43149: -- Summary: When CTAS with USING fails to store metadata in metastore, data gets left around

[jira] [Comment Edited] (SPARK-43113) Codegen error when full outer join's bound condition has multiple references to the same stream-side column

2023-04-14 Thread Bruce Robbins (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43113?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17711614#comment-17711614 ] Bruce Robbins edited comment on SPARK-43113 at 4/14/23 6:02 AM: PR here: 

[jira] [Commented] (SPARK-43113) Codegen error when full outer join's bound condition has multiple references to the same stream-side column

2023-04-12 Thread Bruce Robbins (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43113?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17711614#comment-17711614 ] Bruce Robbins commented on SPARK-43113: --- PR here: https://github.com/apache/spark/pull/40766/files

[jira] [Created] (SPARK-43113) Codegen error when full outer join's bound condition has multiple references to the same stream-side column

2023-04-12 Thread Bruce Robbins (Jira)
Bruce Robbins created SPARK-43113: - Summary: Codegen error when full outer join's bound condition has multiple references to the same stream-side column Key: SPARK-43113 URL:

[jira] [Commented] (SPARK-42937) Join with subquery in condition can fail with wholestage codegen and adaptive execution disabled

2023-03-27 Thread Bruce Robbins (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42937?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17705702#comment-17705702 ] Bruce Robbins commented on SPARK-42937: --- PR at https://github.com/apache/spark/pull/40569 > Join

[jira] [Updated] (SPARK-42937) Join with subquery in condition can fail with wholestage codegen and adaptive execution disabled

2023-03-27 Thread Bruce Robbins (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42937?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bruce Robbins updated SPARK-42937: -- Affects Version/s: 3.4.0 > Join with subquery in condition can fail with wholestage codegen

[jira] [Updated] (SPARK-42937) Join with subquery in condition can fail with wholestage codegen and adaptive execution disabled

2023-03-27 Thread Bruce Robbins (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42937?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bruce Robbins updated SPARK-42937: -- Affects Version/s: 3.3.2 > Join with subquery in condition can fail with wholestage codegen

[jira] [Created] (SPARK-42937) Join with subquery in condition can fail with wholestage codegen and adaptive execution disabled

2023-03-27 Thread Bruce Robbins (Jira)
Bruce Robbins created SPARK-42937: - Summary: Join with subquery in condition can fail with wholestage codegen and adaptive execution disabled Key: SPARK-42937 URL:

[jira] [Commented] (SPARK-42909) INSERT INTO with column list does not work

2023-03-23 Thread Bruce Robbins (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42909?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17704368#comment-17704368 ] Bruce Robbins commented on SPARK-42909: --- It looks like this capability landed in 3.4/3.5 with

[jira] [Commented] (SPARK-42401) Incorrect results or NPE when inserting null value into array using array_insert/array_append

2023-02-14 Thread Bruce Robbins (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42401?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17688759#comment-17688759 ] Bruce Robbins commented on SPARK-42401: --- There is another case: {noformat} spark-sql> select

[jira] [Updated] (SPARK-42401) Incorrect results or NPE when inserting null value into array using array_insert/array_append

2023-02-12 Thread Bruce Robbins (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42401?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bruce Robbins updated SPARK-42401: -- Summary: Incorrect results or NPE when inserting null value into array using

[jira] [Updated] (SPARK-42401) Incorrect results or NPE when inserting null value using array_insert/array_append

2023-02-10 Thread Bruce Robbins (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42401?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bruce Robbins updated SPARK-42401: -- Labels: correctness (was: ) > Incorrect results or NPE when inserting null value using >

[jira] [Created] (SPARK-42401) Incorrect results or NPE when inserting null value using array_insert/array_append

2023-02-10 Thread Bruce Robbins (Jira)
Bruce Robbins created SPARK-42401: - Summary: Incorrect results or NPE when inserting null value using array_insert/array_append Key: SPARK-42401 URL: https://issues.apache.org/jira/browse/SPARK-42401

[jira] [Updated] (SPARK-42384) Mask function's generated code does not handle null input

2023-02-08 Thread Bruce Robbins (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42384?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bruce Robbins updated SPARK-42384: -- Affects Version/s: 3.4.0 > Mask function's generated code does not handle null input >

[jira] [Created] (SPARK-42384) Mask function's generated code does not handle null input

2023-02-08 Thread Bruce Robbins (Jira)
Bruce Robbins created SPARK-42384: - Summary: Mask function's generated code does not handle null input Key: SPARK-42384 URL: https://issues.apache.org/jira/browse/SPARK-42384 Project: Spark

[jira] [Updated] (SPARK-41991) Interpreted mode subexpression elimination can throw exception during insert

2023-01-11 Thread Bruce Robbins (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41991?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bruce Robbins updated SPARK-41991: -- Affects Version/s: 3.3.1 > Interpreted mode subexpression elimination can throw exception

[jira] [Created] (SPARK-41991) Interpreted mode subexpression elimination can throw exception during insert

2023-01-11 Thread Bruce Robbins (Jira)
Bruce Robbins created SPARK-41991: - Summary: Interpreted mode subexpression elimination can throw exception during insert Key: SPARK-41991 URL: https://issues.apache.org/jira/browse/SPARK-41991

[jira] [Updated] (SPARK-41804) InterpretedUnsafeProjection doesn't properly handle an array of UDTs

2022-12-31 Thread Bruce Robbins (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41804?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bruce Robbins updated SPARK-41804: -- Description: Reproduction steps: {noformat} // create a file of vector data import

  1   2   3   4   5   >