[jira] [Updated] (SPARK-49555) SQL Pipe Syntax

2024-09-09 Thread Daniel (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49555?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel updated SPARK-49555: --- Description: This umbrella Jira ticket tracks implementing new support for issuing SQL queries using pipe s

[jira] [Created] (SPARK-49563) WINDOW operator

2024-09-09 Thread Daniel (Jira)
Daniel created SPARK-49563: -- Summary: WINDOW operator Key: SPARK-49563 URL: https://issues.apache.org/jira/browse/SPARK-49563 Project: Spark Issue Type: Sub-task Components: SQL Affect

[jira] [Created] (SPARK-49562) AGGREGATE operator

2024-09-09 Thread Daniel (Jira)
Daniel created SPARK-49562: -- Summary: AGGREGATE operator Key: SPARK-49562 URL: https://issues.apache.org/jira/browse/SPARK-49562 Project: Spark Issue Type: Sub-task Components: SQL Aff

[jira] [Updated] (SPARK-49555) SQL Pipe Syntax

2024-09-09 Thread Daniel (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49555?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel updated SPARK-49555: --- Description: This umbrella Jira ticket tracks implementing new support for issuing SQL queries using pipe s

[jira] [Updated] (SPARK-49555) SQL Pipe Syntax

2024-09-09 Thread Daniel (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49555?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel updated SPARK-49555: --- Description: This umbrella Jira ticket tracks implementing new support for issuing SQL queries using pipe s

[jira] [Created] (SPARK-49505) Create SQL functions to generate random strings or numbers within ranges

2024-09-03 Thread Daniel (Jira)
Daniel created SPARK-49505: -- Summary: Create SQL functions to generate random strings or numbers within ranges Key: SPARK-49505 URL: https://issues.apache.org/jira/browse/SPARK-49505 Project: Spark

[jira] [Resolved] (SPARK-49503) Create SQL functions to compute the name of a day or month as a string

2024-09-03 Thread Daniel (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49503?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel resolved SPARK-49503. Resolution: Won't Fix > Create SQL functions to compute the name of a day or month as a string > -

[jira] [Created] (SPARK-49503) Create SQL functions to compute the name of a day or month as a string

2024-09-03 Thread Daniel (Jira)
Daniel created SPARK-49503: -- Summary: Create SQL functions to compute the name of a day or month as a string Key: SPARK-49503 URL: https://issues.apache.org/jira/browse/SPARK-49503 Project: Spark I

[jira] [Created] (SPARK-49364) Create DataFrame API endpoints for new 'zeroifnull' and 'nullifzero' functions

2024-08-22 Thread Daniel (Jira)
Daniel created SPARK-49364: -- Summary: Create DataFrame API endpoints for new 'zeroifnull' and 'nullifzero' functions Key: SPARK-49364 URL: https://issues.apache.org/jira/browse/SPARK-49364 Project: Spark

[jira] [Created] (SPARK-49342) Make TO_AVRO SQL function 'jsonFormatSchema' argument optional

2024-08-21 Thread Daniel (Jira)
Daniel created SPARK-49342: -- Summary: Make TO_AVRO SQL function 'jsonFormatSchema' argument optional Key: SPARK-49342 URL: https://issues.apache.org/jira/browse/SPARK-49342 Project: Spark Issue Typ

[jira] [Created] (SPARK-49306) Create SQL function aliases for 'zeroifnull' and 'nullifzero'

2024-08-19 Thread Daniel (Jira)
Daniel created SPARK-49306: -- Summary: Create SQL function aliases for 'zeroifnull' and 'nullifzero' Key: SPARK-49306 URL: https://issues.apache.org/jira/browse/SPARK-49306 Project: Spark Issue Type

[jira] [Commented] (SPARK-49302) Implement pipe operator syntax in Spark SQL

2024-08-19 Thread Daniel (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17874961#comment-17874961 ] Daniel commented on SPARK-49302: I can help with this one > Implement pipe operator syn

[jira] [Created] (SPARK-49302) Implement pipe operator syntax in Spark SQL

2024-08-19 Thread Daniel (Jira)
Daniel created SPARK-49302: -- Summary: Implement pipe operator syntax in Spark SQL Key: SPARK-49302 URL: https://issues.apache.org/jira/browse/SPARK-49302 Project: Spark Issue Type: Improvement

[jira] [Updated] (SPARK-48966) Improve error message with invalid unresolved column reference in UDTF call

2024-07-22 Thread Daniel (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48966?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel updated SPARK-48966: --- Description: This bug covers improving an error message in the event of invalid UDTF calls. For example: {

[jira] [Created] (SPARK-48966) Improve error message with invalid unresolved column reference in UDTF call

2024-07-22 Thread Daniel (Jira)
Daniel created SPARK-48966: -- Summary: Improve error message with invalid unresolved column reference in UDTF call Key: SPARK-48966 URL: https://issues.apache.org/jira/browse/SPARK-48966 Project: Spark

[jira] [Created] (SPARK-48688) Return reasonable error when calling SQL to_avro and from_avro functions but Avro is not loaded by default

2024-06-21 Thread Daniel (Jira)
Daniel created SPARK-48688: -- Summary: Return reasonable error when calling SQL to_avro and from_avro functions but Avro is not loaded by default Key: SPARK-48688 URL: https://issues.apache.org/jira/browse/SPARK-48688

[jira] [Updated] (SPARK-48545) Create to_avro and from_avro SQL functions to match PySpark equivalent

2024-06-13 Thread Daniel (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48545?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel updated SPARK-48545: --- Summary: Create to_avro and from_avro SQL functions to match PySpark equivalent (was: Create from_avro SQL

[jira] [Resolved] (SPARK-48607) Support more types of expressions in VALUES lists

2024-06-12 Thread Daniel (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48607?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel resolved SPARK-48607. Resolution: Duplicate > Support more types of expressions in VALUES lists > --

[jira] [Created] (SPARK-48607) Support more types of expressions in VALUES lists

2024-06-12 Thread Daniel (Jira)
Daniel created SPARK-48607: -- Summary: Support more types of expressions in VALUES lists Key: SPARK-48607 URL: https://issues.apache.org/jira/browse/SPARK-48607 Project: Spark Issue Type: Bug

[jira] [Created] (SPARK-48566) [Bug] Partition indices are incorrect when UDTF analyze() uses both select and partitionColumns

2024-06-07 Thread Daniel (Jira)
Daniel created SPARK-48566: -- Summary: [Bug] Partition indices are incorrect when UDTF analyze() uses both select and partitionColumns Key: SPARK-48566 URL: https://issues.apache.org/jira/browse/SPARK-48566 P

[jira] [Created] (SPARK-48546) Fix ExpressionEncoder after replacing NullPointerExceptions with proper error classes in AssertNotNull expression

2024-06-05 Thread Daniel (Jira)
Daniel created SPARK-48546: -- Summary: Fix ExpressionEncoder after replacing NullPointerExceptions with proper error classes in AssertNotNull expression Key: SPARK-48546 URL: https://issues.apache.org/jira/browse/SPARK-48

[jira] [Created] (SPARK-48545) Create from_avro SQL function to match PySpark equivalent

2024-06-05 Thread Daniel (Jira)
Daniel created SPARK-48545: -- Summary: Create from_avro SQL function to match PySpark equivalent Key: SPARK-48545 URL: https://issues.apache.org/jira/browse/SPARK-48545 Project: Spark Issue Type: Bug

[jira] [Created] (SPARK-48461) Replace NullPointerExceptions with proper error classes in AssertNotNull expression

2024-05-29 Thread Daniel (Jira)
Daniel created SPARK-48461: -- Summary: Replace NullPointerExceptions with proper error classes in AssertNotNull expression Key: SPARK-48461 URL: https://issues.apache.org/jira/browse/SPARK-48461 Project: Spar

[jira] [Commented] (SPARK-48180) Analyzer bug with multiple ORDER BY items for input table argument

2024-05-07 Thread Daniel (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48180?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17844471#comment-17844471 ] Daniel commented on SPARK-48180: Fix here: [https://github.com/apache/spark/pull/46451]

[jira] [Commented] (SPARK-48180) Analyzer bug with multiple ORDER BY items for input table argument

2024-05-07 Thread Daniel (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48180?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17844467#comment-17844467 ] Daniel commented on SPARK-48180: The bug is that parentheses are required around the two

[jira] [Created] (SPARK-48180) Analyzer bug with multiple ORDER BY items for input table argument

2024-05-07 Thread Daniel (Jira)
Daniel created SPARK-48180: -- Summary: Analyzer bug with multiple ORDER BY items for input table argument Key: SPARK-48180 URL: https://issues.apache.org/jira/browse/SPARK-48180 Project: Spark Issue

[jira] [Comment Edited] (SPARK-47578) Spark core: Migrate logWarn with variables to structured logging framework

2024-04-30 Thread Daniel (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47578?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17835552#comment-17835552 ] Daniel edited comment on SPARK-47578 at 4/30/24 7:11 PM: - Edit:

[jira] [Comment Edited] (SPARK-47578) Spark core: Migrate logWarn with variables to structured logging framework

2024-04-30 Thread Daniel (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47578?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17835552#comment-17835552 ] Daniel edited comment on SPARK-47578 at 4/30/24 6:57 PM: - Edit:

[jira] [Comment Edited] (SPARK-47578) Spark core: Migrate logWarn with variables to structured logging framework

2024-04-30 Thread Daniel (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47578?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17835552#comment-17835552 ] Daniel edited comment on SPARK-47578 at 4/30/24 6:43 PM: - Edit:

[jira] [Comment Edited] (SPARK-47578) Spark core: Migrate logWarn with variables to structured logging framework

2024-04-30 Thread Daniel (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47578?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17835552#comment-17835552 ] Daniel edited comment on SPARK-47578 at 4/30/24 5:55 PM: - Edit:

[jira] [Comment Edited] (SPARK-47578) Spark core: Migrate logWarn with variables to structured logging framework

2024-04-29 Thread Daniel (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47578?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17835552#comment-17835552 ] Daniel edited comment on SPARK-47578 at 4/29/24 11:11 PM: -- Edit

[jira] [Comment Edited] (SPARK-47578) Spark core: Migrate logWarn with variables to structured logging framework

2024-04-29 Thread Daniel (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47578?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17835552#comment-17835552 ] Daniel edited comment on SPARK-47578 at 4/29/24 10:21 PM: -- Edit

[jira] [Comment Edited] (SPARK-47578) Spark core: Migrate logWarn with variables to structured logging framework

2024-04-29 Thread Daniel (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47578?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17835552#comment-17835552 ] Daniel edited comment on SPARK-47578 at 4/29/24 10:19 PM: -- Edit

[jira] [Comment Edited] (SPARK-47578) Spark core: Migrate logWarn with variables to structured logging framework

2024-04-29 Thread Daniel (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47578?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17835552#comment-17835552 ] Daniel edited comment on SPARK-47578 at 4/29/24 10:01 PM: -- Edit

[jira] [Comment Edited] (SPARK-47578) Spark core: Migrate logWarn with variables to structured logging framework

2024-04-29 Thread Daniel (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47578?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17835552#comment-17835552 ] Daniel edited comment on SPARK-47578 at 4/29/24 9:58 PM: - Edit:

[jira] [Comment Edited] (SPARK-47578) Spark core: Migrate logWarn with variables to structured logging framework

2024-04-29 Thread Daniel (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47578?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17835552#comment-17835552 ] Daniel edited comment on SPARK-47578 at 4/29/24 9:44 PM: - Edit:

[jira] [Comment Edited] (SPARK-47578) Spark core: Migrate logWarn with variables to structured logging framework

2024-04-29 Thread Daniel (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47578?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17835552#comment-17835552 ] Daniel edited comment on SPARK-47578 at 4/29/24 9:38 PM: - Edit:

[jira] [Comment Edited] (SPARK-47578) Spark core: Migrate logWarn with variables to structured logging framework

2024-04-29 Thread Daniel (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47578?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17835552#comment-17835552 ] Daniel edited comment on SPARK-47578 at 4/29/24 9:26 PM: - Edit:

[jira] [Comment Edited] (SPARK-47578) Spark core: Migrate logWarn with variables to structured logging framework

2024-04-29 Thread Daniel (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47578?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17835552#comment-17835552 ] Daniel edited comment on SPARK-47578 at 4/29/24 9:15 PM: - Edit:

[jira] [Comment Edited] (SPARK-47578) Spark core: Migrate logWarn with variables to structured logging framework

2024-04-29 Thread Daniel (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47578?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17835552#comment-17835552 ] Daniel edited comment on SPARK-47578 at 4/29/24 8:54 PM: - Edit:

[jira] [Comment Edited] (SPARK-47578) Spark core: Migrate logWarn with variables to structured logging framework

2024-04-29 Thread Daniel (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47578?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17835552#comment-17835552 ] Daniel edited comment on SPARK-47578 at 4/29/24 8:23 PM: - Edit:

[jira] [Comment Edited] (SPARK-47578) Spark core: Migrate logWarn with variables to structured logging framework

2024-04-29 Thread Daniel (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47578?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17835552#comment-17835552 ] Daniel edited comment on SPARK-47578 at 4/29/24 8:07 PM: - Edit:

[jira] [Comment Edited] (SPARK-47578) Spark core: Migrate logWarn with variables to structured logging framework

2024-04-29 Thread Daniel (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47578?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17835552#comment-17835552 ] Daniel edited comment on SPARK-47578 at 4/29/24 4:50 PM: - Edit:

[jira] [Commented] (SPARK-47585) SQL core: Migrate logInfo with variables to structured logging framework

2024-04-29 Thread Daniel (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17842094#comment-17842094 ] Daniel commented on SPARK-47585: [~panbingkun] here are the affected files: spark$ grep

[jira] [Commented] (SPARK-47578) Spark core: Migrate logWarn with variables to structured logging framework

2024-04-29 Thread Daniel (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47578?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17842092#comment-17842092 ] Daniel commented on SPARK-47578: Sounds good, I start on this one now > Spark core: Mig

[jira] [Comment Edited] (SPARK-47597) Streaming: Migrate logInfo with variables to structured logging framework

2024-04-23 Thread Daniel (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47597?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17840213#comment-17840213 ] Daniel edited comment on SPARK-47597 at 4/23/24 9:44 PM: - spark$

[jira] [Comment Edited] (SPARK-47597) Streaming: Migrate logInfo with variables to structured logging framework

2024-04-23 Thread Daniel (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47597?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17840213#comment-17840213 ] Daniel edited comment on SPARK-47597 at 4/23/24 9:14 PM: - spark$

[jira] [Comment Edited] (SPARK-47597) Streaming: Migrate logInfo with variables to structured logging framework

2024-04-23 Thread Daniel (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47597?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17840213#comment-17840213 ] Daniel edited comment on SPARK-47597 at 4/23/24 9:04 PM: - spark$

[jira] [Comment Edited] (SPARK-47597) Streaming: Migrate logInfo with variables to structured logging framework

2024-04-23 Thread Daniel (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47597?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17840213#comment-17840213 ] Daniel edited comment on SPARK-47597 at 4/23/24 8:31 PM: - spark$

[jira] [Comment Edited] (SPARK-47597) Streaming: Migrate logInfo with variables to structured logging framework

2024-04-23 Thread Daniel (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47597?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17840213#comment-17840213 ] Daniel edited comment on SPARK-47597 at 4/23/24 8:27 PM: - spark$

[jira] [Commented] (SPARK-47597) Streaming: Migrate logInfo with variables to structured logging framework

2024-04-23 Thread Daniel (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47597?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17840213#comment-17840213 ] Daniel commented on SPARK-47597: spark$ grep logInfo sql/core/src/main/* -R | cut -d ':'

[jira] [Commented] (SPARK-47578) Spark core: Migrate logWarn with variables to structured logging framework

2024-04-09 Thread Daniel (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47578?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17835552#comment-17835552 ] Daniel commented on SPARK-47578: spark-2$ grep logWarning sql/core/src/main/* -R | cut -

[jira] [Comment Edited] (SPARK-47583) SQL core: Migrate logError with variables to structured logging framework

2024-04-09 Thread Daniel (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17835546#comment-17835546 ] Daniel edited comment on SPARK-47583 at 4/9/24 11:26 PM: - spark$

[jira] [Comment Edited] (SPARK-47583) SQL core: Migrate logError with variables to structured logging framework

2024-04-09 Thread Daniel (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17835546#comment-17835546 ] Daniel edited comment on SPARK-47583 at 4/9/24 11:16 PM: - spark$

[jira] [Comment Edited] (SPARK-47583) SQL core: Migrate logError with variables to structured logging framework

2024-04-09 Thread Daniel (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17835546#comment-17835546 ] Daniel edited comment on SPARK-47583 at 4/9/24 11:14 PM: - spark$

[jira] [Comment Edited] (SPARK-47583) SQL core: Migrate logError with variables to structured logging framework

2024-04-09 Thread Daniel (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17835546#comment-17835546 ] Daniel edited comment on SPARK-47583 at 4/9/24 11:14 PM: - spark$

[jira] [Comment Edited] (SPARK-47583) SQL core: Migrate logError with variables to structured logging framework

2024-04-09 Thread Daniel (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17835546#comment-17835546 ] Daniel edited comment on SPARK-47583 at 4/9/24 11:07 PM: - spark$

[jira] [Comment Edited] (SPARK-47583) SQL core: Migrate logError with variables to structured logging framework

2024-04-09 Thread Daniel (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17835546#comment-17835546 ] Daniel edited comment on SPARK-47583 at 4/9/24 11:07 PM: - spark$

[jira] [Comment Edited] (SPARK-47583) SQL core: Migrate logError with variables to structured logging framework

2024-04-09 Thread Daniel (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17835546#comment-17835546 ] Daniel edited comment on SPARK-47583 at 4/9/24 10:59 PM: - spark$

[jira] [Comment Edited] (SPARK-47583) SQL core: Migrate logError with variables to structured logging framework

2024-04-09 Thread Daniel (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17835546#comment-17835546 ] Daniel edited comment on SPARK-47583 at 4/9/24 10:55 PM: - spark$

[jira] [Comment Edited] (SPARK-47583) SQL core: Migrate logError with variables to structured logging framework

2024-04-09 Thread Daniel (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17835546#comment-17835546 ] Daniel edited comment on SPARK-47583 at 4/9/24 10:53 PM: - spark$

[jira] [Comment Edited] (SPARK-47583) SQL core: Migrate logError with variables to structured logging framework

2024-04-09 Thread Daniel (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17835546#comment-17835546 ] Daniel edited comment on SPARK-47583 at 4/9/24 10:47 PM: - spark$

[jira] [Comment Edited] (SPARK-47583) SQL core: Migrate logError with variables to structured logging framework

2024-04-09 Thread Daniel (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17835546#comment-17835546 ] Daniel edited comment on SPARK-47583 at 4/9/24 10:32 PM: - spark$

[jira] [Comment Edited] (SPARK-47583) SQL core: Migrate logError with variables to structured logging framework

2024-04-09 Thread Daniel (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17835546#comment-17835546 ] Daniel edited comment on SPARK-47583 at 4/9/24 10:30 PM: - spark$

[jira] [Commented] (SPARK-47583) SQL core: Migrate logError with variables to structured logging framework

2024-04-09 Thread Daniel (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17835546#comment-17835546 ] Daniel commented on SPARK-47583: spark$ grep logError sql/core/src/main/* -R | cut -d ':

[jira] [Updated] (SPARK-47509) Incorrect results for LambdaFunctions or HigherOrderFunctions in subquery expression plans

2024-03-21 Thread Daniel (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47509?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel updated SPARK-47509: --- Summary: Incorrect results for LambdaFunctions or HigherOrderFunctions in subquery expression plans (was: I

[jira] [Updated] (SPARK-47509) Incorrect results for LambdaFunction or HigherOrderFunction within subquery expressions

2024-03-21 Thread Daniel (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47509?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel updated SPARK-47509: --- Summary: Incorrect results for LambdaFunction or HigherOrderFunction within subquery expressions (was: Inco

[jira] [Created] (SPARK-47509) Incorrect results for LambdaFunction or HigherOrderFunction with subquery expressions

2024-03-21 Thread Daniel (Jira)
Daniel created SPARK-47509: -- Summary: Incorrect results for LambdaFunction or HigherOrderFunction with subquery expressions Key: SPARK-47509 URL: https://issues.apache.org/jira/browse/SPARK-47509 Project: Sp

[jira] [Resolved] (SPARK-47032) Create API for 'analyze' method to send input column(s) to output table unchanged

2024-03-04 Thread Daniel (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47032?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel resolved SPARK-47032. Resolution: Won't Fix After investigating this, we found that any performance benefits from skipping send

[jira] [Created] (SPARK-47214) Create API for 'analyze' method to differentiate constant NULL arguments and other types of arguments

2024-02-28 Thread Daniel (Jira)
Daniel created SPARK-47214: -- Summary: Create API for 'analyze' method to differentiate constant NULL arguments and other types of arguments Key: SPARK-47214 URL: https://issues.apache.org/jira/browse/SPARK-47214

[jira] [Closed] (SPARK-46638) Create API to acquire execution memory for 'eval' and 'terminate' methods

2024-02-13 Thread Daniel (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46638?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel closed SPARK-46638. -- > Create API to acquire execution memory for 'eval' and 'terminate' methods >

[jira] [Created] (SPARK-47032) Create API for 'analyze' method to send input column(s) to output table unchanged

2024-02-13 Thread Daniel (Jira)
Daniel created SPARK-47032: -- Summary: Create API for 'analyze' method to send input column(s) to output table unchanged Key: SPARK-47032 URL: https://issues.apache.org/jira/browse/SPARK-47032 Project: Spark

[jira] [Created] (SPARK-47002) Enforce that 'AnalyzeResult' 'orderBy' field is a list of pyspark.sql.functions.OrderingColumn

2024-02-07 Thread Daniel (Jira)
Daniel created SPARK-47002: -- Summary: Enforce that 'AnalyzeResult' 'orderBy' field is a list of pyspark.sql.functions.OrderingColumn Key: SPARK-47002 URL: https://issues.apache.org/jira/browse/SPARK-47002 Pr

[jira] [Created] (SPARK-46966) Create API for 'analyze' method to indicate subset of input table columns to select

2024-02-02 Thread Daniel (Jira)
Daniel created SPARK-46966: -- Summary: Create API for 'analyze' method to indicate subset of input table columns to select Key: SPARK-46966 URL: https://issues.apache.org/jira/browse/SPARK-46966 Project: Spar

[jira] [Resolved] (SPARK-46638) Create API to acquire execution memory for 'eval' and 'terminate' methods

2024-02-02 Thread Daniel (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46638?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel resolved SPARK-46638. Resolution: Won't Fix > Create API to acquire execution memory for 'eval' and 'terminate' methods > --

[jira] (SPARK-46890) CSV fails on a column with default and without enforcing schema

2024-01-29 Thread Daniel (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46890 ] Daniel deleted comment on SPARK-46890: was (Author: JIRAUSER285772): I think this `tokenIndexArr` within Spark's `UnivocityParser` class has different values in the passing and failing cases: {c

[jira] [Commented] (SPARK-46890) CSV fails on a column with default and without enforcing schema

2024-01-29 Thread Daniel (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46890?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17812072#comment-17812072 ] Daniel commented on SPARK-46890: [~maxgekk] I created a bug fix here: [https://github.c

[jira] [Comment Edited] (SPARK-46890) CSV fails on a column with default and without enforcing schema

2024-01-29 Thread Daniel (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46890?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17812059#comment-17812059 ] Daniel edited comment on SPARK-46890 at 1/29/24 9:27 PM: - I thin

[jira] [Commented] (SPARK-46890) CSV fails on a column with default and without enforcing schema

2024-01-29 Thread Daniel (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46890?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17812059#comment-17812059 ] Daniel commented on SPARK-46890: I think this `tokenIndexArr` within Spark's `UnivocityP

[jira] [Commented] (SPARK-46890) CSV fails on a column with default and without enforcing schema

2024-01-29 Thread Daniel (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46890?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17812058#comment-17812058 ] Daniel commented on SPARK-46890: The bug happens when the Univocity parser is converting

[jira] [Updated] (SPARK-46890) CSV fails on a column with default and without enforcing schema

2024-01-29 Thread Daniel (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46890?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel updated SPARK-46890: --- Attachment: image-2024-01-29-13-22-05-326.png > CSV fails on a column with default and without enforcing sch

[jira] [Commented] (SPARK-46890) CSV fails on a column with default and without enforcing schema

2024-01-29 Thread Daniel (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46890?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17812051#comment-17812051 ] Daniel commented on SPARK-46890: The exception comes from here: [https://github.com/apa

[jira] [Commented] (SPARK-46890) CSV fails on a column with default and without enforcing schema

2024-01-29 Thread Daniel (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46890?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17812047#comment-17812047 ] Daniel commented on SPARK-46890: Thanks [~maxgekk] both of the above tests reproduce the

[jira] [Commented] (SPARK-46890) CSV fails on a column with default and without enforcing schema

2024-01-29 Thread Daniel (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46890?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17812040#comment-17812040 ] Daniel commented on SPARK-46890: I tried the exact command in the bug description, and i

[jira] [Commented] (SPARK-46890) CSV fails on a column with default and without enforcing schema

2024-01-29 Thread Daniel (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46890?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17812034#comment-17812034 ] Daniel commented on SPARK-46890: This unit test does not seem to reproduce the problem:

[jira] [Comment Edited] (SPARK-46890) CSV fails on a column with default and without enforcing schema

2024-01-29 Thread Daniel (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46890?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17812034#comment-17812034 ] Daniel edited comment on SPARK-46890 at 1/29/24 7:31 PM: - This u

[jira] [Commented] (SPARK-46890) CSV fails on a column with default and without enforcing schema

2024-01-29 Thread Daniel (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46890?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17812002#comment-17812002 ] Daniel commented on SPARK-46890: Thanks [~maxgekk] for writing down the details here. It

[jira] [Updated] (SPARK-46879) Run optimizer on REPLACE TABLE column defaults

2024-01-26 Thread Daniel (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46879?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel updated SPARK-46879: --- Summary: Run optimizer on REPLACE TABLE column defaults (was: Run optimizer on CREATE TABLE column defaults

[jira] [Created] (SPARK-46879) Run optimizer on CREATE TABLE column defaults

2024-01-26 Thread Daniel (Jira)
Daniel created SPARK-46879: -- Summary: Run optimizer on CREATE TABLE column defaults Key: SPARK-46879 URL: https://issues.apache.org/jira/browse/SPARK-46879 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-46849) Run optimizer on CREATE TABLE column defaults

2024-01-24 Thread Daniel (Jira)
Daniel created SPARK-46849: -- Summary: Run optimizer on CREATE TABLE column defaults Key: SPARK-46849 URL: https://issues.apache.org/jira/browse/SPARK-46849 Project: Spark Issue Type: Sub-task

[jira] [Updated] (SPARK-46638) Create API to acquire execution memory for 'eval' and 'terminate' methods

2024-01-09 Thread Daniel (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46638?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel updated SPARK-46638: --- Summary: Create API to acquire execution memory for 'eval' and 'terminate' methods (was: Create API to acqu

[jira] [Created] (SPARK-46638) Create API to acquire execution memory for eval and terminate methods

2024-01-09 Thread Daniel (Jira)
Daniel created SPARK-46638: -- Summary: Create API to acquire execution memory for eval and terminate methods Key: SPARK-46638 URL: https://issues.apache.org/jira/browse/SPARK-46638 Project: Spark Is

[jira] [Commented] (SPARK-46370) Fix bug when querying from table after changing defaults

2023-12-11 Thread Daniel (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46370?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17795514#comment-17795514 ] Daniel commented on SPARK-46370: The bug is here:   {{drop table if exists t;}} {{crea

[jira] [Created] (SPARK-46370) Fix bug when querying from table after changing defaults

2023-12-11 Thread Daniel (Jira)
Daniel created SPARK-46370: -- Summary: Fix bug when querying from table after changing defaults Key: SPARK-46370 URL: https://issues.apache.org/jira/browse/SPARK-46370 Project: Spark Issue Type: Sub-

[jira] [Created] (SPARK-46040) Update API for 'analyze' partitioning/ordering columns to support general expressions

2023-11-21 Thread Daniel (Jira)
Daniel created SPARK-46040: -- Summary: Update API for 'analyze' partitioning/ordering columns to support general expressions Key: SPARK-46040 URL: https://issues.apache.org/jira/browse/SPARK-46040 Project: Sp

[jira] [Created] (SPARK-45810) Create API to stop consuming rows from the input table

2023-11-06 Thread Daniel (Jira)
Daniel created SPARK-45810: -- Summary: Create API to stop consuming rows from the input table Key: SPARK-45810 URL: https://issues.apache.org/jira/browse/SPARK-45810 Project: Spark Issue Type: Sub-ta

[jira] [Created] (SPARK-45746) Return specific error messages if UDTF 'analyze' method accepts or returns wrong values

2023-10-31 Thread Daniel (Jira)
Daniel created SPARK-45746: -- Summary: Return specific error messages if UDTF 'analyze' method accepts or returns wrong values Key: SPARK-45746 URL: https://issues.apache.org/jira/browse/SPARK-45746 Project:

[jira] [Created] (SPARK-45523) Return useful error message if UDTF returns None for non-nullable column

2023-10-12 Thread Daniel (Jira)
Daniel created SPARK-45523: -- Summary: Return useful error message if UDTF returns None for non-nullable column Key: SPARK-45523 URL: https://issues.apache.org/jira/browse/SPARK-45523 Project: Spark

[jira] [Created] (SPARK-45402) Add API for 'analyze' method to return a buffer to be consumed on each class creation

2023-10-03 Thread Daniel (Jira)
Daniel created SPARK-45402: -- Summary: Add API for 'analyze' method to return a buffer to be consumed on each class creation Key: SPARK-45402 URL: https://issues.apache.org/jira/browse/SPARK-45402 Project: Sp

[jira] [Created] (SPARK-45362) Project out PARTITION BY expressions before 'eval' method consumes input rows

2023-09-27 Thread Daniel (Jira)
Daniel created SPARK-45362: -- Summary: Project out PARTITION BY expressions before 'eval' method consumes input rows Key: SPARK-45362 URL: https://issues.apache.org/jira/browse/SPARK-45362 Project: Spark

  1   2   3   >