[jira] [Resolved] (SPARK-44837) Improve error message for ALTER TABLE ALTER COLUMN on partition columns in non-delta tables

2023-10-19 Thread Max Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44837?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk resolved SPARK-44837. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 42524

[jira] [Commented] (SPARK-45608) Migrate SchemaColumnConvertNotSupportedException onto DATATYPE_MISMATCH error classes

2023-10-19 Thread Max Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1574#comment-1574 ] Max Gekk commented on SPARK-45608: -- The ticket came from

[jira] [Resolved] (SPARK-45366) Remove productHash from TreeNode

2023-10-19 Thread BingKun Pan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45366?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] BingKun Pan resolved SPARK-45366. - Resolution: Duplicate > Remove productHash from TreeNode > > >

[jira] [Updated] (SPARK-44753) XML: Add Python and sparkR binding including Spark Connect

2023-10-19 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44753?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-44753: --- Labels: pull-request-available (was: ) > XML: Add Python and sparkR binding including

[jira] [Resolved] (SPARK-45289) ClassCastException when reading Delta table on AWS S3

2023-10-19 Thread Tanawat Panmongkol (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45289?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tanawat Panmongkol resolved SPARK-45289. Resolution: Fixed > ClassCastException when reading Delta table on AWS S3 >

[jira] [Created] (SPARK-45614) Assign name to _LEGACY_ERROR_TEMP_215[6,7,8]

2023-10-19 Thread Deng Ziming (Jira)
Deng Ziming created SPARK-45614: --- Summary: Assign name to _LEGACY_ERROR_TEMP_215[6,7,8] Key: SPARK-45614 URL: https://issues.apache.org/jira/browse/SPARK-45614 Project: Spark Issue Type:

[jira] [Resolved] (SPARK-45591) Upgrade ASM to 9.6

2023-10-19 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45591?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie resolved SPARK-45591. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 43431

[jira] [Comment Edited] (SPARK-45610) Handle "Auto-application to `()` is deprecated."

2023-10-19 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45610?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1539#comment-1539 ] Yang Jie edited comment on SPARK-45610 at 10/20/23 2:31 AM: Okay, I can

[jira] [Commented] (SPARK-45610) Handle "Auto-application to `()` is deprecated."

2023-10-19 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45610?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1539#comment-1539 ] Yang Jie commented on SPARK-45610: -- Okay, I can start preparing this PR.           > Handle

[jira] [Commented] (SPARK-44405) Reduce code duplication in group-based DELETE and MERGE tests

2023-10-19 Thread Min Zhao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44405?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1538#comment-1538 ] Min Zhao commented on SPARK-44405: -- Hello, are you working on it and would you like me to try it? >

[jira] [Resolved] (SPARK-45601) stackoverflow when executing rule ExtractWindowExpressions

2023-10-19 Thread JacobZheng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45601?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] JacobZheng resolved SPARK-45601. Fix Version/s: 3.3.0 Resolution: Resolved > stackoverflow when executing rule

[jira] [Commented] (SPARK-45601) stackoverflow when executing rule ExtractWindowExpressions

2023-10-19 Thread JacobZheng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45601?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1536#comment-1536 ] JacobZheng commented on SPARK-45601: Got it, Thanks [~bersprockets] > stackoverflow when executing

[jira] [Updated] (SPARK-45613) Expose DeterministicLevel as a DeveloperApi

2023-10-19 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45613?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-45613: --- Labels: pull-request-available (was: ) > Expose DeterministicLevel as a DeveloperApi >

[jira] [Created] (SPARK-45613) Expose DeterministicLevel as a DeveloperApi

2023-10-19 Thread Mridul Muralidharan (Jira)
Mridul Muralidharan created SPARK-45613: --- Summary: Expose DeterministicLevel as a DeveloperApi Key: SPARK-45613 URL: https://issues.apache.org/jira/browse/SPARK-45613 Project: Spark

[jira] [Resolved] (SPARK-45603) merge_spark_pr shall notice us about GITHUB_OAUTH_KEY expiry

2023-10-19 Thread Ruifeng Zheng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45603?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng resolved SPARK-45603. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 43447

[jira] [Assigned] (SPARK-45603) merge_spark_pr shall notice us about GITHUB_OAUTH_KEY expiry

2023-10-19 Thread Ruifeng Zheng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45603?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng reassigned SPARK-45603: - Assignee: Kent Yao > merge_spark_pr shall notice us about GITHUB_OAUTH_KEY expiry >

[jira] [Updated] (SPARK-45612) Allow cached RDDs to be migrated to fallback storage during decommission

2023-10-19 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45612?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-45612: --- Labels: pull-request-available (was: ) > Allow cached RDDs to be migrated to fallback

[jira] [Assigned] (SPARK-45584) Execution fails when there are subqueries in TakeOrderedAndProjectExec

2023-10-19 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45584?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-45584: --- Assignee: Allison Wang > Execution fails when there are subqueries in

[jira] [Resolved] (SPARK-45584) Execution fails when there are subqueries in TakeOrderedAndProjectExec

2023-10-19 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45584?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-45584. - Fix Version/s: 3.5.1 4.0.0 Resolution: Fixed Issue resolved by pull

[jira] [Resolved] (SPARK-45611) spark.python.pyspark.sql.functions Typo at date_format Function

2023-10-19 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45611?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-45611. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 43442

[jira] [Assigned] (SPARK-45611) spark.python.pyspark.sql.functions Typo at date_format Function

2023-10-19 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45611?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-45611: Assignee: Mete Can Akar > spark.python.pyspark.sql.functions Typo at date_format

[jira] [Resolved] (SPARK-45428) Add Matomo analytics to all released docs pages

2023-10-19 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45428?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-45428. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 43454

[jira] [Updated] (SPARK-45611) spark.python.pyspark.sql.functions Typo at date_format Function

2023-10-19 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45611?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-45611: --- Labels: pull-request-available (was: ) > spark.python.pyspark.sql.functions Typo at

[jira] [Created] (SPARK-45612) Allow cached RDDs to be migrated to fallback storage during decommission

2023-10-19 Thread Frank Yin (Jira)
Frank Yin created SPARK-45612: - Summary: Allow cached RDDs to be migrated to fallback storage during decommission Key: SPARK-45612 URL: https://issues.apache.org/jira/browse/SPARK-45612 Project: Spark

[jira] [Commented] (SPARK-45583) Spark SQL returning incorrect values for full outer join on keys with the same name.

2023-10-19 Thread Huw (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1502#comment-1502 ] Huw commented on SPARK-45583: - Ahh, apologies, it looks like I was running 3.4.1 when I found this issue.

[jira] [Updated] (SPARK-45583) Spark SQL returning incorrect values for full outer join on keys with the same name.

2023-10-19 Thread Huw (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45583?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Huw updated SPARK-45583: Affects Version/s: 3.4.1 (was: 3.5.0) > Spark SQL returning incorrect values for full

[jira] [Updated] (SPARK-45583) Spark SQL returning incorrect values for full outer join on keys with the same name.

2023-10-19 Thread Huw (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45583?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Huw updated SPARK-45583: Fix Version/s: 3.5.0 > Spark SQL returning incorrect values for full outer join on keys with the > same name. >

[jira] [Updated] (SPARK-45604) Converting timestamp_ntz to array can cause NPE or SEGFAULT on parquet vectorized reader

2023-10-19 Thread Zamil Majdy (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45604?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zamil Majdy updated SPARK-45604: Description: Repro: ``` val path = "/tmp/sample_parquet_file" spark.sql("SELECT

[jira] [Updated] (SPARK-45604) Converting timestamp_ntz to array can cause NPE or SEGFAULT on parquet vectorized reader

2023-10-19 Thread Zamil Majdy (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45604?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zamil Majdy updated SPARK-45604: Description: Repro: val path = "/tmp/sample_parquet_file" spark.sql("SELECT CAST('2019-01-01'

[jira] [Updated] (SPARK-45604) Converting timestamp_ntz to array can cause NPE or SEGFAULT on parquet vectorized reader

2023-10-19 Thread Zamil Majdy (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45604?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zamil Majdy updated SPARK-45604: Description: Repro: {{{}{}}}``` val path = "/tmp/zamil/timestamp" spark.sql("SELECT

[jira] [Updated] (SPARK-45604) Converting timestamp_ntz to array can cause NPE or SEGFAULT on parquet vectorized reader

2023-10-19 Thread Zamil Majdy (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45604?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zamil Majdy updated SPARK-45604: Description: Repro: {{val path = "/tmp/someparquetfile"}} {{spark.sql("SELECT CAST('2019-01-01

[jira] [Updated] (SPARK-45604) Converting timestamp_ntz to array can cause NPE or SEGFAULT on parquet vectorized reader

2023-10-19 Thread Zamil Majdy (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45604?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zamil Majdy updated SPARK-45604: Description: Repro: {{val path = "/tmp/someparquetfile" spark.sql("SELECT CAST('2019-01-01

[jira] [Updated] (SPARK-45611) spark.python.pyspark.sql.functions Typo at date_format Function

2023-10-19 Thread Mete Can Akar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45611?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mete Can Akar updated SPARK-45611: -- Description: In the spark.python.pyspark.sql.functions module, at the {{date_format}}

[jira] [Updated] (SPARK-45611) spark.python.pyspark.sql.functions Typo at date_format Function

2023-10-19 Thread Mete Can Akar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45611?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mete Can Akar updated SPARK-45611: -- Description: In the spark.python.pyspark.sql.functions module, at the {{date_format}}

[jira] [Updated] (SPARK-45611) spark.python.pyspark.sql.functions Typo at date_format Function

2023-10-19 Thread Mete Can Akar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45611?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mete Can Akar updated SPARK-45611: -- Attachment: image-2023-10-19-19-46-22-918.png > spark.python.pyspark.sql.functions Typo at

[jira] [Updated] (SPARK-45611) spark.python.pyspark.sql.functions Typo at date_format Function

2023-10-19 Thread Mete Can Akar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45611?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mete Can Akar updated SPARK-45611: -- Description: In the spark.python.pyspark.sql.functions module, at the {{date_format}}

[jira] [Updated] (SPARK-45611) spark.python.pyspark.sql.functions Typo at date_format Function

2023-10-19 Thread Mete Can Akar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45611?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mete Can Akar updated SPARK-45611: -- Description: In the spark.python.pyspark.sql.functions module, at the {{date_format}}

[jira] [Updated] (SPARK-45611) spark.python.pyspark.sql.functions Typo at date_format Function

2023-10-19 Thread Mete Can Akar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45611?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mete Can Akar updated SPARK-45611: -- Description: In the spark.python.pyspark.sql.functions module, at the {{date_format}}

[jira] [Created] (SPARK-45611) spark.python.pyspark.sql.functions Typo at date_format Function

2023-10-19 Thread Mete Can Akar (Jira)
Mete Can Akar created SPARK-45611: - Summary: spark.python.pyspark.sql.functions Typo at date_format Function Key: SPARK-45611 URL: https://issues.apache.org/jira/browse/SPARK-45611 Project: Spark

[jira] [Commented] (SPARK-45610) Handle "Auto-application to `()` is deprecated."

2023-10-19 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45610?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1405#comment-1405 ] Sean R. Owen commented on SPARK-45610: -- I think it's better to make big changes at major version

[jira] [Commented] (SPARK-45610) Handle "Auto-application to `()` is deprecated."

2023-10-19 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45610?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1396#comment-1396 ] Yang Jie commented on SPARK-45610: -- In Spark, this involves a massive amount of cases. Since this is a

[jira] [Updated] (SPARK-45610) Handle "Auto-application to `()` is deprecated."

2023-10-19 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45610?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie updated SPARK-45610: - Description: For the following case, a compile warning will be issued in Scala 2.13:   {code:java}

[jira] [Created] (SPARK-45610) Handle "Auto-application to `()` is deprecated."

2023-10-19 Thread Yang Jie (Jira)
Yang Jie created SPARK-45610: Summary: Handle "Auto-application to `()` is deprecated." Key: SPARK-45610 URL: https://issues.apache.org/jira/browse/SPARK-45610 Project: Spark Issue Type:

[jira] [Commented] (SPARK-25078) Standalone does not work with spark.authenticate.secret and deploy-mode=cluster

2023-10-19 Thread Yaroslav (Jira)
[ https://issues.apache.org/jira/browse/SPARK-25078?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1377#comment-1377 ] Yaroslav commented on SPARK-25078: -- Hi, this issue is still reproducible. In SPARK-8129  they changed

[jira] [Updated] (SPARK-45609) Include SqlState in SparkThrowable proto message

2023-10-19 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45609?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-45609: --- Labels: pull-request-available (was: ) > Include SqlState in SparkThrowable proto message

[jira] [Created] (SPARK-45609) Include SqlState in SparkThrowable proto message

2023-10-19 Thread Yihong He (Jira)
Yihong He created SPARK-45609: - Summary: Include SqlState in SparkThrowable proto message Key: SPARK-45609 URL: https://issues.apache.org/jira/browse/SPARK-45609 Project: Spark Issue Type: Test

[jira] [Comment Edited] (SPARK-45598) Delta table 3.0.0 not working with Spark Connect 3.5.0

2023-10-19 Thread Faiz Halde (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45598?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1246#comment-1246 ] Faiz Halde edited comment on SPARK-45598 at 10/19/23 4:04 PM: -- Hi

[jira] [Updated] (SPARK-45368) Remove scala2.12 compatibility logic for DoubleType, FloatType, Decimal

2023-10-19 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45368?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-45368: --- Labels: pull-request-available (was: ) > Remove scala2.12 compatibility logic for

[jira] [Commented] (SPARK-45601) stackoverflow when executing rule ExtractWindowExpressions

2023-10-19 Thread Bruce Robbins (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45601?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1304#comment-1304 ] Bruce Robbins commented on SPARK-45601: --- Possibly SPARK-38666 > stackoverflow when executing rule

[jira] [Created] (SPARK-45608) Migrate SchemaColumnConvertNotSupportedException onto DATATYPE_MISMATCH error classes

2023-10-19 Thread Zamil Majdy (Jira)
Zamil Majdy created SPARK-45608: --- Summary: Migrate SchemaColumnConvertNotSupportedException onto DATATYPE_MISMATCH error classes Key: SPARK-45608 URL: https://issues.apache.org/jira/browse/SPARK-45608

[jira] [Updated] (SPARK-45569) Assign name to _LEGACY_ERROR_TEMP_2152

2023-10-19 Thread Deng Ziming (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45569?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Deng Ziming updated SPARK-45569: Description: Choose a proper name for the error class *_LEGACY_ERROR_TEMP_2152* defined in

[jira] [Updated] (SPARK-45573) Assign name to _LEGACY_ERROR_TEMP_2153

2023-10-19 Thread Deng Ziming (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45573?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Deng Ziming updated SPARK-45573: Description: Choose a proper name for the error class *_LEGACY_ERROR_TEMP_2153* defined in

[jira] [Commented] (SPARK-45598) Delta table 3.0.0 not working with Spark Connect 3.5.0

2023-10-19 Thread Faiz Halde (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45598?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1246#comment-1246 ] Faiz Halde commented on SPARK-45598: Hi [~sdaberdaku] , corrected the title. I tested it with 3.0.0

[jira] [Updated] (SPARK-45598) Delta table 3.0.0 not working with Spark Connect 3.5.0

2023-10-19 Thread Faiz Halde (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45598?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Faiz Halde updated SPARK-45598: --- Summary: Delta table 3.0.0 not working with Spark Connect 3.5.0 (was: Delta table 3.0-rc2 not

[jira] [Resolved] (SPARK-45573) Assign name to _LEGACY_ERROR_TEMP_2153

2023-10-19 Thread Max Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45573?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk resolved SPARK-45573. -- Resolution: Fixed > Assign name to _LEGACY_ERROR_TEMP_2153 > -- >

[jira] [Commented] (SPARK-45573) Assign name to _LEGACY_ERROR_TEMP_2153

2023-10-19 Thread Max Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45573?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1238#comment-1238 ] Max Gekk commented on SPARK-45573: -- Resolved by https://github.com/apache/spark/pull/43414 > Assign

[jira] [Resolved] (SPARK-45543) InferWindowGroupLimit causes bug if the other window functions haven't the same window frame as the rank-like functions

2023-10-19 Thread Jiaan Geng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45543?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jiaan Geng resolved SPARK-45543. Fix Version/s: 3.5.1 4.0.0 Resolution: Fixed Issue resolved by pull

[jira] [Commented] (SPARK-45598) Delta table 3.0-rc2 not working with Spark Connect 3.5.0

2023-10-19 Thread Sebastian Daberdaku (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45598?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1172#comment-1172 ] Sebastian Daberdaku commented on SPARK-45598: - Hello [~haldefaiz], you need to use the

[jira] [Commented] (SPARK-45289) ClassCastException when reading Delta table on AWS S3

2023-10-19 Thread Sebastian Daberdaku (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45289?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1170#comment-1170 ] Sebastian Daberdaku commented on SPARK-45289: - Hello [~tanawatpan], you need to use the

[jira] [Assigned] (SPARK-45428) Add Matomo analytics to all released docs pages

2023-10-19 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45428?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-45428: -- Assignee: BingKun Pan (was: Apache Spark) > Add Matomo analytics to all released

[jira] [Assigned] (SPARK-45428) Add Matomo analytics to all released docs pages

2023-10-19 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45428?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-45428: -- Assignee: Apache Spark (was: BingKun Pan) > Add Matomo analytics to all released

[jira] [Assigned] (SPARK-45428) Add Matomo analytics to all released docs pages

2023-10-19 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45428?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-45428: -- Assignee: Apache Spark (was: BingKun Pan) > Add Matomo analytics to all released

[jira] [Assigned] (SPARK-45428) Add Matomo analytics to all released docs pages

2023-10-19 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45428?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-45428: -- Assignee: BingKun Pan (was: Apache Spark) > Add Matomo analytics to all released

[jira] [Assigned] (SPARK-45428) Add Matomo analytics to all released docs pages

2023-10-19 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45428?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-45428: -- Assignee: Apache Spark (was: BingKun Pan) > Add Matomo analytics to all released

[jira] [Assigned] (SPARK-45428) Add Matomo analytics to all released docs pages

2023-10-19 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45428?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-45428: -- Assignee: BingKun Pan (was: Apache Spark) > Add Matomo analytics to all released

[jira] [Assigned] (SPARK-45605) Replace `s.c.MapOps.mapValues` with `s.c.MapOps.view.mapValues`

2023-10-19 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45605?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-45605: -- Assignee: (was: Apache Spark) >Replace `s.c.MapOps.mapValues` with

[jira] [Assigned] (SPARK-45428) Add Matomo analytics to all released docs pages

2023-10-19 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45428?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-45428: -- Assignee: BingKun Pan (was: Apache Spark) > Add Matomo analytics to all released

[jira] [Assigned] (SPARK-45428) Add Matomo analytics to all released docs pages

2023-10-19 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45428?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-45428: -- Assignee: Apache Spark (was: BingKun Pan) > Add Matomo analytics to all released

[jira] [Assigned] (SPARK-45428) Add Matomo analytics to all released docs pages

2023-10-19 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45428?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-45428: -- Assignee: BingKun Pan (was: Apache Spark) > Add Matomo analytics to all released

[jira] [Assigned] (SPARK-45428) Add Matomo analytics to all released docs pages

2023-10-19 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45428?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-45428: -- Assignee: Apache Spark (was: BingKun Pan) > Add Matomo analytics to all released

[jira] [Updated] (SPARK-45428) Add Matomo analytics to all released docs pages

2023-10-19 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45428?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-45428: --- Labels: pull-request-available (was: ) > Add Matomo analytics to all released docs pages >

[jira] [Assigned] (SPARK-45594) Auto repartition before writing data into partitioned or bucket table

2023-10-19 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45594?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-45594: -- Assignee: (was: Apache Spark) > Auto repartition before writing data into

[jira] [Assigned] (SPARK-45594) Auto repartition before writing data into partitioned or bucket table

2023-10-19 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45594?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-45594: -- Assignee: Apache Spark > Auto repartition before writing data into partitioned or

[jira] [Assigned] (SPARK-45594) Auto repartition before writing data into partitioned or bucket table

2023-10-19 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45594?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-45594: -- Assignee: (was: Apache Spark) > Auto repartition before writing data into

[jira] [Assigned] (SPARK-45607) Collapse repartition operators with a project

2023-10-19 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45607?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-45607: -- Assignee: (was: Apache Spark) > Collapse repartition operators with a project >

[jira] [Assigned] (SPARK-45594) Auto repartition before writing data into partitioned or bucket table

2023-10-19 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45594?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-45594: -- Assignee: Apache Spark > Auto repartition before writing data into partitioned or

[jira] [Assigned] (SPARK-45607) Collapse repartition operators with a project

2023-10-19 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45607?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-45607: -- Assignee: Apache Spark > Collapse repartition operators with a project >

[jira] [Updated] (SPARK-45607) Collapse repartition operators with a project

2023-10-19 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45607?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-45607: --- Labels: pull-request-available (was: ) > Collapse repartition operators with a project >

[jira] [Updated] (SPARK-45607) Collapse repartition operators with a project

2023-10-19 Thread Wan Kun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45607?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wan Kun updated SPARK-45607: Summary: Collapse repartition operators with a project (was: Collapse repartition operators with

[jira] [Updated] (SPARK-45607) Collapse repartition operators with project

2023-10-19 Thread Wan Kun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45607?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wan Kun updated SPARK-45607: Summary: Collapse repartition operators with project (was: Collapse repartitions with project) >

[jira] [Updated] (SPARK-45607) Collapse repartitions with project

2023-10-19 Thread Wan Kun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45607?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wan Kun updated SPARK-45607: Summary: Collapse repartitions with project (was: Collapse repartition with project) > Collapse

[jira] [Created] (SPARK-45607) Collapse repartition with project

2023-10-19 Thread Wan Kun (Jira)
Wan Kun created SPARK-45607: --- Summary: Collapse repartition with project Key: SPARK-45607 URL: https://issues.apache.org/jira/browse/SPARK-45607 Project: Spark Issue Type: Improvement

[jira] [Updated] (SPARK-45604) Converting timestamp_ntz to array can cause NPE or SEGFAULT on parquet vectorized reader

2023-10-19 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45604?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-45604: --- Labels: pull-request-available (was: ) > Converting timestamp_ntz to array can cause NPE

[jira] [Updated] (SPARK-45554) Introduce flexible parameter to assertSchemaEqual

2023-10-19 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45554?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-45554: --- Labels: pull-request-available (was: ) > Introduce flexible parameter to assertSchemaEqual

[jira] [Updated] (SPARK-45606) Release restrictions on multi-layer runtime filter

2023-10-19 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45606?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-45606: --- Labels: pull-request-available (was: ) > Release restrictions on multi-layer runtime

[jira] [Updated] (SPARK-45605) Replace `s.c.MapOps.mapValues` with `s.c.MapOps.view.mapValues`

2023-10-19 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45605?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-45605: --- Labels: pull-request-available (was: ) >Replace `s.c.MapOps.mapValues` with

[jira] [Updated] (SPARK-45605) Replace `s.c.MapOps.mapValues` with `s.c.MapOps.view.mapValues`

2023-10-19 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45605?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie updated SPARK-45605: - Description: {code:java} @deprecated("Use .view.mapValues(f). A future version will include a strict

[jira] [Created] (SPARK-45606) Release restrictions on multi-layer runtime filter

2023-10-19 Thread Jiaan Geng (Jira)
Jiaan Geng created SPARK-45606: -- Summary: Release restrictions on multi-layer runtime filter Key: SPARK-45606 URL: https://issues.apache.org/jira/browse/SPARK-45606 Project: Spark Issue Type:

[jira] [Updated] (SPARK-45605) Replace `s.c.MapOps.mapValues` with `s.c.MapOps.view.mapValues`

2023-10-19 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45605?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie updated SPARK-45605: - Description: {code:java} // code placeholder {code} >Replace `s.c.MapOps.mapValues` with

[jira] [Created] (SPARK-45605) Replace `s.c.MapOps.mapValues` with `s.c.MapOps.view.mapValues`

2023-10-19 Thread Yang Jie (Jira)
Yang Jie created SPARK-45605: Summary:Replace `s.c.MapOps.mapValues` with `s.c.MapOps.view.mapValues` Key: SPARK-45605 URL: https://issues.apache.org/jira/browse/SPARK-45605 Project: Spark

[jira] [Updated] (SPARK-45604) Converting timestamp_ntz to array can cause NPE or SEGFAULT on parquet vectorized reader

2023-10-19 Thread Zamil Majdy (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45604?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zamil Majdy updated SPARK-45604: Description: Repro: ``` val path = "/tmp/someparquetfile" val df = sql("SELECT MAP('key',

[jira] [Updated] (SPARK-45604) Converting timestamp_ntz to array can cause NPE or SEGFAULT on parquet vectorized reader

2023-10-19 Thread Zamil Majdy (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45604?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zamil Majdy updated SPARK-45604: Description: Repro: ``` spark.conf.set("spark.databricks.photon.enabled", "false") val path =

[jira] [Updated] (SPARK-45604) Converting timestamp_ntz to array can cause NPE or SEGFAULT on parquet vectorized reader

2023-10-19 Thread Zamil Majdy (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45604?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zamil Majdy updated SPARK-45604: Description: Repro: ``` val path = "/tmp/zamil/timestamp" val df = sql("SELECT MAP('key',

[jira] [Updated] (SPARK-45604) Converting timestamp_ntz to array can cause NPE or SEGFAULT on parquet vectorized reader

2023-10-19 Thread Zamil Majdy (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45604?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zamil Majdy updated SPARK-45604: Description: Repro:   ``` spark.conf.set("spark.databricks.photon.enabled", "false") val path

[jira] [Created] (SPARK-45604) Converting timestamp_ntz to array can cause NPE or SEGFAULT on parquet vectorized reader

2023-10-19 Thread Zamil Majdy (Jira)
Zamil Majdy created SPARK-45604: --- Summary: Converting timestamp_ntz to array can cause NPE or SEGFAULT on parquet vectorized reader Key: SPARK-45604 URL: https://issues.apache.org/jira/browse/SPARK-45604

[jira] [Updated] (SPARK-45604) Converting timestamp_ntz to array can cause NPE or SEGFAULT on parquet vectorized reader

2023-10-19 Thread Zamil Majdy (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45604?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zamil Majdy updated SPARK-45604: Description: Repro:   {{```}} spark.conf.set("spark.databricks.photon.enabled", "false") val

[jira] [Commented] (SPARK-44734) Add documentation for type casting rules in Python UDFs/UDTFs

2023-10-19 Thread BingKun Pan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44734?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1064#comment-1064 ] BingKun Pan commented on SPARK-44734: - [~phildakin] Sorry, I didn't see that the previous PR is

[jira] [Resolved] (SPARK-45569) Assign name to _LEGACY_ERROR_TEMP_2152

2023-10-19 Thread Max Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45569?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk resolved SPARK-45569. -- Resolution: Fixed Issue resolved by pull request 43414 [https://github.com/apache/spark/pull/43414]