[jira] [Updated] (SPARK-48752) Introduce `pyspark.logging` for improved structured logging for PySpark

2024-07-05 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48752?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-48752: --- Labels: pull-request-available (was: ) > Introduce `pyspark.logging` for improved structure

[jira] [Updated] (SPARK-48825) Unify the 'See Also' section formatting across PySpark docstrings

2024-07-05 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48825?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-48825: --- Labels: pull-request-available (was: ) > Unify the 'See Also' section formatting across PyS

[jira] [Created] (SPARK-48825) Unify the 'See Also' section formatting across PySpark docstrings

2024-07-05 Thread Allison Wang (Jira)
Allison Wang created SPARK-48825: Summary: Unify the 'See Also' section formatting across PySpark docstrings Key: SPARK-48825 URL: https://issues.apache.org/jira/browse/SPARK-48825 Project: Spark

[jira] [Updated] (SPARK-47047) Add support for transformWithState operator with state data source reader

2024-07-05 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47047?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-47047: --- Labels: pull-request-available (was: ) > Add support for transformWithState operator with s

[jira] [Created] (SPARK-48824) Add SQL syntax in create/replace table to create an identity column

2024-07-05 Thread Carmen Kwan (Jira)
Carmen Kwan created SPARK-48824: --- Summary: Add SQL syntax in create/replace table to create an identity column Key: SPARK-48824 URL: https://issues.apache.org/jira/browse/SPARK-48824 Project: Spark

[jira] [Updated] (SPARK-48822) Add examples section header to `format_number` docstring

2024-07-05 Thread Thomas Hart (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48822?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Hart updated SPARK-48822: Parent: SPARK-44728 Issue Type: Sub-task (was: Task) > Add examples section header to `fo

[jira] [Updated] (SPARK-48821) Support Update in DataFrameWriterV2

2024-07-05 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48821?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-48821: --- Labels: pull-request-available (was: ) > Support Update in DataFrameWriterV2 >

[jira] [Created] (SPARK-48822) Add examples section header to `format_number` docstring

2024-07-05 Thread Thomas Hart (Jira)
Thomas Hart created SPARK-48822: --- Summary: Add examples section header to `format_number` docstring Key: SPARK-48822 URL: https://issues.apache.org/jira/browse/SPARK-48822 Project: Spark Issue

[jira] [Created] (SPARK-48821) Support Update in DataFrameWriterV2

2024-07-05 Thread Szehon Ho (Jira)
Szehon Ho created SPARK-48821: - Summary: Support Update in DataFrameWriterV2 Key: SPARK-48821 URL: https://issues.apache.org/jira/browse/SPARK-48821 Project: Spark Issue Type: Task Comp

[jira] [Comment Edited] (SPARK-18105) LZ4 failed to decompress a stream of shuffled data

2024-07-05 Thread Cheng Pan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-18105?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17863386#comment-17863386 ] Cheng Pan edited comment on SPARK-18105 at 7/5/24 4:36 PM: --- th

[jira] [Commented] (SPARK-18105) LZ4 failed to decompress a stream of shuffled data

2024-07-05 Thread Cheng Pan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-18105?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17863386#comment-17863386 ] Cheng Pan commented on SPARK-18105: --- there is an XFS kernel bug identified by the Bili

[jira] [Commented] (SPARK-48725) Use lowerCaseCodePoints in string functions for UTF8_LCASE

2024-07-05 Thread psyren99 (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17863222#comment-17863222 ] psyren99 commented on SPARK-48725: -- [~uros-db] yes > Use lowerCaseCodePoints in string

[jira] [Resolved] (SPARK-48640) Perf improvement for format hex from byte array

2024-07-05 Thread Kent Yao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48640?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao resolved SPARK-48640. -- Resolution: Not A Problem > Perf improvement for format hex from byte array >

[jira] [Assigned] (SPARK-48792) INSERT with partial column list to table with char/varchar crashes

2024-07-05 Thread Kent Yao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48792?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao reassigned SPARK-48792: Assignee: Kent Yao > INSERT with partial column list to table with char/varchar crashes > ---

[jira] [Resolved] (SPARK-48792) INSERT with partial column list to table with char/varchar crashes

2024-07-05 Thread Kent Yao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48792?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao resolved SPARK-48792. -- Fix Version/s: 4.0.0 Resolution: Fixed > INSERT with partial column list to table with char/var

[jira] [Resolved] (SPARK-48815) Update environment when stoping connect session

2024-07-05 Thread XiDuo You (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48815?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] XiDuo You resolved SPARK-48815. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 47223 [https://github.c

[jira] [Assigned] (SPARK-48820) Correct the examples for Collate function

2024-07-05 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48820?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-48820: -- Assignee: Jiaan Geng (was: Apache Spark) > Correct the examples for Collate function

[jira] [Assigned] (SPARK-48820) Correct the examples for Collate function

2024-07-05 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48820?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-48820: -- Assignee: Apache Spark (was: Jiaan Geng) > Correct the examples for Collate function

[jira] [Assigned] (SPARK-48820) Correct the examples for Collate function

2024-07-05 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48820?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-48820: -- Assignee: Apache Spark (was: Jiaan Geng) > Correct the examples for Collate function

[jira] [Assigned] (SPARK-48820) Correct the examples for Collate function

2024-07-05 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48820?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-48820: -- Assignee: Jiaan Geng (was: Apache Spark) > Correct the examples for Collate function

[jira] [Assigned] (SPARK-48816) Perf improvement for CSV UnivocityParser with ANSI Intervals

2024-07-05 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48816?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-48816: -- Assignee: (was: Apache Spark) > Perf improvement for CSV UnivocityParser with ANS

[jira] [Assigned] (SPARK-48816) Perf improvement for CSV UnivocityParser with ANSI Intervals

2024-07-05 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48816?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-48816: -- Assignee: Apache Spark > Perf improvement for CSV UnivocityParser with ANSI Intervals

[jira] [Updated] (SPARK-48816) Perf improvement for CSV UnivocityParser with ANSI Intervals

2024-07-05 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48816?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-48816: --- Labels: pull-request-available (was: ) > Perf improvement for CSV UnivocityParser with ANSI

[jira] [Created] (SPARK-48820) Correct the examples for Collate function

2024-07-05 Thread Jiaan Geng (Jira)
Jiaan Geng created SPARK-48820: -- Summary: Correct the examples for Collate function Key: SPARK-48820 URL: https://issues.apache.org/jira/browse/SPARK-48820 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-48817) MultiInsert is split to multiple sql executions, resulting in no exchange reuse

2024-07-05 Thread Zhen Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48817?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhen Wang updated SPARK-48817: -- Attachment: image-2024-07-05-16-42-34-738.png > MultiInsert is split to multiple sql executions, resul

[jira] [Updated] (SPARK-48817) MultiInsert is split to multiple sql executions, resulting in no exchange reuse

2024-07-05 Thread Zhen Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48817?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhen Wang updated SPARK-48817: -- Description: MultiInsert is split to multiple sql executions, resulting in no exchange reuse.   Repr

[jira] [Updated] (SPARK-48817) MultiInsert is split to multiple sql executions, resulting in no exchange reuse

2024-07-05 Thread Zhen Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48817?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhen Wang updated SPARK-48817: -- Attachment: image-2024-07-05-16-42-17-817.png > MultiInsert is split to multiple sql executions, resul

[jira] [Updated] (SPARK-48817) MultiInsert is split to multiple sql executions, resulting in no exchange reuse

2024-07-05 Thread Zhen Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48817?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhen Wang updated SPARK-48817: -- Attachment: image-2024-07-05-16-42-46-500.png > MultiInsert is split to multiple sql executions, resul

[jira] [Updated] (SPARK-48817) MultiInsert is split to multiple sql executions, resulting in no exchange reuse

2024-07-05 Thread Zhen Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48817?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhen Wang updated SPARK-48817: -- Attachment: image-2024-07-05-16-42-27-033.png > MultiInsert is split to multiple sql executions, resul

[jira] [Updated] (SPARK-48817) MultiInsert is split to multiple sql executions, resulting in no exchange reuse

2024-07-05 Thread Zhen Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48817?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhen Wang updated SPARK-48817: -- Attachment: image-2024-07-05-16-42-01-973.png > MultiInsert is split to multiple sql executions, resul

[jira] [Commented] (SPARK-48725) Use lowerCaseCodePoints in string functions for UTF8_LCASE

2024-07-05 Thread Jira
[ https://issues.apache.org/jira/browse/SPARK-48725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17863160#comment-17863160 ] Uroš Bojanić commented on SPARK-48725: -- sorry, it's already in progress I can try

[jira] [Resolved] (SPARK-48767) Fix Variant error prompt

2024-07-05 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48767?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-48767. - Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 47162 [https://gith

[jira] [Updated] (SPARK-48810) [CONNECT] session.stop() should be best effort

2024-07-05 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48810?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-48810: --- Labels: pull-request-available (was: ) > [CONNECT] session.stop() should be best effort > -

[jira] [Updated] (SPARK-48819) Fix SchemaOfJson Expression to work with Collations

2024-07-05 Thread Mihailo Milosevic (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48819?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mihailo Milosevic updated SPARK-48819: -- Description: It was noticed that SchemaOfJson uses: {code:java} private lazy val jsonI

[jira] [Updated] (SPARK-48819) Fix SchemaOfJson Expression to work with Collations

2024-07-05 Thread Mihailo Milosevic (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48819?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mihailo Milosevic updated SPARK-48819: -- Description: It was noticed that SchemaOfJson uses ```private lazy val jsonInferSchema

[jira] [Updated] (SPARK-48819) Fix SchemaOfJson Expression to work with Collations

2024-07-05 Thread Mihailo Milosevic (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48819?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mihailo Milosevic updated SPARK-48819: -- Description: It was noticed that SchemaOfJson uses: ``` private lazy val jsonInferSc

[jira] [Updated] (SPARK-48819) Fix SchemaOfJson Expression to work with Collations

2024-07-05 Thread Mihailo Milosevic (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48819?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mihailo Milosevic updated SPARK-48819: -- Description: It was noticed that SchemaOfJson uses `private lazy val jsonInferSchema =

[jira] [Created] (SPARK-48819) Fix SchemaOfJson Expression to work with Collations

2024-07-05 Thread Mihailo Milosevic (Jira)
Mihailo Milosevic created SPARK-48819: - Summary: Fix SchemaOfJson Expression to work with Collations Key: SPARK-48819 URL: https://issues.apache.org/jira/browse/SPARK-48819 Project: Spark

[jira] [Updated] (SPARK-48818) Simplify `percentile` functions

2024-07-05 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48818?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-48818: --- Labels: pull-request-available (was: ) > Simplify `percentile` functions >

[jira] [Created] (SPARK-48818) Simplify `percentile` functions

2024-07-05 Thread Ruifeng Zheng (Jira)
Ruifeng Zheng created SPARK-48818: - Summary: Simplify `percentile` functions Key: SPARK-48818 URL: https://issues.apache.org/jira/browse/SPARK-48818 Project: Spark Issue Type: Improvement

[jira] [Resolved] (SPARK-47912) File format of insert overwrite dir does not take effect

2024-07-05 Thread Zhen Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47912?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhen Wang resolved SPARK-47912. --- Resolution: Won't Fix > File format of insert overwrite dir does not take effect > -

[jira] [Updated] (SPARK-48817) MultiInsert is split to multiple sql executions, resulting in no exchange reuse

2024-07-05 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48817?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-48817: --- Labels: pull-request-available (was: ) > MultiInsert is split to multiple sql executions, r

[jira] [Updated] (SPARK-48817) MultiInsert is split to multiple sql executions, resulting in no exchange reuse

2024-07-05 Thread Zhen Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48817?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhen Wang updated SPARK-48817: -- Description: MultiInsert is split to multiple sql executions, resulting in no exchange reuse.   Repr

[jira] [Updated] (SPARK-48817) MultiInsert is split to multiple sql executions, resulting in no exchange reuse

2024-07-05 Thread Zhen Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48817?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhen Wang updated SPARK-48817: -- Attachment: image-2024-07-05-15-00-09-181.png > MultiInsert is split to multiple sql executions, resul

[jira] [Updated] (SPARK-48817) MultiInsert is split to multiple sql executions, resulting in no exchange reuse

2024-07-05 Thread Zhen Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48817?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhen Wang updated SPARK-48817: -- Description: MultiInsert is split to multiple sql executions, resulting in no exchange reuse.   Repr

[jira] [Updated] (SPARK-48817) MultiInsert is split to multiple sql executions, resulting in no exchange reuse

2024-07-05 Thread Zhen Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48817?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhen Wang updated SPARK-48817: -- Attachment: image-2024-07-05-15-00-17-693.png > MultiInsert is split to multiple sql executions, resul

[jira] [Updated] (SPARK-48817) MultiInsert is split to multiple sql executions, resulting in no exchange reuse

2024-07-05 Thread Zhen Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48817?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhen Wang updated SPARK-48817: -- Attachment: image-2024-07-05-15-00-01-805.png > MultiInsert is split to multiple sql executions, resul

[jira] [Updated] (SPARK-48817) MultiInsert is split to multiple sql executions, resulting in no exchange reuse

2024-07-05 Thread Zhen Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48817?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhen Wang updated SPARK-48817: -- Attachment: image-2024-07-05-14-59-55-291.png > MultiInsert is split to multiple sql executions, resul

[jira] [Updated] (SPARK-48817) MultiInsert is split to multiple sql executions, resulting in no exchange reuse

2024-07-05 Thread Zhen Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48817?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhen Wang updated SPARK-48817: -- Description: MultiInsert is split to multiple sql executions, resulting in no exchange reuse.   Repr

[jira] [Created] (SPARK-48817) MultiInsert is split to multiple sql executions, resulting in no exchange reuse

2024-07-05 Thread Zhen Wang (Jira)
Zhen Wang created SPARK-48817: - Summary: MultiInsert is split to multiple sql executions, resulting in no exchange reuse Key: SPARK-48817 URL: https://issues.apache.org/jira/browse/SPARK-48817 Project: Sp

[jira] [Updated] (SPARK-48817) MultiInsert is split to multiple sql executions, resulting in no exchange reuse

2024-07-05 Thread Zhen Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48817?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhen Wang updated SPARK-48817: -- Attachment: image-2024-07-05-14-59-35-340.png > MultiInsert is split to multiple sql executions, resul