[jira] [Updated] (SPARK-47475) Jar Download Under K8s Cluster Mode Causes Executors Scaling Issues

2024-03-19 Thread Jiale Tan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47475?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jiale Tan updated SPARK-47475: -- Component/s: Spark Core > Jar Download Under K8s Cluster Mode Causes Executors Scaling Issues > -

[jira] [Updated] (SPARK-47479) Optimize cannot write data to relations with multiple paths error log

2024-03-19 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47479?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-47479: --- Labels: pull-request-available (was: ) > Optimize cannot write data to relations with multi

[jira] [Updated] (SPARK-47475) Jar Download Under K8s Cluster Mode Causes Executors Scaling Issues

2024-03-19 Thread Jiale Tan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47475?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jiale Tan updated SPARK-47475: -- Description: {*}Context{*}: To submit spark jobs to Kubernetes under cluster mode, the {{spark-submit}

[jira] [Updated] (SPARK-47475) Jar Download Under K8s Cluster Mode Causes Executors Scaling Issues

2024-03-19 Thread Jiale Tan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47475?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jiale Tan updated SPARK-47475: -- Description: {*}Context{*}: To submit spark jobs to Kubernetes under cluster mode, the {{spark-submit}

[jira] [Created] (SPARK-47479) Optimize cannot write data to relations with multiple paths error log

2024-03-19 Thread xiaoping.huang (Jira)
xiaoping.huang created SPARK-47479: -- Summary: Optimize cannot write data to relations with multiple paths error log Key: SPARK-47479 URL: https://issues.apache.org/jira/browse/SPARK-47479 Project: Sp

[jira] [Updated] (SPARK-47475) Jar Download Under K8s Cluster Mode Causes Executors Scaling Issues

2024-03-19 Thread Jiale Tan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47475?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jiale Tan updated SPARK-47475: -- Description: {*}Context{*}: To submit spark jobs to Kubernetes under cluster mode, the {{spark-submit}

[jira] [Updated] (SPARK-47475) Jar Download Under K8s Cluster Mode Causes Executors Scaling Issues

2024-03-19 Thread Jiale Tan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47475?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jiale Tan updated SPARK-47475: -- Description: {*}Issues{*}: - The executor will download 2 duplicate copies of primary resource, one w

[jira] [Updated] (SPARK-47478) Improve test coverage for mysql bool synonyms

2024-03-19 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47478?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-47478: --- Labels: pull-request-available (was: ) > Improve test coverage for mysql bool synonyms > --

[jira] [Updated] (SPARK-47359) StringTranslate (all collations)

2024-03-19 Thread Jira
[ https://issues.apache.org/jira/browse/SPARK-47359?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Uroš Bojanić updated SPARK-47359: - Description: Enable collation support for the *StringTranslate* built-in string function in Spa

[jira] [Updated] (SPARK-47411) StringInstr, FindInSet (all collations)

2024-03-19 Thread Jira
[ https://issues.apache.org/jira/browse/SPARK-47411?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Uroš Bojanić updated SPARK-47411: - Description: Enable collation support for the *StringInstr* and *FindInSet* built-in string fun

[jira] [Updated] (SPARK-47409) StringTrim & StringTrimLeft/Right/Both (all collations)

2024-03-19 Thread Jira
[ https://issues.apache.org/jira/browse/SPARK-47409?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Uroš Bojanić updated SPARK-47409: - Description: Enable collation support for the *StringTrim* built-in string function in Spark (i

[jira] [Updated] (SPARK-47476) StringReplace (all collations)

2024-03-19 Thread Jira
[ https://issues.apache.org/jira/browse/SPARK-47476?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Uroš Bojanić updated SPARK-47476: - Description: Enable collation support for the *StringReplace* built-in string function in Spark

[jira] [Updated] (SPARK-47359) StringTranslate (all collations)

2024-03-19 Thread Jira
[ https://issues.apache.org/jira/browse/SPARK-47359?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Uroš Bojanić updated SPARK-47359: - Description: Enable collation support for the *StringTranslate* built-in string function in Spa

[jira] [Updated] (SPARK-47477) SubstringIndex, StringLocate (all collations)

2024-03-19 Thread Jira
[ https://issues.apache.org/jira/browse/SPARK-47477?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Uroš Bojanić updated SPARK-47477: - Description: Enable collation support for the *StringInstr* and *FindInSet* built-in string fun

[jira] [Created] (SPARK-47478) Improve test coverage for mysql bool synonyms

2024-03-19 Thread Kent Yao (Jira)
Kent Yao created SPARK-47478: Summary: Improve test coverage for mysql bool synonyms Key: SPARK-47478 URL: https://issues.apache.org/jira/browse/SPARK-47478 Project: Spark Issue Type: Sub-task

[jira] [Updated] (SPARK-47357) Upper, Lower, InitCap (binary & lowercase collation only)

2024-03-19 Thread Jira
[ https://issues.apache.org/jira/browse/SPARK-47357?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Uroš Bojanić updated SPARK-47357: - Summary: Upper, Lower, InitCap (binary & lowercase collation only) (was: Upper, Lower, InitCap

[jira] [Updated] (SPARK-47356) ConcatWs & Elt (binary & lowercase collation only)

2024-03-19 Thread Jira
[ https://issues.apache.org/jira/browse/SPARK-47356?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Uroš Bojanić updated SPARK-47356: - Summary: ConcatWs & Elt (binary & lowercase collation only) (was: ConcatWs & Elt (lowercase col

[jira] [Updated] (SPARK-47355) regexp_replace & regexp_instr (binary & lowercase collation only)

2024-03-19 Thread Jira
[ https://issues.apache.org/jira/browse/SPARK-47355?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Uroš Bojanić updated SPARK-47355: - Summary: regexp_replace & regexp_instr (binary & lowercase collation only) (was: regexp_replace

[jira] [Updated] (SPARK-47353) regexp_count & regexp_substr (binary & lowercase collation only)

2024-03-19 Thread Jira
[ https://issues.apache.org/jira/browse/SPARK-47353?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Uroš Bojanić updated SPARK-47353: - Summary: regexp_count & regexp_substr (binary & lowercase collation only) (was: regexp_count &

[jira] [Updated] (SPARK-47354) regexp_extract & regexp_extract_all (binary & lowercase collation only)

2024-03-19 Thread Jira
[ https://issues.apache.org/jira/browse/SPARK-47354?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Uroš Bojanić updated SPARK-47354: - Summary: regexp_extract & regexp_extract_all (binary & lowercase collation only) (was: regexp_e

[jira] [Updated] (SPARK-47351) ilike & rlike (binary & lowercase collation only)

2024-03-19 Thread Jira
[ https://issues.apache.org/jira/browse/SPARK-47351?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Uroš Bojanić updated SPARK-47351: - Summary: ilike & rlike (binary & lowercase collation only) (was: ilike & rlike (lowercase colla

[jira] [Updated] (SPARK-47350) like (binary & lowercase collation only)

2024-03-19 Thread Jira
[ https://issues.apache.org/jira/browse/SPARK-47350?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Uroš Bojanić updated SPARK-47350: - Summary: like (binary & lowercase collation only) (was: like (lowercase collation)) > like (bi

[jira] [Created] (SPARK-47477) SubstringIndex, StringLocate

2024-03-19 Thread Jira
Uroš Bojanić created SPARK-47477: Summary: SubstringIndex, StringLocate Key: SPARK-47477 URL: https://issues.apache.org/jira/browse/SPARK-47477 Project: Spark Issue Type: Sub-task C

[jira] [Updated] (SPARK-47477) SubstringIndex, StringLocate (all collations)

2024-03-19 Thread Jira
[ https://issues.apache.org/jira/browse/SPARK-47477?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Uroš Bojanić updated SPARK-47477: - Summary: SubstringIndex, StringLocate (all collations) (was: SubstringIndex, StringLocate) > S

[jira] [Updated] (SPARK-47477) SubstringIndex, StringLocate

2024-03-19 Thread Jira
[ https://issues.apache.org/jira/browse/SPARK-47477?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Uroš Bojanić updated SPARK-47477: - Description: Enable collation support for the *StringInstr* and *FindInSet* built-in string fun

[jira] [Updated] (SPARK-41625) Feature parity: Streaming support

2024-03-19 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41625?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-41625: --- Labels: pull-request-available (was: ) > Feature parity: Streaming support > --

[jira] [Updated] (SPARK-47476) StringReplace (all collations)

2024-03-19 Thread Jira
[ https://issues.apache.org/jira/browse/SPARK-47476?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Uroš Bojanić updated SPARK-47476: - Description: Enable collation support for the *StringReplace* built-in string function in Spark

[jira] [Created] (SPARK-47476) StringReplace (all collations)

2024-03-19 Thread Jira
Uroš Bojanić created SPARK-47476: Summary: StringReplace (all collations) Key: SPARK-47476 URL: https://issues.apache.org/jira/browse/SPARK-47476 Project: Spark Issue Type: Sub-task

[jira] [Updated] (SPARK-47358) StringRepeat (all collations)

2024-03-19 Thread Jira
[ https://issues.apache.org/jira/browse/SPARK-47358?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Uroš Bojanić updated SPARK-47358: - Description: Enable collation support for the *StringRepeat* built-in string function in Spark.

[jira] [Updated] (SPARK-47358) StringRepeat (all collations)

2024-03-19 Thread Jira
[ https://issues.apache.org/jira/browse/SPARK-47358?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Uroš Bojanić updated SPARK-47358: - Summary: StringRepeat (all collations) (was: StringReplace, StringRepeat (all collations)) > S

[jira] [Updated] (SPARK-47413) Substring, Right, Left (all collations)

2024-03-19 Thread Jira
[ https://issues.apache.org/jira/browse/SPARK-47413?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Uroš Bojanić updated SPARK-47413: - Description: Enable collation support for the *Substring* built-in string function in Spark (in

[jira] [Updated] (SPARK-47475) Jar Download Under K8s Cluster Mode Causes Executors Scaling Issues

2024-03-19 Thread Jiale Tan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47475?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jiale Tan updated SPARK-47475: -- Description: *Issues*: - The executor will download 2 duplicate copies of primary resource, one with

[jira] [Updated] (SPARK-47411) StringInstr, FindInSet (all collations)

2024-03-19 Thread Jira
[ https://issues.apache.org/jira/browse/SPARK-47411?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Uroš Bojanić updated SPARK-47411: - Description: Enable collation support for the *StringInstr* and *FindInSet* built-in string fun

[jira] [Created] (SPARK-47475) Jar Download Under K8s Cluster Mode Causes Executors Scaling Issues

2024-03-19 Thread Jiale Tan (Jira)
Jiale Tan created SPARK-47475: - Summary: Jar Download Under K8s Cluster Mode Causes Executors Scaling Issues Key: SPARK-47475 URL: https://issues.apache.org/jira/browse/SPARK-47475 Project: Spark

[jira] [Updated] (SPARK-47359) StringTranslate (all collations)

2024-03-19 Thread Jira
[ https://issues.apache.org/jira/browse/SPARK-47359?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Uroš Bojanić updated SPARK-47359: - Description: Enable collation support for the *StringTranslate* built-in string function in Spa

[jira] [Updated] (SPARK-47411) StringInstr, FindInSet (all collations)

2024-03-19 Thread Jira
[ https://issues.apache.org/jira/browse/SPARK-47411?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Uroš Bojanić updated SPARK-47411: - Summary: StringInstr, FindInSet (all collations) (was: StringInstr, SubstringIndex, StringLocat

[jira] [Updated] (SPARK-47352) improve error handling and collation exception messages

2024-03-19 Thread Jira
[ https://issues.apache.org/jira/browse/SPARK-47352?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Uroš Bojanić updated SPARK-47352: - Summary: improve error handling and collation exception messages (was: split (lowercase collati

[jira] [Updated] (SPARK-47297) split (binary & lowercase collation only)

2024-03-19 Thread Jira
[ https://issues.apache.org/jira/browse/SPARK-47297?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Uroš Bojanić updated SPARK-47297: - Summary: split (binary & lowercase collation only) (was: StringSplit (binary & lowercase collat

[jira] [Updated] (SPARK-47295) startswith, endswith (all collations)

2024-03-19 Thread Jira
[ https://issues.apache.org/jira/browse/SPARK-47295?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Uroš Bojanić updated SPARK-47295: - Summary: startswith, endswith (all collations) (was: startswith, endswith (non-binary collation

[jira] [Updated] (SPARK-47421) Split, SplitPart (binary & lowercase collation only)

2024-03-19 Thread Jira
[ https://issues.apache.org/jira/browse/SPARK-47421?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Uroš Bojanić updated SPARK-47421: - Summary: Split, SplitPart (binary & lowercase collation only) (was: Split, SplitPart (lowercase

[jira] [Updated] (SPARK-47297) StringSplit (binary & lowercase collation only)

2024-03-19 Thread Jira
[ https://issues.apache.org/jira/browse/SPARK-47297?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Uroš Bojanić updated SPARK-47297: - Summary: StringSplit (binary & lowercase collation only) (was: regular expressions) > StringSp

[jira] [Updated] (SPARK-47248) contains allcollations)

2024-03-19 Thread Jira
[ https://issues.apache.org/jira/browse/SPARK-47248?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Uroš Bojanić updated SPARK-47248: - Summary: contains allcollations) (was: contains (non-binary collations)) > contains allcollatio

[jira] [Updated] (SPARK-47248) contains (all collations)

2024-03-19 Thread Jira
[ https://issues.apache.org/jira/browse/SPARK-47248?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Uroš Bojanić updated SPARK-47248: - Summary: contains (all collations) (was: contains allcollations)) > contains (all collations) >

[jira] [Updated] (SPARK-47131) contains, startswith, endswith (binary & lowercase collation only)

2024-03-19 Thread Jira
[ https://issues.apache.org/jira/browse/SPARK-47131?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Uroš Bojanić updated SPARK-47131: - Summary: contains, startswith, endswith (binary & lowercase collation only) (was: contains, sta

[jira] [Updated] (SPARK-47409) StringTrim & StringTrimLeft/Right/Both (all collations)

2024-03-19 Thread Jira
[ https://issues.apache.org/jira/browse/SPARK-47409?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Uroš Bojanić updated SPARK-47409: - Description: Enable collation support for the *StringTrim* built-in string function in Spark (i

[jira] [Updated] (SPARK-47409) StringTrim & StringTrimLeft/Right/Both (all collations)

2024-03-19 Thread Jira
[ https://issues.apache.org/jira/browse/SPARK-47409?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Uroš Bojanić updated SPARK-47409: - Summary: StringTrim & StringTrimLeft/Right/Both (all collations) (was: StringTrim, StringTrimBo

[jira] [Updated] (SPARK-47359) StringTranslate (all collations)

2024-03-19 Thread Jira
[ https://issues.apache.org/jira/browse/SPARK-47359?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Uroš Bojanić updated SPARK-47359: - Description: Enable collation support for the *StringTranslate* built-in string function in Spa

[jira] [Updated] (SPARK-47359) StringTranslate (all collations)

2024-03-19 Thread Jira
[ https://issues.apache.org/jira/browse/SPARK-47359?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Uroš Bojanić updated SPARK-47359: - Description: Enable collation support for the *StringTranslate* built-in string function in Spa

[jira] [Resolved] (SPARK-47447) Allow reading Parquet TimestampLTZ as TimestampNTZ

2024-03-19 Thread Gengliang Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47447?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang resolved SPARK-47447. Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45571 [https:

[jira] [Updated] (SPARK-47474) Revert change of SPARK-47461 and add some comments

2024-03-19 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47474?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-47474: --- Labels: pull-request-available (was: ) > Revert change of SPARK-47461 and add some comments

[jira] [Created] (SPARK-47474) Revert change of SPARK-47461 and add some comments

2024-03-19 Thread Yang Jie (Jira)
Yang Jie created SPARK-47474: Summary: Revert change of SPARK-47461 and add some comments Key: SPARK-47474 URL: https://issues.apache.org/jira/browse/SPARK-47474 Project: Spark Issue Type: Improv

[jira] [Commented] (SPARK-44101) Support pandas 2

2024-03-19 Thread Haejoon Lee (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44101?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17828607#comment-17828607 ] Haejoon Lee commented on SPARK-44101: - Got it. Thanks for the notice, [~dongjoon] !

[jira] [Updated] (SPARK-47473) Correctness issue of converting postgres INFINITIES timestamps

2024-03-19 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47473?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-47473: --- Labels: correctness pull-request-available (was: correctness) > Correctness issue of conver

[jira] [Updated] (SPARK-47473) Correctness issue of converting postgres INFINITIES timestamps

2024-03-19 Thread Kent Yao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47473?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao updated SPARK-47473: - Labels: correctness (was: ) > Correctness issue of converting postgres INFINITIES timestamps >

[jira] [Created] (SPARK-47473) Correctness issue of converting postgres INFINITIES timestamps

2024-03-19 Thread Kent Yao (Jira)
Kent Yao created SPARK-47473: Summary: Correctness issue of converting postgres INFINITIES timestamps Key: SPARK-47473 URL: https://issues.apache.org/jira/browse/SPARK-47473 Project: Spark Issue

[jira] [Assigned] (SPARK-47007) Add SortMap function

2024-03-19 Thread Max Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47007?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk reassigned SPARK-47007: Assignee: Stevo Mitric > Add SortMap function > > > Key: SPA

[jira] [Resolved] (SPARK-47007) Add SortMap function

2024-03-19 Thread Max Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47007?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk resolved SPARK-47007. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45069 [https://github.com

[jira] [Updated] (SPARK-47471) Support order-insensitive lateral column alias

2024-03-19 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47471?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-47471: --- Labels: pull-request-available (was: ) > Support order-insensitive lateral column alias > -

[jira] [Assigned] (SPARK-47472) Pin `numpy` to 1.23.5 in `dev/infra/Dockerfile`

2024-03-19 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47472?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-47472: - Assignee: Dongjoon Hyun > Pin `numpy` to 1.23.5 in `dev/infra/Dockerfile` > ---

[jira] [Updated] (SPARK-47472) Pin `numpy` to 1.23.5 in `dev/infra/Dockerfile`

2024-03-19 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47472?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-47472: --- Labels: pull-request-available (was: ) > Pin `numpy` to 1.23.5 in `dev/infra/Dockerfile` >

[jira] [Resolved] (SPARK-47472) Pin `numpy` to 1.23.5 in `dev/infra/Dockerfile`

2024-03-19 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47472?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-47472. --- Fix Version/s: 3.4.3 Resolution: Fixed Issue resolved by pull request 45595 [https://

[jira] [Created] (SPARK-47472) Pin `numpy` to 1.23.5 in `dev/infra/Dockerfile`

2024-03-19 Thread Dongjoon Hyun (Jira)
Dongjoon Hyun created SPARK-47472: - Summary: Pin `numpy` to 1.23.5 in `dev/infra/Dockerfile` Key: SPARK-47472 URL: https://issues.apache.org/jira/browse/SPARK-47472 Project: Spark Issue Type:

[jira] [Updated] (SPARK-47471) Support order-insensitive lateral column alias

2024-03-19 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47471?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-47471: Component/s: SQL (was: Block Manager) > Support order-insensitive lateral col

[jira] [Resolved] (SPARK-47470) Ignore `IntentionallyFaultyConnectionProvider` error in `CliSuite`

2024-03-19 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47470?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-47470. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45597 [https://

[jira] [Created] (SPARK-47471) Support order-insensitive lateral column alias

2024-03-19 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-47471: --- Summary: Support order-insensitive lateral column alias Key: SPARK-47471 URL: https://issues.apache.org/jira/browse/SPARK-47471 Project: Spark Issue Type: Impr

[jira] [Assigned] (SPARK-47470) Ignore `IntentionallyFaultyConnectionProvider` error in `CliSuite`

2024-03-19 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47470?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-47470: - Assignee: Dongjoon Hyun > Ignore `IntentionallyFaultyConnectionProvider` error in `CliS

[jira] [Updated] (SPARK-47470) Ignore `IntentionallyFaultyConnectionProvider` error in `CliSuite`

2024-03-19 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47470?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-47470: -- Summary: Ignore `IntentionallyFaultyConnectionProvider` error in `CliSuite` (was: Make `CliSu

[jira] [Updated] (SPARK-47470) Make `CliSuite` ignores `IntentionallyFaultyConnectionProvider` error

2024-03-19 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47470?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-47470: --- Labels: pull-request-available (was: ) > Make `CliSuite` ignores `IntentionallyFaultyConnec

[jira] [Created] (SPARK-47470) Make `CliSuite` ignores `IntentionallyFaultyConnectionProvider` error

2024-03-19 Thread Dongjoon Hyun (Jira)
Dongjoon Hyun created SPARK-47470: - Summary: Make `CliSuite` ignores `IntentionallyFaultyConnectionProvider` error Key: SPARK-47470 URL: https://issues.apache.org/jira/browse/SPARK-47470 Project: Spar

[jira] [Resolved] (SPARK-47330) XML: Add XmlExpressionsSuite

2024-03-19 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47330?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-47330. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45445 [https://gi

[jira] [Assigned] (SPARK-47330) XML: Add XmlExpressionsSuite

2024-03-19 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47330?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-47330: Assignee: Yousof Hosny > XML: Add XmlExpressionsSuite > - >

[jira] [Updated] (SPARK-47469) Add availNow tests for TWS operator

2024-03-19 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47469?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-47469: --- Labels: pull-request-available (was: ) > Add availNow tests for TWS operator >

[jira] [Comment Edited] (SPARK-44101) Support pandas 2

2024-03-19 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44101?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17828556#comment-17828556 ] Dongjoon Hyun edited comment on SPARK-44101 at 3/20/24 1:00 AM: --

[jira] [Assigned] (SPARK-44101) Support pandas 2

2024-03-19 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44101?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-44101: - Assignee: Haejoon Lee > Support pandas 2 > > > Key: SP

[jira] [Updated] (SPARK-44101) Support pandas 2

2024-03-19 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44101?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-44101: -- Labels: releasenotes (was: ) > Support pandas 2 > > > Key: S

[jira] [Commented] (SPARK-44101) Support pandas 2

2024-03-19 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44101?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17828556#comment-17828556 ] Dongjoon Hyun commented on SPARK-44101: --- I added `releasenotes` label to this JIRA

[jira] [Closed] (SPARK-44149) Support DataFrame Merge API

2024-03-19 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44149?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun closed SPARK-44149. - > Support DataFrame Merge API > --- > > Key: SPARK-44149 >

[jira] [Updated] (SPARK-46207) Support MergeInto in DataFrameWriterV2

2024-03-19 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46207?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-46207: -- Parent: SPARK-44111 Issue Type: Sub-task (was: New Feature) > Support MergeInto in Da

[jira] [Resolved] (SPARK-47468) Exclude `logback` dependency from SBT like Maven

2024-03-19 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47468?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-47468. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45594 [https://

[jira] [Updated] (SPARK-47432) Add `pyarrow` upper bound requirement, `<13.0.0`

2024-03-19 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47432?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-47432: -- Fix Version/s: 3.4.3 > Add `pyarrow` upper bound requirement, `<13.0.0` >

[jira] [Resolved] (SPARK-47449) Refactor and split list/timer unit tests

2024-03-19 Thread Jungtaek Lim (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47449?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim resolved SPARK-47449. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45573 [https://gi

[jira] [Updated] (SPARK-46349) Prevent Multiple SortOrders for an Expression

2024-03-19 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46349?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-46349: --- Labels: pull-request-available (was: ) > Prevent Multiple SortOrders for an Expression > --

[jira] [Resolved] (SPARK-47290) Extend CustomTaskMetric to allow metric values from multiple sources

2024-03-19 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47290?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-47290. --- Resolution: Invalid > Extend CustomTaskMetric to allow metric values from multiple sources >

[jira] [Closed] (SPARK-47290) Extend CustomTaskMetric to allow metric values from multiple sources

2024-03-19 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47290?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun closed SPARK-47290. - Sure. I closed this. > Extend CustomTaskMetric to allow metric values from multiple sources > -

[jira] [Resolved] (SPARK-47464) Update labeler.yml

2024-03-19 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47464?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-47464. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45590 [https://

[jira] [Updated] (SPARK-47464) Update labeler.yml for module common/sketch and common/variant

2024-03-19 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47464?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-47464: -- Summary: Update labeler.yml for module common/sketch and common/variant (was: Update labeler.

[jira] [Updated] (SPARK-47468) Exclude `logback` dependency from SBT like Maven

2024-03-19 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47468?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-47468: -- Summary: Exclude `logback` dependency from SBT like Maven (was: Exclude `logback` from SBT de

[jira] [Assigned] (SPARK-47468) Exclude `logback` from SBT dependency like Maven

2024-03-19 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47468?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-47468: - Assignee: Dongjoon Hyun > Exclude `logback` from SBT dependency like Maven > --

[jira] [Updated] (SPARK-47468) Exclude `logback` from SBT dependency like Maven

2024-03-19 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47468?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-47468: --- Labels: pull-request-available (was: ) > Exclude `logback` from SBT dependency like Maven >

[jira] [Created] (SPARK-47468) Exclude `logback` from SBT dependency like Maven

2024-03-19 Thread Dongjoon Hyun (Jira)
Dongjoon Hyun created SPARK-47468: - Summary: Exclude `logback` from SBT dependency like Maven Key: SPARK-47468 URL: https://issues.apache.org/jira/browse/SPARK-47468 Project: Spark Issue Type

[jira] [Updated] (SPARK-47359) StringTranslate (all collations)

2024-03-19 Thread Jira
[ https://issues.apache.org/jira/browse/SPARK-47359?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Uroš Bojanić updated SPARK-47359: - Description: Enable collation support for the *StringTranslate* built-in string function in Spa

[jira] [Updated] (SPARK-47359) StringTranslate (all collations)

2024-03-19 Thread Jira
[ https://issues.apache.org/jira/browse/SPARK-47359?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Uroš Bojanić updated SPARK-47359: - Description: Enable collation support for the *StringTranslate* built-in string function in Spa

[jira] [Created] (SPARK-47467) Error message regressed when creating hive table with illegal column name

2024-03-19 Thread Serge Rielau (Jira)
Serge Rielau created SPARK-47467: Summary: Error message regressed when creating hive table with illegal column name Key: SPARK-47467 URL: https://issues.apache.org/jira/browse/SPARK-47467 Project: Sp

[jira] [Updated] (SPARK-47449) Refactor and split list/timer unit tests

2024-03-19 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47449?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-47449: --- Labels: pull-request-available (was: ) > Refactor and split list/timer unit tests > ---

[jira] [Commented] (SPARK-47290) Extend CustomTaskMetric to allow metric values from multiple sources

2024-03-19 Thread Parth Chandra (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17828481#comment-17828481 ] Parth Chandra commented on SPARK-47290: --- [~dongjoon] can we close this issue? As

[jira] [Resolved] (SPARK-47461) Remove unused private function `totalRunningTasksPerResourceProfile` from `ExecutorAllocationManager`

2024-03-19 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47461?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-47461. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45587 [https://

[jira] [Assigned] (SPARK-47461) Remove unused private function `totalRunningTasksPerResourceProfile` from `ExecutorAllocationManager`

2024-03-19 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47461?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-47461: - Assignee: Yang Jie > Remove unused private function `totalRunningTasksPerResourceProfil

[jira] [Commented] (SPARK-38958) Override S3 Client in Spark Write/Read calls

2024-03-19 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38958?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17828458#comment-17828458 ] ASF GitHub Bot commented on SPARK-38958: steveloughran commented on PR #6550: UR

[jira] [Created] (SPARK-47466) Add PySpark DataFrame method to return iterator of PyArrow RecordBatches

2024-03-19 Thread Ian Cook (Jira)
Ian Cook created SPARK-47466: Summary: Add PySpark DataFrame method to return iterator of PyArrow RecordBatches Key: SPARK-47466 URL: https://issues.apache.org/jira/browse/SPARK-47466 Project: Spark

[jira] [Updated] (SPARK-47365) Add _toArrow() DataFrame method to PySpark

2024-03-19 Thread Ian Cook (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47365?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ian Cook updated SPARK-47365: - Component/s: Connect > Add _toArrow() DataFrame method to PySpark >

  1   2   >