[jira] [Commented] (SPARK-39519) Test failure in SPARK-39387 with JDK 11

[ https://issues.apache.org/jira/browse/SPARK-39519?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17557229#comment-17557229 ] Yang Jie commented on SPARK-39519: -- The default -XX:NewRatio is 2, change it to 3 for sql/core module

[jira] [Resolved] (SPARK-39536) to_date function is returning incorrect value

[ https://issues.apache.org/jira/browse/SPARK-39536?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-39536. -- Resolution: Invalid > to_date function is returning incorrect value >

[jira] [Updated] (SPARK-39549) How to get access to the data created in different Spark Applications

[ https://issues.apache.org/jira/browse/SPARK-39549?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-39549: - Component/s: (was: Project Infra) > How to get access to the data created in different

[jira] [Resolved] (SPARK-39549) How to get access to the data created in different Spark Applications

[ https://issues.apache.org/jira/browse/SPARK-39549?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-39549. -- Resolution: Invalid For questions, let's leverage Spark mailing list. > How to get access to

[jira] [Commented] (SPARK-39549) How to get access to the data created in different Spark Applications

[ https://issues.apache.org/jira/browse/SPARK-39549?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17557202#comment-17557202 ] Hyukjin Kwon commented on SPARK-39549: -- You should either write down into a file or a table, and

[jira] [Commented] (SPARK-39551) Add AQE invalid plan check

[ https://issues.apache.org/jira/browse/SPARK-39551?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17557201#comment-17557201 ] Apache Spark commented on SPARK-39551: -- User 'maryannxue' has created a pull request for this

[jira] [Commented] (SPARK-39551) Add AQE invalid plan check

[ https://issues.apache.org/jira/browse/SPARK-39551?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17557199#comment-17557199 ] Apache Spark commented on SPARK-39551: -- User 'maryannxue' has created a pull request for this

[jira] [Assigned] (SPARK-39551) Add AQE invalid plan check

[ https://issues.apache.org/jira/browse/SPARK-39551?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-39551: Assignee: (was: Apache Spark) > Add AQE invalid plan check >

[jira] [Assigned] (SPARK-39551) Add AQE invalid plan check

[ https://issues.apache.org/jira/browse/SPARK-39551?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-39551: Assignee: Apache Spark > Add AQE invalid plan check > -- > >

[jira] [Created] (SPARK-39551) Add AQE invalid plan check

Wei Xue created SPARK-39551: --- Summary: Add AQE invalid plan check Key: SPARK-39551 URL: https://issues.apache.org/jira/browse/SPARK-39551 Project: Spark Issue Type: Improvement

[jira] [Updated] (SPARK-39545) Override `concat` method for `ExpressionSet` in Scala 2.13 to improve the performance

[ https://issues.apache.org/jira/browse/SPARK-39545?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie updated SPARK-39545: - Description: {{ExpressionSet ++}} method in the master branch a little slower than the branch-3.3 with

[jira] [Assigned] (SPARK-39541) [Yarn] Diagnostics of yarn UI did not display the exception of driver when driver exit before regiserAM

[ https://issues.apache.org/jira/browse/SPARK-39541?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-39541: Assignee: (was: Apache Spark) > [Yarn] Diagnostics of yarn UI did not display the

[jira] [Commented] (SPARK-39541) [Yarn] Diagnostics of yarn UI did not display the exception of driver when driver exit before regiserAM

[ https://issues.apache.org/jira/browse/SPARK-39541?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17557190#comment-17557190 ] Apache Spark commented on SPARK-39541: -- User 'lyy-pineapple' has created a pull request for this

[jira] [Assigned] (SPARK-39541) [Yarn] Diagnostics of yarn UI did not display the exception of driver when driver exit before regiserAM

[ https://issues.apache.org/jira/browse/SPARK-39541?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-39541: Assignee: Apache Spark > [Yarn] Diagnostics of yarn UI did not display the exception of

[jira] [Commented] (SPARK-38614) After Spark update, df.show() shows incorrect F.percent_rank results

[ https://issues.apache.org/jira/browse/SPARK-38614?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17557179#comment-17557179 ] Yuming Wang commented on SPARK-38614: - Thank you for reporting this issue. Workaround: {code:sql}

[jira] [Updated] (SPARK-39533) Deprecate scoreLabelsWeight in BinaryClassificationMetrics

[ https://issues.apache.org/jira/browse/SPARK-39533?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhengruifeng updated SPARK-39533: - Summary: Deprecate scoreLabelsWeight in BinaryClassificationMetrics (was: Remove

[jira] [Updated] (SPARK-39533) Deprecate scoreLabelsWeight in BinaryClassificationMetrics

[ https://issues.apache.org/jira/browse/SPARK-39533?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhengruifeng updated SPARK-39533: - Description: scoreLabelsWeight in BinaryClassificationMetrics is a public variable, but it

[jira] [Resolved] (SPARK-39540) Upgrade mysql-connector-java to 8.0.28

[ https://issues.apache.org/jira/browse/SPARK-39540?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-39540. --- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 36938

[jira] [Assigned] (SPARK-39540) Upgrade mysql-connector-java to 8.0.28

[ https://issues.apache.org/jira/browse/SPARK-39540?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-39540: - Assignee: Bjørn Jørgensen > Upgrade mysql-connector-java to 8.0.28 >

[jira] [Updated] (SPARK-39540) Upgrade mysql-connector-java to 8.0.29

[ https://issues.apache.org/jira/browse/SPARK-39540?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-39540: -- Summary: Upgrade mysql-connector-java to 8.0.29 (was: Upgrade mysql-connector-java to

[jira] [Assigned] (SPARK-38614) After Spark update, df.show() shows incorrect F.percent_rank results

[ https://issues.apache.org/jira/browse/SPARK-38614?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38614: Assignee: Apache Spark > After Spark update, df.show() shows incorrect F.percent_rank

[jira] [Commented] (SPARK-38614) After Spark update, df.show() shows incorrect F.percent_rank results

[ https://issues.apache.org/jira/browse/SPARK-38614?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17557160#comment-17557160 ] Apache Spark commented on SPARK-38614: -- User 'bersprockets' has created a pull request for this

[jira] [Commented] (SPARK-38614) After Spark update, df.show() shows incorrect F.percent_rank results

[ https://issues.apache.org/jira/browse/SPARK-38614?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17557159#comment-17557159 ] Apache Spark commented on SPARK-38614: -- User 'bersprockets' has created a pull request for this

[jira] [Assigned] (SPARK-38614) After Spark update, df.show() shows incorrect F.percent_rank results

[ https://issues.apache.org/jira/browse/SPARK-38614?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38614: Assignee: (was: Apache Spark) > After Spark update, df.show() shows incorrect

[jira] [Updated] (SPARK-39494) Support `createDataFrame` from a list of scalars

[ https://issues.apache.org/jira/browse/SPARK-39494?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-39494: - Description: Currently, DataFrame creation from a list of scalars is unsupported as below: |>>>

[jira] [Updated] (SPARK-39496) Inline eval path cannot handle null structs

[ https://issues.apache.org/jira/browse/SPARK-39496?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-39496: - Fix Version/s: 3.1.3 > Inline eval path cannot handle null structs >

[jira] [Commented] (SPARK-39519) Test failure in SPARK-39387 with JDK 11

[ https://issues.apache.org/jira/browse/SPARK-39519?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17557155#comment-17557155 ] Hyukjin Kwon commented on SPARK-39519: -- Thanks for your investigation. > Test failure in

[jira] [Updated] (SPARK-39494) Support `createDataFrame` from a list of scalars

[ https://issues.apache.org/jira/browse/SPARK-39494?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-39494: - Description: Currently, DataFrame creation from a list of scalars is unsupported as below: |>>>

[jira] [Updated] (SPARK-39550) Fix `MultiIndex.value_counts()` when Arrow Execution is enabled

[ https://issues.apache.org/jira/browse/SPARK-39550?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-39550: - Description:   When Arrow Execution is enabled, {code:java} >>>

[jira] [Commented] (SPARK-39550) Fix `MultiIndex.value_counts()` when Arrow Execution is enabled

[ https://issues.apache.org/jira/browse/SPARK-39550?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17557145#comment-17557145 ] Xinrong Meng commented on SPARK-39550: -- I am working on that. > Fix `MultiIndex.value_counts()`

[jira] [Created] (SPARK-39550) Fix `MultiIndex.value_counts()` when Arrow Execution is enabled

Xinrong Meng created SPARK-39550: Summary: Fix `MultiIndex.value_counts()` when Arrow Execution is enabled Key: SPARK-39550 URL: https://issues.apache.org/jira/browse/SPARK-39550 Project: Spark

[jira] [Updated] (SPARK-39549) How to get access to the data created in different Spark Applications

[ https://issues.apache.org/jira/browse/SPARK-39549?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chenyang Zhang updated SPARK-39549: --- Priority: Major (was: Critical) > How to get access to the data created in different Spark

[jira] [Created] (SPARK-39549) How to get access to the data created in different Spark Applications

Chenyang Zhang created SPARK-39549: -- Summary: How to get access to the data created in different Spark Applications Key: SPARK-39549 URL: https://issues.apache.org/jira/browse/SPARK-39549 Project:

[jira] [Commented] (SPARK-38796) Implement the to_number and try_to_number SQL functions according to a new specification

[ https://issues.apache.org/jira/browse/SPARK-38796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17557130#comment-17557130 ] Apache Spark commented on SPARK-38796: -- User 'dtenedor' has created a pull request for this issue:

[jira] [Commented] (SPARK-39496) Inline eval path cannot handle null structs

[ https://issues.apache.org/jira/browse/SPARK-39496?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17557128#comment-17557128 ] Apache Spark commented on SPARK-39496: -- User 'bersprockets' has created a pull request for this

[jira] [Commented] (SPARK-39496) Inline eval path cannot handle null structs

[ https://issues.apache.org/jira/browse/SPARK-39496?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17557125#comment-17557125 ] Apache Spark commented on SPARK-39496: -- User 'bersprockets' has created a pull request for this

[jira] [Commented] (SPARK-39548) CreateView Command with a window clause query hit a wrong window definition not found issue

[ https://issues.apache.org/jira/browse/SPARK-39548?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17557119#comment-17557119 ] Apache Spark commented on SPARK-39548: -- User 'amaliujia' has created a pull request for this issue:

[jira] [Assigned] (SPARK-39548) CreateView Command with a window clause query hit a wrong window definition not found issue

[ https://issues.apache.org/jira/browse/SPARK-39548?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-39548: Assignee: Apache Spark > CreateView Command with a window clause query hit a wrong

[jira] [Assigned] (SPARK-39548) CreateView Command with a window clause query hit a wrong window definition not found issue

[ https://issues.apache.org/jira/browse/SPARK-39548?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-39548: Assignee: (was: Apache Spark) > CreateView Command with a window clause query hit a

[jira] [Updated] (SPARK-39548) CreateView Command with a window clause query hit a wrong window definition not found issue

[ https://issues.apache.org/jira/browse/SPARK-39548?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rui Wang updated SPARK-39548: - Summary: CreateView Command with a window clause query hit a wrong window definition not found issue

[jira] [Created] (SPARK-39548) CreateView Command with a window clause query hit a wrong window definition not found issue.

Rui Wang created SPARK-39548: Summary: CreateView Command with a window clause query hit a wrong window definition not found issue. Key: SPARK-39548 URL: https://issues.apache.org/jira/browse/SPARK-39548

[jira] [Commented] (SPARK-39500) Ivy doesn't work correctly on IPv6-only environment

[ https://issues.apache.org/jira/browse/SPARK-39500?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17557101#comment-17557101 ] Erik Krogen commented on SPARK-39500: - Thanks for clarifying! > Ivy doesn't work correctly on

[jira] [Commented] (SPARK-39547) V2SessionCatalog should not throw NoSuchDatabaseException in loadNamespaceMetadata

[ https://issues.apache.org/jira/browse/SPARK-39547?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17557062#comment-17557062 ] Apache Spark commented on SPARK-39547: -- User 'singhpk234' has created a pull request for this

[jira] [Assigned] (SPARK-39547) V2SessionCatalog should not throw NoSuchDatabaseException in loadNamespaceMetadata

[ https://issues.apache.org/jira/browse/SPARK-39547?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-39547: Assignee: Apache Spark > V2SessionCatalog should not throw NoSuchDatabaseException in >

[jira] [Assigned] (SPARK-39547) V2SessionCatalog should not throw NoSuchDatabaseException in loadNamespaceMetadata

[ https://issues.apache.org/jira/browse/SPARK-39547?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-39547: Assignee: (was: Apache Spark) > V2SessionCatalog should not throw

[jira] [Commented] (SPARK-39547) V2SessionCatalog should not throw NoSuchDatabaseException in loadNamespaceMetadata

[ https://issues.apache.org/jira/browse/SPARK-39547?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17557063#comment-17557063 ] Apache Spark commented on SPARK-39547: -- User 'singhpk234' has created a pull request for this

[jira] [Created] (SPARK-39547) V2SessionCatalog should not throw NoSuchDatabaseException in loadNamespaceMetadata

Prashant Singh created SPARK-39547: -- Summary: V2SessionCatalog should not throw NoSuchDatabaseException in loadNamespaceMetadata Key: SPARK-39547 URL: https://issues.apache.org/jira/browse/SPARK-39547

[jira] [Commented] (SPARK-39547) V2SessionCatalog should not throw NoSuchDatabaseException in loadNamespaceMetadata

[ https://issues.apache.org/jira/browse/SPARK-39547?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17557056#comment-17557056 ] Prashant Singh commented on SPARK-39547: will post a pr for it shortly. > V2SessionCatalog

[jira] [Resolved] (SPARK-38647) Add SupportsReportOrdering mix in interface for Scan

[ https://issues.apache.org/jira/browse/SPARK-38647?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun resolved SPARK-38647. -- Fix Version/s: 3.4.0 Assignee: Enrico Minack Resolution: Fixed > Add

[jira] [Resolved] (SPARK-39542) Improve YARN client mode to support IPv6

[ https://issues.apache.org/jira/browse/SPARK-39542?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-39542. --- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 36939

[jira] [Assigned] (SPARK-39542) Improve YARN client mode to support IPv6

[ https://issues.apache.org/jira/browse/SPARK-39542?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-39542: - Assignee: Dongjoon Hyun > Improve YARN client mode to support IPv6 >

[jira] [Created] (SPARK-39546) Respect port defininitions on K8S pod templates for both driver and executor

Oliver Koeth created SPARK-39546: Summary: Respect port defininitions on K8S pod templates for both driver and executor Key: SPARK-39546 URL: https://issues.apache.org/jira/browse/SPARK-39546

[jira] [Commented] (SPARK-39340) DS v2 agg pushdown should allow dots in the name of top-level columns

[ https://issues.apache.org/jira/browse/SPARK-39340?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17556982#comment-17556982 ] Apache Spark commented on SPARK-39340: -- User 'cloud-fan' has created a pull request for this issue:

[jira] [Updated] (SPARK-37939) Use error classes in the parsing errors of properties

[ https://issues.apache.org/jira/browse/SPARK-37939?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk updated SPARK-37939: - Fix Version/s: 3.3.1 > Use error classes in the parsing errors of properties >

[jira] [Commented] (SPARK-39195) Spark OutputCommitCoordinator should abort stage when committed file not consistent with task status

[ https://issues.apache.org/jira/browse/SPARK-39195?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17556916#comment-17556916 ] Apache Spark commented on SPARK-39195: -- User 'AngersZh' has created a pull request for this

[jira] [Commented] (SPARK-39195) Spark OutputCommitCoordinator should abort stage when committed file not consistent with task status

[ https://issues.apache.org/jira/browse/SPARK-39195?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17556915#comment-17556915 ] Apache Spark commented on SPARK-39195: -- User 'AngersZh' has created a pull request for this

[jira] [Commented] (SPARK-39519) Test failure in SPARK-39387 with JDK 11

[ https://issues.apache.org/jira/browse/SPARK-39519?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17556911#comment-17556911 ] Yang Jie commented on SPARK-39519: -- I will continue to investigate this issue   > Test failure in

[jira] [Commented] (SPARK-39519) Test failure in SPARK-39387 with JDK 11

[ https://issues.apache.org/jira/browse/SPARK-39519?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17556910#comment-17556910 ] Yang Jie commented on SPARK-39519: -- [~hyukjin.kwon] Sorry, I think we should reopen this issue, from

[jira] [Updated] (SPARK-39519) Test failure in SPARK-39387 with JDK 11

[ https://issues.apache.org/jira/browse/SPARK-39519?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie updated SPARK-39519: - Attachment: image-2022-06-21-21-26-26-563.png > Test failure in SPARK-39387 with JDK 11 >

[jira] [Updated] (SPARK-39519) Test failure in SPARK-39387 with JDK 11

[ https://issues.apache.org/jira/browse/SPARK-39519?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie updated SPARK-39519: - Attachment: image-2022-06-21-21-26-38-146.png > Test failure in SPARK-39387 with JDK 11 >

[jira] [Updated] (SPARK-39519) Test failure in SPARK-39387 with JDK 11

[ https://issues.apache.org/jira/browse/SPARK-39519?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie updated SPARK-39519: - Attachment: image-2022-06-21-21-26-06-586.png > Test failure in SPARK-39387 with JDK 11 >

[jira] [Updated] (SPARK-39519) Test failure in SPARK-39387 with JDK 11

[ https://issues.apache.org/jira/browse/SPARK-39519?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie updated SPARK-39519: - Attachment: image-2022-06-21-21-25-35-951.png > Test failure in SPARK-39387 with JDK 11 >

[jira] [Commented] (SPARK-39519) Test failure in SPARK-39387 with JDK 11

[ https://issues.apache.org/jira/browse/SPARK-39519?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17556896#comment-17556896 ] Yang Jie commented on SPARK-39519: -- I get a oom dump and will analyze it later   > Test failure in

[jira] [Commented] (SPARK-39545) Override `concat` method for `ExpressionSet` in Scala 2.13 to improve the performance

[ https://issues.apache.org/jira/browse/SPARK-39545?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17556894#comment-17556894 ] Apache Spark commented on SPARK-39545: -- User 'LuciferYang' has created a pull request for this

[jira] [Assigned] (SPARK-39545) Override `concat` method for `ExpressionSet` in Scala 2.13 to improve the performance

[ https://issues.apache.org/jira/browse/SPARK-39545?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-39545: Assignee: Apache Spark > Override `concat` method for `ExpressionSet` in Scala 2.13 to

[jira] [Commented] (SPARK-39545) Override `concat` method for `ExpressionSet` in Scala 2.13 to improve the performance

[ https://issues.apache.org/jira/browse/SPARK-39545?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17556893#comment-17556893 ] Apache Spark commented on SPARK-39545: -- User 'LuciferYang' has created a pull request for this

[jira] [Assigned] (SPARK-39545) Override `concat` method for `ExpressionSet` in Scala 2.13 to improve the performance

[ https://issues.apache.org/jira/browse/SPARK-39545?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-39545: Assignee: (was: Apache Spark) > Override `concat` method for `ExpressionSet` in

[jira] [Updated] (SPARK-39545) Override `concat` method for `ExpressionSet` in Scala 2.13 to improve the performance

[ https://issues.apache.org/jira/browse/SPARK-39545?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie updated SPARK-39545: - Description: ExpressionSet ++ with  > Override `concat` method for `ExpressionSet` in Scala 2.13 to

[jira] [Updated] (SPARK-39544) setPredictionCol for OneVsRest does not persist when saving model to disk

[ https://issues.apache.org/jira/browse/SPARK-39544?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] koba updated SPARK-39544: - Issue Type: Bug (was: Improvement) > setPredictionCol for OneVsRest does not persist when saving model to disk

[jira] [Created] (SPARK-39545) Override `concat` method for `ExpressionSet` in Scala 2.13 to improve the performance

Yang Jie created SPARK-39545: Summary: Override `concat` method for `ExpressionSet` in Scala 2.13 to improve the performance Key: SPARK-39545 URL: https://issues.apache.org/jira/browse/SPARK-39545

[jira] [Updated] (SPARK-39544) setPredictionCol for OneVsRest does not persist when saving model to disk

[ https://issues.apache.org/jira/browse/SPARK-39544?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] koba updated SPARK-39544: - Description: The naming of rawPredcitionCol in OneVsRest does not persist after saving and loading a trained

[jira] [Updated] (SPARK-39544) setPredictionCol for OneVsRest does not persist when saving model to disk

[ https://issues.apache.org/jira/browse/SPARK-39544?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] koba updated SPARK-39544: - Description: The naming of rawPredcitionCol in OneVsRest does not persist after saving and loading a trained

[jira] [Updated] (SPARK-39544) setPredictionCol for OneVsRest does not persist when saving model to disk

[ https://issues.apache.org/jira/browse/SPARK-39544?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] koba updated SPARK-39544: - Description: The naming of rawPredcitionCol in OneVsRest does not persist after saving and loading a trained

[jira] [Updated] (SPARK-39544) setPredictionCol for OneVsRest does not persist when saving model to disk

[ https://issues.apache.org/jira/browse/SPARK-39544?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] koba updated SPARK-39544: - Description: The naming of rawPredcitionCol in OneVsRest does not persist after saving and loading a trained

[jira] [Updated] (SPARK-39544) setPredictionCol for OneVsRest does not persist when saving model to disk

[ https://issues.apache.org/jira/browse/SPARK-39544?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] koba updated SPARK-39544: - Description: The naming of rawPredcitionCol in OneVsRest does not persist after saving and loading a trained

[jira] [Updated] (SPARK-39544) setPredictionCol for OneVsRest does not persist when saving model to disk

[ https://issues.apache.org/jira/browse/SPARK-39544?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] koba updated SPARK-39544: - Description: The naming of rawPredcitionCol in OneVsRest does not persist after saving and loading a trained

[jira] [Created] (SPARK-39544) setPredictionCol for OneVsRest does not persist when saving model to disk

koba created SPARK-39544: Summary: setPredictionCol for OneVsRest does not persist when saving model to disk Key: SPARK-39544 URL: https://issues.apache.org/jira/browse/SPARK-39544 Project: Spark

[jira] [Commented] (SPARK-38292) Support `na_filter` for pyspark.pandas.read_csv

[ https://issues.apache.org/jira/browse/SPARK-38292?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17556859#comment-17556859 ] Hyukjin Kwon commented on SPARK-38292: -- you could try to leverage the approach like

[jira] [Commented] (SPARK-38292) Support `na_filter` for pyspark.pandas.read_csv

[ https://issues.apache.org/jira/browse/SPARK-38292?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17556858#comment-17556858 ] Hyukjin Kwon commented on SPARK-38292: -- can we control the options e.g., emptyValue or nullValue in

[jira] [Assigned] (SPARK-39543) The option of DataFrameWriterV2 should be passed to storage properties if fallback to v1

[ https://issues.apache.org/jira/browse/SPARK-39543?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-39543: Assignee: Apache Spark > The option of DataFrameWriterV2 should be passed to storage

[jira] [Commented] (SPARK-39543) The option of DataFrameWriterV2 should be passed to storage properties if fallback to v1

[ https://issues.apache.org/jira/browse/SPARK-39543?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17556857#comment-17556857 ] Apache Spark commented on SPARK-39543: -- User 'Yikf' has created a pull request for this issue:

[jira] [Assigned] (SPARK-39543) The option of DataFrameWriterV2 should be passed to storage properties if fallback to v1

[ https://issues.apache.org/jira/browse/SPARK-39543?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-39543: Assignee: (was: Apache Spark) > The option of DataFrameWriterV2 should be passed to

[jira] [Updated] (SPARK-39543) The option of DataFrameWriterV2 should be passed to storage properties if fallback to v1

[ https://issues.apache.org/jira/browse/SPARK-39543?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] yikf updated SPARK-39543: - Description: The option of DataFrameWriterV2 should be passed to storage properties if fallback to v1, to

[jira] [Commented] (SPARK-38292) Support `na_filter` for pyspark.pandas.read_csv

[ https://issues.apache.org/jira/browse/SPARK-38292?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17556851#comment-17556851 ] pralabhkumar commented on SPARK-38292: -- [~itholic] [~hyukjin.kwon]  Would like to discuss the

[jira] [Updated] (SPARK-39543) The option of DataFrameWriterV2 should be passed to storage properties if fallback to v1

[ https://issues.apache.org/jira/browse/SPARK-39543?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] yikf updated SPARK-39543: - Description: The option of DataFrameWriterV2 should be passed to storage properties if fallback to v1, to

[jira] [Assigned] (SPARK-39515) Improve/recover scheduled jobs in GitHub Actions

[ https://issues.apache.org/jira/browse/SPARK-39515?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-39515: Assignee: Hyukjin Kwon > Improve/recover scheduled jobs in GitHub Actions >

[jira] [Updated] (SPARK-39074) Fail on uploading test files, not when downloading them

[ https://issues.apache.org/jira/browse/SPARK-39074?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-39074: - Parent: (was: SPARK-39515) Issue Type: Bug (was: Sub-task) > Fail on uploading

[jira] [Commented] (SPARK-39529) Refactor and merge all related job selection logic into precondition

[ https://issues.apache.org/jira/browse/SPARK-39529?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17556837#comment-17556837 ] Apache Spark commented on SPARK-39529: -- User 'HyukjinKwon' has created a pull request for this

[jira] [Commented] (SPARK-39529) Refactor and merge all related job selection logic into precondition

[ https://issues.apache.org/jira/browse/SPARK-39529?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17556838#comment-17556838 ] Apache Spark commented on SPARK-39529: -- User 'HyukjinKwon' has created a pull request for this

[jira] [Updated] (SPARK-39543) The option of DataFrameWriterV2 should be passed to storage properties if fallback to v1

[ https://issues.apache.org/jira/browse/SPARK-39543?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] yikf updated SPARK-39543: - Summary: The option of DataFrameWriterV2 should be passed to storage properties if fallback to v1 (was: The

[jira] [Updated] (SPARK-39543) The option of DataFrameWriterV2 should be passed to storage Properties if fallback to v1

[ https://issues.apache.org/jira/browse/SPARK-39543?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] yikf updated SPARK-39543: - Summary: The option of DataFrameWriterV2 should be passed to storage Properties if fallback to v1 (was: The

[jira] [Updated] (SPARK-39543) The option of DataFrameWriterV2 should be passed to storage Properties if fallback to v1

[ https://issues.apache.org/jira/browse/SPARK-39543?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] yikf updated SPARK-39543: - Description: The option of DataFrameWriterV2 should be passed to storage properties if fallback to v1, to

[jira] [Created] (SPARK-39543) The option of DataFrameWriterV2 should be passed to storage Properties

yikf created SPARK-39543: Summary: The option of DataFrameWriterV2 should be passed to storage Properties Key: SPARK-39543 URL: https://issues.apache.org/jira/browse/SPARK-39543 Project: Spark

[jira] [Commented] (SPARK-39542) Improve YARN client mode to support IPv6

[ https://issues.apache.org/jira/browse/SPARK-39542?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17556818#comment-17556818 ] Apache Spark commented on SPARK-39542: -- User 'dongjoon-hyun' has created a pull request for this

[jira] [Assigned] (SPARK-39542) Improve YARN client mode to support IPv6

[ https://issues.apache.org/jira/browse/SPARK-39542?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-39542: Assignee: (was: Apache Spark) > Improve YARN client mode to support IPv6 >

[jira] [Assigned] (SPARK-39542) Improve YARN client mode to support IPv6

[ https://issues.apache.org/jira/browse/SPARK-39542?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-39542: Assignee: Apache Spark > Improve YARN client mode to support IPv6 >

[jira] [Updated] (SPARK-39542) Improve YARN client mode to support IPv6

[ https://issues.apache.org/jira/browse/SPARK-39542?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-39542: -- Component/s: PySpark > Improve YARN client mode to support IPv6 >

[jira] [Created] (SPARK-39542) Improve YARN client mode to support IPv6

Dongjoon Hyun created SPARK-39542: - Summary: Improve YARN client mode to support IPv6 Key: SPARK-39542 URL: https://issues.apache.org/jira/browse/SPARK-39542 Project: Spark Issue Type:

[jira] [Commented] (SPARK-39541) [Yarn] Diagnostics of yarn UI did not display the exception of driver when driver exit before regiserAM

[ https://issues.apache.org/jira/browse/SPARK-39541?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17556806#comment-17556806 ] liangyongyuan commented on SPARK-39541: --- I want to try to solve this problem. I already have a

[jira] [Created] (SPARK-39541) [Yarn] Diagnostics of yarn UI did not display the exception of driver when driver exit before regiserAM

liangyongyuan created SPARK-39541: - Summary: [Yarn] Diagnostics of yarn UI did not display the exception of driver when driver exit before regiserAM Key: SPARK-39541 URL:

  1   2   >