[jira] [Commented] (SPARK-32923) Add support to properly handle different type of stage retries

[ https://issues.apache.org/jira/browse/SPARK-32923?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17391333#comment-17391333 ] Apache Spark commented on SPARK-32923: -- User 'venkata91' has created a pull request for this issue:

[jira] [Comment Edited] (SPARK-30602) SPIP: Support push-based shuffle to improve shuffle efficiency

[ https://issues.apache.org/jira/browse/SPARK-30602?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17391318#comment-17391318 ] Mridul Muralidharan edited comment on SPARK-30602 at 8/2/21, 5:30 AM:

[jira] [Reopened] (SPARK-36378) Minor changes to address a few identified server side inefficiencies

[ https://issues.apache.org/jira/browse/SPARK-36378?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Min Shen reopened SPARK-36378: -- > Minor changes to address a few identified server side inefficiencies >

[jira] [Updated] (SPARK-36378) Minor changes to address a few identified server side inefficiencies

[ https://issues.apache.org/jira/browse/SPARK-36378?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Min Shen updated SPARK-36378: - Parent: SPARK-33235 Issue Type: Sub-task (was: Bug) > Minor changes to address a few

[jira] [Commented] (SPARK-30602) SPIP: Support push-based shuffle to improve shuffle efficiency

[ https://issues.apache.org/jira/browse/SPARK-30602?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17391324#comment-17391324 ] Mridul Muralidharan commented on SPARK-30602: - Thanks for all the work in getting this

[jira] [Updated] (SPARK-36378) Minor changes to address a few identified server side inefficiencies

[ https://issues.apache.org/jira/browse/SPARK-36378?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Min Shen updated SPARK-36378: - Parent: (was: SPARK-30602) Issue Type: Bug (was: Sub-task) > Minor changes to address a

[jira] [Commented] (SPARK-36378) Minor changes to address a few identified server side inefficiencies

[ https://issues.apache.org/jira/browse/SPARK-36378?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17391322#comment-17391322 ] Min Shen commented on SPARK-36378: -- If moving this outside of the SPIP is preferred, then will move

[jira] [Assigned] (SPARK-36266) Rename classes in shuffle RPC used for block push operations

[ https://issues.apache.org/jira/browse/SPARK-36266?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan reassigned SPARK-36266: --- Assignee: Min Shen > Rename classes in shuffle RPC used for block push

[jira] [Commented] (SPARK-36378) Minor changes to address a few identified server side inefficiencies

[ https://issues.apache.org/jira/browse/SPARK-36378?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17391320#comment-17391320 ] Min Shen commented on SPARK-36378: -- Would prefer to merge this in if possible. > Minor changes to

[jira] [Resolved] (SPARK-30602) SPIP: Support push-based shuffle to improve shuffle efficiency

[ https://issues.apache.org/jira/browse/SPARK-30602?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan resolved SPARK-30602. - Resolution: Fixed The only pending task here is documentation. [~vsowrirajan]

[jira] [Assigned] (SPARK-30602) SPIP: Support push-based shuffle to improve shuffle efficiency

[ https://issues.apache.org/jira/browse/SPARK-30602?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan reassigned SPARK-30602: --- Assignee: Mridul Muralidharan > SPIP: Support push-based shuffle to

[jira] [Updated] (SPARK-32923) Add support to properly handle different type of stage retries

[ https://issues.apache.org/jira/browse/SPARK-32923?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan updated SPARK-32923: Fix Version/s: 3.2.0 > Add support to properly handle different type of stage

[jira] [Assigned] (SPARK-32923) Add support to properly handle different type of stage retries

[ https://issues.apache.org/jira/browse/SPARK-32923?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan reassigned SPARK-32923: --- Assignee: Venkata krishnan Sowrirajan > Add support to properly handle

[jira] [Issue Comment Deleted] (SPARK-32923) Add support to properly handle different type of stage retries

[ https://issues.apache.org/jira/browse/SPARK-32923?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan updated SPARK-32923: Comment: was deleted (was: This has been handled by SPARK-32923) > Add support

[jira] [Resolved] (SPARK-32923) Add support to properly handle different type of stage retries

[ https://issues.apache.org/jira/browse/SPARK-32923?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan resolved SPARK-32923. - Resolution: Fixed This has been handled by SPARK-32923 > Add support to

[jira] [Assigned] (SPARK-36378) Minor changes to address a few identified server side inefficiencies

[ https://issues.apache.org/jira/browse/SPARK-36378?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan reassigned SPARK-36378: --- Assignee: Mridul Muralidharan > Minor changes to address a few identified

[jira] [Resolved] (SPARK-36378) Minor changes to address a few identified server side inefficiencies

[ https://issues.apache.org/jira/browse/SPARK-36378?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan resolved SPARK-36378. - Resolution: Won't Fix Let us move this outside of the SPIP and into individual

[jira] [Assigned] (SPARK-35917) Disable push-based shuffle until the feature is complete

[ https://issues.apache.org/jira/browse/SPARK-35917?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan reassigned SPARK-35917: --- Assignee: Mridul Muralidharan > Disable push-based shuffle until the

[jira] [Resolved] (SPARK-35917) Disable push-based shuffle until the feature is complete

[ https://issues.apache.org/jira/browse/SPARK-35917?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan resolved SPARK-35917. - Resolution: Won't Fix > Disable push-based shuffle until the feature is

[jira] [Commented] (SPARK-35917) Disable push-based shuffle until the feature is complete

[ https://issues.apache.org/jira/browse/SPARK-35917?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17391313#comment-17391313 ] Mridul Muralidharan commented on SPARK-35917: - Closing this Jira - as push based shuffle has

[jira] [Updated] (SPARK-36378) Minor changes to address a few identified server side inefficiencies

[ https://issues.apache.org/jira/browse/SPARK-36378?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Min Shen updated SPARK-36378: - Description: With the SPIP ticket close to being finished, we have done some performance evaluations

[jira] [Commented] (SPARK-36377) Fix documentation in spark-env.sh.template

[ https://issues.apache.org/jira/browse/SPARK-36377?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17391298#comment-17391298 ] Apache Spark commented on SPARK-36377: -- User 'yutoacts' has created a pull request for this issue:

[jira] [Commented] (SPARK-35917) Disable push-based shuffle until the feature is complete

[ https://issues.apache.org/jira/browse/SPARK-35917?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17391299#comment-17391299 ] Gengliang Wang commented on SPARK-35917: [~csingh] Shall we mark this one as won't do since

[jira] [Assigned] (SPARK-36377) Fix documentation in spark-env.sh.template

[ https://issues.apache.org/jira/browse/SPARK-36377?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36377: Assignee: Apache Spark > Fix documentation in spark-env.sh.template >

[jira] [Assigned] (SPARK-36377) Fix documentation in spark-env.sh.template

[ https://issues.apache.org/jira/browse/SPARK-36377?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36377: Assignee: (was: Apache Spark) > Fix documentation in spark-env.sh.template >

[jira] [Commented] (SPARK-36377) Fix documentation in spark-env.sh.template

[ https://issues.apache.org/jira/browse/SPARK-36377?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17391297#comment-17391297 ] Apache Spark commented on SPARK-36377: -- User 'yutoacts' has created a pull request for this issue:

[jira] [Created] (SPARK-36378) Minor changes to address a few identified server side inefficiencies

Min Shen created SPARK-36378: Summary: Minor changes to address a few identified server side inefficiencies Key: SPARK-36378 URL: https://issues.apache.org/jira/browse/SPARK-36378 Project: Spark

[jira] [Commented] (SPARK-36303) Refactor fourteenth set of 20 query execution errors to use error classes

[ https://issues.apache.org/jira/browse/SPARK-36303?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17391294#comment-17391294 ] dgd_contributor commented on SPARK-36303: - working on this > Refactor fourteenth set of 20

[jira] [Commented] (SPARK-36302) Refactor thirteenth set of 20 query execution errors to use error classes

[ https://issues.apache.org/jira/browse/SPARK-36302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17391283#comment-17391283 ] dgd_contributor commented on SPARK-36302: - working on this. > Refactor thirteenth set of 20

[jira] [Commented] (SPARK-36301) Refactor twelfth set of 20 query execution errors to use error classes

[ https://issues.apache.org/jira/browse/SPARK-36301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17391279#comment-17391279 ] dgd_contributor commented on SPARK-36301: - working on this > Refactor twelfth set of 20 query

[jira] [Commented] (SPARK-36298) Refactor ninth set of 20 query execution errors to use error classes

[ https://issues.apache.org/jira/browse/SPARK-36298?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17391277#comment-17391277 ] dgd_contributor commented on SPARK-36298: - working on this. > Refactor ninth set of 20 query

[jira] [Commented] (SPARK-36299) Refactor tenth set of 20 query execution errors to use error classes

[ https://issues.apache.org/jira/browse/SPARK-36299?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17391278#comment-17391278 ] dgd_contributor commented on SPARK-36299: - working on this. > Refactor tenth set of 20 query

[jira] [Commented] (SPARK-30602) SPIP: Support push-based shuffle to improve shuffle efficiency

[ https://issues.apache.org/jira/browse/SPARK-30602?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17391276#comment-17391276 ] Gengliang Wang commented on SPARK-30602: [~zhouyejoe] Great, thanks! > SPIP: Support push-based

[jira] [Created] (SPARK-36377) Fix documentation in spark-env.sh.template

Yuto Akutsu created SPARK-36377: --- Summary: Fix documentation in spark-env.sh.template Key: SPARK-36377 URL: https://issues.apache.org/jira/browse/SPARK-36377 Project: Spark Issue Type:

[jira] [Assigned] (SPARK-36376) Collapse repartitions if there is a project between them

[ https://issues.apache.org/jira/browse/SPARK-36376?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36376: Assignee: (was: Apache Spark) > Collapse repartitions if there is a project between

[jira] [Assigned] (SPARK-36376) Collapse repartitions if there is a project between them

[ https://issues.apache.org/jira/browse/SPARK-36376?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36376: Assignee: Apache Spark > Collapse repartitions if there is a project between them >

[jira] [Commented] (SPARK-36376) Collapse repartitions if there is a project between them

[ https://issues.apache.org/jira/browse/SPARK-36376?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17391269#comment-17391269 ] Apache Spark commented on SPARK-36376: -- User 'wangyum' has created a pull request for this issue:

[jira] [Commented] (SPARK-36375) Flaky Test: BasicSchedulerIntegrationSuite - 'super simple job' and 'multi-stage job'

[ https://issues.apache.org/jira/browse/SPARK-36375?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17391268#comment-17391268 ] wuyi commented on SPARK-36375: -- [~hyukjin.kwon] I'd like to take a look first. Thanks for the ping. >

[jira] [Created] (SPARK-36376) Collapse repartitions if there is a project between them

Yuming Wang created SPARK-36376: --- Summary: Collapse repartitions if there is a project between them Key: SPARK-36376 URL: https://issues.apache.org/jira/browse/SPARK-36376 Project: Spark Issue

[jira] [Commented] (SPARK-36375) Flaky Test: BasicSchedulerIntegrationSuite - 'super simple job' and 'multi-stage job'

[ https://issues.apache.org/jira/browse/SPARK-36375?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17391264#comment-17391264 ] Hyukjin Kwon commented on SPARK-36375: -- [~wuyi] do you have any idea on this? > Flaky Test:

[jira] [Updated] (SPARK-36375) Flaky Test: BasicSchedulerIntegrationSuite - 'super simple job' and 'multi-stage job'

[ https://issues.apache.org/jira/browse/SPARK-36375?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-36375: - Issue Type: Test (was: Improvement) > Flaky Test: BasicSchedulerIntegrationSuite - 'super

[jira] [Created] (SPARK-36375) Flaky Test: BasicSchedulerIntegrationSuite - 'super simple job' and 'multi-stage job'

Hyukjin Kwon created SPARK-36375: Summary: Flaky Test: BasicSchedulerIntegrationSuite - 'super simple job' and 'multi-stage job' Key: SPARK-36375 URL: https://issues.apache.org/jira/browse/SPARK-36375

[jira] [Commented] (SPARK-20415) SPARK job hangs while writing DataFrame to HDFS

[ https://issues.apache.org/jira/browse/SPARK-20415?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17391259#comment-17391259 ] Yu Gan commented on SPARK-20415: Did you find the root cause? I came across the same issue in our azure

[jira] [Assigned] (SPARK-36372) ALTER TABLE ADD COLUMNS should check duplicates for the specified columns for v2 command

[ https://issues.apache.org/jira/browse/SPARK-36372?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36372: Assignee: (was: Apache Spark) > ALTER TABLE ADD COLUMNS should check duplicates for

[jira] [Assigned] (SPARK-36372) ALTER TABLE ADD COLUMNS should check duplicates for the specified columns for v2 command

[ https://issues.apache.org/jira/browse/SPARK-36372?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36372: Assignee: Apache Spark > ALTER TABLE ADD COLUMNS should check duplicates for the

[jira] [Commented] (SPARK-36372) ALTER TABLE ADD COLUMNS should check duplicates for the specified columns for v2 command

[ https://issues.apache.org/jira/browse/SPARK-36372?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17391258#comment-17391258 ] Apache Spark commented on SPARK-36372: -- User 'imback82' has created a pull request for this issue:

[jira] [Created] (SPARK-36374) Push-based shuffle documentation

Venkata krishnan Sowrirajan created SPARK-36374: --- Summary: Push-based shuffle documentation Key: SPARK-36374 URL: https://issues.apache.org/jira/browse/SPARK-36374 Project: Spark

[jira] [Commented] (SPARK-36329) show api of Dataset should get as input the output method

[ https://issues.apache.org/jira/browse/SPARK-36329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17391159#comment-17391159 ] Izek Greenfield commented on SPARK-36329: - # if you do it like that you should write it again

[jira] [Assigned] (SPARK-36373) DecimalPrecision only add necessary cast

[ https://issues.apache.org/jira/browse/SPARK-36373?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36373: Assignee: (was: Apache Spark) > DecimalPrecision only add necessary cast >

[jira] [Assigned] (SPARK-36373) DecimalPrecision only add necessary cast

[ https://issues.apache.org/jira/browse/SPARK-36373?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36373: Assignee: Apache Spark > DecimalPrecision only add necessary cast >

[jira] [Commented] (SPARK-36373) DecimalPrecision only add necessary cast

[ https://issues.apache.org/jira/browse/SPARK-36373?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17391150#comment-17391150 ] Apache Spark commented on SPARK-36373: -- User 'wangyum' has created a pull request for this issue:

[jira] [Resolved] (SPARK-36092) Migrate to GitHub Actions Codecov from Jenkins

[ https://issues.apache.org/jira/browse/SPARK-36092?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-36092. -- Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 33591

[jira] [Assigned] (SPARK-36092) Migrate to GitHub Actions Codecov from Jenkins

[ https://issues.apache.org/jira/browse/SPARK-36092?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-36092: Assignee: Hyukjin Kwon > Migrate to GitHub Actions Codecov from Jenkins >

[jira] [Created] (SPARK-36373) DecimalPrecision only add necessary cast

Yuming Wang created SPARK-36373: --- Summary: DecimalPrecision only add necessary cast Key: SPARK-36373 URL: https://issues.apache.org/jira/browse/SPARK-36373 Project: Spark Issue Type:

[jira] [Commented] (SPARK-36362) Omnibus Java code static analyzer warning fixes

[ https://issues.apache.org/jira/browse/SPARK-36362?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1739#comment-1739 ] Apache Spark commented on SPARK-36362: -- User 'dongjoon-hyun' has created a pull request for this

[jira] [Commented] (SPARK-36362) Omnibus Java code static analyzer warning fixes

[ https://issues.apache.org/jira/browse/SPARK-36362?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17391110#comment-17391110 ] Apache Spark commented on SPARK-36362: -- User 'dongjoon-hyun' has created a pull request for this