[jira] [Resolved] (SPARK-36530) Avoid finalizing when there's no push at all in a shuffle

[ https://issues.apache.org/jira/browse/SPARK-36530?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wuyi resolved SPARK-36530. -- Resolution: Won't Fix > Avoid finalizing when there's no push at all in a shuffle >

[jira] [Commented] (SPARK-36530) Avoid finalizing when there's no push at all in a shuffle

[ https://issues.apache.org/jira/browse/SPARK-36530?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17400118#comment-17400118 ] wuyi commented on SPARK-36530: -- SGTM. Could you update SPARK-33701 to include this part? Then, I'll close

[jira] [Commented] (SPARK-36530) Avoid finalizing when there's no push at all in a shuffle

[ https://issues.apache.org/jira/browse/SPARK-36530?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17400089#comment-17400089 ] wuyi commented on SPARK-36530: -- cc [~mridulm80] [~mshen] any thoughts? > Avoid finalizing when there's no

[jira] [Created] (SPARK-36530) Avoid finalizing when there's no push at all in a shuffle

wuyi created SPARK-36530: Summary: Avoid finalizing when there's no push at all in a shuffle Key: SPARK-36530 URL: https://issues.apache.org/jira/browse/SPARK-36530 Project: Spark Issue Type:

[jira] [Resolved] (SPARK-36378) Minor changes to address a few identified server side inefficiencies

[ https://issues.apache.org/jira/browse/SPARK-36378?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wuyi resolved SPARK-36378. -- Fix Version/s: 3.3.0 3.2.0 Assignee: Min Shen Resolution: Fixed Issue

[jira] [Resolved] (SPARK-36332) Cleanup RemoteBlockPushResolver log messages

[ https://issues.apache.org/jira/browse/SPARK-36332?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wuyi resolved SPARK-36332. -- Fix Version/s: 3.3.0 3.2.0 Assignee: Venkata krishnan Sowrirajan

[jira] [Resolved] (SPARK-36391) When fetch chunk throw NPE, improve the error message

[ https://issues.apache.org/jira/browse/SPARK-36391?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wuyi resolved SPARK-36391. -- Fix Version/s: 3.3.0 3.2.0 Resolution: Fixed Issue resolved by 

[jira] [Updated] (SPARK-36383) NullPointerException throws during executor shutdown

[ https://issues.apache.org/jira/browse/SPARK-36383?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wuyi updated SPARK-36383: - Issue Type: Bug (was: Improvement) > NullPointerException throws during executor shutdown >

[jira] [Created] (SPARK-36384) Add documentation for shuffle checksum

wuyi created SPARK-36384: Summary: Add documentation for shuffle checksum Key: SPARK-36384 URL: https://issues.apache.org/jira/browse/SPARK-36384 Project: Spark Issue Type: Sub-task

[jira] [Commented] (SPARK-35275) Add checksum for shuffle blocks

[ https://issues.apache.org/jira/browse/SPARK-35275?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17391654#comment-17391654 ] wuyi commented on SPARK-35275: -- [~mridulm80] Yes, we shall have the doc task. Let me create it. > Add

[jira] [Updated] (SPARK-36383) NullPointerException throws during executor shutdown

[ https://issues.apache.org/jira/browse/SPARK-36383?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wuyi updated SPARK-36383: - Summary: NullPointerException throws during executor shutdown (was: Avoid NullPointerException during executor

[jira] [Created] (SPARK-36383) Avoid NullPointerException during executor shutdown

wuyi created SPARK-36383: Summary: Avoid NullPointerException during executor shutdown Key: SPARK-36383 URL: https://issues.apache.org/jira/browse/SPARK-36383 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-36375) Flaky Test: BasicSchedulerIntegrationSuite - 'super simple job' and 'multi-stage job'

[ https://issues.apache.org/jira/browse/SPARK-36375?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17391268#comment-17391268 ] wuyi commented on SPARK-36375: -- [~hyukjin.kwon] I'd like to take a look first. Thanks for the ping. >

[jira] [Created] (SPARK-36284) Add shuffle checksum support for push-based shuffle

wuyi created SPARK-36284: Summary: Add shuffle checksum support for push-based shuffle Key: SPARK-36284 URL: https://issues.apache.org/jira/browse/SPARK-36284 Project: Spark Issue Type: Sub-task

[jira] [Resolved] (SPARK-36255) FileNotFoundException from the shuffle push can cause the executor to terminate

[ https://issues.apache.org/jira/browse/SPARK-36255?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wuyi resolved SPARK-36255. -- Fix Version/s: 3.3.0 3.2.0 Assignee: Chandni Singh Resolution: Fixed Issue

[jira] [Resolved] (SPARK-36242) Ensure spill file closed before set success to true in ExternalSorter.spillMemoryIteratorToDisk method

[ https://issues.apache.org/jira/browse/SPARK-36242?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wuyi resolved SPARK-36242. -- Fix Version/s: 3.3.0 3.2.0 Assignee: Yang Jie Resolution: Fixed Issue

[jira] [Updated] (SPARK-36206) Diagnose shuffle data corruption by checksum

[ https://issues.apache.org/jira/browse/SPARK-36206?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wuyi updated SPARK-36206: - Description: After adding checksums in SPARK-35276, we can leverage the checksums to do diagnosis for shuffle

[jira] [Created] (SPARK-36206) Diagnose shuffle data corruption by checksum

wuyi created SPARK-36206: Summary: Diagnose shuffle data corruption by checksum Key: SPARK-36206 URL: https://issues.apache.org/jira/browse/SPARK-36206 Project: Spark Issue Type: Sub-task

[jira] [Resolved] (SPARK-36048) Wrong HealthTrackerSuite.allExecutorAndHostIds

[ https://issues.apache.org/jira/browse/SPARK-36048?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wuyi resolved SPARK-36048. -- Fix Version/s: 3.3.0 Assignee: wuyi Resolution: Fixed Issue resolved by 

[jira] [Updated] (SPARK-36048) Wrong HealthTrackerSuite.allExecutorAndHostIds

[ https://issues.apache.org/jira/browse/SPARK-36048?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wuyi updated SPARK-36048: - Summary: Wrong HealthTrackerSuite.allExecutorAndHostIds (was: Fix HealthTrackerSuite.allExecutorAndHostIds) >

[jira] [Created] (SPARK-36048) Fix HealthTrackerSuite.allExecutorAndHostIds

wuyi created SPARK-36048: Summary: Fix HealthTrackerSuite.allExecutorAndHostIds Key: SPARK-36048 URL: https://issues.apache.org/jira/browse/SPARK-36048 Project: Spark Issue Type: Test

[jira] [Created] (SPARK-35869) Cannot run program "python" error when run do-release-docker.sh

wuyi created SPARK-35869: Summary: Cannot run program "python" error when run do-release-docker.sh Key: SPARK-35869 URL: https://issues.apache.org/jira/browse/SPARK-35869 Project: Spark Issue Type:

[jira] [Updated] (SPARK-34534) New protocol FetchShuffleBlocks in OneForOneBlockFetcher lead to data loss or correctness

[ https://issues.apache.org/jira/browse/SPARK-34534?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wuyi updated SPARK-34534: - Target Version/s: (was: 3.0.3) > New protocol FetchShuffleBlocks in OneForOneBlockFetcher lead to data loss

[jira] [Updated] (SPARK-34534) New protocol FetchShuffleBlocks in OneForOneBlockFetcher lead to data loss or correctness

[ https://issues.apache.org/jira/browse/SPARK-34534?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wuyi updated SPARK-34534: - Target Version/s: 3.0.3 > New protocol FetchShuffleBlocks in OneForOneBlockFetcher lead to data loss or >

[jira] [Resolved] (SPARK-35011) Avoid Block Manager registerations when StopExecutor msg is in-flight.

[ https://issues.apache.org/jira/browse/SPARK-35011?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wuyi resolved SPARK-35011. -- Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by [https://github.com/apache/spark/pull/32114]

[jira] [Assigned] (SPARK-35011) Avoid Block Manager registerations when StopExecutor msg is in-flight.

[ https://issues.apache.org/jira/browse/SPARK-35011?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wuyi reassigned SPARK-35011: Assignee: Sumeet > Avoid Block Manager registerations when StopExecutor msg is in-flight. >

[jira] [Resolved] (SPARK-35486) MemoryConsumer reservations that trigger a partial self-spill can fail even if memory is available

[ https://issues.apache.org/jira/browse/SPARK-35486?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wuyi resolved SPARK-35486. -- Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by https://github.com/apache/spark/pull/32625 >

[jira] [Created] (SPARK-35454) Ambiguous self-join doesn't fail after transfroming the dataset to dataframe

wuyi created SPARK-35454: Summary: Ambiguous self-join doesn't fail after transfroming the dataset to dataframe Key: SPARK-35454 URL: https://issues.apache.org/jira/browse/SPARK-35454 Project: Spark

[jira] [Created] (SPARK-35445) Reduce the execution time of DeduplicateRelations

wuyi created SPARK-35445: Summary: Reduce the execution time of DeduplicateRelations Key: SPARK-35445 URL: https://issues.apache.org/jira/browse/SPARK-35445 Project: Spark Issue Type: Improvement

[jira] [Created] (SPARK-35330) Tune shuffle requests frequency dynamically in the case of Netty OOM

wuyi created SPARK-35330: Summary: Tune shuffle requests frequency dynamically in the case of Netty OOM Key: SPARK-35330 URL: https://issues.apache.org/jira/browse/SPARK-35330 Project: Spark Issue

[jira] [Created] (SPARK-35276) Write checksum files for shuffle

wuyi created SPARK-35276: Summary: Write checksum files for shuffle Key: SPARK-35276 URL: https://issues.apache.org/jira/browse/SPARK-35276 Project: Spark Issue Type: Sub-task Components:

[jira] [Created] (SPARK-35275) Add checksum for shuffle blocks

wuyi created SPARK-35275: Summary: Add checksum for shuffle blocks Key: SPARK-35275 URL: https://issues.apache.org/jira/browse/SPARK-35275 Project: Spark Issue Type: New Feature

[jira] [Resolved] (SPARK-35135) Duplicate code implementation of `WritablePartitionedIterator`

[ https://issues.apache.org/jira/browse/SPARK-35135?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wuyi resolved SPARK-35135. -- Fix Version/s: 3.2.0 Target Version/s: 3.2.0 Resolution: Fixed Issue resolved by 

[jira] [Assigned] (SPARK-35135) Duplicate code implementation of `WritablePartitionedIterator`

[ https://issues.apache.org/jira/browse/SPARK-35135?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wuyi reassigned SPARK-35135: Assignee: Yang Jie (was: Apache Spark) > Duplicate code implementation of `WritablePartitionedIterator`

[jira] [Created] (SPARK-35234) Reserve format of the stage failureMessage

wuyi created SPARK-35234: Summary: Reserve format of the stage failureMessage Key: SPARK-35234 URL: https://issues.apache.org/jira/browse/SPARK-35234 Project: Spark Issue Type: Improvement

[jira] [Created] (SPARK-35206) Extract common get project path ability as function to SparkFunctionSuite

wuyi created SPARK-35206: Summary: Extract common get project path ability as function to SparkFunctionSuite Key: SPARK-35206 URL: https://issues.apache.org/jira/browse/SPARK-35206 Project: Spark

[jira] [Created] (SPARK-35188) Support json serde for the custom location

wuyi created SPARK-35188: Summary: Support json serde for the custom location Key: SPARK-35188 URL: https://issues.apache.org/jira/browse/SPARK-35188 Project: Spark Issue Type: Sub-task

[jira] [Updated] (SPARK-35186) Define local/hostlocal/remote fetch for custom storage

[ https://issues.apache.org/jira/browse/SPARK-35186?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wuyi updated SPARK-35186: - Description: After introducing the custom storage system, we need to think about how to define

[jira] [Created] (SPARK-35186) Define local/hostlocal/remote fetch for custom storage

wuyi created SPARK-35186: Summary: Define local/hostlocal/remote fetch for custom storage Key: SPARK-35186 URL: https://issues.apache.org/jira/browse/SPARK-35186 Project: Spark Issue Type: Sub-task

[jira] [Resolved] (SPARK-30849) Application failed due to failed to get MapStatuses broadcast

[ https://issues.apache.org/jira/browse/SPARK-30849?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wuyi resolved SPARK-30849. -- Resolution: Duplicate > Application failed due to failed to get MapStatuses broadcast >

[jira] [Created] (SPARK-34942) Abstract Location in MapStatus to enable support for custom storage

wuyi created SPARK-34942: Summary: Abstract Location in MapStatus to enable support for custom storage Key: SPARK-34942 URL: https://issues.apache.org/jira/browse/SPARK-34942 Project: Spark Issue

[jira] [Resolved] (SPARK-32384) repartitionAndSortWithinPartitions avoid shuffle with same partitioner

[ https://issues.apache.org/jira/browse/SPARK-32384?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wuyi resolved SPARK-32384. -- Fix Version/s: 3.2.0 Assignee: zhengruifeng Resolution: Fixed Resolved by 

[jira] [Resolved] (SPARK-34782) Not equal functions (!=, <>) are missing in SQL doc

[ https://issues.apache.org/jira/browse/SPARK-34782?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wuyi resolved SPARK-34782. -- Resolution: Duplicate > Not equal functions (!=, <>) are missing in SQL doc >

[jira] [Commented] (SPARK-34782) Not equal functions (!=, <>) are missing in SQL doc

[ https://issues.apache.org/jira/browse/SPARK-34782?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17304585#comment-17304585 ] wuyi commented on SPARK-34782: -- Oh yea, thanks for catching it. I'll close this. > Not equal functions

[jira] [Updated] (SPARK-34788) Spark throws FileNotFoundException instead of IOException when disk is full

[ https://issues.apache.org/jira/browse/SPARK-34788?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wuyi updated SPARK-34788: - Description: When the disk is full, Spark throws FileNotFoundException instead of IOException with the hint.

[jira] [Commented] (SPARK-34788) Spark throws FileNotFoundException instead of IOException when disk is full

[ https://issues.apache.org/jira/browse/SPARK-34788?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17304581#comment-17304581 ] wuyi commented on SPARK-34788: -- [~dongjoon] I have pasted the error stack. It happened when we reading the

[jira] [Updated] (SPARK-34788) Spark throws FileNotFoundException instead of IOException when disk is full

[ https://issues.apache.org/jira/browse/SPARK-34788?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wuyi updated SPARK-34788: - Description: When the disk is full, Spark throws FileNotFoundException instead of IOException with the hint.

[jira] [Updated] (SPARK-34788) Spark throws FileNotFoundException instead of IOException when disk is full

[ https://issues.apache.org/jira/browse/SPARK-34788?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wuyi updated SPARK-34788: - Component/s: Shuffle > Spark throws FileNotFoundException instead of IOException when disk is full >

[jira] [Commented] (SPARK-34788) Spark throws FileNotFoundException instead of IOException when disk is full

[ https://issues.apache.org/jira/browse/SPARK-34788?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17304185#comment-17304185 ] wuyi commented on SPARK-34788: -- [~wuxiaofei] Could you take a look at this? > Spark throws

[jira] [Created] (SPARK-34788) Spark throws FileNotFoundException instead of IOException when disk is full

wuyi created SPARK-34788: Summary: Spark throws FileNotFoundException instead of IOException when disk is full Key: SPARK-34788 URL: https://issues.apache.org/jira/browse/SPARK-34788 Project: Spark

[jira] [Commented] (SPARK-34782) Not equal functions (!=, <>) are missing in SQL doc

[ https://issues.apache.org/jira/browse/SPARK-34782?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17303811#comment-17303811 ] wuyi commented on SPARK-34782: -- cc [~beliefer] Could you help with this? > Not equal functions (!=, <>)

[jira] [Created] (SPARK-34782) Not equal functions (!=, <>) are missing in SQL doc

wuyi created SPARK-34782: Summary: Not equal functions (!=, <>) are missing in SQL doc Key: SPARK-34782 URL: https://issues.apache.org/jira/browse/SPARK-34782 Project: Spark Issue Type:

[jira] [Updated] (SPARK-34741) MergeIntoTable should avoid ambiguous reference

[ https://issues.apache.org/jira/browse/SPARK-34741?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wuyi updated SPARK-34741: - Description: When resolving the {{UpdateAction}}, which could reference attributes from both target and source

[jira] [Created] (SPARK-34741) MergeIntoTable should avoid ambiguous reference

wuyi created SPARK-34741: Summary: MergeIntoTable should avoid ambiguous reference Key: SPARK-34741 URL: https://issues.apache.org/jira/browse/SPARK-34741 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-34571) Provide a more convenient way to deprecate/remove/alternate configs

wuyi created SPARK-34571: Summary: Provide a more convenient way to deprecate/remove/alternate configs Key: SPARK-34571 URL: https://issues.apache.org/jira/browse/SPARK-34571 Project: Spark Issue

[jira] [Updated] (SPARK-34482) Correct the active SparkSession for streaming query

[ https://issues.apache.org/jira/browse/SPARK-34482?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wuyi updated SPARK-34482: - Description:

[jira] [Created] (SPARK-34482) Correct the active SparkSession for streaming query

wuyi created SPARK-34482: Summary: Correct the active SparkSession for streaming query Key: SPARK-34482 URL: https://issues.apache.org/jira/browse/SPARK-34482 Project: Spark Issue Type: Improvement

[jira] [Updated] (SPARK-34384) Add missing docs for ResourceProfile APIs

[ https://issues.apache.org/jira/browse/SPARK-34384?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wuyi updated SPARK-34384: - Description: ResourceProfile APIs missed a few API docs, we should add them for the better user experience. >

[jira] [Updated] (SPARK-34384) Add missing docs for ResourceProfile APIs

[ https://issues.apache.org/jira/browse/SPARK-34384?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wuyi updated SPARK-34384: - Summary: Add missing docs for ResourceProfile APIs (was: API cleanup for ResourceProfile) > Add missing docs

[jira] [Updated] (SPARK-34461) Collect API feedback and maybe revise some APIs

[ https://issues.apache.org/jira/browse/SPARK-34461?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wuyi updated SPARK-34461: - Description: For example,  `ResourceProfileBuilder.require` uses the same API name for both task and

[jira] [Updated] (SPARK-34461) Collect API feedback and maybe revise some APIs

[ https://issues.apache.org/jira/browse/SPARK-34461?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wuyi updated SPARK-34461: - Summary: Collect API feedback and maybe revise some APIs (was: Collect API feedback and maybe revj) > Collect

[jira] [Created] (SPARK-34461) Collect API feedback and maybe revj

wuyi created SPARK-34461: Summary: Collect API feedback and maybe revj Key: SPARK-34461 URL: https://issues.apache.org/jira/browse/SPARK-34461 Project: Spark Issue Type: Improvement

[jira] [Created] (SPARK-34460) Add missing tests for APIs

wuyi created SPARK-34460: Summary: Add missing tests for APIs Key: SPARK-34460 URL: https://issues.apache.org/jira/browse/SPARK-34460 Project: Spark Issue Type: Improvement Components:

[jira] [Created] (SPARK-34384) API cleanup for ResourceProfile

wuyi created SPARK-34384: Summary: API cleanup for ResourceProfile Key: SPARK-34384 URL: https://issues.apache.org/jira/browse/SPARK-34384 Project: Spark Issue Type: Improvement

[jira] [Created] (SPARK-34354) CostBasedJoinReorder can fail on self-join

wuyi created SPARK-34354: Summary: CostBasedJoinReorder can fail on self-join Key: SPARK-34354 URL: https://issues.apache.org/jira/browse/SPARK-34354 Project: Spark Issue Type: Bug

[jira] [Created] (SPARK-34319) Self-join after cogroup applyInPandas fails due to unresolved conflicting attributes

wuyi created SPARK-34319: Summary: Self-join after cogroup applyInPandas fails due to unresolved conflicting attributes Key: SPARK-34319 URL: https://issues.apache.org/jira/browse/SPARK-34319 Project: Spark

[jira] [Created] (SPARK-34258) scala.reflect.internal.Symbols$CyclicReference: illegal cyclic reference involving type BuilderType

wuyi created SPARK-34258: Summary: scala.reflect.internal.Symbols$CyclicReference: illegal cyclic reference involving type BuilderType Key: SPARK-34258 URL: https://issues.apache.org/jira/browse/SPARK-34258

[jira] [Created] (SPARK-34245) Master may not remove the finished executor when Worker fails to send ExecutorStateChanged

wuyi created SPARK-34245: Summary: Master may not remove the finished executor when Worker fails to send ExecutorStateChanged Key: SPARK-34245 URL: https://issues.apache.org/jira/browse/SPARK-34245 Project:

[jira] [Updated] (SPARK-34178) Copy tags for the new node created by MultiInstanceRelation.newInstance

[ https://issues.apache.org/jira/browse/SPARK-34178?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wuyi updated SPARK-34178: - Summary: Copy tags for the new node created by MultiInstanceRelation.newInstance (was: Copy tags for the new

[jira] [Updated] (SPARK-34178) Copy tags for the new node that created by MultiInstanceRelation.newInstance

[ https://issues.apache.org/jira/browse/SPARK-34178?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wuyi updated SPARK-34178: - Summary: Copy tags for the new node that created by MultiInstanceRelation.newInstance (was: Copy tags when

[jira] [Created] (SPARK-34178) Copy tags when call MultiInstanceRelation.newInstance

wuyi created SPARK-34178: Summary: Copy tags when call MultiInstanceRelation.newInstance Key: SPARK-34178 URL: https://issues.apache.org/jira/browse/SPARK-34178 Project: Spark Issue Type:

[jira] [Created] (SPARK-34091) Shuffle batch fetch can't be disabled once it's enabled previously

wuyi created SPARK-34091: Summary: Shuffle batch fetch can't be disabled once it's enabled previously Key: SPARK-34091 URL: https://issues.apache.org/jira/browse/SPARK-34091 Project: Spark Issue

[jira] [Created] (SPARK-34089) MemoryConsumer's memory mode should respect MemoryManager's memory mode

wuyi created SPARK-34089: Summary: MemoryConsumer's memory mode should respect MemoryManager's memory mode Key: SPARK-34089 URL: https://issues.apache.org/jira/browse/SPARK-34089 Project: Spark

[jira] [Updated] (SPARK-34088) Rename all decommission configurations to use the same namespace "spark.decommission.*"

[ https://issues.apache.org/jira/browse/SPARK-34088?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wuyi updated SPARK-34088: - Description: Currently, decommission configurations are using difference namespaces, e.g.,  *

[jira] [Created] (SPARK-34088) Rename all decommission configurations to fix same namespace "spark.decommission.*"

wuyi created SPARK-34088: Summary: Rename all decommission configurations to fix same namespace "spark.decommission.*" Key: SPARK-34088 URL: https://issues.apache.org/jira/browse/SPARK-34088 Project: Spark

[jira] [Updated] (SPARK-34088) Rename all decommission configurations to use the same namespace "spark.decommission.*"

[ https://issues.apache.org/jira/browse/SPARK-34088?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wuyi updated SPARK-34088: - Summary: Rename all decommission configurations to use the same namespace "spark.decommission.*" (was: Rename

[jira] [Created] (SPARK-34054) BlockManagerDecommissioner cleanup

wuyi created SPARK-34054: Summary: BlockManagerDecommissioner cleanup Key: SPARK-34054 URL: https://issues.apache.org/jira/browse/SPARK-34054 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-33928) Flaky o.a.s.ExecutorAllocationManagerSuite - "SPARK-23365 Don't update target num executors when killing idle executors"

wuyi created SPARK-33928: Summary: Flaky o.a.s.ExecutorAllocationManagerSuite - "SPARK-23365 Don't update target num executors when killing idle executors" Key: SPARK-33928 URL:

[jira] [Created] (SPARK-33923) Fix some tests with AQE enabled

wuyi created SPARK-33923: Summary: Fix some tests with AQE enabled Key: SPARK-33923 URL: https://issues.apache.org/jira/browse/SPARK-33923 Project: Spark Issue Type: Improvement

[jira] [Created] (SPARK-33799) ExecutorMonitor should handle SparkListenerExecutorExcluded/SparkListenerNodeExcluded

wuyi created SPARK-33799: Summary: ExecutorMonitor should handle SparkListenerExecutorExcluded/SparkListenerNodeExcluded Key: SPARK-33799 URL: https://issues.apache.org/jira/browse/SPARK-33799 Project: Spark

[jira] [Updated] (SPARK-33774) "Back to Master" returns 500 error in Standalone cluster

[ https://issues.apache.org/jira/browse/SPARK-33774?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wuyi updated SPARK-33774: - Summary: "Back to Master" returns 500 error in Standalone cluster (was: "Back to Master" returns 500 error in

[jira] [Updated] (SPARK-33774) "Back to Master" returns 500 error in local Standalone cluster

[ https://issues.apache.org/jira/browse/SPARK-33774?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wuyi updated SPARK-33774: - Summary: "Back to Master" returns 500 error in local Standalone cluster (was: "Back to Master" returns 500

[jira] [Created] (SPARK-33774) "Back to Master" returns 500 error

wuyi created SPARK-33774: Summary: "Back to Master" returns 500 error Key: SPARK-33774 URL: https://issues.apache.org/jira/browse/SPARK-33774 Project: Spark Issue Type: Bug Components:

[jira] [Commented] (SPARK-33545) Support Fallback Storage during Worker decommission

[ https://issues.apache.org/jira/browse/SPARK-33545?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17242862#comment-17242862 ] wuyi commented on SPARK-33545: -- Hi [~dongjoon] Why this's a subtask of k8s GA instead of

[jira] [Created] (SPARK-33536) Incorrect join results when joining twice with the same DF

wuyi created SPARK-33536: Summary: Incorrect join results when joining twice with the same DF Key: SPARK-33536 URL: https://issues.apache.org/jira/browse/SPARK-33536 Project: Spark Issue Type: Bug

[jira] [Resolved] (SPARK-31018) Deprecate support of multiple workers on the same host in Standalone

[ https://issues.apache.org/jira/browse/SPARK-31018?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wuyi resolved SPARK-31018. -- Resolution: Fixed resolved by https://github.com/apache/spark/pull/27768 > Deprecate support of multiple

[jira] [Comment Edited] (SPARK-33331) Limit the number of pending blocks in memory and store blocks that collide

[ https://issues.apache.org/jira/browse/SPARK-1?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17226531#comment-17226531 ] wuyi edited comment on SPARK-1 at 11/5/20, 7:12 AM: I like the idea to cache

[jira] [Comment Edited] (SPARK-33331) Limit the number of pending blocks in memory and store blocks that collide

[ https://issues.apache.org/jira/browse/SPARK-1?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17226531#comment-17226531 ] wuyi edited comment on SPARK-1 at 11/5/20, 7:09 AM: I like the idea to cache

[jira] [Comment Edited] (SPARK-33331) Limit the number of pending blocks in memory and store blocks that collide

[ https://issues.apache.org/jira/browse/SPARK-1?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17226531#comment-17226531 ] wuyi edited comment on SPARK-1 at 11/5/20, 6:59 AM: I like the idea to cache

[jira] [Commented] (SPARK-33331) Limit the number of pending blocks in memory and store blocks that collide

[ https://issues.apache.org/jira/browse/SPARK-1?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17226531#comment-17226531 ] wuyi commented on SPARK-1: -- I like the idea to cache the blocks of the worst case instead of throwing

[jira] [Created] (SPARK-32984) Improve showing the differences between approved and actual plans

wuyi created SPARK-32984: Summary: Improve showing the differences between approved and actual plans Key: SPARK-32984 URL: https://issues.apache.org/jira/browse/SPARK-32984 Project: Spark Issue

[jira] [Commented] (SPARK-32937) DecomissionSuite in k8s integration tests is failing.

[ https://issues.apache.org/jira/browse/SPARK-32937?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17199125#comment-17199125 ] wuyi commented on SPARK-32937: -- I'm looking at it. Thanks for reporting! > DecomissionSuite in k8s

[jira] [Updated] (SPARK-32913) Improve ExecutorDecommissionInfo and ExecutorDecommissionState for different use cases

[ https://issues.apache.org/jira/browse/SPARK-32913?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wuyi updated SPARK-32913: - Summary: Improve ExecutorDecommissionInfo and ExecutorDecommissionState for different use cases (was: Improve

[jira] [Created] (SPARK-32913) Improve DecommissionInfo and DecommissionState for different use cases

wuyi created SPARK-32913: Summary: Improve DecommissionInfo and DecommissionState for different use cases Key: SPARK-32913 URL: https://issues.apache.org/jira/browse/SPARK-32913 Project: Spark

[jira] [Commented] (SPARK-32898) totalExecutorRunTimeMs is too big

[ https://issues.apache.org/jira/browse/SPARK-32898?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17197341#comment-17197341 ] wuyi commented on SPARK-32898: -- I think the issue is(for executorRunTimeMs): Before a task reaches to

[jira] [Updated] (SPARK-32898) totalExecutorRunTimeMs is too big

[ https://issues.apache.org/jira/browse/SPARK-32898?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wuyi updated SPARK-32898: - Description: This might be because of incorrectly calculating executorRunTimeMs in Executor.scala The

[jira] [Created] (SPARK-32878) Avoid scheduling TaskSetManager which has no pending tasks

wuyi created SPARK-32878: Summary: Avoid scheduling TaskSetManager which has no pending tasks Key: SPARK-32878 URL: https://issues.apache.org/jira/browse/SPARK-32878 Project: Spark Issue Type:

[jira] [Created] (SPARK-32857) Flaky o.a.s.scheduler.BarrierTaskContextSuite.throw exception if the number of barrier() calls are not the same on every task

wuyi created SPARK-32857: Summary: Flaky o.a.s.scheduler.BarrierTaskContextSuite.throw exception if the number of barrier() calls are not the same on every task Key: SPARK-32857 URL:

[jira] [Created] (SPARK-32850) Simply the RPC message flow of decommission

wuyi created SPARK-32850: Summary: Simply the RPC message flow of decommission Key: SPARK-32850 URL: https://issues.apache.org/jira/browse/SPARK-32850 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-32736) Avoid caching the removed decommissioned executors in TaskSchedulerImpl

wuyi created SPARK-32736: Summary: Avoid caching the removed decommissioned executors in TaskSchedulerImpl Key: SPARK-32736 URL: https://issues.apache.org/jira/browse/SPARK-32736 Project: Spark

<    1   2   3   4   5   >