[jira] [Commented] (SPARK-46336) Cancel zombie tasks when a ShuffleMapStage is finished

[ https://issues.apache.org/jira/browse/SPARK-46336?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17794888#comment-17794888 ] Xingbo Jiang commented on SPARK-46336: -- I'm currently working on this. > Cancel zombie tasks when

[jira] [Created] (SPARK-46336) Cancel zombie tasks when a ShuffleMapStage is finished

Xingbo Jiang created SPARK-46336: Summary: Cancel zombie tasks when a ShuffleMapStage is finished Key: SPARK-46336 URL: https://issues.apache.org/jira/browse/SPARK-46336 Project: Spark Issue

[jira] [Created] (SPARK-46182) Shuffle data lost on decommissioned executor caused by race condition between lastTaskRunningTime and lastShuffleMigrationTime

Xingbo Jiang created SPARK-46182: Summary: Shuffle data lost on decommissioned executor caused by race condition between lastTaskRunningTime and lastShuffleMigrationTime Key: SPARK-46182 URL:

[jira] [Resolved] (SPARK-43043) Improve the performance of MapOutputTracker.updateMapOutput

[ https://issues.apache.org/jira/browse/SPARK-43043?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xingbo Jiang resolved SPARK-43043. -- Fix Version/s: 3.4.1 Resolution: Done > Improve the performance of

[jira] [Created] (SPARK-43515) Add a benchmark to evaluate the performance of migrating a large number of tiny blocks during decommission

Xingbo Jiang created SPARK-43515: Summary: Add a benchmark to evaluate the performance of migrating a large number of tiny blocks during decommission Key: SPARK-43515 URL:

[jira] [Created] (SPARK-43043) Improve the performance of MapOutputTracker.updateMapOutput

Xingbo Jiang created SPARK-43043: Summary: Improve the performance of MapOutputTracker.updateMapOutput Key: SPARK-43043 URL: https://issues.apache.org/jira/browse/SPARK-43043 Project: Spark

[jira] [Assigned] (SPARK-43043) Improve the performance of MapOutputTracker.updateMapOutput

[ https://issues.apache.org/jira/browse/SPARK-43043?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xingbo Jiang reassigned SPARK-43043: Assignee: Xingbo Jiang > Improve the performance of MapOutputTracker.updateMapOutput >

[jira] [Created] (SPARK-42967) Fix SparkListenerTaskStart.stageAttemptId when a task is started after the stage is cancelled

Xingbo Jiang created SPARK-42967: Summary: Fix SparkListenerTaskStart.stageAttemptId when a task is started after the stage is cancelled Key: SPARK-42967 URL: https://issues.apache.org/jira/browse/SPARK-42967

[jira] [Created] (SPARK-39062) Add Standalone backend support for Stage Level Scheduling

Xingbo Jiang created SPARK-39062: Summary: Add Standalone backend support for Stage Level Scheduling Key: SPARK-39062 URL: https://issues.apache.org/jira/browse/SPARK-39062 Project: Spark

[jira] [Commented] (SPARK-38388) Repartition + Stage retries could lead to incorrect data

[ https://issues.apache.org/jira/browse/SPARK-38388?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17502486#comment-17502486 ] Xingbo Jiang commented on SPARK-38388: -- I see in your example you used `rand()` function between

[jira] [Commented] (SPARK-38388) Repartition + Stage retries could lead to incorrect data

[ https://issues.apache.org/jira/browse/SPARK-38388?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17502481#comment-17502481 ] Xingbo Jiang commented on SPARK-38388: -- > upstream stage (data with randomness) I assume you are

[jira] [Commented] (SPARK-38388) Repartition + Stage retries could lead to incorrect data

[ https://issues.apache.org/jira/browse/SPARK-38388?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17500997#comment-17500997 ] Xingbo Jiang commented on SPARK-38388: -- Have you tried override the `getOutputDeterministicLevel`

[jira] [Commented] (SPARK-37891) Add scalastyle check to disable scala.concurrent.ExecutionContext.Implicits.global

[ https://issues.apache.org/jira/browse/SPARK-37891?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17481370#comment-17481370 ] Xingbo Jiang commented on SPARK-37891: -- Issue resolved by pull request:

[jira] [Resolved] (SPARK-37891) Add scalastyle check to disable scala.concurrent.ExecutionContext.Implicits.global

[ https://issues.apache.org/jira/browse/SPARK-37891?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xingbo Jiang resolved SPARK-37891. -- Assignee: Tianhan Hu Resolution: Fixed > Add scalastyle check to disable >

[jira] [Updated] (SPARK-37151) Avoid executor state sync attempt fail continuously in a short timeframe

[ https://issues.apache.org/jira/browse/SPARK-37151?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xingbo Jiang updated SPARK-37151: - Description: A worker would retry sending the ExecutorStateChanged message when the previous

[jira] [Created] (SPARK-37151) Avoid executor state sync attempt fail continuously in a short timeframe

Xingbo Jiang created SPARK-37151: Summary: Avoid executor state sync attempt fail continuously in a short timeframe Key: SPARK-37151 URL: https://issues.apache.org/jira/browse/SPARK-37151 Project:

[jira] [Updated] (SPARK-36500) The temp_shuffle files are not cleaned up when a task is interrupted

[ https://issues.apache.org/jira/browse/SPARK-36500?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xingbo Jiang updated SPARK-36500: - Description: We discovered a potential file leak on the long running cluster. As the cluster

[jira] [Created] (SPARK-36500) The temp_shuffle files are not cleaned up when a task is interrupted

Xingbo Jiang created SPARK-36500: Summary: The temp_shuffle files are not cleaned up when a task is interrupted Key: SPARK-36500 URL: https://issues.apache.org/jira/browse/SPARK-36500 Project: Spark

[jira] [Created] (SPARK-32658) Partition length number overflow in `PartitionWriterStream`

Xingbo Jiang created SPARK-32658: Summary: Partition length number overflow in `PartitionWriterStream` Key: SPARK-32658 URL: https://issues.apache.org/jira/browse/SPARK-32658 Project: Spark

[jira] [Resolved] (SPARK-31824) DAGSchedulerSuite: Improve and reuse completeShuffleMapStageSuccessfully

[ https://issues.apache.org/jira/browse/SPARK-31824?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xingbo Jiang resolved SPARK-31824. -- Fix Version/s: 3.1.0 Assignee: jiaan.geng Resolution: Fixed Resolved by

[jira] [Updated] (SPARK-31764) JsonProtocol doesn't write RDDInfo#isBarrier

[ https://issues.apache.org/jira/browse/SPARK-31764?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xingbo Jiang updated SPARK-31764: - Affects Version/s: (was: 3.1.0) 3.0.0 > JsonProtocol doesn't write

[jira] [Updated] (SPARK-31764) JsonProtocol doesn't write RDDInfo#isBarrier

[ https://issues.apache.org/jira/browse/SPARK-31764?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xingbo Jiang updated SPARK-31764: - Fix Version/s: (was: 3.1.0) 3.0.0 > JsonProtocol doesn't write

[jira] [Commented] (SPARK-31764) JsonProtocol doesn't write RDDInfo#isBarrier

[ https://issues.apache.org/jira/browse/SPARK-31764?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17118203#comment-17118203 ] Xingbo Jiang commented on SPARK-31764: -- The issue is reported as "Improvement" and the affected

[jira] [Resolved] (SPARK-31730) Flaky test: org.apache.spark.scheduler.BarrierTaskContextSuite

[ https://issues.apache.org/jira/browse/SPARK-31730?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xingbo Jiang resolved SPARK-31730. -- Fix Version/s: 3.0.0 Assignee: Xingbo Jiang Resolution: Fixed Fixed by

[jira] [Updated] (SPARK-31764) JsonProtocol doesn't write RDDInfo#isBarrier

[ https://issues.apache.org/jira/browse/SPARK-31764?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xingbo Jiang updated SPARK-31764: - Fix Version/s: 3.1.0 > JsonProtocol doesn't write RDDInfo#isBarrier >

[jira] [Resolved] (SPARK-31764) JsonProtocol doesn't write RDDInfo#isBarrier

[ https://issues.apache.org/jira/browse/SPARK-31764?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xingbo Jiang resolved SPARK-31764. -- Resolution: Fixed > JsonProtocol doesn't write RDDInfo#isBarrier >

[jira] [Commented] (SPARK-31764) JsonProtocol doesn't write RDDInfo#isBarrier

[ https://issues.apache.org/jira/browse/SPARK-31764?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17118118#comment-17118118 ] Xingbo Jiang commented on SPARK-31764: -- Resolved by https://github.com/apache/spark/pull/28583 >

[jira] [Commented] (SPARK-31784) Fix test BarrierTaskContextSuite."share messages with allGather() call"

[ https://issues.apache.org/jira/browse/SPARK-31784?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17113796#comment-17113796 ] Xingbo Jiang commented on SPARK-31784: -- Fixed by https://github.com/apache/spark/pull/28596 > Fix

[jira] [Updated] (SPARK-31784) Fix test BarrierTaskContextSuite."share messages with allGather() call"

[ https://issues.apache.org/jira/browse/SPARK-31784?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xingbo Jiang updated SPARK-31784: - Affects Version/s: (was: 3.1.0) 3.0.0 > Fix test

[jira] [Resolved] (SPARK-31784) Fix test BarrierTaskContextSuite."share messages with allGather() call"

[ https://issues.apache.org/jira/browse/SPARK-31784?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xingbo Jiang resolved SPARK-31784. -- Fix Version/s: 3.0.0 Target Version/s: 3.0.0 Assignee: wuyi

[jira] [Commented] (SPARK-31757) Improve HistoryServerDiskManager.updateAccessTime()

[ https://issues.apache.org/jira/browse/SPARK-31757?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17113794#comment-17113794 ] Xingbo Jiang commented on SPARK-31757: -- Fixed by https://github.com/apache/spark/pull/28579 >

[jira] [Resolved] (SPARK-31757) Improve HistoryServerDiskManager.updateAccessTime()

[ https://issues.apache.org/jira/browse/SPARK-31757?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xingbo Jiang resolved SPARK-31757. -- Assignee: Xingbo Jiang Resolution: Fixed > Improve

[jira] [Created] (SPARK-31757) Improve HistoryServerDiskManager.updateAccessTime()

Xingbo Jiang created SPARK-31757: Summary: Improve HistoryServerDiskManager.updateAccessTime() Key: SPARK-31757 URL: https://issues.apache.org/jira/browse/SPARK-31757 Project: Spark Issue

[jira] [Updated] (SPARK-31651) Improve handling the case where different barrier sync types in a single sync

[ https://issues.apache.org/jira/browse/SPARK-31651?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xingbo Jiang updated SPARK-31651: - Issue Type: Bug (was: Improvement) > Improve handling the case where different barrier sync

[jira] [Resolved] (SPARK-31651) Improve handling the case where different barrier sync types in a single sync

[ https://issues.apache.org/jira/browse/SPARK-31651?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xingbo Jiang resolved SPARK-31651. -- Fix Version/s: 3.0.0 Resolution: Fixed > Improve handling the case where different

[jira] [Assigned] (SPARK-31651) Improve handling the case where different barrier sync types in a single sync

[ https://issues.apache.org/jira/browse/SPARK-31651?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xingbo Jiang reassigned SPARK-31651: Assignee: wuyi > Improve handling the case where different barrier sync types in a

[jira] [Commented] (SPARK-31651) Improve handling the case where different barrier sync types in a single sync

[ https://issues.apache.org/jira/browse/SPARK-31651?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17110917#comment-17110917 ] Xingbo Jiang commented on SPARK-31651: -- Resolved by https://github.com/apache/spark/pull/28462 >

[jira] [Created] (SPARK-31619) Rename config name "spark.dynamicAllocation.shuffleTimeout" to "spark.dynamicAllocation.shuffleTracking.timeout"

Xingbo Jiang created SPARK-31619: Summary: Rename config name "spark.dynamicAllocation.shuffleTimeout" to "spark.dynamicAllocation.shuffleTracking.timeout" Key: SPARK-31619 URL:

[jira] [Resolved] (SPARK-31344) Polish implementation of barrier() and allGather()

[ https://issues.apache.org/jira/browse/SPARK-31344?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xingbo Jiang resolved SPARK-31344. -- Fix Version/s: 3.1.0 Assignee: wuyi Resolution: Fixed Resolved by

[jira] [Commented] (SPARK-31249) Flaky Test: CoarseGrainedSchedulerBackendSuite.custom log url for Spark UI is applied

[ https://issues.apache.org/jira/browse/SPARK-31249?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17066939#comment-17066939 ] Xingbo Jiang commented on SPARK-31249: -- I can't reproduce this failure, maybe the ExecutorAdded

[jira] [Created] (SPARK-31239) Fix flaky test: WorkerDecommissionSuite.verify a task with all workers decommissioned succeeds

Xingbo Jiang created SPARK-31239: Summary: Fix flaky test: WorkerDecommissionSuite.verify a task with all workers decommissioned succeeds Key: SPARK-31239 URL: https://issues.apache.org/jira/browse/SPARK-31239

[jira] [Created] (SPARK-31207) Ensure the total number of blocks to fetch equals to the sum of local/hostLocal/remote blocks

Xingbo Jiang created SPARK-31207: Summary: Ensure the total number of blocks to fetch equals to the sum of local/hostLocal/remote blocks Key: SPARK-31207 URL: https://issues.apache.org/jira/browse/SPARK-31207

[jira] [Resolved] (SPARK-30667) Support simple all gather in barrier task context

[ https://issues.apache.org/jira/browse/SPARK-30667?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xingbo Jiang resolved SPARK-30667. -- Resolution: Done > Support simple all gather in barrier task context >

[jira] [Resolved] (SPARK-31052) Fix flaky test "DAGSchedulerSuite.shuffle fetch failed on speculative task, but original task succeed"

[ https://issues.apache.org/jira/browse/SPARK-31052?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xingbo Jiang resolved SPARK-31052. -- Fix Version/s: 3.0.0 Target Version/s: 3.0.0 Assignee: wuyi

[jira] [Updated] (SPARK-31052) Fix flaky test "DAGSchedulerSuite.shuffle fetch failed on speculative task, but original task succeed"

[ https://issues.apache.org/jira/browse/SPARK-31052?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xingbo Jiang updated SPARK-31052: - Summary: Fix flaky test "DAGSchedulerSuite.shuffle fetch failed on speculative task, but

[jira] [Resolved] (SPARK-30969) Remove resource coordination support from Standalone

[ https://issues.apache.org/jira/browse/SPARK-30969?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xingbo Jiang resolved SPARK-30969. -- Fix Version/s: 3.0.0 Resolution: Fixed Fixed by

[jira] [Resolved] (SPARK-30987) ResourceDiscoveryPluginSuite sometimes fails

[ https://issues.apache.org/jira/browse/SPARK-30987?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xingbo Jiang resolved SPARK-30987. -- Fix Version/s: 3.0.0 Target Version/s: 3.0.0 Resolution: Fixed Resolved by

[jira] [Assigned] (SPARK-30987) ResourceDiscoveryPluginSuite sometimes fails

[ https://issues.apache.org/jira/browse/SPARK-30987?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xingbo Jiang reassigned SPARK-30987: Assignee: Thomas Graves > ResourceDiscoveryPluginSuite sometimes fails >

[jira] [Resolved] (SPARK-30947) Log better message when accelerate resource is empty

[ https://issues.apache.org/jira/browse/SPARK-30947?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xingbo Jiang resolved SPARK-30947. -- Fix Version/s: 3.1.0 Resolution: Fixed Resolved by

[jira] [Assigned] (SPARK-30947) Log better message when accelerate resource is empty

[ https://issues.apache.org/jira/browse/SPARK-30947?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xingbo Jiang reassigned SPARK-30947: Assignee: wuyi > Log better message when accelerate resource is empty >

[jira] [Commented] (SPARK-30969) Remove resource coordination support from Standalone

[ https://issues.apache.org/jira/browse/SPARK-30969?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17046929#comment-17046929 ] Xingbo Jiang commented on SPARK-30969: -- I created https://issues.apache.org/jira/browse/SPARK-30978

[jira] [Updated] (SPARK-30978) Remove multiple workers on the same host support from Standalone backend

[ https://issues.apache.org/jira/browse/SPARK-30978?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xingbo Jiang updated SPARK-30978: - Description: Based on our experience, there is no scenario that necessarily requires deploying

[jira] [Created] (SPARK-30978) Remove multiple workers on the same host support from Standalone backend

Xingbo Jiang created SPARK-30978: Summary: Remove multiple workers on the same host support from Standalone backend Key: SPARK-30978 URL: https://issues.apache.org/jira/browse/SPARK-30978 Project:

[jira] [Updated] (SPARK-29499) Add mapPartitionsWithIndex for RDDBarrier

[ https://issues.apache.org/jira/browse/SPARK-29499?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xingbo Jiang updated SPARK-29499: - Issue Type: Improvement (was: Bug) > Add mapPartitionsWithIndex for RDDBarrier >

[jira] [Commented] (SPARK-29499) Add mapPartitionsWithIndex for RDDBarrier

[ https://issues.apache.org/jira/browse/SPARK-29499?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17021560#comment-17021560 ] Xingbo Jiang commented on SPARK-29499: -- [~dongjoon] Yes, this is master only, since it's a new API

[jira] [Resolved] (SPARK-30417) SPARK-29976 calculation of slots wrong for Standalone Mode

[ https://issues.apache.org/jira/browse/SPARK-30417?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xingbo Jiang resolved SPARK-30417. -- Fix Version/s: 3.0.0 Assignee: Yuchen Huo Resolution: Fixed Resolved by

[jira] [Commented] (SPARK-30417) SPARK-29976 calculation of slots wrong for Standalone Mode

[ https://issues.apache.org/jira/browse/SPARK-30417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17010156#comment-17010156 ] Xingbo Jiang commented on SPARK-30417: -- Good catch! `max(conf.get(EXECUTOR_CORES) /

[jira] [Commented] (SPARK-27558) NPE in TaskCompletionListener due to Spark OOM in UnsafeExternalSorter causing tasks to hang

[ https://issues.apache.org/jira/browse/SPARK-27558?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16977154#comment-16977154 ] Xingbo Jiang commented on SPARK-27558: -- Trying to clarify the behavior: did the task failed, or did

[jira] [Resolved] (SPARK-27558) NPE in TaskCompletionListener due to Spark OOM in UnsafeExternalSorter causing tasks to hang

[ https://issues.apache.org/jira/browse/SPARK-27558?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xingbo Jiang resolved SPARK-27558. -- Assignee: Artsiom Yudovin Resolution: Fixed Resolved by

[jira] [Resolved] (SPARK-29287) Executors should not receive any offers before they are actually constructed

[ https://issues.apache.org/jira/browse/SPARK-29287?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xingbo Jiang resolved SPARK-29287. -- Fix Version/s: 3.0.0 Assignee: Kent Yao Resolution: Done Fixed by

[jira] [Created] (SPARK-29666) Release script fail to publish release under dry run mode

Xingbo Jiang created SPARK-29666: Summary: Release script fail to publish release under dry run mode Key: SPARK-29666 URL: https://issues.apache.org/jira/browse/SPARK-29666 Project: Spark

[jira] [Updated] (SPARK-29666) Release script fail to publish release under dry run mode

[ https://issues.apache.org/jira/browse/SPARK-29666?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xingbo Jiang updated SPARK-29666: - Description: `release-build.sh` fail to publish release under dry run mode with the following

[jira] [Created] (SPARK-29646) Allow pyspark version name format `${versionNumber}-preview` in release script

Xingbo Jiang created SPARK-29646: Summary: Allow pyspark version name format `${versionNumber}-preview` in release script Key: SPARK-29646 URL: https://issues.apache.org/jira/browse/SPARK-29646

[jira] [Commented] (SPARK-29569) doc build fails with `/api/scala/lib/jquery.js` doesn't exist

[ https://issues.apache.org/jira/browse/SPARK-29569?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16958166#comment-16958166 ] Xingbo Jiang commented on SPARK-29569: -- [~dongjoon] Not yet, the release script is still failing,

[jira] [Resolved] (SPARK-29499) Add mapPartitionsWithIndex for RDDBarrier

[ https://issues.apache.org/jira/browse/SPARK-29499?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xingbo Jiang resolved SPARK-29499. -- Assignee: Xianyang Liu Resolution: Fixed > Add mapPartitionsWithIndex for RDDBarrier >

[jira] [Commented] (SPARK-29569) doc build fails with `/api/scala/lib/jquery.js` doesn't exist

[ https://issues.apache.org/jira/browse/SPARK-29569?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16957731#comment-16957731 ] Xingbo Jiang commented on SPARK-29569: -- [~sowen][~dongjoon] Can you take a look at this issue? >

[jira] [Updated] (SPARK-29569) doc build fails with `/api/scala/lib/jquery.js` doesn't exist

[ https://issues.apache.org/jira/browse/SPARK-29569?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xingbo Jiang updated SPARK-29569: - Summary: doc build fails with `/api/scala/lib/jquery.js` doesn't exist (was: doc build fails

[jira] [Created] (SPARK-29569) doc build fails because `/api/scala/lib/jquery.js` doesn't exist

Xingbo Jiang created SPARK-29569: Summary: doc build fails because `/api/scala/lib/jquery.js` doesn't exist Key: SPARK-29569 URL: https://issues.apache.org/jira/browse/SPARK-29569 Project: Spark

[jira] [Resolved] (SPARK-26217) Compliance to SQL standard (SQL:2011)

[ https://issues.apache.org/jira/browse/SPARK-26217?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xingbo Jiang resolved SPARK-26217. -- Resolution: Duplicate This issue is part of SPARK-27764, I'll move all the sub-tasks and mark

[jira] [Updated] (SPARK-26215) define reserved keywords after SQL standard

[ https://issues.apache.org/jira/browse/SPARK-26215?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xingbo Jiang updated SPARK-26215: - Parent: SPARK-27764 (was: SPARK-26217) > define reserved keywords after SQL standard >

[jira] [Updated] (SPARK-26218) Throw exception on overflow for integers

[ https://issues.apache.org/jira/browse/SPARK-26218?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xingbo Jiang updated SPARK-26218: - Parent: SPARK-27764 (was: SPARK-26217) > Throw exception on overflow for integers >

[jira] [Updated] (SPARK-27149) Support LOCALTIMESTAMP when ANSI mode enabled

[ https://issues.apache.org/jira/browse/SPARK-27149?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xingbo Jiang updated SPARK-27149: - Parent: SPARK-27764 (was: SPARK-26217) > Support LOCALTIMESTAMP when ANSI mode enabled >

[jira] [Updated] (SPARK-27148) Support CURRENT_TIME and LOCALTIME when ANSI mode enabled

[ https://issues.apache.org/jira/browse/SPARK-27148?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xingbo Jiang updated SPARK-27148: - Parent: SPARK-27764 (was: SPARK-26217) > Support CURRENT_TIME and LOCALTIME when ANSI mode

[jira] [Updated] (SPARK-29096) The exact math method should be called only when there is a corresponding function in Math

[ https://issues.apache.org/jira/browse/SPARK-29096?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xingbo Jiang updated SPARK-29096: - Parent: SPARK-27764 (was: SPARK-26217) > The exact math method should be called only when

[jira] [Updated] (SPARK-23179) Support option to throw exception if overflow occurs during Decimal arithmetic

[ https://issues.apache.org/jira/browse/SPARK-23179?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xingbo Jiang updated SPARK-23179: - Parent: SPARK-27764 (was: SPARK-26217) > Support option to throw exception if overflow occurs

[jira] [Updated] (SPARK-26976) Forbid reserved keywords as identifiers when ANSI mode is on

[ https://issues.apache.org/jira/browse/SPARK-26976?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xingbo Jiang updated SPARK-26976: - Parent: SPARK-27764 (was: SPARK-26217) > Forbid reserved keywords as identifiers when ANSI

[jira] [Updated] (SPARK-23264) Support interval values without INTERVAL clauses

[ https://issues.apache.org/jira/browse/SPARK-23264?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xingbo Jiang updated SPARK-23264: - Parent: SPARK-27764 (was: SPARK-26217) > Support interval values without INTERVAL clauses >

[jira] [Assigned] (SPARK-28583) Subqueries should not call `onUpdatePlan` in Adaptive Query Execution

[ https://issues.apache.org/jira/browse/SPARK-28583?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xingbo Jiang reassigned SPARK-28583: Assignee: Wei Xue (was: Xingbo Jiang) > Subqueries should not call `onUpdatePlan` in

[jira] [Assigned] (SPARK-28583) Subqueries should not call `onUpdatePlan` in Adaptive Query Execution

[ https://issues.apache.org/jira/browse/SPARK-28583?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xingbo Jiang reassigned SPARK-28583: Assignee: Xingbo Jiang (was: Wei Xue) > Subqueries should not call `onUpdatePlan` in

[jira] [Comment Edited] (SPARK-26431) Update availableSlots by availableCpus for barrier taskset

[ https://issues.apache.org/jira/browse/SPARK-26431?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16939667#comment-16939667 ] Xingbo Jiang edited comment on SPARK-26431 at 9/27/19 6:39 PM: --- Resolved

[jira] [Comment Edited] (SPARK-29263) availableSlots in scheduler can change before being checked by barrier taskset

[ https://issues.apache.org/jira/browse/SPARK-29263?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16939670#comment-16939670 ] Xingbo Jiang edited comment on SPARK-29263 at 9/27/19 6:38 PM: --- Resolved

[jira] [Updated] (SPARK-29263) availableSlots in scheduler can change before being checked by barrier taskset

[ https://issues.apache.org/jira/browse/SPARK-29263?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xingbo Jiang updated SPARK-29263: - Target Version/s: (was: 2.4.0) > availableSlots in scheduler can change before being checked

[jira] [Updated] (SPARK-29263) availableSlots in scheduler can change before being checked by barrier taskset

[ https://issues.apache.org/jira/browse/SPARK-29263?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xingbo Jiang updated SPARK-29263: - Affects Version/s: (was: 3.0.0) 2.4.0 > availableSlots in scheduler

[jira] [Resolved] (SPARK-29263) availableSlots in scheduler can change before being checked by barrier taskset

[ https://issues.apache.org/jira/browse/SPARK-29263?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xingbo Jiang resolved SPARK-29263. -- Fix Version/s: 3.0.0 Target Version/s: 2.4.0 Assignee: Juliusz Sompolski

[jira] [Resolved] (SPARK-26431) Update availableSlots by availableCpus for barrier taskset

[ https://issues.apache.org/jira/browse/SPARK-26431?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xingbo Jiang resolved SPARK-26431. -- Fix Version/s: 3.0.0 Assignee: Juliusz Sompolski Resolution: Fixed Resolved

[jira] [Updated] (SPARK-29001) Print better log when process of events becomes slow

[ https://issues.apache.org/jira/browse/SPARK-29001?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xingbo Jiang updated SPARK-29001: - Description: We shall print better log when process of events becomes slow, to help find out

[jira] [Updated] (SPARK-29001) Print better log when process of events becomes slow

[ https://issues.apache.org/jira/browse/SPARK-29001?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xingbo Jiang updated SPARK-29001: - Summary: Print better log when process of events becomes slow (was: Print event thread stack

[jira] [Created] (SPARK-29001) Print event thread stack trace when EventQueue starts to drop events

Xingbo Jiang created SPARK-29001: Summary: Print event thread stack trace when EventQueue starts to drop events Key: SPARK-29001 URL: https://issues.apache.org/jira/browse/SPARK-29001 Project: Spark

[jira] [Resolved] (SPARK-28561) DAG viz for barrier-execution mode

[ https://issues.apache.org/jira/browse/SPARK-28561?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xingbo Jiang resolved SPARK-28561. -- Resolution: Fixed > DAG viz for barrier-execution mode > -- >

[jira] [Created] (SPARK-28706) Allow upcast from null type to any types

Xingbo Jiang created SPARK-28706: Summary: Allow upcast from null type to any types Key: SPARK-28706 URL: https://issues.apache.org/jira/browse/SPARK-28706 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-27495) SPIP: Support Stage level resource configuration and scheduling

[ https://issues.apache.org/jira/browse/SPARK-27495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16901475#comment-16901475 ] Xingbo Jiang commented on SPARK-27495: -- Just my two cents: 1. I would treat the resource

[jira] [Resolved] (SPARK-27369) Standalone worker can load resource conf and discover resources

[ https://issues.apache.org/jira/browse/SPARK-27369?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xingbo Jiang resolved SPARK-27369. -- Resolution: Done > Standalone worker can load resource conf and discover resources >

[jira] [Created] (SPARK-28115) Fix flaky test: SparkContextSuite.test resource scheduling under local-cluster mode

Xingbo Jiang created SPARK-28115: Summary: Fix flaky test: SparkContextSuite.test resource scheduling under local-cluster mode Key: SPARK-28115 URL: https://issues.apache.org/jira/browse/SPARK-28115

[jira] [Resolved] (SPARK-27823) Add an abstraction layer for accelerator resource handling to avoid manipulating raw confs

[ https://issues.apache.org/jira/browse/SPARK-27823?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xingbo Jiang resolved SPARK-27823. -- Resolution: Fixed > Add an abstraction layer for accelerator resource handling to avoid >

[jira] [Resolved] (SPARK-21136) Misleading error message for typo in SQL

[ https://issues.apache.org/jira/browse/SPARK-21136?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xingbo Jiang resolved SPARK-21136. -- Resolution: Fixed > Misleading error message for typo in SQL >

[jira] [Created] (SPARK-27948) GPU Scheduling - Use ResouceName to represent resource names

Xingbo Jiang created SPARK-27948: Summary: GPU Scheduling - Use ResouceName to represent resource names Key: SPARK-27948 URL: https://issues.apache.org/jira/browse/SPARK-27948 Project: Spark

[jira] [Assigned] (SPARK-27770) Add aggregates.sql - Part1

[ https://issues.apache.org/jira/browse/SPARK-27770?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xingbo Jiang reassigned SPARK-27770: Assignee: Xiao Li > Add aggregates.sql - Part1 > -- > >

[jira] [Resolved] (SPARK-27770) Add aggregates.sql - Part1

[ https://issues.apache.org/jira/browse/SPARK-27770?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xingbo Jiang resolved SPARK-27770. -- Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 24640

[jira] [Commented] (SPARK-21492) Memory leak in SortMergeJoin

[ https://issues.apache.org/jira/browse/SPARK-21492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16844313#comment-16844313 ] Xingbo Jiang commented on SPARK-21492: -- I'm working on this, will post a simple design doc in a few

[jira] [Commented] (SPARK-27520) Introduce a global config system to replace hadoopConfiguration

[ https://issues.apache.org/jira/browse/SPARK-27520?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16836750#comment-16836750 ] Xingbo Jiang commented on SPARK-27520: -- The major problem of `SparkContext.hadoopConfiguration` is

  1   2   >