[
https://issues.apache.org/jira/browse/SPARK-27817?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hyukjin Kwon updated SPARK-27817:
---------------------------------
Target Version/s: (was: 2.4.3)
> Is it possible to achieve global scheduling optimization by predicting task
> execution times (e.g., training models with historical data using machine
> learning)?
> ----------------------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: SPARK-27817
> URL: https://issues.apache.org/jira/browse/SPARK-27817
> Project: Spark
> Issue Type: New Feature
> Components: Spark Core
> Affects Versions: 2.4.3
> Reporter: shenghuang
> Priority: Blocker
> Labels: patch, performance
>
> For example, by predicting the execution time of ANY level task, 3 seconds
> before the task is about to be completed,Copy the data for the next ANY task
> to the current node ahead of time.Thus reduces the total execution time of
> the application.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
