Jose Torres created SPARK-24236:
-----------------------------------
Summary: continuous replacement for ShuffleExchangeExec
Key: SPARK-24236
URL: https://issues.apache.org/jira/browse/SPARK-24236
Project: Spark
Issue Type: Sub-task
Components: Structured Streaming
Affects Versions: 2.4.0
Reporter: Jose Torres
Two major differences from ShuffleExchangeExec:
* The post-shuffle RDD should contain all the tasks, rather than reporting
scheduler-level dependencies.
* The operator will have to manually register the continuous shuffle
dependencies, since they're not reported to the scheduler.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
