[
https://issues.apache.org/jira/browse/SPARK-24541?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16758673#comment-16758673
]
Jungtaek Lim commented on SPARK-24541:
--------------------------------------
Same understanding here: while I think there's pretty less chance for us to
want to deal with lower level than Netty, but we may also want to send amount
of data close to the size of data structure.
Btw, I don't know which thing Spark leverages to send (pull) shuffle data:
whichever, it would be OK to also leverage it in here because it should be
enough considered as a perspective of performance, security, etc.
> TCP based shuffle
> -----------------
>
> Key: SPARK-24541
> URL: https://issues.apache.org/jira/browse/SPARK-24541
> Project: Spark
> Issue Type: Sub-task
> Components: Structured Streaming
> Affects Versions: 2.4.0
> Reporter: Jose Torres
> Priority: Major
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
