[
https://issues.apache.org/jira/browse/SPARK-24541?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16758482#comment-16758482
]
Imran Rashid commented on SPARK-24541:
--------------------------------------
well, rpc is over tcp, so I'm still not really sure what this means. Is the
point sending raw data directly over sockets? I'd be interested in knowing
what the purpose is. I guess to avoid the overhead associated w/ the extra
headers etc from the rpc framework? And if this is really going to try to use
raw sockets, not through netty, then you'd have to reimplement encryption,
manage your own buffers, etc.
> TCP based shuffle
> -----------------
>
> Key: SPARK-24541
> URL: https://issues.apache.org/jira/browse/SPARK-24541
> Project: Spark
> Issue Type: Sub-task
> Components: Structured Streaming
> Affects Versions: 2.4.0
> Reporter: Jose Torres
> Priority: Major
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
