[
https://issues.apache.org/jira/browse/IGNITE-7736?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrey Gura updated IGNITE-7736:
--------------------------------
Fix Version/s: (was: 2.5)
2.6
> SQL COPY: streaming model for network packets instead of request-response
> model
> -------------------------------------------------------------------------------
>
> Key: IGNITE-7736
> URL: https://issues.apache.org/jira/browse/IGNITE-7736
> Project: Ignite
> Issue Type: Task
> Reporter: Vladimir Ozerov
> Priority: Major
> Fix For: 2.6
>
>
> *Problem*
> Currently data transfer in COPY command is implemented as a series of
> request-responses. When request is received, it is parsed synchronously and
> passed to the streamer, then response is sent. This is not very efficient
> approach:
> # We hardly could utilize long fat network channels efficiently as we spend a
> lot of time waiting for a very small response (ack).
> # Parsing takes and adding data to the streamer takes time (especially if we
> reached streamer buffer limitations and are blocked waiting for responses
> from data nodes). During this period network is not utilized and file data is
> not transferred further.
> *Solution*
> Let's fix the problem iteratively as follows:
> # Introduce asynchrony - when request is received, send the response
> immediately before data processing
> # Then consider sending one ack for several requests instead of sending ack
> for every request
> # When multiple simultaneous requests are enabled (previous point), consider
> moving data processing to separate stream, so that we can read data from the
> socket as fast as possible
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
