Vladimir Ozerov created IGNITE-7736:
---------------------------------------
Summary: SQL COPY: streaming model for network packets instead of
request-response model
Key: IGNITE-7736
URL: https://issues.apache.org/jira/browse/IGNITE-7736
Project: Ignite
Issue Type: Task
Reporter: Vladimir Ozerov
Fix For: 2.5
*Problem*
Currently data transfer in COPY command is implemented as a series of
request-responses. When request is received, it is parsed synchronously and
passed to the streamer, then response is sent. This is not very efficient
approach:
# We hardly could utilize long fat network channels efficiently as we spend a
lot of time waiting for a very small response (ack).
# Parsing takes and adding data to the streamer takes time (especially if we
reached streamer buffer limitations and are blocked waiting for responses from
data nodes). During this period network is not utilized and file data is not
transferred further.
*Solution*
Let's fix the problem iteratively as follows:
# Introduce asynchrony - when request is received, send the response
immediately before data processing
# Then consider sending one ack for several requests instead of sending ack for
every request
# When multiple simultaneous requests are enabled (previous point), consider
moving data processing to separate stream, so that we can read data from the
socket as fast as possible
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
