Vladimir Ozerov created IGNITE-6917:
---------------------------------------
Summary: SQL: implement COP command for efficient data loading
Key: IGNITE-6917
URL: https://issues.apache.org/jira/browse/IGNITE-6917
Project: Ignite
Issue Type: Task
Security Level: Public (Viewable by anyone)
Components: sql
Reporter: Vladimir Ozerov
Fix For: 2.4
Inspired by Postgres [1]
Common use case - bulk data load through JDBC/ODBC interface. Currently it is
only possible to execute single commands one by one. We already can batch them
to improve performance, but there is still big room for improvement.
We should think of a completely new command - {{COPY}}. It will accept a file
(or input stream in general case) on the client side, then transfer data to the
cluster, and then execute update inside the cluster, e.g. through streamer.
First of all we need to create quick and dirty prototype to assess potential
performance improvement. It speedup is confirmed, we should build base
implementation which will accept only files. But at the same time we should
understand how it will evolve in future: multiple file formats (probably
including Hadoop formarts, e.g. Parquet), escape characters, input streams,
etc..
[1] https://www.postgresql.org/docs/9.6/static/sql-copy.html
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
