dzamo commented on pull request #2327: URL: https://github.com/apache/drill/pull/2327#issuecomment-949391299
> @dzamo Per your request, I thought about this some more and added the ability to configure the batch size for the `INSERT` queries. What happens now is that the user can set the batch size depending on their environment and the database to which they are inserting data. @cgivre this is great. I thought of one more possible optimisation: creating a parameterised INSERT PreparedStatement of writer_batch_size rows and reusing it for as long as there are >= writer_batch_size rows remaining to insert. I don't know Calcite stuff but I can say I saw a class called SqlDynamicParam in it. This would mean that the receiving DBMS does not need to parse a very long INSERT statement at the start of every batch, a noticeable saving of memory and CPU time for it I would guess. Just a possible optimisation I wanted to share, I view it as something that can also come in a later version. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
