[
https://issues.apache.org/jira/browse/KUDU-1563?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17038031#comment-17038031
]
Grant Henke commented on KUDU-1563:
-----------------------------------
There are a bunch of use cases this would improve:
* Because Spark tasks retry most jobs require `UPSERT `operations and the task
retries result in many updates, even when they are not required.
* Spark retries with DELETE operations fail given there is no DELETE_IGNORE.
* The backup job works around this with a session based workaround. See
below for issues with that workaround.
* Client side session based INSERT and DELETE ignore functionality results in a
full error message returned for every row ignored.
* This error is then ignored client side.
* Client side session based INSERT ignore functionality complicates use cases
like the NiFi integration where ignore and non-ignore operations may be mixed
in the same session. Using separate session results in smaller, and more
complicated, flushing of write operations.
> Add support for INSERT/UPDATE/DELETE IGNORE
> -------------------------------------------
>
> Key: KUDU-1563
> URL: https://issues.apache.org/jira/browse/KUDU-1563
> Project: Kudu
> Issue Type: New Feature
> Reporter: Dan Burkert
> Assignee: Grant Henke
> Priority: Major
> Labels: backup, roadmap-candidate
>
> The Java client currently has an [option to ignore duplicate row key errors|
> https://kudu.apache.org/apidocs/org/kududb/client/AsyncKuduSession.html#setIgnoreAllDuplicateRows-boolean-],
> which is implemented by filtering the errors on the client side. If we are
> going to continue to support this feature (and the consensus seems to be that
> we probably should), we should promote it to a first class operation type
> that is handled on the server side. This would have a modest perf.
> improvement since less errors are returned, and it would allow INSERT IGNORE
> ops to be mixed in the same batch as other INSERT, DELETE, UPSERT, etc. ops.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
