[
https://issues.apache.org/jira/browse/HIVE-23018?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17071220#comment-17071220
]
Vihang Karajgaonkar commented on HIVE-23018:
--------------------------------------------
Thanks for taking a look [~ychena]. the boolean {{isSuccessful}} is a field in
the top-level request object and hence I thought it can be shared for all the
insert events. Currently, I don't see any usage in Hive (or Spark which reuses
the Hive.java code) to send a request which has the successful flag set to
false. In case of Impala also, we don't see a need to fire an event which is
not successful in the first place. In any case if there is a client who wishes
to make a bulk call to fire multiple insert events which has a mix of
successful insert events and unsuccessful events then they can make 2 RPC calls
one for a successful batch and other for the unsuccessful batch. Currently, I
don't see a reason to duplicate the boolean flag for all the insert events in
the request which unnecessarily increases the payload of the call. What do you
think?
> Provide a bulk API to fire multiple insert events
> -------------------------------------------------
>
> Key: HIVE-23018
> URL: https://issues.apache.org/jira/browse/HIVE-23018
> Project: Hive
> Issue Type: Improvement
> Reporter: Vihang Karajgaonkar
> Assignee: Vihang Karajgaonkar
> Priority: Major
> Attachments: HIVE-23018.01.patch, HIVE-23018.02.patch,
> HIVE-23018.03.patch, HIVE-23018.04.patch
>
>
> Metastore provides a API to fire a listener event (currently only supports
> INSERT event). The problem with that API is that it only takes in one
> partition at a time. A typical query may insert data into multiple partitions
> at a time. In such a case query engines like HS2 or Impala will have to issue
> multiple RPCs to metastore sequentially to fire these events. This can show
> up as a slowdown to the user if the query engines do not return the prompt to
> the user until all the events are fired (In case of HS2 and Impala). It would
> be great if we have bulk API which takes in multiple partitions for a table
> so that metastore can generate many such events in one RPC.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
