[
https://issues.apache.org/jira/browse/HIVE-788?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12747151#action_12747151
]
Zheng Shao commented on HIVE-788:
---------------------------------
For A, we can add a command to Hive to return a partition with partition keys
greater than a given partition. Something like "wait for next partition of
table xxx [greater than ds=yyy, ts=zzz]". This command will return the first
partition after the given partition. If the wait command is implemented
natively using hive, the cost of each probe will be just a SQL query on the
metastore (so the probe interval can be something like one minute or less).
Once we get the new partition, the client can decide what to do. This can avoid
hard questions like B.1, B.2, B.3.
> Triggers when a new partition is created for a table
> ----------------------------------------------------
>
> Key: HIVE-788
> URL: https://issues.apache.org/jira/browse/HIVE-788
> Project: Hadoop Hive
> Issue Type: New Feature
> Reporter: Zheng Shao
>
> One requirement for HIVE-787 is that users would like to run a command
> whenever a new partition of a Hive table gets created.
> There are several ways to achieve this functionality:
> A. Probe and wait: We can have the scripts running in a loop checking if a
> new partition is created.
> Pros: easy to write, easy to control
> Cons: will introduce another delay based on the probing interval.
> B. Triggered: The command is registered inside the hive metastore. Whenever a
> partition gets created, we run the registered command.
> Several questions around option B are:
> 1. whether to support registration of HiveQL or shell command;
> 2. which machine/environment to run the command;
> 3. what to do if the registered command failed.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
