[
https://issues.apache.org/jira/browse/HCATALOG-546?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mithun Radhakrishnan updated HCATALOG-546:
------------------------------------------
Attachment: sample.Add.Drop.Table.json
sample.Add.Drop.Partition.json
sample.Add.Drop.Database.json
I've attached samples of JSON notifications for the following operations:
1. Create/Drop Database.
2. Create/Drop Tables.
3. Add/Drop Partitions.
Please note that the JSON messages don't contain the version-string, format-id
or event-id. I've opted instead to put those in the JMS message (as separate
fields), since that makes deserialization a lot simpler.
> Rework HCatalog's JMS Notifications
> ------------------------------------
>
> Key: HCATALOG-546
> URL: https://issues.apache.org/jira/browse/HCATALOG-546
> Project: HCatalog
> Issue Type: Bug
> Components: notification
> Affects Versions: 0.4.1
> Reporter: Mithun Radhakrishnan
> Fix For: 0.4.1
>
> Attachments: sample.Add.Drop.Database.json,
> sample.Add.Drop.Partition.json, sample.Add.Drop.Table.json
>
>
> In 0.4.1, the NotificationListener listens for metastore operations and emits
> JMS notifications containing the entire metastore-objects
> (Database/Table/Partitions) in Java-serialized form. The assumption at the
> time was that consumers might need access to the whole object. This policy
> poses a couple of problems:
> 1. The notifications are verbose, since it conveys a bunch of information
> that's available from querying the metastore anyway.
> 2. Consumers of these JMS notifications (e.g. Oozie) would now be dependent
> on the Java class definitions of metastore-objects. If they change, Oozie
> would also need to be restarted (with updated libs), to consume the
> notifications.
> Ideally, the notifications should convey only the minimum information that
> identifies the metastore-change unambiguously. (Everything else can be
> queried for.) They should be backward compatible. If new fields are added,
> existing consumers shouldn't break (unless they intend to consume the new
> fields). Also, the notification-format ought to be pluggable.
> For the initial rework, we're proposing to use a JSON-string to represent the
> notification-content. We're also proposing a helper-class for the likes of
> Oozie to use, that converts the strings to POJOs, in a backward-compatible
> fashion.
> I'll attach sample notifications and a tentative patch shortly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
