[
https://issues.apache.org/jira/browse/HCATALOG-546?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mithun Radhakrishnan updated HCATALOG-546:
------------------------------------------
Attachment: HCATALOG-546.trunk.patch
Apologies for the delay, all. I had trouble porting the patch over to trunk,
but I think I'm done now.
I've had to make modifications to the build.xmls, to use the hcatalog-core
classes from core/build/classes (instead of the jars pulled from maven).
Without this, newly added symbols simply won't be resolved. (I'd really
appreciate if you'd review that bit. I'd like to be sure that's alright.)
> Rework HCatalog's JMS Notifications
> ------------------------------------
>
> Key: HCATALOG-546
> URL: https://issues.apache.org/jira/browse/HCATALOG-546
> Project: HCatalog
> Issue Type: Bug
> Components: notification
> Affects Versions: 0.4.1
> Reporter: Mithun Radhakrishnan
> Assignee: Mithun Radhakrishnan
> Fix For: 0.4.1
>
> Attachments: HCATALOG-546.patch, HCATALOG-546.trunk.patch,
> sample.Add.Drop.Database.json, sample.Add.Drop.Partition.json,
> sample.Add.Drop.Table.json
>
>
> In 0.4.1, the NotificationListener listens for metastore operations and emits
> JMS notifications containing the entire metastore-objects
> (Database/Table/Partitions) in Java-serialized form. The assumption at the
> time was that consumers might need access to the whole object. This policy
> poses a couple of problems:
> 1. The notifications are verbose, since it conveys a bunch of information
> that's available from querying the metastore anyway.
> 2. Consumers of these JMS notifications (e.g. Oozie) would now be dependent
> on the Java class definitions of metastore-objects. If they change, Oozie
> would also need to be restarted (with updated libs), to consume the
> notifications.
> Ideally, the notifications should convey only the minimum information that
> identifies the metastore-change unambiguously. (Everything else can be
> queried for.) They should be backward compatible. If new fields are added,
> existing consumers shouldn't break (unless they intend to consume the new
> fields). Also, the notification-format ought to be pluggable.
> For the initial rework, we're proposing to use a JSON-string to represent the
> notification-content. We're also proposing a helper-class for the likes of
> Oozie to use, that converts the strings to POJOs, in a backward-compatible
> fashion.
> I'll attach sample notifications and a tentative patch shortly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
