kalyan kumar kalvagadda commented on HIVE-18942:

[~akolb] That may not be the case.

AddPartitionEvent that is generated here is different from the 
NotificationEvent that is inserted in the NOTIFICATION_LOG table.

  public void onAlterTable(AlterTableEvent tableEvent) throws MetaException {
    Table before = tableEvent.getOldTable();
    Table after = tableEvent.getNewTable();
    NotificationEvent event =
        new NotificationEvent(0, now(), EventType.ALTER_TABLE.toString(), 
            .buildAlterTableMessage(before, after, 
    process(event, tableEvent);
This is how later table JSON is constructed as sentry is interested in below 
things in the alter table notification
# Old Db Name
# New Db Name
# Old Table Name
# New Table Name
# Old Location
# New Location

  public JSONAlterTableMessage(String server, String servicePrincipal, Table 
tableObjBefore, Table tableObjAfter, Long timestamp) {
    this.server = server;
    this.servicePrincipal = servicePrincipal;
    this.db = tableObjBefore.getDbName();
    this.table = tableObjBefore.getTableName();
    this.timestamp = timestamp;

    try {
      this.tableObjBeforeJson = 
      this.tableObjAfterJson = 
    } catch (TException var7) {
      throw new IllegalArgumentException("Could not serialize: ", var7);


I think the NotificationEvent inserted in NOTIFICATION_LOG table may not have 
details of the partitions. 

> ALTER TABLE may generate huge event (with all partitions)
> ---------------------------------------------------------
>                 Key: HIVE-18942
>                 URL: https://issues.apache.org/jira/browse/HIVE-18942
>             Project: Hive
>          Issue Type: Bug
>          Components: Metastore
>    Affects Versions: 3.0.0
>            Reporter: Alexander Kolbasov
>            Priority: Major
> ALTER TABLE handler in HiveAlterHandler has this code:
> {code:java}
> if (isPartitionedTable) {
>   parts = msdb.getPartitions(newt.getDbName(), newt.getTableName(), -1);
>   MetaStoreListenerNotifier.notifyEvent(transactionalListeners,
>           EventMessage.EventType.ADD_PARTITION,
>           new AddPartitionEvent(newt, parts, true, handler),
>           environmentContext);
> }{code}
> The problem is that table may contain huge number of partitions and the event 
> will contain all of them. Partition object itself isn't very small either, so 
> we may end up with huge events which would be stored and then transmitted 
> over the wire to consumers.
> [~spena] [~kkalyan] [~lina.li] [~vaidyand] FYI.

This message was sent by Atlassian JIRA

Reply via email to