[
https://issues.apache.org/jira/browse/HIVE-20545?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16640211#comment-16640211
]
Bharathkrishna Guruvayoor Murali edited comment on HIVE-20545 at 10/5/18 6:51
PM:
----------------------------------------------------------------------------------
Hi [~anishek] ,
An example would be when Impala writes stats information to Partition objects
and accesses it, as shown :
[Impala
stats|http://github.mtv.cloudera.com/CDH/Impala/blob/6f2d928734a33ace15ec6abd5659651173b9e69e/fe/src/main/java/org/apache/impala/catalog/PartitionStatsUtil.java#L45]
But this information is not relevant as a Partition metadata for any other
purpose. Also, these parameters have considerably large-size. On running some
preliminary tests, we observe that there is a performance hit on increasing
message size of HMS Notifications, so we can use this configuration to filter
parameters like above mentioned.
was (Author: bharos92):
Hi [~anishek] ,
An example would be when Impala writes stats information to Partition objects
and accesses it, as shown : [Impala reading stats
|http://github.mtv.cloudera.com/CDH/Impala/blob/6f2d928734a33ace15ec6abd5659651173b9e69e/fe/src/main/java/org/apache/impala/catalog/PartitionStatsUtil.java#L45]
But this information is not relevant as a Partition metadata for any other
purpose. Also, these parameters have considerably large-size. On running some
preliminary tests, we observe that there is a performance hit on increasing
message size of HMS Notifications, so we can use this configuration to filter
parameters like above mentioned.
> Ability to exclude potentially large parameters in HMS Notifications
> --------------------------------------------------------------------
>
> Key: HIVE-20545
> URL: https://issues.apache.org/jira/browse/HIVE-20545
> Project: Hive
> Issue Type: Improvement
> Components: Metastore
> Affects Versions: 3.1.0, 4.0.0
> Reporter: Bharathkrishna Guruvayoor Murali
> Assignee: Bharathkrishna Guruvayoor Murali
> Priority: Major
> Attachments: HIVE-20545.1.patch, HIVE-20545.2.patch,
> HIVE-20545.3.branch-3.patch, HIVE-20545.3.patch, HIVE-20545.4.patch,
> HIVE-20545.6.patch, HIVE-20545.7.patch
>
>
> Clients can add large-sized parameters in Table/Partition objects. So we need
> to enable adding regex patterns through HiveConf to match parameters to be
> filtered from table and partition objects before serialization in HMS
> notifications.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
