Quanlong Huang created IMPALA-10265:
---------------------------------------
Summary: Doc about enable_incremental_metadata_updates flag
Key: IMPALA-10265
URL: https://issues.apache.org/jira/browse/IMPALA-10265
Project: IMPALA
Issue Type: Documentation
Reporter: Quanlong Huang
IMPALA-10113 adds a feature flag to turn off the incremental metadata update
feature which defaults to be turned on. This flag decides how catalogd
propagates metadata updates to the catalog topic.
If enable_incremental_metadata_updates is true, catalogd will send metadata
updates in partition granularity. So a table that just has one partition
changed will only have an update on that partition. This reduces the size of
the metadata that needs to be sent from the catalogd.
If enable_incremental_metadata_updates is false, that's the legacy behavior.
Catalogd will send metadata updates in table granularity. So a table that just
has one partition changed will still have an update for the whole table object.
Catalogd still sends the whole table thrift object to the catalog topic.
Note that this is a catalogd-only flag. Don't need to be set on impalads since
impalad can process both incremental and non-incremental catalog updates.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
