Vihang Karajgaonkar has uploaded a new patch set (#2). (
http://gerrit.cloudera.org:8080/12365 )
Change subject: IMPALA-7976 : Add a flag to disable sync using events at a
table level
......................................................................
IMPALA-7976 : Add a flag to disable sync using events at a table level
This change adds the ability to disable event processing at a table or
database level. If the user adds a property with the key
"impala.disableHmsSync" in the table or database properties, event
processing can be turned on/off at the respective level. The value of
the property is string value of Booleans ("true" or "false"). By default
event processing is enabled when the global config
hms_event_polling_interval_s is set to a non-zero value. The value of
property determines if the event processing needs to be disabled for a
particular table or database. So for example, if
"impala.disableHmsSync" is set to "true" then events for that
table/database are ignored. If the property is unset or a value of
"false" is used, the event processing is enabled (although it is not
necessary to explicitly enable event processing, given the global flag).
When both table and db level properties are set, the table level
property takes precedence. If the table level property is not set, then
database level property is used to evaluate if the event needs to be
processed or not.
Users can issue alter table command to change the value of this
property. When the flag value is changed, the alter table event
generated is always processed and subsequent events are either ignored
or processed based on the new value of table flag and database flag. In
theory, a database level property can also be changed. But Impala
doesn't currently support alter database operations. Users who may wish
to change the database level property should use alter database command
from Apache Hive (or any other metastore client)
Testing done: Added new unit tests which execute all supported DDL
operations with all possible combination of db and table flags. Another
test executes alter table operations for changing flags for all combinations
of transitions at table level.
Change-Id: If116edba51bae139bc003be858b3214c3f0104e3
---
M fe/src/main/java/org/apache/impala/catalog/CatalogServiceCatalog.java
M fe/src/main/java/org/apache/impala/catalog/events/MetastoreEventUtils.java
M
fe/src/main/java/org/apache/impala/catalog/events/MetastoreEventsProcessor.java
M
fe/src/test/java/org/apache/impala/catalog/events/MetastoreEventsProcessorTest.java
4 files changed, 696 insertions(+), 72 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/65/12365/2
--
To view, visit http://gerrit.cloudera.org:8080/12365
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: If116edba51bae139bc003be858b3214c3f0104e3
Gerrit-Change-Number: 12365
Gerrit-PatchSet: 2
Gerrit-Owner: Vihang Karajgaonkar <vih...@cloudera.com>
Gerrit-Reviewer: Anonymous Coward (453)
Gerrit-Reviewer: Bharath Vissapragada <bhara...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <impala-public-jenk...@cloudera.com>
Gerrit-Reviewer: Paul Rogers <prog...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <tarmstr...@cloudera.com>
Gerrit-Reviewer: Vihang Karajgaonkar <vih...@cloudera.com>
