Hello Quanlong Huang, Zoltan Borok-Nagy, Impala Public Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/19484
to look at the new patch set (#5).
Change subject: IMPALA-11822: Optimize the Refresh/Invalidate event processing
by skipping unnecessary events
......................................................................
IMPALA-11822: Optimize the Refresh/Invalidate event processing by skipping
unnecessary events
Added a new variable 'lastRefreshEventId' in the
catalogD's table/partition object to store the
processed reload event's ID for the corresponding
table/partition. This will be updated frequently
based on refresh or invalidate commands. This
variable can be used in the event processor to
decide whether to process or skip the
reload event by comparing the event id of the event
with the latest event id from the map variable. It
is enough to store the refresh event's event id,
invalidate event anyway flushes out the object from cache.
Testing: Couldn't test this feature in an end-to-end
test because the events would be filtered as self
events. Tested it manually by removing the self event
logic.
Change-Id: I905957683a96c3ea01ab4bf043d6658ce37b7574
---
M fe/src/main/java/org/apache/impala/catalog/CatalogServiceCatalog.java
M fe/src/main/java/org/apache/impala/catalog/HdfsPartition.java
M fe/src/main/java/org/apache/impala/catalog/HdfsTable.java
M fe/src/main/java/org/apache/impala/catalog/Table.java
M fe/src/main/java/org/apache/impala/catalog/events/MetastoreEvents.java
M fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java
6 files changed, 98 insertions(+), 11 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/84/19484/5
--
To view, visit http://gerrit.cloudera.org:8080/19484
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I905957683a96c3ea01ab4bf043d6658ce37b7574
Gerrit-Change-Number: 19484
Gerrit-PatchSet: 5
Gerrit-Owner: Sai Hemanth Gantasala <[email protected]>
Gerrit-Reviewer: Impala Public Jenkins <[email protected]>
Gerrit-Reviewer: Quanlong Huang <[email protected]>
Gerrit-Reviewer: Sai Hemanth Gantasala <[email protected]>
Gerrit-Reviewer: Zoltan Borok-Nagy <[email protected]>
