[
https://issues.apache.org/jira/browse/IMPALA-12399?focusedWorklogId=878394&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-878394
]
ASF GitHub Bot logged work on IMPALA-12399:
-------------------------------------------
Author: ASF GitHub Bot
Created on: 25/Aug/23 12:01
Start Date: 25/Aug/23 12:01
Worklog Time Spent: 10m
Work Description: VenuReddy2103 opened a new pull request, #58:
URL: https://github.com/apache/impala/pull/58
Notification events like OPEN_TXN are ignored on catalogd
MetastoreEventsProcessor. So, we can pass eventTypeSkipList with OPEN_TXN in
NotificationEventRequest while invoking get_next_notification() to avoid
reading such notification messages from metastore and then ignoring on
catalogd. OPEN_TXN event being more frequent(received even upon describe table
operation from beeline), we can significantly reduce unwanted processing on
both hive metastore and catalogd. Catalogd reads events in batches of
EVENTS_BATCH_SIZE_PER_RPC, skipping such unnecessary events can help catchup
the events faster.
Testing:
- Manually tested
Issue Time Tracking
-------------------
Worklog Id: (was: 878394)
Remaining Estimate: 0h
Time Spent: 10m
> Pass eventTypeSkipList with OPEN_TXN in NotificationEventRequest to avoid
> receiving OPEN_TXN events from HMS
> ------------------------------------------------------------------------------------------------------------
>
> Key: IMPALA-12399
> URL: https://issues.apache.org/jira/browse/IMPALA-12399
> Project: IMPALA
> Issue Type: Improvement
> Components: Catalog
> Reporter: Venugopal Reddy K
> Priority: Major
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Notification events like OPEN_TXN are ignored on catalogd
> {{{}MetastoreEventsProcessor{}}}. So, we can pass eventTypeSkipList with
> OPEN_TXN in NotificationEventRequest while invoking get_next_notification()
> to avoid reading such notification messages from HMS and then ignoring on
> catalogd. OPEN_TXN event being more frequent(received even upon describe
> table operation from beeline), we can significantly reduce unwanted
> processing on both HMS and catalogd. Catalogd reads events in batches of
> EVENTS_BATCH_SIZE_PER_RPC, skipping such unnecessary events can help catchup
> the events faster.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
