Todd Lipcon created IMPALA-7457:
-----------------------------------
Summary: Allow StateStore subscribers to filter keys by a prefix
Key: IMPALA-7457
URL: https://issues.apache.org/jira/browse/IMPALA-7457
Project: IMPALA
Issue Type: Improvement
Components: Backend, Catalog
Reporter: Todd Lipcon
Assignee: Todd Lipcon
StateStore subscriptions are currently all-or-nothing. However, in some cases
it might be useful to filter a subscription. Two use cases come to mind in the
near term:
1) currently the catalogd publishes objects into a topic. In order to publish,
it has to subscribe to that topic. That means that the statestore then
"reflects" all of the info right back to the catalogd causing extra network and
memory pressure, even though the catalogd ignores the updates.
2) in order to implement cache invalidation on "fetch-on-demand" impalads, we
want to publish version numbers of catalog objects without sending the whole
object. In order to keep SYNC_DDL working, it's much more convenient to publish
the version numbers on the same topic as the full objects. However, the
"fetch-on-demand" impalads shouldn't waste bandwidth from the statestore with
all of the detailed object info.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
