[
https://issues.apache.org/jira/browse/IMPALA-11731?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
zhi tang updated IMPALA-11731:
------------------------------
Description:
In the minimal mode of the catalog server, only partition related information
needs to be protected by table locks when processing table updates. So I'm
thinking whether we can avoid the blocking of the table lock on the collection
thread in this way:
1. When an update arrives, we first determine which partitions will be affected
by the update after obtaining the table lock;
2. Then the collection thread collects the change information of the partition.
Because we already know that those partitions will change, we do not need to
hold table locks;
We do not expect the collection thread to be blocked because it will block
other irrelevant updates, such as the problem described in IMPALA-6671. It is
necessary to deal with this problem in a more effective way.
> Consider avoiding table locks from blocking the
> catalog-update-gathering-thread when the catalog server's catalog_topic_mode
> is minimal.
> ----------------------------------------------------------------------------------------------------------------------------------------
>
> Key: IMPALA-11731
> URL: https://issues.apache.org/jira/browse/IMPALA-11731
> Project: IMPALA
> Issue Type: Improvement
> Reporter: zhi tang
> Priority: Major
>
> In the minimal mode of the catalog server, only partition related
> information needs to be protected by table locks when processing table
> updates. So I'm thinking whether we can avoid the blocking of the table lock
> on the collection thread in this way:
> 1. When an update arrives, we first determine which partitions will be
> affected by the update after obtaining the table lock;
> 2. Then the collection thread collects the change information of the
> partition. Because we already know that those partitions will change, we do
> not need to hold table locks;
> We do not expect the collection thread to be blocked because it will block
> other irrelevant updates, such as the problem described in IMPALA-6671. It is
> necessary to deal with this problem in a more effective way.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
