[
https://issues.apache.org/jira/browse/PHOENIX-6092?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chinmay Kulkarni updated PHOENIX-6092:
--------------------------------------
Description:
Currently, if a metadata upgrade is in progress (either triggered by an
explicit "EXECUTE UPGRADE" command or by a new client with autoUpgrade
enabled), in-flight DDLs will generally go through and work as expected.
However, if the upgrade happens to fail, we restore the snapshot of
SYSTEM.CATALOG (and with
[PHOENIX-6086|https://issues.apache.org/jira/browse/PHOENIX-6086] even other
SYSTEM tables) to represent its state before the upgrade started. Due to this,
any DDLs issued after the upgrade began are lost.
There are upgrade steps that need to iterate over each table/index/view in the
cluster and multiple steps that need full table scans on SYSTEM.CATALOG and so
this time window where we could potentially lose client DDLs is not negligible
(could be to the order of minutes).
This Jira is to discuss ways to tackle this problem. Perhaps we should use some
sort of write-ahead log to store DDLs issued while the upgrade is in progress
and replay those DDLs in case we need to restore SYSTEM tables from their
snapshot.
was:
Currently, if a metadata upgrade is in progress (either triggered by an
explicit "EXECUTE UPGRADE" command or by a new client with autoUpgrade
enabled), in-flight DDLs will generally go through and work as expected.
However, if the upgrade happens to fail, we restore the snapshot of
SYSTEM.CATALOG (and with
[PHOENIX-6086|https://issues.apache.org/jira/browse/PHOENIX-6086] even other
SYSTEM tables) to represent its state before the upgrade started. Due to this,
any DDLs issued after the upgrade began are lost.
This Jira is to discuss ways to tackle this problem. Perhaps we should use some
sort of write-ahead log to store DDLs issued while the upgrade is in progress
and replay those DDLs in case we need to restore SYSTEM tables from their
snapshot.
> Queue DDL requests issued while metadata upgrade is in progress and replay on
> upgrade failure
> ---------------------------------------------------------------------------------------------
>
> Key: PHOENIX-6092
> URL: https://issues.apache.org/jira/browse/PHOENIX-6092
> Project: Phoenix
> Issue Type: Improvement
> Affects Versions: 5.0.0, 4.15.0
> Reporter: Chinmay Kulkarni
> Assignee: Chinmay Kulkarni
> Priority: Critical
> Fix For: 5.1.0, 4.16.0
>
>
> Currently, if a metadata upgrade is in progress (either triggered by an
> explicit "EXECUTE UPGRADE" command or by a new client with autoUpgrade
> enabled), in-flight DDLs will generally go through and work as expected.
> However, if the upgrade happens to fail, we restore the snapshot of
> SYSTEM.CATALOG (and with
> [PHOENIX-6086|https://issues.apache.org/jira/browse/PHOENIX-6086] even other
> SYSTEM tables) to represent its state before the upgrade started. Due to
> this, any DDLs issued after the upgrade began are lost.
> There are upgrade steps that need to iterate over each table/index/view in
> the cluster and multiple steps that need full table scans on SYSTEM.CATALOG
> and so this time window where we could potentially lose client DDLs is not
> negligible (could be to the order of minutes).
> This Jira is to discuss ways to tackle this problem. Perhaps we should use
> some sort of write-ahead log to store DDLs issued while the upgrade is in
> progress and replay those DDLs in case we need to restore SYSTEM tables from
> their snapshot.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
