[jira] [Resolved] (PHOENIX-5945) TaskRegionObserver can kick off the same task multiple times if SYSTEM.TASK has split
[ https://issues.apache.org/jira/browse/PHOENIX-5945?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chinmay Kulkarni resolved PHOENIX-5945. --- Resolution: Duplicate > TaskRegionObserver can kick off the same task multiple times if SYSTEM.TASK > has split > - > > Key: PHOENIX-5945 > URL: https://issues.apache.org/jira/browse/PHOENIX-5945 > Project: Phoenix > Issue Type: Bug >Affects Versions: 4.15.0 >Reporter: Chinmay Kulkarni >Priority: Major > Fix For: 5.1.0, 4.16.1, 4.17.0 > > > We don't specify a split policy for > [SYSTEM.TASK|https://github.com/apache/phoenix/blob/5f9364db7e4925229704706e148e62f4cf4ec4c2/phoenix-core/src/main/java/org/apache/phoenix/query/QueryConstants.java#L381], > so by default it will be allowed to split. Now if SYSTEM.TASK spans multiple > regions, each region's > [postOpen|https://github.com/apache/phoenix/blob/5f9364db7e4925229704706e148e62f4cf4ec4c2/phoenix-core/src/main/java/org/apache/phoenix/coprocessor/TaskRegionObserver.java#L137] > schedules the SelfHealingTask at the specified interval and so [each region > will run a FTS on the > table|https://github.com/apache/phoenix/blob/5f9364db7e4925229704706e148e62f4cf4ec4c2/phoenix-core/src/main/java/org/apache/phoenix/coprocessor/TaskRegionObserver.java#L159] > and try to kick-off all the incomplete and non-failed tasks. > This can lead to the same tasks being kicked off multiple times as a corner > race condition in spite of [this > check|https://github.com/apache/phoenix/blob/5f9364db7e4925229704706e148e62f4cf4ec4c2/phoenix-core/src/main/java/org/apache/phoenix/coprocessor/TaskRegionObserver.java#L187-L195] > (which is another FTS) and also lead to unnecessary extra load on the server. > We do not explicitly outline that tasks need to be idempotent, so we should > handle this properly in the TaskRegionObserver so that each region is only > responsible for tasks lying within its boundaries. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (PHOENIX-6032) When phoenix.allow.system.catalog.rollback=true, a view still sees data from a column that was dropped
[
https://issues.apache.org/jira/browse/PHOENIX-6032?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chinmay Kulkarni updated PHOENIX-6032:
--
Attachment: PHOENIX-6032.master.v4.patch
> When phoenix.allow.system.catalog.rollback=true, a view still sees data from
> a column that was dropped
> --
>
> Key: PHOENIX-6032
> URL: https://issues.apache.org/jira/browse/PHOENIX-6032
> Project: Phoenix
> Issue Type: Bug
>Affects Versions: 5.0.0, 4.15.0
>Reporter: Chinmay Kulkarni
>Assignee: Chinmay Kulkarni
>Priority: Blocker
> Fix For: 5.1.0, 4.16.0
>
> Attachments: PHOENIX-6032.master.v1.patch,
> PHOENIX-6032.master.v2.patch, PHOENIX-6032.master.v3.patch,
> PHOENIX-6032.master.v4.patch
>
>
> Start a 4.x server with phoenix.allow.system.catalog.rollback=true,
> phoenix.system.catalog.splittable=false. Connect to it from a 4.x client with
> phoenix.allow.system.catalog.rollback=true. Run the following from the 4.x
> client:
> {code:sql}
> CREATE TABLE IF NOT EXISTS T (A INTEGER PRIMARY KEY, B INTEGER, C VARCHAR, D
> INTEGER);
> CREATE VIEW IF NOT EXISTS V (VA INTEGER, VB INTEGER) AS SELECT * FROM T WHERE
> B=200;
> UPSERT INTO V(A,B,C,D,VA,VB) VALUES (2, 200, 'def', -20, 91, 101);
> SELECT * FROM T;
> ++--+--+--+
> | A | B | C | D |
> ++--+--+--+
> | 2 | 200 | def | -20 |
> ++--+--+--+
> SELECT * FROM V;
> ++--+--+--+-+--+
> | A | B | C | D | VA | VB |
> ++--+--+--+-+--+
> | 2 | 200 | def | -20 | 91 | 101 |
> ++--+--+--+-+--+
> -- as expected
> -- drop a parent column from the view
> ALTER VIEW V DROP COLUMN C;
> SELECT * FROM V;
> +--++--+--+-+--+
> | C | A | B | D | VA | VB |
> +--++--+--+-+--+
> | def | 2 | 200 | -20 | 91 | 101 |
> +--++--+--+-+--+
> -- Column C can still be seen and its ordering is changed for some reason. If
> you run the drop column again, it is actually dropped
> ALTER VIEW V DROP COLUMN C;
> SELECT * FROM V;
> ++--+--+-+--+
> | A | B | D | VA | VB |
> ++--+--+-+--+
> | 2 | 200 | -20 | 91 | 101 |
> ++--+--+-+--+
> -- Gets dropped when drop column is run a second time.
> {code}
> When splittable SYSTEM.CATALOG rollback is enabled, we store the parent's
> column metadata along with the view as well. After the first drop column
> command, metadata for column 'C' of the parent is removed from the view's
> metadata rows however it is not marked diverged, nor is an EXCLUDED_COLUMN
> entry made for that column in the view metadata rows.
> Because of this, when resolving the view we potentially keep combining the
> parent table columns and still get column 'C'. When the second drop column
> command is issued is when we actually add an EXCLUDED_COLUMN linking row for
> 'C' in the view metadata.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Resolved] (PHOENIX-6030) When phoenix.allow.system.catalog.rollback=true, a view still sees data for columns that were dropped from its parent view
[
https://issues.apache.org/jira/browse/PHOENIX-6030?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chinmay Kulkarni resolved PHOENIX-6030.
---
Resolution: Fixed
> When phoenix.allow.system.catalog.rollback=true, a view still sees data for
> columns that were dropped from its parent view
> --
>
> Key: PHOENIX-6030
> URL: https://issues.apache.org/jira/browse/PHOENIX-6030
> Project: Phoenix
> Issue Type: Bug
>Affects Versions: 5.0.0, 4.15.0
>Reporter: Chinmay Kulkarni
>Assignee: Chinmay Kulkarni
>Priority: Blocker
> Fix For: 5.1.0, 4.16.0
>
>
> Start a 4.x server with phoenix.allow.system.catalog.rollback=true,
> phoenix.system.catalog.splittable=false. Connect to it from a 4.x client with
> phoenix.allow.system.catalog.rollback=true. Run the following from the 4.x
> client:
> {code:sql}
> CREATE TABLE IF NOT EXISTS T (A INTEGER PRIMARY KEY, B INTEGER, C VARCHAR, D
> INTEGER);
> CREATE VIEW IF NOT EXISTS V (VA INTEGER, VB INTEGER) AS SELECT * FROM T WHERE
> B=200;
> UPSERT INTO V(A,B,C,D,VA,VB) VALUES (2, 200, 'def', -20, 91, 101);
> SELECT * FROM T;
> ++--+--+--+
> | A | B | C | D |
> ++--+--+--+
> | 2 | 200 | def | -20 |
> ++--+--+--+
> SELECT * FROM V;
> ++--+--+--+-+--+
> | A | B | C | D | VA | VB |
> ++--+--+--+-+--+
> | 2 | 200 | def | -20 | 91 | 101 |
> ++--+--+--+-+--+
> -- as expected
> -- below view can be either a tenant-specific view or a global view, as long
> as its parent is V.
> CREATE VIEW V_t001 AS SELECT * FROM V;
> ALTER VIEW V DROP COLUMN VA;
> SELECT * FROM V;
> ++--+--+--+--+
> | A | B | C | D | VB |
> ++--+--+--+--+
> | 2 | 200 | def | -20 | 101 |
> ++--+--+--+--+
> -- We shouldn't see VA below since it was dropped from the parent
> SELECT * FROM V_T001;
> ++--+--+--+-+--+
> | A | B | C | D | VA | VB |
> ++--+--+--+-+--+
> | 2 | 200 | def | -20 | 91 | 101 |
> ++--+--+--+-+--+
> {code}
> If rollback is enabled, we prevent adding/dropping a column to/from a *table*
> that has child views (see
> [this|https://github.com/apache/phoenix/blob/2fcb8541c9dd7317e62239bd208ff4377ba794e2/phoenix-core/src/main/java/org/apache/phoenix/coprocessor/MetaDataEndpointImpl.java#L2602-L2615]).
> However, we don't prevent adding/dropping columns to/from a *view* that has
> child views (see
> [here|https://github.com/apache/phoenix/blob/2fcb8541c9dd7317e62239bd208ff4377ba794e2/phoenix-core/src/main/java/org/apache/phoenix/coprocessor/MetaDataEndpointImpl.java#L2591]).
> Either we should also prevent column mutations in case of views that have
> children or make sure that dropped columns don't show up when querying a
> child view.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (PHOENIX-6032) When phoenix.allow.system.catalog.rollback=true, a view still sees data from a column that was dropped
[
https://issues.apache.org/jira/browse/PHOENIX-6032?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chinmay Kulkarni updated PHOENIX-6032:
--
Attachment: PHOENIX-6032.master.v3.patch
> When phoenix.allow.system.catalog.rollback=true, a view still sees data from
> a column that was dropped
> --
>
> Key: PHOENIX-6032
> URL: https://issues.apache.org/jira/browse/PHOENIX-6032
> Project: Phoenix
> Issue Type: Bug
>Affects Versions: 5.0.0, 4.15.0
>Reporter: Chinmay Kulkarni
>Assignee: Chinmay Kulkarni
>Priority: Blocker
> Fix For: 5.1.0, 4.16.0
>
> Attachments: PHOENIX-6032.master.v1.patch,
> PHOENIX-6032.master.v2.patch, PHOENIX-6032.master.v3.patch
>
>
> Start a 4.x server with phoenix.allow.system.catalog.rollback=true,
> phoenix.system.catalog.splittable=false. Connect to it from a 4.x client with
> phoenix.allow.system.catalog.rollback=true. Run the following from the 4.x
> client:
> {code:sql}
> CREATE TABLE IF NOT EXISTS T (A INTEGER PRIMARY KEY, B INTEGER, C VARCHAR, D
> INTEGER);
> CREATE VIEW IF NOT EXISTS V (VA INTEGER, VB INTEGER) AS SELECT * FROM T WHERE
> B=200;
> UPSERT INTO V(A,B,C,D,VA,VB) VALUES (2, 200, 'def', -20, 91, 101);
> SELECT * FROM T;
> ++--+--+--+
> | A | B | C | D |
> ++--+--+--+
> | 2 | 200 | def | -20 |
> ++--+--+--+
> SELECT * FROM V;
> ++--+--+--+-+--+
> | A | B | C | D | VA | VB |
> ++--+--+--+-+--+
> | 2 | 200 | def | -20 | 91 | 101 |
> ++--+--+--+-+--+
> -- as expected
> -- drop a parent column from the view
> ALTER VIEW V DROP COLUMN C;
> SELECT * FROM V;
> +--++--+--+-+--+
> | C | A | B | D | VA | VB |
> +--++--+--+-+--+
> | def | 2 | 200 | -20 | 91 | 101 |
> +--++--+--+-+--+
> -- Column C can still be seen and its ordering is changed for some reason. If
> you run the drop column again, it is actually dropped
> ALTER VIEW V DROP COLUMN C;
> SELECT * FROM V;
> ++--+--+-+--+
> | A | B | D | VA | VB |
> ++--+--+-+--+
> | 2 | 200 | -20 | 91 | 101 |
> ++--+--+-+--+
> -- Gets dropped when drop column is run a second time.
> {code}
> When splittable SYSTEM.CATALOG rollback is enabled, we store the parent's
> column metadata along with the view as well. After the first drop column
> command, metadata for column 'C' of the parent is removed from the view's
> metadata rows however it is not marked diverged, nor is an EXCLUDED_COLUMN
> entry made for that column in the view metadata rows.
> Because of this, when resolving the view we potentially keep combining the
> parent table columns and still get column 'C'. When the second drop column
> command is issued is when we actually add an EXCLUDED_COLUMN linking row for
> 'C' in the view metadata.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (PHOENIX-6179) Relax the MaxLookBack age checks during an upgrade
[ https://issues.apache.org/jira/browse/PHOENIX-6179?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Viraj Jasani updated PHOENIX-6179: -- Attachment: (was: PHOENIX-6179.4.x.001.patch) > Relax the MaxLookBack age checks during an upgrade > -- > > Key: PHOENIX-6179 > URL: https://issues.apache.org/jira/browse/PHOENIX-6179 > Project: Phoenix > Issue Type: Bug >Reporter: Jacob Isaac >Assignee: Viraj Jasani >Priority: Critical > Fix For: 5.1.0, 4.16.0 > > Attachments: PHOENIX-6179.4.x.000.patch, PHOENIX-6179.4.x.001.patch, > PHOENIX-6179.master.000.patch, PHOENIX-6179.master.001.patch > > > Getting this error when trying to upgrade cluster - Error: ERROR 538 (42915): > Cannot use SCN to look further back in the past beyond the configured max > lookback age (state=42915,code=538) > > During the upgrade the SCN for the connection is set to be the phoenix > version timestamp which is a small number. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (PHOENIX-6179) Relax the MaxLookBack age checks during an upgrade
[ https://issues.apache.org/jira/browse/PHOENIX-6179?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Viraj Jasani updated PHOENIX-6179: -- Attachment: PHOENIX-6179.4.x.001.patch > Relax the MaxLookBack age checks during an upgrade > -- > > Key: PHOENIX-6179 > URL: https://issues.apache.org/jira/browse/PHOENIX-6179 > Project: Phoenix > Issue Type: Bug >Reporter: Jacob Isaac >Assignee: Viraj Jasani >Priority: Critical > Fix For: 5.1.0, 4.16.0 > > Attachments: PHOENIX-6179.4.x.000.patch, PHOENIX-6179.4.x.001.patch, > PHOENIX-6179.master.000.patch, PHOENIX-6179.master.001.patch > > > Getting this error when trying to upgrade cluster - Error: ERROR 538 (42915): > Cannot use SCN to look further back in the past beyond the configured max > lookback age (state=42915,code=538) > > During the upgrade the SCN for the connection is set to be the phoenix > version timestamp which is a small number. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (PHOENIX-6087) Phoenix Connection leak in UpgradeUtil.addViewIndexToParentLinks()
[ https://issues.apache.org/jira/browse/PHOENIX-6087?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Viraj Jasani updated PHOENIX-6087: -- Attachment: PHOENIX-6087.4.x.000.patch > Phoenix Connection leak in UpgradeUtil.addViewIndexToParentLinks() > -- > > Key: PHOENIX-6087 > URL: https://issues.apache.org/jira/browse/PHOENIX-6087 > Project: Phoenix > Issue Type: Bug >Affects Versions: 5.0.0, 4.15.0 >Reporter: Chinmay Kulkarni >Assignee: Viraj Jasani >Priority: Major > Labels: phoenix-hardening, quality-improvement > Fix For: 5.1.0, 4.16.0 > > Attachments: PHOENIX-6087.4.x.000.patch, PHOENIX-6087.master.000.patch > > > Inside UpgradeUtil.addViewIndexToParentLinks(), we assign a new Phoenix > connection to the existing metaConn variable without closing the previous > cloned tenant connection > ([here|https://github.com/apache/phoenix/blob/1922895dfe5960dc025709b04acfaf974d3959dc/phoenix-core/src/main/java/org/apache/phoenix/util/UpgradeUtil.java#L1206]). > We should close the tenant-connection before re-initializing metaConn. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (PHOENIX-6087) Phoenix Connection leak in UpgradeUtil.addViewIndexToParentLinks()
[ https://issues.apache.org/jira/browse/PHOENIX-6087?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Viraj Jasani updated PHOENIX-6087: -- Attachment: PHOENIX-6087.master.000.patch > Phoenix Connection leak in UpgradeUtil.addViewIndexToParentLinks() > -- > > Key: PHOENIX-6087 > URL: https://issues.apache.org/jira/browse/PHOENIX-6087 > Project: Phoenix > Issue Type: Bug >Affects Versions: 5.0.0, 4.15.0 >Reporter: Chinmay Kulkarni >Assignee: Viraj Jasani >Priority: Major > Labels: phoenix-hardening, quality-improvement > Fix For: 5.1.0, 4.16.0 > > Attachments: PHOENIX-6087.master.000.patch > > > Inside UpgradeUtil.addViewIndexToParentLinks(), we assign a new Phoenix > connection to the existing metaConn variable without closing the previous > cloned tenant connection > ([here|https://github.com/apache/phoenix/blob/1922895dfe5960dc025709b04acfaf974d3959dc/phoenix-core/src/main/java/org/apache/phoenix/util/UpgradeUtil.java#L1206]). > We should close the tenant-connection before re-initializing metaConn. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (PHOENIX-6179) Relax the MaxLookBack age checks during an upgrade
[ https://issues.apache.org/jira/browse/PHOENIX-6179?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Viraj Jasani updated PHOENIX-6179: -- Attachment: (was: PHOENIX-6179.4.x.001.patch) > Relax the MaxLookBack age checks during an upgrade > -- > > Key: PHOENIX-6179 > URL: https://issues.apache.org/jira/browse/PHOENIX-6179 > Project: Phoenix > Issue Type: Bug >Reporter: Jacob Isaac >Assignee: Viraj Jasani >Priority: Critical > Fix For: 5.1.0, 4.16.0 > > Attachments: PHOENIX-6179.4.x.000.patch, PHOENIX-6179.4.x.001.patch, > PHOENIX-6179.master.000.patch, PHOENIX-6179.master.001.patch > > > Getting this error when trying to upgrade cluster - Error: ERROR 538 (42915): > Cannot use SCN to look further back in the past beyond the configured max > lookback age (state=42915,code=538) > > During the upgrade the SCN for the connection is set to be the phoenix > version timestamp which is a small number. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (PHOENIX-6179) Relax the MaxLookBack age checks during an upgrade
[ https://issues.apache.org/jira/browse/PHOENIX-6179?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Viraj Jasani updated PHOENIX-6179: -- Attachment: PHOENIX-6179.4.x.001.patch > Relax the MaxLookBack age checks during an upgrade > -- > > Key: PHOENIX-6179 > URL: https://issues.apache.org/jira/browse/PHOENIX-6179 > Project: Phoenix > Issue Type: Bug >Reporter: Jacob Isaac >Assignee: Viraj Jasani >Priority: Critical > Fix For: 5.1.0, 4.16.0 > > Attachments: PHOENIX-6179.4.x.000.patch, PHOENIX-6179.4.x.001.patch, > PHOENIX-6179.master.000.patch, PHOENIX-6179.master.001.patch > > > Getting this error when trying to upgrade cluster - Error: ERROR 538 (42915): > Cannot use SCN to look further back in the past beyond the configured max > lookback age (state=42915,code=538) > > During the upgrade the SCN for the connection is set to be the phoenix > version timestamp which is a small number. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (PHOENIX-6179) Relax the MaxLookBack age checks during an upgrade
[ https://issues.apache.org/jira/browse/PHOENIX-6179?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Viraj Jasani updated PHOENIX-6179: -- Attachment: PHOENIX-6179.4.x.001.patch > Relax the MaxLookBack age checks during an upgrade > -- > > Key: PHOENIX-6179 > URL: https://issues.apache.org/jira/browse/PHOENIX-6179 > Project: Phoenix > Issue Type: Bug >Reporter: Jacob Isaac >Assignee: Viraj Jasani >Priority: Critical > Fix For: 5.1.0, 4.16.0 > > Attachments: PHOENIX-6179.4.x.000.patch, PHOENIX-6179.4.x.001.patch, > PHOENIX-6179.master.000.patch, PHOENIX-6179.master.001.patch > > > Getting this error when trying to upgrade cluster - Error: ERROR 538 (42915): > Cannot use SCN to look further back in the past beyond the configured max > lookback age (state=42915,code=538) > > During the upgrade the SCN for the connection is set to be the phoenix > version timestamp which is a small number. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (PHOENIX-6179) Relax the MaxLookBack age checks during an upgrade
[ https://issues.apache.org/jira/browse/PHOENIX-6179?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Viraj Jasani updated PHOENIX-6179: -- Attachment: PHOENIX-6179.master.001.patch > Relax the MaxLookBack age checks during an upgrade > -- > > Key: PHOENIX-6179 > URL: https://issues.apache.org/jira/browse/PHOENIX-6179 > Project: Phoenix > Issue Type: Bug >Reporter: Jacob Isaac >Assignee: Viraj Jasani >Priority: Critical > Fix For: 5.1.0, 4.16.0 > > Attachments: PHOENIX-6179.4.x.000.patch, > PHOENIX-6179.master.000.patch, PHOENIX-6179.master.001.patch > > > Getting this error when trying to upgrade cluster - Error: ERROR 538 (42915): > Cannot use SCN to look further back in the past beyond the configured max > lookback age (state=42915,code=538) > > During the upgrade the SCN for the connection is set to be the phoenix > version timestamp which is a small number. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (PHOENIX-6086) Take a snapshot of all SYSTEM tables before attempting to upgrade them
[ https://issues.apache.org/jira/browse/PHOENIX-6086?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Viraj Jasani updated PHOENIX-6086: -- Attachment: PHOENIX-6086.master.002.patch > Take a snapshot of all SYSTEM tables before attempting to upgrade them > -- > > Key: PHOENIX-6086 > URL: https://issues.apache.org/jira/browse/PHOENIX-6086 > Project: Phoenix > Issue Type: Improvement >Affects Versions: 5.0.0, 4.15.0 >Reporter: Chinmay Kulkarni >Assignee: Viraj Jasani >Priority: Critical > Fix For: 5.1.0, 4.16.0 > > Attachments: PHOENIX-6086.master.000.patch, > PHOENIX-6086.master.002.patch > > > Currently we only take a snapshot of SYSTEM.CATALOG before attempting to > upgrade it (see > [this|https://github.com/apache/phoenix/blob/1922895dfe5960dc025709b04acfaf974d3959dc/phoenix-core/src/main/java/org/apache/phoenix/query/ConnectionQueryServicesImpl.java#L3718]). > From 4.15 onwards we also store critical metadata information in other > SYSTEM tables like SYSTEM.CHILD_LINK, so it is beneficial to also snapshot > those tables before upgrading them henceforth. > We also currently don't take a snapshot of SYSTEM.CATALOG on receiving an > [UpgradeRequiredException|https://github.com/apache/phoenix/blob/1922895dfe5960dc025709b04acfaf974d3959dc/phoenix-core/src/main/java/org/apache/phoenix/query/ConnectionQueryServicesImpl.java#L3685-L3707] > which we should do. > In case of any errors during the upgrade, we restore SYSTEM.CATALOG from this > snapshot and we should extend this to all tables. In cases where the table > didn't exist before the upgrade, we need to ensure it is dropped so that a > subsequent upgrade attempt can start afresh. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (PHOENIX-6086) Take a snapshot of all SYSTEM tables before attempting to upgrade them
[ https://issues.apache.org/jira/browse/PHOENIX-6086?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Viraj Jasani updated PHOENIX-6086: -- Attachment: PHOENIX-6086.master.000.patch > Take a snapshot of all SYSTEM tables before attempting to upgrade them > -- > > Key: PHOENIX-6086 > URL: https://issues.apache.org/jira/browse/PHOENIX-6086 > Project: Phoenix > Issue Type: Improvement >Affects Versions: 5.0.0, 4.15.0 >Reporter: Chinmay Kulkarni >Assignee: Viraj Jasani >Priority: Critical > Fix For: 5.1.0, 4.16.0 > > Attachments: PHOENIX-6086.master.000.patch > > > Currently we only take a snapshot of SYSTEM.CATALOG before attempting to > upgrade it (see > [this|https://github.com/apache/phoenix/blob/1922895dfe5960dc025709b04acfaf974d3959dc/phoenix-core/src/main/java/org/apache/phoenix/query/ConnectionQueryServicesImpl.java#L3718]). > From 4.15 onwards we also store critical metadata information in other > SYSTEM tables like SYSTEM.CHILD_LINK, so it is beneficial to also snapshot > those tables before upgrading them henceforth. > We also currently don't take a snapshot of SYSTEM.CATALOG on receiving an > [UpgradeRequiredException|https://github.com/apache/phoenix/blob/1922895dfe5960dc025709b04acfaf974d3959dc/phoenix-core/src/main/java/org/apache/phoenix/query/ConnectionQueryServicesImpl.java#L3685-L3707] > which we should do. > In case of any errors during the upgrade, we restore SYSTEM.CATALOG from this > snapshot and we should extend this to all tables. In cases where the table > didn't exist before the upgrade, we need to ensure it is dropped so that a > subsequent upgrade attempt can start afresh. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (PHOENIX-6086) Take a snapshot of all SYSTEM tables before attempting to upgrade them
[ https://issues.apache.org/jira/browse/PHOENIX-6086?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Viraj Jasani reassigned PHOENIX-6086: - Assignee: Viraj Jasani > Take a snapshot of all SYSTEM tables before attempting to upgrade them > -- > > Key: PHOENIX-6086 > URL: https://issues.apache.org/jira/browse/PHOENIX-6086 > Project: Phoenix > Issue Type: Improvement >Affects Versions: 5.0.0, 4.15.0 >Reporter: Chinmay Kulkarni >Assignee: Viraj Jasani >Priority: Critical > Fix For: 5.1.0, 4.16.0 > > Attachments: PHOENIX-6086.master.000.patch > > > Currently we only take a snapshot of SYSTEM.CATALOG before attempting to > upgrade it (see > [this|https://github.com/apache/phoenix/blob/1922895dfe5960dc025709b04acfaf974d3959dc/phoenix-core/src/main/java/org/apache/phoenix/query/ConnectionQueryServicesImpl.java#L3718]). > From 4.15 onwards we also store critical metadata information in other > SYSTEM tables like SYSTEM.CHILD_LINK, so it is beneficial to also snapshot > those tables before upgrading them henceforth. > We also currently don't take a snapshot of SYSTEM.CATALOG on receiving an > [UpgradeRequiredException|https://github.com/apache/phoenix/blob/1922895dfe5960dc025709b04acfaf974d3959dc/phoenix-core/src/main/java/org/apache/phoenix/query/ConnectionQueryServicesImpl.java#L3685-L3707] > which we should do. > In case of any errors during the upgrade, we restore SYSTEM.CATALOG from this > snapshot and we should extend this to all tables. In cases where the table > didn't exist before the upgrade, we need to ensure it is dropped so that a > subsequent upgrade attempt can start afresh. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (PHOENIX-6002) Fix connection leaks throughout instances where we use QueryUtil.getConnectionOnServer
[ https://issues.apache.org/jira/browse/PHOENIX-6002?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Viraj Jasani updated PHOENIX-6002: -- Attachment: (was: PHOENIX-6002.4.x.000.patch) > Fix connection leaks throughout instances where we use > QueryUtil.getConnectionOnServer > -- > > Key: PHOENIX-6002 > URL: https://issues.apache.org/jira/browse/PHOENIX-6002 > Project: Phoenix > Issue Type: Bug >Affects Versions: 4.15.0 >Reporter: Chinmay Kulkarni >Assignee: Viraj Jasani >Priority: Major > Labels: phoenix-hardening, quality-improvement > Fix For: 5.1.0, 4.16.0 > > Attachments: PHOENIX-6002.4.x.000.patch, PHOENIX-6002.master.000.patch > > > For example inside MetaDataEndpointImpl#dropIndexes. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (PHOENIX-6002) Fix connection leaks throughout instances where we use QueryUtil.getConnectionOnServer
[ https://issues.apache.org/jira/browse/PHOENIX-6002?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Viraj Jasani updated PHOENIX-6002: -- Attachment: PHOENIX-6002.4.x.000.patch > Fix connection leaks throughout instances where we use > QueryUtil.getConnectionOnServer > -- > > Key: PHOENIX-6002 > URL: https://issues.apache.org/jira/browse/PHOENIX-6002 > Project: Phoenix > Issue Type: Bug >Affects Versions: 4.15.0 >Reporter: Chinmay Kulkarni >Assignee: Viraj Jasani >Priority: Major > Labels: phoenix-hardening, quality-improvement > Fix For: 5.1.0, 4.16.0 > > Attachments: PHOENIX-6002.4.x.000.patch, PHOENIX-6002.master.000.patch > > > For example inside MetaDataEndpointImpl#dropIndexes. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (PHOENIX-6002) Fix connection leaks throughout instances where we use QueryUtil.getConnectionOnServer
[ https://issues.apache.org/jira/browse/PHOENIX-6002?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Viraj Jasani updated PHOENIX-6002: -- Attachment: PHOENIX-6002.master.000.patch > Fix connection leaks throughout instances where we use > QueryUtil.getConnectionOnServer > -- > > Key: PHOENIX-6002 > URL: https://issues.apache.org/jira/browse/PHOENIX-6002 > Project: Phoenix > Issue Type: Bug >Affects Versions: 4.15.0 >Reporter: Chinmay Kulkarni >Assignee: Viraj Jasani >Priority: Major > Labels: phoenix-hardening, quality-improvement > Fix For: 5.1.0, 4.16.0 > > Attachments: PHOENIX-6002.4.x.000.patch, PHOENIX-6002.master.000.patch > > > For example inside MetaDataEndpointImpl#dropIndexes. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (PHOENIX-6002) Fix connection leaks throughout instances where we use QueryUtil.getConnectionOnServer
[ https://issues.apache.org/jira/browse/PHOENIX-6002?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Viraj Jasani updated PHOENIX-6002: -- Attachment: PHOENIX-6002.4.x.000.patch > Fix connection leaks throughout instances where we use > QueryUtil.getConnectionOnServer > -- > > Key: PHOENIX-6002 > URL: https://issues.apache.org/jira/browse/PHOENIX-6002 > Project: Phoenix > Issue Type: Bug >Affects Versions: 4.15.0 >Reporter: Chinmay Kulkarni >Assignee: Viraj Jasani >Priority: Major > Labels: phoenix-hardening, quality-improvement > Fix For: 5.1.0, 4.16.0 > > Attachments: PHOENIX-6002.4.x.000.patch > > > For example inside MetaDataEndpointImpl#dropIndexes. -- This message was sent by Atlassian Jira (v8.3.4#803005)
