[jira] [Updated] (PHOENIX-6032) When phoenix.allow.system.catalog.rollback=true, a view still sees data from a column that was dropped
[
https://issues.apache.org/jira/browse/PHOENIX-6032?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chinmay Kulkarni updated PHOENIX-6032:
--
Comment: was deleted
(was: | (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
36s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} dupname {color} | {color:green} 0m
0s{color} | {color:green} No case conflicting files found. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
36s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
52s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
39s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
43s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} spotbugs {color} | {color:blue} 2m
47s{color} | {color:blue} phoenix-core in master has 969 extant spotbugs
warnings. {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
20s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
51s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
51s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
39s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
41s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} spotbugs {color} | {color:green} 3m
1s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 95m 56s{color}
| {color:red} phoenix-core in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
24s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}122m 42s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | ClientAPI=1.40 ServerAPI=1.40 base:
https://ci-hadoop.apache.org/job/PreCommit-PHOENIX-Build/149/artifact/patchprocess/Dockerfile
|
| JIRA Issue | PHOENIX-6032 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/13014595/PHOENIX-6032.master.v5.patch
|
| Optional Tests | dupname asflicense javac javadoc unit spotbugs hbaseanti
checkstyle compile |
| uname | Linux cb5dd8111da4 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9
23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev/phoenix-personality.sh |
| git revision | master / e828ef7 |
| Default Java | Private Build-1.8.0_242-8u242-b08-0ubuntu3~16.04-b08 |
| unit |
https://ci-hadoop.apache.org/job/PreCommit-PHOENIX-Build/149/artifact/patchprocess/patch-unit-phoenix-core.txt
|
| Test Results |
https://ci-hadoop.apache.org/job/PreCommit-PHOENIX-Build/149/testReport/ |
| Max. process+thread count | 6952 (vs. ulimit of 3) |
| modules | C: phoenix-core U: phoenix-core |
| Console output |
https://ci-hadoop.apache.org/job/PreCommit-PHOENIX-Build/149/console |
| versions | git=2.7.4 maven=3.3.9 spotbugs=4.1.3 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
)
> When phoenix.allow.system.catalog.rollback=true, a view still sees data from
> a column that was dropped
>
[jira] [Updated] (PHOENIX-6032) When phoenix.allow.system.catalog.rollback=true, a view still sees data from a column that was dropped
[
https://issues.apache.org/jira/browse/PHOENIX-6032?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chinmay Kulkarni updated PHOENIX-6032:
--
Attachment: PHOENIX-6032.master.v6.patch
> When phoenix.allow.system.catalog.rollback=true, a view still sees data from
> a column that was dropped
> --
>
> Key: PHOENIX-6032
> URL: https://issues.apache.org/jira/browse/PHOENIX-6032
> Project: Phoenix
> Issue Type: Bug
>Affects Versions: 5.0.0, 4.15.0
>Reporter: Chinmay Kulkarni
>Assignee: Chinmay Kulkarni
>Priority: Blocker
> Fix For: 5.1.0, 4.16.0
>
> Attachments: PHOENIX-6032.master.v1.patch,
> PHOENIX-6032.master.v2.patch, PHOENIX-6032.master.v3.patch,
> PHOENIX-6032.master.v4.patch, PHOENIX-6032.master.v5.patch,
> PHOENIX-6032.master.v6.patch
>
>
> Start a 4.x server with phoenix.allow.system.catalog.rollback=true,
> phoenix.system.catalog.splittable=false. Connect to it from a 4.x client with
> phoenix.allow.system.catalog.rollback=true. Run the following from the 4.x
> client:
> {code:sql}
> CREATE TABLE IF NOT EXISTS T (A INTEGER PRIMARY KEY, B INTEGER, C VARCHAR, D
> INTEGER);
> CREATE VIEW IF NOT EXISTS V (VA INTEGER, VB INTEGER) AS SELECT * FROM T WHERE
> B=200;
> UPSERT INTO V(A,B,C,D,VA,VB) VALUES (2, 200, 'def', -20, 91, 101);
> SELECT * FROM T;
> ++--+--+--+
> | A | B | C | D |
> ++--+--+--+
> | 2 | 200 | def | -20 |
> ++--+--+--+
> SELECT * FROM V;
> ++--+--+--+-+--+
> | A | B | C | D | VA | VB |
> ++--+--+--+-+--+
> | 2 | 200 | def | -20 | 91 | 101 |
> ++--+--+--+-+--+
> -- as expected
> -- drop a parent column from the view
> ALTER VIEW V DROP COLUMN C;
> SELECT * FROM V;
> +--++--+--+-+--+
> | C | A | B | D | VA | VB |
> +--++--+--+-+--+
> | def | 2 | 200 | -20 | 91 | 101 |
> +--++--+--+-+--+
> -- Column C can still be seen and its ordering is changed for some reason. If
> you run the drop column again, it is actually dropped
> ALTER VIEW V DROP COLUMN C;
> SELECT * FROM V;
> ++--+--+-+--+
> | A | B | D | VA | VB |
> ++--+--+-+--+
> | 2 | 200 | -20 | 91 | 101 |
> ++--+--+-+--+
> -- Gets dropped when drop column is run a second time.
> {code}
> When splittable SYSTEM.CATALOG rollback is enabled, we store the parent's
> column metadata along with the view as well. After the first drop column
> command, metadata for column 'C' of the parent is removed from the view's
> metadata rows however it is not marked diverged, nor is an EXCLUDED_COLUMN
> entry made for that column in the view metadata rows.
> Because of this, when resolving the view we potentially keep combining the
> parent table columns and still get column 'C'. When the second drop column
> command is issued is when we actually add an EXCLUDED_COLUMN linking row for
> 'C' in the view metadata.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (PHOENIX-6032) When phoenix.allow.system.catalog.rollback=true, a view still sees data from a column that was dropped
[
https://issues.apache.org/jira/browse/PHOENIX-6032?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chinmay Kulkarni updated PHOENIX-6032:
--
Attachment: PHOENIX-6032.master.v5.patch
> When phoenix.allow.system.catalog.rollback=true, a view still sees data from
> a column that was dropped
> --
>
> Key: PHOENIX-6032
> URL: https://issues.apache.org/jira/browse/PHOENIX-6032
> Project: Phoenix
> Issue Type: Bug
>Affects Versions: 5.0.0, 4.15.0
>Reporter: Chinmay Kulkarni
>Assignee: Chinmay Kulkarni
>Priority: Blocker
> Fix For: 5.1.0, 4.16.0
>
> Attachments: PHOENIX-6032.master.v1.patch,
> PHOENIX-6032.master.v2.patch, PHOENIX-6032.master.v3.patch,
> PHOENIX-6032.master.v4.patch, PHOENIX-6032.master.v5.patch
>
>
> Start a 4.x server with phoenix.allow.system.catalog.rollback=true,
> phoenix.system.catalog.splittable=false. Connect to it from a 4.x client with
> phoenix.allow.system.catalog.rollback=true. Run the following from the 4.x
> client:
> {code:sql}
> CREATE TABLE IF NOT EXISTS T (A INTEGER PRIMARY KEY, B INTEGER, C VARCHAR, D
> INTEGER);
> CREATE VIEW IF NOT EXISTS V (VA INTEGER, VB INTEGER) AS SELECT * FROM T WHERE
> B=200;
> UPSERT INTO V(A,B,C,D,VA,VB) VALUES (2, 200, 'def', -20, 91, 101);
> SELECT * FROM T;
> ++--+--+--+
> | A | B | C | D |
> ++--+--+--+
> | 2 | 200 | def | -20 |
> ++--+--+--+
> SELECT * FROM V;
> ++--+--+--+-+--+
> | A | B | C | D | VA | VB |
> ++--+--+--+-+--+
> | 2 | 200 | def | -20 | 91 | 101 |
> ++--+--+--+-+--+
> -- as expected
> -- drop a parent column from the view
> ALTER VIEW V DROP COLUMN C;
> SELECT * FROM V;
> +--++--+--+-+--+
> | C | A | B | D | VA | VB |
> +--++--+--+-+--+
> | def | 2 | 200 | -20 | 91 | 101 |
> +--++--+--+-+--+
> -- Column C can still be seen and its ordering is changed for some reason. If
> you run the drop column again, it is actually dropped
> ALTER VIEW V DROP COLUMN C;
> SELECT * FROM V;
> ++--+--+-+--+
> | A | B | D | VA | VB |
> ++--+--+-+--+
> | 2 | 200 | -20 | 91 | 101 |
> ++--+--+-+--+
> -- Gets dropped when drop column is run a second time.
> {code}
> When splittable SYSTEM.CATALOG rollback is enabled, we store the parent's
> column metadata along with the view as well. After the first drop column
> command, metadata for column 'C' of the parent is removed from the view's
> metadata rows however it is not marked diverged, nor is an EXCLUDED_COLUMN
> entry made for that column in the view metadata rows.
> Because of this, when resolving the view we potentially keep combining the
> parent table columns and still get column 'C'. When the second drop column
> command is issued is when we actually add an EXCLUDED_COLUMN linking row for
> 'C' in the view metadata.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (PHOENIX-6032) When phoenix.allow.system.catalog.rollback=true, a view still sees data from a column that was dropped
[
https://issues.apache.org/jira/browse/PHOENIX-6032?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chinmay Kulkarni updated PHOENIX-6032:
--
Attachment: PHOENIX-6032.master.v4.patch
> When phoenix.allow.system.catalog.rollback=true, a view still sees data from
> a column that was dropped
> --
>
> Key: PHOENIX-6032
> URL: https://issues.apache.org/jira/browse/PHOENIX-6032
> Project: Phoenix
> Issue Type: Bug
>Affects Versions: 5.0.0, 4.15.0
>Reporter: Chinmay Kulkarni
>Assignee: Chinmay Kulkarni
>Priority: Blocker
> Fix For: 5.1.0, 4.16.0
>
> Attachments: PHOENIX-6032.master.v1.patch,
> PHOENIX-6032.master.v2.patch, PHOENIX-6032.master.v3.patch,
> PHOENIX-6032.master.v4.patch
>
>
> Start a 4.x server with phoenix.allow.system.catalog.rollback=true,
> phoenix.system.catalog.splittable=false. Connect to it from a 4.x client with
> phoenix.allow.system.catalog.rollback=true. Run the following from the 4.x
> client:
> {code:sql}
> CREATE TABLE IF NOT EXISTS T (A INTEGER PRIMARY KEY, B INTEGER, C VARCHAR, D
> INTEGER);
> CREATE VIEW IF NOT EXISTS V (VA INTEGER, VB INTEGER) AS SELECT * FROM T WHERE
> B=200;
> UPSERT INTO V(A,B,C,D,VA,VB) VALUES (2, 200, 'def', -20, 91, 101);
> SELECT * FROM T;
> ++--+--+--+
> | A | B | C | D |
> ++--+--+--+
> | 2 | 200 | def | -20 |
> ++--+--+--+
> SELECT * FROM V;
> ++--+--+--+-+--+
> | A | B | C | D | VA | VB |
> ++--+--+--+-+--+
> | 2 | 200 | def | -20 | 91 | 101 |
> ++--+--+--+-+--+
> -- as expected
> -- drop a parent column from the view
> ALTER VIEW V DROP COLUMN C;
> SELECT * FROM V;
> +--++--+--+-+--+
> | C | A | B | D | VA | VB |
> +--++--+--+-+--+
> | def | 2 | 200 | -20 | 91 | 101 |
> +--++--+--+-+--+
> -- Column C can still be seen and its ordering is changed for some reason. If
> you run the drop column again, it is actually dropped
> ALTER VIEW V DROP COLUMN C;
> SELECT * FROM V;
> ++--+--+-+--+
> | A | B | D | VA | VB |
> ++--+--+-+--+
> | 2 | 200 | -20 | 91 | 101 |
> ++--+--+-+--+
> -- Gets dropped when drop column is run a second time.
> {code}
> When splittable SYSTEM.CATALOG rollback is enabled, we store the parent's
> column metadata along with the view as well. After the first drop column
> command, metadata for column 'C' of the parent is removed from the view's
> metadata rows however it is not marked diverged, nor is an EXCLUDED_COLUMN
> entry made for that column in the view metadata rows.
> Because of this, when resolving the view we potentially keep combining the
> parent table columns and still get column 'C'. When the second drop column
> command is issued is when we actually add an EXCLUDED_COLUMN linking row for
> 'C' in the view metadata.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (PHOENIX-6032) When phoenix.allow.system.catalog.rollback=true, a view still sees data from a column that was dropped
[
https://issues.apache.org/jira/browse/PHOENIX-6032?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chinmay Kulkarni updated PHOENIX-6032:
--
Attachment: PHOENIX-6032.master.v3.patch
> When phoenix.allow.system.catalog.rollback=true, a view still sees data from
> a column that was dropped
> --
>
> Key: PHOENIX-6032
> URL: https://issues.apache.org/jira/browse/PHOENIX-6032
> Project: Phoenix
> Issue Type: Bug
>Affects Versions: 5.0.0, 4.15.0
>Reporter: Chinmay Kulkarni
>Assignee: Chinmay Kulkarni
>Priority: Blocker
> Fix For: 5.1.0, 4.16.0
>
> Attachments: PHOENIX-6032.master.v1.patch,
> PHOENIX-6032.master.v2.patch, PHOENIX-6032.master.v3.patch
>
>
> Start a 4.x server with phoenix.allow.system.catalog.rollback=true,
> phoenix.system.catalog.splittable=false. Connect to it from a 4.x client with
> phoenix.allow.system.catalog.rollback=true. Run the following from the 4.x
> client:
> {code:sql}
> CREATE TABLE IF NOT EXISTS T (A INTEGER PRIMARY KEY, B INTEGER, C VARCHAR, D
> INTEGER);
> CREATE VIEW IF NOT EXISTS V (VA INTEGER, VB INTEGER) AS SELECT * FROM T WHERE
> B=200;
> UPSERT INTO V(A,B,C,D,VA,VB) VALUES (2, 200, 'def', -20, 91, 101);
> SELECT * FROM T;
> ++--+--+--+
> | A | B | C | D |
> ++--+--+--+
> | 2 | 200 | def | -20 |
> ++--+--+--+
> SELECT * FROM V;
> ++--+--+--+-+--+
> | A | B | C | D | VA | VB |
> ++--+--+--+-+--+
> | 2 | 200 | def | -20 | 91 | 101 |
> ++--+--+--+-+--+
> -- as expected
> -- drop a parent column from the view
> ALTER VIEW V DROP COLUMN C;
> SELECT * FROM V;
> +--++--+--+-+--+
> | C | A | B | D | VA | VB |
> +--++--+--+-+--+
> | def | 2 | 200 | -20 | 91 | 101 |
> +--++--+--+-+--+
> -- Column C can still be seen and its ordering is changed for some reason. If
> you run the drop column again, it is actually dropped
> ALTER VIEW V DROP COLUMN C;
> SELECT * FROM V;
> ++--+--+-+--+
> | A | B | D | VA | VB |
> ++--+--+-+--+
> | 2 | 200 | -20 | 91 | 101 |
> ++--+--+-+--+
> -- Gets dropped when drop column is run a second time.
> {code}
> When splittable SYSTEM.CATALOG rollback is enabled, we store the parent's
> column metadata along with the view as well. After the first drop column
> command, metadata for column 'C' of the parent is removed from the view's
> metadata rows however it is not marked diverged, nor is an EXCLUDED_COLUMN
> entry made for that column in the view metadata rows.
> Because of this, when resolving the view we potentially keep combining the
> parent table columns and still get column 'C'. When the second drop column
> command is issued is when we actually add an EXCLUDED_COLUMN linking row for
> 'C' in the view metadata.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (PHOENIX-6032) When phoenix.allow.system.catalog.rollback=true, a view still sees data from a column that was dropped
[
https://issues.apache.org/jira/browse/PHOENIX-6032?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chinmay Kulkarni updated PHOENIX-6032:
--
Attachment: PHOENIX-6032.master.v2.patch
> When phoenix.allow.system.catalog.rollback=true, a view still sees data from
> a column that was dropped
> --
>
> Key: PHOENIX-6032
> URL: https://issues.apache.org/jira/browse/PHOENIX-6032
> Project: Phoenix
> Issue Type: Bug
>Affects Versions: 5.0.0, 4.15.0
>Reporter: Chinmay Kulkarni
>Assignee: Chinmay Kulkarni
>Priority: Blocker
> Fix For: 5.1.0, 4.16.0
>
> Attachments: PHOENIX-6032.master.v1.patch,
> PHOENIX-6032.master.v2.patch
>
>
> Start a 4.x server with phoenix.allow.system.catalog.rollback=true,
> phoenix.system.catalog.splittable=false. Connect to it from a 4.x client with
> phoenix.allow.system.catalog.rollback=true. Run the following from the 4.x
> client:
> {code:sql}
> CREATE TABLE IF NOT EXISTS T (A INTEGER PRIMARY KEY, B INTEGER, C VARCHAR, D
> INTEGER);
> CREATE VIEW IF NOT EXISTS V (VA INTEGER, VB INTEGER) AS SELECT * FROM T WHERE
> B=200;
> UPSERT INTO V(A,B,C,D,VA,VB) VALUES (2, 200, 'def', -20, 91, 101);
> SELECT * FROM T;
> ++--+--+--+
> | A | B | C | D |
> ++--+--+--+
> | 2 | 200 | def | -20 |
> ++--+--+--+
> SELECT * FROM V;
> ++--+--+--+-+--+
> | A | B | C | D | VA | VB |
> ++--+--+--+-+--+
> | 2 | 200 | def | -20 | 91 | 101 |
> ++--+--+--+-+--+
> -- as expected
> -- drop a parent column from the view
> ALTER VIEW V DROP COLUMN C;
> SELECT * FROM V;
> +--++--+--+-+--+
> | C | A | B | D | VA | VB |
> +--++--+--+-+--+
> | def | 2 | 200 | -20 | 91 | 101 |
> +--++--+--+-+--+
> -- Column C can still be seen and its ordering is changed for some reason. If
> you run the drop column again, it is actually dropped
> ALTER VIEW V DROP COLUMN C;
> SELECT * FROM V;
> ++--+--+-+--+
> | A | B | D | VA | VB |
> ++--+--+-+--+
> | 2 | 200 | -20 | 91 | 101 |
> ++--+--+-+--+
> -- Gets dropped when drop column is run a second time.
> {code}
> When splittable SYSTEM.CATALOG rollback is enabled, we store the parent's
> column metadata along with the view as well. After the first drop column
> command, metadata for column 'C' of the parent is removed from the view's
> metadata rows however it is not marked diverged, nor is an EXCLUDED_COLUMN
> entry made for that column in the view metadata rows.
> Because of this, when resolving the view we potentially keep combining the
> parent table columns and still get column 'C'. When the second drop column
> command is issued is when we actually add an EXCLUDED_COLUMN linking row for
> 'C' in the view metadata.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (PHOENIX-6032) When phoenix.allow.system.catalog.rollback=true, a view still sees data from a column that was dropped
[
https://issues.apache.org/jira/browse/PHOENIX-6032?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chinmay Kulkarni updated PHOENIX-6032:
--
Fix Version/s: 5.1.0
> When phoenix.allow.system.catalog.rollback=true, a view still sees data from
> a column that was dropped
> --
>
> Key: PHOENIX-6032
> URL: https://issues.apache.org/jira/browse/PHOENIX-6032
> Project: Phoenix
> Issue Type: Bug
>Affects Versions: 5.0.0, 4.15.0
>Reporter: Chinmay Kulkarni
>Priority: Blocker
> Fix For: 5.1.0, 4.16.0
>
>
> Start a 4.x server with phoenix.allow.system.catalog.rollback=true,
> phoenix.system.catalog.splittable=false. Connect to it from a 4.x client with
> phoenix.allow.system.catalog.rollback=true. Run the following from the 4.x
> client:
> {code:sql}
> CREATE TABLE IF NOT EXISTS T (A INTEGER PRIMARY KEY, B INTEGER, C VARCHAR, D
> INTEGER);
> CREATE VIEW IF NOT EXISTS V (VA INTEGER, VB INTEGER) AS SELECT * FROM T WHERE
> B=200;
> UPSERT INTO V(A,B,C,D,VA,VB) VALUES (2, 200, 'def', -20, 91, 101);
> SELECT * FROM T;
> ++--+--+--+
> | A | B | C | D |
> ++--+--+--+
> | 2 | 200 | def | -20 |
> ++--+--+--+
> SELECT * FROM V;
> ++--+--+--+-+--+
> | A | B | C | D | VA | VB |
> ++--+--+--+-+--+
> | 2 | 200 | def | -20 | 91 | 101 |
> ++--+--+--+-+--+
> -- as expected
> -- drop a parent column from the view
> ALTER VIEW V DROP COLUMN C;
> SELECT * FROM V;
> +--++--+--+-+--+
> | C | A | B | D | VA | VB |
> +--++--+--+-+--+
> | def | 2 | 200 | -20 | 91 | 101 |
> +--++--+--+-+--+
> -- Column C can still be seen and its ordering is changed for some reason. If
> you run the drop column again, it is actually dropped
> ALTER VIEW V DROP COLUMN C;
> SELECT * FROM V;
> ++--+--+-+--+
> | A | B | D | VA | VB |
> ++--+--+-+--+
> | 2 | 200 | -20 | 91 | 101 |
> ++--+--+-+--+
> -- Gets dropped when drop column is run a second time.
> {code}
> When splittable SYSTEM.CATALOG rollback is enabled, we store the parent's
> column metadata along with the view as well. After the first drop column
> command, metadata for column 'C' of the parent is removed from the view's
> metadata rows however it is not marked diverged, nor is an EXCLUDED_COLUMN
> entry made for that column in the view metadata rows.
> Because of this, when resolving the view we potentially keep combining the
> parent table columns and still get column 'C'. When the second drop column
> command is issued is when we actually add an EXCLUDED_COLUMN linking row for
> 'C' in the view metadata.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (PHOENIX-6032) When phoenix.allow.system.catalog.rollback=true, a view still sees data from a column that was dropped
[
https://issues.apache.org/jira/browse/PHOENIX-6032?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chinmay Kulkarni updated PHOENIX-6032:
--
Description:
Start a 4.x server with phoenix.allow.system.catalog.rollback=true,
phoenix.system.catalog.splittable=false. Connect to it from a 4.x client with
phoenix.allow.system.catalog.rollback=true. Run the following from the 4.x
client:
{code:sql}
CREATE TABLE IF NOT EXISTS T (A INTEGER PRIMARY KEY, B INTEGER, C VARCHAR, D
INTEGER);
CREATE VIEW IF NOT EXISTS V (VA INTEGER, VB INTEGER) AS SELECT * FROM T WHERE
B=200;
UPSERT INTO V(A,B,C,D,VA,VB) VALUES (2, 200, 'def', -20, 91, 101);
SELECT * FROM T;
++--+--+--+
| A | B | C | D |
++--+--+--+
| 2 | 200 | def | -20 |
++--+--+--+
SELECT * FROM V;
++--+--+--+-+--+
| A | B | C | D | VA | VB |
++--+--+--+-+--+
| 2 | 200 | def | -20 | 91 | 101 |
++--+--+--+-+--+
-- as expected
-- drop a parent column from the view
ALTER VIEW V DROP COLUMN C;
SELECT * FROM V;
+--++--+--+-+--+
| C | A | B | D | VA | VB |
+--++--+--+-+--+
| def | 2 | 200 | -20 | 91 | 101 |
+--++--+--+-+--+
-- Column C can still be seen and its ordering is changed for some reason. If
you run the drop column again, it is actually dropped
ALTER VIEW V DROP COLUMN C;
SELECT * FROM V;
++--+--+-+--+
| A | B | D | VA | VB |
++--+--+-+--+
| 2 | 200 | -20 | 91 | 101 |
++--+--+-+--+
-- Gets dropped when drop column is run a second time.
{code}
When splittable SYSTEM.CATALOG rollback is enabled, we store the parent's
column metadata along with the view as well. After the first drop column
command, metadata for column 'C' of the parent is removed from the view's
metadata rows however it is not marked diverged, nor is an EXCLUDED_COLUMN
entry made for that column in the view metadata rows.
Because of this, when resolving the view we potentially keep combining the
parent table columns and still get column 'C'. When the second drop column
command is issued is when we actually add an EXCLUDED_COLUMN linking row for
'C' in the view metadata.
was:
Start a 4.x server with phoenix.allow.system.catalog.rollback=true,
phoenix.system.catalog.splittable=false. Connect to it from a 4.x client with
phoenix.allow.system.catalog.rollback=true. Run the following from the 4.x
client:
{code:sql}
CREATE TABLE IF NOT EXISTS T (A INTEGER PRIMARY KEY, B INTEGER, C VARCHAR, D
INTEGER);
CREATE VIEW IF NOT EXISTS V (VA INTEGER, VB INTEGER) AS SELECT * FROM T WHERE
B=200;
UPSERT INTO V(A,B,C,D,VA,VB) VALUES (2, 200, 'def', -20, 91, 101);
SELECT * FROM T;
++--+--+--+
| A | B | C | D |
++--+--+--+
| 2 | 200 | def | -20 |
++--+--+--+
SELECT * FROM V;
++--+--+--+-+--+
| A | B | C | D | VA | VB |
++--+--+--+-+--+
| 2 | 200 | def | -20 | 91 | 101 |
++--+--+--+-+--+
-- as expected
-- drop a parent column from the view
ALTER VIEW V DROP COLUMN C;
SELECT * FROM V;
+--++--+--+-+--+
| C | A | B | D | VA | VB |
+--++--+--+-+--+
| def | 2 | 200 | -20 | 91 | 101 |
+--++--+--+-+--+
-- Column C can still be seen and its ordering is changed for some reason. If
you run the drop column again, it is actually dropped
ALTER VIEW V DROP COLUMN C;
SELECT * FROM V;
++--+--+-+--+
| A | B | D | VA | VB |
++--+--+-+--+
| 2 | 200 | -20 | 91 | 101 |
++--+--+-+--+
-- Gets dropped when drop column is run a second time.
{code}
The issue seems to be that when splittable SYSTEM.CATALOG rollback is enabled,
we store the parent's column metadata along with the view as well. After the
first drop column command, metadata for column 'C' of the parent is removed
from the view's metadata rows however it is not marked diverged, nor is an
EXCLUDED_COLUMN entry made for that column in the view metadata rows.
Because of this, when resolving the view we potentially keep combining the
parent table columns and still get column 'C'. When the second drop column
command is issued is when we actually add an EXCLUDED_COLUMN linking row for
'C' in the view metadata.
> When phoenix.allow.system.catalog.rollback=true, a view still sees data from
> a column that was dropped
> --
>
> Key: PHOENIX-6032
> URL: https://issues.apache.org/jira/browse/PHOENIX-6032
> Project:
[jira] [Updated] (PHOENIX-6032) When phoenix.allow.system.catalog.rollback=true, a view still sees data from a column that was dropped
[
https://issues.apache.org/jira/browse/PHOENIX-6032?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chinmay Kulkarni updated PHOENIX-6032:
--
Description:
Start a 4.x server with phoenix.allow.system.catalog.rollback=true,
phoenix.system.catalog.splittable=false. Connect to it from a 4.x client with
phoenix.allow.system.catalog.rollback=true. Run the following from the 4.x
client:
{code:sql}
CREATE TABLE IF NOT EXISTS T (A INTEGER PRIMARY KEY, B INTEGER, C VARCHAR, D
INTEGER);
CREATE VIEW IF NOT EXISTS V (VA INTEGER, VB INTEGER) AS SELECT * FROM T WHERE
B=200;
UPSERT INTO V(A,B,C,D,VA,VB) VALUES (2, 200, 'def', -20, 91, 101);
SELECT * FROM T;
++--+--+--+
| A | B | C | D |
++--+--+--+
| 2 | 200 | def | -20 |
++--+--+--+
SELECT * FROM V;
++--+--+--+-+--+
| A | B | C | D | VA | VB |
++--+--+--+-+--+
| 2 | 200 | def | -20 | 91 | 101 |
++--+--+--+-+--+
-- as expected
-- drop a parent column from the view
ALTER VIEW V DROP COLUMN C;
SELECT * FROM V;
+--++--+--+-+--+
| C | A | B | D | VA | VB |
+--++--+--+-+--+
| def | 2 | 200 | -20 | 91 | 101 |
+--++--+--+-+--+
-- Column C can still be seen and its ordering is changed for some reason. If
you run the drop column again, it is actually dropped
ALTER VIEW V DROP COLUMN C;
SELECT * FROM V;
++--+--+-+--+
| A | B | D | VA | VB |
++--+--+-+--+
| 2 | 200 | -20 | 91 | 101 |
++--+--+-+--+
-- Gets dropped when drop column is run a second time.
{code}
The issue seems to be that when splittable SYSTEM.CATALOG rollback is enabled,
we store the parent's column metadata along with the view as well. After the
first drop column command, metadata for column 'C' of the parent is removed
from the view's metadata rows however it is not marked diverged, nor is an
EXCLUDED_COLUMN entry made for that column in the view metadata rows.
Because of this, when resolving the view we potentially keep combining the
parent table columns and still get column 'C'. When the second drop column
command is issued is when we actually add an EXCLUDED_COLUMN linking row for
'C' in the view metadata.
> When phoenix.allow.system.catalog.rollback=true, a view still sees data from
> a column that was dropped
> --
>
> Key: PHOENIX-6032
> URL: https://issues.apache.org/jira/browse/PHOENIX-6032
> Project: Phoenix
> Issue Type: Bug
>Affects Versions: 5.0.0, 4.15.0
>Reporter: Chinmay Kulkarni
>Priority: Blocker
> Fix For: 4.16.0
>
>
> Start a 4.x server with phoenix.allow.system.catalog.rollback=true,
> phoenix.system.catalog.splittable=false. Connect to it from a 4.x client with
> phoenix.allow.system.catalog.rollback=true. Run the following from the 4.x
> client:
> {code:sql}
> CREATE TABLE IF NOT EXISTS T (A INTEGER PRIMARY KEY, B INTEGER, C VARCHAR, D
> INTEGER);
> CREATE VIEW IF NOT EXISTS V (VA INTEGER, VB INTEGER) AS SELECT * FROM T WHERE
> B=200;
> UPSERT INTO V(A,B,C,D,VA,VB) VALUES (2, 200, 'def', -20, 91, 101);
> SELECT * FROM T;
> ++--+--+--+
> | A | B | C | D |
> ++--+--+--+
> | 2 | 200 | def | -20 |
> ++--+--+--+
> SELECT * FROM V;
> ++--+--+--+-+--+
> | A | B | C | D | VA | VB |
> ++--+--+--+-+--+
> | 2 | 200 | def | -20 | 91 | 101 |
> ++--+--+--+-+--+
> -- as expected
> -- drop a parent column from the view
> ALTER VIEW V DROP COLUMN C;
> SELECT * FROM V;
> +--++--+--+-+--+
> | C | A | B | D | VA | VB |
> +--++--+--+-+--+
> | def | 2 | 200 | -20 | 91 | 101 |
> +--++--+--+-+--+
> -- Column C can still be seen and its ordering is changed for some reason. If
> you run the drop column again, it is actually dropped
> ALTER VIEW V DROP COLUMN C;
> SELECT * FROM V;
> ++--+--+-+--+
> | A | B | D | VA | VB |
> ++--+--+-+--+
> | 2 | 200 | -20 | 91 | 101 |
> ++--+--+-+--+
> -- Gets dropped when drop column is run a second time.
> {code}
> The issue seems to be that when splittable SYSTEM.CATALOG rollback is
> enabled, we store the parent's column metadata along with the view as well.
> After the first drop column command, metadata for column 'C' of the parent is
> removed from the view's metadata rows however it is not marked diverged, nor
> is an EXCLUDED_COLUMN entry made for that column
