[jira] [Commented] (CALCITE-2009) Possible bug in interpreting ( IN ) OR ( IN ) logic
[ https://issues.apache.org/jira/browse/CALCITE-2009?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16318636#comment-16318636 ] Piotr Bojko commented on CALCITE-2009: -- I've created the test case in CALCITE-2088. But when I've run it against 1.16-SNAPSHOT - all was ok. When running against 1.15 - test resolves to invalid plan (pretty much similar to this issue). I've double checked my environment in my project - there was a broken build which was linking both versions of calcite as a depdency (1.15 and 1.16-SNAPSHOT). After tuning - all is ok. So the bug is already patched in 1.16. Sorry for my mistake. Reproduction on 1.15 is here - https://github.com/ptrbojko/calcite/tree/test-1.15-with-or-in end to end test in CALCITE-2088 - I will soon pull request them. > Possible bug in interpreting ( IN ) OR ( IN ) logic > - > > Key: CALCITE-2009 > URL: https://issues.apache.org/jira/browse/CALCITE-2009 > Project: Calcite > Issue Type: Bug >Affects Versions: 1.14.0, 1.13.0, 1.15.0 >Reporter: Piotr Bojko >Assignee: Julian Hyde > Fix For: 1.16.0 > > Attachments: logs-when-or-is-used.log, logs-when-union-is-used.log > > > I have schema > {code:javascript} > { > "version": "1.0", > "defaultSchema": "JIRA", > "schemas": [ > { > "name": "INTERNAL", > "type": "custom", > "factory": "%%UNDERLYING_SCHEMA_FACTORY", > "operand": {} > }, { > "name": "JIRA", > "type": "custom", > "factory": > "org.apache.calcite.schema.impl.AbstractSchema$Factory", > "operand": {}, > "tables": [ > { > "name": "ISSUES", > "type": "view", > "sql": [ > "SELECT JI.ID, JI.SUMMARY, > JI.PROJECT ", > "FROM INTERNAL.JIRAISSUE as JI > ", > "WHERE JI.ID IN (SELECT EAI.ID > FROM EXPLICIT_ALLOWED_ISSUES AS EAI) ", > "UNION ", > "SELECT JI.ID, JI.SUMMARY, > JI.PROJECT ", > "FROM INTERNAL.JIRAISSUE as JI > ", > "WHERE JI.PROJECT IN (SELECT > AP.ID FROM ALLOWED_PROJECTS AS AP)" > ] > }, > { > "name": "ALLOWED_PROJECTS", > "type": "table", > "factory": "%%DELEGATING_TABLE_FACTORY" > }, > { > "name": "EXPLICIT_ALLOWED_ISSUES", > "type": "table", > "factory": "%%DELEGATING_TABLE_FACTORY" > } > ] > } > ] > } > {code} > Where INTERNAL schema points to JDBC native JIRA Schema (through my custom > factory, but it only wraps passing datasource instead of user/password/url to > db) and JIRA schema has one view and two tables (factories generate java data > wrapped in AbstractQueryableTable) > When running "SELECT * FROM ISSUES" all works. > But when changing the view from UNION construction to OR - engine provides no > results: > {code:javascript} > { > "version": "1.0", > "defaultSchema": "JIRA", > "schemas": [ > { > "name": "INTERNAL", > "type": "custom", > "factory": "%%UNDERLYING_SCHEMA_FACTORY", > "operand": {} > }, { > "name": "JIRA", > "type": "custom", > "factory": > "org.apache.calcite.schema.impl.AbstractSchema$Factory", > "operand": {}, > "tables": [ > { > "name": "ISSUES", > "type": "view", > "sql": [ > "SELECT JI.ID, JI.SUMMARY, > JI.PROJECT ", > "FROM INTERNAL.JIRAISSUE as JI > ", > "WHERE JI.ID IN (SELECT EAI.ID > FROM EXPLICIT_ALLOWED_ISSUES AS
[jira] [Commented] (CALCITE-2009) Possible bug in interpreting ( IN ) OR ( IN ) logic
[ https://issues.apache.org/jira/browse/CALCITE-2009?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16316151#comment-16316151 ] Piotr Bojko commented on CALCITE-2009: -- Will try in CALCITE-2088 > Possible bug in interpreting ( IN ) OR ( IN ) logic > - > > Key: CALCITE-2009 > URL: https://issues.apache.org/jira/browse/CALCITE-2009 > Project: Calcite > Issue Type: Bug >Affects Versions: 1.14.0, 1.13.0, 1.15.0 >Reporter: Piotr Bojko >Assignee: Julian Hyde > Fix For: 1.16.0 > > Attachments: logs-when-or-is-used.log, logs-when-union-is-used.log > > > I have schema > {code:javascript} > { > "version": "1.0", > "defaultSchema": "JIRA", > "schemas": [ > { > "name": "INTERNAL", > "type": "custom", > "factory": "%%UNDERLYING_SCHEMA_FACTORY", > "operand": {} > }, { > "name": "JIRA", > "type": "custom", > "factory": > "org.apache.calcite.schema.impl.AbstractSchema$Factory", > "operand": {}, > "tables": [ > { > "name": "ISSUES", > "type": "view", > "sql": [ > "SELECT JI.ID, JI.SUMMARY, > JI.PROJECT ", > "FROM INTERNAL.JIRAISSUE as JI > ", > "WHERE JI.ID IN (SELECT EAI.ID > FROM EXPLICIT_ALLOWED_ISSUES AS EAI) ", > "UNION ", > "SELECT JI.ID, JI.SUMMARY, > JI.PROJECT ", > "FROM INTERNAL.JIRAISSUE as JI > ", > "WHERE JI.PROJECT IN (SELECT > AP.ID FROM ALLOWED_PROJECTS AS AP)" > ] > }, > { > "name": "ALLOWED_PROJECTS", > "type": "table", > "factory": "%%DELEGATING_TABLE_FACTORY" > }, > { > "name": "EXPLICIT_ALLOWED_ISSUES", > "type": "table", > "factory": "%%DELEGATING_TABLE_FACTORY" > } > ] > } > ] > } > {code} > Where INTERNAL schema points to JDBC native JIRA Schema (through my custom > factory, but it only wraps passing datasource instead of user/password/url to > db) and JIRA schema has one view and two tables (factories generate java data > wrapped in AbstractQueryableTable) > When running "SELECT * FROM ISSUES" all works. > But when changing the view from UNION construction to OR - engine provides no > results: > {code:javascript} > { > "version": "1.0", > "defaultSchema": "JIRA", > "schemas": [ > { > "name": "INTERNAL", > "type": "custom", > "factory": "%%UNDERLYING_SCHEMA_FACTORY", > "operand": {} > }, { > "name": "JIRA", > "type": "custom", > "factory": > "org.apache.calcite.schema.impl.AbstractSchema$Factory", > "operand": {}, > "tables": [ > { > "name": "ISSUES", > "type": "view", > "sql": [ > "SELECT JI.ID, JI.SUMMARY, > JI.PROJECT ", > "FROM INTERNAL.JIRAISSUE as JI > ", > "WHERE JI.ID IN (SELECT EAI.ID > FROM EXPLICIT_ALLOWED_ISSUES AS EAI) ", > "OR (JI.PROJECT IN (SELECT > AP.ID FROM ALLOWED_PROJECTS AS AP))" > ] > }, > { > "name": "ALLOWED_PROJECTS", > "type": "table", > "factory": "%%DELEGATING_TABLE_FACTORY" > }, > { > "name": "EXPLICIT_ALLOWED_ISSUES",
[jira] [Commented] (CALCITE-2009) Possible bug in interpreting ( IN ) OR ( IN ) logic
[ https://issues.apache.org/jira/browse/CALCITE-2009?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16313931#comment-16313931 ] Julian Hyde commented on CALCITE-2009: -- This is surprising because view expansion is usually reliable. (I'm not saying you're wrong. Just that an obvious cause doesn't spring to mind.) Have you had any luck creating a test case on other data sets? It's hard to make progress without access to your schema. > Possible bug in interpreting ( IN ) OR ( IN ) logic > - > > Key: CALCITE-2009 > URL: https://issues.apache.org/jira/browse/CALCITE-2009 > Project: Calcite > Issue Type: Bug >Affects Versions: 1.14.0, 1.13.0, 1.15.0 >Reporter: Piotr Bojko >Assignee: Julian Hyde > Fix For: 1.16.0 > > Attachments: logs-when-or-is-used.log, logs-when-union-is-used.log > > > I have schema > {code:javascript} > { > "version": "1.0", > "defaultSchema": "JIRA", > "schemas": [ > { > "name": "INTERNAL", > "type": "custom", > "factory": "%%UNDERLYING_SCHEMA_FACTORY", > "operand": {} > }, { > "name": "JIRA", > "type": "custom", > "factory": > "org.apache.calcite.schema.impl.AbstractSchema$Factory", > "operand": {}, > "tables": [ > { > "name": "ISSUES", > "type": "view", > "sql": [ > "SELECT JI.ID, JI.SUMMARY, > JI.PROJECT ", > "FROM INTERNAL.JIRAISSUE as JI > ", > "WHERE JI.ID IN (SELECT EAI.ID > FROM EXPLICIT_ALLOWED_ISSUES AS EAI) ", > "UNION ", > "SELECT JI.ID, JI.SUMMARY, > JI.PROJECT ", > "FROM INTERNAL.JIRAISSUE as JI > ", > "WHERE JI.PROJECT IN (SELECT > AP.ID FROM ALLOWED_PROJECTS AS AP)" > ] > }, > { > "name": "ALLOWED_PROJECTS", > "type": "table", > "factory": "%%DELEGATING_TABLE_FACTORY" > }, > { > "name": "EXPLICIT_ALLOWED_ISSUES", > "type": "table", > "factory": "%%DELEGATING_TABLE_FACTORY" > } > ] > } > ] > } > {code} > Where INTERNAL schema points to JDBC native JIRA Schema (through my custom > factory, but it only wraps passing datasource instead of user/password/url to > db) and JIRA schema has one view and two tables (factories generate java data > wrapped in AbstractQueryableTable) > When running "SELECT * FROM ISSUES" all works. > But when changing the view from UNION construction to OR - engine provides no > results: > {code:javascript} > { > "version": "1.0", > "defaultSchema": "JIRA", > "schemas": [ > { > "name": "INTERNAL", > "type": "custom", > "factory": "%%UNDERLYING_SCHEMA_FACTORY", > "operand": {} > }, { > "name": "JIRA", > "type": "custom", > "factory": > "org.apache.calcite.schema.impl.AbstractSchema$Factory", > "operand": {}, > "tables": [ > { > "name": "ISSUES", > "type": "view", > "sql": [ > "SELECT JI.ID, JI.SUMMARY, > JI.PROJECT ", > "FROM INTERNAL.JIRAISSUE as JI > ", > "WHERE JI.ID IN (SELECT EAI.ID > FROM EXPLICIT_ALLOWED_ISSUES AS EAI) ", > "OR (JI.PROJECT IN (SELECT > AP.ID FROM ALLOWED_PROJECTS AS AP))" > ] > }, > { > "name": "ALLOWED_PROJECTS", >
[jira] [Commented] (CALCITE-2009) Possible bug in interpreting ( IN ) OR ( IN ) logic
[ https://issues.apache.org/jira/browse/CALCITE-2009?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16313218#comment-16313218 ] Piotr Bojko commented on CALCITE-2009: -- This issue is strictly dependent on to whether the table view (with in-or-in) is involved or the query does is raw (without tables view). Any hints would be appreciated > Possible bug in interpreting ( IN ) OR ( IN ) logic > - > > Key: CALCITE-2009 > URL: https://issues.apache.org/jira/browse/CALCITE-2009 > Project: Calcite > Issue Type: Bug >Affects Versions: 1.14.0, 1.13.0, 1.15.0 >Reporter: Piotr Bojko >Assignee: Julian Hyde > Fix For: 1.16.0 > > Attachments: logs-when-or-is-used.log, logs-when-union-is-used.log > > > I have schema > {code:javascript} > { > "version": "1.0", > "defaultSchema": "JIRA", > "schemas": [ > { > "name": "INTERNAL", > "type": "custom", > "factory": "%%UNDERLYING_SCHEMA_FACTORY", > "operand": {} > }, { > "name": "JIRA", > "type": "custom", > "factory": > "org.apache.calcite.schema.impl.AbstractSchema$Factory", > "operand": {}, > "tables": [ > { > "name": "ISSUES", > "type": "view", > "sql": [ > "SELECT JI.ID, JI.SUMMARY, > JI.PROJECT ", > "FROM INTERNAL.JIRAISSUE as JI > ", > "WHERE JI.ID IN (SELECT EAI.ID > FROM EXPLICIT_ALLOWED_ISSUES AS EAI) ", > "UNION ", > "SELECT JI.ID, JI.SUMMARY, > JI.PROJECT ", > "FROM INTERNAL.JIRAISSUE as JI > ", > "WHERE JI.PROJECT IN (SELECT > AP.ID FROM ALLOWED_PROJECTS AS AP)" > ] > }, > { > "name": "ALLOWED_PROJECTS", > "type": "table", > "factory": "%%DELEGATING_TABLE_FACTORY" > }, > { > "name": "EXPLICIT_ALLOWED_ISSUES", > "type": "table", > "factory": "%%DELEGATING_TABLE_FACTORY" > } > ] > } > ] > } > {code} > Where INTERNAL schema points to JDBC native JIRA Schema (through my custom > factory, but it only wraps passing datasource instead of user/password/url to > db) and JIRA schema has one view and two tables (factories generate java data > wrapped in AbstractQueryableTable) > When running "SELECT * FROM ISSUES" all works. > But when changing the view from UNION construction to OR - engine provides no > results: > {code:javascript} > { > "version": "1.0", > "defaultSchema": "JIRA", > "schemas": [ > { > "name": "INTERNAL", > "type": "custom", > "factory": "%%UNDERLYING_SCHEMA_FACTORY", > "operand": {} > }, { > "name": "JIRA", > "type": "custom", > "factory": > "org.apache.calcite.schema.impl.AbstractSchema$Factory", > "operand": {}, > "tables": [ > { > "name": "ISSUES", > "type": "view", > "sql": [ > "SELECT JI.ID, JI.SUMMARY, > JI.PROJECT ", > "FROM INTERNAL.JIRAISSUE as JI > ", > "WHERE JI.ID IN (SELECT EAI.ID > FROM EXPLICIT_ALLOWED_ISSUES AS EAI) ", > "OR (JI.PROJECT IN (SELECT > AP.ID FROM ALLOWED_PROJECTS AS AP))" > ] > }, > { > "name": "ALLOWED_PROJECTS", > "type": "table", > "factory":
[jira] [Commented] (CALCITE-2009) Possible bug in interpreting ( IN ) OR ( IN ) logic
[ https://issues.apache.org/jira/browse/CALCITE-2009?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16313176#comment-16313176 ] Piotr Bojko commented on CALCITE-2009: -- Update. The problem is somewhere at the SqlToRelConverter.convertQuery level. The query {code} select * from ISSUES {code} is resolved to {code} SELECT `JI`.`ID`, `JI`.`SUMMARY`, `JI`.`PROJECT`, `JI`.`ISSUENUM` AS `NUM`, `JI`.`CREATOR`, `JI`.`ASSIGNEE`, `JI`.`REPORTER`, `JI`.`CREATED`, `JI`.`UPDATED`, `JI`.`DUEDATE` AS `DUE`, `JI`.`RESOLUTIONDATE` AS `RESOLVED`, `JI`.`DESCRIPTION`, `JI`.`ENVIRONMENT`, `JI`.`ISSUETYPE` AS `TYPE`, `JI`.`PRIORITY`, `JI`.`RESOLUTION`, `JI`.`ISSUESTATUS` AS `STATUS`, `JI`.`SECURITY` AS `SECURITYLEVEL`, `JI`.`VOTES`, `JI`.`WATCHES`, `JI`.`TIMEORIGINALESTIMATE` AS `ORIGINALESTIMATE`, `JI`.`TIMEESTIMATE` AS `ESTIMATE`, `JI`.`TIMESPENT` FROM `INTERNAL`.`JIRAISSUE` AS `JI` WHERE `JI`.`ID` IN (SELECT `EAI`.`ID` FROM `EXPLICIT_ALLOWED_ISSUES` AS `EAI`) OR `JI`.`PROJECT` IN (SELECT `AP`.`ID` FROM `EXPLICIT_ALLOWED_PROJECTS` AS `AP`) {code} which is resolved to {code} LogicalProject(ID=[$0], SUMMARY=[$8], PROJECT=[$3], NUM=[$2], CREATOR=[$6], ASSIGNEE=[$5], REPORTER=[$4], CREATED=[$14], UPDATED=[$15], DUE=[$16], RESOLVED=[$17], DESCRIPTION=[$9], ENVIRONMENT=[$10], TYPE=[$7], PRIORITY=[$11], RESOLUTION=[$12], STATUS=[$13], SECURITYLEVEL=[$24], VOTES=[$18], WATCHES=[$19], ORIGINALESTIMATE=[$20], ESTIMATE=[$21], TIMESPENT=[$22]) LogicalFilter(condition=[OR(true, true)]) LogicalJoin(condition=[=($3, $28)], joinType=[inner]) LogicalJoin(condition=[=($0, $27)], joinType=[inner]) JdbcTableScan(table=[[INTERNAL, JIRAISSUE]]) LogicalAggregate(group=[{0}]) LogicalProject(ID=[$0]) EnumerableTableScan(table=[[JIRA, EXPLICIT_ALLOWED_ISSUES]]) LogicalAggregate(group=[{0}]) LogicalProject(ID=[$0]) EnumerableTableScan(table=[[JIRA, EXPLICIT_ALLOWED_PROJECTS]]) {code} which is invalid. but when pasting resolved query (not using the declared view ISSUES) explicitly, plan resolves to: {code} LogicalProject(ID=[$0], SUMMARY=[$8], PROJECT=[$3], NUM=[$2], CREATOR=[$6], ASSIGNEE=[$5], REPORTER=[$4], CREATED=[$14], UPDATED=[$15], DUE=[$16], RESOLVED=[$17], DESCRIPTION=[$9], ENVIRONMENT=[$10], TYPE=[$7], PRIORITY=[$11], RESOLUTION=[$12], STATUS=[$13], SECURITYLEVEL=[$24], VOTES=[$18], WATCHES=[$19], ORIGINALESTIMATE=[$20], ESTIMATE=[$21], TIMESPENT=[$22]) LogicalFilter(condition=[OR(IN($0, {LogicalProject(ID=[$0]) EnumerableTableScan(table=[[JIRA, EXPLICIT_ALLOWED_ISSUES]]) }), IN($3, { LogicalProject(ID=[$0]) EnumerableTableScan(table=[[JIRA, EXPLICIT_ALLOWED_PROJECTS]]) }))]) JdbcTableScan(table=[[INTERNAL, JIRAISSUE]]) {code} which is valid. > Possible bug in interpreting ( IN ) OR ( IN ) logic > - > > Key: CALCITE-2009 > URL: https://issues.apache.org/jira/browse/CALCITE-2009 > Project: Calcite > Issue Type: Bug >Affects Versions: 1.14.0, 1.13.0, 1.15.0 >Reporter: Piotr Bojko >Assignee: Julian Hyde > Fix For: 1.16.0 > > Attachments: logs-when-or-is-used.log, logs-when-union-is-used.log > > > I have schema > {code:javascript} > { > "version": "1.0", > "defaultSchema": "JIRA", > "schemas": [ > { > "name": "INTERNAL", > "type": "custom", > "factory": "%%UNDERLYING_SCHEMA_FACTORY", > "operand": {} > }, { > "name": "JIRA", > "type": "custom", > "factory": > "org.apache.calcite.schema.impl.AbstractSchema$Factory", > "operand": {}, > "tables": [ > { > "name": "ISSUES", > "type": "view", > "sql": [ > "SELECT JI.ID, JI.SUMMARY, > JI.PROJECT ", > "FROM INTERNAL.JIRAISSUE as JI > ", > "WHERE JI.ID IN (SELECT EAI.ID > FROM EXPLICIT_ALLOWED_ISSUES AS EAI) ", > "UNION ", > "SELECT JI.ID, JI.SUMMARY, > JI.PROJECT ", > "FROM INTERNAL.JIRAISSUE as JI > ", > "WHERE JI.PROJECT IN (SELECT > AP.ID FROM ALLOWED_PROJECTS AS AP)" > ] > }, > { > "name": "ALLOWED_PROJECTS", >
[jira] [Commented] (CALCITE-2009) Possible bug in interpreting ( IN ) OR ( IN ) logic
[ https://issues.apache.org/jira/browse/CALCITE-2009?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16311520#comment-16311520 ] Piotr Bojko commented on CALCITE-2009: -- I've checked that before volcano planner comes in - the plan is as follows: {code} LogicalProject(subset=[rel#8075:Subset#11.ENUMERABLE.[]], ID=[$0], SUMMARY=[$7], PROJECT=[$2], NUM=[$1], CREATOR=[$5], ASSIGNEE=[$4], REPORTER=[$3], CREATED=[$13], UPDATED=[$14], DUE=[$15], RESOLVED=[$16], DESCRIPTION=[$8], ENVIRONMENT=[$9], TYPE=[$6], PRIORITY=[$10], RESOLUTION=[$11], STATUS=[$12], SECURITYLEVEL=[$22], VOTES=[$17], WATCHES=[$18], ORIGINALESTIMATE=[$19], ESTIMATE=[$20], TIMESPENT=[$21]) LogicalFilter(subset=[rel#8063:Subset#10.NONE.[]], condition=[OR(true, true)]) LogicalJoin(subset=[rel#8061:Subset#9.NONE.[]], condition=[=($2, $24)], joinType=[inner]) LogicalJoin(subset=[rel#8054:Subset#5.NONE.[]], condition=[=($0, $23)], joinType=[inner]) LogicalProject(subset=[rel#8047:Subset#1.NONE.[]], ID=[$0], ISSUENUM=[$2], PROJECT=[$3], REPORTER=[$4], ASSIGNEE=[$5], CREATOR=[$6], ISSUETYPE=[$7], SUMMARY=[$8], DESCRIPTION=[$9], ENVIRONMENT=[$10], PRIORITY=[$11], RESOLUTION=[$12], ISSUESTATUS=[$13], CREATED=[$14], UPDATED=[$15], DUEDATE=[$16], RESOLUTIONDATE=[$17], VOTES=[$18], WATCHES=[$19], TIMEORIGINALESTIMATE=[$20], TIMEESTIMATE=[$21], TIMESPENT=[$22], SECURITY=[$24]) JdbcTableScan(subset=[rel#8045:Subset#0.JDBC.INTERNAL.[]], table=[[INTERNAL, JIRAISSUE]]) LogicalAggregate(subset=[rel#8052:Subset#4.NONE.[]], group=[{0}]) LogicalProject(subset=[rel#8050:Subset#3.NONE.[]], ID=[$0]) EnumerableTableScan(subset=[rel#8048:Subset#2.ENUMERABLE.[]], table=[[JIRA, EXPLICIT_ALLOWED_ISSUES]]) LogicalAggregate(subset=[rel#8059:Subset#8.NONE.[]], group=[{0}]) LogicalProject(subset=[rel#8057:Subset#7.NONE.[]], ID=[$0]) EnumerableTableScan(subset=[rel#8055:Subset#6.ENUMERABLE.[]], table=[[JIRA, EXPLICIT_ALLOWED_PROJECTS]]) {code} There are two inner joins where both should be left, I think. So, the possible bug is somewhere between parsing query and producing relnode tree. > Possible bug in interpreting ( IN ) OR ( IN ) logic > - > > Key: CALCITE-2009 > URL: https://issues.apache.org/jira/browse/CALCITE-2009 > Project: Calcite > Issue Type: Bug >Affects Versions: 1.14.0, 1.13.0, 1.15.0 >Reporter: Piotr Bojko >Assignee: Julian Hyde > Fix For: 1.16.0 > > Attachments: logs-when-or-is-used.log, logs-when-union-is-used.log > > > I have schema > {code:javascript} > { > "version": "1.0", > "defaultSchema": "JIRA", > "schemas": [ > { > "name": "INTERNAL", > "type": "custom", > "factory": "%%UNDERLYING_SCHEMA_FACTORY", > "operand": {} > }, { > "name": "JIRA", > "type": "custom", > "factory": > "org.apache.calcite.schema.impl.AbstractSchema$Factory", > "operand": {}, > "tables": [ > { > "name": "ISSUES", > "type": "view", > "sql": [ > "SELECT JI.ID, JI.SUMMARY, > JI.PROJECT ", > "FROM INTERNAL.JIRAISSUE as JI > ", > "WHERE JI.ID IN (SELECT EAI.ID > FROM EXPLICIT_ALLOWED_ISSUES AS EAI) ", > "UNION ", > "SELECT JI.ID, JI.SUMMARY, > JI.PROJECT ", > "FROM INTERNAL.JIRAISSUE as JI > ", > "WHERE JI.PROJECT IN (SELECT > AP.ID FROM ALLOWED_PROJECTS AS AP)" > ] > }, > { > "name": "ALLOWED_PROJECTS", > "type": "table", > "factory": "%%DELEGATING_TABLE_FACTORY" > }, > { > "name": "EXPLICIT_ALLOWED_ISSUES", > "type": "table", > "factory": "%%DELEGATING_TABLE_FACTORY" > } > ] > } > ] > } > {code} > Where INTERNAL schema points to JDBC native JIRA Schema (through my custom > factory, but it only wraps passing
[jira] [Commented] (CALCITE-2009) Possible bug in interpreting ( IN ) OR ( IN ) logic
[ https://issues.apache.org/jira/browse/CALCITE-2009?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16292601#comment-16292601 ] Piotr Bojko commented on CALCITE-2009: -- I've compared the plans for both queries. The good one (with proper result data): {code} explain plan for select * from ( SELECT JI.ID, JI.SUMMARY, JI.PROJECT, JI.ISSUENUM AS NUM, JI.CREATOR, JI.ASSIGNEE, JI.REPORTER, JI.CREATED, JI.UPDATED, JI.DUEDATE AS DUE, JI.RESOLUTIONDATE AS RESOLVED, JI.DESCRIPTION, JI.ENVIRONMENT, JI.ISSUETYPE AS TYPE, JI.PRIORITY, JI.RESOLUTION, JI.ISSUESTATUS AS STATUS, JI.SECURITY AS SECURITYLEVEL, JI.VOTES, JI.WATCHES, JI.TIMEORIGINALESTIMATE AS ORIGINALESTIMATE, JI.TIMEESTIMATE AS ESTIMATE, JI.TIMESPENT FROM INTERNAL.JIRAISSUE as JI WHERE JI.ID IN (SELECT EAI.ID FROM EXPLICIT_ALLOWED_ISSUES AS EAI) OR JI.PROJECT IN (SELECT AP.ID FROM EXPLICIT_ALLOWED_PROJECTS AS AP)) {code} produces plan: {code} EnumerableCalc(expr#0..27=[{inputs}], expr#28=[IS NOT NULL($t24)], expr#29=[IS NOT NULL($t27)], expr#30=[0], expr#31=[=($t25, $t30)], expr#32=[NOT($t31)], expr#33=[AND($t29, $t32)], expr#34=[CAST($t33):BOOLEAN], expr#35=[OR($t28, $t34)], ID=[$t0], SUMMARY=[$t7], PROJECT=[$t2], NUM=[$t1], CREATOR=[$t5], ASSIGNEE=[$t4], REPORTER=[$t3], CREATED=[$t13], UPDATED=[$t14], DUE=[$t15], RESOLVED=[$t16], DESCRIPTION=[$t8], ENVIRONMENT=[$t9], TYPE=[$t6], PRIORITY=[$t10], RESOLUTION=[$t11], STATUS=[$t12], SECURITYLEVEL=[$t22], VOTES=[$t17], WATCHES=[$t18], ORIGINALESTIMATE=[$t19], ESTIMATE=[$t20], TIMESPENT=[$t21], $condition=[$t35]) EnumerableJoin(condition=[=($2, $26)], joinType=[left]) EnumerableJoin(condition=[true], joinType=[inner]) EnumerableJoin(condition=[=($0, $23)], joinType=[left]) JdbcToEnumerableConverter JdbcProject(ID=[$0], ISSUENUM=[$2], PROJECT=[$3], REPORTER=[$4], ASSIGNEE=[$5], CREATOR=[$6], ISSUETYPE=[$7], SUMMARY=[$8], DESCRIPTION=[$9], ENVIRONMENT=[$10], PRIORITY=[$11], RESOLUTION=[$12], ISSUESTATUS=[$13], CREATED=[$14], UPDATED=[$15], DUEDATE=[$16], RESOLUTIONDATE=[$17], VOTES=[$18], WATCHES=[$19], TIMEORIGINALESTIMATE=[$20], TIMEESTIMATE=[$21], TIMESPENT=[$22], SECURITY=[$24]) JdbcTableScan(table=[[INTERNAL, JIRAISSUE]]) EnumerableAggregate(group=[{0, 1}]) EnumerableCalc(expr#0=[{inputs}], expr#1=[true], proj#0..1=[{exprs}]) EnumerableTableScan(table=[[JIRA, EXPLICIT_ALLOWED_ISSUES]]) EnumerableAggregate(group=[{}], c=[COUNT()]) EnumerableTableScan(table=[[JIRA, EXPLICIT_ALLOWED_PROJECTS]]) EnumerableAggregate(group=[{0, 1}]) EnumerableCalc(expr#0=[{inputs}], expr#1=[true], proj#0..1=[{exprs}]) EnumerableTableScan(table=[[JIRA, EXPLICIT_ALLOWED_PROJECTS]]) {code} and following query with use of a view: {code} explain plan for SELECT * FROM JIRA.ISSUES; {code} produces plan {code} EnumerableCalc(expr#0..23=[{inputs}], ID=[$t0], SUMMARY=[$t7], PROJECT=[$t2], NUM=[$t1], CREATOR=[$t5], ASSIGNEE=[$t4], REPORTER=[$t3], CREATED=[$t13], UPDATED=[$t14], DUE=[$t15], RESOLVED=[$t16], DESCRIPTION=[$t8], ENVIRONMENT=[$t9], TYPE=[$t6], PRIORITY=[$t10], RESOLUTION=[$t11], STATUS=[$t12], SECURITYLEVEL=[$t22], VOTES=[$t17], WATCHES=[$t18], ORIGINALESTIMATE=[$t19], ESTIMATE=[$t20], TIMESPENT=[$t21]) EnumerableSemiJoin(condition=[=($0, $24)], joinType=[inner]) EnumerableCalc(expr#0..23=[{inputs}], ID=[$t1], ISSUENUM=[$t2], PROJECT=[$t3], REPORTER=[$t4], ASSIGNEE=[$t5], CREATOR=[$t6], ISSUETYPE=[$t7], SUMMARY=[$t8], DESCRIPTION=[$t9], ENVIRONMENT=[$t10], PRIORITY=[$t11], RESOLUTION=[$t12], ISSUESTATUS=[$t13], CREATED=[$t14], UPDATED=[$t15], DUEDATE=[$t16], RESOLUTIONDATE=[$t17], VOTES=[$t18], WATCHES=[$t19], TIMEORIGINALESTIMATE=[$t20], TIMEESTIMATE=[$t21], TIMESPENT=[$t22], SECURITY=[$t23], ID0=[$t0]) EnumerableJoin(condition=[=($0, $3)], joinType=[inner]) EnumerableAggregate(group=[{0}]) EnumerableTableScan(table=[[JIRA, EXPLICIT_ALLOWED_PROJECTS]]) JdbcToEnumerableConverter JdbcProject(ID=[$0], ISSUENUM=[$2], PROJECT=[$3], REPORTER=[$4], ASSIGNEE=[$5], CREATOR=[$6], ISSUETYPE=[$7], SUMMARY=[$8], DESCRIPTION=[$9], ENVIRONMENT=[$10], PRIORITY=[$11], RESOLUTION=[$12], ISSUESTATUS=[$13], CREATED=[$14], UPDATED=[$15], DUEDATE=[$16], RESOLUTIONDATE=[$17], VOTES=[$18], WATCHES=[$19], TIMEORIGINALESTIMATE=[$20], TIMEESTIMATE=[$21], TIMESPENT=[$22], SECURITY=[$24]) JdbcTableScan(table=[[INTERNAL, JIRAISSUE]]) EnumerableTableScan(table=[[JIRA, EXPLICIT_ALLOWED_ISSUES]]) {code} > Possible bug in interpreting ( IN ) OR ( IN ) logic > - > > Key: CALCITE-2009 > URL: https://issues.apache.org/jira/browse/CALCITE-2009 > Project: Calcite > Issue Type: Bug >Affects Versions: 1.14.0, 1.13.0, 1.15.0 >Reporter: Piotr Bojko >Assignee:
[jira] [Commented] (CALCITE-2009) Possible bug in interpreting ( IN ) OR ( IN ) logic
[ https://issues.apache.org/jira/browse/CALCITE-2009?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16292372#comment-16292372 ] Piotr Bojko commented on CALCITE-2009: -- I've checked - but sill no green. Please note, that there is a difference whether such in-or-in construction occurs in query or in view. Only when it is implemented as view it goes red (no results, just as CALCITE-2071) > Possible bug in interpreting ( IN ) OR ( IN ) logic > - > > Key: CALCITE-2009 > URL: https://issues.apache.org/jira/browse/CALCITE-2009 > Project: Calcite > Issue Type: Bug >Affects Versions: 1.14.0, 1.13.0, 1.15.0 >Reporter: Piotr Bojko >Assignee: Julian Hyde > Fix For: 1.16.0 > > Attachments: logs-when-or-is-used.log, logs-when-union-is-used.log > > > I have schema > {code:javascript} > { > "version": "1.0", > "defaultSchema": "JIRA", > "schemas": [ > { > "name": "INTERNAL", > "type": "custom", > "factory": "%%UNDERLYING_SCHEMA_FACTORY", > "operand": {} > }, { > "name": "JIRA", > "type": "custom", > "factory": > "org.apache.calcite.schema.impl.AbstractSchema$Factory", > "operand": {}, > "tables": [ > { > "name": "ISSUES", > "type": "view", > "sql": [ > "SELECT JI.ID, JI.SUMMARY, > JI.PROJECT ", > "FROM INTERNAL.JIRAISSUE as JI > ", > "WHERE JI.ID IN (SELECT EAI.ID > FROM EXPLICIT_ALLOWED_ISSUES AS EAI) ", > "UNION ", > "SELECT JI.ID, JI.SUMMARY, > JI.PROJECT ", > "FROM INTERNAL.JIRAISSUE as JI > ", > "WHERE JI.PROJECT IN (SELECT > AP.ID FROM ALLOWED_PROJECTS AS AP)" > ] > }, > { > "name": "ALLOWED_PROJECTS", > "type": "table", > "factory": "%%DELEGATING_TABLE_FACTORY" > }, > { > "name": "EXPLICIT_ALLOWED_ISSUES", > "type": "table", > "factory": "%%DELEGATING_TABLE_FACTORY" > } > ] > } > ] > } > {code} > Where INTERNAL schema points to JDBC native JIRA Schema (through my custom > factory, but it only wraps passing datasource instead of user/password/url to > db) and JIRA schema has one view and two tables (factories generate java data > wrapped in AbstractQueryableTable) > When running "SELECT * FROM ISSUES" all works. > But when changing the view from UNION construction to OR - engine provides no > results: > {code:javascript} > { > "version": "1.0", > "defaultSchema": "JIRA", > "schemas": [ > { > "name": "INTERNAL", > "type": "custom", > "factory": "%%UNDERLYING_SCHEMA_FACTORY", > "operand": {} > }, { > "name": "JIRA", > "type": "custom", > "factory": > "org.apache.calcite.schema.impl.AbstractSchema$Factory", > "operand": {}, > "tables": [ > { > "name": "ISSUES", > "type": "view", > "sql": [ > "SELECT JI.ID, JI.SUMMARY, > JI.PROJECT ", > "FROM INTERNAL.JIRAISSUE as JI > ", > "WHERE JI.ID IN (SELECT EAI.ID > FROM EXPLICIT_ALLOWED_ISSUES AS EAI) ", > "OR (JI.PROJECT IN (SELECT > AP.ID FROM ALLOWED_PROJECTS AS AP))" > ] > }, > { > "name": "ALLOWED_PROJECTS", > "type": "table", >
[jira] [Commented] (CALCITE-2009) Possible bug in interpreting ( IN ) OR ( IN ) logic
[ https://issues.apache.org/jira/browse/CALCITE-2009?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16287017#comment-16287017 ] Julian Hyde commented on CALCITE-2009: -- Might this be a duplicate of CALCITE-2071? Can you test against the proposed fix, https://github.com/vineetgarg02/calcite/tree/CALCITE-2071-my-attempt. > Possible bug in interpreting ( IN ) OR ( IN ) logic > - > > Key: CALCITE-2009 > URL: https://issues.apache.org/jira/browse/CALCITE-2009 > Project: Calcite > Issue Type: Bug >Affects Versions: 1.14.0, 1.13.0, 1.15.0 >Reporter: Piotr Bojko >Assignee: Julian Hyde > Attachments: logs-when-or-is-used.log, logs-when-union-is-used.log > > > I have schema > {code:javascript} > { > "version": "1.0", > "defaultSchema": "JIRA", > "schemas": [ > { > "name": "INTERNAL", > "type": "custom", > "factory": "%%UNDERLYING_SCHEMA_FACTORY", > "operand": {} > }, { > "name": "JIRA", > "type": "custom", > "factory": > "org.apache.calcite.schema.impl.AbstractSchema$Factory", > "operand": {}, > "tables": [ > { > "name": "ISSUES", > "type": "view", > "sql": [ > "SELECT JI.ID, JI.SUMMARY, > JI.PROJECT ", > "FROM INTERNAL.JIRAISSUE as JI > ", > "WHERE JI.ID IN (SELECT EAI.ID > FROM EXPLICIT_ALLOWED_ISSUES AS EAI) ", > "UNION ", > "SELECT JI.ID, JI.SUMMARY, > JI.PROJECT ", > "FROM INTERNAL.JIRAISSUE as JI > ", > "WHERE JI.PROJECT IN (SELECT > AP.ID FROM ALLOWED_PROJECTS AS AP)" > ] > }, > { > "name": "ALLOWED_PROJECTS", > "type": "table", > "factory": "%%DELEGATING_TABLE_FACTORY" > }, > { > "name": "EXPLICIT_ALLOWED_ISSUES", > "type": "table", > "factory": "%%DELEGATING_TABLE_FACTORY" > } > ] > } > ] > } > {code} > Where INTERNAL schema points to JDBC native JIRA Schema (through my custom > factory, but it only wraps passing datasource instead of user/password/url to > db) and JIRA schema has one view and two tables (factories generate java data > wrapped in AbstractQueryableTable) > When running "SELECT * FROM ISSUES" all works. > But when changing the view from UNION construction to OR - engine provides no > results: > {code:javascript} > { > "version": "1.0", > "defaultSchema": "JIRA", > "schemas": [ > { > "name": "INTERNAL", > "type": "custom", > "factory": "%%UNDERLYING_SCHEMA_FACTORY", > "operand": {} > }, { > "name": "JIRA", > "type": "custom", > "factory": > "org.apache.calcite.schema.impl.AbstractSchema$Factory", > "operand": {}, > "tables": [ > { > "name": "ISSUES", > "type": "view", > "sql": [ > "SELECT JI.ID, JI.SUMMARY, > JI.PROJECT ", > "FROM INTERNAL.JIRAISSUE as JI > ", > "WHERE JI.ID IN (SELECT EAI.ID > FROM EXPLICIT_ALLOWED_ISSUES AS EAI) ", > "OR (JI.PROJECT IN (SELECT > AP.ID FROM ALLOWED_PROJECTS AS AP))" > ] > }, > { > "name": "ALLOWED_PROJECTS", > "type": "table", > "factory": "%%DELEGATING_TABLE_FACTORY" > }, >
[jira] [Commented] (CALCITE-2009) Possible bug in interpreting ( IN ) OR ( IN ) logic
[ https://issues.apache.org/jira/browse/CALCITE-2009?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16272685#comment-16272685 ] Piotr Bojko commented on CALCITE-2009: -- 1.15.0-SNAPSHOT has the same issue. > Possible bug in interpreting ( IN ) OR ( IN ) logic > - > > Key: CALCITE-2009 > URL: https://issues.apache.org/jira/browse/CALCITE-2009 > Project: Calcite > Issue Type: Bug >Affects Versions: 1.13.0, 1.14.0 >Reporter: Piotr Bojko >Assignee: Julian Hyde > Attachments: logs-when-or-is-used.log, logs-when-union-is-used.log > > > I have schema > {code:javascript} > { > "version": "1.0", > "defaultSchema": "JIRA", > "schemas": [ > { > "name": "INTERNAL", > "type": "custom", > "factory": "%%UNDERLYING_SCHEMA_FACTORY", > "operand": {} > }, { > "name": "JIRA", > "type": "custom", > "factory": > "org.apache.calcite.schema.impl.AbstractSchema$Factory", > "operand": {}, > "tables": [ > { > "name": "ISSUES", > "type": "view", > "sql": [ > "SELECT JI.ID, JI.SUMMARY, > JI.PROJECT ", > "FROM INTERNAL.JIRAISSUE as JI > ", > "WHERE JI.ID IN (SELECT EAI.ID > FROM EXPLICIT_ALLOWED_ISSUES AS EAI) ", > "UNION ", > "SELECT JI.ID, JI.SUMMARY, > JI.PROJECT ", > "FROM INTERNAL.JIRAISSUE as JI > ", > "WHERE JI.PROJECT IN (SELECT > AP.ID FROM ALLOWED_PROJECTS AS AP)" > ] > }, > { > "name": "ALLOWED_PROJECTS", > "type": "table", > "factory": "%%DELEGATING_TABLE_FACTORY" > }, > { > "name": "EXPLICIT_ALLOWED_ISSUES", > "type": "table", > "factory": "%%DELEGATING_TABLE_FACTORY" > } > ] > } > ] > } > {code} > Where INTERNAL schema points to JDBC native JIRA Schema (through my custom > factory, but it only wraps passing datasource instead of user/password/url to > db) and JIRA schema has one view and two tables (factories generate java data > wrapped in AbstractQueryableTable) > When running "SELECT * FROM ISSUES" all works. > But when changing the view from UNION construction to OR - engine provides no > results: > {code:javascript} > { > "version": "1.0", > "defaultSchema": "JIRA", > "schemas": [ > { > "name": "INTERNAL", > "type": "custom", > "factory": "%%UNDERLYING_SCHEMA_FACTORY", > "operand": {} > }, { > "name": "JIRA", > "type": "custom", > "factory": > "org.apache.calcite.schema.impl.AbstractSchema$Factory", > "operand": {}, > "tables": [ > { > "name": "ISSUES", > "type": "view", > "sql": [ > "SELECT JI.ID, JI.SUMMARY, > JI.PROJECT ", > "FROM INTERNAL.JIRAISSUE as JI > ", > "WHERE JI.ID IN (SELECT EAI.ID > FROM EXPLICIT_ALLOWED_ISSUES AS EAI) ", > "OR (JI.PROJECT IN (SELECT > AP.ID FROM ALLOWED_PROJECTS AS AP))" > ] > }, > { > "name": "ALLOWED_PROJECTS", > "type": "table", > "factory": "%%DELEGATING_TABLE_FACTORY" > }, > { > "name": "EXPLICIT_ALLOWED_ISSUES", >
[jira] [Commented] (CALCITE-2009) Possible bug in interpreting ( IN ) OR ( IN ) logic
[ https://issues.apache.org/jira/browse/CALCITE-2009?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16272684#comment-16272684 ] Piotr Bojko commented on CALCITE-2009: -- I've discovered that when launching the followinf query alone - all works ok: {code:sql} SELECT JI.ID, JI.SUMMARY, JI.PROJECT FROM INTERNAL.JIRAISSUE as JI WHERE JI.ID IN (SELECT EAI.ID FROM EXPLICIT_ALLOWED_ISSUES AS EAI) OR (JI.PROJECT IN (SELECT AP.ID FROM ALLOWED_PROJECTS AS AP)) {code} but if the same query is used as a view - then results are not the same (empty results for select * from issues) {code:javascript} { "name": "ISSUES", "type": "view", "sql": [ "SELECT JI.ID, JI.SUMMARY, JI.PROJECT ", "FROM INTERNAL.JIRAISSUE as JI ", "WHERE JI.ID IN (SELECT EAI.ID FROM EXPLICIT_ALLOWED_ISSUES AS EAI) ", "OR (JI.PROJECT IN (SELECT AP.ID FROM ALLOWED_PROJECTS AS AP))" ] } {code} > Possible bug in interpreting ( IN ) OR ( IN ) logic > - > > Key: CALCITE-2009 > URL: https://issues.apache.org/jira/browse/CALCITE-2009 > Project: Calcite > Issue Type: Bug >Affects Versions: 1.13.0, 1.14.0 >Reporter: Piotr Bojko >Assignee: Julian Hyde > Attachments: logs-when-or-is-used.log, logs-when-union-is-used.log > > > I have schema > {code:javascript} > { > "version": "1.0", > "defaultSchema": "JIRA", > "schemas": [ > { > "name": "INTERNAL", > "type": "custom", > "factory": "%%UNDERLYING_SCHEMA_FACTORY", > "operand": {} > }, { > "name": "JIRA", > "type": "custom", > "factory": > "org.apache.calcite.schema.impl.AbstractSchema$Factory", > "operand": {}, > "tables": [ > { > "name": "ISSUES", > "type": "view", > "sql": [ > "SELECT JI.ID, JI.SUMMARY, > JI.PROJECT ", > "FROM INTERNAL.JIRAISSUE as JI > ", > "WHERE JI.ID IN (SELECT EAI.ID > FROM EXPLICIT_ALLOWED_ISSUES AS EAI) ", > "UNION ", > "SELECT JI.ID, JI.SUMMARY, > JI.PROJECT ", > "FROM INTERNAL.JIRAISSUE as JI > ", > "WHERE JI.PROJECT IN (SELECT > AP.ID FROM ALLOWED_PROJECTS AS AP)" > ] > }, > { > "name": "ALLOWED_PROJECTS", > "type": "table", > "factory": "%%DELEGATING_TABLE_FACTORY" > }, > { > "name": "EXPLICIT_ALLOWED_ISSUES", > "type": "table", > "factory": "%%DELEGATING_TABLE_FACTORY" > } > ] > } > ] > } > {code} > Where INTERNAL schema points to JDBC native JIRA Schema (through my custom > factory, but it only wraps passing datasource instead of user/password/url to > db) and JIRA schema has one view and two tables (factories generate java data > wrapped in AbstractQueryableTable) > When running "SELECT * FROM ISSUES" all works. > But when changing the view from UNION construction to OR - engine provides no > results: > {code:javascript} > { > "version": "1.0", > "defaultSchema": "JIRA", > "schemas": [ > { > "name": "INTERNAL", > "type": "custom", > "factory": "%%UNDERLYING_SCHEMA_FACTORY", > "operand": {} > }, { > "name": "JIRA", > "type": "custom", > "factory": > "org.apache.calcite.schema.impl.AbstractSchema$Factory", > "operand": {}, > "tables": [ > { >