[jira] [Commented] (CALCITE-2069) RexSimplify.removeNullabilityCast() always removes cast for operand with ANY type
[
https://issues.apache.org/jira/browse/CALCITE-2069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16274208#comment-16274208
]
Volodymyr Vysotskyi commented on CALCITE-2069:
--
Thank you all for responses.
[~amansinha100], this change did not bring any additional regression or unit
test failures.
[~julianhyde], I have updated the test in pull request in the same commit, so
currently it is the last fix-up from my side.
> RexSimplify.removeNullabilityCast() always removes cast for operand with ANY
> type
> -
>
> Key: CALCITE-2069
> URL: https://issues.apache.org/jira/browse/CALCITE-2069
> Project: Calcite
> Issue Type: Bug
>Reporter: Volodymyr Vysotskyi
>Assignee: Julian Hyde
> Fix For: 1.16.0
>
>
> When a field is received from Dynamic Table, its type left {{ANY}}, and it is
> used in the filter condition with the cast, which actually should produce
> physical cast (for example we are trying to cast varchar to boolean)
> {{RexSimplify.removeNullabilityCast()}} removes this cast and lefts only
> field in condition.
> This test helps to observe this issue:
> {code:java}
> @Test public void testFilterCastAny() {
> final RelBuilder builder = RelBuilder.create(config().build());
> final RelDataType intType =
> builder.getTypeFactory().createSqlType(SqlTypeName.ANY);
> RelNode root =
> builder.scan("EMP")
> .filter(
> builder.cast(
> builder.patternField("varchar_field", intType, 0),
> SqlTypeName.BOOLEAN))
> .build();
> assertThat(str(root),
> is("LogicalFilter(condition=[CAST(varchar_field.$0):BOOLEAN NOT
> NULL])\n"
> + " LogicalTableScan(table=[[scott, EMP]])\n"));
> }
> {code}
> It happens because {{SqlTypeUtil.equalSansNullability()}} returns true if any
> of its arguments has {{ANY}} type.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (CALCITE-2069) RexSimplify.removeNullabilityCast() always removes cast for operand with ANY type
[
https://issues.apache.org/jira/browse/CALCITE-2069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16273774#comment-16273774
]
Julian Hyde commented on CALCITE-2069:
--
OK, thanks everyone; it sounds as if this won't break Drill or Dremio. When we
have the last fix-up from [~vvysotskyi] this can go in.
> RexSimplify.removeNullabilityCast() always removes cast for operand with ANY
> type
> -
>
> Key: CALCITE-2069
> URL: https://issues.apache.org/jira/browse/CALCITE-2069
> Project: Calcite
> Issue Type: Bug
>Reporter: Volodymyr Vysotskyi
>Assignee: Julian Hyde
> Fix For: 1.16.0
>
>
> When a field is received from Dynamic Table, its type left {{ANY}}, and it is
> used in the filter condition with the cast, which actually should produce
> physical cast (for example we are trying to cast varchar to boolean)
> {{RexSimplify.removeNullabilityCast()}} removes this cast and lefts only
> field in condition.
> This test helps to observe this issue:
> {code:java}
> @Test public void testFilterCastAny() {
> final RelBuilder builder = RelBuilder.create(config().build());
> final RelDataType intType =
> builder.getTypeFactory().createSqlType(SqlTypeName.ANY);
> RelNode root =
> builder.scan("EMP")
> .filter(
> builder.cast(
> builder.patternField("varchar_field", intType, 0),
> SqlTypeName.BOOLEAN))
> .build();
> assertThat(str(root),
> is("LogicalFilter(condition=[CAST(varchar_field.$0):BOOLEAN NOT
> NULL])\n"
> + " LogicalTableScan(table=[[scott, EMP]])\n"));
> }
> {code}
> It happens because {{SqlTypeUtil.equalSansNullability()}} returns true if any
> of its arguments has {{ANY}} type.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (CALCITE-2069) RexSimplify.removeNullabilityCast() always removes cast for operand with ANY type
[
https://issues.apache.org/jira/browse/CALCITE-2069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16273705#comment-16273705
]
Laurent Goujon commented on CALCITE-2069:
-
I did not observe any regression with our own test suite, but maybe we are
hitting the same limitation as [~amansinha100] (although we are on Calcite
1.12.0).
> RexSimplify.removeNullabilityCast() always removes cast for operand with ANY
> type
> -
>
> Key: CALCITE-2069
> URL: https://issues.apache.org/jira/browse/CALCITE-2069
> Project: Calcite
> Issue Type: Bug
>Reporter: Volodymyr Vysotskyi
>Assignee: Julian Hyde
>
> When a field is received from Dynamic Table, its type left {{ANY}}, and it is
> used in the filter condition with the cast, which actually should produce
> physical cast (for example we are trying to cast varchar to boolean)
> {{RexSimplify.removeNullabilityCast()}} removes this cast and lefts only
> field in condition.
> This test helps to observe this issue:
> {code:java}
> @Test public void testFilterCastAny() {
> final RelBuilder builder = RelBuilder.create(config().build());
> final RelDataType intType =
> builder.getTypeFactory().createSqlType(SqlTypeName.ANY);
> RelNode root =
> builder.scan("EMP")
> .filter(
> builder.cast(
> builder.patternField("varchar_field", intType, 0),
> SqlTypeName.BOOLEAN))
> .build();
> assertThat(str(root),
> is("LogicalFilter(condition=[CAST(varchar_field.$0):BOOLEAN NOT
> NULL])\n"
> + " LogicalTableScan(table=[[scott, EMP]])\n"));
> }
> {code}
> It happens because {{SqlTypeUtil.equalSansNullability()}} returns true if any
> of its arguments has {{ANY}} type.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (CALCITE-2069) RexSimplify.removeNullabilityCast() always removes cast for operand with ANY type
[
https://issues.apache.org/jira/browse/CALCITE-2069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16271764#comment-16271764
]
Aman Sinha commented on CALCITE-2069:
-
[~vvysotskyi] ok, makes sense. It's good that you are adding a unit test for
this. Assuming you have run Drill regression tests with your change, I am
good with this. +1.
> RexSimplify.removeNullabilityCast() always removes cast for operand with ANY
> type
> -
>
> Key: CALCITE-2069
> URL: https://issues.apache.org/jira/browse/CALCITE-2069
> Project: Calcite
> Issue Type: Bug
>Reporter: Volodymyr Vysotskyi
>Assignee: Julian Hyde
>
> When a field is received from Dynamic Table, its type left {{ANY}}, and it is
> used in the filter condition with the cast, which actually should produce
> physical cast (for example we are trying to cast varchar to boolean)
> {{RexSimplify.removeNullabilityCast()}} removes this cast and lefts only
> field in condition.
> This test helps to observe this issue:
> {code:java}
> @Test public void testFilterCastAny() {
> final RelBuilder builder = RelBuilder.create(config().build());
> final RelDataType intType =
> builder.getTypeFactory().createSqlType(SqlTypeName.ANY);
> RelNode root =
> builder.scan("EMP")
> .filter(
> builder.cast(
> builder.patternField("varchar_field", intType, 0),
> SqlTypeName.BOOLEAN))
> .build();
> assertThat(str(root),
> is("LogicalFilter(condition=[CAST(varchar_field.$0):BOOLEAN NOT
> NULL])\n"
> + " LogicalTableScan(table=[[scott, EMP]])\n"));
> }
> {code}
> It happens because {{SqlTypeUtil.equalSansNullability()}} returns true if any
> of its arguments has {{ANY}} type.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (CALCITE-2069) RexSimplify.removeNullabilityCast() always removes cast for operand with ANY type
[
https://issues.apache.org/jira/browse/CALCITE-2069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16271744#comment-16271744
]
Volodymyr Vysotskyi commented on CALCITE-2069:
--
[~amansinha100], this bug did not observe in the Calcite 1.4.0, since the
simplification of the condition before creating the filter was not happening.
This simplification was added in newer Calcite version, so you could not
reproduce it using Drill version from the Apache master.
> RexSimplify.removeNullabilityCast() always removes cast for operand with ANY
> type
> -
>
> Key: CALCITE-2069
> URL: https://issues.apache.org/jira/browse/CALCITE-2069
> Project: Calcite
> Issue Type: Bug
>Reporter: Volodymyr Vysotskyi
>Assignee: Julian Hyde
>
> When a field is received from Dynamic Table, its type left {{ANY}}, and it is
> used in the filter condition with the cast, which actually should produce
> physical cast (for example we are trying to cast varchar to boolean)
> {{RexSimplify.removeNullabilityCast()}} removes this cast and lefts only
> field in condition.
> This test helps to observe this issue:
> {code:java}
> @Test public void testFilterCastAny() {
> final RelBuilder builder = RelBuilder.create(config().build());
> final RelDataType intType =
> builder.getTypeFactory().createSqlType(SqlTypeName.ANY);
> RelNode root =
> builder.scan("EMP")
> .filter(
> builder.cast(
> builder.patternField("varchar_field", intType, 0),
> SqlTypeName.BOOLEAN))
> .build();
> assertThat(str(root),
> is("LogicalFilter(condition=[CAST(varchar_field.$0):BOOLEAN NOT
> NULL])\n"
> + " LogicalTableScan(table=[[scott, EMP]])\n"));
> }
> {code}
> It happens because {{SqlTypeUtil.equalSansNullability()}} returns true if any
> of its arguments has {{ANY}} type.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (CALCITE-2069) RexSimplify.removeNullabilityCast() always removes cast for operand with ANY type
[
https://issues.apache.org/jira/browse/CALCITE-2069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16271723#comment-16271723
]
Aman Sinha commented on CALCITE-2069:
-
[~vvysotskyi] I didn't fully understand the motivation in the JIRA description.
Suppose I have a table with 2 columns containing the strings 'true' and
'false'. These columns will show as ANY type in Drill. If I run the following
query, I still see the CAST function; it is not dropped.
{noformat}
explain plan for select b from dfs.tmp.test2 where cast(b as boolean) is false;
...
Filter(condition=[IS FALSE(CAST($0):BOOLEAN)]) : rowType =
RecordType(ANY b):
...
{noformat}
(note, I am working with the older calcite version, so it is possible this
behavior may have changed).
> RexSimplify.removeNullabilityCast() always removes cast for operand with ANY
> type
> -
>
> Key: CALCITE-2069
> URL: https://issues.apache.org/jira/browse/CALCITE-2069
> Project: Calcite
> Issue Type: Bug
>Reporter: Volodymyr Vysotskyi
>Assignee: Julian Hyde
>
> When a field is received from Dynamic Table, its type left {{ANY}}, and it is
> used in the filter condition with the cast, which actually should produce
> physical cast (for example we are trying to cast varchar to boolean)
> {{RexSimplify.removeNullabilityCast()}} removes this cast and lefts only
> field in condition.
> This test helps to observe this issue:
> {code:java}
> @Test public void testFilterCastAny() {
> final RelBuilder builder = RelBuilder.create(config().build());
> final RelDataType intType =
> builder.getTypeFactory().createSqlType(SqlTypeName.ANY);
> RelNode root =
> builder.scan("EMP")
> .filter(
> builder.cast(
> builder.patternField("varchar_field", intType, 0),
> SqlTypeName.BOOLEAN))
> .build();
> assertThat(str(root),
> is("LogicalFilter(condition=[CAST(varchar_field.$0):BOOLEAN NOT
> NULL])\n"
> + " LogicalTableScan(table=[[scott, EMP]])\n"));
> }
> {code}
> It happens because {{SqlTypeUtil.equalSansNullability()}} returns true if any
> of its arguments has {{ANY}} type.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (CALCITE-2069) RexSimplify.removeNullabilityCast() always removes cast for operand with ANY type
[
https://issues.apache.org/jira/browse/CALCITE-2069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16271076#comment-16271076
]
Jacques Nadeau commented on CALCITE-2069:
-
Thanks for mentioning me, Julian.
[~laurentgo], can you or someone on your team take a look if this will be
problematic for us?
[~amansinha100], you might want to take a look as well in terms of impact to
Drill.
> RexSimplify.removeNullabilityCast() always removes cast for operand with ANY
> type
> -
>
> Key: CALCITE-2069
> URL: https://issues.apache.org/jira/browse/CALCITE-2069
> Project: Calcite
> Issue Type: Bug
>Reporter: Volodymyr Vysotskyi
>Assignee: Julian Hyde
>
> When a field is received from Dynamic Table, its type left {{ANY}}, and it is
> used in the filter condition with the cast, which actually should produce
> physical cast (for example we are trying to cast varchar to boolean)
> {{RexSimplify.removeNullabilityCast()}} removes this cast and lefts only
> field in condition.
> This test helps to observe this issue:
> {code:java}
> @Test public void testFilterCastAny() {
> final RelBuilder builder = RelBuilder.create(config().build());
> final RelDataType intType =
> builder.getTypeFactory().createSqlType(SqlTypeName.ANY);
> RelNode root =
> builder.scan("EMP")
> .filter(
> builder.cast(
> builder.patternField("varchar_field", intType, 0),
> SqlTypeName.BOOLEAN))
> .build();
> assertThat(str(root),
> is("LogicalFilter(condition=[CAST(varchar_field.$0):BOOLEAN NOT
> NULL])\n"
> + " LogicalTableScan(table=[[scott, EMP]])\n"));
> }
> {code}
> It happens because {{SqlTypeUtil.equalSansNullability()}} returns true if any
> of its arguments has {{ANY}} type.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (CALCITE-2069) RexSimplify.removeNullabilityCast() always removes cast for operand with ANY type
[
https://issues.apache.org/jira/browse/CALCITE-2069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16270696#comment-16270696
]
Volodymyr Vysotskyi commented on CALCITE-2069:
--
Sorry for naming in the test, I forgot to change it.
{{patternField}} was used in other tests of this class, so I also used it.
Replaced it by {{builder.getRexBuilder().makeInputRef(anyType, 0)}}.
> RexSimplify.removeNullabilityCast() always removes cast for operand with ANY
> type
> -
>
> Key: CALCITE-2069
> URL: https://issues.apache.org/jira/browse/CALCITE-2069
> Project: Calcite
> Issue Type: Bug
>Reporter: Volodymyr Vysotskyi
>Assignee: Julian Hyde
>
> When a field is received from Dynamic Table, its type left {{ANY}}, and it is
> used in the filter condition with the cast, which actually should produce
> physical cast (for example we are trying to cast varchar to boolean)
> {{RexSimplify.removeNullabilityCast()}} removes this cast and lefts only
> field in condition.
> This test helps to observe this issue:
> {code:java}
> @Test public void testFilterCastAny() {
> final RelBuilder builder = RelBuilder.create(config().build());
> final RelDataType intType =
> builder.getTypeFactory().createSqlType(SqlTypeName.ANY);
> RelNode root =
> builder.scan("EMP")
> .filter(
> builder.cast(
> builder.patternField("varchar_field", intType, 0),
> SqlTypeName.BOOLEAN))
> .build();
> assertThat(str(root),
> is("LogicalFilter(condition=[CAST(varchar_field.$0):BOOLEAN NOT
> NULL])\n"
> + " LogicalTableScan(table=[[scott, EMP]])\n"));
> }
> {code}
> It happens because {{SqlTypeUtil.equalSansNullability()}} returns true if any
> of its arguments has {{ANY}} type.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (CALCITE-2069) RexSimplify.removeNullabilityCast() always removes cast for operand with ANY type
[
https://issues.apache.org/jira/browse/CALCITE-2069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16270275#comment-16270275
]
Julian Hyde commented on CALCITE-2069:
--
Reviewing
https://github.com/apache/calcite/pull/574/commits/f113abd7f67291941517abfeb875849ae60bdaaa:
* The fix looks reasonable to me, and it doesn't break any tests.
* [~jnadeau], The fix removes code that you added 4 years ago without adding
any tests. So, it might be worth you reviewing this.
* Can you clean up your naming? You have a type called "intType" which is
actually an ANY.
* Why are you using {{patternField}}? That is a *very* obscure method.
> RexSimplify.removeNullabilityCast() always removes cast for operand with ANY
> type
> -
>
> Key: CALCITE-2069
> URL: https://issues.apache.org/jira/browse/CALCITE-2069
> Project: Calcite
> Issue Type: Bug
>Reporter: Volodymyr Vysotskyi
>Assignee: Julian Hyde
>
> When a field is received from Dynamic Table, its type left {{ANY}}, and it is
> used in the filter condition with the cast, which actually should produce
> physical cast (for example we are trying to cast varchar to boolean)
> {{RexSimplify.removeNullabilityCast()}} removes this cast and lefts only
> field in condition.
> This test helps to observe this issue:
> {code:java}
> @Test public void testFilterCastAny() {
> final RelBuilder builder = RelBuilder.create(config().build());
> final RelDataType intType =
> builder.getTypeFactory().createSqlType(SqlTypeName.ANY);
> RelNode root =
> builder.scan("EMP")
> .filter(
> builder.cast(
> builder.patternField("varchar_field", intType, 0),
> SqlTypeName.BOOLEAN))
> .build();
> assertThat(str(root),
> is("LogicalFilter(condition=[CAST(varchar_field.$0):BOOLEAN NOT
> NULL])\n"
> + " LogicalTableScan(table=[[scott, EMP]])\n"));
> }
> {code}
> It happens because {{SqlTypeUtil.equalSansNullability()}} returns true if any
> of its arguments has {{ANY}} type.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (CALCITE-2069) RexSimplify.removeNullabilityCast() always removes cast for operand with ANY type
[
https://issues.apache.org/jira/browse/CALCITE-2069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16268943#comment-16268943
]
Volodymyr Vysotskyi commented on CALCITE-2069:
--
I created pull request for this issue:
https://github.com/apache/calcite/pull/574. Could someone please take a look?
> RexSimplify.removeNullabilityCast() always removes cast for operand with ANY
> type
> -
>
> Key: CALCITE-2069
> URL: https://issues.apache.org/jira/browse/CALCITE-2069
> Project: Calcite
> Issue Type: Bug
>Reporter: Volodymyr Vysotskyi
>Assignee: Julian Hyde
>
> When a field is received from Dynamic Table, its type left {{ANY}}, and it is
> used in the filter condition with the cast, which actually should produce
> physical cast (for example we are trying to cast varchar to boolean)
> {{RexSimplify.removeNullabilityCast()}} removes this cast and lefts only
> field in condition.
> This test helps to observe this issue:
> {code:java}
> @Test public void testFilterCastAny() {
> final RelBuilder builder = RelBuilder.create(config().build());
> final RelDataType intType =
> builder.getTypeFactory().createSqlType(SqlTypeName.ANY);
> RelNode root =
> builder.scan("EMP")
> .filter(
> builder.cast(
> builder.patternField("varchar_field", intType, 0),
> SqlTypeName.BOOLEAN))
> .build();
> assertThat(str(root),
> is("LogicalFilter(condition=[CAST(varchar_field.$0):BOOLEAN NOT
> NULL])\n"
> + " LogicalTableScan(table=[[scott, EMP]])\n"));
> }
> {code}
> It happens because {{SqlTypeUtil.equalSansNullability()}} returns true if any
> of its arguments has {{ANY}} type.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
