[jira] [Commented] (FLINK-35098) Incorrect results for queries like "10 >= y" on tables using Filesystem connector and Orc format
[
https://issues.apache.org/jira/browse/FLINK-35098?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17846194#comment-17846194
]
Sergey Nuyanzin commented on FLINK-35098:
-
Merged as
1.18:
[1f604da2dfc831d04826a20b3cb272d2ad9dfb56|https://github.com/apache/flink/commit/1f604da2dfc831d04826a20b3cb272d2ad9dfb56]
1.19:
[e16da86dfb1fbeee541cd9dfccd5f5f4520b7396|https://github.com/apache/flink/commit/e16da86dfb1fbeee541cd9dfccd5f5f4520b7396]
master:
[4165bac27bda4457e5940a994d923242d4a271dc|https://github.com/apache/flink/commit/4165bac27bda4457e5940a994d923242d4a271dc]
> Incorrect results for queries like "10 >= y" on tables using Filesystem
> connector and Orc format
>
>
> Key: FLINK-35098
> URL: https://issues.apache.org/jira/browse/FLINK-35098
> Project: Flink
> Issue Type: Bug
> Components: Connectors / ORC, Formats (JSON, Avro, Parquet, ORC,
> SequenceFile)
>Affects Versions: 1.12.7, 1.13.6, 1.14.6, 1.15.4, 1.16.3, 1.17.2, 1.19.0,
> 1.18.1
>Reporter: Andrey Gaskov
>Assignee: Andrey Gaskov
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.18.2, 1.20.0, 1.19.1
>
>
> When working with ORC files, there is an issue with evaluation of SQL queries
> containing expressions with a literal as the first operand. Specifically, the
> query *10 >= y* does not always return the correct result.
> This test added to OrcFileSystemITCase.java fails on the second check:
>
> {code:java}
> @TestTemplate
> void testOrcFilterPushDownLiteralFirst() throws ExecutionException,
> InterruptedException {
> super.tableEnv()
> .executeSql("insert into orcLimitTable values('a', 10, 10)")
> .await();
> List expected = Collections.singletonList(Row.of(10));
> check("select y from orcLimitTable where y <= 10", expected);
> check("select y from orcLimitTable where 10 >= y", expected);
> }
> Results do not match for query:
> select y from orcLimitTable where 10 >= y
> Results
> == Correct Result - 1 == == Actual Result - 0 ==
> !+I[10] {code}
> The checks are equivalent and should evaluate to the same result. But the
> second query doesn't return the record with y=10.
> The table is defined as:
> {code:java}
> create table orcLimitTable (
> x string,
> y int,
> a int)
> with (
> 'connector' = 'filesystem',
> 'path' = '/tmp/junit4374176500101507155/junit7109291529844202275/',
> 'format'='orc'){code}
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (FLINK-35098) Incorrect results for queries like "10 >= y" on tables using Filesystem connector and Orc format
[
https://issues.apache.org/jira/browse/FLINK-35098?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17836879#comment-17836879
]
Andrey Gaskov commented on FLINK-35098:
---
[~jeyhunkarimov], thank you! I submitted my PR.
> Incorrect results for queries like "10 >= y" on tables using Filesystem
> connector and Orc format
>
>
> Key: FLINK-35098
> URL: https://issues.apache.org/jira/browse/FLINK-35098
> Project: Flink
> Issue Type: Bug
> Components: Connectors / ORC, Formats (JSON, Avro, Parquet, ORC,
> SequenceFile)
>Affects Versions: 1.12.7, 1.13.6, 1.14.6, 1.15.4, 1.16.3, 1.17.2, 1.19.0,
> 1.18.1
>Reporter: Andrey Gaskov
>Priority: Major
> Labels: pull-request-available
>
> When working with ORC files, there is an issue with evaluation of SQL queries
> containing expressions with a literal as the first operand. Specifically, the
> query *10 >= y* does not always return the correct result.
> This test added to OrcFileSystemITCase.java fails on the second check:
>
> {code:java}
> @TestTemplate
> void testOrcFilterPushDownLiteralFirst() throws ExecutionException,
> InterruptedException {
> super.tableEnv()
> .executeSql("insert into orcLimitTable values('a', 10, 10)")
> .await();
> List expected = Collections.singletonList(Row.of(10));
> check("select y from orcLimitTable where y <= 10", expected);
> check("select y from orcLimitTable where 10 >= y", expected);
> }
> Results do not match for query:
> select y from orcLimitTable where 10 >= y
> Results
> == Correct Result - 1 == == Actual Result - 0 ==
> !+I[10] {code}
> The checks are equivalent and should evaluate to the same result. But the
> second query doesn't return the record with y=10.
> The table is defined as:
> {code:java}
> create table orcLimitTable (
> x string,
> y int,
> a int)
> with (
> 'connector' = 'filesystem',
> 'path' = '/tmp/junit4374176500101507155/junit7109291529844202275/',
> 'format'='orc'){code}
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (FLINK-35098) Incorrect results for queries like "10 >= y" on tables using Filesystem connector and Orc format
[
https://issues.apache.org/jira/browse/FLINK-35098?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17836868#comment-17836868
]
Jeyhun Karimov commented on FLINK-35098:
Hi [~empathy87] sorry, I have been working on the issue in the time between you
created the issue and you commented here, so I did not track your comment here.
In any case, submission might have some issues (now closed), so you are welcome
to submit your PR. Thanks
> Incorrect results for queries like "10 >= y" on tables using Filesystem
> connector and Orc format
>
>
> Key: FLINK-35098
> URL: https://issues.apache.org/jira/browse/FLINK-35098
> Project: Flink
> Issue Type: Bug
> Components: Connectors / ORC, Formats (JSON, Avro, Parquet, ORC,
> SequenceFile)
>Affects Versions: 1.12.7, 1.13.6, 1.14.6, 1.15.4, 1.16.3, 1.17.2, 1.19.0,
> 1.18.1
>Reporter: Andrey Gaskov
>Priority: Major
> Labels: pull-request-available
>
> When working with ORC files, there is an issue with evaluation of SQL queries
> containing expressions with a literal as the first operand. Specifically, the
> query *10 >= y* does not always return the correct result.
> This test added to OrcFileSystemITCase.java fails on the second check:
>
> {code:java}
> @TestTemplate
> void testOrcFilterPushDownLiteralFirst() throws ExecutionException,
> InterruptedException {
> super.tableEnv()
> .executeSql("insert into orcLimitTable values('a', 10, 10)")
> .await();
> List expected = Collections.singletonList(Row.of(10));
> check("select y from orcLimitTable where y <= 10", expected);
> check("select y from orcLimitTable where 10 >= y", expected);
> }
> Results do not match for query:
> select y from orcLimitTable where 10 >= y
> Results
> == Correct Result - 1 == == Actual Result - 0 ==
> !+I[10] {code}
> The checks are equivalent and should evaluate to the same result. But the
> second query doesn't return the record with y=10.
> The table is defined as:
> {code:java}
> create table orcLimitTable (
> x string,
> y int,
> a int)
> with (
> 'connector' = 'filesystem',
> 'path' = '/tmp/junit4374176500101507155/junit7109291529844202275/',
> 'format'='orc'){code}
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (FLINK-35098) Incorrect results for queries like "10 >= y" on tables using Filesystem connector and Orc format
[
https://issues.apache.org/jira/browse/FLINK-35098?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17836830#comment-17836830
]
Andrey Gaskov commented on FLINK-35098:
---
Please, assign this issue to me. I know where it could be fixed.
> Incorrect results for queries like "10 >= y" on tables using Filesystem
> connector and Orc format
>
>
> Key: FLINK-35098
> URL: https://issues.apache.org/jira/browse/FLINK-35098
> Project: Flink
> Issue Type: Bug
> Components: Connectors / ORC, Formats (JSON, Avro, Parquet, ORC,
> SequenceFile)
>Affects Versions: 1.12.7, 1.13.6, 1.14.6, 1.15.4, 1.16.3, 1.17.2, 1.19.0,
> 1.18.1
>Reporter: Andrey Gaskov
>Priority: Major
>
> When working with ORC files, there is an issue with evaluation of SQL queries
> containing expressions with a literal as the first operand. Specifically, the
> query *10 >= y* does not always return the correct result.
> This test added to OrcFileSystemITCase.java fails on the second check:
>
> {code:java}
> @TestTemplate
> void testOrcFilterPushDownLiteralFirst() throws ExecutionException,
> InterruptedException {
> super.tableEnv()
> .executeSql("insert into orcLimitTable values('a', 10, 10)")
> .await();
> List expected = Collections.singletonList(Row.of(10));
> check("select y from orcLimitTable where y <= 10", expected);
> check("select y from orcLimitTable where 10 >= y", expected);
> }
> Results do not match for query:
> select y from orcLimitTable where 10 >= y
> Results
> == Correct Result - 1 == == Actual Result - 0 ==
> !+I[10] {code}
> The checks are equivalent and should evaluate to the same result. But the
> second query doesn't return the record with y=10.
> The table is defined as:
> {code:java}
> create table orcLimitTable (
> x string,
> y int,
> a int)
> with (
> 'connector' = 'filesystem',
> 'path' = '/tmp/junit4374176500101507155/junit7109291529844202275/',
> 'format'='orc'){code}
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
