[jira] [Commented] (CALCITE-6123) DruidAdapterIT#testInterleaveBetweenAggregateAndGroupOrderByOnMetrics fails
[
https://issues.apache.org/jira/browse/CALCITE-6123?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17793805#comment-17793805
]
Benchao Li commented on CALCITE-6123:
-
I verified it in one of my [PR|https://github.com/apache/calcite/pull/3555] for
3 times, Druid tests all pass. Thanks [~zabetak] for the fix.
> DruidAdapterIT#testInterleaveBetweenAggregateAndGroupOrderByOnMetrics fails
> ---
>
> Key: CALCITE-6123
> URL: https://issues.apache.org/jira/browse/CALCITE-6123
> Project: Calcite
> Issue Type: Bug
> Components: tests
>Affects Versions: 1.36.0
>Reporter: Benchao Li
>Assignee: Stamatis Zampetakis
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.37.0
>
> Attachments: image-2023-11-19-15-17-53-699.png
>
>
> The test that is failing is
> testInterleaveBetweenAggregateAndGroupOrderByOnMetrics and the error is shown
> below:
> {noformat}
> FAILURE 0.6sec, org.apache.calcite.test.DruidAdapterIT >
> testInterleaveBetweenAggregateAndGroupOrderByOnMetrics()
> java.lang.AssertionError:
> Expected: "store_state=CA; brand_name=King; A=21.4632\nstore_state=OR;
> brand_name=Symphony; A=32.176\nstore_state=CA; brand_name=Toretti;
> A=32.2465\nstore_state=WA; brand_name=King; A=34.6104\nstore_state=OR;
> brand_name=Toretti; A=36.3"
> but: was "store_state=OR; brand_name=ADJ; A=83.8764\nstore_state=WA;
> brand_name=Akron; A=85.8402\nstore_state=OR; brand_name=American;
> A=86.7898\nstore_state=WA; brand_name=ADJ; A=97.6488\nstore_state=CA;
> brand_name=ADJ; A=98.0076"
> at org.hamcrest.MatcherAssert.assertThat(MatcherAssert.java:18)
> at org.hamcrest.MatcherAssert.assertThat(MatcherAssert.java:6)
> at
> org.apache.calcite.test.CalciteAssert.lambda$checkResult$6(CalciteAssert.java:453)
> at
> org.apache.calcite.test.CalciteAssert.assertQuery(CalciteAssert.java:582)
> at
> org.apache.calcite.test.CalciteAssert$AssertQuery.lambda$returns$1(CalciteAssert.java:1495)
> at
> org.apache.calcite.test.CalciteAssert$AssertQuery.withConnection(CalciteAssert.java:1434)
> at
> org.apache.calcite.test.CalciteAssert$AssertQuery.returns(CalciteAssert.java:1493)
> at
> org.apache.calcite.test.CalciteAssert$AssertQuery.returns(CalciteAssert.java:1483)
> at
> org.apache.calcite.test.CalciteAssert$AssertQuery.returnsOrdered(CalciteAssert.java:1509)
> at
> org.apache.calcite.test.DruidAdapterIT.testInterleaveBetweenAggregateAndGroupOrderByOnMetrics(DruidAdapterIT.java:2336)
> Suppressed: org.apache.calcite.util.TestUtil$ExtraInformation: With
> materializationsEnabled=false, limit=-1, sql=select "store_state",
> "brand_name", "A" from (
> select sum("store_sales")-sum("store_cost") as a, "store_state",
> "brand_name"
> from "foodmart"
> group by "store_state", "brand_name" ) subq
> order by "A" limit 5
> at
> app//org.apache.calcite.util.TestUtil.rethrow(TestUtil.java:389)
> at
> app//org.apache.calcite.test.CalciteAssert.assertQuery(CalciteAssert.java:598)
> ... 6 more
> {noformat}
>
> The test is all failing after
> [https://github.com/apache/calcite/commit/55034513b463c938035e5d2436949bbf734b84b6],
> I'm not sure whether it's related.
> See following jobs:
> * [https://github.com/apache/calcite/actions/runs/6886169664/job/18731435605]
> * [https://github.com/apache/calcite/actions/runs/6885301555/job/18729238762]
> * [https://github.com/apache/calcite/actions/runs/6871630651/job/18688793776]
> * [https://github.com/apache/calcite/actions/runs/6860287671/job/18653876601]
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (CALCITE-6123) DruidAdapterIT#testInterleaveBetweenAggregateAndGroupOrderByOnMetrics fails
[
https://issues.apache.org/jira/browse/CALCITE-6123?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17793780#comment-17793780
]
Stamatis Zampetakis commented on CALCITE-6123:
--
[~jiajunbernoulli] I suspect that it has to do with how text execution
framework (Junit) defines execution order. Although, I haven't confirmed I
suspect that in our setting test order is defined by the [Random|
https://github.com/junit-team/junit5/blob/9115e233b8d589f1082179f2de17a4698a081cc5/junit-jupiter-api/src/main/java/org/junit/jupiter/api/MethodOrderer.java#L264]
strategy which uses {{System.nanoTime}} as a seed. Every time we run the tests
we will have a different seed for random so I guess test order is sensitive to
when the tests are run.
> DruidAdapterIT#testInterleaveBetweenAggregateAndGroupOrderByOnMetrics fails
> ---
>
> Key: CALCITE-6123
> URL: https://issues.apache.org/jira/browse/CALCITE-6123
> Project: Calcite
> Issue Type: Bug
> Components: tests
>Affects Versions: 1.36.0
>Reporter: Benchao Li
>Assignee: Stamatis Zampetakis
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.37.0
>
> Attachments: image-2023-11-19-15-17-53-699.png
>
>
> The test that is failing is
> testInterleaveBetweenAggregateAndGroupOrderByOnMetrics and the error is shown
> below:
> {noformat}
> FAILURE 0.6sec, org.apache.calcite.test.DruidAdapterIT >
> testInterleaveBetweenAggregateAndGroupOrderByOnMetrics()
> java.lang.AssertionError:
> Expected: "store_state=CA; brand_name=King; A=21.4632\nstore_state=OR;
> brand_name=Symphony; A=32.176\nstore_state=CA; brand_name=Toretti;
> A=32.2465\nstore_state=WA; brand_name=King; A=34.6104\nstore_state=OR;
> brand_name=Toretti; A=36.3"
> but: was "store_state=OR; brand_name=ADJ; A=83.8764\nstore_state=WA;
> brand_name=Akron; A=85.8402\nstore_state=OR; brand_name=American;
> A=86.7898\nstore_state=WA; brand_name=ADJ; A=97.6488\nstore_state=CA;
> brand_name=ADJ; A=98.0076"
> at org.hamcrest.MatcherAssert.assertThat(MatcherAssert.java:18)
> at org.hamcrest.MatcherAssert.assertThat(MatcherAssert.java:6)
> at
> org.apache.calcite.test.CalciteAssert.lambda$checkResult$6(CalciteAssert.java:453)
> at
> org.apache.calcite.test.CalciteAssert.assertQuery(CalciteAssert.java:582)
> at
> org.apache.calcite.test.CalciteAssert$AssertQuery.lambda$returns$1(CalciteAssert.java:1495)
> at
> org.apache.calcite.test.CalciteAssert$AssertQuery.withConnection(CalciteAssert.java:1434)
> at
> org.apache.calcite.test.CalciteAssert$AssertQuery.returns(CalciteAssert.java:1493)
> at
> org.apache.calcite.test.CalciteAssert$AssertQuery.returns(CalciteAssert.java:1483)
> at
> org.apache.calcite.test.CalciteAssert$AssertQuery.returnsOrdered(CalciteAssert.java:1509)
> at
> org.apache.calcite.test.DruidAdapterIT.testInterleaveBetweenAggregateAndGroupOrderByOnMetrics(DruidAdapterIT.java:2336)
> Suppressed: org.apache.calcite.util.TestUtil$ExtraInformation: With
> materializationsEnabled=false, limit=-1, sql=select "store_state",
> "brand_name", "A" from (
> select sum("store_sales")-sum("store_cost") as a, "store_state",
> "brand_name"
> from "foodmart"
> group by "store_state", "brand_name" ) subq
> order by "A" limit 5
> at
> app//org.apache.calcite.util.TestUtil.rethrow(TestUtil.java:389)
> at
> app//org.apache.calcite.test.CalciteAssert.assertQuery(CalciteAssert.java:598)
> ... 6 more
> {noformat}
>
> The test is all failing after
> [https://github.com/apache/calcite/commit/55034513b463c938035e5d2436949bbf734b84b6],
> I'm not sure whether it's related.
> See following jobs:
> * [https://github.com/apache/calcite/actions/runs/6886169664/job/18731435605]
> * [https://github.com/apache/calcite/actions/runs/6885301555/job/18729238762]
> * [https://github.com/apache/calcite/actions/runs/6871630651/job/18688793776]
> * [https://github.com/apache/calcite/actions/runs/6860287671/job/18653876601]
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (CALCITE-6123) DruidAdapterIT#testInterleaveBetweenAggregateAndGroupOrderByOnMetrics fails
[
https://issues.apache.org/jira/browse/CALCITE-6123?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17793695#comment-17793695
]
Jiajun Xie commented on CALCITE-6123:
-
[~zabetak], Nice work.
After running CI again, I confirmed that your fix was effective
Cache has always been enabled, but why is it so frequent recently?
Perhaps due to other environmental changes, the unit test execution order has
changed.
> DruidAdapterIT#testInterleaveBetweenAggregateAndGroupOrderByOnMetrics fails
> ---
>
> Key: CALCITE-6123
> URL: https://issues.apache.org/jira/browse/CALCITE-6123
> Project: Calcite
> Issue Type: Bug
> Components: tests
>Affects Versions: 1.36.0
>Reporter: Benchao Li
>Assignee: Stamatis Zampetakis
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.37.0
>
> Attachments: image-2023-11-19-15-17-53-699.png
>
>
> The test that is failing is
> testInterleaveBetweenAggregateAndGroupOrderByOnMetrics and the error is shown
> below:
> {noformat}
> FAILURE 0.6sec, org.apache.calcite.test.DruidAdapterIT >
> testInterleaveBetweenAggregateAndGroupOrderByOnMetrics()
> java.lang.AssertionError:
> Expected: "store_state=CA; brand_name=King; A=21.4632\nstore_state=OR;
> brand_name=Symphony; A=32.176\nstore_state=CA; brand_name=Toretti;
> A=32.2465\nstore_state=WA; brand_name=King; A=34.6104\nstore_state=OR;
> brand_name=Toretti; A=36.3"
> but: was "store_state=OR; brand_name=ADJ; A=83.8764\nstore_state=WA;
> brand_name=Akron; A=85.8402\nstore_state=OR; brand_name=American;
> A=86.7898\nstore_state=WA; brand_name=ADJ; A=97.6488\nstore_state=CA;
> brand_name=ADJ; A=98.0076"
> at org.hamcrest.MatcherAssert.assertThat(MatcherAssert.java:18)
> at org.hamcrest.MatcherAssert.assertThat(MatcherAssert.java:6)
> at
> org.apache.calcite.test.CalciteAssert.lambda$checkResult$6(CalciteAssert.java:453)
> at
> org.apache.calcite.test.CalciteAssert.assertQuery(CalciteAssert.java:582)
> at
> org.apache.calcite.test.CalciteAssert$AssertQuery.lambda$returns$1(CalciteAssert.java:1495)
> at
> org.apache.calcite.test.CalciteAssert$AssertQuery.withConnection(CalciteAssert.java:1434)
> at
> org.apache.calcite.test.CalciteAssert$AssertQuery.returns(CalciteAssert.java:1493)
> at
> org.apache.calcite.test.CalciteAssert$AssertQuery.returns(CalciteAssert.java:1483)
> at
> org.apache.calcite.test.CalciteAssert$AssertQuery.returnsOrdered(CalciteAssert.java:1509)
> at
> org.apache.calcite.test.DruidAdapterIT.testInterleaveBetweenAggregateAndGroupOrderByOnMetrics(DruidAdapterIT.java:2336)
> Suppressed: org.apache.calcite.util.TestUtil$ExtraInformation: With
> materializationsEnabled=false, limit=-1, sql=select "store_state",
> "brand_name", "A" from (
> select sum("store_sales")-sum("store_cost") as a, "store_state",
> "brand_name"
> from "foodmart"
> group by "store_state", "brand_name" ) subq
> order by "A" limit 5
> at
> app//org.apache.calcite.util.TestUtil.rethrow(TestUtil.java:389)
> at
> app//org.apache.calcite.test.CalciteAssert.assertQuery(CalciteAssert.java:598)
> ... 6 more
> {noformat}
>
> The test is all failing after
> [https://github.com/apache/calcite/commit/55034513b463c938035e5d2436949bbf734b84b6],
> I'm not sure whether it's related.
> See following jobs:
> * [https://github.com/apache/calcite/actions/runs/6886169664/job/18731435605]
> * [https://github.com/apache/calcite/actions/runs/6885301555/job/18729238762]
> * [https://github.com/apache/calcite/actions/runs/6871630651/job/18688793776]
> * [https://github.com/apache/calcite/actions/runs/6860287671/job/18653876601]
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
