[
https://issues.apache.org/jira/browse/CALCITE-6123?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17793780#comment-17793780
]
Stamatis Zampetakis edited comment on CALCITE-6123 at 12/6/23 3:25 PM:
-----------------------------------------------------------------------
[~jiajunbernoulli] I suspect that it has to do with how text execution
framework (Junit) defines execution order. Although, I haven't confirmed I
suspect that in our setting test order is defined by the [Random
strategy|https://github.com/junit-team/junit5/blob/9115e233b8d589f1082179f2de17a4698a081cc5/junit-jupiter-api/src/main/java/org/junit/jupiter/api/MethodOrderer.java#L264],
which uses {{System.nanoTime}} as a seed. Every time we run the tests we will
have a different seed for random so I guess test order is sensitive to when the
tests are run.
was (Author: zabetak):
[~jiajunbernoulli] I suspect that it has to do with how text execution
framework (Junit) defines execution order. Although, I haven't confirmed I
suspect that in our setting test order is defined by the [Random|
https://github.com/junit-team/junit5/blob/9115e233b8d589f1082179f2de17a4698a081cc5/junit-jupiter-api/src/main/java/org/junit/jupiter/api/MethodOrderer.java#L264]
strategy which uses {{System.nanoTime}} as a seed. Every time we run the tests
we will have a different seed for random so I guess test order is sensitive to
when the tests are run.
> DruidAdapterIT#testInterleaveBetweenAggregateAndGroupOrderByOnMetrics fails
> ---------------------------------------------------------------------------
>
> Key: CALCITE-6123
> URL: https://issues.apache.org/jira/browse/CALCITE-6123
> Project: Calcite
> Issue Type: Bug
> Components: tests
> Affects Versions: 1.36.0
> Reporter: Benchao Li
> Assignee: Stamatis Zampetakis
> Priority: Major
> Labels: pull-request-available
> Fix For: 1.37.0
>
> Attachments: image-2023-11-19-15-17-53-699.png
>
>
> The test that is failing is
> testInterleaveBetweenAggregateAndGroupOrderByOnMetrics and the error is shown
> below:
> {noformat}
> FAILURE 0.6sec, org.apache.calcite.test.DruidAdapterIT >
> testInterleaveBetweenAggregateAndGroupOrderByOnMetrics()
> java.lang.AssertionError:
> Expected: "store_state=CA; brand_name=King; A=21.4632\nstore_state=OR;
> brand_name=Symphony; A=32.176\nstore_state=CA; brand_name=Toretti;
> A=32.2465\nstore_state=WA; brand_name=King; A=34.6104\nstore_state=OR;
> brand_name=Toretti; A=36.3"
> but: was "store_state=OR; brand_name=ADJ; A=83.8764\nstore_state=WA;
> brand_name=Akron; A=85.8402\nstore_state=OR; brand_name=American;
> A=86.7898\nstore_state=WA; brand_name=ADJ; A=97.6488\nstore_state=CA;
> brand_name=ADJ; A=98.0076"
> at org.hamcrest.MatcherAssert.assertThat(MatcherAssert.java:18)
> at org.hamcrest.MatcherAssert.assertThat(MatcherAssert.java:6)
> at
> org.apache.calcite.test.CalciteAssert.lambda$checkResult$6(CalciteAssert.java:453)
> at
> org.apache.calcite.test.CalciteAssert.assertQuery(CalciteAssert.java:582)
> at
> org.apache.calcite.test.CalciteAssert$AssertQuery.lambda$returns$1(CalciteAssert.java:1495)
> at
> org.apache.calcite.test.CalciteAssert$AssertQuery.withConnection(CalciteAssert.java:1434)
> at
> org.apache.calcite.test.CalciteAssert$AssertQuery.returns(CalciteAssert.java:1493)
> at
> org.apache.calcite.test.CalciteAssert$AssertQuery.returns(CalciteAssert.java:1483)
> at
> org.apache.calcite.test.CalciteAssert$AssertQuery.returnsOrdered(CalciteAssert.java:1509)
> at
> org.apache.calcite.test.DruidAdapterIT.testInterleaveBetweenAggregateAndGroupOrderByOnMetrics(DruidAdapterIT.java:2336)
> Suppressed: org.apache.calcite.util.TestUtil$ExtraInformation: With
> materializationsEnabled=false, limit=-1, sql=select "store_state",
> "brand_name", "A" from (
> select sum("store_sales")-sum("store_cost") as a, "store_state",
> "brand_name"
> from "foodmart"
> group by "store_state", "brand_name" ) subq
> order by "A" limit 5
> at
> app//org.apache.calcite.util.TestUtil.rethrow(TestUtil.java:389)
> at
> app//org.apache.calcite.test.CalciteAssert.assertQuery(CalciteAssert.java:598)
> ... 6 more
> {noformat}
>
> The test is all failing after
> [https://github.com/apache/calcite/commit/55034513b463c938035e5d2436949bbf734b84b6],
> I'm not sure whether it's related.
> See following jobs:
> * [https://github.com/apache/calcite/actions/runs/6886169664/job/18731435605]
> * [https://github.com/apache/calcite/actions/runs/6885301555/job/18729238762]
> * [https://github.com/apache/calcite/actions/runs/6871630651/job/18688793776]
> * [https://github.com/apache/calcite/actions/runs/6860287671/job/18653876601]
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
