I’m running tests on the PR now. Since commit comments are a soap-box of mine, I will remark that rather than
[CALCITE-2428] Fix cassandra unit test initialization. (Andrei Sereda) the commit comment should be [CALCITE-2428] Cassandra unit test fails to parse version string (Andrei Sereda) Julian > On Jul 30, 2018, at 1:13 PM, Michael Mior <[email protected]> wrote: > > Thanks Andrei for digging into this! Since I haven't reproduced this > failure myself, if someone else could check out the PR, that would be > great. > -- > Michael Mior > [email protected] > > > > Le lun. 30 juil. 2018 à 13:29, Andrei Sereda <[email protected]> a écrit : > >> Agree, flaky tests are pretty annoying. I'll try to watch more carefully >> new "embedded data-source" issues (fongo, ES, cassandra). They introduced >> more "non-determinism" because they now run as part of regular build which >> means executed much more often than IT. >> >> Last commit was for a more deterministic issue (incorrect parsing of new >> java version format <http://openjdk.java.net/jeps/223>). >> >> >> On Mon, Jul 30, 2018 at 12:58 PM Julian Hyde <[email protected]> wrote: >> >>> On balance, I don’t think we should back out CassandraAdapterTest. But we >>> do need to continue working to make it more resilient. Any test that >>> generates too many false negatives over the long run should be disabled, >>> and this is no exception. >>> >>> And as the other test results show, it’s not the only flaky part of the >>> test suite, when the machine is stressed. One thing we can do is to avoid >>> timeouts whenever possible. >>> >>> Julian >>> >>> >>>> On Jul 30, 2018, at 9:30 AM, Andrei Sereda <[email protected]> wrote: >>>> >>>> Pls check and confirm that the following PR fixes the issue: >>>> https://github.com/apache/calcite/pull/770 >>>> It addresses build failures due to version parsing. >>>> >>>> Stale folders / files will be addressed in a different PR. >>>> >>>> >>>> On Mon, Jul 30, 2018 at 10:10 AM Andrei Sereda <[email protected]> >> wrote: >>>> >>>>> This must be something specific to Cassandra Unit. Will check >>>>> >>>>> On Mon, Jul 30, 2018, 08:59 Sergey Nuyanzin <[email protected]> >>> wrote: >>>>> >>>>>> There is one more strange thing (at least on Windows): while >> building a >>>>>> file with name ".toDelete" is generated under calcite\cassandra and >>> it's >>>>>> not removed by the end of tests >>>>>> Is there a way to make cassandra generates these files in target >>> directory >>>>>> e.g.? >>>>>> >>>>>> >>>>>> >>>>>> On Mon, Jul 30, 2018 at 3:52 PM, Andrei Sereda <[email protected]> >>> wrote: >>>>>> >>>>>>> Most of the problems are during test init Phase. Most likely with >>>>>> version >>>>>>> string (eg. 11-ea for JDK11). >>>>>>> I'll fix that. >>>>>>> >>>>>>> >>>>>>> On Mon, Jul 30, 2018 at 8:16 AM Michael Mior <[email protected]> >>> wrote: >>>>>>> >>>>>>>> I tested myself a fair bit under Ubuntu before pushing this and >>> didn't >>>>>>> see >>>>>>>> any of these issues myself. That said, I agree that it's important >>> the >>>>>>> test >>>>>>>> suite be stable. I'm fine with reverting for now or (more >> preferably >>>>>> IMO) >>>>>>>> just disabling these tests by default. >>>>>>>> >>>>>>>> -- >>>>>>>> Michael Mior >>>>>>>> [email protected] >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> Le lun. 30 juil. 2018 à 00:59, Julian Hyde <[email protected]> a >>>>>> écrit : >>>>>>>> >>>>>>>>> I ran on several JDK versions, all on Ubuntu Linux. The machine >> was >>>>>>>>> fairly heavily loaded (I was copying one filesystem to another at >>>>>> the >>>>>>>>> time). >>>>>>>>> >>>>>>>>> Here's the error from OpenJDK10: >>>>>>>>> >>>>>>>>> [INFO] Running org.apache.calcite.test.CassandraAdapterTest >>>>>>>>> [ERROR] Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time >>>>>>>>> elapsed: 0.001 s <<< FAILURE! - in >>>>>>>>> org.apache.calcite.test.CassandraAdapterTest >>>>>>>>> [ERROR] org.apache.calcite.test.CassandraAdapterTest Time >> elapsed: >>>>>>>>> 0.001 s <<< ERROR! >>>>>>>>> java.lang.ExceptionInInitializerError >>>>>>>>> at >>>>>>>>> >>>>>>>> >> org.apache.calcite.test.CassandraAdapterTest.initCassandraIfEnabled( >>>>>>> CassandraAdapterTest.java:106) >>>>>>>>> at >>>>>>>>> >>>>>>>> org.apache.calcite.test.CassandraAdapterTest.<clinit>( >>>>>>> CassandraAdapterTest.java:56) >>>>>>>>> Caused by: java.lang.StringIndexOutOfBoundsException: begin 0, end >>>>>> -1, >>>>>>>>> length 5 >>>>>>>>> at >>>>>>>>> >>>>>>>> >> org.apache.calcite.test.CassandraAdapterTest.initCassandraIfEnabled( >>>>>>> CassandraAdapterTest.java:106) >>>>>>>>> at >>>>>>>>> >>>>>>>> org.apache.calcite.test.CassandraAdapterTest.<clinit>( >>>>>>> CassandraAdapterTest.java:56) >>>>>>>>> >>>>>>>>> Here's the error in JDK 11: >>>>>>>>> >>>>>>>>> [INFO] Running org.apache.calcite.test.CassandraAdapterTest >>>>>>>>> [ERROR] Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time >>>>>>>>> elapsed: 0 s <<< FAILURE! - in >>>>>>>>> org.apache.calcite.test.CassandraAdapterTest >>>>>>>>> [ERROR] org.apache.calcite.test.CassandraAdapterTest Time >> elapsed: >>>>>> 0 >>>>>>>>> s <<< ERROR! >>>>>>>>> java.lang.ExceptionInInitializerError >>>>>>>>> at >>>>>>>>> >>>>>>>> org.apache.calcite.test.CassandraAdapterTest.enabled( >>>>>>> CassandraAdapterTest.java:81) >>>>>>>>> at >>>>>>>>> >>>>>>>> >> org.apache.calcite.test.CassandraAdapterTest.initCassandraIfEnabled( >>>>>>> CassandraAdapterTest.java:88) >>>>>>>>> at >>>>>>>>> >>>>>>>> org.apache.calcite.test.CassandraAdapterTest.<clinit>( >>>>>>> CassandraAdapterTest.java:56) >>>>>>>>> >>>>>>>>> Here's the failure from JDK 9: >>>>>>>>> >>>>>>>>> [INFO] Running org.apache.calcite.test.CassandraAdapterTest >>>>>>>>> [ERROR] Tests run: 1, Failures: 1, Errors: 0, Skipped: 0, Time >>>>>>>>> elapsed: 0.001 s <<< FAILURE! - in >>>>>>>>> org.apache.calcite.test.CassandraAdapterTest >>>>>>>>> [ERROR] org.apache.calcite.test.CassandraAdapterTest Time >> elapsed: >>>>>>>>> 0.001 s <<< FAILURE! >>>>>>>>> java.lang.AssertionError: Cassandra daemon did not start within >>>>>> timeout >>>>>>>>> >>>>>>>>> Here's the failure from JDK 10: >>>>>>>>> >>>>>>>>> [INFO] Running org.apache.calcite.jdbc.CalciteRemoteDriverTest >>>>>>>>> [ERROR] Tests run: 28, Failures: 1, Errors: 0, Skipped: 0, Time >>>>>>>>> elapsed: 0.642 s <<< FAILURE! - in >>>>>>>>> org.apache.calcite.jdbc.CalciteRemoteDriverTest >>>>>>>>> [ERROR] >>>>>>>>> testRemoteExecuteQuery(org.apache.calcite.jdbc. >>>>>>> CalciteRemoteDriverTest) >>>>>>>>> Time elapsed: 0.047 s <<< FAILURE! >>>>>>>>> java.lang.AssertionError: >>>>>>>>> >>>>>>>>> Expected: "EXPR$0=1; EXPR$1=a\nEXPR$0=null; EXPR$1=b" >>>>>>>>> but: was "" >>>>>>>>> at >>>>>>>>> >>>>>>>> >>>>>> >> org.apache.calcite.jdbc.CalciteRemoteDriverTest.testRemoteExecuteQuery( >>>>>>> CalciteRemoteDriverTest.java:280) >>>>>>>>> >>>>>>>>> In conclusion: It's worrying that the suite shows 4 different >> cracks >>>>>>>>> under 4 different JDKs. Clearly the load on my machine was making >>>>>>>>> problems worse, and granted, the problems are just testing >> problems, >>>>>>>>> not real bugs. But flaky test suites waste time and effort. There >>>>>> are >>>>>>>>> indications that the new embedded Cassandra test is more flaky >> than >>>>>>>>> most. >>>>>>>>> >>>>>>>>> Julian >>>>>>>>> >>>>>>>>> >>>>>>>>> On Sun, Jul 29, 2018 at 11:33 AM, Andrei Sereda <[email protected] >>> >>>>>>>> wrote: >>>>>>>>>> What version of java / OS / maven do you have ? What is your >> maven >>>>>>>>> command >>>>>>>>>> ? >>>>>>>>>> >>>>>>>>>> Things pass for me on MacOS X java 8,9 and 10. >>>>>>>>>> >>>>>>>>>> On Sun, Jul 29, 2018 at 2:05 PM Julian Hyde <[email protected]> >>>>>>> wrote: >>>>>>>>>> >>>>>>>>>>> I'm seeing the following error when I run the tests on bd0e14002 >>>>>>>>>>> origin/master. Anyone else see it? >>>>>>>>>>> >>>>>>>>>>> [INFO] Running org.apache.calcite.test.CassandraAdapterTest >>>>>>>>>>> [ERROR] Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time >>>>>>>>>>> elapsed: 0.001 s <<< FAILURE! - in >>>>>>>>>>> org.apache.calcite.test.CassandraAdapterTest >>>>>>>>>>> [ERROR] org.apache.calcite.test.CassandraAdapterTest Time >>>>>> elapsed: >>>>>>>>>>> 0.001 s <<< ERROR! >>>>>>>>>>> java.lang.ExceptionInInitializerError >>>>>>>>>>> at >>>>>>>>>>> >>>>>>>>> >>>>>>>> >> org.apache.calcite.test.CassandraAdapterTest.initCassandraIfEnabled( >>>>>>> CassandraAdapterTest.java:106) >>>>>>>>>>> at >>>>>>>>>>> >>>>>>>>> >>>>>>>> org.apache.calcite.test.CassandraAdapterTest.<clinit>( >>>>>>> CassandraAdapterTest.java:56) >>>>>>>>>>> Caused by: java.lang.StringIndexOutOfBoundsException: begin 0, >>>>>> end >>>>>>> -1, >>>>>>>>>>> length 5 >>>>>>>>>>> at >>>>>>>>>>> >>>>>>>>> >>>>>>>> >> org.apache.calcite.test.CassandraAdapterTest.initCassandraIfEnabled( >>>>>>> CassandraAdapterTest.java:106) >>>>>>>>>>> at >>>>>>>>>>> >>>>>>>>> >>>>>>>> org.apache.calcite.test.CassandraAdapterTest.<clinit>( >>>>>>> CassandraAdapterTest.java:56) >>>>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>> >>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> Best regards, >>>>>> Sergey >>>>>> >>>>> >>> >>> >>
