On balance, I don’t think we should back out CassandraAdapterTest. But we do need to continue working to make it more resilient. Any test that generates too many false negatives over the long run should be disabled, and this is no exception.
And as the other test results show, it’s not the only flaky part of the test suite, when the machine is stressed. One thing we can do is to avoid timeouts whenever possible. Julian > On Jul 30, 2018, at 9:30 AM, Andrei Sereda <[email protected]> wrote: > > Pls check and confirm that the following PR fixes the issue: > https://github.com/apache/calcite/pull/770 > It addresses build failures due to version parsing. > > Stale folders / files will be addressed in a different PR. > > > On Mon, Jul 30, 2018 at 10:10 AM Andrei Sereda <[email protected]> wrote: > >> This must be something specific to Cassandra Unit. Will check >> >> On Mon, Jul 30, 2018, 08:59 Sergey Nuyanzin <[email protected]> wrote: >> >>> There is one more strange thing (at least on Windows): while building a >>> file with name ".toDelete" is generated under calcite\cassandra and it's >>> not removed by the end of tests >>> Is there a way to make cassandra generates these files in target directory >>> e.g.? >>> >>> >>> >>> On Mon, Jul 30, 2018 at 3:52 PM, Andrei Sereda <[email protected]> wrote: >>> >>>> Most of the problems are during test init Phase. Most likely with >>> version >>>> string (eg. 11-ea for JDK11). >>>> I'll fix that. >>>> >>>> >>>> On Mon, Jul 30, 2018 at 8:16 AM Michael Mior <[email protected]> wrote: >>>> >>>>> I tested myself a fair bit under Ubuntu before pushing this and didn't >>>> see >>>>> any of these issues myself. That said, I agree that it's important the >>>> test >>>>> suite be stable. I'm fine with reverting for now or (more preferably >>> IMO) >>>>> just disabling these tests by default. >>>>> >>>>> -- >>>>> Michael Mior >>>>> [email protected] >>>>> >>>>> >>>>> >>>>> Le lun. 30 juil. 2018 à 00:59, Julian Hyde <[email protected]> a >>> écrit : >>>>> >>>>>> I ran on several JDK versions, all on Ubuntu Linux. The machine was >>>>>> fairly heavily loaded (I was copying one filesystem to another at >>> the >>>>>> time). >>>>>> >>>>>> Here's the error from OpenJDK10: >>>>>> >>>>>> [INFO] Running org.apache.calcite.test.CassandraAdapterTest >>>>>> [ERROR] Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time >>>>>> elapsed: 0.001 s <<< FAILURE! - in >>>>>> org.apache.calcite.test.CassandraAdapterTest >>>>>> [ERROR] org.apache.calcite.test.CassandraAdapterTest Time elapsed: >>>>>> 0.001 s <<< ERROR! >>>>>> java.lang.ExceptionInInitializerError >>>>>> at >>>>>> >>>>> org.apache.calcite.test.CassandraAdapterTest.initCassandraIfEnabled( >>>> CassandraAdapterTest.java:106) >>>>>> at >>>>>> >>>>> org.apache.calcite.test.CassandraAdapterTest.<clinit>( >>>> CassandraAdapterTest.java:56) >>>>>> Caused by: java.lang.StringIndexOutOfBoundsException: begin 0, end >>> -1, >>>>>> length 5 >>>>>> at >>>>>> >>>>> org.apache.calcite.test.CassandraAdapterTest.initCassandraIfEnabled( >>>> CassandraAdapterTest.java:106) >>>>>> at >>>>>> >>>>> org.apache.calcite.test.CassandraAdapterTest.<clinit>( >>>> CassandraAdapterTest.java:56) >>>>>> >>>>>> Here's the error in JDK 11: >>>>>> >>>>>> [INFO] Running org.apache.calcite.test.CassandraAdapterTest >>>>>> [ERROR] Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time >>>>>> elapsed: 0 s <<< FAILURE! - in >>>>>> org.apache.calcite.test.CassandraAdapterTest >>>>>> [ERROR] org.apache.calcite.test.CassandraAdapterTest Time elapsed: >>> 0 >>>>>> s <<< ERROR! >>>>>> java.lang.ExceptionInInitializerError >>>>>> at >>>>>> >>>>> org.apache.calcite.test.CassandraAdapterTest.enabled( >>>> CassandraAdapterTest.java:81) >>>>>> at >>>>>> >>>>> org.apache.calcite.test.CassandraAdapterTest.initCassandraIfEnabled( >>>> CassandraAdapterTest.java:88) >>>>>> at >>>>>> >>>>> org.apache.calcite.test.CassandraAdapterTest.<clinit>( >>>> CassandraAdapterTest.java:56) >>>>>> >>>>>> Here's the failure from JDK 9: >>>>>> >>>>>> [INFO] Running org.apache.calcite.test.CassandraAdapterTest >>>>>> [ERROR] Tests run: 1, Failures: 1, Errors: 0, Skipped: 0, Time >>>>>> elapsed: 0.001 s <<< FAILURE! - in >>>>>> org.apache.calcite.test.CassandraAdapterTest >>>>>> [ERROR] org.apache.calcite.test.CassandraAdapterTest Time elapsed: >>>>>> 0.001 s <<< FAILURE! >>>>>> java.lang.AssertionError: Cassandra daemon did not start within >>> timeout >>>>>> >>>>>> Here's the failure from JDK 10: >>>>>> >>>>>> [INFO] Running org.apache.calcite.jdbc.CalciteRemoteDriverTest >>>>>> [ERROR] Tests run: 28, Failures: 1, Errors: 0, Skipped: 0, Time >>>>>> elapsed: 0.642 s <<< FAILURE! - in >>>>>> org.apache.calcite.jdbc.CalciteRemoteDriverTest >>>>>> [ERROR] >>>>>> testRemoteExecuteQuery(org.apache.calcite.jdbc. >>>> CalciteRemoteDriverTest) >>>>>> Time elapsed: 0.047 s <<< FAILURE! >>>>>> java.lang.AssertionError: >>>>>> >>>>>> Expected: "EXPR$0=1; EXPR$1=a\nEXPR$0=null; EXPR$1=b" >>>>>> but: was "" >>>>>> at >>>>>> >>>>> >>> org.apache.calcite.jdbc.CalciteRemoteDriverTest.testRemoteExecuteQuery( >>>> CalciteRemoteDriverTest.java:280) >>>>>> >>>>>> In conclusion: It's worrying that the suite shows 4 different cracks >>>>>> under 4 different JDKs. Clearly the load on my machine was making >>>>>> problems worse, and granted, the problems are just testing problems, >>>>>> not real bugs. But flaky test suites waste time and effort. There >>> are >>>>>> indications that the new embedded Cassandra test is more flaky than >>>>>> most. >>>>>> >>>>>> Julian >>>>>> >>>>>> >>>>>> On Sun, Jul 29, 2018 at 11:33 AM, Andrei Sereda <[email protected]> >>>>> wrote: >>>>>>> What version of java / OS / maven do you have ? What is your maven >>>>>> command >>>>>>> ? >>>>>>> >>>>>>> Things pass for me on MacOS X java 8,9 and 10. >>>>>>> >>>>>>> On Sun, Jul 29, 2018 at 2:05 PM Julian Hyde <[email protected]> >>>> wrote: >>>>>>> >>>>>>>> I'm seeing the following error when I run the tests on bd0e14002 >>>>>>>> origin/master. Anyone else see it? >>>>>>>> >>>>>>>> [INFO] Running org.apache.calcite.test.CassandraAdapterTest >>>>>>>> [ERROR] Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time >>>>>>>> elapsed: 0.001 s <<< FAILURE! - in >>>>>>>> org.apache.calcite.test.CassandraAdapterTest >>>>>>>> [ERROR] org.apache.calcite.test.CassandraAdapterTest Time >>> elapsed: >>>>>>>> 0.001 s <<< ERROR! >>>>>>>> java.lang.ExceptionInInitializerError >>>>>>>> at >>>>>>>> >>>>>> >>>>> org.apache.calcite.test.CassandraAdapterTest.initCassandraIfEnabled( >>>> CassandraAdapterTest.java:106) >>>>>>>> at >>>>>>>> >>>>>> >>>>> org.apache.calcite.test.CassandraAdapterTest.<clinit>( >>>> CassandraAdapterTest.java:56) >>>>>>>> Caused by: java.lang.StringIndexOutOfBoundsException: begin 0, >>> end >>>> -1, >>>>>>>> length 5 >>>>>>>> at >>>>>>>> >>>>>> >>>>> org.apache.calcite.test.CassandraAdapterTest.initCassandraIfEnabled( >>>> CassandraAdapterTest.java:106) >>>>>>>> at >>>>>>>> >>>>>> >>>>> org.apache.calcite.test.CassandraAdapterTest.<clinit>( >>>> CassandraAdapterTest.java:56) >>>>>>>> >>>>>> >>>>> >>>> >>> >>> >>> >>> -- >>> Best regards, >>> Sergey >>> >>
