Thanks Andrei for digging into this! Since I haven't reproduced this failure myself, if someone else could check out the PR, that would be great. -- Michael Mior [email protected]
Le lun. 30 juil. 2018 à 13:29, Andrei Sereda <[email protected]> a écrit : > Agree, flaky tests are pretty annoying. I'll try to watch more carefully > new "embedded data-source" issues (fongo, ES, cassandra). They introduced > more "non-determinism" because they now run as part of regular build which > means executed much more often than IT. > > Last commit was for a more deterministic issue (incorrect parsing of new > java version format <http://openjdk.java.net/jeps/223>). > > > On Mon, Jul 30, 2018 at 12:58 PM Julian Hyde <[email protected]> wrote: > > > On balance, I don’t think we should back out CassandraAdapterTest. But we > > do need to continue working to make it more resilient. Any test that > > generates too many false negatives over the long run should be disabled, > > and this is no exception. > > > > And as the other test results show, it’s not the only flaky part of the > > test suite, when the machine is stressed. One thing we can do is to avoid > > timeouts whenever possible. > > > > Julian > > > > > > > On Jul 30, 2018, at 9:30 AM, Andrei Sereda <[email protected]> wrote: > > > > > > Pls check and confirm that the following PR fixes the issue: > > > https://github.com/apache/calcite/pull/770 > > > It addresses build failures due to version parsing. > > > > > > Stale folders / files will be addressed in a different PR. > > > > > > > > > On Mon, Jul 30, 2018 at 10:10 AM Andrei Sereda <[email protected]> > wrote: > > > > > >> This must be something specific to Cassandra Unit. Will check > > >> > > >> On Mon, Jul 30, 2018, 08:59 Sergey Nuyanzin <[email protected]> > > wrote: > > >> > > >>> There is one more strange thing (at least on Windows): while > building a > > >>> file with name ".toDelete" is generated under calcite\cassandra and > > it's > > >>> not removed by the end of tests > > >>> Is there a way to make cassandra generates these files in target > > directory > > >>> e.g.? > > >>> > > >>> > > >>> > > >>> On Mon, Jul 30, 2018 at 3:52 PM, Andrei Sereda <[email protected]> > > wrote: > > >>> > > >>>> Most of the problems are during test init Phase. Most likely with > > >>> version > > >>>> string (eg. 11-ea for JDK11). > > >>>> I'll fix that. > > >>>> > > >>>> > > >>>> On Mon, Jul 30, 2018 at 8:16 AM Michael Mior <[email protected]> > > wrote: > > >>>> > > >>>>> I tested myself a fair bit under Ubuntu before pushing this and > > didn't > > >>>> see > > >>>>> any of these issues myself. That said, I agree that it's important > > the > > >>>> test > > >>>>> suite be stable. I'm fine with reverting for now or (more > preferably > > >>> IMO) > > >>>>> just disabling these tests by default. > > >>>>> > > >>>>> -- > > >>>>> Michael Mior > > >>>>> [email protected] > > >>>>> > > >>>>> > > >>>>> > > >>>>> Le lun. 30 juil. 2018 à 00:59, Julian Hyde <[email protected]> a > > >>> écrit : > > >>>>> > > >>>>>> I ran on several JDK versions, all on Ubuntu Linux. The machine > was > > >>>>>> fairly heavily loaded (I was copying one filesystem to another at > > >>> the > > >>>>>> time). > > >>>>>> > > >>>>>> Here's the error from OpenJDK10: > > >>>>>> > > >>>>>> [INFO] Running org.apache.calcite.test.CassandraAdapterTest > > >>>>>> [ERROR] Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time > > >>>>>> elapsed: 0.001 s <<< FAILURE! - in > > >>>>>> org.apache.calcite.test.CassandraAdapterTest > > >>>>>> [ERROR] org.apache.calcite.test.CassandraAdapterTest Time > elapsed: > > >>>>>> 0.001 s <<< ERROR! > > >>>>>> java.lang.ExceptionInInitializerError > > >>>>>> at > > >>>>>> > > >>>>> > org.apache.calcite.test.CassandraAdapterTest.initCassandraIfEnabled( > > >>>> CassandraAdapterTest.java:106) > > >>>>>> at > > >>>>>> > > >>>>> org.apache.calcite.test.CassandraAdapterTest.<clinit>( > > >>>> CassandraAdapterTest.java:56) > > >>>>>> Caused by: java.lang.StringIndexOutOfBoundsException: begin 0, end > > >>> -1, > > >>>>>> length 5 > > >>>>>> at > > >>>>>> > > >>>>> > org.apache.calcite.test.CassandraAdapterTest.initCassandraIfEnabled( > > >>>> CassandraAdapterTest.java:106) > > >>>>>> at > > >>>>>> > > >>>>> org.apache.calcite.test.CassandraAdapterTest.<clinit>( > > >>>> CassandraAdapterTest.java:56) > > >>>>>> > > >>>>>> Here's the error in JDK 11: > > >>>>>> > > >>>>>> [INFO] Running org.apache.calcite.test.CassandraAdapterTest > > >>>>>> [ERROR] Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time > > >>>>>> elapsed: 0 s <<< FAILURE! - in > > >>>>>> org.apache.calcite.test.CassandraAdapterTest > > >>>>>> [ERROR] org.apache.calcite.test.CassandraAdapterTest Time > elapsed: > > >>> 0 > > >>>>>> s <<< ERROR! > > >>>>>> java.lang.ExceptionInInitializerError > > >>>>>> at > > >>>>>> > > >>>>> org.apache.calcite.test.CassandraAdapterTest.enabled( > > >>>> CassandraAdapterTest.java:81) > > >>>>>> at > > >>>>>> > > >>>>> > org.apache.calcite.test.CassandraAdapterTest.initCassandraIfEnabled( > > >>>> CassandraAdapterTest.java:88) > > >>>>>> at > > >>>>>> > > >>>>> org.apache.calcite.test.CassandraAdapterTest.<clinit>( > > >>>> CassandraAdapterTest.java:56) > > >>>>>> > > >>>>>> Here's the failure from JDK 9: > > >>>>>> > > >>>>>> [INFO] Running org.apache.calcite.test.CassandraAdapterTest > > >>>>>> [ERROR] Tests run: 1, Failures: 1, Errors: 0, Skipped: 0, Time > > >>>>>> elapsed: 0.001 s <<< FAILURE! - in > > >>>>>> org.apache.calcite.test.CassandraAdapterTest > > >>>>>> [ERROR] org.apache.calcite.test.CassandraAdapterTest Time > elapsed: > > >>>>>> 0.001 s <<< FAILURE! > > >>>>>> java.lang.AssertionError: Cassandra daemon did not start within > > >>> timeout > > >>>>>> > > >>>>>> Here's the failure from JDK 10: > > >>>>>> > > >>>>>> [INFO] Running org.apache.calcite.jdbc.CalciteRemoteDriverTest > > >>>>>> [ERROR] Tests run: 28, Failures: 1, Errors: 0, Skipped: 0, Time > > >>>>>> elapsed: 0.642 s <<< FAILURE! - in > > >>>>>> org.apache.calcite.jdbc.CalciteRemoteDriverTest > > >>>>>> [ERROR] > > >>>>>> testRemoteExecuteQuery(org.apache.calcite.jdbc. > > >>>> CalciteRemoteDriverTest) > > >>>>>> Time elapsed: 0.047 s <<< FAILURE! > > >>>>>> java.lang.AssertionError: > > >>>>>> > > >>>>>> Expected: "EXPR$0=1; EXPR$1=a\nEXPR$0=null; EXPR$1=b" > > >>>>>> but: was "" > > >>>>>> at > > >>>>>> > > >>>>> > > >>> > org.apache.calcite.jdbc.CalciteRemoteDriverTest.testRemoteExecuteQuery( > > >>>> CalciteRemoteDriverTest.java:280) > > >>>>>> > > >>>>>> In conclusion: It's worrying that the suite shows 4 different > cracks > > >>>>>> under 4 different JDKs. Clearly the load on my machine was making > > >>>>>> problems worse, and granted, the problems are just testing > problems, > > >>>>>> not real bugs. But flaky test suites waste time and effort. There > > >>> are > > >>>>>> indications that the new embedded Cassandra test is more flaky > than > > >>>>>> most. > > >>>>>> > > >>>>>> Julian > > >>>>>> > > >>>>>> > > >>>>>> On Sun, Jul 29, 2018 at 11:33 AM, Andrei Sereda <[email protected] > > > > >>>>> wrote: > > >>>>>>> What version of java / OS / maven do you have ? What is your > maven > > >>>>>> command > > >>>>>>> ? > > >>>>>>> > > >>>>>>> Things pass for me on MacOS X java 8,9 and 10. > > >>>>>>> > > >>>>>>> On Sun, Jul 29, 2018 at 2:05 PM Julian Hyde <[email protected]> > > >>>> wrote: > > >>>>>>> > > >>>>>>>> I'm seeing the following error when I run the tests on bd0e14002 > > >>>>>>>> origin/master. Anyone else see it? > > >>>>>>>> > > >>>>>>>> [INFO] Running org.apache.calcite.test.CassandraAdapterTest > > >>>>>>>> [ERROR] Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time > > >>>>>>>> elapsed: 0.001 s <<< FAILURE! - in > > >>>>>>>> org.apache.calcite.test.CassandraAdapterTest > > >>>>>>>> [ERROR] org.apache.calcite.test.CassandraAdapterTest Time > > >>> elapsed: > > >>>>>>>> 0.001 s <<< ERROR! > > >>>>>>>> java.lang.ExceptionInInitializerError > > >>>>>>>> at > > >>>>>>>> > > >>>>>> > > >>>>> > org.apache.calcite.test.CassandraAdapterTest.initCassandraIfEnabled( > > >>>> CassandraAdapterTest.java:106) > > >>>>>>>> at > > >>>>>>>> > > >>>>>> > > >>>>> org.apache.calcite.test.CassandraAdapterTest.<clinit>( > > >>>> CassandraAdapterTest.java:56) > > >>>>>>>> Caused by: java.lang.StringIndexOutOfBoundsException: begin 0, > > >>> end > > >>>> -1, > > >>>>>>>> length 5 > > >>>>>>>> at > > >>>>>>>> > > >>>>>> > > >>>>> > org.apache.calcite.test.CassandraAdapterTest.initCassandraIfEnabled( > > >>>> CassandraAdapterTest.java:106) > > >>>>>>>> at > > >>>>>>>> > > >>>>>> > > >>>>> org.apache.calcite.test.CassandraAdapterTest.<clinit>( > > >>>> CassandraAdapterTest.java:56) > > >>>>>>>> > > >>>>>> > > >>>>> > > >>>> > > >>> > > >>> > > >>> > > >>> -- > > >>> Best regards, > > >>> Sergey > > >>> > > >> > > > > >
