+1 I verified things listed on Matthias's checklist and successfully built and ran tests on both hadoop-1 and hadoop-2.
2013/11/8 Josh Wills <[email protected]> > On Thu, Nov 7, 2013 at 9:14 AM, Matthias Friedrich <[email protected]> wrote: > > > Yup, that sounds plausible. In any case, even with a sporadically > > failing integration test, the release is still valid from a legal > > point of view. It's up to the release manager to decide whether > > we need to restart the ceremony :) > > > > It's certainly annoying and something to fix for the next release, but no, > I don't think it merits a new RC. > > > > > > Regards, > > Matthias > > > > On Thursday, 2013-11-07, Josh Wills wrote: > > > That does sound plausible-- I actually just ran into that issue on > > another > > > project I'm working on (i.e., like yesterday.) > > > > > > > > > On Thu, Nov 7, 2013 at 8:23 AM, Gabriel Reid <[email protected]> > > wrote: > > > > > > > On Thu, Nov 7, 2013 at 4:13 PM, Chao Shi <[email protected]> wrote: > > > > > I think this failure may depend on the execution order of tests, > > because: > > > > > > > > > > 1) I can reproduce this failure on some source tree, but cannot on > > > > another > > > > > 2) I can only reproduce it with "mvn install", but "mvn test > > > > > -Dtest=org.apache.crunch.CheckpointIT" cannot > > > > > 3) I was trying to add some printf in the case, after saving the > > file (I > > > > > modified that CheckpointIT.java), I cannot reproduce it now. > > > > > > > > > > Does anyone have some clue? > > > > > > > > I have a feeling that this isn't due to order of execution. If it is > > > > down to that, then as far as I can see it would mean that there is > > > > something wrong with the TemporaryPath class, as that seems to be the > > > > only thing that is shared between tests in that class. > > > > > > > > I'm actually wondering if this could be down to a timing issue in the > > > > checkpointing code. The FileTargetImpl class checks if the > > > > lastModified timestamp of the checkpointed file is greater than the > > > > lastModified timestamp on the input source. Based on a check I just > > > > did locally here on Mac OS X and on a Linux machine with ext4, the > > > > time granularity of the lastModified timestamp on files is only > > > > second-based, not millisecond-based. I'm thinking that this could be > > > > an issue of the checkpointed file being created less than a second > > > > after the startup of the test and creation of the source file > > > > (although that seems really quick), and so the checkpointing wouldn't > > > > think that the source file is older than the checkpointed file, so it > > > > would run the full pipeline again (therefore resulting in two > > > > execution stages instead of one. > > > > > > > > Does that sound plausible? If so, the only thing I can see that would > > > > keep the code correct and prevent the test from failing would be to > > > > add a Thread.sleep() into the test, but I hate having stuff like that > > > > in tests. > > > > > > > > - Gabriel > > > > > > > > > > > > > > > > -- > > > Director of Data Science > > > Cloudera <http://www.cloudera.com> > > > Twitter: @josh_wills <http://twitter.com/josh_wills> > > > > > > -- > Director of Data Science > Cloudera <http://www.cloudera.com> > Twitter: @josh_wills <http://twitter.com/josh_wills> >
