Question regarding test.junit.thread - when it is >1 like on hudson, is there a way to separate the logging messages from the different threads or identify which message belongs to which thread ?
On Tue, Jun 2, 2015 at 10:37 AM, Chris Nauroth <[email protected]> wrote: > I didn't notice any other problems. I was +1 (non-binding) on RC1, aside > from the discussion of these 2 issues. Thanks for your hard work on this > RC, Michi! > > --Chris Nauroth > > > > > On 6/1/15, 10:29 PM, "Michi Mutsuzaki" <[email protected]> wrote: > > >Ok both of these issues are resolved now. Let me know if you guys > >noticed any other issues with RC1. If not, I'll create another RC this > >weekend. > > > >On Sat, May 30, 2015 at 10:55 PM, Chris Nauroth > ><[email protected]> wrote: > >> Thank you, Michi. I filed a patch for this on ZOOKEEPER-2198. > >> > >> --Chris Nauroth > >> > >> > >> > >> > >> On 5/30/15, 1:19 PM, "Michi Mutsuzaki" <[email protected]> wrote: > >> > >>>Ok, since the vote didn't pass anyways, let's fix these problems: > >>> > >>>1. Change the default test.junit.thread to 1. Chris, could you submit > >>>a patch for this? > >>>2. Fix the comment in FinalRequestProcessor.java. I'll submit a patch. > >>> > >>>Let me know if you guys have seen any other problems. Also, please let > >>>me know if the voting period of 2 weeks was too short. I'd like to > >>>make sure everybody gets enough time to vote. > >>> > >>> > >>>On Sat, May 30, 2015 at 8:55 AM, Flavio Junqueira > >>><[email protected]> wrote: > >>>> Another thing that is possibly not a reason to drop the config, but > >>>>I'm > >>>>getting this with this RC: > >>>> > >>>> [javac] > >>>>/home/fpj/code/zookeeper-3.5.1-alpha/src/java/main/org/apache/zookeeper > >>>>/s > >>>>erver/FinalRequestProcessor.java:134: error: unmappable character for > >>>>encoding ASCII > >>>> [javac] // was not being queued ??? ZOOKEEPER-558) > >>>>properly. This happens, for example, > >>>> > >>>> It is a trivial problem to solve, but it does generate a compilation > >>>>error for me. > >>>> > >>>> -Flavio > >>>> > >>>>> On 30 May 2015, at 15:26, Flavio Junqueira > >>>>><[email protected]> wrote: > >>>>> > >>>>> I don't see a reason to -1 the release just because of the number of > >>>>>threads junit is using. I've been a bit distracted with other things, > >>>>>but I'm coming back to the release candidate now. > >>>>> > >>>>> -Flavio > >>>>> > >>>>> > >>>>>> On 23 May 2015, at 22:09, Michi Mutsuzaki <[email protected]> > >>>>>>wrote: > >>>>>> > >>>>>> I can go either way. Flavio, do you think we should set the default > >>>>>> test.junit.threads to 1 and create another release candidate? > >>>>>> > >>>>>> On Fri, May 22, 2015 at 5:08 PM, Chris Nauroth > >>>>>><[email protected]> wrote: > >>>>>>> I haven't been able to repro this locally. Here are the details on > >>>>>>>my Ubuntu VM: > >>>>>>> > >>>>>>> uname -a > >>>>>>> Linux ubuntu 3.16.0-30-generic #40~14.04.1-Ubuntu SMP Thu Jan 15 > >>>>>>>17:43:14 UTC 2015 x86_64 x86_64 x86_64 GNU/Linux > >>>>>>> > >>>>>>> java -version > >>>>>>> java version "1.8.0_45" > >>>>>>> Java(TM) SE Runtime Environment (build 1.8.0_45-b14) > >>>>>>> Java HotSpot(TM) 64-Bit Server VM (build 25.45-b02, mixed mode) > >>>>>>> > >>>>>>> ant -version > >>>>>>> Apache Ant(TM) version 1.9.4 compiled on April 29 2014 > >>>>>>> > >>>>>>> I'm getting 100% passing test runs with multiple concurrent JUnit > >>>>>>>processes, including the tests that you mentioned were failing in > >>>>>>>your environment. > >>>>>>> > >>>>>>> I don't have any immediate ideas for what to try next. Everything > >>>>>>>has been working well on Jenkins and multiple dev machines, so it > >>>>>>>seems like there is some subtle environmental difference in this VM > >>>>>>>that I didn't handle in the ZOOKEEPER-2183 patch. > >>>>>>> > >>>>>>> Is this problematic for the release candidate? If so, then I > >>>>>>>recommend doing a quick change to set the default test.junit.threads > >>>>>>>to 1 in build.xml. That would restore the old single-process > >>>>>>>testing > >>>>>>>behavior. We can change test-patch.sh to pass > >>>>>>>-Dtest.junit.threads=8 > >>>>>>>on the command line, so we'll still get speedy pre-commit runs on > >>>>>>>Jenkins where it is working well. We all can do the same when we > >>>>>>>run > >>>>>>>ant locally too. Let me know if this is important, and I can put > >>>>>>>together a patch quickly. > >>>>>>> > >>>>>>> Thanks! > >>>>>>> > >>>>>>> --Chris Nauroth > >>>>>>> > >>>>>>> From: Flavio Junqueira > >>>>>>><[email protected]<mailto:[email protected]>> > >>>>>>> Date: Friday, May 22, 2015 at 3:37 PM > >>>>>>> To: Chris Nauroth > >>>>>>><[email protected]<mailto:[email protected]>> > >>>>>>> Cc: Zookeeper > >>>>>>><[email protected]<mailto:[email protected]>> > >>>>>>> Subject: Re: [VOTE] Apache ZooKeeper release 3.5.1-alpha candidate > >>>>>>>1 > >>>>>>> > >>>>>>> That's the range I get in the vm. I also checked the load from log > >>>>>>>test and the port it was trying to bind to is 11222. > >>>>>>> > >>>>>>> -Flavio > >>>>>>> > >>>>>>> On 22 May 2015, at 23:14, Chris Nauroth > >>>>>>><[email protected]<mailto:[email protected]>> wrote: > >>>>>>> > >>>>>>> No worries on the delay. Thank you for sharing. > >>>>>>> > >>>>>>> That's interesting. The symptoms look similar to something we had > >>>>>>>seen from an earlier iteration of the ZOOKEEPER-2183 patch that was > >>>>>>>assigning ports from the ephemeral port range. This would cause a > >>>>>>>brief (but noticeable) window in which the OS could assign the same > >>>>>>>ephemeral port to a client socket while a server test still held > >>>>>>>onto > >>>>>>>that port assignment. It was particularly noticeable for tests that > >>>>>>>stop and restart a server on the same port, such as tests covering > >>>>>>>client reconnect logic. In the final committed version of the > >>>>>>>ZOOKEEPER-2183 patch, I excluded the ephemeral port range from use > >>>>>>>by > >>>>>>>port assignment. Typically, that's 32768 - 61000 on Linux. > >>>>>>> > >>>>>>> Is it possible that this VM is configured to use a different > >>>>>>>ephemeral port range? Here is what I get from recent stock Ubuntu > >>>>>>>and CentOS installs: > >>>>>>> > >>>>>>>> cat /proc/sys/net/ipv4/ip_local_port_range > >>>>>>> 32768 61000 > >>>>>>> > >>>>>>> --Chris Nauroth > >>>>>>> > >>>>>>> From: Flavio Junqueira > >>>>>>><[email protected]<mailto:[email protected]>> > >>>>>>> Date: Friday, May 22, 2015 at 2:47 PM > >>>>>>> To: Chris Nauroth > >>>>>>><[email protected]<mailto:[email protected]>> > >>>>>>> Cc: Zookeeper > >>>>>>><[email protected]<mailto:[email protected]>> > >>>>>>> Subject: Re: [VOTE] Apache ZooKeeper release 3.5.1-alpha candidate > >>>>>>>1 > >>>>>>> > >>>>>>> Sorry about the delay, here are the logs: > >>>>>>> > >>>>>>> http://people.apache.org/~fpj/logs-3.5.1-rc1/ > >>>>>>> > >>>>>>> the load test is giving bind exceptions. > >>>>>>> > >>>>>>> -Flavio > >>>>>>> > >>>>>>> On 21 May 2015, at 23:02, Chris Nauroth > >>>>>>><[email protected]<mailto:[email protected]>> wrote: > >>>>>>> > >>>>>>> Thanks, sharing logs would be great. I'll try to repro > >>>>>>>independently with > >>>>>>> JDK8 too. > >>>>>>> > >>>>>>> --Chris Nauroth > >>>>>>> > >>>>>>> > >>>>>>> > >>>>>>> > >>>>>>> On 5/21/15, 2:30 PM, "Flavio Junqueira" > >>>>>>><[email protected]<mailto:[email protected] > > > >>>>>>>> > >>>>>>> wrote: > >>>>>>> > >>>>>>> I accidently removed dev from the response, bringing it back in. > >>>>>>> The tests are failing intermittently for me. In the last run, I got > >>>>>>>these > >>>>>>> failing: > >>>>>>> [junit] Tests run: 8, Failures: 0, Errors: 4, Skipped: 0, Time > >>>>>>>elapsed: > >>>>>>> 30.444 sec[junit] Test org.apache.zookeeper.test.LoadFromLogTest > >>>>>>>FAILED > >>>>>>> [junit] Tests run: 86, Failures: 0, Errors: 2, Skipped: 0, Time > >>>>>>>elapsed: > >>>>>>> 264.272 sec[junit] Test org.apache.zookeeper.test.NioNettySuiteTest > >>>>>>>FAILED > >>>>>>> Still the same setup, linux + jdk 8. I can share logs if necessary. > >>>>>>> -Flavio > >>>>>>> > >>>>>>> > >>>>>>> On Thursday, May 21, 2015 8:28 PM, Chris Nauroth > >>>>>>> <[email protected]<mailto:[email protected]>> wrote: > >>>>>>> > >>>>>>> > >>>>>>> > >>>>>>> Ah, my mistake. I saw "Azure" and my brain jumped right to > >>>>>>>"Windows". > >>>>>>> I suppose the thing for me to check then is JDK8. I believe all > >>>>>>>prior > >>>>>>> testing was on JDK7. > >>>>>>> --Chris Nauroth > >>>>>>> From: Flavio Junqueira > >>>>>>><[email protected]<mailto:[email protected]>> > >>>>>>> Date: Thursday, May 21, 2015 at 12:18 PM > >>>>>>> To: Chris Nauroth > >>>>>>><[email protected]<mailto:[email protected]>> > >>>>>>> Subject: RE: [VOTE] Apache ZooKeeper release 3.5.1-alpha candidate > >>>>>>>1 > >>>>>>> > >>>>>>> Yeah, I started with an Ubuntu vm, so it's Linux. I haven't tested > >>>>>>>the RC > >>>>>>> on windows yet. > >>>>>>> > >>>>>>> -FlavioFrom:Chris Nauroth > >>>>>>> Sent:?5/?21/?2015 6:46 PM > >>>>>>> To:[email protected]<http://zookeeper.apache.org/>;Flavio > >>>>>>>Junqueira > >>>>>>> Subject:Re: [VOTE] Apache ZooKeeper release 3.5.1-alpha candidate 1 > >>>>>>> > >>>>>>> If I understand correctly, you're seeing test failures specifically > >>>>>>>on > >>>>>>> Windows (not Linux) after ZOOKEEPER-2183. Is that right? > >>>>>>> > >>>>>>> Tests have been stable in Linux Jenkins and dev environments after > >>>>>>>that > >>>>>>> patch, but perhaps there is another issue specific to Windows. > >>>>>>>I'll > >>>>>>>take > >>>>>>> a look on Windows. It might also be worthwhile to detect Windows > >>>>>>>and set > >>>>>>> test.junit.threads to 1 automatically in build.xml as a stop-gap. > >>>>>>> > >>>>>>> --Chris Nauroth > >>>>>>> > >>>>>>> > >>>>>>> > >>>>>>> > >>>>>>> On 5/21/15, 9:05 AM, "Flavio Junqueira" > >>>>>>><[email protected]<mailto:[email protected] > > > >>>>>>>> > >>>>>>> wrote: > >>>>>>> > >>>>>>> Yep, that did it. > >>>>>>> -Flavio > >>>>>>> > >>>>>>> > >>>>>>> On Thursday, May 21, 2015 5:23 AM, Michi Mutsuzaki > >>>>>>> <[email protected]<mailto:[email protected]>> wrote: > >>>>>>> > >>>>>>> > >>>>>>> > >>>>>>> I wonder if it's related to ZOOKEEPER-2183. Could you try setting > >>>>>>> test.junit.threads to 1 in build.xml? > >>>>>>> > >>>>>>> On Wed, May 20, 2015 at 1:44 PM, Flavio Junqueira > >>>>>>> > >>>>>>><[email protected]<mailto:[email protected] > > > >>>>>>>> > >>>>>>>wrote: > >>>>>>> I'm not being able to get a clean build for the RC. I'm running it > >>>>>>>on > >>>>>>> an azure vm with ubuntu and oracle jdk8. The java tests failing > >>>>>>>vary. At > >>>>>>> this point, I just wanted to check if I'm the only one seeing > >>>>>>>failures. > >>>>>>> -Flavio > >>>>>>> > >>>>>>> > >>>>>>> On Saturday, May 16, 2015 6:25 AM, Michi Mutsuzaki > >>>>>>> <[email protected]<mailto:[email protected]>> wrote: > >>>>>>> > >>>>>>> > >>>>>>> > >>>>>>> This is the second release candidate for 3.5.1-alpha. This > >>>>>>>candidate > >>>>>>> fixes some issues found in the first candidate, including > >>>>>>> ZOOKEEPER-2171. The full release notes is > >>>>>>> available at: > >>>>>>> > >>>>>>> > >>>>>>> > https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=123 > >>>>>>>10 > >>>>>>>80 > >>>>>>> 1 > >>>>>>> &version=12326786 > >>>>>>> > >>>>>>> *** Please download, test and vote by May 29th 2015, 23:59 UTC+0. > >>>>>>>*** > >>>>>>> > >>>>>>> Source files: > >>>>>>> > http://people.apache.org/~michim/zookeeper-3.5.1-alpha-candidate-1/ > >>>>>>> > >>>>>>> Maven staging repo: > >>>>>>> > >>>>>>> > >>>>>>> > https://repository.apache.org/content/groups/staging/org/apache/zook > >>>>>>>ee > >>>>>>>pe > >>>>>>> r > >>>>>>> /zookeeper/3.5.1-alpha/ > >>>>>>> > >>>>>>> The tag to be voted upon: > >>>>>>> https://svn.apache.org/repos/asf/zookeeper/tags/release-3.5.1-rc1/ > >>>>>>> > >>>>>>> ZooKeeper's KEYS file containing PGP keys we use to sign the > >>>>>>>release: > >>>>>>> http://www.apache.org/dist/zookeeper/KEYS > >>>>>>> > >>>>>>> Should we release this candidate? > >>>>>>> > >>>>>>> > >>>>>>> > >>>>>>> > >>>>>>> > >>>>>>> > >>>>>>> > >>>>>>> > >>>>>>> > >>>>>>> > >>>>>>> > >>>>>>> > >>>>>>> > >>>>>>> > >>>>>>> > >>>>>>> > >>>>> > >>>> > >>> > >> > > > >
