I can go either way. Flavio, do you think we should set the default test.junit.threads to 1 and create another release candidate?
On Fri, May 22, 2015 at 5:08 PM, Chris Nauroth <[email protected]> wrote: > I haven't been able to repro this locally. Here are the details on my Ubuntu > VM: > > uname -a > Linux ubuntu 3.16.0-30-generic #40~14.04.1-Ubuntu SMP Thu Jan 15 17:43:14 UTC > 2015 x86_64 x86_64 x86_64 GNU/Linux > > java -version > java version "1.8.0_45" > Java(TM) SE Runtime Environment (build 1.8.0_45-b14) > Java HotSpot(TM) 64-Bit Server VM (build 25.45-b02, mixed mode) > > ant -version > Apache Ant(TM) version 1.9.4 compiled on April 29 2014 > > I'm getting 100% passing test runs with multiple concurrent JUnit processes, > including the tests that you mentioned were failing in your environment. > > I don't have any immediate ideas for what to try next. Everything has been > working well on Jenkins and multiple dev machines, so it seems like there is > some subtle environmental difference in this VM that I didn't handle in the > ZOOKEEPER-2183 patch. > > Is this problematic for the release candidate? If so, then I recommend doing > a quick change to set the default test.junit.threads to 1 in build.xml. That > would restore the old single-process testing behavior. We can change > test-patch.sh to pass -Dtest.junit.threads=8 on the command line, so we'll > still get speedy pre-commit runs on Jenkins where it is working well. We all > can do the same when we run ant locally too. Let me know if this is > important, and I can put together a patch quickly. > > Thanks! > > --Chris Nauroth > > From: Flavio Junqueira <[email protected]<mailto:[email protected]>> > Date: Friday, May 22, 2015 at 3:37 PM > To: Chris Nauroth <[email protected]<mailto:[email protected]>> > Cc: Zookeeper <[email protected]<mailto:[email protected]>> > Subject: Re: [VOTE] Apache ZooKeeper release 3.5.1-alpha candidate 1 > > That's the range I get in the vm. I also checked the load from log test and > the port it was trying to bind to is 11222. > > -Flavio > > On 22 May 2015, at 23:14, Chris Nauroth > <[email protected]<mailto:[email protected]>> wrote: > > No worries on the delay. Thank you for sharing. > > That's interesting. The symptoms look similar to something we had seen from > an earlier iteration of the ZOOKEEPER-2183 patch that was assigning ports > from the ephemeral port range. This would cause a brief (but noticeable) > window in which the OS could assign the same ephemeral port to a client > socket while a server test still held onto that port assignment. It was > particularly noticeable for tests that stop and restart a server on the same > port, such as tests covering client reconnect logic. In the final committed > version of the ZOOKEEPER-2183 patch, I excluded the ephemeral port range from > use by port assignment. Typically, that's 32768 - 61000 on Linux. > > Is it possible that this VM is configured to use a different ephemeral port > range? Here is what I get from recent stock Ubuntu and CentOS installs: > >> cat /proc/sys/net/ipv4/ip_local_port_range > 32768 61000 > > --Chris Nauroth > > From: Flavio Junqueira <[email protected]<mailto:[email protected]>> > Date: Friday, May 22, 2015 at 2:47 PM > To: Chris Nauroth <[email protected]<mailto:[email protected]>> > Cc: Zookeeper <[email protected]<mailto:[email protected]>> > Subject: Re: [VOTE] Apache ZooKeeper release 3.5.1-alpha candidate 1 > > Sorry about the delay, here are the logs: > > http://people.apache.org/~fpj/logs-3.5.1-rc1/ > > the load test is giving bind exceptions. > > -Flavio > > On 21 May 2015, at 23:02, Chris Nauroth > <[email protected]<mailto:[email protected]>> wrote: > > Thanks, sharing logs would be great. I'll try to repro independently with > JDK8 too. > > --Chris Nauroth > > > > > On 5/21/15, 2:30 PM, "Flavio Junqueira" > <[email protected]<mailto:[email protected]>> > wrote: > > I accidently removed dev from the response, bringing it back in. > The tests are failing intermittently for me. In the last run, I got these > failing: > [junit] Tests run: 8, Failures: 0, Errors: 4, Skipped: 0, Time elapsed: > 30.444 sec[junit] Test org.apache.zookeeper.test.LoadFromLogTest FAILED > [junit] Tests run: 86, Failures: 0, Errors: 2, Skipped: 0, Time elapsed: > 264.272 sec[junit] Test org.apache.zookeeper.test.NioNettySuiteTest FAILED > Still the same setup, linux + jdk 8. I can share logs if necessary. > -Flavio > > > On Thursday, May 21, 2015 8:28 PM, Chris Nauroth > <[email protected]<mailto:[email protected]>> wrote: > > > > Ah, my mistake. I saw "Azure" and my brain jumped right to "Windows". > I suppose the thing for me to check then is JDK8. I believe all prior > testing was on JDK7. > --Chris Nauroth > From: Flavio Junqueira <[email protected]<mailto:[email protected]>> > Date: Thursday, May 21, 2015 at 12:18 PM > To: Chris Nauroth <[email protected]<mailto:[email protected]>> > Subject: RE: [VOTE] Apache ZooKeeper release 3.5.1-alpha candidate 1 > > Yeah, I started with an Ubuntu vm, so it's Linux. I haven't tested the RC > on windows yet. > > -FlavioFrom:Chris Nauroth > Sent:?5/?21/?2015 6:46 PM > To:[email protected]<http://zookeeper.apache.org/>;Flavio Junqueira > Subject:Re: [VOTE] Apache ZooKeeper release 3.5.1-alpha candidate 1 > > If I understand correctly, you're seeing test failures specifically on > Windows (not Linux) after ZOOKEEPER-2183. Is that right? > > Tests have been stable in Linux Jenkins and dev environments after that > patch, but perhaps there is another issue specific to Windows. I'll take > a look on Windows. It might also be worthwhile to detect Windows and set > test.junit.threads to 1 automatically in build.xml as a stop-gap. > > --Chris Nauroth > > > > > On 5/21/15, 9:05 AM, "Flavio Junqueira" > <[email protected]<mailto:[email protected]>> > wrote: > > Yep, that did it. > -Flavio > > > On Thursday, May 21, 2015 5:23 AM, Michi Mutsuzaki > <[email protected]<mailto:[email protected]>> wrote: > > > > I wonder if it's related to ZOOKEEPER-2183. Could you try setting > test.junit.threads to 1 in build.xml? > > On Wed, May 20, 2015 at 1:44 PM, Flavio Junqueira > <[email protected]<mailto:[email protected]>> wrote: > I'm not being able to get a clean build for the RC. I'm running it on > an azure vm with ubuntu and oracle jdk8. The java tests failing vary. At > this point, I just wanted to check if I'm the only one seeing failures. > -Flavio > > > On Saturday, May 16, 2015 6:25 AM, Michi Mutsuzaki > <[email protected]<mailto:[email protected]>> wrote: > > > > This is the second release candidate for 3.5.1-alpha. This candidate > fixes some issues found in the first candidate, including > ZOOKEEPER-2171. The full release notes is > available at: > > https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=1231080 > 1 > &version=12326786 > > *** Please download, test and vote by May 29th 2015, 23:59 UTC+0. *** > > Source files: > http://people.apache.org/~michim/zookeeper-3.5.1-alpha-candidate-1/ > > Maven staging repo: > > https://repository.apache.org/content/groups/staging/org/apache/zookeepe > r > /zookeeper/3.5.1-alpha/ > > The tag to be voted upon: > https://svn.apache.org/repos/asf/zookeeper/tags/release-3.5.1-rc1/ > > ZooKeeper's KEYS file containing PGP keys we use to sign the release: > http://www.apache.org/dist/zookeeper/KEYS > > Should we release this candidate? > > > > > > > > > > > > > > > >
