Hi all,
we (JMeter) have currently a problem running our nightly builds. The
tests fail, when we try to bind to the address of the hostname of the
build server. The server is bb-1604-test.apache.org and resolves to
207.244.88.135. It seems, that that address is not bound to the network
devices the jvm has access to. We have added a debug message to the
tests, that lists ens3 and lo with their IPv4 and IPv6 addresses, but
not the 207.244.88.135.
Is this a misconfiguration of the build server? Is there any way to
determine the "right" hostname that resolves to an address, that is
bound to an accessible network interface?
Regards,
Felix
-------- Weitergeleitete Nachricht --------
Betreff: Re: buildbot failure in on jmeter-nightly
Datum: Mon, 5 Feb 2018 21:38:26 +0100
Von: Felix Schumacher <felix.schumac...@internetallee.de>
Antwort an: d...@jmeter.apache.org, d...@jmeter.apache.org
An: d...@jmeter.apache.org, Gavin McDonald <ga...@16degrees.com.au>
Hi all,
I think I found out, why the build fails (Thanks Rainer).
The server has not bound the address for its hostname on any device
(that can be seen).
This is what the current test suite lists as network devices and their
addresses:
[java] java.rmi.server.hostname=null [java] Interfaces: {name:ens3
(ens3) => [/fe80:0:0:0:0:10ff:fefb:15%ens3, /10.10.4.78], name:lo (lo)
=> [/0:0:0:0:0:0:0:1%lo, /127.0.0.1]} [java]
java.rmi.server.hostname=null [java] bb-1604-test.apache.org/207.244.88.135
We want to bind to 207.244.88.135, but that address is not bound (or at
least, we can't see it).
Should I open a jira issue?
Regards,
Felix
Am 01.02.2018 um 21:51 schrieb Felix Schumacher:
Am 26.01.2018 um 23:11 schrieb Felix Schumacher:
Am 26.01.2018 um 08:08 schrieb Felix Schumacher:
Am 26.01.2018 um 02:12 schrieb Gavin McDonald:
I fixed the node that nightly was building on and ran a manual
build, it failed , but not due the node this time I dont think.
Thanks for looking into this.
The current error log looks like the subversion checkout didn't work
properly. It complains about a java file, that doesn't contain its
own class:
[javac]
/home/buildslave/slave/jmeter-nightly/build/src/core/org/apache/jmeter/rmi/RmiUtils.java:92:
error: cannot access SSLRMIClientSocketFactory [javac] final
SSLRMIClientSocketFactory factory = new SSLRMIClientSocketFactory();
[javac] ^ [javac] bad source file:
/home/buildslave/slave/jmeter-nightly/build/src/core/org/apache/jmeter/rmi/SSLRMIClientSocketFactory.java
[javac] file does not contain class
org.apache.jmeter.rmi.SSLRMIClientSocketFactory [javac] Please
remove or make sure it appears in the correct subdirectory of the
sourcepath.
I try to force another nightly build and hope that subversion
fetches the file.
I managed to correct the broken subversion checkout, by adding some
options to the SVN check in buildbot, but now the nigthly build shows
another problem:
[java] 1) initializationError(org.apache.jmeter.junit.JMeterTest)
[java] java.lang.Exception: Error creating
org.apache.jmeter.samplers.RemoteSampleListenerImpl [java] at
org.apache.jmeter.junit.JMeterTest.getObjects(JMeterTest.java:488)
[java] at
org.apache.jmeter.junit.JMeterTest.suiteSerializableElements(JMeterTest.java:388)
[java] at
org.apache.jmeter.junit.JMeterTest.suite(JMeterTest.java:133) [java]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [java]
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
[java] at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[java] at java.lang.reflect.Method.invoke(Method.java:498) [java] at
org.junit.internal.runners.SuiteMethod.testFromSuiteMethod(SuiteMethod.java:35)
[java] at
org.junit.internal.runners.SuiteMethod.<init>(SuiteMethod.java:24)
[java] at
org.junit.internal.builders.SuiteMethodBuilder.runnerForClass(SuiteMethodBuilder.java:11)
[java] at
org.junit.runners.model.RunnerBuilder.safeRunnerForClass(RunnerBuilder.java:59)
[java] at
org.junit.internal.builders.AllDefaultPossibilitiesBuilder.runnerForClass(AllDefaultPossibilitiesBuilder.java:26)
[java] at org.junit.runner.Computer.getRunner(Computer.java:40)
[java] at
org.junit.experimental.ParallelComputer.getRunner(ParallelComputer.java:64)
[java] at
org.junit.runner.Computer$1.runnerForClass(Computer.java:31) [java]
at
org.junit.runners.model.RunnerBuilder.safeRunnerForClass(RunnerBuilder.java:59)
[java] at
org.junit.runners.model.RunnerBuilder.runners(RunnerBuilder.java:101)
[java] at
org.junit.runners.model.RunnerBuilder.runners(RunnerBuilder.java:87)
[java] at org.junit.runners.Suite.<init>(Suite.java:81) [java] at
org.junit.runner.Computer.getSuite(Computer.java:28) [java] at
org.junit.experimental.ParallelComputer.getSuite(ParallelComputer.java:57)
[java] at org.junit.runner.Request.classes(Request.java:75) [java] at
org.apache.jorphan.test.AllTests.getParallelTests(AllTests.java:231)
[java] at org.apache.jorphan.test.AllTests.main(AllTests.java:218)
[java] Caused by: java.lang.reflect.InvocationTargetException [java]
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native
Method) [java] at
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
[java] at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
[java] at
java.lang.reflect.Constructor.newInstance(Constructor.java:423)
[java] at
org.apache.jmeter.junit.JMeterTest.getObjects(JMeterTest.java:461)
[java] ... 23 more [java] Caused by: java.rmi.server.ExportException:
Port already in use: 0; nested exception is: [java]
java.net.BindException: Cannot assign requested address (Bind failed)
[java] at
sun.rmi.transport.tcp.TCPTransport.listen(TCPTransport.java:341)
[java] at
sun.rmi.transport.tcp.TCPTransport.exportObject(TCPTransport.java:249)
[java] at
sun.rmi.transport.tcp.TCPEndpoint.exportObject(TCPEndpoint.java:411)
[java] at sun.rmi.transport.LiveRef.exportObject(LiveRef.java:147)
[java] at
sun.rmi.server.UnicastServerRef.exportObject(UnicastServerRef.java:236)
[java] at
java.rmi.server.UnicastRemoteObject.exportObject(UnicastRemoteObject.java:383)
[java] at
java.rmi.server.UnicastRemoteObject.exportObject(UnicastRemoteObject.java:346)
[java] at
java.rmi.server.UnicastRemoteObject.<init>(UnicastRemoteObject.java:225)
[java] at
org.apache.jmeter.samplers.RemoteSampleListenerImpl.<init>(RemoteSampleListenerImpl.java:44)
[java] ... 28 more [java] Caused by: java.net.BindException: Cannot
assign requested address (Bind failed) [java] at
java.net.PlainSocketImpl.socketBind(Native Method) [java] at
java.net.AbstractPlainSocketImpl.bind(AbstractPlainSocketImpl.java:387)
[java] at java.net.ServerSocket.bind(ServerSocket.java:375) [java] at
java.net.ServerSocket.<init>(ServerSocket.java:237) [java] at
javax.net.ssl.SSLServerSocket.<init>(SSLServerSocket.java:181) [java]
at
sun.security.ssl.SSLServerSocketImpl.<init>(SSLServerSocketImpl.java:140)
[java] at
sun.security.ssl.SSLServerSocketFactoryImpl.createServerSocket(SSLServerSocketFactoryImpl.java:95)
[java] at
org.apache.jmeter.rmi.SSLRMIServerSocketFactory.createServerSocket(SSLRMIServerSocketFactory.java:134)
[java] at
sun.rmi.transport.tcp.TCPEndpoint.newServerSocket(TCPEndpoint.java:666)
[java] at
sun.rmi.transport.tcp.TCPTransport.listen(TCPTransport.java:330)
[java] ... 36 more
I tried to force the build two times hoping that the (randomly
chosen) port would be free. It didn't work.
I never experienced this problem on my local builds and jenkins build
are fine, too.
Has anyone an idea to fix this, or the cause of the problem?
ping.
Anyone an idea?
I have caught the BindException and rethrown it as an IOException to
get the address the server socket tries to bind to. It looks like the
address, that gets reported earlier as the address for the node:
...
[java] java.rmi.server.hostname=null [java]
bb-1604-test.apache.org/207.244.88.135 [java] isSiteLocalAddress:false
[java] isAnyLocalAddress:false [java] isLinkLocalAddress:false [java]
isLoopbackAddress:false [java] isMulticastAddress:false [java]
localhost/127.0.0.1 [java] isSiteLocalAddress:false [java]
isAnyLocalAddress:false [java] isLinkLocalAddress:false [java]
isLoopbackAddress:true [java] isMulticastAddress:false [java]
==================== [java] bb-1604-test.apache.org/207.244.88.135
[java] isSiteLocalAddress:false [java] isAnyLocalAddress:false [java]
isLinkLocalAddress:false [java] isLoopbackAddress:false [java]
isMulticastAddress:false ... [java] Caused by: java.io.IOException:
Could not bind to bb-1604-test.apache.org/207.244.88.135 using port 0
[java] at
org.apache.jmeter.rmi.SSLRMIServerSocketFactory.createServerSocket(SSLRMIServerSocketFactory.java:139)
[java] at
sun.rmi.transport.tcp.TCPEndpoint.newServerSocket(TCPEndpoint.java:666)
[java] at
sun.rmi.transport.tcp.TCPTransport.listen(TCPTransport.java:330)
[java] ... 36 more [java] Caused by: java.net.BindException: Cannot
assign requested address (Bind failed) [java] at
java.net.PlainSocketImpl.socketBind(Native Method) [java] at
java.net.AbstractPlainSocketImpl.bind(AbstractPlainSocketImpl.java:387)
A port number 0 should be interpreted as choose one available. And if
I read the jdk source code correctly, the error gets thrown by the
following code:
if (NET_Bind(fd, (struct sockaddr *)&him, len) < 0) {
if (errno == EADDRINUSE || errno == EADDRNOTAVAIL ||
errno == EPERM || errno == EACCES) {
NET_ThrowByNameWithLastError(env, JNU_JAVANETPKG
"BindException",
"Bind failed");
} else {
NET_ThrowByNameWithLastError(env, JNU_JAVANETPKG
"SocketException",
"Bind failed");
}
return;
}
So the errno would be one of EADDRINUSE, EADDRNOTAVAIL, EPERM or
EACCES (whatever that means).
One difference to my local build is the value of "isSiteLocalAddress"
for bb-1604-test.apache.org, which is "false" for bb-1604-test and
"true" for my laptops address. But on the other hand, it is "false"
for the jenkins build node, which completes without an error (well,
most of the time).
Another difference to my local build is the strength of JCE, but
bb-1604-test has the same max key length as the jenkins build node.
Anyone any idea?
Felix
Felix
Felix
Gav…
On 26 Jan 2018, at 12:08 pm, build...@apache.org wrote:
The Buildbot has detected a new failure on builder jmeter-nightly
while building . Full details are available at:
https://ci.apache.org/builders/jmeter-nightly/builds/922
Buildbot URL: https://ci.apache.org/
Buildslave for this Build: bb_1604_test_ubuntu
Build Reason: forced: by IRC user <gmcdonald> on channel #asftest:
test after rebooting node
Build Source Stamp: HEAD
Blamelist:
BUILD FAILED: failed shell_5
Sincerely,
-The Buildbot
