Hi all,

we (JMeter) have currently a problem running our nightly builds. The tests fail, when we try to bind to the address of the hostname of the build server. The server is bb-1604-test.apache.org and resolves to 207.244.88.135. It seems, that that address is not bound to the network devices the jvm has access to. We have added a debug message to the tests, that lists ens3 and lo with their IPv4 and IPv6 addresses, but not the 207.244.88.135.


Is this a misconfiguration of the build server? Is there any way to determine the "right" hostname that resolves to an address, that is bound to an accessible network interface?

Regards,

 Felix


-------- Weitergeleitete Nachricht --------
Betreff:        Re: buildbot failure in on jmeter-nightly
Datum:  Mon, 5 Feb 2018 21:38:26 +0100
Von:    Felix Schumacher <felix.schumac...@internetallee.de>
Antwort an:     d...@jmeter.apache.org, d...@jmeter.apache.org
An:     d...@jmeter.apache.org, Gavin McDonald <ga...@16degrees.com.au>



Hi all,

I think I found out, why the build fails (Thanks Rainer).

The server has not bound the address for its hostname on any device
(that can be seen).
This is what the current test suite lists as network devices and their
addresses:

[java] java.rmi.server.hostname=null [java] Interfaces: {name:ens3
(ens3) => [/fe80:0:0:0:0:10ff:fefb:15%ens3, /10.10.4.78], name:lo (lo)
=> [/0:0:0:0:0:0:0:1%lo, /127.0.0.1]} [java]
java.rmi.server.hostname=null [java] bb-1604-test.apache.org/207.244.88.135

We want to bind to 207.244.88.135, but that address is not bound (or at
least, we can't see it).

Should I open a jira issue?

Regards,
  Felix

Am 01.02.2018 um 21:51 schrieb Felix Schumacher:
Am 26.01.2018 um 23:11 schrieb Felix Schumacher:
Am 26.01.2018 um 08:08 schrieb Felix Schumacher:
Am 26.01.2018 um 02:12 schrieb Gavin McDonald:
I fixed the node that nightly was building on and ran a manual build, it failed , but not due the node this time I dont think.
Thanks for looking into this.

The current error log looks like the subversion checkout didn't work properly. It complains about a java file, that doesn't contain its own class:

[javac] /home/buildslave/slave/jmeter-nightly/build/src/core/org/apache/jmeter/rmi/RmiUtils.java:92: error: cannot access SSLRMIClientSocketFactory [javac] final SSLRMIClientSocketFactory factory = new SSLRMIClientSocketFactory(); [javac] ^ [javac] bad source file: /home/buildslave/slave/jmeter-nightly/build/src/core/org/apache/jmeter/rmi/SSLRMIClientSocketFactory.java [javac] file does not contain class org.apache.jmeter.rmi.SSLRMIClientSocketFactory [javac] Please remove or make sure it appears in the correct subdirectory of the sourcepath.

I try to force another nightly build and hope that subversion fetches the file.

I managed to correct the broken subversion checkout, by adding some options to the SVN check in buildbot, but now the nigthly build shows another problem:

[java] 1) initializationError(org.apache.jmeter.junit.JMeterTest) [java] java.lang.Exception: Error creating org.apache.jmeter.samplers.RemoteSampleListenerImpl [java] at org.apache.jmeter.junit.JMeterTest.getObjects(JMeterTest.java:488) [java] at org.apache.jmeter.junit.JMeterTest.suiteSerializableElements(JMeterTest.java:388) [java] at org.apache.jmeter.junit.JMeterTest.suite(JMeterTest.java:133) [java] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [java] at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) [java] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [java] at java.lang.reflect.Method.invoke(Method.java:498) [java] at org.junit.internal.runners.SuiteMethod.testFromSuiteMethod(SuiteMethod.java:35) [java] at org.junit.internal.runners.SuiteMethod.<init>(SuiteMethod.java:24) [java] at org.junit.internal.builders.SuiteMethodBuilder.runnerForClass(SuiteMethodBuilder.java:11) [java] at org.junit.runners.model.RunnerBuilder.safeRunnerForClass(RunnerBuilder.java:59) [java] at org.junit.internal.builders.AllDefaultPossibilitiesBuilder.runnerForClass(AllDefaultPossibilitiesBuilder.java:26) [java] at org.junit.runner.Computer.getRunner(Computer.java:40) [java] at org.junit.experimental.ParallelComputer.getRunner(ParallelComputer.java:64) [java] at org.junit.runner.Computer$1.runnerForClass(Computer.java:31) [java] at org.junit.runners.model.RunnerBuilder.safeRunnerForClass(RunnerBuilder.java:59) [java] at org.junit.runners.model.RunnerBuilder.runners(RunnerBuilder.java:101) [java] at org.junit.runners.model.RunnerBuilder.runners(RunnerBuilder.java:87) [java] at org.junit.runners.Suite.<init>(Suite.java:81) [java] at org.junit.runner.Computer.getSuite(Computer.java:28) [java] at org.junit.experimental.ParallelComputer.getSuite(ParallelComputer.java:57) [java] at org.junit.runner.Request.classes(Request.java:75) [java] at org.apache.jorphan.test.AllTests.getParallelTests(AllTests.java:231) [java] at org.apache.jorphan.test.AllTests.main(AllTests.java:218) [java] Caused by: java.lang.reflect.InvocationTargetException [java] at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) [java] at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) [java] at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) [java] at java.lang.reflect.Constructor.newInstance(Constructor.java:423) [java] at org.apache.jmeter.junit.JMeterTest.getObjects(JMeterTest.java:461) [java] ... 23 more [java] Caused by: java.rmi.server.ExportException: Port already in use: 0; nested exception is: [java] java.net.BindException: Cannot assign requested address (Bind failed) [java] at sun.rmi.transport.tcp.TCPTransport.listen(TCPTransport.java:341) [java] at sun.rmi.transport.tcp.TCPTransport.exportObject(TCPTransport.java:249) [java] at sun.rmi.transport.tcp.TCPEndpoint.exportObject(TCPEndpoint.java:411) [java] at sun.rmi.transport.LiveRef.exportObject(LiveRef.java:147) [java] at sun.rmi.server.UnicastServerRef.exportObject(UnicastServerRef.java:236) [java] at java.rmi.server.UnicastRemoteObject.exportObject(UnicastRemoteObject.java:383) [java] at java.rmi.server.UnicastRemoteObject.exportObject(UnicastRemoteObject.java:346) [java] at java.rmi.server.UnicastRemoteObject.<init>(UnicastRemoteObject.java:225) [java] at org.apache.jmeter.samplers.RemoteSampleListenerImpl.<init>(RemoteSampleListenerImpl.java:44) [java] ... 28 more [java] Caused by: java.net.BindException: Cannot assign requested address (Bind failed) [java] at java.net.PlainSocketImpl.socketBind(Native Method) [java] at java.net.AbstractPlainSocketImpl.bind(AbstractPlainSocketImpl.java:387) [java] at java.net.ServerSocket.bind(ServerSocket.java:375) [java] at java.net.ServerSocket.<init>(ServerSocket.java:237) [java] at javax.net.ssl.SSLServerSocket.<init>(SSLServerSocket.java:181) [java] at sun.security.ssl.SSLServerSocketImpl.<init>(SSLServerSocketImpl.java:140) [java] at sun.security.ssl.SSLServerSocketFactoryImpl.createServerSocket(SSLServerSocketFactoryImpl.java:95) [java] at org.apache.jmeter.rmi.SSLRMIServerSocketFactory.createServerSocket(SSLRMIServerSocketFactory.java:134) [java] at sun.rmi.transport.tcp.TCPEndpoint.newServerSocket(TCPEndpoint.java:666) [java] at sun.rmi.transport.tcp.TCPTransport.listen(TCPTransport.java:330) [java] ... 36 more

I tried to force the build two times hoping that the (randomly chosen) port would be free. It didn't work.

I never experienced this problem on my local builds and jenkins build are fine, too.

Has anyone an idea to fix this, or the cause of the problem?

ping.

Anyone an idea?

I have caught the BindException and rethrown it as an IOException to get the address the server socket tries to bind to. It looks like the address, that gets reported earlier as the address for the node:

...

[java] java.rmi.server.hostname=null [java] bb-1604-test.apache.org/207.244.88.135 [java] isSiteLocalAddress:false [java] isAnyLocalAddress:false [java] isLinkLocalAddress:false [java] isLoopbackAddress:false [java] isMulticastAddress:false [java] localhost/127.0.0.1 [java] isSiteLocalAddress:false [java] isAnyLocalAddress:false [java] isLinkLocalAddress:false [java] isLoopbackAddress:true [java] isMulticastAddress:false [java] ==================== [java] bb-1604-test.apache.org/207.244.88.135 [java] isSiteLocalAddress:false [java] isAnyLocalAddress:false [java] isLinkLocalAddress:false [java] isLoopbackAddress:false [java] isMulticastAddress:false ... [java] Caused by: java.io.IOException: Could not bind to bb-1604-test.apache.org/207.244.88.135 using port 0 [java] at org.apache.jmeter.rmi.SSLRMIServerSocketFactory.createServerSocket(SSLRMIServerSocketFactory.java:139) [java] at sun.rmi.transport.tcp.TCPEndpoint.newServerSocket(TCPEndpoint.java:666) [java] at sun.rmi.transport.tcp.TCPTransport.listen(TCPTransport.java:330) [java] ... 36 more [java] Caused by: java.net.BindException: Cannot assign requested address (Bind failed) [java] at java.net.PlainSocketImpl.socketBind(Native Method) [java] at java.net.AbstractPlainSocketImpl.bind(AbstractPlainSocketImpl.java:387)

A port number 0 should be interpreted as choose one available. And if I read the jdk source code correctly, the error gets thrown by the following code:

    if (NET_Bind(fd, (struct sockaddr *)&him, len) < 0) {
        if (errno == EADDRINUSE || errno == EADDRNOTAVAIL ||
            errno == EPERM || errno == EACCES) {
            NET_ThrowByNameWithLastError(env, JNU_JAVANETPKG "BindException",
                           "Bind failed");
        } else {
            NET_ThrowByNameWithLastError(env, JNU_JAVANETPKG "SocketException",
                           "Bind failed");
        }
        return;
    }

So the errno would be one of EADDRINUSE, EADDRNOTAVAIL, EPERM or EACCES (whatever that means).

One difference to my local build is the value of "isSiteLocalAddress" for bb-1604-test.apache.org, which is "false" for bb-1604-test and "true" for my laptops address. But on the other hand, it is "false" for the jenkins build node, which completes without an error (well, most of the time).

Another difference to my local build is the strength of JCE, but bb-1604-test has the same max key length as the jenkins build node.

Anyone any idea?

Felix


Felix


Felix

Gav…


On 26 Jan 2018, at 12:08 pm, build...@apache.org wrote:

The Buildbot has detected a new failure on builder jmeter-nightly while building . Full details are available at:
https://ci.apache.org/builders/jmeter-nightly/builds/922

Buildbot URL: https://ci.apache.org/

Buildslave for this Build: bb_1604_test_ubuntu

Build Reason: forced: by IRC user <gmcdonald> on channel #asftest: test after rebooting node
Build Source Stamp: HEAD
Blamelist:

BUILD FAILED: failed shell_5

Sincerely,
-The Buildbot











Reply via email to