Help needed [Fwd: Jenkins build is still unstable: Qpid-proton-j » tests #295]
This failure is due to my updates to the SSL certificates and keys used by the SSL unit tests. Specifically: IllegalStateException: java.lang.IllegalStateException: Unable to read PEM object from file /home/jenkins/jenkins-slave/workspace/Qpid-proton-j/trunk/tests/target/classes/proton_tests/ssl_db/server-private-key.pem proton_tests.ssl.SslTest.test_client_server_authentication ..Mar 20, 2013 1:48:59 AM org.apache.qpid.proton.engine.impl.ssl.SslEngineFacadeFactory readPemObject SEVERE: Unable to read PEM object. Perhaps you need the unlimited strength libraries in java-home/jre/lib/security/ ? org.bouncycastle.openssl.PEMException: problem parsing ENCRYPTED PRIVATE KEY: java.security.InvalidKeyException: Illegal key size I've hit this problem before, and have yet to be able to solve it (on my machine, at least). The problem is due to the export restrictions on encryption. I suspect the default java configuration for some machines - certainly OSX - does not allow for exportable key lengths. On such systems, the proton SSL test will fail as the environment cannot handle the key lengths used in the checked in certificates. So why not check in certificates with short keys? That'll fix the problem. But I can't - the Fedora packages do not support creating certs with short key lengths, for security reasons. Therefore I cannot generate universally usable certs in my environment. This is a call for help - is there anyone out there who is seeing the same SSL test failures using the latest trunk? If so, can you regenerate the test certificates on your system? There's a script attached to the end of the README.txt file in qpid-proton/tests/python/proton_tests/ssl_db - simply run that in the ssl_db directory to regenerate the certs. Rerun the SSL tests - they should pass. If they do, send me the diff and I'll check it in. Alternatively, if anyone can figure out how to install weak keysigning algorithms on a Fedora box - I'm all ears. FYI: In order to support the larger key lengths, the following policy files need to be installed: http://www.oracle.com/technetwork/java/javase/downloads/jce-6-download-429243.html - Forwarded Message - From: Apache Jenkins Server jenk...@builds.apache.org To: notificati...@qpid.apache.org Sent: Tuesday, March 19, 2013 9:49:01 PM Subject: Jenkins build is still unstable: Qpid-proton-j » tests #295 See https://builds.apache.org/job/Qpid-proton-j/org.apache.qpid$tests/changes -- -K
Re: Difficulties building Swig-generated C++ code for proton-jni binding (due to nested typedef union)
Hello Cliff I too share your dislike of my suggested change to code.h:) I do prefer your alternative despite the duplication. However I worry that the copy - deep within the java bindings tree would be overlooked when changes are made to codec.h and the two would diverge, leading to obscure defects. I wonder if it is possible to move your workaround into its own .i file which would be located alongside codec.h? java.i would then include it using a %import. This approach would also facilitate sharing with the other bindings (surely they would need the same solution??). I had just got this working yesterday on Linux and was trying a Windows build before posting a review. It fails in cmake scripts working with windows paths (and maybe other problems). I see an issue on Windows (Windows 7/MS Visual Studio 2010/Python 2.7.3) with the generation of protocol.h and encodings.h. It behaves as if the cmd-shell is ignoring the redirection (). I see the generated file flash by in cmd window and no .h is written. I'm working around by executing: python X:/src/proton/proton-c/env.py PYTHONPATH=X:/src/proton/proton-c python X:/src/proton/proton-c/src/protocol.h.py X:/src/proton/buildwin/proton-c/protocol.h python X:/src/proton/proton-c/env.py PYTHONPATH=X:/src/proton/proton-c python X:/src/proton/proton-c/src/codec/encodings.h.py X:/src/proton/buildwin/proton-c/encodings.h Do you see this? My windows box behaves oddly at the best of times. On Windows I still see a number of C++ compilation errors from javaJAVA_wrap.cxx. Do you see these? I haven't started to investigate yet. 11X:\src\proton\proton-c\include\proton/driver_extras.h(40): error C2146: syntax error : missing ';' before identifier 'pn_socket_t' 11X:\src\proton\proton-c\include\proton/driver_extras.h(40): error C4430: missing type specifier - int assumed. Note: C++ does not support default-int 11X:\src\proton\proton-c\include\proton/driver_extras.h(40): error C4430: missing type specifier - int assumed. Note: C++ does not support default-int 11X:\src\proton\proton-c\include\proton/driver_extras.h(53): error C2061: syntax error : identifier 'pn_socket_t' 11X:\src\proton\proton-c\include\proton/driver_extras.h(63): error C2061: syntax error : identifier 'pn_socket_t' 11javaJAVA_wrap.cxx(1062): warning C4018: '' : signed/unsigned mismatch 11javaJAVA_wrap.cxx(1089): warning C4018: '' : signed/unsigned mismatch 11javaJAVA_wrap.cxx(7754): error C2146: syntax error : missing ';' before identifier 'arg2' 11javaJAVA_wrap.cxx(7754): error C2065: 'arg2' : undeclared identifier 11javaJAVA_wrap.cxx(7761): error C2065: 'arg2' : undeclared identifier 11javaJAVA_wrap.cxx(7761): error C2146: syntax error : missing ';' before identifier 'jarg2' 11javaJAVA_wrap.cxx(7763): error C2065: 'arg2' : undeclared identifier 11javaJAVA_wrap.cxx(7772): error C2146: syntax error : missing ';' before identifier 'arg2' 11javaJAVA_wrap.cxx(7772): error C2065: 'arg2' : undeclared identifier 11javaJAVA_wrap.cxx(7779): error C2065: 'arg2' : undeclared identifier 11javaJAVA_wrap.cxx(7779): error C2146: syntax error : missing ';' before identifier 'jarg2' 11javaJAVA_wrap.cxx(7781): error C2065: 'arg2' : undeclared identifier Let me know if you wish to tackle the remaining windows build issues. I can't offer to take these on firstly as my focus is Engine at present, and C/C++ it is no longer my area of expertise (12 years of rust). I will pitch in and help with any questions you may have with the JNI aspects if I can. I am also in the process of getting this build set up on Apache Jenkins. Kind regards, Keith On 20 March 2013 03:49, Cliff Jansen cliffjan...@gmail.com wrote: Hi Keith, Very nice work there, thanks. I've been looking at this too and had an alternate solution to the nested union problem (https://issues.apache.org/jira/secure/attachment/12574482/pn254-cj-0.patch). It is based on the observation that there is nothing wrong with the definition in codec.h that swig complains about to begin with. That, plus it irks me to change a public interface file just to satisfy the one C99 unfriendly compiler in the world. So I looked for a solution that was specific to swig. Sadly, swig handled this just fine in the 1.X series, precisely to handle cases such as here. However that apparently exacerbated problems elsewhere and, as far as I can tell, their new fancy %nestedworkaround fix in swig 2.X can't work in our case and leaves us high and dry. The patch I provided just restores the swig 1.X behaviour. The downside is the (twice) repeated re-definition of the union (bleh, my swig knowledge could find no better solution). Your patch is cleaner in that respect. If my patch seems distasteful in its foibles, I would look to Rafi for his views on changing codec.h. Let me know what you think. I had just got this working yesterday on Linux and was trying a Windows build before posting a review. It fails in cmake scripts working with
Re: Help needed [Fwd: Jenkins build is still unstable: Qpid-proton-j » tests #295]
Resolved by rev. 1458901 On 20 March 2013 13:24, Ken Giusti kgiu...@redhat.com wrote: This failure is due to my updates to the SSL certificates and keys used by the SSL unit tests. Specifically: IllegalStateException: java.lang.IllegalStateException: Unable to read PEM object from file /home/jenkins/jenkins-slave/workspace/Qpid-proton-j/trunk/tests/target/classes/proton_tests/ssl_db/server-private-key.pem proton_tests.ssl.SslTest.test_client_server_authentication ..Mar 20, 2013 1:48:59 AM org.apache.qpid.proton.engine.impl.ssl.SslEngineFacadeFactory readPemObject SEVERE: Unable to read PEM object. Perhaps you need the unlimited strength libraries in java-home/jre/lib/security/ ? org.bouncycastle.openssl.PEMException: problem parsing ENCRYPTED PRIVATE KEY: java.security.InvalidKeyException: Illegal key size I've hit this problem before, and have yet to be able to solve it (on my machine, at least). The problem is due to the export restrictions on encryption. I suspect the default java configuration for some machines - certainly OSX - does not allow for exportable key lengths. On such systems, the proton SSL test will fail as the environment cannot handle the key lengths used in the checked in certificates. So why not check in certificates with short keys? That'll fix the problem. But I can't - the Fedora packages do not support creating certs with short key lengths, for security reasons. Therefore I cannot generate universally usable certs in my environment. This is a call for help - is there anyone out there who is seeing the same SSL test failures using the latest trunk? If so, can you regenerate the test certificates on your system? There's a script attached to the end of the README.txt file in qpid-proton/tests/python/proton_tests/ssl_db - simply run that in the ssl_db directory to regenerate the certs. Rerun the SSL tests - they should pass. If they do, send me the diff and I'll check it in. Alternatively, if anyone can figure out how to install weak keysigning algorithms on a Fedora box - I'm all ears. FYI: In order to support the larger key lengths, the following policy files need to be installed: http://www.oracle.com/technetwork/java/javase/downloads/jce-6-download-429243.html - Forwarded Message - From: Apache Jenkins Server jenk...@builds.apache.org To: notificati...@qpid.apache.org Sent: Tuesday, March 19, 2013 9:49:01 PM Subject: Jenkins build is still unstable: Qpid-proton-j » tests #295 See https://builds.apache.org/job/Qpid-proton-j/org.apache.qpid$tests/changes -- -K
Re: Difficulties building Swig-generated C++ code for proton-jni binding (due to nested typedef union)
Hi Keith, Sorry, the compile errors you see on Windows should go away with the following in java.i: 26a27,29 #if defined(_WIN32) ! defined(__CYGWIN__) #include winsock2.h #endif I am able to generate protocol.h and encodings.h on my Windows systems (a range of XP - Windows 8, VS2008 - VS2012, 32/64 bit). I am using fairly up to date versions of python, cmake and swig. So I am not sure why it is failing for you. Note that Windows support is still a work in progress. For 0.4 I only have completed building proton-c on Windows and the limited ctest infrastructure (i.e. syntax errors and CMakeLists.txt). Some proton-test tests fail and investigation is pending. I wonder if it is possible to move your workaround into its own .i file which would be located alongside codec.h? java.i would then include it using a %import. This approach would also facilitate sharing with the other bindings (surely they would need the same solution??). Let me think about that or perhaps another way to address your (sensible) concern and get back to you later today or tomorrow. Cliff On Wed, Mar 20, 2013 at 11:33 AM, Keith W keith.w...@gmail.com wrote: Hello Cliff I too share your dislike of my suggested change to code.h:) I do prefer your alternative despite the duplication. However I worry that the copy - deep within the java bindings tree would be overlooked when changes are made to codec.h and the two would diverge, leading to obscure defects. I wonder if it is possible to move your workaround into its own .i file which would be located alongside codec.h? java.i would then include it using a %import. This approach would also facilitate sharing with the other bindings (surely they would need the same solution??). I had just got this working yesterday on Linux and was trying a Windows build before posting a review. It fails in cmake scripts working with windows paths (and maybe other problems). I see an issue on Windows (Windows 7/MS Visual Studio 2010/Python 2.7.3) with the generation of protocol.h and encodings.h. It behaves as if the cmd-shell is ignoring the redirection (). I see the generated file flash by in cmd window and no .h is written. I'm working around by executing: python X:/src/proton/proton-c/env.py PYTHONPATH=X:/src/proton/proton-c python X:/src/proton/proton-c/src/protocol.h.py X:/src/proton/buildwin/proton-c/protocol.h python X:/src/proton/proton-c/env.py PYTHONPATH=X:/src/proton/proton-c python X:/src/proton/proton-c/src/codec/encodings.h.py X:/src/proton/buildwin/proton-c/encodings.h Do you see this? My windows box behaves oddly at the best of times. On Windows I still see a number of C++ compilation errors from javaJAVA_wrap.cxx. Do you see these? I haven't started to investigate yet. 11X:\src\proton\proton-c\include\proton/driver_extras.h(40): error C2146: syntax error : missing ';' before identifier 'pn_socket_t' 11X:\src\proton\proton-c\include\proton/driver_extras.h(40): error C4430: missing type specifier - int assumed. Note: C++ does not support default-int 11X:\src\proton\proton-c\include\proton/driver_extras.h(40): error C4430: missing type specifier - int assumed. Note: C++ does not support default-int 11X:\src\proton\proton-c\include\proton/driver_extras.h(53): error C2061: syntax error : identifier 'pn_socket_t' 11X:\src\proton\proton-c\include\proton/driver_extras.h(63): error C2061: syntax error : identifier 'pn_socket_t' 11javaJAVA_wrap.cxx(1062): warning C4018: '' : signed/unsigned mismatch 11javaJAVA_wrap.cxx(1089): warning C4018: '' : signed/unsigned mismatch 11javaJAVA_wrap.cxx(7754): error C2146: syntax error : missing ';' before identifier 'arg2' 11javaJAVA_wrap.cxx(7754): error C2065: 'arg2' : undeclared identifier 11javaJAVA_wrap.cxx(7761): error C2065: 'arg2' : undeclared identifier 11javaJAVA_wrap.cxx(7761): error C2146: syntax error : missing ';' before identifier 'jarg2' 11javaJAVA_wrap.cxx(7763): error C2065: 'arg2' : undeclared identifier 11javaJAVA_wrap.cxx(7772): error C2146: syntax error : missing ';' before identifier 'arg2' 11javaJAVA_wrap.cxx(7772): error C2065: 'arg2' : undeclared identifier 11javaJAVA_wrap.cxx(7779): error C2065: 'arg2' : undeclared identifier 11javaJAVA_wrap.cxx(7779): error C2146: syntax error : missing ';' before identifier 'jarg2' 11javaJAVA_wrap.cxx(7781): error C2065: 'arg2' : undeclared identifier Let me know if you wish to tackle the remaining windows build issues. I can't offer to take these on firstly as my focus is Engine at present, and C/C++ it is no longer my area of expertise (12 years of rust). I will pitch in and help with any questions you may have with the JNI aspects if I can. I am also in the process of getting this build set up on Apache Jenkins. Kind regards, Keith
[jira] [Assigned] (PROTON-214) Test proton_tests.messenger.MessengerTest.testSendBogus failed
[ https://issues.apache.org/jira/browse/PROTON-214?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Philip Harvey reassigned PROTON-214: Assignee: Gordon Sim (was: Philip Harvey) Hi Gordon, Would you mind looking at my commit to fix this race condition please? It's r1459089. Thanks. Test proton_tests.messenger.MessengerTest.testSendBogus failed Key: PROTON-214 URL: https://issues.apache.org/jira/browse/PROTON-214 Project: Qpid Proton Issue Type: Bug Affects Versions: 0.3, 0.4 Environment: Run mvn test from a clean checkout - this uses proton-j by default. Reporter: Philip Harvey Assignee: Gordon Sim Priority: Minor Attachments: JNIMessenger.patch The system test proton_tests.messenger.MessengerTest.testSendBogus is failing against both the proton-jni profile and, on certain computers, when run against proton-j too. The proton-jni problem seems to be caused by the fact the the JNI Messenger implementation ignores proton-c's error return codes. I think I've seen this test pass occasionally against proton-j so I suspect there's something unreliable about the test. Here is the output. proton_tests.messenger.MessengerTest.testSendBogus ..Feb 4, 2013 2:07:19 PM org.apache.qpid.proton.messenger.impl.MessengerImpl processActive SEVERE: Error processing connection java.io.IOException: Connection reset by peer at sun.nio.ch.FileDispatcher.read0(Native Method) at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:21) at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:233) at sun.nio.ch.IOUtil.read(IOUtil.java:206) at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:236) at org.apache.qpid.proton.driver.impl.ConnectorImpl.read(ConnectorImpl.java:95) at org.apache.qpid.proton.driver.impl.ConnectorImpl.process(ConnectorImpl.java:80) at org.apache.qpid.proton.messenger.impl.MessengerImpl.processActive(MessengerImpl.java:426) at org.apache.qpid.proton.messenger.impl.MessengerImpl.waitUntil(MessengerImpl.java:525) at org.apache.qpid.proton.messenger.impl.MessengerImpl.waitUntil(MessengerImpl.java:506) at org.apache.qpid.proton.messenger.impl.MessengerImpl.send(MessengerImpl.java:205) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.python.core.PyReflectedFunction.__call__(PyReflectedFunction.java:186) at org.python.core.PyReflectedFunction.__call__(PyReflectedFunction.java:204) at org.python.core.PyObject.__call__(PyObject.java:387) at org.python.core.PyObject.__call__(PyObject.java:391) at org.python.core.PyMethod.__call__(PyMethod.java:109) at proton$py.send$201(__pyclasspath__/proton.py:997) at proton$py.call_function(__pyclasspath__/proton.py) at org.python.core.PyTableCode.call(PyTableCode.java:165) at org.python.core.PyBaseCode.call(PyBaseCode.java:134) at org.python.core.PyFunction.__call__(PyFunction.java:317) at org.python.core.PyMethod.__call__(PyMethod.java:109) at proton_tests.messenger$py.teardown$4(/fast/.jenkins/jobs/Trunk-Proton-J/workspace/proton/tests/target/classes/proton_tests/messenger.py:52) at proton_tests.messenger$py.call_function(/fast/.jenkins/jobs/Trunk-Proton-J/workspace/proton/tests/target/classes/proton_tests/messenger.py) at org.python.core.PyTableCode.call(PyTableCode.java:165) at org.python.core.PyBaseCode.call(PyBaseCode.java:134) at org.python.core.PyFunction.__call__(PyFunction.java:317) at org.python.core.PyMethod.__call__(PyMethod.java:109) at org.python.pycode._pyx1.run$36(/fast/.jenkins/jobs/Trunk-Proton-J/workspace/proton/tests/target/classes/proton-test:344) at org.python.pycode._pyx1.call_function(/fast/.jenkins/jobs/Trunk-Proton-J/workspace/proton/tests/target/classes/proton-test) at org.python.core.PyTableCode.call(PyTableCode.java:165) at org.python.core.PyBaseCode.call(PyBaseCode.java:166) at org.python.core.PyFunction.__call__(PyFunction.java:338) at org.python.core.PyMethod.__call__(PyMethod.java:139) at org.python.pycode._pyx1._run$55(/fast/.jenkins/jobs/Trunk-Proton-J/workspace/proton/tests/target/classes/proton-test:484) at org.python.pycode._pyx1.call_function(/fast/.jenkins/jobs/Trunk-Proton-J/workspace/proton/tests/target/classes/proton-test) at
