Dear pypy3 maintainers, There is already a pull request upstream [1] that allows pypy3 to build with OpenSSL 4, but it applies to the main branch rather than the packaged version. So I adapted it from scratch.
I verified that the patched package builds with OpenSSL 4.0, all autopkgtests pass, and that the pypy test suite shows no regressions when run locally. The FTBFS was caused by several OpenSSL 4 API changes: opaque ASN.1 structures, constified X509 functions, and removed TLS method functions. The patch updates the affected bindings and call sites accordingly; the details are all in the attached patch. Please note that this patch drops backward compatibility with OpenSSL 3 in favor of a simpler, cleaner implementation. Building against OpenSSL 3 may now fail. Please let me know if you need anything else. Best wishes, Ruan [1]: https://github.com/pypy/pypy/pull/5489 [2]: https://launchpad.net/~ruancomelli/+archive/ubuntu/fix-lp2154869/+sourcepub/18742598/+listing-archive-extra [3]: https://bugs.debian.org/1138322
Description: Fix for OpenSSL 4 compatibility OpenSSL 4 made the ASN1_STRING structure opaque, so direct member access (->data, ->length) is no longer possible. The cffi bindings now declare the ASN.1 string types as opaque, keeping their shared underlying struct (struct asn1_string_st) exactly as OpenSSL 4's own asn1.h and types.h declare them, so no casts between the aliased types are needed. The remaining direct member accesses in lib_pypy/_cffi_ssl/_stdssl/certificate.py were replaced with the accessor functions (ASN1_STRING_get0_data, ASN1_STRING_length). See also the equivalent pyca/cryptography fix: https://github.com/pyca/cryptography/commit/76ad3dd2c30c16cec8cbc79db3240418e5d26614 . OpenSSL 4 also constified a large family of X509 functions. Return types for X509_get_subject_name, X509_get_issuer_name, X509_get_ext, X509_EXTENSION_get_data, X509_EXTENSION_get_object, X509_NAME_get_entry, X509_NAME_ENTRY_get_object, X509_NAME_ENTRY_get_data, X509_REVOKED_get_ext, X509_CRL_get_ext, X509_CRL_get_issuer, X509_REQ_get_subject_name, X509_alias_get0, and the OCSP_*_get_ext family were updated to return const pointers in the cffi bindings. . The version-specific TLS method functions removed in OpenSSL 4 (TLSv1_method, TLSv1_1_method, TLSv1_2_method, SSLv3_method, SSLv3_server_method, SSLv3_client_method) were removed from the cffi ssl bindings. PROTOCOL_TLSv1/1_1/1_2 remain supported: they now use the generic TLS_method() and pin the protocol version with SSL_OP_NO_TLSv* options. The unused X509_STORE_CTX_get_issuer_fn typedef and related declarations were removed from x509_vfy.py, and the engine.h include is guarded with #ifndef OPENSSL_NO_ENGINE. . cadata DER EOF: OpenSSL 4 reports ASN1_R_NOT_ENOUGH_DATA (instead of ASN1_R_HEADER_TOO_LONG) at the end of a DER certificate read in load_verify_locations(), so that reason code is now accepted as a benign EOF indicator (when at least one certificate was loaded). . Test fixes: lib-python/3/test/test_ssl.py version bound checks were updated from < 4.0 to < 5.0, and test_wrong_cert_tls13 now also accepts ECONNRESET, since OpenSSL 4 may close a TLS 1.3 connection with a TCP reset instead of an SSL alert when the server rejects the client certificate. . Upstream has a pull request for this (https://github.com/pypy/pypy/pull/5489), but, because our packaged version differs significantly from the main branch, this patch was adapted from scratch to apply to the current Debian/Ubuntu package. Author: Ruan Cardoso Comelli <[email protected]> Origin: backport, https://github.com/pypy/pypy/pull/5489 Bug-Debian: http://bugs.debian.org/1138322 Bug-Ubuntu: https://bugs.launchpad.net/ubuntu/+source/pypy3/+bug/2154869 Forwarded: https://github.com/pypy/pypy/pull/5489 Last-Update: 2026-09-21 --- Index: pypy3/lib-python/3/test/test_ssl.py =================================================================== --- pypy3.orig/lib-python/3/test/test_ssl.py +++ pypy3/lib-python/3/test/test_ssl.py @@ -539,11 +539,11 @@ # Some sanity checks follow # >= 1.1.1 self.assertGreaterEqual(n, 0x10101000) - # < 4.0 - self.assertLess(n, 0x40000000) + # < 5.0 + self.assertLess(n, 0x50000000) major, minor, fix, patch, status = t self.assertGreaterEqual(major, 1) - self.assertLess(major, 4) + self.assertLess(major, 5) self.assertGreaterEqual(minor, 0) self.assertLess(minor, 256) self.assertGreaterEqual(fix, 0) @@ -3360,15 +3360,23 @@ server_hostname=hostname, suppress_ragged_eofs=False) as s: s.connect((HOST, server.port)) - with self.assertRaisesRegex( - ssl.SSLError, - 'alert unknown ca|EOF occurred' - ): + # OpenSSL 4 may close the TLS 1.3 connection with a TCP reset + # (ECONNRESET) instead of an SSL alert when the server rejects + # the client certificate, so accept both failure modes. + try: # TLS 1.3 perform client cert exchange after handshake s.write(b'data') s.read(1000) s.write(b'should have failed already') s.read(1000) + self.fail("server did not reject the client certificate") + except ssl.SSLError as exc: + self.assertIsNotNone( + re.search('alert unknown ca|EOF occurred', str(exc)), + msg=str(exc)) + except OSError as exc: + if exc.errno != errno.ECONNRESET: + raise def test_rude_shutdown(self): """A brutal shutdown of an SSL server should raise an OSError Index: pypy3/lib_pypy/_cffi_ssl/_cffi_src/openssl/asn1.py =================================================================== --- pypy3.orig/lib_pypy/_cffi_ssl/_cffi_src/openssl/asn1.py +++ pypy3/lib_pypy/_cffi_ssl/_cffi_src/openssl/asn1.py @@ -13,13 +13,7 @@ typedef ... ASN1_INTEGER; -struct asn1_string_st { - int length; - int type; - unsigned char *data; - long flags; -}; - +struct asn1_string_st; typedef struct asn1_string_st ASN1_OCTET_STRING; typedef struct asn1_string_st ASN1_IA5STRING; typedef struct asn1_string_st ASN1_BIT_STRING; Index: pypy3/lib_pypy/_cffi_ssl/_cffi_src/openssl/engine.py =================================================================== --- pypy3.orig/lib_pypy/_cffi_ssl/_cffi_src/openssl/engine.py +++ pypy3/lib_pypy/_cffi_ssl/_cffi_src/openssl/engine.py @@ -5,7 +5,10 @@ from __future__ import absolute_import, division, print_function INCLUDES = """ +#include <openssl/configuration.h> +#ifndef OPENSSL_NO_ENGINE #include <openssl/engine.h> +#endif """ TYPES = """ Index: pypy3/lib_pypy/_cffi_ssl/_cffi_src/openssl/err.py =================================================================== --- pypy3.orig/lib_pypy/_cffi_ssl/_cffi_src/openssl/err.py +++ pypy3/lib_pypy/_cffi_ssl/_cffi_src/openssl/err.py @@ -89,6 +89,7 @@ static const int ASN1_R_NO_MULTIPART_BODY_FAILURE; static const int ASN1_R_NO_MULTIPART_BOUNDARY; static const int ASN1_R_HEADER_TOO_LONG; +static const int ASN1_R_NOT_ENOUGH_DATA; static const int DH_R_INVALID_PUBKEY; Index: pypy3/lib_pypy/_cffi_ssl/_cffi_src/openssl/ocsp.py =================================================================== --- pypy3.orig/lib_pypy/_cffi_ssl/_cffi_src/openssl/ocsp.py +++ pypy3/lib_pypy/_cffi_ssl/_cffi_src/openssl/ocsp.py @@ -33,21 +33,21 @@ const X509_NAME **); const X509_ALGOR *OCSP_resp_get0_tbs_sigalg(const OCSP_BASICRESP *); const OCSP_RESPDATA *OCSP_resp_get0_respdata(const OCSP_BASICRESP *); -X509_EXTENSION *OCSP_BASICRESP_get_ext(OCSP_BASICRESP *, int); +const X509_EXTENSION *OCSP_BASICRESP_get_ext(OCSP_BASICRESP *, int); int OCSP_resp_count(OCSP_BASICRESP *); OCSP_SINGLERESP *OCSP_resp_get0(OCSP_BASICRESP *, int); int OCSP_SINGLERESP_get_ext_count(OCSP_SINGLERESP *); -X509_EXTENSION *OCSP_SINGLERESP_get_ext(OCSP_SINGLERESP *, int); +const X509_EXTENSION *OCSP_SINGLERESP_get_ext(OCSP_SINGLERESP *, int); int OCSP_single_get0_status(OCSP_SINGLERESP *, int *, ASN1_GENERALIZEDTIME **, ASN1_GENERALIZEDTIME **, ASN1_GENERALIZEDTIME **); int OCSP_REQUEST_get_ext_count(OCSP_REQUEST *); -X509_EXTENSION *OCSP_REQUEST_get_ext(OCSP_REQUEST *, int); +const X509_EXTENSION *OCSP_REQUEST_get_ext(OCSP_REQUEST *, int); int OCSP_request_onereq_count(OCSP_REQUEST *); OCSP_ONEREQ *OCSP_request_onereq_get0(OCSP_REQUEST *, int); int OCSP_ONEREQ_get_ext_count(OCSP_ONEREQ *); -X509_EXTENSION *OCSP_ONEREQ_get_ext(OCSP_ONEREQ *, int); +const X509_EXTENSION *OCSP_ONEREQ_get_ext(OCSP_ONEREQ *, int); OCSP_CERTID *OCSP_onereq_get0_id(OCSP_ONEREQ *); OCSP_ONEREQ *OCSP_request_add0_id(OCSP_REQUEST *, OCSP_CERTID *); OCSP_CERTID *OCSP_cert_to_id(const EVP_MD *, const X509 *, const X509 *); Index: pypy3/lib_pypy/_cffi_ssl/_cffi_src/openssl/ssl.py =================================================================== --- pypy3.orig/lib_pypy/_cffi_ssl/_cffi_src/openssl/ssl.py +++ pypy3/lib_pypy/_cffi_ssl/_cffi_src/openssl/ssl.py @@ -393,19 +393,6 @@ /* methods */ -/* - * TLSv1_1 and TLSv1_2 are recent additions. Only sufficiently new versions of - * OpenSSL support them. - */ -const SSL_METHOD *TLSv1_method(void); -const SSL_METHOD *TLSv1_1_method(void); -const SSL_METHOD *TLSv1_2_method(void); - -const SSL_METHOD *SSLv3_method(void); -const SSL_METHOD *SSLv3_server_method(void); -const SSL_METHOD *SSLv3_client_method(void); - - /* Added in 1.0.2 */ const SSL_METHOD *DTLS_method(void); const SSL_METHOD *DTLS_server_method(void); @@ -669,14 +656,7 @@ */ static const long Cryptography_HAS_SSL2 = 0; -#ifdef OPENSSL_NO_SSL3_METHOD static const long Cryptography_HAS_SSL3_METHOD = 0; -const SSL_METHOD* (*SSLv3_method)(void) = NULL; -const SSL_METHOD* (*SSLv3_client_method)(void) = NULL; -const SSL_METHOD* (*SSLv3_server_method)(void) = NULL; -#else -static const long Cryptography_HAS_SSL3_METHOD = 1; -#endif static const long Cryptography_HAS_TLSEXT_HOSTNAME = 1; static const long Cryptography_HAS_TLSEXT_STATUS_REQ_CB = 1; Index: pypy3/lib_pypy/_cffi_ssl/_cffi_src/openssl/x509.py =================================================================== --- pypy3.orig/lib_pypy/_cffi_ssl/_cffi_src/openssl/x509.py +++ pypy3/lib_pypy/_cffi_ssl/_cffi_src/openssl/x509.py @@ -58,7 +58,7 @@ EVP_PKEY *X509_get_pubkey(X509 *); int X509_set_pubkey(X509 *, EVP_PKEY *); -unsigned char *X509_alias_get0(X509 *, int *); +const unsigned char *X509_alias_get0(X509 *, int *); int X509_sign(X509 *, EVP_PKEY *, const EVP_MD *); int X509_digest(const X509 *, const EVP_MD *, unsigned char *, unsigned int *); @@ -74,7 +74,7 @@ int X509_add_ext(X509 *, X509_EXTENSION *, int); X509_EXTENSION *X509_EXTENSION_dup(X509_EXTENSION *); -ASN1_OBJECT *X509_EXTENSION_get_object(X509_EXTENSION *); +const ASN1_OBJECT *X509_EXTENSION_get_object(X509_EXTENSION *); void X509_EXTENSION_free(X509_EXTENSION *); int i2d_X509(X509 *, unsigned char **); @@ -92,7 +92,7 @@ X509_EXTENSIONS *X509_REQ_get_extensions(X509_REQ *); int X509V3_EXT_print(BIO *, X509_EXTENSION *, unsigned long, int); -ASN1_OCTET_STRING *X509_EXTENSION_get_data(X509_EXTENSION *); +const ASN1_OCTET_STRING *X509_EXTENSION_get_data(X509_EXTENSION *); X509_REVOKED *X509_REVOKED_new(void); void X509_REVOKED_free(X509_REVOKED *); @@ -158,10 +158,10 @@ /* These became const X509 in 1.1.0 */ int X509_get_ext_count(X509 *); -X509_EXTENSION *X509_get_ext(X509 *, int); +const X509_EXTENSION *X509_get_ext(X509 *, int); int X509_get_ext_by_NID(X509 *, int, int); -X509_NAME *X509_get_subject_name(X509 *); -X509_NAME *X509_get_issuer_name(X509 *); +const X509_NAME *X509_get_subject_name(X509 *); +const X509_NAME *X509_get_issuer_name(X509 *); /* This became const ASN1_OBJECT * in 1.1.0 */ X509_EXTENSION *X509_EXTENSION_create_by_OBJ(X509_EXTENSION **, @@ -174,10 +174,10 @@ /* This became const X509_REVOKED * in 1.1.0 */ int X509_REVOKED_get_ext_count(X509_REVOKED *); -X509_EXTENSION *X509_REVOKED_get_ext(X509_REVOKED *, int); +const X509_EXTENSION *X509_REVOKED_get_ext(X509_REVOKED *, int); /* This became const X509_CRL * in 1.1.0 */ -X509_EXTENSION *X509_CRL_get_ext(X509_CRL *, int); +const X509_EXTENSION *X509_CRL_get_ext(X509_CRL *, int); int X509_CRL_get_ext_count(X509_CRL *); int X509_CRL_get0_by_serial(X509_CRL *, X509_REVOKED **, ASN1_INTEGER *); @@ -199,7 +199,7 @@ ASN1_TIME *X509_get_notAfter(X509 *); long X509_REQ_get_version(X509_REQ *); -X509_NAME *X509_REQ_get_subject_name(X509_REQ *); +const X509_NAME *X509_REQ_get_subject_name(X509_REQ *); Cryptography_STACK_OF_X509 *sk_X509_new_null(void); void sk_X509_free(Cryptography_STACK_OF_X509 *); @@ -226,7 +226,7 @@ X509_CRL *sk_X509_CRL_value(Cryptography_STACK_OF_X509_CRL *, int); long X509_CRL_get_version(X509_CRL *); -X509_NAME *X509_CRL_get_issuer(X509_CRL *); +const X509_NAME *X509_CRL_get_issuer(X509_CRL *); Cryptography_STACK_OF_X509_REVOKED *X509_CRL_get_REVOKED(X509_CRL *); /* These aren't macros these arguments are all const X on openssl > 1.0.x */ Index: pypy3/lib_pypy/_cffi_ssl/_cffi_src/openssl/x509_vfy.py =================================================================== --- pypy3.orig/lib_pypy/_cffi_ssl/_cffi_src/openssl/x509_vfy.py +++ pypy3/lib_pypy/_cffi_ssl/_cffi_src/openssl/x509_vfy.py @@ -34,8 +34,6 @@ typedef ... X509_VERIFY_PARAM; typedef ... X509_STORE_CTX; -typedef int (*X509_STORE_CTX_get_issuer_fn)(X509 **, X509_STORE_CTX *, X509 *); - /* While these are defined in the source as ints, they're tagged here as longs, just in case they ever grow to large, such as what we saw with OP_ALL. */ @@ -217,8 +215,6 @@ /* added in 1.1.0 */ X509 *X509_STORE_CTX_get0_cert(X509_STORE_CTX *); -X509_STORE_CTX_get_issuer_fn X509_STORE_get_get_issuer(X509_STORE *); -void X509_STORE_set_get_issuer(X509_STORE *, X509_STORE_CTX_get_issuer_fn); /* added in 3.3.0 */ Cryptography_STACK_OF_X509_OBJECT *X509_STORE_get1_objects(X509_STORE *); @@ -337,15 +333,7 @@ } #endif -#if CRYPTOGRAPHY_OPENSSL_LESS_THAN_110 static const long Cryptography_HAS_X509_STORE_CTX_GET_ISSUER = 0; -typedef void *X509_STORE_CTX_get_issuer_fn; -X509_STORE_CTX_get_issuer_fn (*X509_STORE_get_get_issuer)(X509_STORE *) = NULL; -void (*X509_STORE_set_get_issuer)(X509_STORE *, - X509_STORE_CTX_get_issuer_fn) = NULL; -#else -static const long Cryptography_HAS_X509_STORE_CTX_GET_ISSUER = 1; -#endif #if CRYPTOGRAPHY_OPENSSL_LESS_THAN_330 static X509_OBJECT *x509_object_dup(const X509_OBJECT *obj) Index: pypy3/lib_pypy/_cffi_ssl/_cffi_src/openssl/x509name.py =================================================================== --- pypy3.orig/lib_pypy/_cffi_ssl/_cffi_src/openssl/x509name.py +++ pypy3/lib_pypy/_cffi_ssl/_cffi_src/openssl/x509name.py @@ -38,13 +38,13 @@ int Cryptography_X509_NAME_ENTRY_set(X509_NAME_ENTRY *); /* These became const X509_NAME * in 1.1.0 */ int X509_NAME_entry_count(X509_NAME *); -X509_NAME_ENTRY *X509_NAME_get_entry(X509_NAME *, int); +const X509_NAME_ENTRY *X509_NAME_get_entry(X509_NAME *, int); char *X509_NAME_oneline(X509_NAME *, char *, int); int X509_NAME_print_ex(BIO *, X509_NAME *, int, unsigned long); /* These became const X509_NAME_ENTRY * in 1.1.0 */ -ASN1_OBJECT *X509_NAME_ENTRY_get_object(X509_NAME_ENTRY *); -ASN1_STRING *X509_NAME_ENTRY_get_data(X509_NAME_ENTRY *); +const ASN1_OBJECT *X509_NAME_ENTRY_get_object(X509_NAME_ENTRY *); +const ASN1_STRING *X509_NAME_ENTRY_get_data(X509_NAME_ENTRY *); int X509_NAME_add_entry(X509_NAME *, X509_NAME_ENTRY *, int, int); /* this became const unsigned char * in 1.1.0 */ Index: pypy3/lib_pypy/_cffi_ssl/_stdssl/__init__.py =================================================================== --- pypy3.orig/lib_pypy/_cffi_ssl/_stdssl/__init__.py +++ pypy3/lib_pypy/_cffi_ssl/_stdssl/__init__.py @@ -1172,13 +1172,13 @@ self.keylog_bio = None if protocol == PROTOCOL_TLSv1: warnings.warn("ssl.PROTOCOL_TLSv1 is deprecated", DeprecationWarning) - method = lib.TLSv1_method() + method = lib.TLS_method() elif lib.Cryptography_HAS_TLSv1_1 and protocol == PROTOCOL_TLSv1_1: warnings.warn("ssl.PROTOCOL_TLSv1_1 is deprecated", DeprecationWarning) - method = lib.TLSv1_1_method() + method = lib.TLS_method() elif lib.Cryptography_HAS_TLSv1_2 and protocol == PROTOCOL_TLSv1_2 : warnings.warn("ssl.PROTOCOL_TLSv1_2 is deprecated", DeprecationWarning) - method = lib.TLSv1_2_method() + method = lib.TLS_method() elif SSLv3_method_ok and protocol == PROTOCOL_SSLv3: warnings.warn("ssl.PROTOCOL_SSLv3 is deprecated", DeprecationWarning) method = lib.SSLv3_method() @@ -1215,6 +1215,18 @@ options |= lib.SSL_OP_NO_SSLv2 if not SSLv3_method_ok or protocol != PROTOCOL_SSLv3: options |= lib.SSL_OP_NO_SSLv3 + # The version-specific TLSv1/1_1/1_2_method() functions were removed + # in OpenSSL 4. PROTOCOL_TLSv1/1_1/1_2 now use the generic TLS_method() + # and disable the other protocol versions via SSL_OP_NO_TLSv*. + if protocol == PROTOCOL_TLSv1: + options |= (lib.SSL_OP_NO_TLSv1_1 | lib.SSL_OP_NO_TLSv1_2 | + lib.SSL_OP_NO_TLSv1_3) + elif protocol == PROTOCOL_TLSv1_1: + options |= (lib.SSL_OP_NO_TLSv1 | lib.SSL_OP_NO_TLSv1_2 | + lib.SSL_OP_NO_TLSv1_3) + elif protocol == PROTOCOL_TLSv1_2: + options |= (lib.SSL_OP_NO_TLSv1 | lib.SSL_OP_NO_TLSv1_1 | + lib.SSL_OP_NO_TLSv1_3) # Minimal security flags for server and client side context. # Client sockets ignore server-side parameters. options |= lib.SSL_OP_NO_COMPRESSION @@ -1627,7 +1639,7 @@ elif (ca_file_type == lib.SSL_FILETYPE_ASN1 and loaded > 0 and lib.ERR_GET_LIB(err) == lib.ERR_LIB_ASN1 and - lib.ERR_GET_REASON(err) == lib.ASN1_R_HEADER_TOO_LONG): + lib.ERR_GET_REASON(err) == lib.ASN1_R_NOT_ENOUGH_DATA): # EOF ASN1 file, not an error lib.ERR_clear_error() elif (ca_file_type == lib.SSL_FILETYPE_PEM and Index: pypy3/lib_pypy/_cffi_ssl/_stdssl/certificate.py =================================================================== --- pypy3.orig/lib_pypy/_cffi_ssl/_stdssl/certificate.py +++ pypy3/lib_pypy/_cffi_ssl/_stdssl/certificate.py @@ -151,7 +151,7 @@ ad.location.type != lib.GEN_URI: continue uri = ad.location.d.uniformResourceIdentifier - ostr = _str_with_len(uri.data, uri.length) + ostr = _string_from_asn1(uri) lst.append(ostr) lib.AUTHORITY_INFO_ACCESS_free(info) @@ -187,9 +187,9 @@ raise ssl_error("No method for internalizing subjectAltName!") ext_data = lib.X509_EXTENSION_get_data(ext) - ext_data_len = ext_data.length + ext_data_len = lib.ASN1_STRING_length(ext_data) ext_data_value = ffi.new("unsigned char**", ffi.NULL) - ext_data_value[0] = ext_data.data + ext_data_value[0] = lib.ASN1_STRING_get0_data(ext_data) if method.it != ffi.NULL: names = lib.ASN1_item_d2i(ffi.NULL, ext_data_value, ext_data_len, lib.ASN1_ITEM_ptr(method.it)) @@ -387,7 +387,7 @@ continue uri = gn.d.uniformResourceIdentifier; - ouri = _str_with_len(uri.data, uri.length) + ouri = _string_from_asn1(uri) lst.append(ouri) lib.CRL_DIST_POINTS_free(dps);
