Re: [Python-Dev] TLS 1.3 support
my country, mauritius, had a participation in the dev, internet society vid : https://youtu.be/u6rz4PWA_As?t=4524 the red blue yellow green flag is our flag Abdur-Rahmaan Janhangeer https://github.com/Abdur-rahmaanJ Mauritius On Thu, 16 Aug 2018, 17:27 Christian Heimes, wrote: > Hi, > > I have some exciting news. RFC 8446 [1] for TLS 1.3 was finalized a > couple of days ago. The new TLS standard comes with several major > improvements, but also with major changes. > > First some good news: > Python's ssl module not compiles with OpenSSL 1.1.1-pre. It's also able > to negotiate and establish TLS 1.3 connections successfully. Almost all > unit tests are passing, too. Yesterday I pushed some patches to fix 2.7, > 3.6, 3.7, and master. > > Of course, there is a catch. While TLS 1.0 to 1.2 were just gradual > improvements over SSLv3, TLS 1.3 behaves differently on several levels. > The article [2] from Nick Sullivan explains the biggest changes well. > Most improvements like 1-RTT handshake, enforced perfect forward secrecy > and improved cryptography are not visible to applications. However some > changes and new features are going to need additional work and new APIs. > > I'm open to discuss the matter at the core dev sprints in Seattle next > month. > > I'm still in the process of reviewing and assessing changes. OpenSSL > 1.1.1's APIs are not finalized yet, too. I'm working closely with Red > Hat's crypto team and OpenSSL devs on the topic. I'll walk you through > some changes and possibly new APIs that I have identified so far. > > > New cipher suites > = > > Old ciphers suites in TLS 1.2 and earlier looked like > >TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 > > The parameters are key agreement/key exchange (ECDHE), authentication > (RSA), bulk encryption (AES-256) with block and/or authenticated > encryption mode (GCM) and finally a pseudo random function for MAC. > > In TLS 1.3, the cipher suites only specify bulk encryption and PRF. The > parameters for key agreement and authentication are configured > differently. TLS 1.3 uses dedicated TLS extensions for that, too. This > allows more fine grained control to specific supported EC curves, > signature algorithms (RSA PKCS#1.5, RSASSA-PSS) and so on. Some settings > also apply to TLS 1.2 already. > > Further more, OpenSSL has introduced new APIs to configure TLS 1.3 > cipher suites. SSLContext.set_ciphers() cannot be used to disable TLS > 1.3 suites. For now, that is fine. All default algorithms are secure and > state-of-the-art. > > For 3.8, I'm planning to make some of the settings configurable. I might > need to backport the new APIs to 3.7 and 3.6, though. > > > post handshake authentication > = > > TLS 1.3 introduced the concept of post handshake authentication (PHA). A > server can either request a client to provide client cert authentication > during the handshake, directly after the handshake, or at any later > point. It makes optional authentication for HTTP much more sane. For > example a server may only require authentication for POST requests and > GET requests to /secure/* path. In TLS 1.2, optional client cert auth > requires a complete renegotiation of the handshake. > > OpenSSL 1.1.1-pre9 will most likely disable PHA by default [3], because > PHA breaks some assumptions of TLS 1.2 clients and servers. But PHA > might be required for path specific authentication. I need to wait and > see how mod_ssl implements the feature. > > I might have to add SSL_VERIFY_POST_HANDSHAKE, > SSL_verify_client_post_handshake(), and > SSL_CTX_set_post_handshake_auth() from [3][4] to Python 2.7, 3.6, 3.7 > and master. > > > session handling > > > TLS 1.3 uses a different approach to handle sessions. Simple speaking, > session data is exchanged after the handshake, there can be more than > one ticket, tickets must not be reused, and session resumption involved > DHE. ssl.SSLSession [5] feature only applies to TLS 1.2 and earlier. I > need to come up with a different approach here. For the time being, TLS > 1.3 won't be compatible with manual session control. It's not a big deal > since it is an advanced feature any way. > > The new session system also seems to break FTP over TLS. FTP uses two > different TCP connections (control channel, data channel). For FTP over > TLS, the data channel must reuse a session of the control channel. For > now, I'm plannung to restrict FTP to TLSv1.2. > > > 0-RTT resumption > = > > 0 round trip time handshake on session resumption is an optional and > advanced feature. I might expose the feature in Python 3.8. > > Regards, > Christian > > > [1] https://tools.ietf.org/html/rfc8446 > [2] https://blog.cloudflare.com/rfc-8446-aka-tls-1-3/ > [3] https://github.com/openssl/openssl/issues/6933 > [4] https://www.openssl.org/docs/man1.1.1/man3/SSL_CTX_set_verify.html > [5] https://docs.python.org/3/library/ssl.html#ssl-session > > ___
Re: [Python-Dev] TLS 1.3 support
Thank you Christian for your great work! On Thu, Aug 16, 2018 at 3:28 PM Christian Heimes wrote: > Hi, > > I have some exciting news. RFC 8446 [1] for TLS 1.3 was finalized a > couple of days ago. The new TLS standard comes with several major > improvements, but also with major changes. > > First some good news: > Python's ssl module not compiles with OpenSSL 1.1.1-pre. It's also able > to negotiate and establish TLS 1.3 connections successfully. Almost all > unit tests are passing, too. Yesterday I pushed some patches to fix 2.7, > 3.6, 3.7, and master. > > Of course, there is a catch. While TLS 1.0 to 1.2 were just gradual > improvements over SSLv3, TLS 1.3 behaves differently on several levels. > The article [2] from Nick Sullivan explains the biggest changes well. > Most improvements like 1-RTT handshake, enforced perfect forward secrecy > and improved cryptography are not visible to applications. However some > changes and new features are going to need additional work and new APIs. > > I'm open to discuss the matter at the core dev sprints in Seattle next > month. > > I'm still in the process of reviewing and assessing changes. OpenSSL > 1.1.1's APIs are not finalized yet, too. I'm working closely with Red > Hat's crypto team and OpenSSL devs on the topic. I'll walk you through > some changes and possibly new APIs that I have identified so far. > > > New cipher suites > = > > Old ciphers suites in TLS 1.2 and earlier looked like > >TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 > > The parameters are key agreement/key exchange (ECDHE), authentication > (RSA), bulk encryption (AES-256) with block and/or authenticated > encryption mode (GCM) and finally a pseudo random function for MAC. > > In TLS 1.3, the cipher suites only specify bulk encryption and PRF. The > parameters for key agreement and authentication are configured > differently. TLS 1.3 uses dedicated TLS extensions for that, too. This > allows more fine grained control to specific supported EC curves, > signature algorithms (RSA PKCS#1.5, RSASSA-PSS) and so on. Some settings > also apply to TLS 1.2 already. > > Further more, OpenSSL has introduced new APIs to configure TLS 1.3 > cipher suites. SSLContext.set_ciphers() cannot be used to disable TLS > 1.3 suites. For now, that is fine. All default algorithms are secure and > state-of-the-art. > > For 3.8, I'm planning to make some of the settings configurable. I might > need to backport the new APIs to 3.7 and 3.6, though. > > > post handshake authentication > = > > TLS 1.3 introduced the concept of post handshake authentication (PHA). A > server can either request a client to provide client cert authentication > during the handshake, directly after the handshake, or at any later > point. It makes optional authentication for HTTP much more sane. For > example a server may only require authentication for POST requests and > GET requests to /secure/* path. In TLS 1.2, optional client cert auth > requires a complete renegotiation of the handshake. > > OpenSSL 1.1.1-pre9 will most likely disable PHA by default [3], because > PHA breaks some assumptions of TLS 1.2 clients and servers. But PHA > might be required for path specific authentication. I need to wait and > see how mod_ssl implements the feature. > > I might have to add SSL_VERIFY_POST_HANDSHAKE, > SSL_verify_client_post_handshake(), and > SSL_CTX_set_post_handshake_auth() from [3][4] to Python 2.7, 3.6, 3.7 > and master. > > > session handling > > > TLS 1.3 uses a different approach to handle sessions. Simple speaking, > session data is exchanged after the handshake, there can be more than > one ticket, tickets must not be reused, and session resumption involved > DHE. ssl.SSLSession [5] feature only applies to TLS 1.2 and earlier. I > need to come up with a different approach here. For the time being, TLS > 1.3 won't be compatible with manual session control. It's not a big deal > since it is an advanced feature any way. > > The new session system also seems to break FTP over TLS. FTP uses two > different TCP connections (control channel, data channel). For FTP over > TLS, the data channel must reuse a session of the control channel. For > now, I'm plannung to restrict FTP to TLSv1.2. > > > 0-RTT resumption > = > > 0 round trip time handshake on session resumption is an optional and > advanced feature. I might expose the feature in Python 3.8. > > Regards, > Christian > > > [1] https://tools.ietf.org/html/rfc8446 > [2] https://blog.cloudflare.com/rfc-8446-aka-tls-1-3/ > [3] https://github.com/openssl/openssl/issues/6933 > [4] https://www.openssl.org/docs/man1.1.1/man3/SSL_CTX_set_verify.html > [5] https://docs.python.org/3/library/ssl.html#ssl-session > > ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/
Re: [Python-Dev] TLS 1.3 support
-BEGIN PGP SIGNED MESSAGE- Hash: SHA512 Argh, typo! Thanks for pointing that out, Miro. I meant to write "now compiles with OpenSSL 1.1.1-pre". On 2018-08-16 15:27, Christian Heimes wrote: > First some good news: Python's ssl module not compiles with OpenSSL > 1.1.1-pre. It's also able ^^^ Python's ssl module NOW compiles with OpenSSL 1.1.1-pre. The latest release OpenSSL 1.1.1-pre8 from June has some bugs, e.g. twp way shutdown is broken under some circumstances [1]. The current git master of CPython is mostly compatible with the latest git snapshot of OpenSSL. I'm waiting for OpenSSL devs to merge PR 6938 [2] and release pre9. Christian [1] https://github.com/openssl/openssl/issues/6262 [2] https://github.com/openssl/openssl/pull/6938 -BEGIN PGP SIGNATURE- iQEzBAEBCgAdFiEEL7+ElqIWjf0BBClLhmhSSSEttokFAlt1hTIACgkQhmhSSSEt tokiMAgApG/ruPxq0WW5FsOuQquH+jH6coifKzEsHsZfIhHLGQwVlpUjRhtQdj+T YpprzjRU6AS+R7J+/fr7t032TPfogSSpCHSh22RIeY5BlYkNBjjyhxuItwGu8SiP pvKThAsLiu4x4SCJHMJHgjv/M9ljAsUM9VgUjCy8UQ8p6zfKjC2B/Vwg8WLO0XOn 90ggMEpMfyb6t0daUIEe2eEPBmfwN5K0cctWg1rb98LUbfOdq3MMEWjKgYBBsCS7 vk3RIKE1ZMtzDJaO3M09wHYLu2QXtUpo9k3VU2CABC65Jw9SkyMDUGyUFZjV5GU/ dW0K5DyHr/6JHfHWfq37Y3y/yBXNsA== =gVcQ -END PGP SIGNATURE- ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] TLS 1.3 support
Hi, I have some exciting news. RFC 8446 [1] for TLS 1.3 was finalized a couple of days ago. The new TLS standard comes with several major improvements, but also with major changes. First some good news: Python's ssl module not compiles with OpenSSL 1.1.1-pre. It's also able to negotiate and establish TLS 1.3 connections successfully. Almost all unit tests are passing, too. Yesterday I pushed some patches to fix 2.7, 3.6, 3.7, and master. Of course, there is a catch. While TLS 1.0 to 1.2 were just gradual improvements over SSLv3, TLS 1.3 behaves differently on several levels. The article [2] from Nick Sullivan explains the biggest changes well. Most improvements like 1-RTT handshake, enforced perfect forward secrecy and improved cryptography are not visible to applications. However some changes and new features are going to need additional work and new APIs. I'm open to discuss the matter at the core dev sprints in Seattle next month. I'm still in the process of reviewing and assessing changes. OpenSSL 1.1.1's APIs are not finalized yet, too. I'm working closely with Red Hat's crypto team and OpenSSL devs on the topic. I'll walk you through some changes and possibly new APIs that I have identified so far. New cipher suites = Old ciphers suites in TLS 1.2 and earlier looked like TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 The parameters are key agreement/key exchange (ECDHE), authentication (RSA), bulk encryption (AES-256) with block and/or authenticated encryption mode (GCM) and finally a pseudo random function for MAC. In TLS 1.3, the cipher suites only specify bulk encryption and PRF. The parameters for key agreement and authentication are configured differently. TLS 1.3 uses dedicated TLS extensions for that, too. This allows more fine grained control to specific supported EC curves, signature algorithms (RSA PKCS#1.5, RSASSA-PSS) and so on. Some settings also apply to TLS 1.2 already. Further more, OpenSSL has introduced new APIs to configure TLS 1.3 cipher suites. SSLContext.set_ciphers() cannot be used to disable TLS 1.3 suites. For now, that is fine. All default algorithms are secure and state-of-the-art. For 3.8, I'm planning to make some of the settings configurable. I might need to backport the new APIs to 3.7 and 3.6, though. post handshake authentication = TLS 1.3 introduced the concept of post handshake authentication (PHA). A server can either request a client to provide client cert authentication during the handshake, directly after the handshake, or at any later point. It makes optional authentication for HTTP much more sane. For example a server may only require authentication for POST requests and GET requests to /secure/* path. In TLS 1.2, optional client cert auth requires a complete renegotiation of the handshake. OpenSSL 1.1.1-pre9 will most likely disable PHA by default [3], because PHA breaks some assumptions of TLS 1.2 clients and servers. But PHA might be required for path specific authentication. I need to wait and see how mod_ssl implements the feature. I might have to add SSL_VERIFY_POST_HANDSHAKE, SSL_verify_client_post_handshake(), and SSL_CTX_set_post_handshake_auth() from [3][4] to Python 2.7, 3.6, 3.7 and master. session handling TLS 1.3 uses a different approach to handle sessions. Simple speaking, session data is exchanged after the handshake, there can be more than one ticket, tickets must not be reused, and session resumption involved DHE. ssl.SSLSession [5] feature only applies to TLS 1.2 and earlier. I need to come up with a different approach here. For the time being, TLS 1.3 won't be compatible with manual session control. It's not a big deal since it is an advanced feature any way. The new session system also seems to break FTP over TLS. FTP uses two different TCP connections (control channel, data channel). For FTP over TLS, the data channel must reuse a session of the control channel. For now, I'm plannung to restrict FTP to TLSv1.2. 0-RTT resumption = 0 round trip time handshake on session resumption is an optional and advanced feature. I might expose the feature in Python 3.8. Regards, Christian [1] https://tools.ietf.org/html/rfc8446 [2] https://blog.cloudflare.com/rfc-8446-aka-tls-1-3/ [3] https://github.com/openssl/openssl/issues/6933 [4] https://www.openssl.org/docs/man1.1.1/man3/SSL_CTX_set_verify.html [5] https://docs.python.org/3/library/ssl.html#ssl-session signature.asc Description: OpenPGP digital signature ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [issue17180] shutil copy* unsafe on POSIX - they preserve setuid/setgit bits
I want to believe this can be resolved - without breakage on POSIX. Clarification: while Mac/OS falls under "posix" in python terms - maybe "breakage" will need to be accepted, or, for "back-ports" Mac/OS will be "as if root or super-user" and use an additional (optional) argument in 3.8 and beyond to keep backwards compatibility. Short text: to proceed I think we should start with getting some additional tests into test_shutil.py asap so that we can see how systems respond without any changes. My experience is that cp -r[pP] behaves the same as shutil.copy*() when the EUID==0, aka superuser, but strips special bits from files and cannot copy the UID/GID owner bits of the inode. I would appreciate someone helping me writing more extensive testing. We need to test: * root * not-root, but owner * not-root, not owner, but in group * not-root, "other", other "read" access exists * if the test does not already exist - also check behavior when directories have/do not have "search" (x-bit) enabled. I am working on a patch to address these different conditions. Ideally, the "not-owner" and "not-group" tests can be run by creating the "copy.me" area as root, setting perms, etc. and then using su -c to run the shutil.copy*() call and back as root make the verification. ±±± Perspective ±±± If this is too much discussion, please reply with suggestions - privately - on what I could do better to not waste your time. The issue seems unchanged since original posting. The original report states: hen copying the mode of a file with copy, copy2, copymode, copystat or copytree, all permission bits are copied (including setuid and setgit), but the owner of the file is not. This can be used for privilege escalation. ...snip... The behaviour of copymode/copystat in this case is the same as `chmod --reference', and there can be some expectation of unsafety, but copy/copy2/copytree's behaviour differs from that of `cp -p', and this is a non-obvious difference. For clarity: GNU chmod states: --reference=RFILE use RFILE's mode instead of MODE values Additionally, the chmod man page reminds us the "special bit" masking behavior is different for files and directries. Specifically, SUID, SGID and SVTX should not be cleared unless specifically requested by a chmod "u-s,g-s" specification. "... a directory's unmentioned set user and group ID bits are not affected" Additional comments discuss: short window of opportunity (files are copied first, then mode bits copied) breakage with the past (copytree used as "backup", regardless of version) And the comment/opinion that shutil.copy() should emulate cp (implies emulate "cp -r", so neither -p nor -P) it seems shutil.copy2() is adding the -p (or -P if follow_symlinks=false) There was a modification to test_shutil.py suggested as part of a patch. I added that to verify the issue is still current. ±±± diff --git a/Lib/test/test_shutil.py b/Lib/test/test_shutil.py index 7e0a3292e0..7ceefd1ebc 100644 --- a/Lib/test/test_shutil.py +++ b/Lib/test/test_shutil.py @@ -1471,6 +1471,24 @@ class TestShutil(unittest.TestCase): rv = shutil.copytree(src_dir, dst_dir) self.assertEqual(['foo'], os.listdir(rv)) + @unittest.skipUnless((os.name == "posix" and os.geteuid() != 0), "Requires POSIX compatible OS and non-root userid") + def test_copy_remove_setuid(self): + src_dir = self.mkdtemp() + src_file = os.path.join(src_dir, 'foo') + write_file(src_file, 'foo') + dst_file = os.path.join(src_dir, 'bar') + harmful_mode = stat.S_IRUSR | stat.S_IXUSR | stat.S_ISUID + harmless_mode = stat.S_IRUSR | stat.S_IXUSR + + # set mode and verify + os.chmod(src_file, harmful_mode) + mode = stat.S_IMODE(os.stat(src_file).st_mode) + self.assertTrue(oct(mode), oct(harmful_mode)) + + # check that copy does not preserve harmful bits + shutil.copy(src_file, dst_file) + mode = stat.S_IMODE(os.stat(dst_file).st_mode) + self.assertEqual(oct(mode), oct(harmless_mode)) class TestWhich(unittest.TestCase): ±±± The result is: root@x066:[/data/prj/python/python3-3.8]./python -m test -v test_shutil == CPython 3.8.0a0 (heads/master:cca4eec3c0, Aug 13 2018, 04:53:15) [C] == AIX-1-00C291F54C00-powerpc-32bit big-endian == cwd: /data/prj/python/python3-3.8/build/test_python_10944516 == CPU count: 8 == encodings: locale=ISO8859-1, FS=iso8859-1 Run tests sequentially ... test_copy_remove_setuid (test.test_shutil.TestShutil) ... FAIL ... == FAIL: test_copy_remove_setuid (test.test_shutil.TestShutil) -- Traceback (most recent call last): File "/data/prj/python/git/python3-3.8/Lib/test/test_shutil.py", line 1491, in test_copy_remove_setuid self.assertEqual(oct(mode), oct(harmless_mode)) AssertionError: '0o4500' != '0o500' - 0o4500 ? - + 0o500 -
