[issue32394] socket lib beahavior change in 3.6.4

Change by Steve Dower : -- assignee: -> steve.dower stage: backport needed -> resolved status: open -> closed ___ Python tracker

[issue32394] socket lib beahavior change in 3.6.4

miss-islington added the comment: New changeset 53d3f8a89971bac3d2f454ff9f923066ecc3a6d9 by Miss Islington (bot) in branch '3.7': bpo-32394: Remove some TCP options on old version Windows. (GH-5523)

[issue32394] socket lib beahavior change in 3.6.4

Steve Dower added the comment: Agreed. I've merged these (and Miss Islington should get the 3.7 backport when CI completes) -- resolution: -> fixed stage: patch review -> backport needed ___ Python tracker

[issue32394] socket lib beahavior change in 3.6.4

Steve Dower added the comment: New changeset 1278c21f5234477aab21531773d65ca7ebd1b81f by Steve Dower (animalize) in branch '3.6': [3.6] bpo-32394: Remove some TCP options on older version Windows. (GH-5585)

[issue32394] socket lib beahavior change in 3.6.4

Change by miss-islington : -- pull_requests: +5680 ___ Python tracker ___

[issue32394] socket lib beahavior change in 3.6.4

Steve Dower added the comment: New changeset 19e7d48ce89422091f9af93038b9fee075d46e9e by Steve Dower (animalize) in branch 'master': bpo-32394: Remove some TCP options on old version Windows. (GH-5523)

[issue32394] socket lib beahavior change in 3.6.4

Andrew Svetlov added the comment: I think PRs could be merged -- ___ Python tracker ___

[issue32394] socket lib beahavior change in 3.6.4

Ned Deily added the comment: What's the status of this issue? 3.7.0b2 is tagging in 48 hours or so and 3.6.5rc1 is in less than 2 weeks. -- ___ Python tracker

[issue32394] socket lib beahavior change in 3.6.4

Ma Lin added the comment: PR 5523 is prepared for master/3.7 branches. PR 5585 is prepared for 3.6 branch. -- ___ Python tracker ___

[issue32394] socket lib beahavior change in 3.6.4

Ma Lin added the comment: > What's about other OS/flags? > Should we commit that every exposed socket flag is supported in runtime? > It looks like very heavy burden. Let alone run-time check. Flags only depend on .C code and the building SDK, therefore, for a certain

[issue32394] socket lib beahavior change in 3.6.4

Ma Lin added the comment: Here is PR 5585 for 3.6 branch. For 3.7+, I would suggest patch in socketmodule.c like this: PyMODINIT_FUNC PyInit__socket(void) { PyObject *m, *has_ipv6; ... ... ... +#ifdef MS_WINDOWS + return remove_unusable_flags(m); +#else

[issue32394] socket lib beahavior change in 3.6.4

Change by Ma Lin : -- pull_requests: +5403 ___ Python tracker ___ ___ Python-bugs-list

[issue32394] socket lib beahavior change in 3.6.4

Andrew Svetlov added the comment: Ok -- ___ Python tracker ___ ___

[issue32394] socket lib beahavior change in 3.6.4

Steve Dower added the comment: Oh, and checking Windows version is hard. Better for us to get it right than every single library to risk getting it wrong. (The code used in the second PR is not right - there are IsWindowsVersion* macros that are better.) --

[issue32394] socket lib beahavior change in 3.6.4

Steve Dower added the comment: In this case I like the flags disappearing on older versions, just as they would if you built CPython on a version of Linux that didn't have the flags. The problem is that the Windows SDK always defines enum values for all Windows

[issue32394] socket lib beahavior change in 3.6.4

Ned Deily added the comment: > Ned Deily, what do you think about? I would like to have Steve's opinion. -- ___ Python tracker ___

[issue32394] socket lib beahavior change in 3.6.4

Andrew Svetlov added the comment: If the fix will land into 3.6 bugfix release only -- I can live with it, while the overall looks tricky. Ned Deily, what do you think about? -- ___ Python tracker

[issue32394] socket lib beahavior change in 3.6.4

Nathaniel Smith added the comment: I'm sympathetic to the idea that we don't want to carry around these checks, but also to the idea that this caused a regression in a micro release and that's not cool. Hence the idea that maybe we should keep everything the way it is in 3.7,

[issue32394] socket lib beahavior change in 3.6.4

Andrew Svetlov added the comment: I suggest closing the issue as "won't fix": checking in runtime for only for TCP flags on Windows is a weird exception. Checking all flags on all supported platforms is impossible. Client libraries should process such situations

[issue32394] socket lib beahavior change in 3.6.4

Ma Lin added the comment: > Or the issue is specific for TCP_KEEPCNT for Windows only? Four flags involved. In this table, result column is search results from GitHub. keyword available result TCP_FASTOPEN win10 1607+ 778 TCP_KEEPCNTwin10 1703+ 3356

[issue32394] socket lib beahavior change in 3.6.4

Nathaniel Smith added the comment: I definitely don't think we should get into the game of trying to guess which flags are supported at runtime and only exposing those. It's not as simple as keeping a table of OS versions -- which would be hard enough to get right -- but on

[issue32394] socket lib beahavior change in 3.6.4

INADA Naoki added the comment: > What's about other OS/flags? > Should we commit that every exposed socket flag is supported in runtime? It looks like very heavy burden. I agree with you. It almost impossible. > Or the issue is specific for TCP_KEEPCNT for Windows

[issue32394] socket lib beahavior change in 3.6.4

Andrew Svetlov added the comment: What's about other OS/flags? Should we commit that every exposed socket flag is supported in runtime? It looks like very heavy burden. Or the issue is specific for TCP_KEEPCNT for Windows only? --

[issue32394] socket lib beahavior change in 3.6.4

INADA Naoki added the comment: It seems Linux has TCP_KEEPCNT from very old ages and just checking it's existence was OK for many years. So I'm +0.5 on this Python-side fix. -- ___ Python tracker

[issue32394] socket lib beahavior change in 3.6.4

INADA Naoki added the comment: > On Linux/Unix, the compile-time headers always consist with the system, so > there should not has this problem. > Correct me if I'm wrong. No. Compile-time and run-time system is not always consist. Kernel version may be upgraded /

[issue32394] socket lib beahavior change in 3.6.4

Ma Lin added the comment: > We don't remove unsupported socket flags on Unix, why should we do it for > Windows? We have this problem because: compile with new Windows SDK, but run on old version Windows. On Linux/Unix, the compile-time headers always consist with the

[issue32394] socket lib beahavior change in 3.6.4

Nathaniel Smith added the comment: The other option would be to always hide the new constant on Windows in 3.6, and make it unconditionally available on 3.7. -- nosy: +njs ___ Python tracker

[issue32394] socket lib beahavior change in 3.6.4

Andrew Svetlov added the comment: Socket constants a compile time values, obviously concrete operation system might not support a flag -- but we do nothing with it in runtime. All flags available on compile time are exposed, it's true for modules like socket, os,

[issue32394] socket lib beahavior change in 3.6.4

Andrew Svetlov added the comment: We don't remove unsupported socket flags on Unix, why should we do it for Windows? -- nosy: +asvetlov ___ Python tracker

[issue32394] socket lib beahavior change in 3.6.4

Ma Lin added the comment: I create a new one (PR 5523), I'm not a C & socket expert, so if you want to improve/polish the patch, feel free to create a new PR based on (or not) it. -- ___ Python tracker

[issue32394] socket lib beahavior change in 3.6.4

Change by Ma Lin : -- pull_requests: +5352 ___ Python tracker ___ ___ Python-bugs-list

[issue32394] socket lib beahavior change in 3.6.4

Kamil added the comment: I am sorry, this is the right version CHANGE: #ifdef TCP_KEEPCNT PyModule_AddIntMacro(m, TCP_KEEPCNT); #endif TO: #ifdef TCP_KEEPCNT #ifdef MS_WINDOWS #if defined(_MSC_VER) && _MSC_VER >= 1800 //on Windows avaible only from

[issue32394] socket lib beahavior change in 3.6.4

Kamil added the comment: With сorrect comments: CHANGE: #ifdef TCP_KEEPCNT PyModule_AddIntMacro(m, TCP_KEEPCNT); #endif TO: #ifdef TCP_KEEPCNT #if defined(_MSC_VER) && _MSC_VER >= 1800 if (IsWindows10CreatorsOrGreater()) { //Windows 10 1703(Build:15063

[issue32394] socket lib beahavior change in 3.6.4

Kamil added the comment: I suggest inserting the following code into socketmodule.c: CHANGE: #ifdef TCP_KEEPCNT PyModule_AddIntMacro(m, TCP_KEEPCNT); #endif TO: #ifdef TCP_KEEPCNT #if defined(_MSC_VER) && _MSC_VER >= 1800 // Windows 10 1703 (15063)

[issue32394] socket lib beahavior change in 3.6.4

Ma Lin added the comment: Glad to see PR 5468 not merged, I found it makes socket.py complicated. Now I'm inclined to patch the code in PyInit__socket(void) of socketmodule.c https://github.com/python/cpython/blob/3.6/Modules/socketmodule.c#L6504 There already has a

[issue32394] socket lib beahavior change in 3.6.4

Steve Dower added the comment: PR looks good to me. Unfortunately without SMS (I'm travelling) I can't get into my account from my phone, so if someone else wants to add the backport tags (3.7 and 3.6) and finish it then feel free. -- versions: +Python 3.7,

[issue32394] socket lib beahavior change in 3.6.4

Steve Dower added the comment: Yes, adding the member back will put you back in the broken state, but there's nothing we can do to stop it. Thanks for the PR - I'll take a look, and if I'm able to log into GitHub on my phone maybe finish it. --

[issue32394] socket lib beahavior change in 3.6.4

Ma Lin added the comment: So we either need to explicitly exclude this symbol on Windows (at least until we drop support for pre-Windows 10 versions) or silently ignore setsockopt errors for future good arguments. I'm inclined to do the former - other opinions?

[issue32394] socket lib beahavior change in 3.6.4

Change by Ma Lin : -- keywords: +patch pull_requests: +5294 stage: test needed -> patch review ___ Python tracker ___

[issue32394] socket lib beahavior change in 3.6.4

Steve Dower added the comment: Nice. Think you can turn that into a pull request on GitHub? -- ___ Python tracker ___

[issue32394] socket lib beahavior change in 3.6.4

Ma Lin added the comment: Base on Kamil's code, a little improvements. if hasattr(sys, 'getwindowsversion'): WIN_MAJOR, _, WIN_BUILD, *_ = sys.getwindowsversion() if WIN_MAJOR == 10: if WIN_BUILD >= 15063:# Windows 10 1703 pass elif

[issue32394] socket lib beahavior change in 3.6.4

Steve Dower added the comment: I like Kamil's suggestion. -- ___ Python tracker ___

[issue32394] socket lib beahavior change in 3.6.4

Ned Deily added the comment: Steve, is there something to be done here for 3.7.0? -- ___ Python tracker ___

[issue32394] socket lib beahavior change in 3.6.4

Kamil added the comment: I suggest inserting the following code into socket.py: if hasattr(sys, 'getwindowsversion') and sys.getwindowsversion()[0] < 10: del socket.TCP_KEEPCNT del socket.TCP_FASTOPEN -- ___ Python tracker

[issue32394] socket lib beahavior change in 3.6.4

TJG added the comment: Excluding for now seems like a simple option. (Basically a reversion to previous behaviour). And allows us easily to include again later easily. Messing with setsockopt seems a little more risky. In short: I'm with you -- exclude for now.

[issue32394] socket lib beahavior change in 3.6.4

Steve Dower added the comment: Okay, so it looks like we don't have any prior art to conditionally omit constants from _socket based on OS level, and nothing in the setsockopt() doc suggests that it may fail like this. So we either need to explicitly exclude this

[issue32394] socket lib beahavior change in 3.6.4

Kamil added the comment: websocket-client 0.44.0 https://pypi.python.org/pypi/websocket-client/0.44.0 My script gives the following Erroe: File "C:\Program Files\Python36\lib\site-packages\websocket\_http.py", line 108, in _open_socket sock.setsockopt(*opts) OSError:

[issue32394] socket lib beahavior change in 3.6.4

Steve Dower added the comment: Yes, I recognise that the change happened. I don't understand what breaks as a result. You said it breaks "many libraries" - can you name some of them and provide sample code? -- ___ Python

[issue32394] socket lib beahavior change in 3.6.4

Kamil added the comment: My OS version is Windows 7 x64. I ran the script on the same computer, but with different versions of the python: import socket import platform print('1) OS Info: ', platform.architecture(), platform.platform()) print('2) Python Info: ',

[issue32394] socket lib beahavior change in 3.6.4

Steve Dower added the comment: Yeah, I updated the build machine before doing 3.6.4rc1 (I never update between rc and final releases). I'm intrigued to see why it breaks libraries though - typically unsupported enum values like this are silently ignored on older

[issue32394] socket lib beahavior change in 3.6.4

TJG added the comment: It's a compile-time option in socketmodule.c. https://github.com/python/cpython/blob/3.6/Modules/socketmodule.c#L7466 The MSDN page suggests that it was added for Win10:

[issue32394] socket lib beahavior change in 3.6.4

Steve Dower added the comment: Which libraries break? And which version of Windows are you running on? If TCP_KEEPCNT was only added recently (perhaps in an SDK update) then it may not really be available on all versions. (+Ned for awareness of the regression - not

[issue32394] socket lib beahavior change in 3.6.4

Change by Antoine Pitrou : -- nosy: +paul.moore, steve.dower, tim.golden, zach.ware ___ Python tracker ___

[issue32394] socket lib beahavior change in 3.6.4

Srinivas Reddy T added the comment: dtdev@dtdev-centos $ python3 Python 3.6.3 (default, Oct 11 2017, 18:17:01) [GCC 4.4.7 20120313 (Red Hat 4.4.7-18)] on linux Type "help", "copyright", "credits" or "license" for more information. >>> import socket >>>

[issue32394] socket lib beahavior change in 3.6.4

New submission from Kamil : On Windows, python 3.6.3 code "hasattr(socket, 'TCP_KEEPCNT')" gives False, python 3.6.4 gives True. This behavior breaks many libraries that i use. -- components: Library (Lib) messages: 308837 nosy: skn78 priority: normal severity: normal