[issue25924] investigate if getaddrinfo(3) on OSX is thread-safe

A. Jesse Jiryu Davis added the comment: Here's a retelling of this bug report as a silly fantasy saga: https://engineering.mongodb.com/post/the-saga-of-concurrent-dns-in-python-and-the-defeat-of-the-wicked-mutex-troll/ -- ___ Python tracker

[issue25924] investigate if getaddrinfo(3) on OSX is thread-safe

A. Jesse Jiryu Davis added the comment: Related to #26406, a fix for NetBSD and OpenBSD. -- ___ Python tracker ___

[issue25924] investigate if getaddrinfo(3) on OSX is thread-safe

Ned Deily added the comment: I've committed a revised version of the "try 2" patch for release in 2.7.12, 3.5.2, and 3.6.0. The revisions were to better follow the somewhat arcane conventions of Apple's availability macros, in particular, to take into account the value of

[issue25924] investigate if getaddrinfo(3) on OSX is thread-safe

Roundup Robot added the comment: New changeset 58ebfa7c1361 by Ned Deily in branch '2.7': Issue #25924: Avoid unnecessary serialization of getaddrinfo(3) calls on OS X https://hg.python.org/cpython/rev/58ebfa7c1361 New changeset 86ddb4d747f8 by Ned Deily in branch '3.5': Issue #25924: Avoid

[issue25924] investigate if getaddrinfo(3) on OSX is thread-safe

A. Jesse Jiryu Davis added the comment: I think we have consensus for "try 2". I'm not a core dev, would one of you please merge 25924-getaddrinfo-is-thread-safe-2.patch? Thanks! -- ___ Python tracker

[issue25924] investigate if getaddrinfo(3) on OSX is thread-safe

A. Jesse Jiryu Davis added the comment: Martin, here's a third patch, "try 3", which does a runtime version check instead of compile-time. It's a bit complex, compared to "try 2", which did a compile-time check instead. I do NOT think this extra complexity is worth it to support Pythons that

[issue25924] investigate if getaddrinfo(3) on OSX is thread-safe

Ned Deily added the comment: I agree that the approach in "try 2" is fine and the runtime check in "try 3" is overkill. While it is possible to do so, we've never really supported building on an OS X release n for release m, where m < n, without using the m SDK on n (and in particular for

[issue25924] investigate if getaddrinfo(3) on OSX is thread-safe

Martin Panter added the comment: I am happy to defer to Ned who probably knows a lot more than me about the OS X situation. FTR it was the code added in 2006 by r45660 (Issue 1471925) that motivated my concern. Since that was added by Ronald, it would be good to hear his opinion :)

[issue25924] investigate if getaddrinfo(3) on OSX is thread-safe

A. Jesse Jiryu Davis added the comment: Thanks Martin. The MAC_OS_X_VERSION_10_5 macro ensures that Python is still compatible with Mac OS before version 10.5: if it's built on 10.4 or older, it locks around getaddrinfo (since it's not thread-safe there), and on 10.5 and later it doesn't

[issue25924] investigate if getaddrinfo(3) on OSX is thread-safe

Yury Selivanov added the comment: Hi Jesse, could you please update your patch with a detailed comment summarizing your research? Also it would be great if you can provide a separate patch for 2.7. -- nosy: +yselivanov ___ Python tracker

[issue25924] investigate if getaddrinfo(3) on OSX is thread-safe

A. Jesse Jiryu Davis added the comment: Second patch with a comment about how we know Mac OS 10.5+'s getaddrinfo is thread-safe. I'll wait for this to be merged before submitting another for Python 2.7. -- Added file:

[issue25924] investigate if getaddrinfo(3) on OSX is thread-safe

A. Jesse Jiryu Davis added the comment: Great, how do we ensure this gets merged soon? -- ___ Python tracker ___

[issue25924] investigate if getaddrinfo(3) on OSX is thread-safe

Guido van Rossum added the comment: Find an active developer who cares -- e.g. Martin Panter, Serhiy, Yury. On Tue, Feb 2, 2016 at 5:46 AM, A. Jesse Jiryu Davis wrote: > > A. Jesse Jiryu Davis added the comment: > > Great, how do we ensure this gets merged soon? > >

[issue25924] investigate if getaddrinfo(3) on OSX is thread-safe

Martin Panter added the comment: Hello Jesse. Maybe Yury might be more qualified here than me to handle OS X stuff (correct me if I’m wrong). I mainly just work with Linux. But reading around socketmodule.c, it sounds like we want to be able to build it on newer OS versions, but still have it

[issue25924] investigate if getaddrinfo(3) on OSX is thread-safe

A. Jesse Jiryu Davis added the comment: I've created a Mac OS 10.4 virtual machine and reproduced the getaddrinfo concurrency bug there using the attached h_resolv.c. The man page on that OS version indeed includes the "not thread-safe" warning. The same test passes on my Mac OS 10.10 system.

[issue25924] investigate if getaddrinfo(3) on OSX is thread-safe

Guido van Rossum added the comment: Thanks for the thorough work! Hopefully we can apply this fix to 3.5, 3.6 and even 2.7. -- nosy: +gvanrossum ___ Python tracker

[issue25924] investigate if getaddrinfo(3) on OSX is thread-safe

A. Jesse Jiryu Davis added the comment: An Apple Developer Relations engineer tells me it's "reasonable to assume that getaddrinfo() is thread safe" on OS X 10.5 and later. (He mentioned that iOS inherited the OS X 10.5 DNS architecture, so Apple phones, watches, TVs, hairdryers, etc. can all

[issue25924] investigate if getaddrinfo(3) on OSX is thread-safe

A. Jesse Jiryu Davis added the comment: Related to #1288833, when FreeBSD 5.3's getaddrinfo was declared thread-safe. -- ___ Python tracker ___

[issue25924] investigate if getaddrinfo(3) on OSX is thread-safe

Changes by A. Jesse Jiryu Davis : Added file: http://bugs.python.org/file41597/d_mach ___ Python tracker ___

[issue25924] investigate if getaddrinfo(3) on OSX is thread-safe

Changes by A. Jesse Jiryu Davis : Added file: http://bugs.python.org/file41596/h_resolv.c ___ Python tracker ___

[issue25924] investigate if getaddrinfo(3) on OSX is thread-safe

A. Jesse Jiryu Davis added the comment: NetBSD has a concurrency test for getaddrinfo, so I tried it on Mac to see if getaddrinfo is thread-safe here, too. It appears that it is thread-safe. I copied NetBSD's getaddrinfo concurrency test "h_resolv.c" and the test's data file "d_mach" from

[issue25924] investigate if getaddrinfo(3) on OSX is thread-safe

A. Jesse Jiryu Davis added the comment: In Apple's Libinfo version 222.4.12 (corresponding to the last OS X 10.4 release), the man page says getaddrinfo isn't thread-safe: http://www.opensource.apple.com/source/Libinfo/Libinfo-222.4.12/lookup.subproj/getaddrinfo.3 And here's its source:

[issue25924] investigate if getaddrinfo(3) on OSX is thread-safe

New submission from Ronald Oussoren: https://emptysqua.re/blog/getaddrinfo-deadlock/ claims that getaddrinfo may deadlock when using threads and fork on (amongst others) OSX due to using a global lock. That lock is used when getaddrinfo is believed to be not thread safe, see the relevant