[issue13829] exception error in _scproxy.so when called after fork

[Ronald Oussoren]

Ronald Oussoren  added the comment:

I propose closing this as 3th-party or out-of-date:

1) The crash in _scproxy is due to limitations in Apple's libraries, in 
particular they don't work in child processes created with os.fork() without 
calling execv*()

2) The primary way to run into this is by the use of the multiprocessing 
library. The default spawn strategy for multiprocessing was changes to "spawn" 
instead of "fork" in 3.8 (for macOS) because of problems like this.

--
resolution:  -> out of date
stage: needs patch -> resolved
status: open -> pending

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13829] exception error in _scproxy.so when called after fork

[ThePokestarFan]

ThePokestarFan  added the comment:

Still present in python 3.8 and issue 38658. Another workaround is in 
https://bugs.python.org/issue30385#msg293958

--
nosy: +ThePokestarFan
versions: +Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13829] exception error in _scproxy.so when called after fork

[triccare triccare]

triccare triccare  added the comment:

I have run into this bug which can be reliably reproduced by importing tkinter.

However, I have found another workaround if one does not want to deal with the 
environmental variable and may point to a possible bug fix. Before forking, any 
use of `urlopen` alleviates the bug. This is demonstrated in the attached 
example.

Example works under Python 3.6.4 and Mac OS 10.12.6

--
nosy: +triccare triccare
Added file: https://bugs.python.org/file47503/debug_urlopen.py

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13829] exception error in _scproxy.so when called after fork

[Eitan Adler]

Change by Eitan Adler :


--
nosy: +eadler

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13829] exception error in _scproxy.so when called after fork

[Ned Deily]

Changes by Ned Deily :


--
priority: low -> normal
stage:  -> needs patch
versions: +Python 3.5, Python 3.6, Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13829] exception error in _scproxy.so when called after fork

[Evan Jones]

Evan Jones added the comment:

This is effectively the same as issue 27126. The discussion there convinced me 
that this may not be working around, since it isn't the only place this can 
happen! This same crash, caused by libdispatch not being fork compatible, also 
happens with sqlite and tk. We can probably add a workaround specifically for 
_scproxy.so, but those others would still cause the crash.

--
nosy: +evanj

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13829] exception error in _scproxy.so when called after fork

[R. David Murray]

R. David Murray added the comment:

Why is it out of date?  Has it been fixed?  Has apple changed their abort 
policy?

Obviously it isn't high priority, though :)

--
assignee: ronaldoussoren -> 
nosy: +r.david.murray
priority: normal -> low
resolution: out of date -> 
status: closed -> open
title: exception error in _scproxy.so -> exception error in _scproxy.so when 
called after fork

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13829] exception error in _scproxy.so

[Carol Willing]

Carol Willing added the comment:

Closed as an out of date issue.

--
nosy: +willingc
resolution:  -> out of date
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13829] exception error in _scproxy.so

[Ronald Oussoren]

Ronald Oussoren added the comment:

Sigh... The crash is a subprocess started with fork is not entirely unexpected, 
although I had hoped that the current version of _scproxy would be safe enough.

A number of Apple frameworks detect that a process called fork after they 
initialized and then explictly crash (basicly a call to abort()). In this case 
it appears to just not work correctly.

I'm not sure how to fix this, other than spawning a small helper program to 
query the proxy settings. That could get prohibitively expensive when fetching 
a lot of URLs though (and an RPC system might run into the a similar problem 
after calling fork...)

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13829
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13829] exception error in _scproxy.so

[Julian Scheid]

Julian Scheid added the comment:

FWIW, I've run into the same issue in a homegrown application with 2.6.8, 2.7.2 
and 2.7.3 (these were the only versions I've tested).

Looking around a little bit, I suspect this might be a bug in 
SCDynamicStoreCopyProxies that's only present on OS X 10.7 and only triggered 
when invoked in a forked child (?) [1][2].

I've tried working around it by invoking SCDynamicStoreCopyProxies with a 
non-NULL SCDynamicStoreRef but to no avail. Since I don't need this Python 
application to use HTTP proxies, I'm working around it now like this:

proxy_handler = urllib2.ProxyHandler({})
opener = urllib2.build_opener(proxy_handler)
request = urllib2.Request(...)
response = opener.open(request)

[1] http://forums.macrumors.com/archive/index.php/t-1295113.html
[2] https://github.com/suminb/spider/issues/7

--
nosy: +Julian.Scheid

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13829
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13829] exception error in _scproxy.so

[Ronald Oussoren]

Ronald Oussoren added the comment:

I've once again reviewed the _scproxy code and that code seems correct 
(although that doesn't say too much for subtle bugs because I wrote the initial 
version of the module).

Dan: is it possible to tell moviegrabber to use another python installation (in 
particular /Library/Frameworks/Python.framework)? If so, is the problem 
reproducable with the latest binaries on www.python.org?

The crash report says that the actual crash occurs inside a function called by 
SCDynamicStoreCopyProxies and I wouldn't know how Python's use of that function 
is wrong. 

The crash could still be caused by the way the moviegrabber application uses 
python, but I'd consider that a bug in moviegrabber unless there is a clear 
indication of a bug in python itself.

BTW. Is Moviegrabber this software? : 
http://sourceforge.net/projects/moviegrabber/

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13829
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13829] exception error in _scproxy.so

[Ronald Oussoren]

Ronald Oussoren added the comment:

If it is the moviegrabber I linked to: that's a 100% python script using some 
other opensource libraries.

It does use multiprocessing, and that might mean this is the same problem as 
issue9405. That issue should be fixed in the repository (for a long time, the 
issue is not yet closed because I wanted to write a test case).

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13829
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13829] exception error in _scproxy.so

[Brett Cannon]

Changes by Brett Cannon br...@python.org:


--
nosy:  -brett.cannon

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13829
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13829] exception error in _scproxy.so

[Ned Deily]

Ned Deily n...@acm.org added the comment:

The problem was originally discussed in these threads:

http://mail.python.org/pipermail/pythonmac-sig/2011-December/023428.html
http://mail.python.org/pipermail/pythonmac-sig/2011-December/023430.html

Because _scproxy is used to get information about Internet proxies, a test case 
should be constructed on OS X 10.7.x with a proxy configured.

--
assignee:  - ronaldoussoren
components: +Macintosh
nosy: +ned.deily, ronaldoussoren
resolution: invalid - 
status: closed - open
title: exception error - exception error in _scproxy.so

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13829
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13829] exception error

[Dan kamp]

Dan kamp bitbuc...@roontoon.com added the comment:

I have received this from the macpython listserv it that helps. Would really 
like to find this issue.

From the traceback, it appears that there is a problem with Python's
_scproxy module; that's an internal helper C module that provides an
interface to the OS X System Configuration framework to access Internet
proxy configurations for the urllib module. You should open an issue
for this at bugs.python.org. Please include the original crash report
traceback.

On Jan 21, 2012, at 12:03 PM, Brett Cannon wrote:

 
 Brett Cannon br...@python.org added the comment:
 
 Then I'm going to assume the bug lies with Moviegrabber doing something wrong 
 and it isn't Python's direct fault.
 
 --
 resolution:  - invalid
 status: open - closed
 
 ___
 Python tracker rep...@bugs.python.org
 http://bugs.python.org/issue13829
 ___

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13829
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13829] exception error

[Brett Cannon]

Brett Cannon br...@python.org added the comment:

Then I'm going to assume the bug lies with Moviegrabber doing something wrong 
and it isn't Python's direct fault.

--
resolution:  - invalid
status: open - closed

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13829
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13829] exception error

[Brett Cannon]

Brett Cannon br...@python.org added the comment:

Can you isolate the cause? There is way too much in that core dump to try to 
debug the problem. Without knowing what code in this Moviegrabber app caused 
the bug we can't do anything to debug the issue.

--
nosy: +brett.cannon
status: open - pending

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13829
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13829] exception error

[Dan kamp]

Dan kamp bitbuc...@roontoon.com added the comment:

On Jan 20, 2012, at 4:20 PM, Brett Cannon wrote:

 Can you isolate the cause? There is way too much in that core dump to try to 
 debug the problem. Without knowing what code in this Moviegrabber app caused 
 the bug we can't do anything to debug the issue.

All I know is that this happens with Moviegrabber and no other app that I am 
running (5 of them). Attached is the code I am using. This is WAY above what I 
know how to do but I can tell you that I am running it on a mac with 2.7.1 
installed and OSX lion. Attached is the source. Thanks

--
status: pending - open

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13829
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13829] exception error

[Dan kamp]

New submission from Dan kamp roont...@gmail.com:

Get this error when trying to run Moviegrabber on a mac running v2.7. Crash 
report below.

Process: Python [2444]
Path:
/System/Library/Frameworks/Python.framework/Versions/2.7/Resources/Python.app/C 
ontents/MacOS/Python
Identifier:  Python
Version: ??? (???)
Code Type:   X86-64 (Native)
Parent Process:  Python [2431]
 
 
Date/Time:   2011-12-21 20:32:41.233 -0500
OS Version:  Mac OS X 10.7.2 (11C74)
Report Version:  9
 
 
Interval Since Last Report:  357638 sec
Crashes Since Last Report:   17
Per-App Crashes Since Last Report:   405
Anonymous UUID:  3C387DB7-0AA3-4F34-A100-31D1736D2668
 
 
Crashed Thread:  0
 
 
Exception Type:  EXC_BAD_ACCESS (SIGSEGV)
Exception Codes: KERN_INVALID_ADDRESS at 0x0108
 
 
VM Regions Near 0x108:
--
__TEXT 00010c5c5000-00010c5c6000 [4K] r-x/rwx 
SM=COW  
/System/Library/Frameworks/Python.framework/Versions/2.7/Resources/Python.app/C 
ontents/MacOS/Python
 
 
Application Specific Information:
objc[2429]: garbage collection is OFF
 
 
Thread 0 Crashed:
0   libdispatch.dylib   0x7fff8b14ace9 _dispatch_wakeup 
+ 108
1   libdispatch.dylib   0x7fff8b14d876 
_dispatch_resume_slow + 20
2   com.apple.CoreFoundation0x7fff8af8262c 
-[NSXPCConnection start] + 92
3   com.apple.CoreFoundation0x7fff8af84ca3 
__CFXNotificationCenterSetupConnection + 387
4   com.apple.CoreFoundation0x7fff8af84b11 
__CFXNotificationCenterCreate + 273
5   com.apple.CoreFoundation0x7fff8af849ea 
__CFNotificationCenterGetDistributedCenter_block_invoke_1 + 26
6   libdispatch.dylib   0x7fff8b14d224 dispatch_once_f 
+ 53
7   com.apple.CoreFoundation0x7fff8af73e0a 
CFNotificationCenterGetDistributedCenter + 74
8   com.apple.CoreFoundation0x7fff8afb2eb8 
CFXPreferencesGetSourceForTriplet_block_invoke_1 + 40
9   libdispatch.dylib   0x7fff8b14d224 dispatch_once_f 
+ 53
10  com.apple.CoreFoundation0x7fff8af7fa4a 
__CFXPreferencesGetSourceForTriplet + 58
11  com.apple.CoreFoundation0x7fff8af8be67 
__CFXPreferencesGetSearchListForBundleID + 215
12  com.apple.CoreFoundation0x7fff8af8bd48 
___CFXPreferencesCopyAppValue_block_invoke_1 + 24
13  com.apple.CoreFoundation0x7fff8af8bcea 
CFPreferencesCopyAppValue + 218
14  com.apple.SystemConfiguration   0x7fff928f388c 
SCDynamicStoreCopyProxies + 43
15  _scproxy.so 0x00010ceae9ba 0x10ceae000 + 
2490
16  org.python.python   0x00010c651b58 
PyEval_EvalFrameEx + 13318
17  org.python.python   0x00010c654df7 0x10c5ca000 + 
568823
18  org.python.python   0x00010c651e0a 
PyEval_EvalFrameEx + 14008
19  org.python.python   0x00010c654df7 0x10c5ca000 + 
568823
20  org.python.python   0x00010c651e0a 
PyEval_EvalFrameEx + 14008
21  org.python.python   0x00010c654cd8 
PyEval_EvalCodeEx + 1996
22  org.python.python   0x00010c5f2abf 0x10c5ca000 + 
166591
23  org.python.python   0x00010c5d1d32 PyObject_Call + 
97
24  org.python.python   0x00010c5e06e9 0x10c5ca000 + 
91881
25  org.python.python   0x00010c5d1d32 PyObject_Call + 
97
26  org.python.python   0x00010c64dc40 
PyEval_CallObjectWithKeywords + 180
27  org.python.python   0x00010c5dd489 PyInstance_New + 
273
28  org.python.python   0x00010c5d1d32 PyObject_Call + 
97
29  org.python.python   0x00010c651f63 
PyEval_EvalFrameEx + 14353
30  org.python.python   0x00010c654cd8 
PyEval_EvalCodeEx + 1996
31  org.python.python   0x00010c654e6c 0x10c5ca000 + 
568940
32  org.python.python   0x00010c651e0a 
PyEval_EvalFrameEx + 14008
33  org.python.python   0x00010c654cd8 
PyEval_EvalCodeEx + 1996
34  org.python.python   0x00010c654e6c 0x10c5ca000 + 
568940
35  org.python.python   0x00010c651e0a 
PyEval_EvalFrameEx + 14008
36  org.python.python   0x00010c654cd8 
PyEval_EvalCodeEx + 1996
37  org.python.python   0x00010c5f2abf 0x10c5ca000 + 
166591
38  org.python.python   0x00010c5d1d32 PyObject_Call + 
97
39  org.python.python   0x00010c6525ec 
PyEval_EvalFrameEx + 16026
40  org.python.python   0x00010c654df7 0x10c5ca000 + 
568823
41  org.python.python