[issue35969] Interpreter crashes with "can't initialize init_sys_streams" when abc fails to import

[STINNER Victor]

STINNER Victor  added the comment:

> I agree with @p-ganssle, If exist some problem on initialize Python 
> interpreter it would be great if it return a non-zero status and avoid an 
> abort()

Since Python 3.8 (PEP 587), Python no longer call abort() on initialization 
error.

$ echo bug > abc.py

$ PYTHONPATH=$PWD python3.8 -c pass
Fatal Python error: init_sys_streams: can't initialize sys standard streams
Python runtime state: core initialized
Traceback (most recent call last):
  File "/usr/lib64/python3.8/io.py", line 52, in 
  File "/home/vstinner/abc.py", line 1, in 
NameError: name 'bug' is not defined

I don't think that it's worth it to change Python 3.7.

The initial issue was that abort() was called in this case. It's no longer the 
case, so I close the issue.

If someone cares abou the more general issue with the current directory being 
added to sys.path, please open a separated issue (or look for existing issues).

--
components: +Interpreter Core
resolution:  -> fixed
stage:  -> resolved
status: open -> closed
versions: +Python 3.9 -Python 3.5, Python 3.6, Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35969] Interpreter crashes with "can't initialize init_sys_streams" when abc fails to import

[Emmanuel Arias]

Emmanuel Arias  added the comment:

> Otherwise I'd say it would be nice to refactor in such a way that avoids the 
> core dump when the problem is on the Python side 

I agree with @p-ganssle, If exist some problem on initialize Python interpreter 
it would be great if it return a non-zero status and avoid an abort()

+ Python3.5. I test it and has the same result.

--
versions: +Python 3.5

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35969] Interpreter crashes with "can't initialize init_sys_streams" when abc fails to import

[Guido van Rossum]

Change by Guido van Rossum :


--
nosy:  -gvanrossum

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35969] Interpreter crashes with "can't initialize init_sys_streams" when abc fails to import

[Paul Ganssle]

Paul Ganssle  added the comment:

> @Guido, I recall a while back you explained the value of adding CWD to 
> sys.path.  Would you mind providing some insight here

I think bpo-35971 or bpo-13475 are better places to discuss that, because this 
issue doesn't require CWD to be in the path. You can trigger this by any 
mechanism that would raise an exception in the Python code used to initialize 
the interpreter.

Even if we restrict it to the most common case - shadowing a part of the 
standard library - you would still be able to trigger this as long as you have 
*some* mechanism to manipulate the python path in the environment that allows 
you to put paths on there with higher precedence than the standard library.

I think the core question in this issue is whether errors in the Python code 
run during the interpreter initialization should crash the interpreter or exit 
gracefully with an error code.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35969] Interpreter crashes with "can't initialize init_sys_streams" when abc fails to import

[Eric Snow]

Eric Snow  added the comment:

@Guido, I recall a while back you explained the value of adding CWD to 
sys.path.  Would you mind providing some insight here?

--
nosy: +gvanrossum

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35969] Interpreter crashes with "can't initialize init_sys_streams" when abc fails to import

[Eric Snow]

Eric Snow  added the comment:

related: issue #13475

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35969] Interpreter crashes with "can't initialize init_sys_streams" when abc fails to import

[Eric Snow]

Eric Snow  added the comment:

On Tue, Feb 12, 2019 at 3:57 AM STINNER Victor  wrote:
> A long term solution would to enable -I behavior by default: don't add the 
> current directory to sys.path when using -c CMD.

>From what I can recall, it was a conscious decision to include CWD in 
>sys.path.  Doing so benefits some scripting use cases.  The downside is the 
>effect when running an app.

--
nosy: +eric.snow

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35969] Interpreter crashes with "can't initialize init_sys_streams" when abc fails to import

[STINNER Victor]

STINNER Victor  added the comment:

See also bpo-35971: "Documentation should warn about code injection from 
current working directory" which discuss changing the default behavior.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35969] Interpreter crashes with "can't initialize init_sys_streams" when abc fails to import

[Paul Ganssle]

Paul Ganssle  added the comment:

> One enhancement of this new API is that it now reports the name of the C 
> function which causes the error. The initial bug report says "Fatal Python 
> error: init_sys_streams: can't initialize sys standard 
streams": init_sys_streams() function raised the fatal error.

I think that's a welcome change, the main question is whether it's necessary to 
hard-crash the interpreter and dump the core, or if, in most cases, the error 
message + non-zero return code would be sufficient.

It's not clear to me how useful the traceback itself is, but of course I'm 
giving a toy example where the Python exception is sufficient to debug what's 
failing. I would think that *in general* if the interpreter is failing to 
initialize because of a problem in the Python stack, printing the Python 
traceback should be sufficient. If there's good reason to think otherwise, then 
I'm fine with closing this bug. Otherwise I'd say it would be nice to refactor 
in such a way that avoids the core dump when the problem is on the Python side 
(leaving in place the error message you added that shows where it failed on the 
C side).

> A workaround is the usage of -I option:

In this case I don't think we need a workaround, because the issue is really 
"should it be possible for a pure python crash to cause the interpreter to 
crash rather than exit with an error code?" Shadowing a standard library module 
from your PYTHONPATH is just the easiest way to manifest this. (Though probably 
more people should be using -I, so it's good to bring it up).


> A long term solution would to enable -I behavior by default: don't add the 
> current directory to sys.path when using -c CMD.

If you've been following the recent issues with the PEP 517 rollout in 
setuptools, you'll see that I'm deeply sympathetic to this view (though in this 
case I'm not amazingly optimistic about its feasibility). Probably we shouldn't 
derail this issue too much with a discussion of our revolutionary views on the 
right default Python path, though.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35969] Interpreter crashes with "can't initialize init_sys_streams" when abc fails to import

[STINNER Victor]

STINNER Victor  added the comment:

A workaround is the usage of -I option:

vstinner@apu$ PYTHONPATH=: python3 -c ""
Fatal Python error: init_sys_streams: can't initialize sys standard streams
Traceback (most recent call last):
  File "/usr/lib64/python3.7/io.py", line 52, in 
  File "/tmp/demo/abc.py", line 1, in 
Exception: Hi
Aborted (core dumped)

vstinner@apu$ PYTHONPATH=: python3 -I -c ""


A long term solution would to enable -I behavior by default: don't add the 
current directory to sys.path when using -c CMD.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35969] Interpreter crashes with "can't initialize init_sys_streams" when abc fails to import

[STINNER Victor]

STINNER Victor  added the comment:

Last year, I reworked Python initialization to introduce a new _PyInitError 
structure which allows to report a failure to the caller of a function instead 
of calling directly Py_FatalError() which always call abort() immediately.

_PyInitError allows to distinguish "user error" and "internal error". User 
error is caused by an user mistake, whereas internal errors are all other 
errors. User errors don't call abort() to avoid dumping a core dump.

It's not always easy to decide if a bug is caused by the user or not.

For example, a memory allocation failure is now considered as an "user error":

#define _Py_INIT_NO_MEMORY() _Py_INIT_USER_ERR("memory allocation failed")

See Include/coreconfig.h for the definition of _PyInitError and related macros.

One enhancement of this new API is that it now reports the name of the C 
function which causes the error. The initial bug report says "Fatal Python 
error: init_sys_streams: can't initialize sys standard 
streams": init_sys_streams() function raised the fatal error.

--
nosy: +vstinner

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35969] Interpreter crashes with "can't initialize init_sys_streams" when abc fails to import

[Emmanuel Arias]

Emmanuel Arias  added the comment:

Hi,

The reason of the aborted(core dump) of the interpreter is because when the 
init process was wrong, the 

```_Py_NO_RETURN fatal_error()```

call and abort()

So, I change it for a exit(status) to avoid break the interpreter (I attach a 
patch)

What do you think about it? If all it's ok I can prepare the PR

--
keywords: +patch
nosy: +eamanu
Added file: 
https://bugs.python.org/file48134/0001-avoid-abort-when-initialize-python-fail.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35969] Interpreter crashes with "can't initialize init_sys_streams" when abc fails to import

[Karthikeyan Singaravelan]

Karthikeyan Singaravelan  added the comment:

Sorry by previous reports I was talking about some of the issues where the 
installers were not correct though I couldn't find the issue at the moment. I 
just presented it as a data point where some necessary modules that cannot be 
imported could lead to crash. But if this can be dealt with in a better way I 
think it will be a good improvement.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35969] Interpreter crashes with "can't initialize init_sys_streams" when abc fails to import

[Paul Ganssle]

Paul Ganssle  added the comment:

@Karthikeyan I would certainly consider this a duplicate of the encodings 
bug/behavior that you demonstrate. I don't see an existing bug on the tracker 
for it, though.

I think it's pretty clear that the interpreter needs to fail, but it seems to 
*crash* rather than exit gracefully with a non-zero return code, which I think 
is the main problem. I'll note that it *does* print some stuff out before it 
crashes, which makes me think that even though io and/or encodings hasn't 
loaded, you still are able to do the "print an error message" part of "print an 
error message and exit with non-zero error code".

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35969] Interpreter crashes with "can't initialize init_sys_streams" when abc fails to import

[Karthikeyan Singaravelan]

Karthikeyan Singaravelan  added the comment:

I hope there are certain modules that CPython interpreter expects to load 
properly. There were some cases in the past where encodings module caused the 
interpreter to crash. A similar scenario.

➜  cpython git:(master) echo "raise Exception('a')" > encodings.py
➜  cpython git:(master) ✗ PYTHONPATH=: ./python.exe
Fatal Python error: initfsencoding: failed to get the Python codec of the 
filesystem encoding
Traceback (most recent call last):
  File "/Users/karthikeyansingaravelan/stuff/python/cpython/encodings.py", line 
1, in 
Exception: a
[1]29005 abort  PYTHONPATH=: ./python.exe

--
nosy: +xtreak

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35969] Interpreter crashes with "can't initialize init_sys_streams" when abc fails to import

[Paul Ganssle]

Paul Ganssle  added the comment:

Tested with 3.6 and 2.7 - core dump on 3.6, no core dump on 2.7. The crash on 
2.7 goes through a different path, it goes site.py > os.py >= UserDict.py > 
_abcoll.py > abc.py. That may account for why it's not crashing the interpreter 
itself.

--
versions: +Python 3.6 -Python 2.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35969] Interpreter crashes with "can't initialize init_sys_streams" when abc fails to import

[Paul Ganssle]

New submission from Paul Ganssle :

Just noticed this (tested on Python 3.7 and 3.8):

mkdir /tmp/demo
cd /tmp/demo
cat << EOF > abc.py
raise Exception("Hi")
EOF
PYTHONPATH=: python -c ""


This will crash the interpreter with:

Fatal Python error: init_sys_streams: can't initialize sys standard 
streams
Traceback (most recent call last):
  File ".../lib/python3.7/io.py", line 52, in 
  File "/tmp/demo/abc.py", line 1, in 
Exception: Hi
Aborted (core dumped)


It seems that the problem is that the io module imports the abc module, which 
raises an exception, so io fails to load. Evidently the io module is necessary 
to load the interpreter, so the interpreter crashes.

Here's the backtrace for 3.7.2 on Arch Linux:

(gdb) bt
#0  0x7f234b3f0d7f in raise () from /usr/lib/libc.so.6
#1  0x7f234b3db672 in abort () from /usr/lib/libc.so.6
#2  0x7f234b7db490 in fatal_error (prefix=prefix@entry=0x7f234b9d5fe0 
<__func__.16645> "init_sys_streams", 
msg=msg@entry=0x7f234ba01f60 "can't initialize sys standard streams", 
status=status@entry=-1)
at Python/pylifecycle.c:2179
#3  0x7f234b8460cb in _Py_FatalInitError (err=...) at 
Python/pylifecycle.c:2198
#4  0x7f234b8495a9 in pymain_init (pymain=0x7fff971cca70) at 
Modules/main.c:3019
#5  0x555dfa560050 in ?? ()
#6  0x7fff971ccbc0 in ?? ()
#7  0x in ?? ()


I'm not sure if anything can or should be done about this. It's very fair for 
the interpreter to fail to start, though I would guess that it should do that 
without dumping a core.

--
messages: 335244
nosy: p-ganssle
priority: normal
severity: normal
status: open
title: Interpreter crashes with "can't initialize init_sys_streams" when abc 
fails to import
type: crash
versions: Python 2.7, Python 3.7, Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com