[issue34087] int(s), float(s) and others may cause segmentation fault

Change by INADA Naoki : -- resolution: -> fixed stage: patch review -> resolved status: open -> closed versions: +Python 3.8 ___ Python tracker ___

[issue34087] int(s), float(s) and others may cause segmentation fault

miss-islington added the comment: New changeset c721472fb83d1f7c7606bcf33ba2d42d6127a764 by Miss Islington (bot) in branch '3.7': bpo-34087: Fix buffer overflow in int(s) and similar functions (GH-8274) https://github.com/python/cpython/commit/c721472fb83d1f7c7606bcf33ba2d42d6127a764

[issue33955] Implement PyOS_CheckStack on macOS using pthread_get_stack*_np

Dong-hee Na added the comment: FYI, I've updated the patch to use rlim_cur() on the main thread when pthread_get_stacksize_np() is not accurate. This way works pretty well on my local machine. Enjoy your travel! -- ___ Python tracker

[issue34087] int(s), float(s) and others may cause segmentation fault

Change by miss-islington : -- pull_requests: +7814 ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue34087] int(s), float(s) and others may cause segmentation fault

INADA Naoki added the comment: New changeset 16dfca4d829e45f36e71bf43f83226659ce49315 by INADA Naoki in branch 'master': bpo-34087: Fix buffer overflow in int(s) and similar functions (GH-8274) https://github.com/python/cpython/commit/16dfca4d829e45f36e71bf43f83226659ce49315 --

[issue34087] int(s), float(s) and others may cause segmentation fault

Change by INADA Naoki : -- pull_requests: +7815 ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue34097] ZIP does not support timestamps before 1980

Change by Marcel Plch : -- keywords: +patch pull_requests: +7805 stage: -> patch review ___ Python tracker ___ ___

[issue34087] django: segmentation fault on random places

Jonas H. added the comment: Here's a Docker image that reproduces the bug. FROM ubuntu:18.04 RUN apt update && apt install -y python3.7-dbg python3.7-venv python3-venv wget RUN python3.7 -m venv venv RUN venv/bin/pip install django RUN wget https://bugs.python.org/file47688/testproj.tar.gz -O

[issue34097] ZIP does not support timestamps before 1980

Serhiy Storchaka added the comment: There are ZIP extensions which allow to save timestamps before 1980 and with better than 2-seconds resolution. It would be better to use this feature. But first we need to resolve issue17681. -- nosy: +serhiy.storchaka

[issue34108] 2to3 munges new lines on Windows

New submission from Jason R. Coombs : In issue11594, we attempted to solve the newlines issue but inadvertently introduced extraneous CR before CRLF newlines when running on Windows. See https://github.com/python/cpython/pull/6483#issuecomment-401957049 for more details. --

[issue34108] 2to3 munges new lines on Windows

Change by Jason R. Coombs : -- keywords: +patch pull_requests: +7806 stage: commit review -> patch review ___ Python tracker ___

[issue34097] ZIP does not support timestamps before 1980

Serhiy Storchaka added the comment: You can't, because Python 2.7 doesn't support it. But you will be able to pack files in a ZIP archive and extract them without a loss on 3.8 and with a loss of a timestamp on 2.7. And you will be able to use a third-party utilities for extracting files

[issue4260] Document that ctypes.xFUNCTYPE are decorators.

Berker Peksag added the comment: New changeset 379e9d639a52766f79c7a206c5096c8333d1896f by Berker Peksag (Andrés Delfino) in branch 'master': bpo-4260: Document that ctypes.xFUNCTYPE are decorators (GH-7924) https://github.com/python/cpython/commit/379e9d639a52766f79c7a206c5096c8333d1896f

[issue34087] float(unicode) may cause segmentation fault

INADA Naoki added the comment: Very thanks! I found I set `export PYTHONMALLOC=pymalloc_debug`. That's why I can't reproduce it. Now I can reproduce and I get where memory is broken, and understood what cause: Python traceback is: Traceback (most recent call first): File

[issue34087] django: segmentation fault on random places

INADA Naoki added the comment: @jonash What Python binary do you use? * OS (distro and version) * Installed from package? if so, complete package name and version. * Installed from source? if so, step you used to install Python 3.7 (e.g. pyenv install 3.7) --

[issue34097] ZIP does not support timestamps before 1980

Marcel Plch added the comment: I have created a PR for this: https://github.com/python/cpython/pull/8270 -- ___ Python tracker ___

[issue17681] Work with an extra field of gzip and zip files

Change by Serhiy Storchaka : -- versions: +Python 3.8 -Python 3.4 ___ Python tracker ___ ___ Python-bugs-list mailing list

[issue34097] ZIP does not support timestamps before 1980

Petr Viktorin added the comment: I'm not sure the extensions will solve this problem fully. If an implementation that doesn't support these extensions, how does it handle them? For example, if Python 3.8 implements the extensions, I use py3.8 to create a zipfile containing old timestamps,

[issue4260] Document that ctypes.xFUNCTYPE are decorators.

Change by miss-islington : -- pull_requests: +7807 ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue4260] Document that ctypes.xFUNCTYPE are decorators.

Change by miss-islington : -- pull_requests: +7808 ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue34087] django: segmentation fault on random places

Rodrigo Pinheiro Marques de Araújo added the comment: I can reproduce the segmentation fault using 'testproj.tar.gz' with homebrew and compiled from source. MacOS X High Sierra 10.13.5 (17F77). -- ___ Python tracker

[issue34087] django: segmentation fault on garbage collection in visit_decref()

INADA Naoki added the comment: I received two core files ("core" and "core.23") from fenrrir, thank you. But I can't investigate well, sorry. I think someone breaks memory, and SEGV happened later. I need to reproduce it on my machine... In core.23, SEGV happens in kill() syscall. I can't

[issue33859] Spelling mistakes found using aspell

Karthikeyan Singaravelan added the comment: Closing this since the related PRs are merged. -- resolution: -> fixed stage: patch review -> resolved status: open -> closed ___ Python tracker

[issue34087] django: segmentation fault on random places

Jonas H. added the comment: Reduced it to something that seems unicode related? No extension modules involved. Vanilla Django project with a single url + template. See testproj/urls.py and tmpl/index.html -- Added file: https://bugs.python.org/file47688/testproj.tar.gz

[issue34087] django: segmentation fault on random places

Change by INADA Naoki : -- title: django: segmentation fault on garbage collection in visit_decref() -> django: segmentation fault on random places ___ Python tracker ___

[issue34096] [2.7] test_audioop.test_max() failed: AssertionError: -2147483648 != 2147483648L

STINNER Victor added the comment: I installed Fedora 28 in 32-bit (GCC 8.1.1) in a VM to try to reproduce the bug, but I failed to reproduce it. Erich Eckner: IMHO your issue comes from a C flag. Try to loop on: touch Modules/audioop.c && make && ./python -m test test_audioop And remove

[issue34087] float(unicode) may cause segmentation fault

STINNER Victor added the comment: It might be a duplicate of bpo-33954: bug in _PyUnicode_InsertThousandsGrouping(). -- ___ Python tracker ___

[issue34096] [2.7] test_audioop.test_max() failed: AssertionError: -2147483648 != 2147483648L

Erich Eckner added the comment: yes, indeed, I just tested with your three flags-variables set and the build+test succeeded - I'll have to figure out what the minimal change is, though :-) -- ___ Python tracker

[issue34097] ZIP does not support timestamps before 1980

Petr Viktorin added the comment: Which third-party utilities support these? As I said, AFAIK `zip` on my system does not. I assume the loss of data is the reason we have an error now -- if that wasn't a concern, zipfile could just silently bump the timestamp to 1980. When the extensions

[issue34087] int(s), float(s) and others may cause segmentation fault

Change by INADA Naoki : -- title: float(unicode) may cause segmentation fault -> int(s), float(s) and others may cause segmentation fault ___ Python tracker ___

[issue31979] Simplify converting non-ASCII strings to int, float and complex

Change by INADA Naoki : -- pull_requests: +7810 ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue34087] int(s), float(s) and others may cause segmentation fault

Change by INADA Naoki : -- keywords: +patch pull_requests: +7809 stage: -> patch review ___ Python tracker ___ ___

[issue32046] 2to3 fix for operator.isCallable()

Dong-hee Na added the comment: @serhiy.storchaka If we don't have plans with backporting patches than we can close this issue. -- ___ Python tracker ___

[issue21446] Update reload fixer to use importlib instead of imp

Berker Peksag added the comment: Brett, what do you think about this issue since Python 3.4 is pretty old now. Should I convert my patch to GitHub PR or close this issue as 'wontfix'? -- type: -> enhancement versions: +Python 3.8 -Python 3.5 ___

[issue4260] Document that ctypes.xFUNCTYPE are decorators.

miss-islington added the comment: New changeset 08c1da71030e0d6019b3493e0ffaf060a6aa5d8e by Miss Islington (bot) in branch '3.7': bpo-4260: Document that ctypes.xFUNCTYPE are decorators (GH-7924) https://github.com/python/cpython/commit/08c1da71030e0d6019b3493e0ffaf060a6aa5d8e --

[issue34087] django: segmentation fault on random places

STINNER Victor added the comment: If someone expects progress on these issues, we need to get a repository that we can clone to reproduce the bug. Then the code must be simplified as much as possible to reduce the code base. The best is when you still trigger the crash with less than 1000

[issue34087] django: segmentation fault on random places

INADA Naoki added the comment: @jonash Thanks! Would you give me concrete step to run the project and reproduce the segfault? -- ___ Python tracker ___

[issue34087] django: segmentation fault on random places

Jonas H. added the comment: Sure. Unpack archive, create new 3.7 venv with Django (latest version is fine), ./manage.py runserver, curl localhost:8000. -- ___ Python tracker

[issue34096] [2.7] test_audioop.test_max() failed: AssertionError: -2147483648 != 2147483648L

STINNER Victor added the comment: Extract of attached test.pythoninfo output: sys.maxsize: 2147483647 sysconfig[PY_CFLAGS]: -fno-strict-aliasing -march=i686 -mtune=generic -O2 -pipe -fstack-protector-strong -fno-plt -DNDEBUG -march=i686 -mtune=generic -O2 -pipe -fstack-protector-strong

[issue34087] django: segmentation fault on random places

INADA Naoki added the comment: I tried it but I can't reproduce... -- ___ Python tracker ___ ___ Python-bugs-list mailing list

[issue34107] root.warning('msg') output format modified by logging.warning('msg')

Change by Karthikeyan Singaravelan : -- nosy: +xtreak ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue34107] root.warning('msg') output format modified by logging.warning('msg')

Vinay Sajip added the comment: This behaviour is as expected. If no handlers are configured for logging, an internal "handler of last resort" is used, with just the message output. See: https://docs.python.org/3/howto/logging.html#what-happens-if-no-configuration-is-provided If you call

[issue34096] [2.7] test_audioop.test_max() failed: AssertionError: -2147483648 != 2147483648L

Erich Eckner added the comment: yes, I'm using gcc (GCC) 8.1.1 20180531 I attached the created pythoninfo -- Added file: https://bugs.python.org/file47687/pythoninfo ___ Python tracker

[issue34091] REPL does not work in msys2, ConEmu terminals on windows

Segev Finer added the comment: I'd assume this is using ConEmu's ChildGui which means it hacks up a MinTTY window to appear inside ConEmu's GUI. MinTTY doesn't use a real terminal but rather pipes as the processes stdio. Any native (non-MSYS2) Windows program would have a problem with this.

[issue34093] Reproducible pyc: FLAG_REF is not stable.

Christian Tismer added the comment: Why must this become slower? To my knowledge, many projects prefer marshal over pickle for suitable simple objects because it is so very fast. I would not throw that away: Would it not be easy to add a named optional keyword argument, like "stable=True"?

[issue4260] Document that ctypes.xFUNCTYPE are decorators.

miss-islington added the comment: New changeset 083a836937b734fcc81b2482d2c52825caff0115 by Miss Islington (bot) in branch '3.6': bpo-4260: Document that ctypes.xFUNCTYPE are decorators (GH-7924) https://github.com/python/cpython/commit/083a836937b734fcc81b2482d2c52825caff0115 --

[issue34087] int(s), float(s) and others may cause segmentation fault

Serhiy Storchaka added the comment: Is it possible to write a reproducer in Python? The string returned by _PyUnicode_TransformDecimalAndSpaceToASCII() is still a NUL-terminated, but the NUL can be far past the end of the string. -- nosy: +serhiy.storchaka

[issue34108] 2to3 munges new lines on Windows

Jason R. Coombs added the comment: The regression never got to 3.6, so this issue likely only affects 3.7+. -- versions: -Python 3.6 ___ Python tracker ___

[issue34087] int(s), float(s) and others may cause segmentation fault

Jonas H. added the comment: I don't think this can be tested with Python code, unless you can make sure the target buffer _PyUnicode_TransformDecimalAndSpaceToASCII operates on is initialised with garbage bytes. -- ___ Python tracker

[issue4260] Document that ctypes.xFUNCTYPE are decorators.

Berker Peksag added the comment: I think we can now close this one. Thanks for the report, David, and thanks for the PR Andreas. (And thanks for creating and maintaining this awesome module for years, Thomas :)) -- resolution: -> fixed stage: patch review -> resolved status: open

[issue34087] int(s), float(s) and others may cause segmentation fault

STINNER Victor added the comment: > I don't think this can be tested with Python code, unless you can make sure > the target buffer _PyUnicode_TransformDecimalAndSpaceToASCII operates on is > initialised with garbage bytes. We just have to call float() with a non-ASCII string, no? An

[issue34087] int(s), float(s) and others may cause segmentation fault

Jonas H. added the comment: The assertion in the patched code, yes. The segfault in the unpatched code, no. -- ___ Python tracker ___

[issue32046] 2to3 fix for operator.isCallable()

Change by Guido van Rossum : -- nosy: -gvanrossum ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue34108] 2to3 munges new lines on Windows

Jason R. Coombs added the comment: New changeset cafaf0447b950fd4f59edd8cbde040c61ae528f8 by Jason R. Coombs in branch 'master': bpo-34108: Fix double carriage return in 2to3 on Windows (#8271) https://github.com/python/cpython/commit/cafaf0447b950fd4f59edd8cbde040c61ae528f8 --

[issue34108] 2to3 munges new lines on Windows

Change by miss-islington : -- pull_requests: +7811 ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue34075] asyncio: We should prohibit setting a ProcessPoolExecutor in with set_default_executor

Andrew Svetlov added the comment: Agree, restricting to ThreadPoolExecutor sounds reasonable. A custom executor may have the same problem as ProcessPoolExecutor. Moreover it can work under same scenarios but crash with other third-party libs. --

[issue34087] int(s), float(s) and others may cause segmentation fault

INADA Naoki added the comment: When I commented out the fix: $ ./python -c 'int("こんにちは")' python: Objects/unicodeobject.c:484: _PyUnicode_CheckConsistency: Assertion `PyUnicode_READ(kind, data, ascii->length) == 0' failed. Aborted I added this to test cases. --

[issue34087] int(s), float(s) and others may cause segmentation fault

Change by Ezio Melotti : -- nosy: +ezio.melotti ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue34093] Reproducible pyc: FLAG_REF is not stable.

INADA Naoki added the comment: > Would it not be easy to add a named optional keyword > argument, like "stable=True"? My pull request did it. But for now, I get hint on ML and overwrote my PR with another way: Use FLAG_REF for all interned strings. --

[issue34109] Accumulator bug

New submission from Victor Pires : A function to return a number from 1 to 5 (inclusive) sometimes returns -1 when called thousands of times. from random import randint import sys def rand5(): """Returns a random integer from 1 to 5 (inclusive)""" r5 = -5 # This *should* accumulate

[issue34109] Accumulator bug

Victor Pires added the comment: Sorry, you are right -- resolution: -> not a bug stage: -> resolved status: open -> closed ___ Python tracker ___

[issue27741] datetime.datetime.strptime functionality description incorrect

Karthikeyan Singaravelan added the comment: The same part repeated at https://docs.python.org/3/library/datetime.html#strftime-and-strptime-behavior can also be corrected. It seems both of them use _strptime which returns microseconds but only datetime.datetime.strptime uses it with

[issue34110] cPickle may raise AttributeError when loading concurrently in threads

New submission from Guoqiang Zhang : If two threads use cPickle to load objects simultaneously, one thread may raise an AttributeError. This problem is caused by the partially loaded module. To reproduce, create a file 'foo.py': import time time.sleep(0.1) class foo(): pass Then in

[issue34108] 2to3 munges new lines on Windows

Jason R. Coombs added the comment: New changeset cf21d0031dd84544d4108765553c2b03dfe726c5 by Jason R. Coombs (Miss Islington (bot)) in branch '3.7': bpo-34108: Fix double carriage return in 2to3 on Windows (GH-8271) (#8275)

[issue34108] 2to3 munges new lines on Windows

Change by Jason R. Coombs : -- resolution: -> fixed stage: patch review -> resolved status: open -> closed ___ Python tracker ___

[issue34066] Possible resource warning in "with open()"

Change by Jakub Wilk : -- nosy: +jwilk ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue34109] Accumulator bug

Tim Peters added the comment: ? I expect your code to return -1 about once per 7**4 = 2401 times, which would be about 400 times per million tries, which is what your output shows. If you start with -5, and randint(1, 7) returns 1 four times in a row, r5 is left at -5 + 4 = -1. --

[issue34095] [2.7] test_idle fails with: /usr/bin/xvfb-run: line 181: 3617 Segmentation fault

Terry J. Reedy added the comment: I think we should first determine whether xvfb is required to get the segfault, and second, where it fails. Does -m test.regrtest -v -m test_io test_idle fail and fail in the same place, the 11th and last test run in the module? If not, what about

[issue34110] cPickle may raise AttributeError when loading concurrently in threads

Change by Guoqiang Zhang : -- keywords: +patch pull_requests: +7812 stage: -> patch review ___ Python tracker ___ ___

[issue21446] Update reload fixer to use importlib instead of imp

Brett Cannon added the comment: I would still fix it as I suspect there will be a burst of users of this come 2020 and at this point it won't be a compatibility issue going forward since 3.4 is so old. :) -- ___ Python tracker

[issue34063] binhex REASONABLY_LARGE = 32768 so what is 128000

Karthikeyan Singaravelan added the comment: I tried changing it to REASONABLY_LARGE and could see no test failures. It seems the literal was added as part of a220e67a9ed94d66b81e393a3bb9e6acd10068c1 (23/03/1996) and REASONABLY_LARGE was before that as added part of

[issue34109] Accumulator bug

Change by Victor Pires : Removed file: https://bugs.python.org/file47689/Bug.html ___ Python tracker ___ ___ Python-bugs-list mailing list

[issue34037] asyncio: BaseEventLoop.close() shutdowns the executor without waiting causing leak of dangling threads

Change by Guido van Rossum : -- nosy: -gvanrossum ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue33073] Add as_integer_ratio() to int() objects

Change by Guido van Rossum : -- nosy: -gvanrossum ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue33073] Add as_integer_ratio() to int() objects

Guido van Rossum added the comment: I assume it's decided what to do -- it may not be easy to do it, but I wouldn't call it controversial at this point. -- ___ Python tracker

[issue32430] Simplify Modules/Setup{,.dist,.local}

Change by Eric V. Smith : -- nosy: +eric.smith ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue34102] None.splitlines raises AttributeError in email.feedparser

Terry J. Reedy added the comment: What you can do is report to pip and ansible-lint that there is a bug somewhere in one of the two packages, or try to figure out which and report it to one of them. -- ___ Python tracker

[issue33361] readline() + seek() on codecs.EncodedFile breaks next readline()

Diego Argueta added the comment: Bug still present in 3.7.0, now seeing it in 3.8.0a0 as well. -- versions: +Python 3.8 ___ Python tracker ___

[issue32584] Uninitialized free_extra in code_dealloc

Brett Cannon added the comment: co_extra_freefuncs is an array of pointers, so there's no way for it to be uninitialized unless you didn't initialize the interpreter state (https://github.com/python/cpython/blob/b193fa996a746111252156f11fb14c12fd6267e6/Include/pystate.h#L155). And looking

[issue30400] Race condition in shutil.copyfile(): source file replaced file during copy

Preston Moore added the comment: I like Victor's idea for updating public functions to support file descriptors. I could submit a patch for this instead if desired. In the meantime, I've updated the pull request for this issue so the patch I originally created that compares inode numbers

[issue33361] readline() + seek() on codecs.EncodedFile breaks next readline()

Change by Ammar Askar : -- keywords: +patch pull_requests: +7813 stage: -> patch review ___ Python tracker ___ ___

[issue34107] root.warning('msg') output format modified by logging.warning('msg')

Michael Kearney added the comment: Thanks for the clarification and doc pointers. I saw "lastResort" attribute in the code but had not fully understood what was going on. I am not certain I really do now, but that's ok. That's why we read docs, read and write code. Eventually it starts

[issue34102] local variable 'parts' referenced before assignment in feedparser in email module

Terry J. Reedy added the comment: An exception is not a crash. An intended exception is not even a bug. A bug is a discrepancy between behavior and doc. In this case, email.parser.Feedparser is imported from email.feedparser. Its doc

[issue34102] None.splitlines raises AttributeError in email.feedparser

Change by Terry J. Reedy : -- title: local variable 'parts' referenced before assignment in feedparser in email module -> None.splitlines raises AttributeError in email.feedparser ___ Python tracker

[issue34110] cPickle may raise AttributeError when loading concurrently in threads

Pablo Galindo Salgado added the comment: This behaviour also happens on the current master. -- nosy: +pablogsal ___ Python tracker ___

[issue34110] cPickle may raise AttributeError when loading concurrently in threads

Change by Pablo Galindo Salgado : -- versions: +Python 3.6, Python 3.7, Python 3.8 ___ Python tracker ___ ___ Python-bugs-list

[issue34100] Same constants in tuples are not merged while compile()

Terry J. Reedy added the comment: I think (space) 'performance' would be a better label, as this is strictly an implementation improvement, not a language change. But we often (usually? sometimes?) limit performance improvements to the 'next version' so we have the alpha/beta/candidate