[issue36502] The behavior of str.isspace() for U+00A0 and U+202F is different from what is documented
New submission from Jun : I was looking for a list of Unicode codepoints that str.isspace() returns true. According to https://docs.python.org/3/library/stdtypes.html#str.isspace, it's "Whitespace characters are those characters defined in the Unicode character database as “Other” or “Separator” and those with bidirectional property being one of “WS”, “B”, or “S”." However, for U+202F(https://www.fileformat.info/info/unicode/char/202f/index.htm) which is a "Separator" and its bidirectional property is "CS", str.isspace() returns True while it shouldn't if we follow the definition above. >>> "\u202f".isspace() True I'm not sure either the documentation should be updated or behavior should be updated, but at least those should be consistent. -- assignee: docs@python components: Documentation, Unicode messages: 339317 nosy: Jun, docs@python, ezio.melotti, vstinner priority: normal severity: normal status: open title: The behavior of str.isspace() for U+00A0 and U+202F is different from what is documented type: behavior versions: Python 2.7, Python 3.5, Python 3.6 ___ Python tracker <https://bugs.python.org/issue36502> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue44155] Race condition when using multiprocessing BaseManager and Pool in Python3
Change by Jun : -- keywords: +patch nosy: +junnplus nosy_count: 1.0 -> 2.0 pull_requests: +24924 stage: -> patch review pull_request: https://github.com/python/cpython/pull/26332 ___ Python tracker <https://bugs.python.org/issue44155> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43207] InspectLoader.is_package is not an abstract method
New submission from Jun : In the documentation of importlib: https://docs.python.org/3/library/importlib.html#importlib.abc.InspectLoader It says InspectLoader.is_package is an abstract method while it is not in the implementation. @abc.abstractmethod decorator was removed with this commit https://github.com/python/cpython/commit/b523f8433a8982e10eb41a3e2b37ee0e6d6a6e00 -- assignee: docs@python components: Documentation messages: 386866 nosy: Jun, docs@python priority: normal severity: normal status: open title: InspectLoader.is_package is not an abstract method versions: Python 3.10 ___ Python tracker <https://bugs.python.org/issue43207> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue25634] Add a dedicated subclass for attribute missing errors
New submission from Jun Wang: See this simple example: class A(): def __init__(self, x=None): self.x = x @property def t(self): return self.x.t def __getattr__(self, name): return 'default' print(A().t) AttributeError is raised as "'NoneType' object has no attribute 't'". Currently __getattr__ is called if any AttributeError is raised, so the result of a.t is *default*, while an AttributeError is the desired behavior. The most intuitive solution seems to add a subclass of AttributeError, say AttributeMissError, which triggers __getattr__. At present, I have to do some tricky and ugly things to __getattribute__ to show where the AttributeError occurs, or it's quite hard to figure out what happened with no informative traceback messages. -- components: Interpreter Core messages: 254722 nosy: 王珺 priority: normal severity: normal status: open title: Add a dedicated subclass for attribute missing errors type: enhancement versions: Python 3.6 ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue25634> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue25634] Add a dedicated subclass for attribute missing errors
Jun Wang added the comment:
I think this is a common problem while using both __getattr__ and
descriptor/property. A descriptor example:
class Descriptor():
def __get__(self, instance, owner=None):
raise AttributeError('Implicitly suppressed')
class A():
d = Descriptor()
def __getattr__(self, name):
return 'default'
print(A().d)
Without descriptor, unexpected AttributeError could only come from overriding
__getattribute__, which is a rare case, although still an imperfection. But in
descriptor/property, AttributeError which is too general just occurs frequently
like in normal method.
Surely any modification would break the backward compatibility, although I
wonder how often it is used of raising AttributeError purposely, maybe in
__getattribute__, to call __getattr__, instead of explicitly calling
__getattr__. In my understanding this is the only case that will be affected.
"An unexpected exception should not result in subtly altered behaviour, but
should cause a noisy and easily-debugged traceback. "—from PEP479
About the implementation, maybe something like "RuntimeError: descriptor raised
AttributeError" simulating PEP479. Or in my lay opinion, the best solution is:
add object.__getattr__, with the only behavior of raising AttributeError; when
normal attribute lookup fails, object.__getattribute__ calls __getattr__
explicitly; __getattr__ not triggered by AttributeError anymore.
I know little about the CPython implementation, so I might be completely wrong.
However this seems deserving more detailed discussion.
--
___
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/issue25634>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1633941] for line in sys.stdin: doesn't notice EOF the first time
Jun Wu added the comment: haypo: The file.__iter__ EINTR issue may be better discussed as a separate bug report. It's not related to stdin or EOF or Windows. Since we have some EINTR fixes for Python 2.7.4, I think it's reasonable to fix the remaining EINTR issues for 2.7.13+. If I have read fileobject.c correctly, readahead() is the only remaining place having the EINTR issue. If you agree that we should fix readahead(), I can prepare the patch. -- nosy: +quark ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue1633941> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue34509] Starting to use gcc-8 on CI
Jun Aruga added the comment: > it would be better to set up a buildbot running GCC 8, or switching one of > our existing workers to it. We want to keep the pre-merge CI as stable as > possible, and using a bleeding-edge compiler doesn't strike me as > particularly stable. Oh it seems that we have already had the CI environment as buildbot. I did not know that. Yeah it might be better to add it as a buildbot. https://buildbot.python.org/all/#/ -- ___ Python tracker <https://bugs.python.org/issue34509> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue34509] Starting to use gcc-8 on CI
Change by Jun Aruga : -- keywords: +patch pull_requests: +8418 stage: -> patch review ___ Python tracker <https://bugs.python.org/issue34509> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue34509] Starting to use gcc-8 on CI
New submission from Jun Aruga : This ticket is from below conversation on python-dev mailing list. https://mail.python.org/pipermail/python-dev/2018-August/155011.html Right now the Python project Travis CI is running on gcc-4.8.4 for coverage testing. I want to replace it or add the latest version gcc-8 case. The benefit to use gcc-8 is, * It is important to run on the old version 4.8 but, it is also meaningful to follow the latest version and care issues such as warnings on the latest version gcc. * I am working in Fedora project that is using gcc version 8 as a main c compiler. If the Python project care gcc-8, that might be helpful for people who maintain Python Debian/Fedora package or Installer on Mac/Windows. The demerit is * Right now the gcc-4.8 is used for the coverage. The total running time on gcc-8 might be longer than current one. * To run the latest gcc-8, we can add the repository [1] with the way [2][3], but I am not sure whether the repository is stable. I would show you 2 cases to use gcc-8 on Linux. [1] https://launchpad.net/~ubuntu-toolchain-r/+archive/ubuntu/test [2] https://docs.travis-ci.com/user/installing-dependencies/#installing-packages-from-a-custom-apt-repository [3] https://docs.travis-ci.com/user/languages/c/#gcc-on-os-x [4] Ruby https://travis-ci.org/junaruga/ruby/builds/418242410 https://github.com/junaruga/ruby/blob/feature/ci-new-gcc/.travis.yml https://github.com/ruby/ruby/pull/1937 [5] Trinity Example for gcc-8 on both Linux and Mac. https://travis-ci.org/trinityrnaseq/trinityrnaseq https://github.com/trinityrnaseq/trinityrnaseq/blob/master/.travis.yml -- messages: 324125 nosy: junaruga priority: normal severity: normal status: open title: Starting to use gcc-8 on CI type: enhancement versions: Python 3.8 ___ Python tracker <https://bugs.python.org/issue34509> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue34509] Starting to use gcc-8 on CI
Jun Aruga added the comment: Sure, alright. Thanks for informing me. On Thu, 16 May 2019 at 00:08, Zachary Ware wrote: > > > Zachary Ware added the comment: > > We do now have at least one builder using GCC 8 (ware-gentoo-x86) and one > using GCC 9 (cstratak-fedora), so I'm closing the issue. Thanks for bringing > this up, and sorry it fell through the cracks for a while! > > -- > resolution: -> fixed > stage: patch review -> resolved > status: open -> closed > > ___ > Python tracker > <https://bugs.python.org/issue34509> > ___ -- ___ Python tracker <https://bugs.python.org/issue34509> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue44687] io.BufferedReader:peek() closes underlying file, breaking peek/read expectations
Change by Jun De : -- keywords: +patch nosy: +AngstyDuck nosy_count: 1.0 -> 2.0 pull_requests: +26859 stage: -> patch review pull_request: https://github.com/python/cpython/pull/28457 ___ Python tracker <https://bugs.python.org/issue44687> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue44687] io.BufferedReader:peek() closes underlying file, breaking peek/read expectations
Jun De added the comment: Yeap Steve, I was thinking the same thing. I've taken the buffered data into consideration when checking if the file is closed in my fix, feel free to take a look and let me know what you think :) -- ___ Python tracker <https://bugs.python.org/issue44687> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue44687] io.BufferedReader:peek() closes underlying file, breaking peek/read expectations
Jun De added the comment: Awesome! To be honest I was very inclined to add test cases for this edge case, but while some existing test cases use native objects like io.BytesIO, the one that our edge case uses is HTTPResponse from `urllib3.response.py`. Should I still go ahead with writing the test cases in this case? I worry about creating new dependencies for the python repository. -- ___ Python tracker <https://bugs.python.org/issue44687> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue31865] html.unescape does not work as per documentation
New submission from Jun Hui Lee <cardinle...@gmail.com>:
html.unescape(s)
Convert all named and numeric character references (e.g. , , )
But running this gives:
>>> html.unescape(', , ')
'>, >, '
--
assignee: docs@python
components: Documentation
messages: 304957
nosy: Jun Hui Lee, docs@python
priority: normal
severity: normal
status: open
title: html.unescape does not work as per documentation
type: behavior
versions: Python 3.6
___
Python tracker <rep...@bugs.python.org>
<https://bugs.python.org/issue31865>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue36260] Cpython/Lib vulnerability found and request a patch submission
JUN-WEI SONG added the comment: Hello Python community, With Christian Heimes’ suggestion, we manipulate appropriate warning to inform users that they may encounter zip bomb issues when using the zipfile module. The warning we would like to add in the zipfile documentation is shown below : https://github.com/python/cpython/blob/3.7/Doc/library/zipfile.rst .. warning:: Never extract files from untrusted sources without prior inspection. It is possible that the file may contain zip bomb issues such as 42.zip. The zip bomb will usually be a small file before decompression, but once it is decompressed, it will exhaust system resources. You can protect your system by limiting system resources, limiting compression ratio (zip bombs are usually quite high), and checking for nested zip files. We are also pleasure to provide a patch to enhance the zipfile module to provide basic information. In zipfile.py https://github.com/python/cpython/blob/master/Lib/zipfile.py Inside the ZipFile class : def filecount(self): """Return total count of files in the archive.""" return len(self.filelist) def total_compressed_size(self): """Return total compressed size in the archive.""" return sum([data.compress_size for data in self.filelist]) def total_uncompressed_size(self): """Return total uncompressed size in the archive.""" return sum([data.file_size for data in self.filelist]) -- resolution: -> remind status: closed -> open ___ Python tracker <https://bugs.python.org/issue36260> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue36462] CVE-2019-9674 : zip bomb vulnerability in Lib/zipfile.py
New submission from JUN-WEI SONG :
Dear Python Community,
we found a python module vulnerability during these days and we got a CVE
number, CVE-2019-9674 after reported it to cve.mitre.org.
https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2019-9674
The reserved information of CVE-2019-9674 is shown below:
[Description]
Lib/zipfile.py in Python through 3.7.2 allows remote
attackers to cause a denial of service (resource consumption)
via a ZIP bomb.
[Additional Information]
The python zipfile library version 3.2, 3.3, 3.4, 3.5, 3.6,
3.7, 3.8. Allow attackers to cause a denial of service (disk
volume exhaustion) via a ZIP bomb.
We have found python standard library zipfile doesn't have
ZIP bomb detection and protection. If the user uses zipfile
library to unzip a ZIP bomb file, this might cause a denial
of service of the localhost.
[VulnerabilityType Other]
Denial-of-Service
Our proposed solutions:
1.The compression ratio:
Compression ratio = Uncompressed file size / Compressed file size
Since ZIP bomb file has a higher compression ratio (1028) than
normal ZIP file (1 to 3). Therefore, we calculate the compression
ratio and set a threshold for the detection.
2.Nested zip file
There is a high chance that it is zip bomb if it is a nested zip
file.
3.By limiting resources such as CPU, memory, disk usage.
Unsolved issue
However, we have not yet determined the compression ratio. We
temporarily set the compression ratio to 10, and if it exceeds, it
may be a ZIP bomb.
It is likely that detection may misjudge nested compressed files.
For example, under normal circumstances, compressed files are
included in the zip file.
Our solution code：
"""For ratio"""
def _exam_ratio(self, threshold=10):
"""If the ratio exceeds threshold, it may be a ZIP Bomb."""
sum_file_size = sum([data.file_size for data in self.filelist])
sum_compress_size = sum([data.compress_size for data in self.filelist])
ratio = sum_file_size / sum_compress_size
if (ratio > threshold):
raise BadZipFile("Zip Bomb Detected")
"""For Nested zip file"""
if(members.filename.endswith(".zip")):
raise BadZipFile("Nested Zip File Detected")
Thanks!
--
components: Library (Lib)
messages: 339053
nosy: krnick
priority: normal
severity: normal
status: open
title: CVE-2019-9674 : zip bomb vulnerability in Lib/zipfile.py
type: security
versions: Python 3.5, Python 3.6, Python 3.7, Python 3.8
___
Python tracker
<https://bugs.python.org/issue36462>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue36462] CVE-2019-9674 : zip bomb vulnerability in Lib/zipfile.py
JUN-WEI SONG added the comment: Thanks to the python community, both of these issues are the same. I also think it's a good thing to make related documentation to reduce this type of problem rather than implementing it on a low-level zipfile module. Perhaps we can customize such a requirement through a pip package. -- ___ Python tracker <https://bugs.python.org/issue36462> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue36260] Cpython/Lib vulnerability found and request a patch submission
JUN-WEI SONG added the comment: Thank you python community, these two issues are indeed the same problem. I also think that it is good to make a related document to reduce such problems. -- stage: -> resolved status: -> closed ___ Python tracker <https://bugs.python.org/issue36260> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue36260] Zip Bomb vulnerability
Change by JUN-WEI SONG : -- keywords: +patch pull_requests: +13288 stage: resolved -> patch review ___ Python tracker <https://bugs.python.org/issue36260> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue36260] Zip Bomb vulnerability
JUN-WEI SONG added the comment: Dear friends, We moved a little bit forward to improve the writing. :) -- ___ Python tracker <https://bugs.python.org/issue36260> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue36988] zipfile: string IndexError on extract
JUN-WEI SONG added the comment: The following output throws error when using unzip -t $ unzip -t file0.zip Output: Archive: file0.zip : mismatching "local" filename (zipfile_extract.pyUT^I), continuing with "central" filename version testing: error: invalid compressed data to inflate At least one error was detected in file0.zip. It looks like the zip file is corrupted. Maybe we could add some detection mechanisms before extract it like unzip, for example, unsupported characters or file corrupted check. -- nosy: +krnick ___ Python tracker <https://bugs.python.org/issue36988> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue36260] Cpython/Lib vulnerability found and request a patch submission
JUN-WEI SONG added the comment: Thank you very much for your reply. Based on discussions above, consensuses are improving the zipfile documentation. And we (JUN-WEI SONG & KunYu Chen) would like to work on this. With opinions of Serhiy Storchaka, Christian Heimes and the ideas we have, possible pitfalls are listed below. 1. From file itself: Decompression may fail due to an incorrect password, an incorrect CRC checksum, an incorrect PKZIP format, an unsupported compression method, or an unsupported decryption. 2. File system: Each file system has different limitations such as allowable characters in directory entries, the max length of file name, the max length of path name, the max size of single file, the max number of files, the max number of files in a single directory, etc. Decompression will fail as long as these limitations are exceeded. 3. Operating system: The lack of memory or disk space would lead to decompression failed (see also Zip Bomb). 4. Interrupt: Users should be careful in interrupting the process of decompression, such as control-C or killing the process during decompression, which may result in incomplete decompression of the archive. 5. Different default behaviors: Users should figure out different default extraction behaviors, such as when extracting into the existing tree, it will overwriting an existing file without asking, or when in a case-insensitive file system, it keeps only one file when extracting an archive which contains many files that have the same name but different case. Please let us know if anything’s missing. -- ___ Python tracker <https://bugs.python.org/issue36260> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue38507] Improve the documentation of the nested list initialization
New submission from JUN-WEI SONG : When I used the nested list, I need to initialize the nested list, so I used this expression: >>> nested_list = [[]] * 5 see also: https://stackoverflow.com/questions/12791501/python-initializing-a-list-of-lists So I later learned that such an expression would make the list inside the list have the same reference, which would cause the problem that you modified one element would lead to all elements changed in the nested list. For example: >>> nested_list[0].append(1) >>> nested_list [[1], [1], [1], [1], [1]] Therefore, maybe we could tell users how to initialize the list on the documentation like below: If you need to initialize the nested list, you could follow the below example, also, be aware of the expression like ``[[]] * 5``, this will cause the five lists in the nested list to have the same reference. >>> nested_list = [[] for _ in range(5)] -- assignee: docs@python components: Documentation messages: 354844 nosy: docs@python, krnick priority: normal severity: normal status: open title: Improve the documentation of the nested list initialization type: enhancement versions: Python 3.8 ___ Python tracker <https://bugs.python.org/issue38507> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue38507] Improve the documentation of the nested list initialization
JUN-WEI SONG added the comment: sorry that I did not notice it already documented -- resolution: -> duplicate stage: -> resolved status: open -> closed ___ Python tracker <https://bugs.python.org/issue38507> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
