[issue43650] MemoryError on zip.read in shutil._unpack_zipfile

[Gregory P. Smith]

Gregory P. Smith  added the comment:

thanks for the patch!

--
resolution: remind -> fixed
stage: patch review -> commit review
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43650] MemoryError on zip.read in shutil._unpack_zipfile

[Gregory P. Smith]

Gregory P. Smith  added the comment:


New changeset 7a588621c2854bcef6ce9eeb54349b84ac078c45 by Miss Islington (bot) 
in branch '3.10':
bpo-43650: Fix MemoryError on zip.read in shutil._unpack_zipfile for large 
files (GH-25058) (GH-26190)
https://github.com/python/cpython/commit/7a588621c2854bcef6ce9eeb54349b84ac078c45


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43650] MemoryError on zip.read in shutil._unpack_zipfile

[miss-islington]

miss-islington  added the comment:


New changeset 049c4125f8a2b482c6129db68463f58e20c31526 by Miss Islington (bot) 
in branch '3.9':
bpo-43650: Fix MemoryError on zip.read in shutil._unpack_zipfile for large 
files (GH-25058)
https://github.com/python/cpython/commit/049c4125f8a2b482c6129db68463f58e20c31526


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43650] MemoryError on zip.read in shutil._unpack_zipfile

[miss-islington]

Change by miss-islington :


--
pull_requests: +24808
pull_request: https://github.com/python/cpython/pull/26191

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43650] MemoryError on zip.read in shutil._unpack_zipfile

[miss-islington]

Change by miss-islington :


--
nosy: +miss-islington
nosy_count: 4.0 -> 5.0
pull_requests: +24807
pull_request: https://github.com/python/cpython/pull/26190

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43650] MemoryError on zip.read in shutil._unpack_zipfile

[Gregory P. Smith]

Change by Gregory P. Smith :


--
assignee:  -> gregory.p.smith
nosy: +gregory.p.smith
type: crash -> resource usage
versions: +Python 3.10, Python 3.11 -Python 3.6, Python 3.7, Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43650] MemoryError on zip.read in shutil._unpack_zipfile

[Ma Lin]

Change by Ma Lin :


--
nosy: +malin

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43650] MemoryError on zip.read in shutil._unpack_zipfile

[Igor Bolshakov]

Igor Bolshakov  added the comment:

pls review

--
resolution:  -> remind

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43650] MemoryError on zip.read in shutil._unpack_zipfile

[Roundup Robot]

Change by Roundup Robot :


--
keywords: +patch
nosy: +python-dev
nosy_count: 1.0 -> 2.0
pull_requests: +23805
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/25058

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43650] MemoryError on zip.read in shutil._unpack_zipfile

[igor voltaic]

New submission from igor voltaic :

MemoryError: null
  ...
  File "", line 13, in repack__file
shutil.unpack_archive(local_file_path, local_dir)
  File "python3.6/shutil.py", line 983, in unpack_archive
func(filename, extract_dir, **kwargs)
  File "python3.6/shutil.py", line 901, in _unpack_zipfile
data = zip.read(info.filename)
  File "python3.6/zipfile.py", line 1338, in read
return fp.read()
  File "python3.6/zipfile.py", line 858, in read
buf += self._read1(self.MAX_N)
  File "python3.6/zipfile.py", line 948, in _read1
data = self._decompressor.decompress(data, n)

shutil.unpack_archive tries to read the whole file into memory, making use of 
any buffer at all. Python crashes for really large files. In my case — archive: 
~1.7G, unpacked: ~10G. Interestingly zipfile.ZipFile.extractall handles this 
case more effective.

--
components: Library (Lib)
messages: 389652
nosy: igorvoltaic
priority: normal
severity: normal
status: open
title: MemoryError on zip.read in shutil._unpack_zipfile
type: crash
versions: Python 3.6, Python 3.7, Python 3.8, Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com