[issue23649] tarfile not re-entrant for multi-threading

Change by STINNER Victor : -- nosy: -vstinner ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.p

[issue23649] tarfile not re-entrant for multi-threading

Change by Jeffrey Kintscher : -- nosy: +Jeffrey.Kintscher ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: htt

[issue23649] tarfile not re-entrant for multi-threading

Xavier de Gaye added the comment: extract_from_pkgs() in the attached extract_from_packages.py script extracts /etc files from the tar files in PKG_DIR into WORK_DIR using a ThreadPoolExecutor (a ThreadPoolExecutor, when used to extract all the /etc files from the packages that build a whole

[issue23649] tarfile not re-entrant for multi-threading

STINNER Victor added the comment: Lars Gustäbel added the comment: > I agree with David that there is no need for tarfile to be thread-safe. There > is nothing to be gained from distributing one TarFile object among multiple > threads because it operates on a single resource which has to be acc

[issue23649] tarfile not re-entrant for multi-threading

Lars Gustäbel added the comment: I agree with David that there is no need for tarfile to be thread-safe. There is nothing to be gained from distributing one TarFile object among multiple threads because it operates on a single resource which has to be accessed sequentially anyway. So, it seems

[issue23649] tarfile not re-entrant for multi-threading

Srdjan Grubor added the comment: The whole lib still needs the threading locks added but the patch submitted should fix things for people that do the locking from their code. -- ___ Python tracker

[issue23649] tarfile not re-entrant for multi-threading

Srdjan Grubor added the comment: Patch for the multithreaded expansion of files and use of makedirs. -- keywords: +patch Added file: http://bugs.python.org/file38462/mutithreading_tarfile.patch ___ Python tracker _

[issue23649] tarfile not re-entrant for multi-threading

Srdjan Grubor added the comment: After some thinking, for the makedirs it should only need makedirs(exist_ok=True) -- ___ Python tracker ___

[issue23649] tarfile not re-entrant for multi-threading

Srdjan Grubor added the comment: I don't know if that's true of core libraries. Why complicate things for end users when those issues could be done in the library itself and be completely transparent to the devs? A simple RLock latch wouldn't pose almost any speed degradation but would work in

[issue23649] tarfile not re-entrant for multi-threading

R. David Murray added the comment: If you want to use an object that has state in more than one thread you generally have to put some locking around it. Unless I'm missing something (which I might be) I don't think it is tarfile's responsibility to do this. -- nosy: +r.david.murray _

[issue23649] tarfile not re-entrant for multi-threading

Changes by STINNER Victor : -- nosy: +haypo, lars.gustaebel versions: +Python 3.5 ___ Python tracker ___ ___ Python-bugs-list mailing

[issue23649] tarfile not re-entrant for multi-threading

Srdjan Grubor added the comment: The code around tarfile multi-threading was fixed for me on the user-side with threading.Lock() usage so it might work to use this within the library and the directory creation could be improved by probably doing a try/except around the makedirs() call with ign

[issue23649] tarfile not re-entrant for multi-threading

Changes by Srdjan Grubor : -- type: enhancement -> behavior ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: htt

[issue23649] tarfile not re-entrant for multi-threading

Srdjan Grubor added the comment: Also, extract_member in tarfile.py is not thread-safe since the check for folder existence might occur during another thread's creation of that same dir causing the code to error out. File "/usr/lib/python3.4/concurrent/futures/thread.py", line 54, in run

[issue23649] tarfile not re-entrant for multi-threading

Changes by Srdjan Grubor : -- type: behavior -> enhancement ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: htt

[issue23649] tarfile not re-entrant for multi-threading

New submission from Srdjan Grubor: When running tarfile.extract through multiple threads, the archive reading pointer is not protected from simultaneous seeks and causes various convoluted bugs: self.archive_object.extract(member, extraction_path) File "/usr/lib/python3.4/tarfile.py",