[issue24831] Load average in test suite too high

STINNER Victor added the comment: No activity for 5 years, so far this issue is more an open discussion than a list of concrete changes, so I close the issue as out of date. If some tests are too CPU or I/O intensive, you can skip them using the -x option. Use --ignorefile to only skip

[issue24831] Load average in test suite too high

Change by Irit Katriel : -- versions: +Python 3.10, Python 3.8, Python 3.9 -Python 3.6 ___ Python tracker ___ ___ Python-bugs-list

[issue24831] Load average in test suite too high

STINNER Victor added the comment: There is not option to choose the temporary directory like $TMPDIR? I had issues with /tmp when creating large files because /tmp is now a tmpfs on Fedora by default. Maybe we can blacklist some filesystems to choose the temporary directory. --

[issue24831] Load average in test suite too high

Serhiy Storchaka added the comment: Lib/test/regrtest.py: # When tests are run from the Python build directory, it is best practice # to keep the test files in a subfolder. This eases the cleanup of leftover # files using the make distclean command. if sysconfig.is_python_build(): TEMPDIR =

[issue24831] Load average in test suite too high

Stefan Krah added the comment: Actually, for ecryptfs we could just check for ~/.ecryptfs and then use /tmp. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue24831 ___

[issue24831] Load average in test suite too high

Stefan Krah added the comment: Yes, the current scheme is probably good for easy buildbot cleanup. Maybe we can blacklist some filesystems to choose the temporary directory. If that's easily possible, it would be a good solution at least for ecryptfs. I've removed ecryptfs from my setup,

[issue24831] Load average in test suite too high

Stefan Krah added the comment: It's very likely an I/O issue. There are several large file tests that run even if 'largefile' is disabled. It already helps a lot if the following tests are decorated with requires_resource('largefile'): test_mmap: class LargeMmapTests() test_io:

[issue24831] Load average in test suite too high

Antoine Pitrou added the comment: Should we perhaps create temp files in /tmp rather than in a subdir of the build directory? That would sound reasonable, I don't know why it hasn't been done like that from the start. Perhaps Ezio or Victor know. -- nosy: +ezio.melotti, haypo

[issue24831] Load average in test suite too high

Stefan Krah added the comment: Thanks, this clears up the mystery! -- I made the mistake of choosing the eCryptfs-homedir option when installing Ubuntu, so now I have eCryptfs on top of LVM on top of full-disk encryption. So I can't create sparse files in my home directory, but I *can* create

[issue24831] Load average in test suite too high

New submission from Stefan Krah: On my machine (Lenovo T400, Linux) the load average during running the 2.7 test suite is about 0.5 (with some exceptions like test_io). The system is still usable even during test_io. In 3.6 the load average is 2.0, with some tests making the system

[issue24831] Load average in test suite too high

Stefan Krah added the comment: Scratch the comment about test_io in 2.7: It also renders the system unusable. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue24831 ___

[issue24831] Load average in test suite too high

Antoine Pitrou added the comment: Just to diagnose this, try to run the test suite with nice. If it still make the system unusable, then it's probably an I/O issue rather than CPU scheduling. -- nosy: +pitrou ___ Python tracker

[issue24831] Load average in test suite too high

Serhiy Storchaka added the comment: Any specific tests or all tests? 3.x has many new features in comparison with 2.x, and new tests for these features. For example zipfile and tarfile tests run much longer due to testing with bzip2 and lzma compressions. -- nosy: +serhiy.storchaka