[issue36044] PROFILE_TASK for PGO build is not a good workload

Neil Schemenauer added the comment: Closing as I think PR 14702 mostly resolves this. The gc_collect() issue has not been resolved but based on some investigation, I think it is not worth the effort to try to reduce the time spend in that function. Reducing the number of unit tests being

[issue36044] PROFILE_TASK for PGO build is not a good workload

Neil Schemenauer added the comment: I think expanding the list of tests used by --pgo is fine, as long as we put a little thought into each one we add. The ones added by Steve look fine to me. It seems useful to run a profiler and see if there are any unusually expensive tests being added.

[issue36044] PROFILE_TASK for PGO build is not a good workload

Steve Dower added the comment: Thanks for approving my PR, Greg! -- ___ Python tracker ___ ___ Python-bugs-list mailing list

[issue36044] PROFILE_TASK for PGO build is not a good workload

Steve Dower added the comment: > What matters there is compiling the underlying lzma, bz2, and sqlite3 > libraries with PGO. that isn't done as part of our build system. On Windows they are linked in as source files, which is why it matters for us. On platforms where we rely on system

[issue36044] PROFILE_TASK for PGO build is not a good workload

miss-islington added the comment: New changeset 36fd7b6f01127bc6a8b4a37a363e0aa9cfd76506 by Miss Islington (bot) in branch '3.8': bpo-36044: Avoid warnings in Windows PGO build and add lzma, bz2 and sqlite coverage (GH-14985)

[issue36044] PROFILE_TASK for PGO build is not a good workload

Gregory P. Smith added the comment: i doubt test_lzma test_bz2 and test_sqlite matter. What matters there is compiling the underlying lzma, bz2, and sqlite3 libraries with PGO. that isn't done as part of our build system. regardless, those tests are fast enough so i've approved the PR to

[issue36044] PROFILE_TASK for PGO build is not a good workload

Change by miss-islington : -- pull_requests: +14763 pull_request: https://github.com/python/cpython/pull/14997 ___ Python tracker ___

[issue36044] PROFILE_TASK for PGO build is not a good workload

miss-islington added the comment: New changeset e1b900247227dad49d8231f1d028872412230ab4 by Miss Islington (bot) (Steve Dower) in branch 'master': bpo-36044: Avoid warnings in Windows PGO build and add lzma, bz2 and sqlite coverage (GH-14985)

[issue36044] PROFILE_TASK for PGO build is not a good workload

Inada Naoki added the comment: @steve.dower Does the PGO have significant benefit for those modules? -- ___ Python tracker ___ ___

[issue36044] PROFILE_TASK for PGO build is not a good workload

Steve Dower added the comment: I also added test_bz2 and test_lzma into the PGO profile, as the extension modules for those on Windows were barely being covered (they get imported, apparently, but that seems to be it). -- ___ Python tracker

[issue36044] PROFILE_TASK for PGO build is not a good workload

Change by Steve Dower : -- pull_requests: +14753 pull_request: https://github.com/python/cpython/pull/14985 ___ Python tracker ___

[issue36044] PROFILE_TASK for PGO build is not a good workload

Steve Dower added the comment: I just ran this on Windows and noticed that there is no data for _msi.pyd, winsound.pyd or _sqlite3.pyd. The first two don't matter - we should just suppress PGO on those projects to avoid the warnings (add a false property). That's a note to myself, unless

[issue36044] PROFILE_TASK for PGO build is not a good workload

Gregory P. Smith added the comment: tracking that as a release blocker in https://bugs.python.org/issue37667 -- ___ Python tracker ___

[issue36044] PROFILE_TASK for PGO build is not a good workload

Zackery Spytz added the comment: Travis only runs 40 tests after this change (https://travis-ci.org/python/cpython/jobs/562362678). -- nosy: +ZackerySpytz ___ Python tracker

[issue36044] PROFILE_TASK for PGO build is not a good workload

miss-islington added the comment: New changeset 2406672984e4c1b18629e615edad52928a72ffcc by Miss Islington (bot) in branch '3.8': bpo-36044: Reduce number of unit tests run for PGO build (GH-14702) https://github.com/python/cpython/commit/2406672984e4c1b18629e615edad52928a72ffcc --

[issue36044] PROFILE_TASK for PGO build is not a good workload

Change by miss-islington : -- pull_requests: +14685 pull_request: https://github.com/python/cpython/pull/14910 ___ Python tracker ___

[issue36044] PROFILE_TASK for PGO build is not a good workload

Neil Schemenauer added the comment: New changeset 4e16a4a3112161a5c6981c0588142d4a4673a934 by Neil Schemenauer in branch 'master': bpo-36044: Reduce number of unit tests run for PGO build (GH-14702) https://github.com/python/cpython/commit/4e16a4a3112161a5c6981c0588142d4a4673a934

[issue36044] PROFILE_TASK for PGO build is not a good workload

Neil Schemenauer added the comment: I changed configure.in to use AC_ARG_VAR instead. So, you can override it as you suggest: ./configure [..] PROFILE_TASK="-m test --pgo-extended" -- ___ Python tracker

[issue36044] PROFILE_TASK for PGO build is not a good workload

Inada Naoki added the comment: >./configure [...] --with-profile-task='-m test --pgo-extended' I think this is abusing of `--with` options. `--with-*` is for external software [1]. But `--with-lto` option abuses it already. How about `./configure PROFILE_TASKS="-m test --pgo-extend"`?

[issue36044] PROFILE_TASK for PGO build is not a good workload

Neil Schemenauer added the comment: Thanks for the feedback. I agree that putting the tests in regrtest is better. I've made the following changes: - the --pgo option now uses the shorter list of tests by default - I added --pgo-extended as a regrtest option that enables the old behavior

[issue36044] PROFILE_TASK for PGO build is not a good workload

Gregory P. Smith added the comment: As far as where to put the lists of tests, you're probably right. putting it within test.regrtest itself under the --pgo banner makes sense. (though we should keep logic to accept a list of explicit tests to add or exclude if the user has also provided

[issue36044] PROFILE_TASK for PGO build is not a good workload

Steve Dower added the comment: Looking at PR 14702, I would much rather update what `-m test --pgo` does than make this exclusive to Makefile. Back when we added the --pgo flag, the intent was always to move toward an equivalently good profile, but for practical reasons we settled at the

[issue36044] PROFILE_TASK for PGO build is not a good workload

Neil Schemenauer added the comment: +-+---+-+ | Benchmark | task-all2 | task-short7 | +=+===+=+ | 2to3| 304 ms| 305 ms:

[issue36044] PROFILE_TASK for PGO build is not a good workload

Neil Schemenauer added the comment: I tweaked the list of unit tests a little more, trying to incorporate some from your Docker build settings. Not sure what's going on with the pickle results. Below are new pyperformance runs, comparing my PR to the "master" version it is based on. I

[issue36044] PROFILE_TASK for PGO build is not a good workload

Change by Neil Schemenauer : -- keywords: +patch pull_requests: +14501 stage: -> patch review pull_request: https://github.com/python/cpython/pull/14702 ___ Python tracker

[issue36044] PROFILE_TASK for PGO build is not a good workload

Gregory P. Smith added the comment: yeah I pulled a similar looking list of tests for the PROFILE_TASK to suggest to the Docker Python image maintainers - see https://github.com/docker-library/python/pull/404 and https://github.com/docker-library/python/issues/160. I haven't run

[issue36044] PROFILE_TASK for PGO build is not a good workload

Neil Schemenauer added the comment: > Decreasing the total wall time for a default --enable-optimizations build > would > be a good thing for everyone, provided the resulting interpreter remains > "effectively similar" in speed. If you somehow manage to find something that > actually

[issue36044] PROFILE_TASK for PGO build is not a good workload

Gregory P. Smith added the comment: In my experience, people overthink what needs to go into a CPython profiling run. Sure, our default PROFILE_TASK is rather unfortunate because it takes a very long time by including runs of super slow tests that won't meaningfully contribute profile data

[issue36044] PROFILE_TASK for PGO build is not a good workload

Change by Tianon : -- nosy: +tianon ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue36044] PROFILE_TASK for PGO build is not a good workload

Change by Inada Naoki : -- nosy: +inada.naoki ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue36044] PROFILE_TASK for PGO build is not a good workload

Pablo Galindo Salgado added the comment: Also, the test suite exercises a lot of branches (like passing incorrect types to function to check errors) that will hurt the branch prediction that PGO generates. -- nosy: +pablogsal ___ Python tracker

[issue36044] PROFILE_TASK for PGO build is not a good workload

New submission from Neil Schemenauer : I was doing some 'perf' profiling runs of Python. I decided to try running PROFILE_TASK to see what the profile looks like. I was surprised that gc_collect dominated runtime: Children Self Symbol + 93.93% 6.00% [.]