[issue24915] Profile Guided Optimization improvements (better training, llvm support, etc)

Brett Cannon added the comment: Please add the fix to the issue that reported the problem so that the fix can be tracked with the bug report. -- resolution: -> fixed status: open -> closed ___ Python tracker

[issue24915] Profile Guided Optimization improvements (better training, llvm support, etc)

Changes by Alecsandru Patrascu : -- resolution: fixed -> status: closed -> open ___ Python tracker ___

[issue24915] Profile Guided Optimization improvements (better training, llvm support, etc)

Changes by Alecsandru Patrascu : Added file: http://bugs.python.org/file41917/pgofix-cpython3.patch ___ Python tracker ___

[issue24915] Profile Guided Optimization improvements (better training, llvm support, etc)

Alecsandru Patrascu added the comment: I've added a fix for the PGO builds after a issue pointed out in #26307. Thank you Christos for your observation! -- Added file: http://bugs.python.org/file41916/pgofix-cpython2.patch ___ Python tracker

[issue24915] Profile Guided Optimization improvements (better training, llvm support, etc)

Χρήστος Γεωργίου (Christos Georgiou) added the comment: Perhaps I'm missing something obvious here, but… … $(MAKE) build_all_merge_profile @echo "Rebuilding with profile guided optimizations:" $(MAKE) clean $(MAKE) build_all_use_profile … the `$(MAKE) clean` does an `rm

[issue24915] Profile Guided Optimization improvements (better training, llvm support, etc)

Alecsandru Patrascu added the comment: Hello and thank you for your feedback. For CPython this does not apply because due to the structure of the build system, inside the "build" directory there are no PGO profiles saved. You can run find . -name '*.gc??' to see. --

[issue24915] Profile Guided Optimization improvements (better training, llvm support, etc)

Χρήστος Γεωργίου (Christos Georgiou) added the comment: There are. (Check issue #26307 that explains this cpio file. This is a x32 build of Python, because the memory savings are very welcome for the multiple worker processes of a project I work on.) $ cpio -it <_modules.gcda.cpio

[issue24915] Profile Guided Optimization improvements (better training, llvm support, etc)

Χρήστος Γεωργίου (Christos Georgiou) added the comment: First, let's make sure we're on the same page. - These files are created during the `$(MAKE) run_profile_task` stage. - They get removed during the `$(MAKE) clean` stage, along with the build directory. - The build directory gets

[issue24915] Profile Guided Optimization improvements (better training, llvm support, etc)

Alecsandru Patrascu added the comment: That's interesting. Even on CPython3, I still don't see any gcda's inside the build directory, nor the tree structure you are seeing there. Can you please give me a couple of details regarding your environment (os, distribution, gcc version, 32/64 bit,

[issue24915] Profile Guided Optimization improvements (better training, llvm support, etc)

Alecsandru Patrascu added the comment: For my responses, I modified locally the Makefile so that it will not remove the build directory and any of the gcda files. I will make some more tests tomorrow, but i think that this problem will solve simpler if the removal of the build directory is

[issue24915] Profile Guided Optimization improvements (better training, llvm support, etc)

Brett Cannon added the comment: regrtest changes are now in 2.7, 3.5, and default in spite of Victor changing everything underneath me constantly in default. =) That should make the buildbot happy again. -- resolution: -> fixed status: open -> closed

[issue24915] Profile Guided Optimization improvements (better training, llvm support, etc)

STINNER Victor added the comment: > regrtest changes are now in 2.7, 3.5, and default in spite of Victor changing > everything underneath me constantly in default. =) That should make the > buildbot happy again. Yeah sorry, it was my regrtest week :-) --

[issue24915] Profile Guided Optimization improvements (better training, llvm support, etc)

Brett Cannon added the comment: It's fine. =) Glad it was for good reasons. Just took quite a while to manually apply the old patch to the new layout and then fix merges. -- ___ Python tracker

[issue24915] Profile Guided Optimization improvements (better training, llvm support, etc)

Changes by Brett Cannon : -- dependencies: +regrtest.py improvement for Profile Guided Optimization builds ___ Python tracker ___

[issue24915] Profile Guided Optimization improvements (better training, llvm support, etc)

Alecsandru Patrascu added the comment: Thank you for upstreaming this in both branches of Python! Do you think that a different version of regrtest.py, that will be used only for PGO training, should be better in this case? I mean, by implementing a custom version, I think we can control

[issue24915] Profile Guided Optimization improvements (better training, llvm support, etc)

Brett Cannon added the comment: A separate issue is fine, Alecsandru, since we can make it a dependency of this issue. -- ___ Python tracker ___

[issue24915] Profile Guided Optimization improvements (better training, llvm support, etc)

Antoine Pitrou added the comment: I think the --pgo flag needs only work in single process mode, since multi-process would probably not write out the profiling data properly. -- ___ Python tracker

[issue24915] Profile Guided Optimization improvements (better training, llvm support, etc)

Skip Montanaro added the comment: Would it be possible to grep out the warning messages, but let everything else through? On Sep 19, 2015 1:34 PM, "Brett Cannon" wrote: > > Brett Cannon added the comment: > > The problem with the output is that error cases are

[issue24915] Profile Guided Optimization improvements (better training, llvm support, etc)

Alecsandru Patrascu added the comment: I can work on modifying the existing regrtest and adding a distinct flag, --pgo for example, as Antoine suggested. Indeed, it will not be trivial as regrtest has a dual approach (single process and multi process), but I will give it a try and post a

[issue24915] Profile Guided Optimization improvements (better training, llvm support, etc)

Antoine Pitrou added the comment: Instead of writing a custom test runner from scratch, I would suggest adding a hidden --option to regrtest that would disable reporting errors. -- ___ Python tracker

[issue24915] Profile Guided Optimization improvements (better training, llvm support, etc)

Brett Cannon added the comment: It didn't break gcc, the buildbot simply wasn't patient enough for the PGO run of the test suite to complete: http://buildbot.python.org/all/builders/AMD64%20Debian%20PGO%203.5/builds/274/steps/compile/logs/stdio . It takes a good amount of time to run the test

[issue24915] Profile Guided Optimization improvements (better training, llvm support, etc)

Antoine Pitrou added the comment: Le 19/09/2015 20:18, Brett Cannon a écrit : > > And I don't want to add output back simply to appease the buildbot as the output means nothing to a user who is doing the build themselves. The output is actually a good indication of progress, so I don't think

[issue24915] Profile Guided Optimization improvements (better training, llvm support, etc)

Brett Cannon added the comment: The problem with the output is that error cases are unimportant and yet it fooled Skip into temporarily caring until he finally noticed the warning message. So my worry is that someone doesn't notice the "NOTE: ignore errors as they don't affect anything" and

[issue24915] Profile Guided Optimization improvements (better training, llvm support, etc)

STINNER Victor added the comment: Hum, the change 7fcff838d09e broke the buildbot "AMD64 Debian PGO 3.5". It would nice to add Clang support without loosing GCC support :-D http://buildbot.python.org/all/builders/AMD64%20Debian%20PGO%203.5/builds/274 -- nosy: +haypo resolution: fixed

[issue24915] Profile Guided Optimization improvements (better training, llvm support, etc)

Brett Cannon added the comment: I gave the custom test runner a try using unittest's discovery facility, but it started to execute the whole test suite again, so it's a bit more complicated than you might think (I guess it imported regrtest or something?). -- Added file:

[issue24915] Profile Guided Optimization improvements (better training, llvm support, etc)

Brett Cannon added the comment: Thanks to Alecsandru and Intel for the patches! -- resolution: -> fixed stage: commit review -> resolved status: open -> closed ___ Python tracker

[issue24915] Profile Guided Optimization improvements (better training, llvm support, etc)

Antoine Pitrou added the comment: Thank you Brett for committing this. -- ___ Python tracker ___ ___

[issue24915] Profile Guided Optimization improvements (better training, llvm support, etc)

Roundup Robot added the comment: New changeset 7fcff838d09e by Brett Cannon in branch '3.5': Issue #24915: Add Clang support to PGO builds and use the test suite https://hg.python.org/cpython/rev/7fcff838d09e New changeset 7749fc0a5ea6 by Brett Cannon in branch 'default': Merge for issue #24915

[issue24915] Profile Guided Optimization improvements (better training, llvm support, etc)

Roundup Robot added the comment: New changeset 0f4e6c303531 by Brett Cannon in branch '2.7': Issue #24915: Make PGO builds support Clang and use the test suite for https://hg.python.org/cpython/rev/0f4e6c303531 New changeset f211c8f554f9 by Brett Cannon in branch '2.7': Give proper credit for

[issue24915] Profile Guided Optimization improvements (better training, llvm support, etc)

Brett Cannon added the comment: Attached is what I plan to commit to Python 2.7 assuming everyone is happy with the outcome. I tweaked the echoed messages from Alecsandru's patch, pulled in the README changes, and dropped the x86 checks as Antoine and Stefan requested. Assuming people are

[issue24915] Profile Guided Optimization improvements (better training, llvm support, etc)

Alecsandru Patrascu added the comment: If you are talking just about the 3.6 patch, it is called this way to emphasize the fact that it is intended for the development branch. It is perfectly compatible with 3.5, therefore it is not needed for packagers to maintain two distinct versions. I've

[issue24915] Profile Guided Optimization improvements (better training, llvm support, etc)

Antoine Pitrou added the comment: As for the dual 2.7/3.6 aspect: I don't really understand it. If this is committed to 2.7 it should also be committed to 3.5. It doesn't threaten the stability of the interpreter in any way, given it does not affect the default build path. There's no reason

[issue24915] Profile Guided Optimization improvements (better training, llvm support, etc)

Alecsandru Patrascu added the comment: I've updated the patches with redirect to /dev/null, as is it is more clearer to the user what is our intent, without having him to necessarily read the regrtest documentation. I've also added a warning message regarding the output and ported all these

[issue24915] Profile Guided Optimization improvements (better training, llvm support, etc)

Alecsandru Patrascu added the comment: I named this task PROFILE_TASK_X86 because it is rigorously tested and we have proven that regrtest performs better on this architecture. Until any other clear evidence and solid measurements that regrtest is performing better on other architectures

[issue24915] Profile Guided Optimization improvements (better training, llvm support, etc)

Changes by Alecsandru Patrascu : Added file: http://bugs.python.org/file40387/python2.7-pgo-v07.patch ___ Python tracker ___

[issue24915] Profile Guided Optimization improvements (better training, llvm support, etc)

Changes by Alecsandru Patrascu : Added file: http://bugs.python.org/file40388/python3.6-pgo-v07.patch ___ Python tracker ___

[issue24915] Profile Guided Optimization improvements (better training, llvm support, etc)

Changes by Alecsandru Patrascu : Added file: http://bugs.python.org/file40389/README2.7-pgo-v02.patch ___ Python tracker ___

[issue24915] Profile Guided Optimization improvements (better training, llvm support, etc)

Changes by Alecsandru Patrascu : Added file: http://bugs.python.org/file40390/README3.6-pgo-v02.patch ___ Python tracker ___

[issue24915] Profile Guided Optimization improvements (better training, llvm support, etc)

Antoine Pitrou added the comment: Please don't call it "PROFILE_TASK_X86" - the architecture should have nothing to do with it. Actually, there shouldn't be any architecture-specific check at all. -- ___ Python tracker

[issue24915] Profile Guided Optimization improvements (better training, llvm support, etc)

Changes by Alecsandru Patrascu : Removed file: http://bugs.python.org/file40387/python2.7-pgo-v07.patch ___ Python tracker ___

[issue24915] Profile Guided Optimization improvements (better training, llvm support, etc)

Changes by Alecsandru Patrascu : Added file: http://bugs.python.org/file40392/python3.6-pgo-v07.patch ___ Python tracker ___

[issue24915] Profile Guided Optimization improvements (better training, llvm support, etc)

Changes by Alecsandru Patrascu : Removed file: http://bugs.python.org/file40388/python3.6-pgo-v07.patch ___ Python tracker ___

[issue24915] Profile Guided Optimization improvements (better training, llvm support, etc)

Changes by Alecsandru Patrascu : Added file: http://bugs.python.org/file40391/python2.7-pgo-v07.patch ___ Python tracker ___

[issue24915] Profile Guided Optimization improvements (better training, llvm support, etc)

Stefan Krah added the comment: Just (hopefully) for extra clarity: As you mentioned, the 3.6 patch is perfect for 3.5, too. The reason why 3.5 was brought up is to ask Larry, our release manager, to allow it already for 3.5. Technically it's an enhancement/new feature, but practically it is

[issue24915] Profile Guided Optimization improvements (better training, llvm support, etc)

Stefan Krah added the comment: I don't think we should provide any performance guarantees in the Makefile. +1 for not special-casing x86 (does it include amd64?). As I understood, Antoine was not talking about a unified patch but about applying the 3.6 patch to 3.5 right away. --

[issue24915] Profile Guided Optimization improvements (better training, llvm support, etc)

Changes by Brett Cannon : -- assignee: -> brett.cannon ___ Python tracker ___ ___

[issue24915] Profile Guided Optimization improvements (better training, llvm support, etc)

Stefan Behnel added the comment: > The only thing that was a bit mystifying were the errors during the > initial profile run. There is so much that floats by in the terminal > window that I completely missed the warnings about errors during the > test run not being anything to worry about. Then

[issue24915] Profile Guided Optimization improvements (better training, llvm support, etc)

Brett Cannon added the comment: I guess the test output -- both stdout and stderr -- could be redirected to /dev/null as simply using -q with regrtest will still lead to failures being emitted and random output which no one cares about except people inspecting the test output. Just need to

[issue24915] Profile Guided Optimization improvements (better training, llvm support, etc)

Changes by Alecsandru Patrascu alecsandru.patra...@intel.com: Added file: http://bugs.python.org/file40299/README2.7-pgo-v01.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue24915 ___

[issue24915] Profile Guided Optimization improvements (better training, llvm support, etc)

Changes by Alecsandru Patrascu alecsandru.patra...@intel.com: Added file: http://bugs.python.org/file40300/README3.6-pgo-v01.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue24915 ___

[issue24915] Profile Guided Optimization improvements (better training, llvm support, etc)

Alecsandru Patrascu added the comment: That's a good point Skip. I added another set of patches, just for the README files, explaining the entire procedure, so now anyone reading it will see that PGO is available, what are the steps involved and a brief comment about the warning. --

[issue24915] Profile Guided Optimization improvements (better training, llvm support, etc)

Changes by Alecsandru Patrascu alecsandru.patra...@intel.com: Added file: http://bugs.python.org/file40289/python3.6-pgo-v06.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue24915 ___

[issue24915] Profile Guided Optimization improvements (better training, llvm support, etc)

Changes by Alecsandru Patrascu alecsandru.patra...@intel.com: Added file: http://bugs.python.org/file40288/python2.7-pgo-v06.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue24915 ___

[issue24915] Profile Guided Optimization improvements (better training, llvm support, etc)

Alecsandru Patrascu added the comment: I've updated the patches after review and implemented the checkup for llvm-profdata for both Linux and OSX. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue24915

[issue24915] Profile Guided Optimization improvements (better training, llvm support, etc)

Skip Montanaro added the comment: The latest patch worked fine for me (Mac OS X Yosemite). I've only tried with 2.7 so far. The only thing that was a bit mystifying were the errors during the initial profile run. There is so much that floats by in the terminal window that I completely missed

[issue24915] Profile Guided Optimization improvements (better training, llvm support, etc)

Skip Montanaro added the comment: Not knowing a darn thing about this, I went ahead and made a provisional change to the README file. -- Added file: http://bugs.python.org/file40297/README.pgo ___ Python tracker rep...@bugs.python.org

[issue24915] Profile Guided Optimization improvements (better training, llvm support, etc)

Brett Cannon added the comment: I did another round of review. I noticed that the configure part of the patch is missing and that .hgignore and .gitignore should get updated to ignore the profile files. Otherwise the only other comment was making an echoed comment a bit clearer. And in case

[issue24915] Profile Guided Optimization improvements (better training, llvm support, etc)

Alecsandru Patrascu added the comment: The profile merging is necessary in case you want to use a pure clang compiler or you use GCC in OSX. For example, a general profiling action using clang will result in at least one binary profile. For our case, when using regrtest, we will have multiple

[issue24915] Profile Guided Optimization improvements (better training, llvm support, etc)

Antoine Pitrou added the comment: Can you explain what the profile-merging thing is achieving? -- nosy: +pitrou ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue24915 ___

[issue24915] Profile Guided Optimization improvements (better training, llvm support, etc)

Changes by Alecsandru Patrascu alecsandru.patra...@intel.com: Added file: http://bugs.python.org/file40274/python3.6-pgo-v05.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue24915 ___

[issue24915] Profile Guided Optimization improvements (better training, llvm support, etc)

Changes by Alecsandru Patrascu alecsandru.patra...@intel.com: Added file: http://bugs.python.org/file40273/python2.7-pgo-v05.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue24915 ___

[issue24915] Profile Guided Optimization improvements (better training, llvm support, etc)

Alecsandru Patrascu added the comment: I fixed the files after the review. Regarding the PROFILE_OPT_OLD line, I think that it is better to keep also the old task used for PGO, until clear evidence and measurements that regrtest is performing better on other architectures exists. --

[issue24915] Profile Guided Optimization improvements (better training, llvm support, etc)

Changes by Alecsandru Patrascu alecsandru.patra...@intel.com: Added file: http://bugs.python.org/file40268/python3.6-pgo-v04.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue24915 ___

[issue24915] Profile Guided Optimization improvements (better training, llvm support, etc)

Changes by Alecsandru Patrascu alecsandru.patra...@intel.com: Added file: http://bugs.python.org/file40267/python2.7-pgo-v04.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue24915 ___

[issue24915] Profile Guided Optimization improvements (better training, llvm support, etc)

Changes by Alecsandru Patrascu alecsandru.patra...@intel.com: Removed file: http://bugs.python.org/file40263/python3.6-pgo-v04.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue24915 ___

[issue24915] Profile Guided Optimization improvements (better training, llvm support, etc)

Changes by Alecsandru Patrascu alecsandru.patra...@intel.com: Removed file: http://bugs.python.org/file40262/python2.7-pgo-v04.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue24915 ___

[issue24915] Profile Guided Optimization improvements (better training, llvm support, etc)

Changes by Alecsandru Patrascu alecsandru.patra...@intel.com: Added file: http://bugs.python.org/file40262/python2.7-pgo-v04.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue24915 ___

[issue24915] Profile Guided Optimization improvements (better training, llvm support, etc)

Alecsandru Patrascu added the comment: I modified the patches with clang support. Also, I added an important check for the architecture on which PGO is running. Our proposal targets x86 platforms, since our measurements are made only on x86 hardware. --

[issue24915] Profile Guided Optimization improvements (better training, llvm support, etc)

Changes by Alecsandru Patrascu alecsandru.patra...@intel.com: Added file: http://bugs.python.org/file40263/python3.6-pgo-v04.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue24915 ___

[issue24915] Profile Guided Optimization improvements (better training, llvm support, etc)

Changes by Alecsandru Patrascu alecsandru.patra...@intel.com: Added file: http://bugs.python.org/file40255/python3.5-pgo-v03.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue24915 ___

[issue24915] Profile Guided Optimization improvements (better training, llvm support, etc)

Changes by Alecsandru Patrascu alecsandru.patra...@intel.com: Added file: http://bugs.python.org/file40254/python2.7-pgo-v03.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue24915 ___

[issue24915] Profile Guided Optimization improvements (better training, llvm support, etc)

Alecsandru Patrascu added the comment: I modified the patches to be compatible with both environments. The new versions modify the configure.ac file also, therefore you will need to run autoconf by hand. Also, in case of MaOS you will need to have llvm-profdata installed and in your path. I

[issue24915] Profile Guided Optimization improvements (better training, llvm support, etc)

Brett Cannon added the comment: Any specific reason the v3 patch, Alecsandru, is listed as against 3.5 in the filename? Or is that just a typo? P.S.: I did another review asking about explicit Clang support and also supporting Greg's request to use `-m test` instead of the explicit file

[issue24915] Profile Guided Optimization improvements (better training, llvm support, etc)

Gregory P. Smith added the comment: i'm updating the title to be more accurate. turning it on by default is likely not desirable as the makefile is primarily used by developers who are iterating on changes. but having it use a good workload (regrtest) and work with llvm and os x are good. :)