[issue17449] dev guide appears not to cover the benchmarking suite
Roundup Robot added the comment: New changeset 428b6350307e by Benjamin Peterson in branch 'default': add a pointer to the benchmarks repo (closes #17449) http://hg.python.org/devguide/rev/428b6350307e -- nosy: +python-dev resolution: - fixed stage: needs patch - resolved status: open - closed ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17449 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17449] dev guide appears not to cover the benchmarking suite
Kavya Joshi added the comment: I added a `Benchmarking` section in the dev guide with the relevant links and the intended use (testing rather than optimizing). I also added an entry for the section in the Index. Testing/verification: I built the docs and visually inspected them. -- keywords: +patch nosy: +kavya Added file: http://bugs.python.org/file35521/issue17449.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17449 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17449] dev guide appears not to cover the benchmarking suite
Antoine Pitrou added the comment: Jeff: yes it could :) Do you want to provide a patch? -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17449 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17449] dev guide appears not to cover the benchmarking suite
Jeff Ramnani added the comment: Now that bug #18586 is closed, could the Dev Guide point benchmarkers to the benchmarks repo and its README? http://hg.python.org/benchmarks/file/9a1136898539/README.txt -- nosy: +jramnani ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17449 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17449] dev guide appears not to cover the benchmarking suite
Ronald Oussoren added the comment: Documenting this would be useful because it appears to be non-trivial to actually run the benchmark suite with Python 3. * Using python perf.py -b default fails due to some code in the benchmarks that is not python3 compatible * Using make_perf3.sh and then running ... -b default still fails, because not all libraries are translated (the test run stopped because splitfire couldn't be found) * Using -b 2n3 instead instead of -b default or -b all works, but runs only a subset of the benchmark. -- nosy: +ronaldoussoren ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17449 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17449] dev guide appears not to cover the benchmarking suite
Antoine Pitrou added the comment: * Using -b 2n3 instead instead of -b default or -b all works, but runs only a subset of the benchmark. I think we should include a copy of the 2to3'ed third-party libs in the benchmarks repo, this would running avoid the length and tedious conversion on every user's computer. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17449 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17449] dev guide appears not to cover the benchmarking suite
Ronald Oussoren added the comment: The overhead of 2to3 isn't too bad, especially because only a subset of the benchmarks is translated in the first place. It is more important that there is better documentation than there is now, it seems that you're supposed to use either '-b 2n3' or make_perf3.sh with '-b py3k'. Neither of which are mentioned in README.txt. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17449 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17449] dev guide appears not to cover the benchmarking suite
Antoine Pitrou added the comment: The overhead of 2to3 isn't too bad, especially because only a subset of the benchmarks is translated in the first place. Well, the biggest time is spent translating the third-party libraries, not benchmark files. Also, it's simply tedious manual work (you have to create a separate directory etc.). It is more important that there is better documentation than there is now, it seems that you're supposed to use either '-b 2n3' or make_perf3.sh with '-b py3k'. Neither of which are mentioned in README.txt. That's because the benchmark suite was initially 2.x-only, and the make_perf3.sh was a quick hack to make some of the benchmarks work on 3.x. Nowadays the goal is for benchmark files to be 2.x and 3.x compatible in the same codebase, so that only third-party libs remain to translate. Adding 3.x-compatible versions of the third-party libs would eliminate that last step. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17449 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17449] dev guide appears not to cover the benchmarking suite
Ronald Oussoren added the comment: The attached patch for the benchmark README.txt should document the status quo. -- Added file: http://bugs.python.org/file31073/issue-18181-full-v3.txt ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17449 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17449] dev guide appears not to cover the benchmarking suite
Dave Malcolm added the comment: On Mon, 2013-07-29 at 14:01 +, Ronald Oussoren wrote: Ronald Oussoren added the comment: The attached patch for the benchmark README.txt should document the status quo. -- Added file: http://bugs.python.org/file31073/issue-18181-full-v3.txt It looks like you attached the wrong patch: the patch you attached seems to be unrelated to benchmarking (and presumably relates to issue 18181) -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17449 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17449] dev guide appears not to cover the benchmarking suite
Ronald Oussoren added the comment: Sorry about that. I stored the patch file in the wrong directory and attached the most recent file in the directory where I had expected the patch to be. -- Added file: http://bugs.python.org/file31074/issue-17449-readme.txt ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17449 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17449] dev guide appears not to cover the benchmarking suite
Changes by Ronald Oussoren ronaldousso...@mac.com: Removed file: http://bugs.python.org/file31073/issue-18181-full-v3.txt ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17449 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17449] dev guide appears not to cover the benchmarking suite
Brett Cannon added the comment: I agree with Antoine that the time has come to drop make_perf3.sh and make it so that Python 3 doesn't require a separate build of the benchmark directory. Created http://bugs.python.org/issue18586 to track this idea. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17449 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17449] dev guide appears not to cover the benchmarking suite
Meador Inge added the comment: I think this will be useful information. I tend to doing benchmarks rather infrequently. Whenever I do them I forgot how I setup the suite the previous time around. In fact, I found this issue just now while googling for how to setup and run the official benchmark suite :-) -- nosy: +meador.inge ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17449 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17449] dev guide appears not to cover the benchmarking suite
Changes by Edmond Burnett eburn...@gmail.com: -- nosy: +edmond.burnett ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17449 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17449] dev guide appears not to cover the benchmarking suite
Changes by Ezio Melotti ezio.melo...@gmail.com: -- stage: - needs patch type: - enhancement ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17449 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17449] dev guide appears not to cover the benchmarking suite
Antoine Pitrou added the comment: Mmmh... I'm not very warm on this one. If we start documenting the benchmark suite in the devguide, then people will start to be all gung ho about it, be obsessed with benchmark numbers, etc. I guess what I'm saying is that *if* it's covered, then the very beginning of the section should explain that benchmarking is not important in most cases, and time spent on Python is generally better spent fixing bugs and adding features. -- nosy: +pitrou ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17449 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17449] dev guide appears not to cover the benchmarking suite
Ezio Melotti added the comment: Benchmarking is also useful to test that existing patches (that fix bug or add features) don't have a negative effect on the performance. Some kind of warning that advises against using benchmarks just to try and optimize everything sounds like a good idea though. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17449 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17449] dev guide appears not to cover the benchmarking suite
Brett Cannon added the comment: No it does not, but it should. -- assignee: brett.cannon - keywords: +easy ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17449 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17449] dev guide appears not to cover the benchmarking suite
New submission from Dave Malcolm: Does the devguide document the benchmarking suite anywhere? I can't see it anywhere in the index on http://docs.python.org/devguide/ and google doesn't seem to show anything. suggested content: * how to run the benchmarks for a Python 2 implementation * how to run the benchmarks for a Python 3 implementation Sorry if it exists and I missed it -- assignee: brett.cannon components: Devguide messages: 184421 nosy: brett.cannon, dmalcolm, ezio.melotti priority: normal severity: normal status: open title: dev guide appears not to cover the benchmarking suite ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17449 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
