Re: callgrind metrics was: Re: minutes of ESC call ...
On Sat, 2013-12-07 at 12:08 +, Michael Meeks wrote: What files should it test ? Currently it's empty.ods, empty.odt, and sample.xlsx - just some numbers in there. I guess we should get QA to create some small files with a chart, a small image, a bit of text attributes, bold, italic etc. Sounds good, how to achieve this? :-) Anyone with commit access can just push documents to buildbot.git/loperf/docs and they will be automatically tested. I'd be happy if we only ran one performance test cycle each eight hours - you see how stable the results are :-) so - I'd be eager to have some larger documents so we can catch new N^2ds that creep in [ the most common cause of grief ], so some 10k row spreadsheets, and some 100 page documents would be good. right, makes sense I guess we should also have import (and ideally export ;-) of all our major formats. So - it looks beautiful - any chance to make it slower more comprehensive ? [ build less, and test more ]. Ok, so I've added --convert-to option which hopefully helps to achieve this. It just needs more documents. I might add some random files there. Where should I upload this history.csv ? Post to the list once in a week, or..? Having it available on-line would be cool; ideally as flat-odf ;-) [ saves a bit of import time ], and we'll graph it. Did you mean .fods ? It does produce .fods now too. It's in http://dev-builds.libreoffice.org/callgrind_report/ Best, Matus ___ LibreOffice mailing list LibreOffice@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/libreoffice
Re: callgrind metrics was: Re: minutes of ESC call ...
On Wed, Dec 11, 2013 at 9:55 AM, Matúš Kukan matus.ku...@collabora.com wrote: On Sat, 2013-12-07 at 12:08 +, Michael Meeks wrote: What files should it test ? Currently it's empty.ods, empty.odt, and sample.xlsx - just some numbers in there. I guess we should get QA to create some small files with a chart, a small image, a bit of text attributes, bold, italic etc. Sure, QA can assemble/make up a set of some test docs. We can try to assemble documents created in other applications, if preferable (and if we can find them :-) Sounds good, how to achieve this? :-) Anyone with commit access can just push documents to buildbot.git/loperf/docs and they will be automatically tested. - you see how stable the results are :-) so - I'd be eager to have some larger documents so we can catch new N^2ds that creep in [ the most common cause of grief ], so some 10k row spreadsheets, and some 100 page documents would be good. The buildbot repo is pretty small right now, which can be useful. How much test data total are we thinking about adding to it? (maybe a separate repo for test documents could be useful beyond perf testing) Thanks, --R ___ LibreOffice mailing list LibreOffice@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/libreoffice
Re: callgrind metrics was: Re: minutes of ESC call ...
On Wed, 2013-12-11 at 15:55 +0100, Matúš Kukan wrote: Anyone with commit access can just push documents to buildbot.git/loperf/docs and they will be automatically tested. Nice =) I guess we should also have import (and ideally export ;-) of all our major formats. So - it looks beautiful - any chance to make it slower more comprehensive ? [ build less, and test more ]. Ok, so I've added --convert-to option which hopefully helps to achieve this. It just needs more documents. I might add some random files there. Cool - well; please construct some larger documents for calc - it'd be nice if we had some that triggered re-calculation on load too. For Impress it'd be nice to have some slide-decks with a number of large images in them etc. Then of course, having the (same?) data in multiple formats would be a nice comparison point. In general it's best to have single files that exercise one feature each - so we can chart how that feature performs on import over time. Having it available on-line would be cool; ideally as flat-odf ;-) [ saves a bit of import time ], and we'll graph it. Did you mean .fods ? It does produce .fods now too. It's in http://dev-builds.libreoffice.org/callgrind_report/ That looks beautiful =) I assume that between: 2013-12-11 09:58:25+01:00 cd7bce9e3701942438e420c9dde69f44eab5fa00 2013-12-11 10:46:56+01:00 326eb94142e0fe9fbb81243fea58261f0203ddcc the ~6x slow-down in sample.xlsx is due to the file itself changing ;-) Oh - also, adding a number format like: #,##0 to each of the callgrind pseudo-cycle numbers makes it massively more readable =) actually rounding those numbers to millions of cycles would make them much more tractable too. But hey - this is really awesome progress - really glad to have that there generating stats. Thanks ! Michael. -- michael.me...@collabora.com , Pseudo Engineer, itinerant idiot ___ LibreOffice mailing list LibreOffice@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/libreoffice
Re: callgrind metrics was: Re: minutes of ESC call ...
On Wed, 2013-12-11 at 10:19 -0500, Robinson Tryon wrote: The buildbot repo is pretty small right now, which can be useful. How much test data total are we thinking about adding to it? (maybe a separate repo for test documents could be useful beyond perf testing) Well - big documents don't have to be 100's of Mb large; the zip compression helps a lot for spreadsheets at least. Also - if we're going to load them into a LibreOffice 100x slower than real-time, we should prolly make sure we don't have too many that are too large - particularly if we want to compare filter performance by having the same files in multiple formats =) But it'd be awesome to have help from QA constructing some initially: but when made, we want to ~permanently freeze the documents so we can compare LibreOffice over the long term without changes there. Matus - out of interest - are we still timing / charting startup ? I'd love to see 2x measurements there if possible: first the first ever start where we do a ton of UNO / component bootstrapping and then also a second start =) HTH, Michael. -- michael.me...@collabora.com , Pseudo Engineer, itinerant idiot ___ LibreOffice mailing list LibreOffice@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/libreoffice
Re: callgrind metrics was: Re: minutes of ESC call ...
On Wed, Dec 11, 2013 at 8:55 AM, Matúš Kukan matus.ku...@collabora.com wrote: On Sat, 2013-12-07 at 12:08 +, Michael Meeks wrote: What files should it test ? Currently it's empty.ods, empty.odt, and sample.xlsx - just some numbers in there. I guess we should get QA to create some small files with a chart, a small image, a bit of text attributes, bold, italic etc. Sounds good, how to achieve this? :-) Anyone with commit access can just push documents to buildbot.git/loperf/docs and they will be automatically tested. hold on... we do have a git repo for test files, aptly named test-files.git I'd rather not bloat buibot.git too much as this need to be cloned on every tb Norbert ___ LibreOffice mailing list LibreOffice@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/libreoffice
Re: callgrind metrics was: Re: minutes of ESC call ...
Hi Matus, On Fri, 2013-12-06 at 13:18 +0100, Matúš Kukan wrote: So - it's running and producing callgrind profiles - Is it possible somebody would be interested in them ? They are deleted after 10 days, so that should be enough in that case I hope. IMHO we want as long a series as possible even if the profiles are deleted - but it'd be really great to have that data in a form that is workable. Charting it it is -awesome- to see how incredibly flat the data is =) What files should it test ? Currently it's empty.ods, empty.odt, and sample.xlsx - just some numbers in there. I guess we should get QA to create some small files with a chart, a small image, a bit of text attributes, bold, italic etc. There can't be too many files nor big ones, otherwise it takes too much time. I'd be happy if we only ran one performance test cycle each eight hours - you see how stable the results are :-) so - I'd be eager to have some larger documents so we can catch new N^2ds that creep in [ the most common cause of grief ], so some 10k row spreadsheets, and some 100 page documents would be good. I guess we should also have import (and ideally export ;-) of all our major formats. So - it looks beautiful - any chance to make it slower more comprehensive ? [ build less, and test more ]. Any ideas how to make this useful for you ? Btw. it's in buildbot.git/loperf Neato. Where should I upload this history.csv ? Post to the list once in a week, or..? Having it available on-line would be cool; ideally as flat-odf ;-) [ saves a bit of import time ], and we'll graph it. Mercifully we don't need to look at it -too- often since we'll immediately see performance problems and can get the git hashes / revisions easily. Anyhow - great to see your progress here ! Thanks, Michael. -- michael.me...@collabora.com , Pseudo Engineer, itinerant idiot ___ LibreOffice mailing list LibreOffice@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/libreoffice
callgrind metrics was: Re: minutes of ESC call ...
On Thu, 2013-12-05 at 17:23 +, Michael Meeks wrote: * Pending Action Items: + actually produce callgrind performance metrics from VM (Matus) So - it's running and producing callgrind profiles - Is it possible somebody would be interested in them ? They are deleted after 10 days, so that should be enough in that case I hope. And the result (attached) is something like: date git-commit CEst-for-file1 CEst-for-file2 .. What files should it test ? Currently it's empty.ods, empty.odt, and sample.xlsx - just some numbers in there. There can't be too many files nor big ones, otherwise it takes too much time. Any ideas how to make this useful for you ? Btw. it's in buildbot.git/loperf Where should I upload this history.csv ? Post to the list once in a week, or..? Thanks, Matus date time git-commit offload empty.ods empty.odt sample.xlsx 2013-12-03 16:44:09+01:00 86268546a09c3bdd0d5cb6bc047408db779e057c 3054566195 2039663691 2371898423 6836419695 2013-12-03 19:01:41+01:00 d83328b233f51d4a70bfeaae90129a68dccf825b 2955548755 2023025385 2359846135 6860023737 2013-12-03 19:13:20+01:00 c3760e9099db9cf1be696347e2a0743a3cae1b20 2955782274 2023300742 2359931669 6840258596 2013-12-03 20:45:37+01:00 af43ae6f85f9ca840889d568f15b6123c98037a4 2955731942 2023142394 2358388597 6843090795 2013-12-03 21:25:41+01:00 c4d79527ce3a0d9b466ea291a1932bcd66474827 2956440946 2023869893 2359756512 6860276096 2013-12-03 21:35:39+01:00 37d9edc4a7da90da502e62e2adde67396d049bae 2956922327 2023310541 2360547300 6846361226 2013-12-03 21:53:59+01:00 7eb3e0b3892c90a5a2fbaaeaf7693bffeb80a360 2956542166 2023207897 2360088501 6861016596 2013-12-03 22:15:16+01:00 bfd1909c87d0d645f1bbb74a142172ecc15100e8 2957476907 2023555981 2359559775 6855215977 2013-12-04 01:11:53+01:00 882665d821a2fc705b7ae03372c2ae7593028210 2956721914 2023837055 2360925339 6851398137 2013-12-04 08:55:02+01:00 d6de313b043154e70a84c0fc29cbae94fe7541b1 2957353416 2023426462 2359526504 6846275353 2013-12-04 10:32:20+01:00 79eab004dca8413cf99ea688291083df2d146230 2957444016 2023977593 2359756193 6832959376 2013-12-04 12:53:30+01:00 a073e81c3acb0c4aa3bc4fde146b6eb9869738e1 2959508563 2023896575 2360345563 6847003507 ___ LibreOffice mailing list LibreOffice@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/libreoffice
Re: callgrind metrics was: Re: minutes of ESC call ...
Good question, Matúš I have been thinking about that also, and was thinking of asking the qa people for a list of file load/save complaints from fdo. The test ods should definitely have a graph in it. That is always a speed bottleneck in Calc. Unfortunately some of the interpretation of the graph data is deferred now until the user clicks the graph. That means that some of the biggest loading bottleneck will not be measured by loperf. The idea was once to have loperf produce a single perfomance indicator, and present that as a graph somewhere of the performance over time. Only retain the underlying data for a short time for dev's interested in the causes of performance regressions. Greetings, Stephan 2013/12/6 Matúš Kukan matus.ku...@gmail.com: On Thu, 2013-12-05 at 17:23 +, Michael Meeks wrote: * Pending Action Items: + actually produce callgrind performance metrics from VM (Matus) So - it's running and producing callgrind profiles - Is it possible somebody would be interested in them ? They are deleted after 10 days, so that should be enough in that case I hope. And the result (attached) is something like: date git-commit CEst-for-file1 CEst-for-file2 .. What files should it test ? Currently it's empty.ods, empty.odt, and sample.xlsx - just some numbers in there. There can't be too many files nor big ones, otherwise it takes too much time. Any ideas how to make this useful for you ? Btw. it's in buildbot.git/loperf Where should I upload this history.csv ? Post to the list once in a week, or..? Thanks, Matus ___ LibreOffice mailing list LibreOffice@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/libreoffice ___ LibreOffice mailing list LibreOffice@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/libreoffice
