On Thu, Feb 25, 2010 at 9:25 AM, Lionel Dricot <[email protected]> wrote: > > Thanks for your input. I didn't touch anything related to tags but, > indeed, it seems that it is sometimes broken. After some test, it appears > that the task view are, in fact, quite efficient. I did some very very > basic optimisations yesterday and I'm currently between 100% and 200% of > the trunk loading time (when I wrote the mail yesterday, we were at 1000% > so it looks like the concept is good). > > The problem is that taskbrowser was the root of everything. Everything was > in it, from filtering tasks/tags to windows management and tasks updating. > > Every filtering feature should be moved to the filterbank. I will create a > new module, called view_manager, that will contains the current manager.py > in core and will be responsible of loading the preferences and launching > the gtk loop. > > Also, it looks like the tag store itself might benefit from some > refactoring and some love. > > > Bertrand : I removed the refresh because it's not needed anymore. That's > the beauty of all of that. The only global refreshes now happen when a > filter is applied/unapplied. We will even be able to reduce the artificial > 7 second delay before updating that we introduced in the TaskEditor. > > > PS : I'm quite bad at undestanding profiling output, so thanks for your > interpretation. I remember that you found a visualisation tool. What was > the name again ?
It's runsnakerun. But here's the deal: - cumulative time is the total time spent in a function, including the time spent in its callees. Accordingly, we spent the most time in gtg.main and such since they're the ones that launch everything - total time is the total time spent ONLY in the function itself, not in its callees. - percall, is the mean time per call So you generally must have a look at cumulative time, and try to identify what the worst contributor is. For instance, on_task_modified time is heavily due to update_tags_for_task, which is called inside it. Having a look at the most called functions is also worth it, since you really want ot optimize functions that are called a lot of time (e.g. get_value() ) > > On Thu, 25 Feb 2010 01:27:16 +0100, Bertrand Rousseau > <[email protected]> wrote: >> On Wed, Feb 24, 2010 at 5:57 PM, Lionel Dricot <[email protected]> wrote: >>> >>> I've very good news for myself : I see the light. >>> >>> The FilteredTree is now working nearly perfectly. It is rock-solid, > even >>> when loading 300 tasks at the same time. I also implemented the Closed >>> Tasks pane with it. The FiltersBank is also working great. >> >> Great news! >> >>> Bad news : with a lot of tasks, the start time is really really really >>> slow. (like 10 times the current trunk launch time). On the other hand, >>> it >>> looks like other performance issues are mostly solved (like renaming a >>> subtask in a task). >> >> Task renaming is maybe due to the fact that you disabled the refresh >> methods from the browser. Originally, it can from the fact that >> browser refresh is called at each letter you type by the editor. It >> should be worth looking at this too or it will probably show again >> sometime. >> >>> So now, I need *you*. I need testing and profiling. >>> >>> Do : bzr branch lp:~gtg/gtg/gtg-refactor >>> and launch it first with debugging data then with actual data. Do some >>> profiling ( http://live.gnome.org/gtg/development ) and send me the >>> results. >> >> Here's mine (on my netbook, so slownesses are sensible here): >> >> Ordered by: cumulative time >> List reduced from 947 to 15 due to restriction <15> >> >> ncalls tottime percall cumtime percall filename:lineno(function) >> 1 0.000 0.000 10.889 10.889 >> /home/rousseau/workspace/gtg/gtg-refactor/GTG/gtg.py:91(main) >> 1 0.000 0.000 10.127 10.127 >> > /home/rousseau/workspace/gtg/gtg-refactor/GTG/core/manager.py:49(show_browser) >> 1 0.000 0.000 9.047 9.047 >> > /home/rousseau/workspace/gtg/gtg-refactor/GTG/taskbrowser/browser.py:1795(main) >> 1 4.523 4.523 8.653 8.653 {gtk._gtk.main} >> 244 0.030 0.000 3.011 0.012 >> > /home/rousseau/workspace/gtg/gtg-refactor/GTG/taskbrowser/browser.py:1642(on_task_modified) >> 244 0.027 0.000 2.595 0.011 >> > /home/rousseau/workspace/gtg/gtg-refactor/GTG/taskbrowser/tagtree.py:53(update_tags_for_task) >> 4032 0.263 0.000 2.457 0.001 {method 'row_changed' of >> 'gtk.TreeModel' objects} >> 7208 0.394 0.000 1.734 0.000 {method 'get_value' of >> 'gtk.TreeModel' objects} >> 7495 0.060 0.000 1.380 0.000 >> > /home/rousseau/workspace/gtg/gtg-refactor/GTG/taskbrowser/tagtree.py:82(on_get_value) >> 732 0.023 0.000 1.347 0.002 >> > /home/rousseau/workspace/gtg/gtg-refactor/GTG/taskbrowser/browser.py:788(tag_visible_func) >> 879 0.297 0.000 1.295 0.001 >> > /home/rousseau/workspace/gtg/gtg-refactor/GTG/core/tagstore.py:305(get_tasks_nbr) >> 12660 0.107 0.000 1.140 0.000 >> > /home/rousseau/workspace/gtg/gtg-refactor/GTG/core/requester.py:135(get_task) >> 1 0.000 0.000 1.074 1.074 >> > /home/rousseau/workspace/gtg/gtg-refactor/GTG/taskbrowser/browser.py:83(__init__) >> 12660 0.231 0.000 1.034 0.000 >> > /home/rousseau/workspace/gtg/gtg-refactor/GTG/core/datastore.py:51(get_task) >> 2709 0.175 0.000 0.825 0.000 >> > /home/rousseau/workspace/gtg/gtg-refactor/GTG/taskbrowser/browser.py:858(tag_sort_func) >> >> As you can see, a lot of time is spent in update_tags_for_task. If you >> comment this out, get_value and on_task_modified don't even show up in >> the profiling log: >> >> Ordered by: cumulative time >> List reduced from 944 to 15 due to restriction <15> >> >> ncalls tottime percall cumtime percall filename:lineno(function) >> 1 0.000 0.000 9.428 9.428 >> /home/rousseau/workspace/gtg/gtg-refactor/GTG/gtg.py:91(main) >> 1 0.000 0.000 8.639 8.639 >> > /home/rousseau/workspace/gtg/gtg-refactor/GTG/core/manager.py:49(show_browser) >> 1 0.000 0.000 7.443 7.443 >> > /home/rousseau/workspace/gtg/gtg-refactor/GTG/taskbrowser/browser.py:1795(main) >> 1 5.892 5.892 7.012 7.012 {gtk._gtk.main} >> 1 0.001 0.001 1.173 1.173 >> > /home/rousseau/workspace/gtg/gtg-refactor/GTG/taskbrowser/browser.py:83(__init__) >> 1 0.000 0.000 0.776 0.776 >> > /home/rousseau/workspace/gtg/gtg-refactor/GTG/taskbrowser/browser.py:525(_init_plugin_engine) >> 1 0.002 0.002 0.775 0.775 >> > /home/rousseau/workspace/gtg/gtg-refactor/GTG/core/plugins/engine.py:147(load_plugins) >> 12 0.002 0.000 0.587 0.049 >> > /home/rousseau/workspace/gtg/gtg-refactor/GTG/core/plugins/engine.py:40(__init__) >> 12 0.001 0.000 0.544 0.045 >> > /home/rousseau/workspace/gtg/gtg-refactor/GTG/core/plugins/engine.py:108(_load_module) >> 12 0.003 0.000 0.514 0.043 {imp.load_module} >> 29 0.003 0.000 0.442 0.015 >> > /home/rousseau/workspace/gtg/gtg-refactor/GTG/tools/cleanxml.py:98(savexml) >> 2 0.000 0.000 0.430 0.215 >> > /home/rousseau/workspace/gtg/gtg-refactor/GTG/core/__init__.py:160(save_datastore) >> 10 0.356 0.036 0.395 0.040 {method 'show' of >> 'gtk.Widget' objects} >> 975 0.064 0.000 0.383 0.000 >> > /home/rousseau/workspace/gtg/gtg-refactor/GTG/taskbrowser/tasktree.py:94(on_get_value) >> 29 0.001 0.000 0.334 0.012 >> /usr/lib/python2.6/xml/dom/minidom.py:47(toprettyxml) >> >> It's probably not the task of the browser to refresh each tag related >> to a task. Since Tags have references to their tasks, they are >> modified as well when a Task is edited. So we should probably let the >> tag emit a signal to warn everyone they changed (much like the Task >> are doing now). Let the objets which care about it to register >> themselve to this signal through the requester. >> >> That would imply having a "on_tag_modified" in the browser that would >> trigger some job to perform on the browser (probably not much since, >> if I understood well, most of the filtering work is now done outside >> the browser) >> >>> If you want to contribute, there's a huge cleaning to do in the >>> browser.py >>> ! A lot of cruft should be removed (and I don't understand everything >>> there). >> >> I agree with what you say, but it would be more efficient if you could >> be more specific and point out some of these cruft. >> >>> >>> What does not work : >>> - tag filtering. They are implemented. We just need to do >>> "self.req.apply_filter(tagname)" when a particular tag is selected but > I >>> don't know where it is in browser.py and I'm a bit tired. (any help is >>> welcome here) >>> - drag-n-drop : code is commented out in tasktree.py. Maybe it will > work >>> without any trouble. >>> - multiparent (not tested yet) >>> - tag count in tagtree >>> >>> What is broken : >>> - plugins and plugin_api >>> - dbus interface >>> >>> >>> I was expecting better performances but FilteredTree appeared to be a > lot >>> more complex than expected. So now, it's time to optimize it a bit and >>> I'm >>> wondering where is the bottleneck. >> >> Like I wrote above, it seems to be situated around tags update >> mechanisms and TagTreeModel updates. >> >>> >>> Thanks for listenning, >>> >>> >>> Lionel >>> >>> _______________________________________________ >>> Mailing list: https://launchpad.net/~gtg-contributors >>> Post to : [email protected] >>> Unsubscribe : https://launchpad.net/~gtg-contributors >>> More help : https://help.launchpad.net/ListHelp >>> > -- Bertrand Rousseau _______________________________________________ Mailing list: https://launchpad.net/~gtg-contributors Post to : [email protected] Unsubscribe : https://launchpad.net/~gtg-contributors More help : https://help.launchpad.net/ListHelp
