Re: [Zim-wiki] way slow on Ubuntu
On May 12, 2014 4:20 PM, wrote: > > YES! > > Setting it to "shared" - I also deleted the .zim structure and > re-indexed - makes things much snappier - I still see the python > processes hitting the CPU quite a bit, not quite pegging but close, > but the UI gives me the new pages when navigating, creating links etc > in a second or two. > > Only tested for a few minutes so far, but so far so good. . . > > BTW just out of curiosity, where are the indexes now? They end up in XDG_CACHE_HOME which defaults to ~/.cache > > On Mon, May 12, 2014 at 8:54 AM, Jaap Karssenberg > wrote: > > > > Hans, > > > > In general zim should not have issues handling a few thousand files - it is > > built rather scalable. However there may be some issue where the way zim > > accesses the drive conflicts with assumptions in encfs. > > > > One thing that comes to mind is to set the "shared" property - be aware: > > this will cache the index outside of the encfs folder. See if the index > > access is the bottleneck here. > > > > Otherwise we would need to think how to insert timers to figure out the > > exact piece of code that causes the delay. > > > > Regards, > > > > Jaap > > > > > > > > > > On Fri, May 9, 2014 at 5:11 PM, wrote: > >> > >> files - 1100 including attachments (docroot), 860 just for the wiki > >> > >> folders - 280 total, 200 just for the wiki > >> > >> timedstats for *.txt > >> 16673 69699 590562 total > >> > >> real 0m0.439s > >> user 0m0.059s > >> sys 0m0.064s > >> > >> I don't do inline attachments or pasted images at all, everything not > >> *.txt is in the docroot tree > >> > >> I have no trouble at all accessing the tree with other tools, only in > >> Zim, in System Monitor or htop can see the Python processes spiking on > >> both processors to 100% while the graying out occurs. > >> > >> Shutting down everything python and then re-starting doesn't help, > >> symptom can occur right after a restart. > >> > >> > >> On Fri, May 9, 2014 at 10:50 AM, Sylvain Viart > >> wrote: > >> > Hi Hans, > >> > > >> > I guess… > >> > > >> > Le 09/05/2014 16:03, hans...@gmail.com a écrit : > >> > > >> >> Just created a new notebook and WOW is that nice and snappy. > >> >> > >> >> Confirmed it wasn't the fact it was on an encfs filesystem, nor that > >> >> dropbox is sync'ing while things are unencrypted > >> >> > >> >> I guess I just let my "main" tree get too large? > >> > > >> > > >> > could you give use a idea of your notebook size? > >> > > >> > # all files: > >> > $ find ~/Notebooks/Notes/ | wc -l > >> > 404 > >> > # folders: > >> > $ find ~/Notebooks/Notes/ -type d | wc -l > >> > 79 > >> > # pasted images > >> > $ find ~/Notebooks/Notes/ -name pasted_image\* | wc -l > >> > 43 > >> > # stats of all .txt files (lines words character) > >> > $ find ~/Notebooks/Notes/ -name \*.txt -print0 | wc --files0-from=- | > >> > grep > >> > 'total$' > >> > 12162 65938 523762 total > >> > > >> > You can prefix command with time: > >> > > >> > time find ~/Notebooks/Notes/ -name \*.txt -print0 | wc --files0-from=- | > >> > grep 'total$' > >> > 12162 65938 523762 total > >> > > >> > real0m0.081s > >> > user0m0.060s > >> > sys0m0.024s > >> > > >> > > >> > Regards, > >> > Sylvain. > >> > > >> > > >> > > >> > > >> > ___ > >> > Mailing list: https://launchpad.net/~zim-wiki > >> > Post to : zim-wiki@lists.launchpad.net > >> > Unsubscribe : https://launchpad.net/~zim-wiki > >> > More help : https://help.launchpad.net/ListHelp > >> > >> ___ > >> Mailing list: https://launchpad.net/~zim-wiki > >> Post to : zim-wiki@lists.launchpad.net > >> Unsubscribe : https://launchpad.net/~zim-wiki > >> More help : https://help.launchpad.net/ListHelp > > > > ___ Mailing list: https://launchpad.net/~zim-wiki Post to : zim-wiki@lists.launchpad.net Unsubscribe : https://launchpad.net/~zim-wiki More help : https://help.launchpad.net/ListHelp
Re: [Zim-wiki] way slow on Ubuntu
YES! Setting it to "shared" - I also deleted the .zim structure and re-indexed - makes things much snappier - I still see the python processes hitting the CPU quite a bit, not quite pegging but close, but the UI gives me the new pages when navigating, creating links etc in a second or two. Only tested for a few minutes so far, but so far so good. . . BTW just out of curiosity, where are the indexes now? On Mon, May 12, 2014 at 8:54 AM, Jaap Karssenberg wrote: > > Hans, > > In general zim should not have issues handling a few thousand files - it is > built rather scalable. However there may be some issue where the way zim > accesses the drive conflicts with assumptions in encfs. > > One thing that comes to mind is to set the "shared" property - be aware: > this will cache the index outside of the encfs folder. See if the index > access is the bottleneck here. > > Otherwise we would need to think how to insert timers to figure out the > exact piece of code that causes the delay. > > Regards, > > Jaap > > > > > On Fri, May 9, 2014 at 5:11 PM, wrote: >> >> files - 1100 including attachments (docroot), 860 just for the wiki >> >> folders - 280 total, 200 just for the wiki >> >> timedstats for *.txt >> 16673 69699 590562 total >> >> real 0m0.439s >> user 0m0.059s >> sys 0m0.064s >> >> I don't do inline attachments or pasted images at all, everything not >> *.txt is in the docroot tree >> >> I have no trouble at all accessing the tree with other tools, only in >> Zim, in System Monitor or htop can see the Python processes spiking on >> both processors to 100% while the graying out occurs. >> >> Shutting down everything python and then re-starting doesn't help, >> symptom can occur right after a restart. >> >> >> On Fri, May 9, 2014 at 10:50 AM, Sylvain Viart >> wrote: >> > Hi Hans, >> > >> > I guess… >> > >> > Le 09/05/2014 16:03, hans...@gmail.com a écrit : >> > >> >> Just created a new notebook and WOW is that nice and snappy. >> >> >> >> Confirmed it wasn't the fact it was on an encfs filesystem, nor that >> >> dropbox is sync'ing while things are unencrypted >> >> >> >> I guess I just let my "main" tree get too large? >> > >> > >> > could you give use a idea of your notebook size? >> > >> > # all files: >> > $ find ~/Notebooks/Notes/ | wc -l >> > 404 >> > # folders: >> > $ find ~/Notebooks/Notes/ -type d | wc -l >> > 79 >> > # pasted images >> > $ find ~/Notebooks/Notes/ -name pasted_image\* | wc -l >> > 43 >> > # stats of all .txt files (lines words character) >> > $ find ~/Notebooks/Notes/ -name \*.txt -print0 | wc --files0-from=- | >> > grep >> > 'total$' >> > 12162 65938 523762 total >> > >> > You can prefix command with time: >> > >> > time find ~/Notebooks/Notes/ -name \*.txt -print0 | wc --files0-from=- | >> > grep 'total$' >> > 12162 65938 523762 total >> > >> > real0m0.081s >> > user0m0.060s >> > sys0m0.024s >> > >> > >> > Regards, >> > Sylvain. >> > >> > >> > >> > >> > ___ >> > Mailing list: https://launchpad.net/~zim-wiki >> > Post to : zim-wiki@lists.launchpad.net >> > Unsubscribe : https://launchpad.net/~zim-wiki >> > More help : https://help.launchpad.net/ListHelp >> >> ___ >> Mailing list: https://launchpad.net/~zim-wiki >> Post to : zim-wiki@lists.launchpad.net >> Unsubscribe : https://launchpad.net/~zim-wiki >> More help : https://help.launchpad.net/ListHelp > > ___ Mailing list: https://launchpad.net/~zim-wiki Post to : zim-wiki@lists.launchpad.net Unsubscribe : https://launchpad.net/~zim-wiki More help : https://help.launchpad.net/ListHelp
Re: [Zim-wiki] way slow on Ubuntu
Hans, In general zim should not have issues handling a few thousand files - it is built rather scalable. However there may be some issue where the way zim accesses the drive conflicts with assumptions in encfs. One thing that comes to mind is to set the "shared" property - be aware: this will cache the index outside of the encfs folder. See if the index access is the bottleneck here. Otherwise we would need to think how to insert timers to figure out the exact piece of code that causes the delay. Regards, Jaap On Fri, May 9, 2014 at 5:11 PM, wrote: > files - 1100 including attachments (docroot), 860 just for the wiki > > folders - 280 total, 200 just for the wiki > > timedstats for *.txt > 16673 69699 590562 total > > real 0m0.439s > user 0m0.059s > sys 0m0.064s > > I don't do inline attachments or pasted images at all, everything not > *.txt is in the docroot tree > > I have no trouble at all accessing the tree with other tools, only in > Zim, in System Monitor or htop can see the Python processes spiking on > both processors to 100% while the graying out occurs. > > Shutting down everything python and then re-starting doesn't help, > symptom can occur right after a restart. > > > On Fri, May 9, 2014 at 10:50 AM, Sylvain Viart > wrote: > > Hi Hans, > > > > I guess… > > > > Le 09/05/2014 16:03, hans...@gmail.com a écrit : > > > >> Just created a new notebook and WOW is that nice and snappy. > >> > >> Confirmed it wasn't the fact it was on an encfs filesystem, nor that > >> dropbox is sync'ing while things are unencrypted > >> > >> I guess I just let my "main" tree get too large? > > > > > > could you give use a idea of your notebook size? > > > > # all files: > > $ find ~/Notebooks/Notes/ | wc -l > > 404 > > # folders: > > $ find ~/Notebooks/Notes/ -type d | wc -l > > 79 > > # pasted images > > $ find ~/Notebooks/Notes/ -name pasted_image\* | wc -l > > 43 > > # stats of all .txt files (lines words character) > > $ find ~/Notebooks/Notes/ -name \*.txt -print0 | wc --files0-from=- | > grep > > 'total$' > > 12162 65938 523762 total > > > > You can prefix command with time: > > > > time find ~/Notebooks/Notes/ -name \*.txt -print0 | wc --files0-from=- | > > grep 'total$' > > 12162 65938 523762 total > > > > real0m0.081s > > user0m0.060s > > sys0m0.024s > > > > > > Regards, > > Sylvain. > > > > > > > > > > ___ > > Mailing list: https://launchpad.net/~zim-wiki > > Post to : zim-wiki@lists.launchpad.net > > Unsubscribe : https://launchpad.net/~zim-wiki > > More help : https://help.launchpad.net/ListHelp > > ___ > Mailing list: https://launchpad.net/~zim-wiki > Post to : zim-wiki@lists.launchpad.net > Unsubscribe : https://launchpad.net/~zim-wiki > More help : https://help.launchpad.net/ListHelp > ___ Mailing list: https://launchpad.net/~zim-wiki Post to : zim-wiki@lists.launchpad.net Unsubscribe : https://launchpad.net/~zim-wiki More help : https://help.launchpad.net/ListHelp
Re: [Zim-wiki] way slow on Ubuntu
files - 1100 including attachments (docroot), 860 just for the wiki folders - 280 total, 200 just for the wiki timedstats for *.txt 16673 69699 590562 total real 0m0.439s user 0m0.059s sys 0m0.064s I don't do inline attachments or pasted images at all, everything not *.txt is in the docroot tree I have no trouble at all accessing the tree with other tools, only in Zim, in System Monitor or htop can see the Python processes spiking on both processors to 100% while the graying out occurs. Shutting down everything python and then re-starting doesn't help, symptom can occur right after a restart. On Fri, May 9, 2014 at 10:50 AM, Sylvain Viart wrote: > Hi Hans, > > I guess… > > Le 09/05/2014 16:03, hans...@gmail.com a écrit : > >> Just created a new notebook and WOW is that nice and snappy. >> >> Confirmed it wasn't the fact it was on an encfs filesystem, nor that >> dropbox is sync'ing while things are unencrypted >> >> I guess I just let my "main" tree get too large? > > > could you give use a idea of your notebook size? > > # all files: > $ find ~/Notebooks/Notes/ | wc -l > 404 > # folders: > $ find ~/Notebooks/Notes/ -type d | wc -l > 79 > # pasted images > $ find ~/Notebooks/Notes/ -name pasted_image\* | wc -l > 43 > # stats of all .txt files (lines words character) > $ find ~/Notebooks/Notes/ -name \*.txt -print0 | wc --files0-from=- | grep > 'total$' > 12162 65938 523762 total > > You can prefix command with time: > > time find ~/Notebooks/Notes/ -name \*.txt -print0 | wc --files0-from=- | > grep 'total$' > 12162 65938 523762 total > > real0m0.081s > user0m0.060s > sys0m0.024s > > > Regards, > Sylvain. > > > > > ___ > Mailing list: https://launchpad.net/~zim-wiki > Post to : zim-wiki@lists.launchpad.net > Unsubscribe : https://launchpad.net/~zim-wiki > More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~zim-wiki Post to : zim-wiki@lists.launchpad.net Unsubscribe : https://launchpad.net/~zim-wiki More help : https://help.launchpad.net/ListHelp
Re: [Zim-wiki] way slow on Ubuntu
Hi Hans, I guess… Le 09/05/2014 16:03, hans...@gmail.com a écrit : Just created a new notebook and WOW is that nice and snappy. Confirmed it wasn't the fact it was on an encfs filesystem, nor that dropbox is sync'ing while things are unencrypted I guess I just let my "main" tree get too large? could you give use a idea of your notebook size? # all files: $ find ~/Notebooks/Notes/ | wc -l 404 # folders: $ find ~/Notebooks/Notes/ -type d | wc -l 79 # pasted images $ find ~/Notebooks/Notes/ -name pasted_image\* | wc -l 43 # stats of all .txt files (lines words character) $ find ~/Notebooks/Notes/ -name \*.txt -print0 | wc --files0-from=- | grep 'total$' 12162 65938 523762 total You can prefix command with time: time find ~/Notebooks/Notes/ -name \*.txt -print0 | wc --files0-from=- | grep 'total$' 12162 65938 523762 total real0m0.081s user0m0.060s sys0m0.024s Regards, Sylvain. ___ Mailing list: https://launchpad.net/~zim-wiki Post to : zim-wiki@lists.launchpad.net Unsubscribe : https://launchpad.net/~zim-wiki More help : https://help.launchpad.net/ListHelp
Re: [Zim-wiki] way slow on Ubuntu
Just created a new notebook and WOW is that nice and snappy. Confirmed it wasn't the fact it was on an encfs filesystem, nor that dropbox is sync'ing while things are unencrypted I guess I just let my "main" tree get too large? Are there any guidelines as to number of nodes, size of files where things start to bog down like that? Really has been getting pretty much unusable, sometimes over a minute of "greying out" to move to a different node, sometimes another cycle to get the cursor back ready to insert new text into the file. . . I guess I'll just need to regularly parse out from an old to a new notebook. Or maybe I'll try having separate "sub-notebooks" at the head of particularly large sub-topic trees. I'm also starting to research using Org-mode in Emacs - anyone have suggestions of good package combos to give Zim-like functionality there? On Tue, Apr 29, 2014 at 4:18 PM, Jaap Karssenberg wrote: > > On Thu, Apr 17, 2014 at 11:37 PM, wrote: >> >> OK, didn't reboot but shutdown Zim, killed Python processes still >> running, then launched from console >> zim --debug >> >> Here's the output: http://gist.github.com/anonymous/11012787 >> >> Note my .config under ~/ as well as the Zim/python installs are on a >> normal ext3 filesystem, but looking at this reminded me my data folder >> is encrypted with encfs and (once decrypted) sync'd via Dropbox. >> >> I have tested after killing the Dropbox process, no change. >> >> Next step - if you guys don't see anything in the gist - will be to >> move the data folder and test again. >> >> If that is the problem, then suggestions for an alternative >> user-transparent encryption solution that doesn't slow things down >> would be greatly appreciated. >> >> PS thanks so much to the developer(s) and other contributors for this >> excellent software, finally weaned myself away from Evernote after >> nearly a decade's addiction. > > > The debug log looks entirely normal, no hints to be found there. > > One thing that might be happening is that the index update that is running > after starting zim is having to do a lot of maintenance. When you start zim, > wait a bit until you see a debug message that the index update is done > before trying other stuff. Or you can force an index update by running "zim > --index" after closing the gui. > > THis stuff of course depends on the number of pages, lenght of pages etc. > But unless those numbers are really ridiculously large it should resolve > itself. > > If that does not help I would indeed think about the filesystem speed, or > other things that could interact. > > Regards, > > Jaap > On Tue, Apr 29, 2014 at 4:18 PM, Jaap Karssenberg wrote: > > On Thu, Apr 17, 2014 at 11:37 PM, wrote: >> >> OK, didn't reboot but shutdown Zim, killed Python processes still >> running, then launched from console >> zim --debug >> >> Here's the output: http://gist.github.com/anonymous/11012787 >> >> Note my .config under ~/ as well as the Zim/python installs are on a >> normal ext3 filesystem, but looking at this reminded me my data folder >> is encrypted with encfs and (once decrypted) sync'd via Dropbox. >> >> I have tested after killing the Dropbox process, no change. >> >> Next step - if you guys don't see anything in the gist - will be to >> move the data folder and test again. >> >> If that is the problem, then suggestions for an alternative >> user-transparent encryption solution that doesn't slow things down >> would be greatly appreciated. >> >> PS thanks so much to the developer(s) and other contributors for this >> excellent software, finally weaned myself away from Evernote after >> nearly a decade's addiction. > > > The debug log looks entirely normal, no hints to be found there. > > One thing that might be happening is that the index update that is running > after starting zim is having to do a lot of maintenance. When you start zim, > wait a bit until you see a debug message that the index update is done > before trying other stuff. Or you can force an index update by running "zim > --index" after closing the gui. > > THis stuff of course depends on the number of pages, lenght of pages etc. > But unless those numbers are really ridiculously large it should resolve > itself. > > If that does not help I would indeed think about the filesystem speed, or > other things that could interact. > > Regards, > > Jaap > ___ Mailing list: https://launchpad.net/~zim-wiki Post to : zim-wiki@lists.launchpad.net Unsubscribe : https://launchpad.net/~zim-wiki More help : https://help.launchpad.net/ListHelp
Re: [Zim-wiki] way slow on Ubuntu
On Thu, Apr 17, 2014 at 11:37 PM, wrote: > OK, didn't reboot but shutdown Zim, killed Python processes still > running, then launched from console > zim --debug > > Here's the output: http://gist.github.com/anonymous/11012787 > > Note my .config under ~/ as well as the Zim/python installs are on a > normal ext3 filesystem, but looking at this reminded me my data folder > is encrypted with encfs and (once decrypted) sync'd via Dropbox. > > I have tested after killing the Dropbox process, no change. > > Next step - if you guys don't see anything in the gist - will be to > move the data folder and test again. > > If that is the problem, then suggestions for an alternative > user-transparent encryption solution that doesn't slow things down > would be greatly appreciated. > > PS thanks so much to the developer(s) and other contributors for this > excellent software, finally weaned myself away from Evernote after > nearly a decade's addiction. > The debug log looks entirely normal, no hints to be found there. One thing that might be happening is that the index update that is running after starting zim is having to do a lot of maintenance. When you start zim, wait a bit until you see a debug message that the index update is done before trying other stuff. Or you can force an index update by running "zim --index" after closing the gui. THis stuff of course depends on the number of pages, lenght of pages etc. But unless those numbers are really ridiculously large it should resolve itself. If that does not help I would indeed think about the filesystem speed, or other things that could interact. Regards, Jaap ___ Mailing list: https://launchpad.net/~zim-wiki Post to : zim-wiki@lists.launchpad.net Unsubscribe : https://launchpad.net/~zim-wiki More help : https://help.launchpad.net/ListHelp
Re: [Zim-wiki] way slow on Ubuntu
OK, didn't reboot but shutdown Zim, killed Python processes still running, then launched from console zim --debug Here's the output: http://gist.github.com/anonymous/11012787 Note my .config under ~/ as well as the Zim/python installs are on a normal ext3 filesystem, but looking at this reminded me my data folder is encrypted with encfs and (once decrypted) sync'd via Dropbox. I have tested after killing the Dropbox process, no change. Next step - if you guys don't see anything in the gist - will be to move the data folder and test again. If that is the problem, then suggestions for an alternative user-transparent encryption solution that doesn't slow things down would be greatly appreciated. PS thanks so much to the developer(s) and other contributors for this excellent software, finally weaned myself away from Evernote after nearly a decade's addiction. > On Tue, Apr 15, 2014, at 15:22, hans...@gmail.com wrote: >> Zim .60 via PPA, both with and without the tray icon runnung >> >> 12.04 LTS Unity, Python 2.7.3 >> >> Clicking on another node, or even just putting focus on a page already >> selected, everything just hangs for 5-20 seconds at a time, the Zim >> window goes grey/blue, CPU pegs on python during that time? >> >> Thought I'd try the dev repo - noob-friendly instructions? >> >> Or maybe going to a newer Python? > > Thank you for reaching out to the mailing list on this Hans. Can you try > starting Zim from a terminal with the command > >zim --debug > > That will print chatty debug messages to the console that you can look > at and maybe share. It might help identify exactly what step in > navigation is getting hung up. > > Also, is your home folder (your user profile folder) stored on local > storage, or NFS, or something more exotic? > > Brendan Kidwell ___ Mailing list: https://launchpad.net/~zim-wiki Post to : zim-wiki@lists.launchpad.net Unsubscribe : https://launchpad.net/~zim-wiki More help : https://help.launchpad.net/ListHelp
Re: [Zim-wiki] way slow on Ubuntu
PLEASE DISREGARD MY PREVIOUS MESSAGE. My mail client replied with the wrong From address and it'll be rejected by the mailing list. On Tue, Apr 15, 2014, at 15:22, hans...@gmail.com wrote: > Zim .60 via PPA, both with and without the tray icon runnung > > 12.04 LTS Unity, Python 2.7.3 > > Clicking on another node, or even just putting focus on a page already > selected, everything just hangs for 5-20 seconds at a time, the Zim > window goes grey/blue, CPU pegs on python during that time? > > Thought I'd try the dev repo - noob-friendly instructions? > > Or maybe going to a newer Python? Thank you for reaching out to the mailing list on this Hans. Can you try starting Zim from a terminal with the command zim --debug That will print chatty debug messages to the console that you can look at and maybe share. It might help identify exactly what step in navigation is getting hung up. Also, is your home folder (your user profile folder) stored on local storage, or NFS, or something more exotic? Brendan Kidwell ___ Mailing list: https://launchpad.net/~zim-wiki Post to : zim-wiki@lists.launchpad.net Unsubscribe : https://launchpad.net/~zim-wiki More help : https://help.launchpad.net/ListHelp
Re: [Zim-wiki] way slow on Ubuntu
On Tue, Apr 15, 2014, at 15:22, hans...@gmail.com wrote: > Zim .60 via PPA, both with and without the tray icon runnung > > 12.04 LTS Unity, Python 2.7.3 > > Clicking on another node, or even just putting focus on a page already > selected, everything just hangs for 5-20 seconds at a time, the Zim > window goes grey/blue, CPU pegs on python during that time? > > Thought I'd try the dev repo - noob-friendly instructions? > > Or maybe going to a newer Python? Thank you for reaching out to the mailing list on this Hans. Can you try starting Zim from a terminal with the command zim --debug That will print chatty debug messages to the console that you can look at and maybe share. It might help identify exactly what step in navigation is getting hung up. Also, is your home folder (your user profile folder) stored on local storage, or NFS, or something more exotic? Brendan Kidwell ___ Mailing list: https://launchpad.net/~zim-wiki Post to : zim-wiki@lists.launchpad.net Unsubscribe : https://launchpad.net/~zim-wiki More help : https://help.launchpad.net/ListHelp
[Zim-wiki] way slow on Ubuntu
Zim .60 via PPA, both with and without the tray icon runnung 12.04 LTS Unity, Python 2.7.3 Clicking on another node, or even just putting focus on a page already selected, everything just hangs for 5-20 seconds at a time, the Zim window goes grey/blue, CPU pegs on python during that time? Thought I'd try the dev repo - noob-friendly instructions? Or maybe going to a newer Python? ___ Mailing list: https://launchpad.net/~zim-wiki Post to : zim-wiki@lists.launchpad.net Unsubscribe : https://launchpad.net/~zim-wiki More help : https://help.launchpad.net/ListHelp
