Re: [O] Very slow performance in Org-mode on 10k line file?
On Wed, Aug 7, 2013 at 4:39 PM, Rainer Stengele rainer.steng...@online.de wrote: Am 07.08.2013 22:25, schrieb John Hendy: Greetings, I just started experiencing major lag in Org-mode on my main work notes file, which is at about 10k lines. Is that getting up to the point where files get unwieldy? In googling around, I found a few suggestions: - Fiddle with linum settings http://stackoverflow.com/questions/5229705/emacs-org-mode-turn-off-line-numbers I set linum-eager to off and linum-delay to on for the current setting via the customize interface and didn't perceive an effect. Any keystroke in my org file takes 1-2 seconds to appear. - Fontification? http://comments.gmane.org/gmane.emacs.orgmode/45197 Comments there have files in the 5-137k line range and many say there's no/little lag unless running agenda commands. Any other suggestions? I use this file almost daily, mostly for reference, not adding... that's to say it hasn't grown majorly in the past even 3months (maybe a few hundred lines), but performance *definitely* wasn't anything like this until the last week or so. Thanks for any suggestions on improving or tracking down the source. In the mean time, I'm going to revert to a few git commits ago and see if that does anything for me. Best regards, John Hi, just jumping on the bandwagon. My one and only biggest issue with wonderful Orgmode is slow Emacs. I run Emacs on i7 hardware with lots of memory and still have an Emacs Orgmode that answers rather slowly. Slow meaning it is just not snappy at all. Creating any aganda takes several seconds which is a long time to wait for the result. I already spent a lot of thought into how to optimise the performance of my system, archiving and splitting Org files, using sticky agenda etc. I know there is no quick solution to the slowness because of the limitation of threading in Emacs - I just wanted to mention that very unmodern behaviour of Emacs running Orgmode. I have to use Windows 7 so this makes it even slower. I assume my environment would run faster on Linux. I'm running Arch Linux 64bit on an HP EliteBook 8540w with an i7 and 8G of RAM. HD is 54% full at present. I agree that this shouldn't be a big burden, and whatever happened recently really made this unusable for me. Oddly, generating an agenda only takes a couple of seconds, which is plenty fast for me, even using search. So yes, this is nothing more than something like a rant. For me Orgmode is still the killer app in Emacs, it is just sad to have a slow environment on quite modern hardware with some bigger Org files. My files are of size: $ wc *org 1241690 31670 file1.org 1555 11829 97805 file2.org 35022 262820 2314234 file3.org 9994968 105854 file4.org 5574029 30586 file5.org 2523 20324 162165 file6.org 2447 19974 139768 file7.org 6894703 36495 file8.org 6789 58782 461211 file9.org 53078 403126 3531142 total $ wc *.org 23 90867 bibliography.org 42192 1756 clocking.org 2137 18286 122303 devel.org 9837 74994 494234 projects.org 1536 9692 77261 reference.org 1057 6673 48309 tf.org 14632 109927 744730 total projects.org is my most used file by far, and the biggest, but nothing compared to some of the folks posting on the link I showed who have 30-130k line files! John Rainer
Re: [O] Very slow performance in Org-mode on 10k line file?
Just an update: - I reverted back to commit release_8.0.2-73-g9998f2 (early May) with no change in behavior, so perhaps it wasn't anything other than growing file size - I just created archive files for work journal entries in 2011 and 2012, storing them in separate archive files - I'm now down to ~6500 lines, and lag is unnoticeable Perhaps there's some magical cutoff between 6,000 and 10,000 lines that starts to really bog things down? John On Wed, Aug 7, 2013 at 4:47 PM, John Hendy jw.he...@gmail.com wrote: On Wed, Aug 7, 2013 at 4:39 PM, Rainer Stengele rainer.steng...@online.de wrote: Am 07.08.2013 22:25, schrieb John Hendy: Greetings, I just started experiencing major lag in Org-mode on my main work notes file, which is at about 10k lines. Is that getting up to the point where files get unwieldy? In googling around, I found a few suggestions: - Fiddle with linum settings http://stackoverflow.com/questions/5229705/emacs-org-mode-turn-off-line-numbers I set linum-eager to off and linum-delay to on for the current setting via the customize interface and didn't perceive an effect. Any keystroke in my org file takes 1-2 seconds to appear. - Fontification? http://comments.gmane.org/gmane.emacs.orgmode/45197 Comments there have files in the 5-137k line range and many say there's no/little lag unless running agenda commands. Any other suggestions? I use this file almost daily, mostly for reference, not adding... that's to say it hasn't grown majorly in the past even 3months (maybe a few hundred lines), but performance *definitely* wasn't anything like this until the last week or so. Thanks for any suggestions on improving or tracking down the source. In the mean time, I'm going to revert to a few git commits ago and see if that does anything for me. Best regards, John Hi, just jumping on the bandwagon. My one and only biggest issue with wonderful Orgmode is slow Emacs. I run Emacs on i7 hardware with lots of memory and still have an Emacs Orgmode that answers rather slowly. Slow meaning it is just not snappy at all. Creating any aganda takes several seconds which is a long time to wait for the result. I already spent a lot of thought into how to optimise the performance of my system, archiving and splitting Org files, using sticky agenda etc. I know there is no quick solution to the slowness because of the limitation of threading in Emacs - I just wanted to mention that very unmodern behaviour of Emacs running Orgmode. I have to use Windows 7 so this makes it even slower. I assume my environment would run faster on Linux. I'm running Arch Linux 64bit on an HP EliteBook 8540w with an i7 and 8G of RAM. HD is 54% full at present. I agree that this shouldn't be a big burden, and whatever happened recently really made this unusable for me. Oddly, generating an agenda only takes a couple of seconds, which is plenty fast for me, even using search. So yes, this is nothing more than something like a rant. For me Orgmode is still the killer app in Emacs, it is just sad to have a slow environment on quite modern hardware with some bigger Org files. My files are of size: $ wc *org 1241690 31670 file1.org 1555 11829 97805 file2.org 35022 262820 2314234 file3.org 9994968 105854 file4.org 5574029 30586 file5.org 2523 20324 162165 file6.org 2447 19974 139768 file7.org 6894703 36495 file8.org 6789 58782 461211 file9.org 53078 403126 3531142 total $ wc *.org 23 90867 bibliography.org 42192 1756 clocking.org 2137 18286 122303 devel.org 9837 74994 494234 projects.org 1536 9692 77261 reference.org 1057 6673 48309 tf.org 14632 109927 744730 total projects.org is my most used file by far, and the biggest, but nothing compared to some of the folks posting on the link I showed who have 30-130k line files! John Rainer
Re: [O] Very slow performance in Org-mode on 10k line file?
On Wed, Aug 7, 2013 at 5:12 PM, Russell Adams rlad...@adamsinfoserv.com wrote: John, I have a 17,000 file I work out of constantly in Org 7.8.10 with very little lag. Just another data point. Is it inconvenient for you to git pull and try on Org 8.0 to see if there's any difference? John Thanks. On Wed, Aug 07, 2013 at 05:06:51PM -0500, John Hendy wrote: Just an update: - I reverted back to commit release_8.0.2-73-g9998f2 (early May) with no change in behavior, so perhaps it wasn't anything other than growing file size - I just created archive files for work journal entries in 2011 and 2012, storing them in separate archive files - I'm now down to ~6500 lines, and lag is unnoticeable Perhaps there's some magical cutoff between 6,000 and 10,000 lines that starts to really bog things down? John On Wed, Aug 7, 2013 at 4:47 PM, John Hendy jw.he...@gmail.com wrote: On Wed, Aug 7, 2013 at 4:39 PM, Rainer Stengele rainer.steng...@online.de wrote: Am 07.08.2013 22:25, schrieb John Hendy: Greetings, I just started experiencing major lag in Org-mode on my main work notes file, which is at about 10k lines. Is that getting up to the point where files get unwieldy? In googling around, I found a few suggestions: - Fiddle with linum settings http://stackoverflow.com/questions/5229705/emacs-org-mode-turn-off-line-numbers I set linum-eager to off and linum-delay to on for the current setting via the customize interface and didn't perceive an effect. Any keystroke in my org file takes 1-2 seconds to appear. - Fontification? http://comments.gmane.org/gmane.emacs.orgmode/45197 Comments there have files in the 5-137k line range and many say there's no/little lag unless running agenda commands. Any other suggestions? I use this file almost daily, mostly for reference, not adding... that's to say it hasn't grown majorly in the past even 3months (maybe a few hundred lines), but performance *definitely* wasn't anything like this until the last week or so. Thanks for any suggestions on improving or tracking down the source. In the mean time, I'm going to revert to a few git commits ago and see if that does anything for me. Best regards, John Hi, just jumping on the bandwagon. My one and only biggest issue with wonderful Orgmode is slow Emacs. I run Emacs on i7 hardware with lots of memory and still have an Emacs Orgmode that answers rather slowly. Slow meaning it is just not snappy at all. Creating any aganda takes several seconds which is a long time to wait for the result. I already spent a lot of thought into how to optimise the performance of my system, archiving and splitting Org files, using sticky agenda etc. I know there is no quick solution to the slowness because of the limitation of threading in Emacs - I just wanted to mention that very unmodern behaviour of Emacs running Orgmode. I have to use Windows 7 so this makes it even slower. I assume my environment would run faster on Linux. I'm running Arch Linux 64bit on an HP EliteBook 8540w with an i7 and 8G of RAM. HD is 54% full at present. I agree that this shouldn't be a big burden, and whatever happened recently really made this unusable for me. Oddly, generating an agenda only takes a couple of seconds, which is plenty fast for me, even using search. So yes, this is nothing more than something like a rant. For me Orgmode is still the killer app in Emacs, it is just sad to have a slow environment on quite modern hardware with some bigger Org files. My files are of size: $ wc *org 1241690 31670 file1.org 1555 11829 97805 file2.org 35022 262820 2314234 file3.org 9994968 105854 file4.org 5574029 30586 file5.org 2523 20324 162165 file6.org 2447 19974 139768 file7.org 6894703 36495 file8.org 6789 58782 461211 file9.org 53078 403126 3531142 total $ wc *.org 23 90867 bibliography.org 42192 1756 clocking.org 2137 18286 122303 devel.org 9837 74994 494234 projects.org 1536 9692 77261 reference.org 1057 6673 48309 tf.org 14632 109927 744730 total projects.org is my most used file by far, and the biggest, but nothing compared to some of the folks posting on the link I showed who have 30-130k line files! John Rainer -- Russell Adamsrlad...@adamsinfoserv.com PGP Key ID: 0x1160DCB3 http://www.adamsinfoserv.com/ Fingerprint:1723 D8CA 4280 1EC9 557F 66E8 1154 E018 1160 DCB3
Re: [O] Very slow performance in Org-mode on 10k line file?
On Wed, Aug 7, 2013 at 5:22 PM, Nick Dokos ndo...@gmail.com wrote: John Hendy jw.he...@gmail.com writes: ... Thanks for any suggestions on improving or tracking down the source. In the mean time, I'm going to revert to a few git commits ago and see if that does anything for me. That's definitely a good idea: it sounds as if something got worse suddenly so it may have been a minor change. Well, see my follow-up email. Reverting back to early May did not change the situation. Before you try going back, can you also try to get a profile of a slow operation and then repeat the profile on the same operation with the earlier org? M-x elp-instrument-package org M-x elp-reset-all run your workload M-x elp-results Would it help to do this on a 6k file vs. a 10k file? Reducing my file size made a huge difference, so if those results would be of interest/help, I can definitely do that? John should be enough as a first approximation. -- Nick
Re: [O] Very slow performance in Org-mode on 10k line file?
John Hendy jw.he...@gmail.com writes: ... Thanks for any suggestions on improving or tracking down the source. In the mean time, I'm going to revert to a few git commits ago and see if that does anything for me. That's definitely a good idea: it sounds as if something got worse suddenly so it may have been a minor change. Before you try going back, can you also try to get a profile of a slow operation and then repeat the profile on the same operation with the earlier org? M-x elp-instrument-package org M-x elp-reset-all run your workload M-x elp-results should be enough as a first approximation. -- Nick
Re: [O] Very slow performance in Org-mode on 10k line file?
John Hendy jw.he...@gmail.com writes: M-x elp-instrument-package org M-x elp-reset-all run your workload M-x elp-results Would it help to do this on a 6k file vs. a 10k file? Reducing my file size made a huge difference, so if those results would be of interest/help, I can definitely do that? The more data the better, so yes, I think it's a useful exercise. -- Nick
Re: [O] Very slow performance in Org-mode on 10k line file?
John, I have a 17,000 file I work out of constantly in Org 7.8.10 with very little lag. Just another data point. Thanks. On Wed, Aug 07, 2013 at 05:06:51PM -0500, John Hendy wrote: Just an update: - I reverted back to commit release_8.0.2-73-g9998f2 (early May) with no change in behavior, so perhaps it wasn't anything other than growing file size - I just created archive files for work journal entries in 2011 and 2012, storing them in separate archive files - I'm now down to ~6500 lines, and lag is unnoticeable Perhaps there's some magical cutoff between 6,000 and 10,000 lines that starts to really bog things down? John On Wed, Aug 7, 2013 at 4:47 PM, John Hendy jw.he...@gmail.com wrote: On Wed, Aug 7, 2013 at 4:39 PM, Rainer Stengele rainer.steng...@online.de wrote: Am 07.08.2013 22:25, schrieb John Hendy: Greetings, I just started experiencing major lag in Org-mode on my main work notes file, which is at about 10k lines. Is that getting up to the point where files get unwieldy? In googling around, I found a few suggestions: - Fiddle with linum settings http://stackoverflow.com/questions/5229705/emacs-org-mode-turn-off-line-numbers I set linum-eager to off and linum-delay to on for the current setting via the customize interface and didn't perceive an effect. Any keystroke in my org file takes 1-2 seconds to appear. - Fontification? http://comments.gmane.org/gmane.emacs.orgmode/45197 Comments there have files in the 5-137k line range and many say there's no/little lag unless running agenda commands. Any other suggestions? I use this file almost daily, mostly for reference, not adding... that's to say it hasn't grown majorly in the past even 3months (maybe a few hundred lines), but performance *definitely* wasn't anything like this until the last week or so. Thanks for any suggestions on improving or tracking down the source. In the mean time, I'm going to revert to a few git commits ago and see if that does anything for me. Best regards, John Hi, just jumping on the bandwagon. My one and only biggest issue with wonderful Orgmode is slow Emacs. I run Emacs on i7 hardware with lots of memory and still have an Emacs Orgmode that answers rather slowly. Slow meaning it is just not snappy at all. Creating any aganda takes several seconds which is a long time to wait for the result. I already spent a lot of thought into how to optimise the performance of my system, archiving and splitting Org files, using sticky agenda etc. I know there is no quick solution to the slowness because of the limitation of threading in Emacs - I just wanted to mention that very unmodern behaviour of Emacs running Orgmode. I have to use Windows 7 so this makes it even slower. I assume my environment would run faster on Linux. I'm running Arch Linux 64bit on an HP EliteBook 8540w with an i7 and 8G of RAM. HD is 54% full at present. I agree that this shouldn't be a big burden, and whatever happened recently really made this unusable for me. Oddly, generating an agenda only takes a couple of seconds, which is plenty fast for me, even using search. So yes, this is nothing more than something like a rant. For me Orgmode is still the killer app in Emacs, it is just sad to have a slow environment on quite modern hardware with some bigger Org files. My files are of size: $ wc *org 1241690 31670 file1.org 1555 11829 97805 file2.org 35022 262820 2314234 file3.org 9994968 105854 file4.org 5574029 30586 file5.org 2523 20324 162165 file6.org 2447 19974 139768 file7.org 6894703 36495 file8.org 6789 58782 461211 file9.org 53078 403126 3531142 total $ wc *.org 23 90867 bibliography.org 42192 1756 clocking.org 2137 18286 122303 devel.org 9837 74994 494234 projects.org 1536 9692 77261 reference.org 1057 6673 48309 tf.org 14632 109927 744730 total projects.org is my most used file by far, and the biggest, but nothing compared to some of the folks posting on the link I showed who have 30-130k line files! John Rainer -- Russell Adamsrlad...@adamsinfoserv.com PGP Key ID: 0x1160DCB3 http://www.adamsinfoserv.com/ Fingerprint:1723 D8CA 4280 1EC9 557F 66E8 1154 E018 1160 DCB3
Re: [O] Very slow performance in Org-mode on 10k line file?
On Wed, Aug 07, 2013 at 05:17:53PM -0500, John Hendy wrote: Is it inconvenient for you to git pull and try on Org 8.0 to see if there's any difference? I'm not prepared to upgrade at this time. I just thought it odd the line length you were experiencing issues at compared to mine. Good luck! -- Russell Adamsrlad...@adamsinfoserv.com PGP Key ID: 0x1160DCB3 http://www.adamsinfoserv.com/ Fingerprint:1723 D8CA 4280 1EC9 557F 66E8 1154 E018 1160 DCB3
Re: [O] Very slow performance in Org-mode on 10k line file?
Alrighty. Here you are:
#+begin_src minimal emacs (which I've moved to ~/.emacs) and started a
fresh session
(add-to-list 'load-path ~/.elisp/org.git/contrib/lisp)
(add-to-list 'load-path ~/.elisp/org.git/lisp/)
#+end_src
Really simple operation, but that's that had huge lag on the 10k file.
All I did was delete a word and re-write it, and then insert a line
above a table (#+attr_latex: :align llp{7cm}). Here's the process:
- Start emacs (/usr/bin/emacs)
- Navigate to file (6k ~/org/projects.org file)
- Run: M-x elp-instrument-package [RET] org
- Run: M-x elp-reset-all
- Navigate to the headline I was trying to work on, delete a word a
character at a time, re-type it, add the above latex line above a
table
- Run: M-x elp-results
- Copied 2011 and 2012 journal entries back into projects.org, saved,
quit emacs, and repeated
Results with 6k line file
==
org-self-insert-command 41
2.026747151 0.0494328573
org-activate-dates 54
0.0175029940 0.0003241295
org-fontify-meta-lines-and-blocks 161
0.006726777 4.178...e-05
org-fontify-meta-lines-and-blocks-1 161
0.0047893350 2.974...e-05
org-delete-backward-char12
0.002401629 0.0002001357
org-activate-footnote-links 54
0.002203194 4.079...e-05
org-activate-plain-links54
0.0020726379 3.838...e-05
org-unfontify-region54
0.0020696900 3.832...e-05
org-at-table-p 54
0.0016839339 3.118...e-05
org-footnote-next-reference-or-definition 54
0.0015487719 2.868...e-05
org-do-latex-and-related54
0.0014039899 2.599...e-05
org-do-emphasis-faces 54
0.001028353 1.904...e-05
org-string-nw-p 54
0.0008700800 1.611...e-05
org-activate-angle-links54
0.0008193899 1.517...e-05
org-return 1
0.000758448 0.000758448
org-activate-tags 54
0.0005812419 1.076...e-05
org-in-item-p 1
0.000436519 0.000436519
org-activate-bracket-links 54
0.0004324309 8.007...e-06
org-activate-code 54
0.000416118 7.705...e-06
org-fix-tags-on-the-fly 53
0.000332257 6.269e-06
org-font-lock-add-priority-faces54
0.0003180110 5.889...e-06
org-string-match-p 54
0.000269684 4.994...e-06
org-remove-font-lock-display-properties 54
0.0002692999 4.987...e-06
org-list-context1
0.000268703 0.000268703
org-hide-wide-columns 54
0.000228154 4.225...e-06
org-before-change-function 54
0.0001231199 2.279...e-06
org-remove-flyspell-overlays-in 21
0.0001044190 4.972...e-06
org-font-lock-hook 54
8.081...e-05 1.496...e-06
org-check-before-invisible-edit 53
7.8635e-051.483...e-06
org-activate-target-links 54
7.542...e-05 1.396...e-06
org-fontify-entities54
6.752e-05 1.250...e-06
org-raise-scripts 54
6.415...e-05 1.188...e-06
org-font-lock-add-tag-faces 54
5.994...e-05 1.110...e-06
org-in-src-block-p 1
5.1549e-055.1549e-05
org-in-regexp 1
3.4068e-053.4068e-05
org-back-to-heading 1
3.3785e-053.3785e-05
org-get-limited-outline-regexp 1
2.1714e-052.1714e-05
org-at-heading-p1
1.3884e-051.3884e-05
org-get-indentation 2
1.337...e-05 6.689...e-06
org-item-re 1
4.116e-06 4.116e-06
==
Results with ~10k line file
==
org-self-insert-command 39
0.855967410.0219478823
org-cycle 6
0.107894177 0.0179823628
org-cycle-internal-local6
0.1058569380 0.0176428230
org-optimize-window-after-visibility-change 6
0.047916078 0.007986013
org-subtree-end-visible-p 5
0.046233295 0.009246659
org-end-of-subtree 27
0.029686826 0.0010995120
org-outline-level 394
0.0162884779 4.134...e-05
org-cycle-show-empty-lines 6
0.008189213 0.0013648688
org-fontify-meta-lines-and-blocks 182
0.0078306620 4.302...e-05
org-fontify-meta-lines-and-blocks-1 182
0.0057315880 3.149...e-05
Re: [O] Very slow performance in Org-mode on 10k line file?
John Hendy writes: - make clean make make doc, which is my standard update process I should maybe point out again that this has been redundant for a long time and a simple make would suffice if you removed the line oldorg: from your local.mk file. Regards, Achim. -- +[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]+ SD adaptation for Waldorf rackAttack V1.04R1: http://Synth.Stromeko.net/Downloads.html#WaldorfSDada
