Re: [O] another example of org being slow, with some analysis
On Monday, 22 Jun 2015 at 08:48, Daniel Bausch wrote: [...] I am doing each day. Maybe I will look again how to set the expected LC_... environment variables for emacs daemon. This may be the way forward. My LC related environment variables are all set to en_GB.UTF-8. This seems to be set in /etc/default/locale on my systems. -- : Eric S Fraga (0xFFFCF67D), Emacs 25.0.50.1, Org release_8.3beta-1231-ga0a883
Re: [O] another example of org being slow, with some analysis
Eric S Fraga e.fr...@ucl.ac.uk writes: On Friday, 19 Jun 2015 at 10:28, Daniel Bausch wrote: [...] If anyone could give me a hint how to reliably set the preferred (or internal) encoding I could check wether it might have something to do with the system locale. I have (only) the following encoding related settings in my initialisation, some of which are very historical and may or may not be needed... #+begin_src emacs-lisp (prefer-coding-system 'utf-8) (set-charset-priority 'unicode) (setq default-process-coding-system '(utf-8-unix . utf-8-unix)) #+end_src I think the first one is the most important. Thank you for the code, but unfortunately it does not help. The first line I already had and adding the other two did not help either. Executing M-x prefer-coding-system interactively works and that is what I am doing each day. Maybe I will look again how to set the expected LC_... environment variables for emacs daemon. Regards, Daniel -- MSc. Daniel Bausch Research Assistant (Computer Science) Technische Universität Darmstadt http://www.dvs.tu-darmstadt.de/staff/dbausch
Re: [O] another example of org being slow, with some analysis
Nick Dokos ndo...@gmail.com writes: Eric S Fraga e.fr...@ucl.ac.uk writes: On Friday, 19 Jun 2015 at 08:19, Daniel Bausch wrote: [...] Line 6000 is indeed quite lame. I have similar problems like Eric. A table recalculation at line 43868 takes about a minute at my quite fast machine. I also tracked that down to org-current-line. One interesting detail is that this depends on the buffer encoding. With ASCII the recalculation takes less than a second, with utf-8 about a minute. Adding some data: my table is at line 8438 in the buffer but character position 398345 (I have very long lines as I use visual-line-mode in org exclusively with org-indent). I do use utf-8 encoding. I have just tried updating the table on a different laptop (i7-2760, 8 cores, 8 GB RAM, Ubuntu) and it was very fast. The two laptops are running different versions of emacs (tracking latest emacs developments on Ubuntu and Debian testing lead to different versions unfortunately) so my gut feeling is that there is an emacs issue here and possibly one related to utf-8 as Daniel suggests. I'll try to do more instrumenting on my other laptop when I get a chance. What is the setting of cache-long-scans you are using? Does it differ on the two laptops? Ivan Andrus suggested setting it to nil, but it seems that for this case, leaving it at t (the default) should be much faster. But there may be a bug in the cache code. It *should* be faster but it isn't :-( At least not when using org mode. I've had this in my org file where I recalculate tables regularly. # -*- cache-long-scans: nil; -*- # This makes forward-line much faster and thus org-goto-line and thus org-table-sum (C-c +) Before it took forever to recalculate tables; now it's fast :-) Org mode file with 10500 lines. The tables I recalculate are at the bottom. Current org mode version is 8.2.10 (org-plus-contrib 20150601) Best regards, Jacob
Re: [O] another example of org being slow, with some analysis
Daniel Bausch bau...@dvs.tu-darmstadt.de writes: I think it actually is not an org-mode problem but depends on how (count-lines 1 (point)) works, as it is using regex searches for the line endings. I can imagine that the regex parser for utf-8 can be inefficient. I just looked again at Eric's profile output. Why does count-lines not show up there at all? Could it be goto-char, bolp, or even save-excursion instead? I suspected count-lines because it is really doing heavy lifting. Regards, Daniel -- MSc. Daniel Bausch Research Assistant (Computer Science) Technische Universität Darmstadt http://www.dvs.tu-darmstadt.de/staff/dbausch
Re: [O] another example of org being slow, with some analysis
Hi! Nick Dokos ndo...@gmail.com writes: Eric S Fraga e.fr...@ucl.ac.uk writes: The output of the ELP profiler is here: ... org-goto-line 104 10.761145733 0.1034725551 .. org-current-line66 6.8422078910 0.1036698165 ... I find these two difficult to explain: they account for the vast majority of the time, they don't call anything other than basic emacs lisp functions (which should be very fast) and they take an unbelievably long 0.1 s/call - I did a profile of a single call of each in a file where wc reports these stats: 12961 270362 4317977 /home/nick/lib/notes/notes.org and I got 0.002s for the first (going to line 6000, about the middle of the file) and 0.0004s for the second: a factor of 50 smaller for the first and a factor of 250 smaller for the second. Maybe it's an artifact of profiling, but maybe you can try instrumenting these two functions and doing something similar. Do you still get 0.1s for each call? Line 6000 is indeed quite lame. I have similar problems like Eric. A table recalculation at line 43868 takes about a minute at my quite fast machine. I also tracked that down to org-current-line. One interesting detail is that this depends on the buffer encoding. With ASCII the recalculation takes less than a second, with utf-8 about a minute. I think it actually is not an org-mode problem but depends on how (count-lines 1 (point)) works, as it is using regex searches for the line endings. I can imagine that the regex parser for utf-8 can be inefficient. Regards, Daniel -- MSc. Daniel Bausch Research Assistant (Computer Science) Technische Universität Darmstadt http://www.dvs.tu-darmstadt.de/staff/dbausch
Re: [O] another example of org being slow, with some analysis
Hello, Daniel Bausch bau...@dvs.tu-darmstadt.de writes: Line 6000 is indeed quite lame. I have similar problems like Eric. A table recalculation at line 43868 takes about a minute at my quite fast machine. I also tracked that down to org-current-line. One interesting detail is that this depends on the buffer encoding. With ASCII the recalculation takes less than a second, with utf-8 about a minute. I think it actually is not an org-mode problem but depends on how (count-lines 1 (point)) works, as it is using regex searches for the line endings. I can imagine that the regex parser for utf-8 can be inefficient. Actually `org-count-lines' and `org-goto-line' are inefficient functions. Unfortunately, Org Table overuses the latter. Functions calling it should be refactored to use, e.g., `goto-char' instead. However, this might require some deep changes in the library. Regards, -- Nicolas Goaziou
Re: [O] another example of org being slow, with some analysis
Eric S Fraga e.fr...@ucl.ac.uk writes: On Friday, 19 Jun 2015 at 08:19, Daniel Bausch wrote: [...] Line 6000 is indeed quite lame. I have similar problems like Eric. A table recalculation at line 43868 takes about a minute at my quite fast machine. I also tracked that down to org-current-line. One interesting detail is that this depends on the buffer encoding. With ASCII the recalculation takes less than a second, with utf-8 about a minute. Adding some data: my table is at line 8438 in the buffer but character position 398345 (I have very long lines as I use visual-line-mode in org exclusively with org-indent). I do use utf-8 encoding. I have just tried updating the table on a different laptop (i7-2760, 8 cores, 8 GB RAM, Ubuntu) and it was very fast. The two laptops are running different versions of emacs (tracking latest emacs developments on Ubuntu and Debian testing lead to different versions unfortunately) so my gut feeling is that there is an emacs issue here and possibly one related to utf-8 as Daniel suggests. I'll try to do more instrumenting on my other laptop when I get a chance. What is the setting of cache-long-scans you are using? Does it differ on the two laptops? Ivan Andrus suggested setting it to nil, but it seems that for this case, leaving it at t (the default) should be much faster. But there may be a bug in the cache code. -- Nick
Re: [O] another example of org being slow, with some analysis
On Friday, 19 Jun 2015 at 08:19, Daniel Bausch wrote: [...] Line 6000 is indeed quite lame. I have similar problems like Eric. A table recalculation at line 43868 takes about a minute at my quite fast machine. I also tracked that down to org-current-line. One interesting detail is that this depends on the buffer encoding. With ASCII the recalculation takes less than a second, with utf-8 about a minute. Adding some data: my table is at line 8438 in the buffer but character position 398345 (I have very long lines as I use visual-line-mode in org exclusively with org-indent). I do use utf-8 encoding. I have just tried updating the table on a different laptop (i7-2760, 8 cores, 8 GB RAM, Ubuntu) and it was very fast. The two laptops are running different versions of emacs (tracking latest emacs developments on Ubuntu and Debian testing lead to different versions unfortunately) so my gut feeling is that there is an emacs issue here and possibly one related to utf-8 as Daniel suggests. I'll try to do more instrumenting on my other laptop when I get a chance. thanks all, eric -- : Eric S Fraga (0xFFFCF67D), Emacs 25.0.50.1, Org release_8.3beta-1147-g0e5069.dirty
Re: [O] another example of org being slow, with some analysis
Eric S Fraga e.fr...@ucl.ac.uk writes: On Friday, 19 Jun 2015 at 08:19, Daniel Bausch wrote: [...] Line 6000 is indeed quite lame. I have similar problems like Eric. A table recalculation at line 43868 takes about a minute at my quite fast machine. I also tracked that down to org-current-line. One interesting detail is that this depends on the buffer encoding. With ASCII the recalculation takes less than a second, with utf-8 about a minute. Adding some data: my table is at line 8438 in the buffer but character position 398345 (I have very long lines as I use visual-line-mode in org exclusively with org-indent). I do use utf-8 encoding. I have just tried updating the table on a different laptop (i7-2760, 8 cores, 8 GB RAM, Ubuntu) and it was very fast. The two laptops are running different versions of emacs (tracking latest emacs developments on Ubuntu and Debian testing lead to different versions unfortunately) so my gut feeling is that there is an emacs issue here and possibly one related to utf-8 as Daniel suggests. Maybe it is related (and maybe not - just some thoughts): Since some months I keep having a problem with my preferred encoding. Gentoo changed some things how the LC_... variables are set during boot. Now emacs daemon starts with no encoding. I tried to fix that by inserting (prefer-coding-system 'utf-8) into mit ~/.emacs file, but that does not help. I still need to manually execute prefer-coding-system by hand and revert my org-file (which is automatically opened during login) with C-x RET r utf-8 every day. But if I remember correctly, I started observing the slowness of the table recalculation long before that change in Gentoo. Maybe it got worse since then but that might also be just because the file grew further. If anyone could give me a hint how to reliably set the preferred (or internal) encoding I could check wether it might have something to do with the system locale. (I can imagine that in some constellations there could be some otherwise unnecessary conversion happening. E.g. maybe utf-8 support is faster if the system locale is also utf-8.) Is there anything like an internal encoding in emacs at all? As far as I know I can type any character in any buffer regardless of the buffer encoding. I just cannot save the buffer if the encoding does not support a character. How is then a utf-8 *buffer* different from a buffer with no encoding (until being saved). Does goto-char work on bytes or really on characters? How does it maybe deal different with characters made up of multiple bytes in the on-disk encoding of the buffer? Regards, Daniel -- MSc. Daniel Bausch Research Assistant (Computer Science) Technische Universität Darmstadt http://www.dvs.tu-darmstadt.de/staff/dbausch
Re: [O] another example of org being slow, with some analysis
On Friday, 19 Jun 2015 at 10:28, Daniel Bausch wrote: [...] If anyone could give me a hint how to reliably set the preferred (or internal) encoding I could check wether it might have something to do with the system locale. I have (only) the following encoding related settings in my initialisation, some of which are very historical and may or may not be needed... #+begin_src emacs-lisp (prefer-coding-system 'utf-8) (set-charset-priority 'unicode) (setq default-process-coding-system '(utf-8-unix . utf-8-unix)) #+end_src I think the first one is the most important. -- : Eric S Fraga (0xFFFCF67D), Emacs 25.0.50.1, Org release_8.3beta-1231-ga0a883
Re: [O] another example of org being slow, with some analysis
On Friday, 19 Jun 2015 at 09:53, Nick Dokos wrote: [...] What is the setting of cache-long-scans you are using? Does it differ on the two laptops? Ivan Andrus suggested setting it to nil, but it seems that for this case, leaving it at t (the default) should be much faster. But there may be a bug in the cache code. It's the default (t) in both cases. My emacs initialisation is the same on all of my systems and I didn't even know about this setting! -- : Eric S Fraga (0xFFFCF67D), Emacs 25.0.50.1, Org release_8.3beta-1231-ga0a883
[O] another example of org being slow, with some analysis
Hello, there have been a few threads recently mentioning poor performance. Although some of these have been resolved (e.g. the use of =linum=), I wonder if I could add a data point. I have a medium (for me) sized (385 kB) org file with all of my tasks in a date-tree structure. One of the tasks requires me to edit a small table every day for a couple of weeks (see below for the table). Asking for a recalculation of the table (C-u C-c C-c) takes 17 seconds on a new laptop (16 GB RAM, SSD, i7 cpu, Debian linux testing) which is quite fast at most other tasks. The output of the ELP profiler is here: --8---cut here---start-8--- org-ctrl-c-ctrl-c 117.655796272 17.655796272 org-table-recalculate 117.652738791 17.652738791 org-table-get-range 14 13.166332242 0.940452303 org-table-eval-formula 19 13.101183716 0.6895359851 org-goto-line 104 10.761145733 0.1034725551 org-table-copy-region 13 7.362366204 0.5663358618 org-current-line66 6.8422078910 0.1036698165 org-table-get-descriptor-line 28 1.659497877 0.0592677813 org-table-align 10.415633428 0.415633428 org-table-get-specials 10.206756703 0.206756703 org-table-expand-lhs-ranges 10.103551707 0.103551707 org-element--parse-to 60.0031655370 0.0005275895 org-babel-execute-safely-maybe 10.002872891 0.002872891 org-babel-execute-maybe 10.002870729 0.002870729 org-babel-execute-src-block-maybe 10.002846968 0.002846968 org-element-at-point30.0028012790 0.0009337596 org-babel-where-is-src-block-head 10.002664822 0.002664822 org-element--cache-compare 303 0.0025883040 8.542...e-06 org-table-current-column119 0.002235665 1.878...e-05 --8---cut here---end---8--- The table, for completeness, is here: #+begin_src org | date | TBD | Victoria | London Bridge | Blackfriars | City | Farrindgon | St Pancras | Totals | Miles | | | 2.5 |5 | 4.8 | 3.2 | 2.9 | 5 |1.6 || | |--+-+--+---+-+--++++---| | [2015-06-08 Mon] | 2 | | | |1 | | 1 |9.5 | 5.9 | | [2015-06-12 Fri] | 2 | | | 1 | | | 1 |9.8 | 6.0 | | [2015-06-15 Mon] | 2 | | 1 | | | | 1 | 11.4 | 7.0 | | [2015-06-17 Wed] | 2 | | | 1 | | | 1 |9.8 | 6.0 | | [2015-06-18 Thu] | 1 | | | 1 | | ||5.7 | 3.5 | |--+-+--+---+-+--++++---| | Totals | 9 |0 | 1 | 3 |1 | 0 | 4 | 46.2 | 28.5 | ,#+TBLFM: @$2..@$=vsum(@I..@II)::$9=($2..$)*(@$2..@$);EN::$10=$9/1.62;%.1f #+end_src If I change, for instance, the entry in the TBD column for today from 1 to 2, in the original file, it takes 17 seconds. If I do it in this email buffer (changing mode to org etc.), it is essentially instantaneous. Large buffers seem to be causing problems with org. I've noticed this elsewhere as I do tend to work with org files that are often 0.25-1 MB in size. Having looked at the profiler output, my gut feeling was that the buffer is being repeatedly parsed (or some similar activity) given where the time is being spent. So... as the table is currently 99% of the way into the buffer, I have copied the table to the start of the same large document and, lo and behold, the table recalculation on the copy is instantaneous! (the moral of the story, for me, may be to use a reverse date-tree structure for my tasks, if such were possible... maybe it is, given the hidden gems in org! ;-) I would be happy to instrument more than just org, if it would help. I should add that the above profile was done using emacs -Q so no customisations. I have no real issue with the performance of org overall but I thought I'd add a data point in case it helps. thanks, eric -- : Eric S Fraga (0xFFFCF67D), Emacs 24.4.1, Org release_8.3beta-1216-gb856f6
Re: [O] another example of org being slow, with some analysis
Eric S Fraga e.fr...@ucl.ac.uk writes: The output of the ELP profiler is here: ... org-goto-line 104 10.761145733 0.1034725551 .. org-current-line66 6.8422078910 0.1036698165 ... I find these two difficult to explain: they account for the vast majority of the time, they don't call anything other than basic emacs lisp functions (which should be very fast) and they take an unbelievably long 0.1 s/call - I did a profile of a single call of each in a file where wc reports these stats: 12961 270362 4317977 /home/nick/lib/notes/notes.org and I got 0.002s for the first (going to line 6000, about the middle of the file) and 0.0004s for the second: a factor of 50 smaller for the first and a factor of 250 smaller for the second. Maybe it's an artifact of profiling, but maybe you can try instrumenting these two functions and doing something similar. Do you still get 0.1s for each call? Nick
Re: [O] another example of org being slow, with some analysis
I’m jumping into the middle of the thread, but have you tried (setq cache-long-scans nil) That solved some performance issues for me. I can’t remember where I got the advice. -Ivan On Jun 18, 2015, at 10:28 AM, Eric S Fraga e.fr...@ucl.ac.uk wrote: Hello, there have been a few threads recently mentioning poor performance. Although some of these have been resolved (e.g. the use of =linum=), I wonder if I could add a data point. I have a medium (for me) sized (385 kB) org file with all of my tasks in a date-tree structure. One of the tasks requires me to edit a small table every day for a couple of weeks (see below for the table). Asking for a recalculation of the table (C-u C-c C-c) takes 17 seconds on a new laptop (16 GB RAM, SSD, i7 cpu, Debian linux testing) which is quite fast at most other tasks. The output of the ELP profiler is here: --8---cut here---start-8--- org-ctrl-c-ctrl-c 117.655796272 17.655796272 org-table-recalculate 117.652738791 17.652738791 org-table-get-range 14 13.166332242 0.940452303 org-table-eval-formula 19 13.101183716 0.6895359851 org-goto-line 104 10.761145733 0.1034725551 org-table-copy-region 13 7.362366204 0.5663358618 org-current-line66 6.8422078910 0.1036698165 org-table-get-descriptor-line 28 1.659497877 0.0592677813 org-table-align 10.415633428 0.415633428 org-table-get-specials 10.206756703 0.206756703 org-table-expand-lhs-ranges 10.103551707 0.103551707 org-element--parse-to 60.0031655370 0.0005275895 org-babel-execute-safely-maybe 10.002872891 0.002872891 org-babel-execute-maybe 10.002870729 0.002870729 org-babel-execute-src-block-maybe 10.002846968 0.002846968 org-element-at-point30.0028012790 0.0009337596 org-babel-where-is-src-block-head 10.002664822 0.002664822 org-element--cache-compare 303 0.0025883040 8.542...e-06 org-table-current-column119 0.002235665 1.878...e-05 --8---cut here---end---8--- The table, for completeness, is here: #+begin_src org | date | TBD | Victoria | London Bridge | Blackfriars | City | Farrindgon | St Pancras | Totals | Miles | | | 2.5 |5 | 4.8 | 3.2 | 2.9 | 5 |1.6 || | |--+-+--+---+-+--++++---| | [2015-06-08 Mon] | 2 | | | |1 | | 1 |9.5 | 5.9 | | [2015-06-12 Fri] | 2 | | | 1 | | | 1 |9.8 | 6.0 | | [2015-06-15 Mon] | 2 | | 1 | | | | 1 | 11.4 | 7.0 | | [2015-06-17 Wed] | 2 | | | 1 | | | 1 |9.8 | 6.0 | | [2015-06-18 Thu] | 1 | | | 1 | | ||5.7 | 3.5 | |--+-+--+---+-+--++++---| | Totals | 9 |0 | 1 | 3 |1 | 0 | 4 | 46.2 | 28.5 | ,#+TBLFM: @$2..@$=vsum(@I..@II)::$9=($2..$)*(@$2..@ $);EN::$10=$9/1.62;%.1f #+end_src If I change, for instance, the entry in the TBD column for today from 1 to 2, in the original file, it takes 17 seconds. If I do it in this email buffer (changing mode to org etc.), it is essentially instantaneous. Large buffers seem to be causing problems with org. I've noticed this elsewhere as I do tend to work with org files that are often 0.25-1 MB in size. Having looked at the profiler output, my gut feeling was that the buffer is being repeatedly parsed (or some similar activity) given where the time is being spent. So... as the table is currently 99% of the way into the buffer, I have copied the table to the start of the same large document and, lo and behold, the table recalculation on the copy is instantaneous! (the moral of the story, for me, may be to use a reverse date-tree structure for my tasks, if such were possible... maybe it is, given the hidden gems in org! ;-) I would be happy to instrument more than just org, if it would help. I should add that the above profile was done using emacs -Q so no customisations. I have no real issue with the performance of org overall but I thought I'd add a data point in case it helps. thanks, eric -- : Eric S Fraga (0xFFFCF67D), Emacs 24.4.1, Org release_8.3beta-1216-gb856f6
Re: [O] another example
rasmus == rasmus Rasmus writes:
Uwe Brauer o...@mat.ucm.es writes:
Look the following examples
\begin{displaymath}
Xs,δ:=Hs,δ× Hs,δ+1× Hs,δ+1×
Hs+1,δ+2
\end{displaymath}
org-preview-latex-fragment does not work, change displaymath for
equation, again org-preview-latex-fragment does not work, but
\[ \]
does work, this is odd
1. The example you posted contains utf8 symbols and you'd likely need
unicodemath (the package) to display it. For this you'd need
xelatex or lualatex. But probably you wrote it as plain LaTeX code
in your document.
Hm it seems that the coding was changed when sending the formula, since
I wrote \delta in the original mail not δ.
In any case: *why* does \[ \] work, if it is question of a missing package?
2. displaymath is from amsmath. This works on my system:
#+BEGIN_SRC org
#+LATEX_HEADER: \usepackage{amsmath}
\begin{displaymath}
x=1
\end{displaymath}
#+END_SRC
Really?
Here is a minimal example
of a LaTeX file, which is not using amsmath, but the file is compiled as
it should be:
\documentclass{article}
\begin{document}
\begin{displaymath}
\int dx
\end{displaymath}
\end{document}
I also extended the corresponding configuration in org.el
(defcustom org-format-latex-header \\documentclass[12pt]{article}
\\usepackage[usenames]{color}
\\usepackage{amsmath}
\\usepackage{plain}
\\usepackage[mathscr]{eucal}
Uwe
smime.p7s
Description: S/MIME cryptographic signature
[O] another example (was: org-preview-latex-fragment is very picky)
#secure method=smime mode=sign
"rasmus" == rasmus Rasmus writes:
Uwe Brauer o...@mat.ucm.es writes:
Probably not.
Note these are Org features irrespective of fragment preview. There
was some talk about it not so long ago.
Basically, it just happens that works as desired most of the time.
The proper way in LaTeX as well as in Org is . Often you'll find
that in challenging cases you're better off with .
Look the following examples
\begin{displaymath}
Xs,:=Hs, Hs,+1 Hs,+1
Hs+1,+2
\end{displaymath}
org-preview-latex-fragment does not work, change displaymath for
equation, again org-preview-latex-fragment does not work, but
does work, this is odd
Re: [O] another example
Uwe Brauer o...@mat.ucm.es writes:
Look the following examples
\begin{displaymath}
Xs,δ:=Hs,δ× Hs,δ+1× Hs,δ+1×
Hs+1,δ+2
\end{displaymath}
org-preview-latex-fragment does not work, change displaymath for
equation, again org-preview-latex-fragment does not work, but
\[ \]
does work, this is odd
1. The example you posted contains utf8 symbols and you'd likely need
unicodemath (the package) to display it. For this you'd need
xelatex or lualatex. But probably you wrote it as plain LaTeX code
in your document.
2. displaymath is from amsmath. This works on my system:
#+BEGIN_SRC org
#+LATEX_HEADER: \usepackage{amsmath}
\begin{displaymath}
x=1
\end{displaymath}
#+END_SRC
-Rasmus
--
Got mashed potatoes. Ain't got no T-Bone. No T-Bone
