Re: [O] working with tables can be quite painful...
Hello, Eric S Fragawrites: > I cannot send the actual table I am working on as it relates to grades > for students. I will try to create a similar table with random > entries. You could also try the imperfect (make sure to eyeball the result): (defun scramble-contents () (interactive) (let ((tree (org-element-parse-buffer))) (org-element-map tree '(code comment comment-block example-block fixed-width keyword link node-property plain-text verbatim) (lambda (obj) (cl-case (org-element-type obj) ((code comment comment-block example-block fixed-width keyword node-property verbatim) (let ((value (org-element-property :value obj))) (org-element-put-property obj :value (replace-regexp-in-string "[[:alnum:]]" "x" value (link (unless (string= (org-element-property :type obj) "radio") (org-element-put-property obj :raw-link "http://orgmode.org;))) (plain-text (org-element-set-element obj (replace-regexp-in-string "[[:alnum:]]" "x" obj) nil nil nil t) (let ((buffer (get-buffer-create "*Scrambled text*"))) (with-current-buffer buffer (insert (org-element-interpret-data tree)) (goto-char (point-min))) (switch-to-buffer buffer Regards, -- Nicolas Goaziou
Re: [O] working with tables can be quite painful...
On Tuesday, 20 Sep 2016 at 08:54, Jacob Nielsen wrote: > Eric S Fragawrites: > > Hi, I've had these lines in my org files for a long time. Perhaps they > help ? > > # -*- cache-long-scans: nil; -*- > # This makes forward-line much faster and thus org-goto-line > # and thus org-table-sum (C-c +) Interesting. Reading the documentation for this variable notes that having this set to t is most useful for buffers with very long lines. As I use visual-line-mode, most of my lines are whole paragraphs so frequently longer than the 500 characters that the documentation mentions as "long lines". So it would seem that I should keep the default setting of t... but it may be worthwhile playing around with this variable. There are obviously quite a few interacting elements which can affect the performance of emacs. Thanks! -- : Eric S Fraga (0xFFFCF67D), Emacs 25.1.50.1, Org release_8.3.6-1149-g582233
Re: [O] working with tables can be quite painful...
Eric S Fragawrites: Hi, I've had these lines in my org files for a long time. Perhaps they help ? # -*- cache-long-scans: nil; -*- # This makes forward-line much faster and thus org-goto-line # and thus org-table-sum (C-c +) Best regards, Jacob > Hello, > > I am working with a table. It has approximately 130 rows and 20 columns > so it's not huge but also not small. Three columns are text but the > rest are all numbers with some degree of sparsity. Instrumenting org > while working on this table, manipulating the entries in just one row, I > get the following: > > | Function | calls | elapsed time | average time | > |--+---+--+--| > | org-cycle|20 | 41.550963140 | 2.0775481570 | > | org-table-next-field |20 | 41.544266727 | 2.0772133363 | > | org-table-align | 5 | 41.470595702 | 8.2941191404 | > | org-mode-flyspell-verify |52 | 1.0647362189 | 0.0204756965 | > | org-do-latex-and-related |21 | 0.6656267140 | 0.0316965101 | > | org-element-at-point | 125 | 0.6356939890 | 0.0050855519 | > | org-element--parse-to| 125 | 0.6086256940 | 0.0048690055 | > | org-element--cache-put | 1399 | 0.4963533770 | 0.0003547915 | > > From this, it would seem that the table align function is killing the > performance. 8 seconds per call? On an 8 core Intel(R) Core(TM) > i7-2760QM CPU @ 2.40GHz... so not my wee Pandora where I expect > slowness! > > This is with a not quite up to date org. I'm avoiding upgrading as I am > preparing material for teaching which starts soon and I don't want to > run into issues due to changes in org... so I apologise if things have > changed recently. This performance issue has existed for quite some time > now, however. > > Any suggestions on speeding things up? > > Thanks, > eric
Re: [O] working with tables can be quite painful...
Hello, Eric S Fragawrites: > On Monday, 19 Sep 2016 at 12:58, Michael Welle wrote: > > [...] > >> the output of your profiler run doesn't ring a bell, but a few years ago >> I had the problem that after some uptime (my Emacs uptime is typical >> several days, up to a few weeks) Org operations like building the agenda >> became slw. Restarting Emacs fixed that, until the problem occurred >> again after some time. > > Interesting. The performance I was referring to last week was indeed in > an emacs that had been running for quite some time. I did close down > that emacs instance on Friday for unrelated reasons and the instance > this morning had only been running for minutes. Maybe the problem is > not org alone but relates to emacs as well. yepp, that's what I thought. Maybe a minor mode or some function that got evaluated only once in a while. Or something, that depends on the input data of that something. >> If I remember correctly I asked the list for advise. But it looked like >> one of these problems only I have ;). Maybe some other lisp package had >> its hands in the game, I don't know. Eventually, after some Emacs, Org >> and other updates the problem was gone. > > Well, maybe it's a problem that others do have! ;) >> As I said, I don't think, you suffer from the same problem. But you said >> you don't use the newest software versions, so just in case. > > To clarify, when I said I wasn't using the latest version of org, it's > that I hadn't updated from git in a few weeks but I am using a snapshot > of emacs tracking the development version (and now I have org up to date). Hm, my problem was definitely way more in the past. The only performance related problem I have from time to time is when the vc-mode hooks get in the way. But that's nothing Org can change. But on the other hand, my tables are much smaller than yours and contain mostly static data. Regards hmw
Re: [O] working with tables can be quite painful...
On Monday, 19 Sep 2016 at 12:58, Michael Welle wrote: [...] > the output of your profiler run doesn't ring a bell, but a few years ago > I had the problem that after some uptime (my Emacs uptime is typical > several days, up to a few weeks) Org operations like building the agenda > became slw. Restarting Emacs fixed that, until the problem occurred > again after some time. Interesting. The performance I was referring to last week was indeed in an emacs that had been running for quite some time. I did close down that emacs instance on Friday for unrelated reasons and the instance this morning had only been running for minutes. Maybe the problem is not org alone but relates to emacs as well. > If I remember correctly I asked the list for advise. But it looked like > one of these problems only I have ;). Maybe some other lisp package had > its hands in the game, I don't know. Eventually, after some Emacs, Org > and other updates the problem was gone. Well, maybe it's a problem that others do have! > As I said, I don't think, you suffer from the same problem. But you said > you don't use the newest software versions, so just in case. To clarify, when I said I wasn't using the latest version of org, it's that I hadn't updated from git in a few weeks but I am using a snapshot of emacs tracking the development version (and now I have org up to date). -- : Eric S Fraga (0xFFFCF67D), Emacs 25.1.50.1, Org release_8.3.6-1149-g582233
Re: [O] working with tables can be quite painful...
Hello, Eric S Fragawrites: > On Saturday, 17 Sep 2016 at 08:48, Nicolas Goaziou wrote: > > [...] > >> Could you send a profiler report so that I can get a better glimpse on >> what part of `org-table-align' is lagging? > > Hi Nicolas, > > this morning, working on the same table is much less painful. I've run > the profile and did some movements and changes to the table and here is > the output from the profiler (I can send you the full report if you > wish). > > I know that if I narrow the buffer to just the section (headline and > contents) that contains the table, the performance is > better. Anecdotally, it would seem that the performance degrades over > time so I wonder if there is a cache issue? the output of your profiler run doesn't ring a bell, but a few years ago I had the problem that after some uptime (my Emacs uptime is typical several days, up to a few weeks) Org operations like building the agenda became slw. Restarting Emacs fixed that, until the problem occurred again after some time. If I remember correctly I asked the list for advise. But it looked like one of these problems only I have ;). Maybe some other lisp package had its hands in the game, I don't know. Eventually, after some Emacs, Org and other updates the problem was gone. As I said, I don't think, you suffer from the same problem. But you said you don't use the newest software versions, so just in case. Regards hmw
Re: [O] working with tables can be quite painful...
On Saturday, 17 Sep 2016 at 08:48, Nicolas Goaziou wrote: [...] > Could you send a profiler report so that I can get a better glimpse on > what part of `org-table-align' is lagging? Hi Nicolas, this morning, working on the same table is much less painful. I've run the profile and did some movements and changes to the table and here is the output from the profiler (I can send you the full report if you wish). I know that if I narrow the buffer to just the section (headline and contents) that contains the table, the performance is better. Anecdotally, it would seem that the performance degrades over time so I wonder if there is a cache issue? My gut feeling is that it is related to fontification as well. --8<---cut here---start->8--- - command-execute2420 53% - call-interactively2420 53% - funcall-interactively2325 51% - pabbrev-expand-maybe1244 27% - if 1244 27% - pabbrev-expand-maybe-full 1244 27% - cond 1244 27% - pabbrev-call-previous-tab-binding 1205 26% - let1205 26% - if1205 26% - let 1205 26% - if 1205 26% - funcall1205 26% - org-cycle 1205 26% - call-interactively 1198 26% - funcall-interactively 1198 26% - org-table-next-field 1198 26% - org-table-align 1122 24% + font-lock-fontify-region 258 5% + mapcar219 4% + org-indent-refresh-maybe 152 3% apply 45 0% org-element--cache-before-change 32 0% org-element--cache-after-change 32 0% + jit-lock-after-change 16 0% org-table-begin 9 0% + org-table-maybe-eval-formula 4 0% org-table-end4 0% --8<---cut here---end--->8--- > An ECM would help too. I cannot send the actual table I am working on as it relates to grades for students. I will try to create a similar table with random entries. thanks, eric -- : Eric S Fraga (0xFFFCF67D), Emacs 25.0.94.1, Org release_8.3.5-1070-g190476
Re: [O] working with tables can be quite painful...
On Saturday, 17 Sep 2016 at 15:37, Michael Brand wrote: > Hi Eric > > Question, out of curiosity: Is there a difference when you delete all lines > above and below the table, with and without adding a headline above? Hi Michael, Maybe. I know that org's performance improves when I narrow the buffer to just the section I am working on, when I work on long documents so the lines above and below the table may affect things. > One of my tables fluctuates around 150 rows and around 20 to 40 > columns, overall a few hundred characters wide (columns with some > history window for each row). Almost all fields are just text although > I think this doesn't matter at all. The table starts at about line 50 > and is followed by about 7600 lines. Since years I manipulate it at > least every few days without any slowness, often on much weaker > hardware than yours. How long do you spend manipulating the table at any one time? I have an impression that performance degrades over time in a single session. But I have no evidence for this. It's just an impression. thanks, eric -- : Eric S Fraga (0xFFFCF67D), Emacs 25.0.94.1, Org release_8.3.5-1070-g190476
Re: [O] working with tables can be quite painful...
Hi Eric Question, out of curiosity: Is there a difference when you delete all lines above and below the table, with and without adding a headline above? One of my tables fluctuates around 150 rows and around 20 to 40 columns, overall a few hundred characters wide (columns with some history window for each row). Almost all fields are just text although I think this doesn't matter at all. The table starts at about line 50 and is followed by about 7600 lines. Since years I manipulate it at least every few days without any slowness, often on much weaker hardware than yours. Maybe you can find a temporary workaround. Michael
Re: [O] working with tables can be quite painful...
In case this helps anyone, I've found this code makes profiling a lot easier. It automatically instruments the desired functions, runs the code you want to test, removes the instrumentation, and presents the results. #+BEGIN_SRC elisp (defmacro profile-org (times body) `(let (output) (dolist (p '("org-")) ; symbol prefixes to instrument (elp-instrument-package p)) (dotimes (x ,times) ,@body) (elp-results) (elp-restore-all) (point-min) (forward-line 20) (delete-region (point) (point-max)) (setq output (buffer-substring-no-properties (point-min) (point-max))) (kill-buffer) (delete-window) output)) ;; Used like this: (profile-org 10 (org-table-next-field) (org-table-align)) #+END_SRC
Re: [O] working with tables can be quite painful...
Hello, Eric S Fragawrites: > I am working with a table. It has approximately 130 rows and 20 columns > so it's not huge but also not small. Three columns are text but the > rest are all numbers with some degree of sparsity. Instrumenting org > while working on this table, manipulating the entries in just one row, I > get the following: > > | Function | calls | elapsed time | average time | > |--+---+--+--| > | org-cycle|20 | 41.550963140 | 2.0775481570 | > | org-table-next-field |20 | 41.544266727 | 2.0772133363 | > | org-table-align | 5 | 41.470595702 | 8.2941191404 | > | org-mode-flyspell-verify |52 | 1.0647362189 | 0.0204756965 | > | org-do-latex-and-related |21 | 0.6656267140 | 0.0316965101 | > | org-element-at-point | 125 | 0.6356939890 | 0.0050855519 | > | org-element--parse-to| 125 | 0.6086256940 | 0.0048690055 | > | org-element--cache-put | 1399 | 0.4963533770 | 0.0003547915 | > > From this, it would seem that the table align function is killing the > performance. 8 seconds per call? On an 8 core Intel(R) Core(TM) > i7-2760QM CPU @ 2.40GHz... so not my wee Pandora where I expect > slowness! > > This is with a not quite up to date org. I'm avoiding upgrading as I am > preparing material for teaching which starts soon and I don't want to > run into issues due to changes in org... so I apologise if things have > changed recently. This performance issue has existed for quite some time > now, however. > > Any suggestions on speeding things up? Could you send a profiler report so that I can get a better glimpse on what part of `org-table-align' is lagging? An ECM would help too. Regards, -- Nicolas Goaziou
[O] working with tables can be quite painful...
Hello, I am working with a table. It has approximately 130 rows and 20 columns so it's not huge but also not small. Three columns are text but the rest are all numbers with some degree of sparsity. Instrumenting org while working on this table, manipulating the entries in just one row, I get the following: | Function | calls | elapsed time | average time | |--+---+--+--| | org-cycle|20 | 41.550963140 | 2.0775481570 | | org-table-next-field |20 | 41.544266727 | 2.0772133363 | | org-table-align | 5 | 41.470595702 | 8.2941191404 | | org-mode-flyspell-verify |52 | 1.0647362189 | 0.0204756965 | | org-do-latex-and-related |21 | 0.6656267140 | 0.0316965101 | | org-element-at-point | 125 | 0.6356939890 | 0.0050855519 | | org-element--parse-to| 125 | 0.6086256940 | 0.0048690055 | | org-element--cache-put | 1399 | 0.4963533770 | 0.0003547915 | >From this, it would seem that the table align function is killing the performance. 8 seconds per call? On an 8 core Intel(R) Core(TM) i7-2760QM CPU @ 2.40GHz... so not my wee Pandora where I expect slowness! This is with a not quite up to date org. I'm avoiding upgrading as I am preparing material for teaching which starts soon and I don't want to run into issues due to changes in org... so I apologise if things have changed recently. This performance issue has existed for quite some time now, however. Any suggestions on speeding things up? Thanks, eric -- : Eric S Fraga (0xFFFCF67D), Emacs 25.0.94.1, Org release_8.3.5-1070-g190476