Re: [O] [PATH] Speedups to org-table-recalculate
Hello, Nathaniel Flath flat0...@gmail.com writes: Fixed. Applied, with TINYCHANGE tag. Thank you. Regards, -- Nicolas Goaziou
Re: [O] [PATH] Speedups to org-table-recalculate
Hi Nathaniel The macro did not set the time to nil, the current function does when t1 is nil but I think it must not. This led me to the following ideas, sorry for not thinking enough earlier: Why not change the macro's (setq ,t1 curtime) to a function's (set t1 curtime), still conditional as it was in the macro and call the function with 'log-*-time, quoted? Why not factor out the action from the function that would then only return whether any action has to be taken and in consequence also factor out the sometimes needed test for all? (when (or (not all) (org-once-per-second 'log-*-time)) (message [...])) The function would not have to test t1 for nil anymore. In my opinion this functionality should go into org-macs.el, hence omitting -table- in the function name. Michael
Re: [O] [PATH] Speedups to org-table-recalculate
Fixed. On Fri, Dec 5, 2014 at 3:57 PM, Nicolas Goaziou m...@nicolasgoaziou.fr wrote: Nathaniel Flath flat0...@gmail.com writes: Sorry, that was incorrect - real patches attached. Thanks. +(defun org-table-message-once-per-second (t1 rest args) + If there has been more than one second since T1, display message. +ARGS are passed as arguments to the 'message' function. Returns +current time if a message is printed, otherwise returns t1.. If +T1 is nil, always messages. + (let ((curtime (current-time))) +(when (or (not t1) ( 0 (nth 1 (time-subtract curtime t1 + (apply message args) + curtime)) +t1) The docstring seems incorrect, as the function always returns T1, no matter if a message is printed or not. + (setq log-last-time + (org-table-message-once-per-second +(when all log-last-time) Nitpick: (and all log-last-time) + (when all log-last-time) Ditto. + (when all log-first-time) Ditto. Regards, 0001-org-table.el-org-table-recalculate-early-returns(1).patch Description: Binary data 0001-org-table.el-org-table-recalculate-is-quieter.patch Description: Binary data
Re: [O] [PATH] Speedups to org-table-recalculate
Nathaniel Flath flat0...@gmail.com writes: Sorry, that was incorrect - real patches attached. Thanks. +(defun org-table-message-once-per-second (t1 rest args) + If there has been more than one second since T1, display message. +ARGS are passed as arguments to the 'message' function. Returns +current time if a message is printed, otherwise returns t1.. If +T1 is nil, always messages. + (let ((curtime (current-time))) +(when (or (not t1) ( 0 (nth 1 (time-subtract curtime t1 + (apply message args) + curtime)) +t1) The docstring seems incorrect, as the function always returns T1, no matter if a message is printed or not. + (setq log-last-time + (org-table-message-once-per-second +(when all log-last-time) Nitpick: (and all log-last-time) + (when all log-last-time) Ditto. + (when all log-first-time) Ditto. Regards,
Re: [O] [PATH] Speedups to org-table-recalculate
Changed it to just a function - you are correct, for this purpose I don't need a macro. On Fri, Nov 21, 2014 at 6:30 PM, Nicolas Goaziou m...@nicolasgoaziou.fr wrote: Nathaniel Flath flat0...@gmail.com writes: OK, I think I fixed that. Thanks. I didn't verify it compiles, but your macro still looks suspicious. +(defmacro org-table-execute-once-per-second (t1 rest body) + If there has been more than one second since T1, execute BODY. +Updates T1 to 'current-time' if this condition is met. If T1 is +nil, always execute body. + `(let ((t1 ,t1)) + (if t1 + (let ((curtime (current-time))) +(when ( 0 (nth 1 (time-subtract curtime t1))) + (setq t1 curtime) +,@body)) + ,@body))) You shouldn't splice BODY twice in your macro. Also, I don't get why you need to (setq t1 curtime). Do you need a macro at all for this task? ISTM you only need to display a message conditionally and update a time value. Regard, 0001-org-table.el-org-table-recalculate-is-quieter.patch Description: Binary data 0001-org-table.el-org-table-recalculate-early-returns(1).patch Description: Binary data
Re: [O] [PATH] Speedups to org-table-recalculate
Sorry, that was incorrect - real patches attached. On Mon, Dec 1, 2014 at 1:02 AM, Nathaniel Flath flat0...@gmail.com wrote: Changed it to just a function - you are correct, for this purpose I don't need a macro. On Fri, Nov 21, 2014 at 6:30 PM, Nicolas Goaziou m...@nicolasgoaziou.fr wrote: Nathaniel Flath flat0...@gmail.com writes: OK, I think I fixed that. Thanks. I didn't verify it compiles, but your macro still looks suspicious. +(defmacro org-table-execute-once-per-second (t1 rest body) + If there has been more than one second since T1, execute BODY. +Updates T1 to 'current-time' if this condition is met. If T1 is +nil, always execute body. + `(let ((t1 ,t1)) + (if t1 + (let ((curtime (current-time))) +(when ( 0 (nth 1 (time-subtract curtime t1))) + (setq t1 curtime) +,@body)) + ,@body))) You shouldn't splice BODY twice in your macro. Also, I don't get why you need to (setq t1 curtime). Do you need a macro at all for this task? ISTM you only need to display a message conditionally and update a time value. Regard, 0001-org-table.el-org-table-recalculate-early-returns(1).patch Description: Binary data 0001-org-table.el-org-table-recalculate-is-quieter.patch Description: Binary data
Re: [O] [PATH] Speedups to org-table-recalculate
OK, I think I fixed that. Yes, I have signed FSF papers. On Sat, Nov 15, 2014 at 4:07 AM, Nicolas Goaziou m...@nicolasgoaziou.fr wrote: Hello, Michael Brand michael.ch.br...@gmail.com writes: In my opinion the newest patches can be applied, tests passed on my side. Thank you. Thanks. However, I get compilation errors, probably due to the macro `org-table-execute-once-per-second'. In particular, using multiple ,t1 isn't a good idea. BTW, Nathaniel, these patches are no tiny changes. Have you signed FSF papers yet? Regards, -- Nicolas Goaziou 0001-org-table.el-org-table-recalculate-is-quieter.patch Description: Binary data 0001-org-table.el-org-table-recalculate-early-returns(1).patch Description: Binary data
Re: [O] [PATH] Speedups to org-table-recalculate
Nathaniel Flath flat0...@gmail.com writes: OK, I think I fixed that. Thanks. I didn't verify it compiles, but your macro still looks suspicious. +(defmacro org-table-execute-once-per-second (t1 rest body) + If there has been more than one second since T1, execute BODY. +Updates T1 to 'current-time' if this condition is met. If T1 is +nil, always execute body. + `(let ((t1 ,t1)) + (if t1 + (let ((curtime (current-time))) +(when ( 0 (nth 1 (time-subtract curtime t1))) + (setq t1 curtime) +,@body)) + ,@body))) You shouldn't splice BODY twice in your macro. Also, I don't get why you need to (setq t1 curtime). Do you need a macro at all for this task? ISTM you only need to display a message conditionally and update a time value. Regard,
Re: [O] [PATH] Speedups to org-table-recalculate
Aaaand another try. On Thu, Nov 13, 2014 at 12:39 AM, Michael Brand michael.ch.br...@gmail.com wrote: Hi Nathaniel On Wed, Nov 12, 2014 at 12:51 PM, Nathaniel Flath flat0...@gmail.com wrote: New patches attached! Now that they apply I found: + Re-applying formulas to full table...(line %d Missing cnt. + (message Re-applying formulas...done cnt)) Superfluous cnt. Michael 0002-org-table.el-org-table-recalculate-is-quieter.patch Description: Binary data 0001-org-table.el-org-table-recalculate-early-returns(1).patch Description: Binary data
Re: [O] [PATH] Speedups to org-table-recalculate
Hi Nathaniel On Fri, Nov 14, 2014 at 2:33 PM, Nathaniel Flath flat0...@gmail.com wrote: Aaaand another try. I tested them and got an error because of a missing cnt: + (message Re-applying formulas to %d lines...done Michael
Re: [O] [PATH] Speedups to org-table-recalculate
Bleh. Took out the wrong cnt, and that didn't show up in my testing for some reason. On Fri, Nov 14, 2014 at 11:10 PM, Michael Brand michael.ch.br...@gmail.com wrote: Hi Nathaniel On Fri, Nov 14, 2014 at 2:33 PM, Nathaniel Flath flat0...@gmail.com wrote: Aaaand another try. I tested them and got an error because of a missing cnt: + (message Re-applying formulas to %d lines...done Michael 0001-org-table.el-org-table-recalculate-early-returns(1).patch Description: Binary data 0002-org-table.el-org-table-recalculate-is-quieter.patch Description: Binary data
Re: [O] [PATH] Speedups to org-table-recalculate
Hi Nathaniel On Fri, Nov 14, 2014 at 7:00 PM, Nathaniel Flath flat0...@gmail.com wrote: Bleh. Took out the wrong cnt, and that didn't show up in my testing for some reason. In my opinion the newest patches can be applied, tests passed on my side. Thank you. Michael
Re: [O] [PATH] Speedups to org-table-recalculate
Hello, Michael Brand michael.ch.br...@gmail.com writes: In my opinion the newest patches can be applied, tests passed on my side. Thank you. Thanks. However, I get compilation errors, probably due to the macro `org-table-execute-once-per-second'. In particular, using multiple ,t1 isn't a good idea. BTW, Nathaniel, these patches are no tiny changes. Have you signed FSF papers yet? Regards, -- Nicolas Goaziou
Re: [O] [PATH] Speedups to org-table-recalculate
New patches attached! On Sun, Nov 9, 2014 at 9:12 PM, Michael Brand michael.ch.br...@gmail.com wrote: Hi Nathaniel On Sun, Nov 9, 2014 at 11:18 AM, Nathaniel Flath flat0...@gmail.com wrote: Updated patches attached. The second does not apply after the first on today's release_8.3beta-552-ga95cfeb. Unrelated: The second has new closing parentheses on an own line. Michael 0002-org-table.el-org-table-recalculate-is-quieter.patch Description: Binary data 0001-org-table.el-org-table-recalculate-early-returns.patch Description: Binary data
Re: [O] [PATH] Speedups to org-table-recalculate
Hi Nathaniel On Wed, Nov 12, 2014 at 12:51 PM, Nathaniel Flath flat0...@gmail.com wrote: New patches attached! Now that they apply I found: + Re-applying formulas to full table...(line %d Missing cnt. + (message Re-applying formulas...done cnt)) Superfluous cnt. Michael
Re: [O] [PATH] Speedups to org-table-recalculate
Updated patches attached. 0001-org-table.el-org-table-recalculate-early-returns.patch Description: Binary data 0002-org-table.el-org-table-recalculate-is-quieter.patch Description: Binary data
Re: [O] [PATH] Speedups to org-table-recalculate
Hi Nathaniel On Sun, Nov 9, 2014 at 11:18 AM, Nathaniel Flath flat0...@gmail.com wrote: Updated patches attached. The second does not apply after the first on today's release_8.3beta-552-ga95cfeb. Unrelated: The second has new closing parentheses on an own line. Michael
Re: [O] [PATH] Speedups to org-table-recalculate
Hi Nathaniel On Sun, Oct 26, 2014 at 2:27 AM, Nathaniel Flath flat0...@gmail.com wrote: I'm fine with adding the (when all log-first-time). I don't want to leave it unconditional Ok, then I agree on (when all log-first-time). Michael
Re: [O] [PATH] Speedups to org-table-recalculate
Hi Michael On Mon, Oct 20, 2014 at 12:41 PM, Michael Brand michael.ch.br...@gmail.com wrote: Hi Nathaniel On Mon, Oct 20, 2014 at 3:56 AM, Nathaniel Flath flat0...@gmail.com wrote: Thanks for the review! Updated patches attached. I believe I've fixed everything you mentioned - Yes, thank you. I tried them out. let me know if I missed something. Subject: [PATCH 1/2] org-table.el: Add early return check to org-table-recalculate Subject: [PATCH 2/2] org-table.el: Print far fewer messages when recalculating tables. http://orgmode.org/worg/org-contribute.html says: Line 2 is an empty line. I try to limit my commit subjects to 50 chars which is a rule for some projects and helpful for e. g. git log with certain options. OK, will fix these in next set of pathces. +(and all (org-table-execute-once-per-second + log-last-time + (message + Re-applying formulas to full table...(line %d) + (setq cnt (1+ cnt) Before trying out I have overseen it, setq does not increment on each line any more so it has to move before org-table-execute-once-per-second. Now probably better wrapped in a `when' instead of the original `and' with an added `progn'. OK, good catch +(org-table-execute-once-per-second + log-first-time + (message Re-applying formulas...done cnt)) cnt is superfluous. OK. Only during trying out I noticed: This message could still be kept as a feedback at least for single row updates, like e. g. Tab on a row with # in the first column, by changing log-first-time to (when all log-first-time). Or more conservative why not leave this message unconditional to avoid that any user could complain for any use case that all feedback messages disappeared like it would be the case for e. g. the typical C-c C-c on TBLFM? I think that would be a simple, safe compromise. I'm fine with adding the (when all log-first-time). I don't want to leave it unconditional because as we discussed before, one message in my case significantly slows down the table recalculation - and if there has been no feedback then we are dealing with a small case where it could be significant. Otherwise the right but not so simple thing to do would be to log this message for a simple C-c C-c on TBLFM and to not log it for only those cases that throw after other feedback messages like e. g. sometimes C-u C-u C-c C-c on TBLFM (Table was already stable), M-x org-table-iterate-buffer-tables and more. These repetition intensive use cases are the only ones that can profit noticeable from the conditional removal of this last message, aren't they? I got tired of having to force tables to be reformated, so I advised org-cycle to always recalculate a table, so I notice whenever I'm in a table. Michael
Re: [O] [PATH] Speedups to org-table-recalculate
Hi Nathaniel On Mon, Oct 20, 2014 at 3:56 AM, Nathaniel Flath flat0...@gmail.com wrote: Thanks for the review! Updated patches attached. I believe I've fixed everything you mentioned - Yes, thank you. I tried them out. let me know if I missed something. Subject: [PATCH 1/2] org-table.el: Add early return check to org-table-recalculate Subject: [PATCH 2/2] org-table.el: Print far fewer messages when recalculating tables. http://orgmode.org/worg/org-contribute.html says: Line 2 is an empty line. I try to limit my commit subjects to 50 chars which is a rule for some projects and helpful for e. g. git log with certain options. +(and all (org-table-execute-once-per-second + log-last-time + (message + Re-applying formulas to full table...(line %d) + (setq cnt (1+ cnt) Before trying out I have overseen it, setq does not increment on each line any more so it has to move before org-table-execute-once-per-second. Now probably better wrapped in a `when' instead of the original `and' with an added `progn'. +(org-table-execute-once-per-second + log-first-time + (message Re-applying formulas...done cnt)) cnt is superfluous. Only during trying out I noticed: This message could still be kept as a feedback at least for single row updates, like e. g. Tab on a row with # in the first column, by changing log-first-time to (when all log-first-time). Or more conservative why not leave this message unconditional to avoid that any user could complain for any use case that all feedback messages disappeared like it would be the case for e. g. the typical C-c C-c on TBLFM? I think that would be a simple, safe compromise. Otherwise the right but not so simple thing to do would be to log this message for a simple C-c C-c on TBLFM and to not log it for only those cases that throw after other feedback messages like e. g. sometimes C-u C-u C-c C-c on TBLFM (Table was already stable), M-x org-table-iterate-buffer-tables and more. These repetition intensive use cases are the only ones that can profit noticeable from the conditional removal of this last message, aren't they? Michael
Re: [O] [PATH] Speedups to org-table-recalculate
Hi Nathaniel On Sat, Oct 18, 2014 at 7:11 AM, Nathaniel Flath flat0...@gmail.com wrote: Patches are attached. I am not an expert for all the following comments, please correct me or contradict where necessary. The patches do not apply on current master, so I did not try them out yet. You might want to add a def-edebug-spec like there is one for many other defmacro in Org. Limit lines to max. 80 chars. It will make it easier for the maintainer Bastien to apply the patches when you format them with git including a changelog etc. as described here http://orgmode.org/worg/org-contribute.html + (if (not all) (message Re-applying formula to field: %s (car eq)) +(org-table-execute-once-per-second log-last-time (message Re-applying formula to field: %s (car eq Good idea to still log always when only one table row is recalculated. The doubling of the message makes it more complicated to maintain its string. I suggest to change the macro to allow (org-table-execute-once-per-second (when all log-last-time) ; Log just always when `all' is nil. (message Re-applying formula to field: %s (car eq))) Why not test `all' also for the other message with to field? + ,@body + ))) Parentheses not alone on an own line. +(and all (org-table-execute-once-per-second log-last-time (message Re-applying formulas to %d lines...done cnt Shouldn't this use `log-first-time'? Michael
Re: [O] [PATH] Speedups to org-table-recalculate
Hi Michael, Thanks for the review! Updated patches attached. I believe I've fixed everything you mentioned - let me know if I missed something. On Sun, Oct 19, 2014 at 3:57 PM, Michael Brand michael.ch.br...@gmail.com wrote: Hi Nathaniel On Sat, Oct 18, 2014 at 7:11 AM, Nathaniel Flath flat0...@gmail.com wrote: Patches are attached. I am not an expert for all the following comments, please correct me or contradict where necessary. The patches do not apply on current master, so I did not try them out yet. You might want to add a def-edebug-spec like there is one for many other defmacro in Org. Limit lines to max. 80 chars. It will make it easier for the maintainer Bastien to apply the patches when you format them with git including a changelog etc. as described here http://orgmode.org/worg/org-contribute.html + (if (not all) (message Re-applying formula to field: %s (car eq)) +(org-table-execute-once-per-second log-last-time (message Re-applying formula to field: %s (car eq Good idea to still log always when only one table row is recalculated. The doubling of the message makes it more complicated to maintain its string. I suggest to change the macro to allow (org-table-execute-once-per-second (when all log-last-time) ; Log just always when `all' is nil. (message Re-applying formula to field: %s (car eq))) Why not test `all' also for the other message with to field? + ,@body + ))) +(and all (org-table-execute-once-per-second log-last-time (message Re-applying formulas to %d lines...done cnt Shouldn't this use `log-first-time'? Michael 0001-org-table.el-Add-early-return-check-to-org-table-rec.patch Description: Binary data 0002-org-table.el-Print-far-fewer-messages-when-recalcula.patch Description: Binary data
Re: [O] [PATH] Speedups to org-table-recalculate
Hi On Sat, Oct 11, 2014 at 12:16 PM, Michael Brand michael.ch.br...@gmail.com wrote: Hi Nathaniel On Fri, Oct 10, 2014 at 9:43 PM, Nathaniel Flath flat0...@gmail.com wrote: Mine is a pretty simple table (takes less than a second even in the original case): Earlier I assumed that the issue is a very high number of messages from the loops. Now your example table clarifies to me that the issue is that already just one single message can take a significant time of a message-less table recalculation (your 17 ms). Only with this I understand now why you want to remove also the beginning/end processing messages. Good point as it should be noticeable at least for org-table-iterate-buffer-tables in a buffer with many tables where each one takes a short time to recalculate. I really wonder what the reason is that this (progn (message %d (random)) ;; Prevent collapsing of message lines. (time (message nil))) in a terminal (emacs -nw) shows most of the times only 0.05 to 0.10 ms but in a window it shows most of the times 8 to 22 ms (here: GNU Emacs 24.3.1 on Mac OS X 10.9). Which Emacs version and OS are you using? I'm running 24.3.1 on Mac OS X 10.8.4. For your patch I suggest to remove only the first message and to add the time check to all other messages. This should not make the patch noticeably slower but would keep showing the progress for table recalculations that last more than one second. To clean up the last loop message from the mode line I suggest to check the end messages against the very first log time in contrast to the next log time used for the loop messages (variables log_first and log_next instead of just log). I suggest you split your patch: One for (when eqlist and one for the messages. The change with the messages will then become human readable also with a simple line diff. OK, sounds reasonable. Patches are attached. Original recalculation: (0 0 396224 0) Version w/ time checks for per-field messages (still always printing at beginning/end of processing):(0 0 56929 0) Version w/ time checks and removing all beginning/end of processing messages: (0 0 22077 0) My patch: (0 0 17405 0) I could not reproduce a reliable difference between the last two. As expected both did not log any message. Can you? Rrunning more iterations they seemed to be mostly equal. Patches are attached. Michael org-table-speedup-1.patch Description: Binary data org-table-speedup-2.patch Description: Binary data
Re: [O] [PATH] Speedups to org-table-recalculate
Hi Nathaniel On Fri, Oct 10, 2014 at 9:43 PM, Nathaniel Flath flat0...@gmail.com wrote: Mine is a pretty simple table (takes less than a second even in the original case): Earlier I assumed that the issue is a very high number of messages from the loops. Now your example table clarifies to me that the issue is that already just one single message can take a significant time of a message-less table recalculation (your 17 ms). Only with this I understand now why you want to remove also the beginning/end processing messages. Good point as it should be noticeable at least for org-table-iterate-buffer-tables in a buffer with many tables where each one takes a short time to recalculate. I really wonder what the reason is that this (progn (message %d (random)) ;; Prevent collapsing of message lines. (time (message nil))) in a terminal (emacs -nw) shows most of the times only 0.05 to 0.10 ms but in a window it shows most of the times 8 to 22 ms (here: GNU Emacs 24.3.1 on Mac OS X 10.9). Which Emacs version and OS are you using? For your patch I suggest to remove only the first message and to add the time check to all other messages. This should not make the patch noticeably slower but would keep showing the progress for table recalculations that last more than one second. To clean up the last loop message from the mode line I suggest to check the end messages against the very first log time in contrast to the next log time used for the loop messages (variables log_first and log_next instead of just log). I suggest you split your patch: One for (when eqlist and one for the messages. The change with the messages will then become human readable also with a simple line diff. Original recalculation: (0 0 396224 0) Version w/ time checks for per-field messages (still always printing at beginning/end of processing):(0 0 56929 0) Version w/ time checks and removing all beginning/end of processing messages: (0 0 22077 0) My patch: (0 0 17405 0) I could not reproduce a reliable difference between the last two. As expected both did not log any message. Can you? Michael
Re: [O] [PATH] Speedups to org-table-recalculate
Hi Nathaniel On Fri, Oct 10, 2014 at 7:56 AM, Nathaniel Flath flat0...@gmail.com wrote: That's still much more slow than not doing it - slightly modifying your example,: (progn (setq start (current-time)) (let ((row 0) (log (time-add (current-time) '(0 1 0 0 (while ( row 6543210) (setq row (1+ row)) (when (time-less-p log (current-time)) (setq log (time-add (current-time) '(0 1 0 0))) (message row %d row (setq end (current-time)) (print (time-subtract end start))) prints (0 43 386499 0) on my computer. Removing the when clause: (progn (setq start (current-time)) (let ((row 0) (log (time-add (current-time) '(0 1 0 0 (while ( row 6543210) (setq row (1+ row (setq end (current-time)) (print (time-subtract end start))) Results in: (0 1 277641 0) So adding the logging here slows it down by about 43x - It doesn't seem worth it. Your measurement shows that (when (time-less-p log (current-time)) [...] takes 6.4 microseconds or can run 150'000 times per second. I would expect it to be negligible compared to what Org has to do for each row or field like parse, calculate, format etc. Otherwise it would mean that Org can perform more or not significantly less than 150'000 rows or fields per second on an appropriate example table. Tersely formulated I expect this performance comparison: nothing or empty loop a conditional message with time check Org performs a simple formula on one row or field an unconditional message Can you make a performance comparison on your table between (a) your patch and (b) without your patch but with (when (time-less-p log (current-time)) [...] plus describe or share this table? Michael
Re: [O] [PATH] Speedups to org-table-recalculate
Mine is a pretty simple table (takes less than a second even in the original case): | Category | Budget | Spent | Remaining | |--++---+---| | A|100 | 0 | 100 | | B|100 | 0 | 100 | | C|100 | 0 | 100 | | D|100 | 0 | 100 | | E|100 | 0 | 100 | | F|100 | 0 | 100 | | G|100 | 0 | 100 | | H|100 | 0 | 100 | | I|100 | 0 | 100 | | J|100 | 0 | 100 | | K|100 | 0 | 100 | | L|100 | 0 | 100 | | M|100 | 0 | 100 | | N|100 | 0 | 100 | | O|100 | 0 | 100 | | K|100 | 0 | 100 | |--++---+---| | Total| 1600 | 0 | 1600 | #+TBLFM: $4=$2-$3::@18$2=vsum(@2$2..@-1)::@18$3=vsum(@2$3..@-1) With the macro: (defmacro time (block) `(let (start end) (setq start (current-time)) ,block (setq end (current-time)) (print (time-subtract end start and running (time (org-table-recalculate t)) Original recalculation: (0 0 396224 0) Version w/ time checks for per-field messages (still always printing at beginning/end of processing):(0 0 56929 0) Version w/ time checks and removing all beginning/end of processing messages: (0 0 22077 0) My patch: (0 0 17405 0) So, it's still a 26% performance degradation to going with the patch and removing the 'global' messaging, but I could probably live with that - qualitatively, there doesn't seem to be too much difference between my patch and doing that, but the original version is obviously slow and with the on-begin/end calculation messages the delay is much more noticable. On Fri, Oct 10, 2014 at 3:35 AM, Michael Brand michael.ch.br...@gmail.com wrote: Hi Nathaniel On Fri, Oct 10, 2014 at 7:56 AM, Nathaniel Flath flat0...@gmail.com wrote: That's still much more slow than not doing it - slightly modifying your example,: (progn (setq start (current-time)) (let ((row 0) (log (time-add (current-time) '(0 1 0 0 (while ( row 6543210) (setq row (1+ row)) (when (time-less-p log (current-time)) (setq log (time-add (current-time) '(0 1 0 0))) (message row %d row (setq end (current-time)) (print (time-subtract end start))) prints (0 43 386499 0) on my computer. Removing the when clause: (progn (setq start (current-time)) (let ((row 0) (log (time-add (current-time) '(0 1 0 0 (while ( row 6543210) (setq row (1+ row (setq end (current-time)) (print (time-subtract end start))) Results in: (0 1 277641 0) So adding the logging here slows it down by about 43x - It doesn't seem worth it. Your measurement shows that (when (time-less-p log (current-time)) [...] takes 6.4 microseconds or can run 150'000 times per second. I would expect it to be negligible compared to what Org has to do for each row or field like parse, calculate, format etc. Otherwise it would mean that Org can perform more or not significantly less than 150'000 rows or fields per second on an appropriate example table. Tersely formulated I expect this performance comparison: nothing or empty loop a conditional message with time check Org performs a simple formula on one row or field an unconditional message Can you make a performance comparison on your table between (a) your patch and (b) without your patch but with (when (time-less-p log (current-time)) [...] plus describe or share this table? Michael
Re: [O] [PATH] Speedups to org-table-recalculate
Sorry for the late response - missed this for a while. That's still much more slow than not doing it - slightly modifying your example,: (progn (setq start (current-time)) (let ((row 0) (log (time-add (current-time) '(0 1 0 0 (while ( row 6543210) (setq row (1+ row)) (when (time-less-p log (current-time)) (setq log (time-add (current-time) '(0 1 0 0))) (message row %d row (setq end (current-time)) (print (time-subtract end start))) prints (0 43 386499 0) on my computer. Removing the when clause: (progn (setq start (current-time)) (let ((row 0) (log (time-add (current-time) '(0 1 0 0 (while ( row 6543210) (setq row (1+ row (setq end (current-time)) (print (time-subtract end start))) Results in: (0 1 277641 0) So adding the logging here slows it down by about 43x - It doesn't seem worth it. On Sun, Aug 17, 2014 at 6:39 AM, Michael Brand michael.ch.br...@gmail.com wrote: Hi Nathaniel On Thu, Aug 7, 2014 at 4:57 PM, Nathaniel Flath flat0...@gmail.com wrote: I'd be fine with displaying every second, but I don't see a good way of doing this - do you have any suggestions? I thought of something like in this example: (let ((row 0) (log (time-add (current-time) '(0 1 0 0 (while ( row 6543210) (setq row (1+ row)) (when (time-less-p log (current-time)) (setq log (time-add (current-time) '(0 1 0 0))) (message row %d row Michael
Re: [O] [PATH] Speedups to org-table-recalculate
Hi Nathaniel On Thu, Aug 7, 2014 at 4:57 PM, Nathaniel Flath flat0...@gmail.com wrote: I'd be fine with displaying every second, but I don't see a good way of doing this - do you have any suggestions? I thought of something like in this example: (let ((row 0) (log (time-add (current-time) '(0 1 0 0 (while ( row 6543210) (setq row (1+ row)) (when (time-less-p log (current-time)) (setq log (time-add (current-time) '(0 1 0 0))) (message row %d row Michael
Re: [O] [PATH] Speedups to org-table-recalculate
Yes, that wouldn't be supported - although certainly in my case what was making it slow *were* the messages. I'd be fine with displaying every second, but I don't see a good way of doing this - do you have any suggestions? On Fri, Aug 1, 2014 at 2:56 PM, Michael Brand michael.ch.br...@gmail.com wrote: Hi Nathaniel On Tue, Jul 29, 2014 at 2:03 PM, Nathaniel Flath flat0...@gmail.com wrote: This patch speeds up org-table-recalculate by removing all 'message' function calls. Additionally adds an early check for whether there are any formulas and only executes the rest of the function if so. As far as I understand these `message' were added to see the progress of a long lasting recalculation which would not be supported any more with your patch. But I agree that it would be sufficient to report the current field/line only every let's say one second and suppress the others to gain speed for use cases like yours. Michael
Re: [O] [PATH] Speedups to org-table-recalculate
Hi Nathaniel On Tue, Jul 29, 2014 at 2:03 PM, Nathaniel Flath flat0...@gmail.com wrote: This patch speeds up org-table-recalculate by removing all 'message' function calls. Additionally adds an early check for whether there are any formulas and only executes the rest of the function if so. As far as I understand these `message' were added to see the progress of a long lasting recalculation which would not be supported any more with your patch. But I agree that it would be sufficient to report the current field/line only every let's say one second and suppress the others to gain speed for use cases like yours. Michael
[O] [PATH] Speedups to org-table-recalculate
This patch speeds up org-table-recalculate by removing all 'message' function calls. Additionally adds an early check for whether there are any formulas and only executes the rest of the function if so. org-table-speedup.patch Description: Binary data
Re: [O] [PATH] Speedups to org-table-recalculate
Hi Nathaniel, Nathaniel Flath flat0...@gmail.com writes: This patch speeds up org-table-recalculate by removing all 'message' function calls. Additionally adds an early check for whether there are any formulas and only executes the rest of the function if so. The patch is bigger than what can be accepted without requiring a copyright assignment. Here is the form you would need to send if you agree: http://orgmode.org/cgit.cgi/org-mode.git/plain/request-assign-future.txt It can take a month, especially since this is summer time. Thanks in advance! -- Bastien
Re: [O] [PATH] Speedups to org-table-recalculate
Hi Bastien, I believe I did that a long time ago, juste been inactive. I've definitely contributed to org before, although searching through my email I can only find a copyright assignment for Emacs itself (dated 8/23/09), do I have to re-do this? Thanks, Nathaniel Flath On Tue, Jul 29, 2014 at 2:30 PM, Bastien b...@gnu.org wrote: Hi Nathaniel, Nathaniel Flath flat0...@gmail.com writes: This patch speeds up org-table-recalculate by removing all 'message' function calls. Additionally adds an early check for whether there are any formulas and only executes the rest of the function if so. The patch is bigger than what can be accepted without requiring a copyright assignment. Here is the form you would need to send if you agree: http://orgmode.org/cgit.cgi/org-mode.git/plain/request-assign-future.txt It can take a month, especially since this is summer time. Thanks in advance! -- Bastien
Re: [O] [PATH] Speedups to org-table-recalculate
Hi Nathaniel, Nathaniel Flath flat0...@gmail.com writes: I believe I did that a long time ago, juste been inactive. I've definitely contributed to org before, although searching through my email I can only find a copyright assignment for Emacs itself (dated 8/23/09), do I have to re-do this? Of course you don't have to redo this -- sorry I forgot. I'll have a look at the patch when I'm back from holidays in three weeks. Thanks, -- Bastien
