Re: Emacs bug 37890; killing capture buffer
Michael Heerdegen writes: > Or (really better IMHO) consider a different implementation where the > original buffer is not modified until the user explicitly confirms the > stuff to capture with C-c C-c. That would be helpful in some ways, but harmful in others. For example, consider a capture that is started while in a meeting, on a phone call, away from one's desk, etc., with some notes in it, clock start time, etc. (You can find examples of this workflow in, e.g. Bernt Hansen's Org config.) If Emacs were interrupted (crash, power failure, reboot, etc), the un-finalized capture would still be present in the auto-save file and could be recovered when restarting Emacs and finding the file again. But if the original buffer were not modified until the capture is finalized, where would the unfinalized data be, and how would it be recovered into the desired capture location? The way Org uses indirect, narrowed buffers for capturing is an elegant use of Emacs features that helps protect user data from accidental loss.
Re: Asynchronous org-agenda-redo
Ihor Radchenko writes: >>> Asynchronous code is not faster; it's generally slower because of >>> yielding and synchronization. > >> Anyway, I will try to throw yields into agenda code just to check how >> bad the performance can degrade. > > With the following code, org-agenda-redo runs for 21 second on my > system, while without threads it is 16 seconds. However, emacs remains > responsive during rebuilding agenda! > > (define-advice org-agenda-redo (:around (oldfun all) make-async) > (make-thread (lambda () (funcall oldfun all)) "org-agenda-redo")) > (define-advice org-agenda-skip-eval (:around (oldfun form) make-async) > (thread-join (make-thread (lambda () (funcall oldfun form)) > "org-agenda-skip-eval"))) That's a very elegant way to add the threading! The performance penalty is noticeable, but the tradeoff could be worth it in some cases, like a background agenda refresh on a timer, or after a "remote" edit. I can imagine an org-agenda-refresh-async command that would add that advice and remove them in an unwind-protect. > The problem, of course, is that touching agenda buffer and org buffers > may be risky while org-agenda-redo is running. > Wondering if it is possible to block user commands during that time. The first thing that comes to mind is to set buffer-read-only on each buffer before it is scanned, and unset it when done with a buffer. That might not be doable with advice.
Re: Asynchronous org-agenda-redo
Ihor Radchenko writes: >> org-ql doesn't use skip functions, just queries. > > Skip functions are essentially used-defined queries as soon as the > queries are tested against every headline. Skip functions aren't necessary with org-ql, because the query itself can have arbitrary Lisp code. So, of course, you can call custom functions in queries, even your own skip functions (with `not', of course), but in most cases, they can be covered with built-in predicates. > I can rewrite my skip functions into queries, but I don't expect much > improvement since org-ql seems to use org-entry-get, which is the main > performance bottleneck for my agenda generation. org-entry-get is only called for the (property) predicate. It's the correct way to get Org properties, because it handles special properties, inheritance, etc. However, when possible, queries are optimized to a whole-buffer regexp search that finds possible matches. So, for example, a query like '(property "owner" "yantar") would be optimized to a whole-buffer regexp search that would be very fast. See function org-ql--query-preamble.
Re: [PATCH] Fix verbatim block fontification to end blocks on headlines
Adam Porter writes: > May I recommend using the rx macro for regexps? They are much easier > for humans to parse, which helps reduce errors like the ones mentioned > here. And they are about to gain some very useful new features > in Emacs 27. Yep. I'll switch the regex in over to use rx. An unrelated question. I've written some basic tests for this and I couldn't find any other tests that seemed to deal with fontification at all. In order to get fontification tests to work I added a call to `font-lock-ensure' inside `org-test-with-temp-text' (see excerpted patch bit below). Given how frequently `org-test-with-temp-text' is used, does it make sense to create a separate version of that macro just for testing with fontification? I have no idea what the performance impact would be, so any guidance is appreciated. diff --git a/testing/org-test.el b/testing/org-test.el index c3e21eb30..e97e2eaa4 100644 --- a/testing/org-test.el +++ b/testing/org-test.el @@ -198,7 +198,8 @@ otherwise place the point at the beginning of the inserted text." (insert (replace-match "" nil nil inside-text)) (goto-char (1+ (match-beginning 0 (insert inside-text) - (goto-char (point-min + (goto-char (point-min))) +(font-lock-ensure (point-min) (point-max))) ,@body))) (def-edebug-spec org-test-with-temp-text (form body))
RE: [O] FW: [RFC] Link-type for attachments, more attach options
FYI, pushed to master. Commit 26ace9004 /Gustav > -Original Message- > From: Emacs-orgmode On Behalf > Of Gustav Wikström > Sent: den 13 december 2019 22:37 > To: numbch...@gmail.com > Cc: emacs-orgmode@gnu.org > Subject: RE: [O] FW: [RFC] Link-type for attachments, more attach options > > Hi, > > > -Original Message- > > From: stardiviner > > Sent: den 13 december 2019 14:39 > > To: Gustav Wikström > > Cc: numbch...@gmail.com; emacs-orgmode@gnu.org > > Subject: Re: [O] FW: [RFC] Link-type for attachments, more attach options > > > > ... > > > I tried to reproduce that using emacs -q just now but couldn't... Is there > a > > > customization that you've enabled? > > > > Aha, seems yes. I checked my config. I have option enabled: > > > > #+begin_src emacs-lisp > > (setq org-attach-store-link-p 'attached) > > #+end_src > > > > About this feature, WDYT? > > Ah, yes of course. What you propose make sense. More sense than the current > functionality! > > If no one objects I'll change the link that is stored for the 'attached option > to be an > attachment link instead of a file link. > > Regards > Gustav
Emacs bug 37890; killing capture buffer
Hi, I want to speak about my Emacs bug report 37890 about org-capture. Seems my main point: | I want to capture an APPT with `org-capture'. I the pop-up buffer to | edit the item I move the date to the second line and add text after the | date (personal preference). That loses the final newline in | CAPTURE-todo.org. As a result, the headline of the item following the | item to be inserted gets appended to the last line of the text: | | ** APPT Abc |<2019-10-23 Mi> | text... ** APPT 8:30 Important Appointment | | breaking the whole item. The user should somehow be prevented from that | happening. has been resolved with the latest merge into Emacs master - is that correct? Then I will close that report. The report also included a feature request which I now want to tell here: I often kill the capture buffer instead of hitting C-c C-k. Just by habit. It's wrong and I now it but I guess it happens to others. I guess the capture buffer is just a narrowed indirect buffer copy of the buffer visiting the according org file. Because of this, when you kill the capture buffer, the original buffer reflects the partial and uncomplete entry one tried to add. This may damage your file. But such internals are not known to all, so it would be good if killing the capture buffer, or even just closing the window, would warn the user and offer to undo the partial changes. Or (really better IMHO) consider a different implementation where the original buffer is not modified until the user explicitly confirms the stuff to capture with C-c C-c. TIA, Michael.
RE: [O] FW: [RFC] Link-type for attachments, more attach options
Hi, > -Original Message- > From: stardiviner > Sent: den 13 december 2019 14:39 > To: Gustav Wikström > Cc: numbch...@gmail.com; emacs-orgmode@gnu.org > Subject: Re: [O] FW: [RFC] Link-type for attachments, more attach options > > ... > > I tried to reproduce that using emacs -q just now but couldn't... Is there a > > customization that you've enabled? > > Aha, seems yes. I checked my config. I have option enabled: > > #+begin_src emacs-lisp > (setq org-attach-store-link-p 'attached) > #+end_src > > About this feature, WDYT? Ah, yes of course. What you propose make sense. More sense than the current functionality! If no one objects I'll change the link that is stored for the 'attached option to be an attachment link instead of a file link. Regards Gustav
Re: [O] FW: [RFC] Link-type for attachments, more attach options
Gustav Wikström writes: > Hi, > >> -Original Message- >> From: stardiviner >> Sent: den 12 december 2019 10:53 >> To: Gustav Wikström >> Cc: numbch...@gmail.com; emacs-orgmode@gnu.org >> Subject: Re: [O] FW: [RFC] Link-type for attachments, more attach options >> >> ... >> >> For example, I press =[Ctrl-c Ctrl-a]= to attach a file. Then I press >> =[Ctrl-c >> Ctrl-l]= (~org-insert-link~) to insert link which will show a list of >> completions >> which are all link types prefix like ~attachment:~, and >> =file:data/a2//attachFile.png=. I mean the second link. it is already in >> completion list, but ~attchment:~ does not have this support. > > Hmm, I'm not sure I follow. Is it in the same suggestion list for link type > prefixes > that you also get the file-link to the newly attached file? Yes. > I tried to reproduce that using emacs -q just now but couldn't... Is there a > customization that you've enabled? Aha, seems yes. I checked my config. I have option enabled: #+begin_src emacs-lisp (setq org-attach-store-link-p 'attached) #+end_src About this feature, WDYT? > > Regards > Gustav -- [ stardiviner ] I try to make every word tell the meaning what I want to express. Blog: https://stardiviner.github.io/ IRC(freenode): stardiviner, Matrix: stardiviner GPG: F09F650D7D674819892591401B5DF1C95AE89AC3
Re: Asynchronous org-agenda-redo
>> Asynchronous code is not faster; it's generally slower because of >> yielding and synchronization. > Anyway, I will try to throw yields into agenda code just to check how > bad the performance can degrade. With the following code, org-agenda-redo runs for 21 second on my system, while without threads it is 16 seconds. However, emacs remains responsive during rebuilding agenda! (define-advice org-agenda-redo (:around (oldfun all) make-async) (make-thread (lambda () (funcall oldfun all)) "org-agenda-redo")) (define-advice org-agenda-skip-eval (:around (oldfun form) make-async) (thread-join (make-thread (lambda () (funcall oldfun form)) "org-agenda-skip-eval"))) The problem, of course, is that touching agenda buffer and org buffers may be risky while org-agenda-redo is running. Wondering if it is possible to block user commands during that time. Best, Ihor Ihor Radchenko writes: >> Org Agenda code does not wait for keyboard input; it's busy building the >> agenda. This is the case with most code in Emacs: it's not written to >> be asynchronous, and it doesn't return to the main thread until done. >> So you can sprinkle yields here and there and maybe be able to move >> point around while some code is running, but that will decrease >> performance, as well as introducing another level of complexity and >> another class of bugs (e.g. what if the user modifies a buffer while the >> agenda code is scanning it?). > > Thanks for the explanation. > >> AFAIK there exists no way to do such a thing. Buffers are not designed >> to be serialized/deserialized like that. You could try writing some >> Elisp code to do it, but the end result would probably be much slower >> than existing agenda code, as well as more difficult to debug. > > Yeah. Even re-initialisation of, for example, overlays in org buffer is > likely to take too much time. > >> As you can see in org-agenda.el, it's complicated. Remember that an >> Emacs process is like a Lisp image, full of state. The more symbols and >> other structures you copy to the async Emacs process (by printing and >> reading them as text, remember), the slower it's going to be--and it >> will always be slower than not using async. > >> Asynchronous code is not faster; it's generally slower because of >> yielding and synchronization. > > I see now that generating agenda in separate process will cause too much > overheads. > Anyway, I will try to throw yields into agenda code just to check how > bad the performance can degrade. > >> org-ql doesn't use skip functions, just queries. > > Skip functions are essentially used-defined queries as soon as the > queries are tested against every headline. > I can rewrite my skip functions into queries, but I don't expect much > improvement since org-ql seems to use org-entry-get, which is the main > performance bottleneck for my agenda generation. > > Best, > Ihor > > adam Porter writes: > >> Ihor Radchenko writes: >> Be sure to read the Emacs Lisp manual regarding threads. They are cooperative, so functions called as threads must yield back to the main thread for Emacs to do anything else before the function returns. >>> >>> I tried to read the manual, but I clearly misunderstand something. >>> The manual says: >>> Currently, thread switching will occur upon explicit request via ‘thread-yield’, when waiting for keyboard input... >>> >>> So, except directly calling thread-yield, it should be possible to >>> trigger switching the current thread when keyboard input is expected. >>> I tried the following demo code: >>> >>> (defun test () >>> (let ((a 0)) >>> (dotimes (_ 5) >>> (setq a (1+ a)) >>> (sleep-for 2) >>> (message "%s" a >>> >>> (progn ;This should return to command loop quickly >>> (make-thread #'test) >>> (message "Executed...")); `eval-last-sexp' here >>> >>> I can move around the buffer while the progn is running. >>> However, it is not the case with `org-agenda-redo' for a reason I do not >>> fully understand. >> >> Org Agenda code does not wait for keyboard input; it's busy building the >> agenda. This is the case with most code in Emacs: it's not written to >> be asynchronous, and it doesn't return to the main thread until done. >> So you can sprinkle yields here and there and maybe be able to move >> point around while some code is running, but that will decrease >> performance, as well as introducing another level of complexity and >> another class of bugs (e.g. what if the user modifies a buffer while the >> agenda code is scanning it?). >> 1. The process would have to load the same Org buffers, which takes time, especially in large buffers. Depending on configuration, it can take some time, indeed. >>> 3. Ensuring that configuration and state between the main Emacs process and the separate, agenda-generating process is not necessarily simple. Consider as well that if a buffer had unsaved
Re: Asynchronous org-agenda-redo
> Org Agenda code does not wait for keyboard input; it's busy building the > agenda. This is the case with most code in Emacs: it's not written to > be asynchronous, and it doesn't return to the main thread until done. > So you can sprinkle yields here and there and maybe be able to move > point around while some code is running, but that will decrease > performance, as well as introducing another level of complexity and > another class of bugs (e.g. what if the user modifies a buffer while the > agenda code is scanning it?). Thanks for the explanation. > AFAIK there exists no way to do such a thing. Buffers are not designed > to be serialized/deserialized like that. You could try writing some > Elisp code to do it, but the end result would probably be much slower > than existing agenda code, as well as more difficult to debug. Yeah. Even re-initialisation of, for example, overlays in org buffer is likely to take too much time. > As you can see in org-agenda.el, it's complicated. Remember that an > Emacs process is like a Lisp image, full of state. The more symbols and > other structures you copy to the async Emacs process (by printing and > reading them as text, remember), the slower it's going to be--and it > will always be slower than not using async. > Asynchronous code is not faster; it's generally slower because of > yielding and synchronization. I see now that generating agenda in separate process will cause too much overheads. Anyway, I will try to throw yields into agenda code just to check how bad the performance can degrade. > org-ql doesn't use skip functions, just queries. Skip functions are essentially used-defined queries as soon as the queries are tested against every headline. I can rewrite my skip functions into queries, but I don't expect much improvement since org-ql seems to use org-entry-get, which is the main performance bottleneck for my agenda generation. Best, Ihor adam Porter writes: > Ihor Radchenko writes: > >>> Be sure to read the Emacs Lisp manual regarding threads. They are >>> cooperative, so functions called as threads must yield back to the main >>> thread for Emacs to do anything else before the function returns. >> >> I tried to read the manual, but I clearly misunderstand something. >> The manual says: >> >>> Currently, thread switching will occur upon explicit request via >>> ‘thread-yield’, when waiting for keyboard input... >> >> So, except directly calling thread-yield, it should be possible to >> trigger switching the current thread when keyboard input is expected. >> I tried the following demo code: >> >> (defun test () >> (let ((a 0)) >> (dotimes (_ 5) >> (setq a (1+ a)) >> (sleep-for 2) >> (message "%s" a >> >> (progn ;This should return to command loop quickly >> (make-thread #'test) >> (message "Executed...")); `eval-last-sexp' here >> >> I can move around the buffer while the progn is running. >> However, it is not the case with `org-agenda-redo' for a reason I do not >> fully understand. > > Org Agenda code does not wait for keyboard input; it's busy building the > agenda. This is the case with most code in Emacs: it's not written to > be asynchronous, and it doesn't return to the main thread until done. > So you can sprinkle yields here and there and maybe be able to move > point around while some code is running, but that will decrease > performance, as well as introducing another level of complexity and > another class of bugs (e.g. what if the user modifies a buffer while the > agenda code is scanning it?). > >>> 1. The process would have to load the same Org buffers, which takes >>> time, especially in large buffers. Depending on configuration, it >>> can take some time, indeed. >> >>> 3. Ensuring that configuration and state between the main Emacs process >>> and the separate, agenda-generating process is not necessarily >>> simple. Consider as well that if a buffer had unsaved changes, >>> those would not be readable by the other process, which would lead >>> to invalid results. One could force the buffers to be saved first, >>> but that may not always be desirable, as saving buffers can have >>> side effects. >> >> Why cannot org-buffer simply be copied into the subordinate process? If >> all be buffer-locals, text properties, and overlays are copied directly >> from the main emacs process, there may be no need to even initialise >> org-mode (the idea is to do something similar to clone-buffer). > > AFAIK there exists no way to do such a thing. Buffers are not designed > to be serialized/deserialized like that. You could try writing some > Elisp code to do it, but the end result would probably be much slower > than existing agenda code, as well as more difficult to debug. > >> The question though is whether buffer-locals + overlays + propertized >> .org files text + org-agenda-buffer copy can be sufficient to make the >> org-agenda-redo run properly. Are
