Re: Inconsistent handling of multi-line properties
Greg Sullivan writes: > I would really like the ability to use the ":myProperty+: " to create > multi-line properties for the org-export-taskjuggler exporter. > However, the mentioned patch is quite large and mostly focused on headline > caching, as far as I can tell. > @Ihor or @Hanno, Is there a patch just to get the :myProperty+: syntax to > work properly with org-element-property? That patch has been merged. However, the part about merging :myProperty+: into :myProperty: did not work as well as I thought initially. Now, all the instances of :myProperty+: do get merged and can be accessed via (org-element-property :MYPROPERTY+ element). However, they are not merged into :myProperty: because it would interfere with property inheritance. Merging everything into :MYPROPERTY is possible, but it might break third-party packages that do not expect element properties to be lists (currently, most export backends assume element properties to be strings). I am reluctant to work on this unless Nicolas displays a clear support of such change. An alternative could be modifying org-element-property or org-export-get-node-property to support multiline properties. Meanwhile, export backends might be changed to look into :PROPERTY+ in addition to :PROPERTY. For org-export-taskjuggler, you may implement this feature yourself. Note that org-export-taskjuggler is not a part of Org core. It is in org-contrib and currently not maintained. Best, Ihor
Re: How do you manage complex project with Org-mode
Tim Cross writes: > For example, I > would not have a task which says to review my tasks twice a week. Do you > really need a task to remind you to do this twice a week? Do you really > need to track that you have done this? I would classify such tasks as > 'noise' tasks. They really don't perform any real purpose. It is > similar to brain dead project policies which state things like "every > function must have a unit test". Many functions don't need a unit test > and forcing people to write pointless unit tests does more damage than > not having them. I disagree about this specific example (not with your whole idea). Reviewing tasks is important for people who can easily fall into continuous routine. Dedicating several hours per week to look into what you are doing in a broader perspective is valuable. Note that I do not mean "review" as looking through _all_ the tasks. Rather looking into current projects and deciding what to do, and, more importantly, what not to do in the coming week/days/months. See https://www.benkuhn.net/weekly/ Best, Ihor
Re: How do you manage complex project with Org-mode
Sébastien Gendre writes: > So, if you have any suggestion on how to manage, in Org-mode, projects > with: > * Lot of work to do (many days) > * Short deadline (not enough time) > * High importance (disastrous consequences in my future in case of fail) > * Many of them in the same time > * Progression need to be followed to chose where to sacrifice time to > limit the damages I think that your existing system is already a good starting point. I would not recommend changing it drastically. Every possible time management workflow must be based on existing workflow habits (like daily inbox review) and only introduce new habits one by one and slowly. For your specific situation with many simultaneous important projects, you are not alone. Every student meets similar issue. My recommendations for managing multiple projects: 1. Similar to daily inbox review, do weekly/bi-weekly project progress review. You are already doing this. However, I am not sure about the details. When doing project review, I find it useful to create a custom agenda listing all the ongoing projects. Every time I review the project list, I select up to 3 projects I plan to work on from now till the next project review. All other projects are marked "HOLD" and their tasks will never be listed in agenda (unless there is some hard deadline or unskippable meeting). Review is the time when you decide which projects to sacrifice if you have insufficient time. Coming back to those HOLD projects after the review time is a no-no, unless you complete the planned projects and still have remaining time. 2. If you cannot complete a project within initially planned time, it may be tempting to continue until completion. Do not do this. It is better to try finishing the other planned project work first and come back to the partially completed projects if time permits. In Org, there are several tools you can use to address this: - You may dedicate each single day to no more than a single project. The project tasks will be scheduled to specific days and you can create a custom agenda that does not show tasks scheduled in past days. - You may use effort estimates for projects/tasks shorter than a day + non-nil org-clock-sound. If you create a habit to clock-in regularly, Emacs will play a sound when you run out of time. - You may create a rule to have at least a single easy 2-3 min task as "entry" point to a project you plan to switch to. Having a simple task really makes it easier to switch from working on current and already familiar project to other one. *This rule sounds obvious and simple, but I cannot stress more how much it changed my workflow once I got to follow this regularly* The above suggestions are simple to list and somewhat obvious, but not-so-easy to master. It is important to stick to them as much as possible until they become a habit. They may take months to master. > To manage school big work, I think of managing them as projects. > > I want to apply a simple "Project" workflow: > > * Each project is a headline with the status "PROJECT" > * Each project have the deadline defined by the school work deadline > * Each project have a complete description with every info needed to work > * Each project have one or many tasks (as sub headlines with a status) > * Each task have a importance, time and effort estimation > * Each task have its own deadline, distributed along the remaining time > * When I set a task deadline, I look at its estimations and also other > projects tasks > * To create a new project, I use Org-capture with a template Just one comment. If you tend to have a large number of tasks in your agenda, you are likely overusing deadlines and scheduling. I would avoid setting deadlines for every single task. Too long agenda lists are *counterproductive* in many ways. I recommend scheduling the whole project (or subproject, it the main project is huge) + 1-2 individual tasks. When you complete the 1-2 individual tasks, it is more productive to look inside the project, and select next tasks depending on the new results from the first tasks. Hope my comments are useful. Best, Ihor
Re: [PATCH] Re: [BUG] org-cite: 10 second hang opening a ~4k org file with 10MB bibtex library
psychosis writes: > I have a related problem (see > https://list.orgmode.org/77km6r.3bgd0z3z4p...@disroot.org/ for > reference) and Ihor’s patch reduced the initial load time of my > biblatex bibliography file from almost ten minutes to about 10 to 20 > seconds. I have seen your email. Persisting the cache between sessions is a more tricky. It will require a dedicated effort - significantly more than the few-liner patch here. I assume that your bibliography is much larger than 10Mb. Am I correct? > For me the changed cite-key turns only red if I restart Emacs. This is bad. Would you mind constructing a reproducer? (using the patched Org version) > Thanks for the patch! Please let me know, if I can provide information > for further improvement. You definitely can. 1. You may share your bibtex file. It will make things much easier for me. I only have 1Mb example bibliography, which I used before I switched to storing references in Org files. 2. You may provide profiling information using M-x profiler-start M-x profiler-report and using ELP as I described in the previous messages. 3. We may arrange a live jitsi meeting and profile things together. Best, Ihor
Re: [PATCH] Re: [BUG] org-cite: 10 second hang opening a ~4k org file with 10MB bibtex library
Hello, I have a related problem (see https://list.orgmode.org/77km6r.3bgd0z3z4p...@disroot.org/ for reference) and Ihor’s patch reduced the initial load time of my biblatex bibliography file from almost ten minutes to about 10 to 20 seconds. For me the changed cite-key turns only red if I restart Emacs. Thanks for the patch! Please let me know, if I can provide information for further improvement. Regards Paul On Sa, Mär 19 2022 at 20:12:39 +0800, Ihor Radchenko wrote: Jamie Matthews writes: Hmm, this doesn't fix what I'm seeing - still hang on first load and quicker after killing and finding again. Very possible there's something else obvious I'm missing though, I'm very much an emacs 'end user'! This is strange. I would then try to run the profiler and look at the results. The very first run might be somewhat slow because org-cite still needs to parse the whole bibliography at the very beginning. But it should be still faster than 10 seconds you observed earlier. Best, Ihor
Re: [BUG] Resolving idle clocks needs multiple keystrokes [9.5.2 (release_9.5.2-24-g668205 @ /home/ignacio/repos/emacs/lisp/org/)]
Hmm, I see the problem. I didn't think about that. Thank you very much for your suggestion. But what about using read-char-exclusive? It seems to have a timeout argument too, and should apparently fix the issue (ie. the prompt will no longer disappear at the first unintentional click). Romeo Ignacio Casso writes: >> Hi, >> >> I don't know how closely it is related to your problem, but I've >> reported another issue revolving around the use of read-char for the >> prompt to resolve clocks. See >> [[https://lists.gnu.org/archive/html/emacs-orgmode/2022-02/msg00278.html]]. >> >> Unfortunately I an not advanced enough in Elisp to know whether using >> another function than read-char would solve your problem as well as >> mine (maybe read-char-choice waits for a valid char, while resetting >> the idle timer ?), but it might be a hint. What do you think ? > > The problem with `read-char-choice' is that it does not seem to have a > timeout argument. `read-char' has, and `org-clock-resolve' uses it to > update the prompt message with the current idle time every 45 seconds, > calling (read-char ... 45) in a loop until it returns non-nil. With > `read-char-choice' that would not be possible, and if > `org-clock-idle-time' was N, after the N idle minutes, the prompt would > appear saying something like "Emacs was idle for N minutes, what do you > want to do?", but M minutes later the message would still be the same > instead of replacing N with N+M. > > The patch I sent already fixes the bug I reported anyway. For yours, I > suggest just wrapping `read-char' in `condition-case', unless someone > knows of some other appropriate function.
Re: [PATCH] Re: [BUG] org-cite: 10 second hang opening a ~4k org file with 10MB bibtex library
Jamie Matthews writes: > Hmm, this doesn't fix what I'm seeing - still hang on first load and quicker > after killing and finding again. Very possible there's something else obvious > I'm missing though, I'm very much an emacs 'end user'! This is strange. I would then try to run the profiler and look at the results. The very first run might be somewhat slow because org-cite still needs to parse the whole bibliography at the very beginning. But it should be still faster than 10 seconds you observed earlier. Best, Ihor
Re: [PATCH] Re: [BUG] org-cite: 10 second hang opening a ~4k org file with 10MB bibtex library
> Great. Then, I am attaching the patch with the new version of the > function. I will let Nicolas decide if it is good enough to be merged. I > do not feel confident enough with org-cite code to judge if my approach > is not missing some edge cases. Great, thanks for your help! > If I am correct, putting (require 'oc-basic) before defun will fix what you are seeing. Hmm, this doesn't fix what I'm seeing - still hang on first load and quicker after killing and finding again. Very possible there's something else obvious I'm missing though, I'm very much an emacs 'end user'! From: Ihor Radchenko Sent: Saturday, March 19, 2022 11:47 AM To: Jamie Matthews ; Nicolas Goaziou Cc: emacs-orgmode@gnu.org Subject: [PATCH] Re: [BUG] org-cite: 10 second hang opening a ~4k org file with 10MB bibtex library Jamie Matthews writes: > I can confirm that the key turns red on insert when I altered the key outside > of emacs (with that second version of `org-cite-basic--parse-bibliography`). Great. Then, I am attaching the patch with the new version of the function. I will let Nicolas decide if it is good enough to be merged. I do not feel confident enough with org-cite code to judge if my approach is not missing some edge cases. > However, I'm now noticing that the hang improvement earlier (< a second down > from ~10) doesn't always occur. > > Specifically, if I > > 1. emacs -Q > 2. eval your code in scratch > 3. C-x C-f to the org file > > I get the hang. However, if I then > > > 1. kill the org buffer > 2. eval the code again > 3. re-find the org file > > the hang is gone. Without evaling your code in between, killing the org > buffer and finding it again in the same emacs session reproduces the hang > everytime, which was probably what I did the first time I report the > improvement, as in I didn't check it worked from startup. It is most likely because you defun the function after emacs -Q before org is loaded, then open Org file (opening Org file autoloads org-mode), and then org-mode overwrites the manual defun. If I am correct, putting (require 'oc-basic) before defun will fix what you are seeing. Best, Ihor
[PATCH] Re: [BUG] org-cite: 10 second hang opening a ~4k org file with 10MB bibtex library
Jamie Matthews writes: > I can confirm that the key turns red on insert when I altered the key outside > of emacs (with that second version of `org-cite-basic--parse-bibliography`). Great. Then, I am attaching the patch with the new version of the function. I will let Nicolas decide if it is good enough to be merged. I do not feel confident enough with org-cite code to judge if my approach is not missing some edge cases. > However, I'm now noticing that the hang improvement earlier (< a second down > from ~10) doesn't always occur. > > Specifically, if I > > 1. emacs -Q > 2. eval your code in scratch > 3. C-x C-f to the org file > > I get the hang. However, if I then > > > 1. kill the org buffer > 2. eval the code again > 3. re-find the org file > > the hang is gone. Without evaling your code in between, killing the org > buffer and finding it again in the same emacs session reproduces the hang > everytime, which was probably what I did the first time I report the > improvement, as in I didn't check it worked from startup. It is most likely because you defun the function after emacs -Q before org is loaded, then open Org file (opening Org file autoloads org-mode), and then org-mode overwrites the manual defun. If I am correct, putting (require 'oc-basic) before defun will fix what you are seeing. Best, Ihor >From 100b722708c68bc65af637c3ad4e289943cccd7c Mon Sep 17 00:00:00 2001 Message-Id: <100b722708c68bc65af637c3ad4e289943cccd7c.1647690044.git.yanta...@gmail.com> From: Ihor Radchenko Date: Sat, 19 Mar 2022 19:24:55 +0800 Subject: [PATCH] oc-basic: Speed up cached bibliography retrival * lisp/oc-basic.el (org-cite-basic--file-id-cache): New variable storing hash of bibliography file contents. (org-cite-basic--parse-bibliography): Skip buffer hash calculation when bibliography file is unchanged on disk. Calculating buffer hash on every call is slow when bibliography file is large. * lisp/org-compat.el: (org-file-has-changed-p--hash-table, org-file-has-changed-p): Define `file-has-changed-p' from Emacs 29 if necessary. See https://list.orgmode.org/lo2p265mb1758e12e04832dc712f5a8e9dc...@lo2p265mb1758.gbrp265.prod.outlook.com/T/#t --- lisp/oc-basic.el | 11 +-- lisp/org-compat.el | 29 + 2 files changed, 38 insertions(+), 2 deletions(-) diff --git a/lisp/oc-basic.el b/lisp/oc-basic.el index e2dfc0603..d1ea620ad 100644 --- a/lisp/oc-basic.el +++ b/lisp/oc-basic.el @@ -233,6 +233,8 @@ (defun org-cite-basic--parse-bibtex (dialect) entries))) entries)) +(defvar org-cite-basic--file-id-cache nil + "Hash table linking files to their hash.") (defun org-cite-basic--parse-bibliography ( info) "List all entries available in the buffer. @@ -245,14 +247,19 @@ (defun org-cite-basic--parse-bibliography ( info) as symbols, and values as strings or nil. Optional argument INFO is the export state, as a property list." + (unless (hash-table-p org-cite-basic--file-id-cache) +(setq org-cite-basic--file-id-cache (make-hash-table :test #'equal))) (if (plist-member info :cite-basic/bibliography) (plist-get info :cite-basic/bibliography) (let ((results nil)) (dolist (file (org-cite-list-bibliography-files)) (when (file-readable-p file) (with-temp-buffer -(insert-file-contents file) - (let* ((file-id (cons file (org-buffer-hash))) +(when (or (org-file-has-changed-p file) + (not (gethash file org-cite-basic--file-id-cache))) + (insert-file-contents file) + (puthash file (org-buffer-hash) org-cite-basic--file-id-cache)) + (let* ((file-id (cons file (gethash file org-cite-basic--file-id-cache))) (entries (or (cdr (assoc file-id org-cite-basic--bibliography-cache)) (let ((table diff --git a/lisp/org-compat.el b/lisp/org-compat.el index 38d330de6..e96dcba8b 100644 --- a/lisp/org-compat.el +++ b/lisp/org-compat.el @@ -73,6 +73,35 @@ (defvar org-table-dataline-regexp) (defvar org-table-tab-recognizes-table.el) (defvar org-table1-hline-regexp) + +;;; Emacs < 29 compatibility + +(defvar org-file-has-changed-p--hash-table (make-hash-table :test #'equal) + "Internal variable used by `org-file-has-changed-p'.") + +(if (fboundp 'file-has-changed-p) +(defalias org-file-has-changed-p #'file-has-changed-p) + (defun org-file-has-changed-p (file tag) +"Return non-nil if FILE has changed. +The size and modification time of FILE are compared to the size +and modification time of the same FILE during a previous +invocation of `org-file-has-changed-p'. Thus, the first invocation +of `org-file-has-changed-p' always returns non-nil when FILE exists. +The optional argument TAG, which must be a symbol, can be used to +limit the comparison to invocations with identical tags; it can be +the symbol of the calling function, for example." +(let*
Re: [BUG] Resolving idle clocks needs multiple keystrokes [9.5.2 (release_9.5.2-24-g668205 @ /home/ignacio/repos/emacs/lisp/org/)]
> Hi, > > I don't know how closely it is related to your problem, but I've > reported another issue revolving around the use of read-char for the > prompt to resolve clocks. See > [[https://lists.gnu.org/archive/html/emacs-orgmode/2022-02/msg00278.html]]. > > Unfortunately I an not advanced enough in Elisp to know whether using > another function than read-char would solve your problem as well as > mine (maybe read-char-choice waits for a valid char, while resetting > the idle timer ?), but it might be a hint. What do you think ? The problem with `read-char-choice' is that it does not seem to have a timeout argument. `read-char' has, and `org-clock-resolve' uses it to update the prompt message with the current idle time every 45 seconds, calling (read-char ... 45) in a loop until it returns non-nil. With `read-char-choice' that would not be possible, and if `org-clock-idle-time' was N, after the N idle minutes, the prompt would appear saying something like "Emacs was idle for N minutes, what do you want to do?", but M minutes later the message would still be the same instead of replacing N with N+M. The patch I sent already fixes the bug I reported anyway. For yours, I suggest just wrapping `read-char' in `condition-case', unless someone knows of some other appropriate function.
Re: Move or rename a file in a link
Hi João, João Pedro de Amorim Paula writes: > I mean org attachments. I use org-attach extensively to store documents > with notes. So I'd have a heading like so > > * Documents > :PROPERTIES: > :DIR: data/docs/ > :END: > > - [[Registration][attachment:registration.pdf]] :: My registration. > > And say I'd like to rename the file. I would need to rename it inside > data/docs and I would also need to edit the link, so I would like to > have a way to do it automatically. I just got a computer back so I will > be trying to adapt what you did to work with org-attach, but if you > figure out a way to do it as well, please, let me know. I see. This is a new version that would also work with org attachment links. I haven't tested it much and the function is a bit tricky, but I think it works fine: #+begin_src emacs-lisp (defun my-org-rename-link-file-at-point () (interactive) (let* ((curr-dir (if (equal (org-element-property :type (org-element-context)) "attachment") (concat (abbreviate-file-name (org-attach-dir)) "/") (abbreviate-file-name default-directory))) (current-path (if (equal (org-element-property :type (org-element-context)) "attachment") (concat curr-dir (org-element-property :path (org-element-context))) (org-element-property :path (org-element-context (new-path (read-file-name "Rename file at point to: " current-path))) (rename-file current-path new-path) (message (concat "moved to: " new-path)) (if (directory-name-p new-path) (setq new-path (concat new-path (file-name-nondirectory current-path))) (setq new-path new-path)) (if (equal (org-element-property :type (org-element-context)) "attachment") (my-org-replace-link-file (file-name-nondirectory current-path) (replace-regexp-in-string curr-dir "" new-path)) (my-org-replace-link-file current-path (replace-regexp-in-string curr-dir "" new-path) (defun my-org-replace-link-file (from to) (save-excursion (goto-char (point-min)) (while (re-search-forward org-bracket-link-regexp nil t) (cond ((string-match-p (concat "attachment:" from) (match-string 1)) (replace-match (concat "[[attachment:" to "]]"))) ((string-match-p from (match-string 1)) (replace-match (concat "[[file:" to "]]"))) #+end_src Best regards, Juan Manuel
Re: [BUG] org-cite: 10 second hang opening a ~4k org file with 10MB bibtex library [9.5.2 (9.5.2-g91681f @ /home/jdm204/.config/emacs/straight/build/org/)]
I can confirm that the key turns red on insert when I altered the key outside
of emacs (with that second version of `org-cite-basic--parse-bibliography`).
However, I'm now noticing that the hang improvement earlier (< a second down
from ~10) doesn't always occur.
Specifically, if I
1. emacs -Q
2. eval your code in scratch
3. C-x C-f to the org file
I get the hang. However, if I then
1. kill the org buffer
2. eval the code again
3. re-find the org file
the hang is gone. Without evaling your code in between, killing the org buffer
and finding it again in the same emacs session reproduces the hang everytime,
which was probably what I did the first time I report the improvement, as in I
didn't check it worked from startup.
From: Ihor Radchenko
Sent: Saturday, March 19, 2022 10:28 AM
To: Jamie Matthews
Cc: emacs-orgmode@gnu.org
Subject: Re: [BUG] org-cite: 10 second hang opening a ~4k org file with 10MB
bibtex library [9.5.2 (9.5.2-g91681f @
/home/jdm204/.config/emacs/straight/build/org/)]
Jamie Matthews writes:
> It seems a very significant improvement - the lag on scrolling after the
> buffer is loaded is not noticeable, and the initial loading hang essentially
> disappears!
Great! Do you mean that there is no apparent slowness at all?
> I'm slightly surprised that elp-results doesn't seem to show as dramatic an
> improvement as it feels for me:
>
> ```
> org-cite-basic-activate 123 6.5652978389 0.0533764051
> org-cite-basic--all-keys123 6.2679872230 0.0509592457
> org-cite-basic--parse-bibliography 270 6.2513217860 0.0231530436
> org-cite-basic--get-entry 147 0.1550045990 0.0010544530
> ```
It does. You need to look at the last column. The total time in the
second column is large probably because you interacted (scrolled?) with
the buffer a bit more than first time and the functions got called more
(see second column).
Also, can you try the following version of
org-cite-basic--parse-bibliography again and try to change cite key of
one of the cited entries of the bibliography outside emacs? The key
should be marked red if you try to insert the changed entry in Org.
(defvar org-cite-basic--file-id-cache nil
"Hash table linking files to their hash.")
(defun org-cite-basic--parse-bibliography ( info)
"List all entries available in the buffer.
Each association follows the pattern
(FILE . ENTRIES)
where FILE is the absolute file name of the BibTeX file, and ENTRIES is a hash
table where keys are references and values are association lists between fields,
as symbols, and values as strings or nil.
Optional argument INFO is the export state, as a property list."
(unless (hash-table-p org-cite-basic--file-id-cache)
(setq org-cite-basic--file-id-cache (make-hash-table :test #'equal)))
(if (plist-member info :cite-basic/bibliography)
(plist-get info :cite-basic/bibliography)
(let ((results nil))
(dolist (file (org-cite-list-bibliography-files))
(when (file-readable-p file)
(with-temp-buffer
(when (or (file-has-changed-p file)
(not (gethash file org-cite-basic--file-id-cache)))
(insert-file-contents file)
(puthash file (org-buffer-hash) org-cite-basic--file-id-cache))
(let* ((file-id (cons file (gethash file
org-cite-basic--file-id-cache)))
(entries
(or (cdr (assoc file-id org-cite-basic--bibliography-cache))
(let ((table
(pcase (file-name-extension file)
("json" (org-cite-basic--parse-json))
("bib" (org-cite-basic--parse-bibtex
'biblatex))
("bibtex" (org-cite-basic--parse-bibtex
'BibTeX))
(ext
(user-error "Unknown bibliography extension:
%S"
ext)
(push (cons file-id table)
org-cite-basic--bibliography-cache)
table
(push (cons file entries) results)
(when info (plist-put info :cite-basic/bibliography results))
results)))
Best,
Ihor
Re: Org and multimedia (tips?)
Max Nikulin writes: > org-web-tools is an interesting project, but if you have access to > files it should be easier to extract all meta information directly > using e.g. > > exiftool -json file.mp3 > > or another tool suitable to particular format. It seems emms has > interface to various tools. > > P.S. You may try to adapt common LISP implementation of ID3 parser > https://gigamonkeys.com/book/practical-an-id3-parser.html Maxim, thanks a lot for the ideas. I take note. The reason for using org-web-tools is that the web page that GNUMP3d serves is extremely simple, with an alphabetical list of artists, titles, and links to the .m3u to stream. The list is easily reusable in Org. In the end I managed to write a function to create in each node a property drawer with album title, date and artist, accessing each m3u and obtaining the information with ffprobe (screenshot: https://i.imgur.com/1ALe4Ah.png). Best regards, Juan Manuel
Re: [BUG] Resolving idle clocks needs multiple keystrokes [9.5.2 (release_9.5.2-24-g668205 @ /home/ignacio/repos/emacs/lisp/org/)]
Hi, I don't know how closely it is related to your problem, but I've reported another issue revolving around the use of read-char for the prompt to resolve clocks. See [[https://lists.gnu.org/archive/html/emacs-orgmode/2022-02/msg00278.html]]. Unfortunately I an not advanced enough in Elisp to know whether using another function than read-char would solve your problem as well as mine (maybe read-char-choice waits for a valid char, while resetting the idle timer ?), but it might be a hint. What do you think ? Regards, Romeo Ignacio Casso writes: >> So the problem is in (org-user-idle-seconds), which in my window system >> boils down to a call to (current-idle-time). It should return 0 after >> answering the prompt, but in my system it keeps counting up. At this >> point I stopped investigating since that function is defined in C. >> > > I have investigated this a little bit further and it seems that reading > a character with read-char only resets the idle timer if the SECONDS > argument is nil (which is not the case here) . I reported it as a bug to > bug-gnu-em...@gnu.org, but it seems there is a reason for that. See > https://debbugs.gnu.org/cgi/bugreport.cgi?bug=54371 > >> Since this bug is already old and I have not found much information >> about it, I assume that in most systems answering the prompt does reset >> the idle timer and this bug does not occur. > > It also seems to happen for all systems, so either much less people than > I thought use this feature, or there is something else that makes this > bug particular to my setup, although I can't think of anything else. > >> As a quick fix for those >> systems where this is an issue, we could reset the idle timer ourselves >> after the prompt in org-resolve-clocks-if-idle. > > I have thought of a better way to fix this, and currently there is no > way to do reset the idle timer with Elisp anyway. I attach the patch: it > just cancels the timer before prompting the user and sets it again if > needed after the prompt is answered. What do you think? > > [2. Patch for org-resolve-clocks-if-idle bug --- text/x-diff; > 0001-fixed-bug.patch] > From 288b25ea95699596762b199088ce6828a5e9a0ed Mon Sep 17 00:00:00 2001 > From: Ignacio > Date: Sun, 13 Mar 2022 21:41:43 +0100 > Subject: [PATCH] fixed bug > > --- > lisp/org/org-clock.el | 9 +++-- > 1 file changed, 7 insertions(+), 2 deletions(-) > > diff --git a/lisp/org/org-clock.el b/lisp/org/org-clock.el > index dce5d9d4c0..81d4a85782 100644 > --- a/lisp/org/org-clock.el > +++ b/lisp/org/org-clock.el > @@ -1228,7 +1228,9 @@ org-resolve-clocks-if-idle > (org-clock-user-idle-start > (org-time-since org-clock-user-idle-seconds)) > (org-clock-resolving-clocks-due-to-idleness t)) > - (if (> org-clock-user-idle-seconds (* 60 org-clock-idle-time)) > + (when (> org-clock-user-idle-seconds (* 60 org-clock-idle-time)) > + (cancel-timer org-clock-idle-timer) > + (setq org-clock-idle-timer nil) > (org-clock-resolve > (cons org-clock-marker >org-clock-start-time) > @@ -1237,7 +1239,10 @@ org-resolve-clocks-if-idle >(/ (float-time >(time-since org-clock-user-idle-start)) > 60))) > -org-clock-user-idle-start) > +org-clock-user-idle-start) > + (when (and (org-clocking-p) (not org-clock-idle-timer)) > +(setq org-clock-idle-timer > + (run-with-timer 60 60 #'org-resolve-clocks-if-idle))) > > (defvar org-clock-current-task nil "Task currently clocked in.") > (defvar org-clock-out-time nil) ; store the time of the last clock-out
Re: [BUG] org-cite: 10 second hang opening a ~4k org file with 10MB bibtex library [9.5.2 (9.5.2-g91681f @ /home/jdm204/.config/emacs/straight/build/org/)]
Jamie Matthews writes:
> It seems a very significant improvement - the lag on scrolling after the
> buffer is loaded is not noticeable, and the initial loading hang essentially
> disappears!
Great! Do you mean that there is no apparent slowness at all?
> I'm slightly surprised that elp-results doesn't seem to show as dramatic an
> improvement as it feels for me:
>
> ```
> org-cite-basic-activate 123 6.5652978389 0.0533764051
> org-cite-basic--all-keys123 6.2679872230 0.0509592457
> org-cite-basic--parse-bibliography 270 6.2513217860 0.0231530436
> org-cite-basic--get-entry 147 0.1550045990 0.0010544530
> ```
It does. You need to look at the last column. The total time in the
second column is large probably because you interacted (scrolled?) with
the buffer a bit more than first time and the functions got called more
(see second column).
Also, can you try the following version of
org-cite-basic--parse-bibliography again and try to change cite key of
one of the cited entries of the bibliography outside emacs? The key
should be marked red if you try to insert the changed entry in Org.
(defvar org-cite-basic--file-id-cache nil
"Hash table linking files to their hash.")
(defun org-cite-basic--parse-bibliography ( info)
"List all entries available in the buffer.
Each association follows the pattern
(FILE . ENTRIES)
where FILE is the absolute file name of the BibTeX file, and ENTRIES is a hash
table where keys are references and values are association lists between fields,
as symbols, and values as strings or nil.
Optional argument INFO is the export state, as a property list."
(unless (hash-table-p org-cite-basic--file-id-cache)
(setq org-cite-basic--file-id-cache (make-hash-table :test #'equal)))
(if (plist-member info :cite-basic/bibliography)
(plist-get info :cite-basic/bibliography)
(let ((results nil))
(dolist (file (org-cite-list-bibliography-files))
(when (file-readable-p file)
(with-temp-buffer
(when (or (file-has-changed-p file)
(not (gethash file org-cite-basic--file-id-cache)))
(insert-file-contents file)
(puthash file (org-buffer-hash) org-cite-basic--file-id-cache))
(let* ((file-id (cons file (gethash file
org-cite-basic--file-id-cache)))
(entries
(or (cdr (assoc file-id org-cite-basic--bibliography-cache))
(let ((table
(pcase (file-name-extension file)
("json" (org-cite-basic--parse-json))
("bib" (org-cite-basic--parse-bibtex
'biblatex))
("bibtex" (org-cite-basic--parse-bibtex
'BibTeX))
(ext
(user-error "Unknown bibliography extension:
%S"
ext)
(push (cons file-id table)
org-cite-basic--bibliography-cache)
table
(push (cons file entries) results)
(when info (plist-put info :cite-basic/bibliography results))
results)))
Best,
Ihor
Re: [BUG] org-cite: 10 second hang opening a ~4k org file with 10MB bibtex library [9.5.2 (9.5.2-g91681f @ /home/jdm204/.config/emacs/straight/build/org/)]
It seems a very significant improvement - the lag on scrolling after the buffer
is loaded is not noticeable, and the initial loading hang essentially
disappears!
I'm slightly surprised that elp-results doesn't seem to show as dramatic an
improvement as it feels for me:
```
org-cite-basic-activate 123 6.5652978389 0.0533764051
org-cite-basic--all-keys123 6.2679872230 0.0509592457
org-cite-basic--parse-bibliography 270 6.2513217860 0.0231530436
org-cite-basic--get-entry 147 0.1550045990 0.0010544530
```
From: Ihor Radchenko
Sent: Saturday, March 19, 2022 9:57 AM
To: Jamie Matthews
Cc: emacs-orgmode@gnu.org
Subject: Re: [BUG] org-cite: 10 second hang opening a ~4k org file with 10MB
bibtex library [9.5.2 (9.5.2-g91681f @
/home/jdm204/.config/emacs/straight/build/org/)]
Jamie Matthews writes:
> Thanks:
>
> ```
> org-cite-basic-activate 59 10.724349447 0.1817686346
> org-cite-basic--parse-bibliography 129 10.559936049 0.0818599693
> org-cite-basic--all-keys59 7.830202561 0.1327152976
> org-cite-basic--get-entry 70 2.7772344940 0.0396747784
> ```
org-cite-basic--parse-bibliography appears to be the main bottleneck.
I tried to write a quick fix (untested).
Can you try to redefine org-cite-basic--parse-bibliography to the
version below (note an extra defvar) and let me know how it goes:
(defvar org-cite-basic--file-id-cache nil
"Hash table linking files to their hash.")
(defun org-cite-basic--parse-bibliography ( info)
"List all entries available in the buffer.
Each association follows the pattern
(FILE . ENTRIES)
where FILE is the absolute file name of the BibTeX file, and ENTRIES is a hash
table where keys are references and values are association lists between fields,
as symbols, and values as strings or nil.
Optional argument INFO is the export state, as a property list."
(unless (hash-table-p org-cite-basic--file-id-cache)
(setq org-cite-basic--file-id-cache (make-hash-table :test #'equal)))
(if (plist-member info :cite-basic/bibliography)
(plist-get info :cite-basic/bibliography)
(let ((results nil))
(dolist (file (org-cite-list-bibliography-files))
(when (file-readable-p file)
(with-temp-buffer
(when (or (file-has-changed-p file)
(not (gethash file org-cite-basic--file-id-cache)))
(insert-file-contents file))
(unless (gethash file org-cite-basic--file-id-cache)
(puthash file (org-buffer-hash) org-cite-basic--file-id-cache))
(let* ((file-id (cons file (gethash file
org-cite-basic--file-id-cache)))
(entries
(or (cdr (assoc file-id org-cite-basic--bibliography-cache))
(let ((table
(pcase (file-name-extension file)
("json" (org-cite-basic--parse-json))
("bib" (org-cite-basic--parse-bibtex
'biblatex))
("bibtex" (org-cite-basic--parse-bibtex
'BibTeX))
(ext
(user-error "Unknown bibliography extension:
%S"
ext)
(push (cons file-id table)
org-cite-basic--bibliography-cache)
table
(push (cons file entries) results)
(when info (plist-put info :cite-basic/bibliography results))
results)))
Best,
Ihor
Re: [BUG] org-cite: 10 second hang opening a ~4k org file with 10MB bibtex library [9.5.2 (9.5.2-g91681f @ /home/jdm204/.config/emacs/straight/build/org/)]
Jamie Matthews writes:
> Thanks:
>
> ```
> org-cite-basic-activate 59 10.724349447 0.1817686346
> org-cite-basic--parse-bibliography 129 10.559936049 0.0818599693
> org-cite-basic--all-keys59 7.830202561 0.1327152976
> org-cite-basic--get-entry 70 2.7772344940 0.0396747784
> ```
org-cite-basic--parse-bibliography appears to be the main bottleneck.
I tried to write a quick fix (untested).
Can you try to redefine org-cite-basic--parse-bibliography to the
version below (note an extra defvar) and let me know how it goes:
(defvar org-cite-basic--file-id-cache nil
"Hash table linking files to their hash.")
(defun org-cite-basic--parse-bibliography ( info)
"List all entries available in the buffer.
Each association follows the pattern
(FILE . ENTRIES)
where FILE is the absolute file name of the BibTeX file, and ENTRIES is a hash
table where keys are references and values are association lists between fields,
as symbols, and values as strings or nil.
Optional argument INFO is the export state, as a property list."
(unless (hash-table-p org-cite-basic--file-id-cache)
(setq org-cite-basic--file-id-cache (make-hash-table :test #'equal)))
(if (plist-member info :cite-basic/bibliography)
(plist-get info :cite-basic/bibliography)
(let ((results nil))
(dolist (file (org-cite-list-bibliography-files))
(when (file-readable-p file)
(with-temp-buffer
(when (or (file-has-changed-p file)
(not (gethash file org-cite-basic--file-id-cache)))
(insert-file-contents file))
(unless (gethash file org-cite-basic--file-id-cache)
(puthash file (org-buffer-hash) org-cite-basic--file-id-cache))
(let* ((file-id (cons file (gethash file
org-cite-basic--file-id-cache)))
(entries
(or (cdr (assoc file-id org-cite-basic--bibliography-cache))
(let ((table
(pcase (file-name-extension file)
("json" (org-cite-basic--parse-json))
("bib" (org-cite-basic--parse-bibtex
'biblatex))
("bibtex" (org-cite-basic--parse-bibtex
'BibTeX))
(ext
(user-error "Unknown bibliography extension:
%S"
ext)
(push (cons file-id table)
org-cite-basic--bibliography-cache)
table
(push (cons file entries) results)
(when info (plist-put info :cite-basic/bibliography results))
results)))
Best,
Ihor
Re: [BUG] org-cite: 10 second hang opening a ~4k org file with 10MB bibtex library [9.5.2 (9.5.2-g91681f @ /home/jdm204/.config/emacs/straight/build/org/)]
Thanks: ``` org-cite-basic-activate 59 10.724349447 0.1817686346 org-cite-basic--parse-bibliography 129 10.559936049 0.0818599693 org-cite-basic--all-keys59 7.830202561 0.1327152976 org-cite-basic--get-entry 70 2.7772344940 0.0396747784 ``` From: Ihor Radchenko Sent: Saturday, March 19, 2022 9:23 AM To: Jamie Matthews Cc: emacs-orgmode@gnu.org Subject: Re: [BUG] org-cite: 10 second hang opening a ~4k org file with 10MB bibtex library [9.5.2 (9.5.2-g91681f @ /home/jdm204/.config/emacs/straight/build/org/)] Jamie Matthews writes: > Hi Ihor, > > I tried to run that code but get errors for the elp-instrument-function > calls (both with my config and emacs -Q): > > ``` > Debugger entered--Lisp error: (error "ELP cannot profile the function: > org-cite-basic--g...") Then also add (require 'oc-basic) before the code I provided. I seems that org-cite has not been loaded yet when you tried to execute the code. Best, Ihor
Re: [BUG] org-cite: 10 second hang opening a ~4k org file with 10MB bibtex library [9.5.2 (9.5.2-g91681f @ /home/jdm204/.config/emacs/straight/build/org/)]
Jamie Matthews writes: > Hi Ihor, > > I tried to run that code but get errors for the elp-instrument-function > calls (both with my config and emacs -Q): > > ``` > Debugger entered--Lisp error: (error "ELP cannot profile the function: > org-cite-basic--g...") Then also add (require 'oc-basic) before the code I provided. I seems that org-cite has not been loaded yet when you tried to execute the code. Best, Ihor
Re: [BUG] org-cite: 10 second hang opening a ~4k org file with 10MB bibtex library [9.5.2 (9.5.2-g91681f @ /home/jdm204/.config/emacs/straight/build/org/)]
Hi Ihor,
I tried to run that code but get errors for the elp-instrument-function calls
(both with my config and emacs -Q):
```
Debugger entered--Lisp error: (error "ELP cannot profile the function:
org-cite-basic--g...")
error("ELP cannot profile the function: %s" org-cite-basic--get-entry)
elp-instrument-function(org-cite-basic--get-entry)
(progn (elp-instrument-function #'org-cite-basic--get-entry))
elisp--eval-last-sexp(nil)
eval-last-sexp(nil)
funcall-interactively(eval-last-sexp nil)
command-execute(eval-last-sexp)
```
Best,
Jamie
From: Ihor Radchenko
Sent: Saturday, March 19, 2022 8:28 AM
To: Jamie Matthews
Cc: emacs-orgmode@gnu.org
Subject: Re: [BUG] org-cite: 10 second hang opening a ~4k org file with 10MB
bibtex library [9.5.2 (9.5.2-g91681f @
/home/jdm204/.config/emacs/straight/build/org/)]
Jamie Matthews writes:
> # Issue
> Starting emacs with emacs -Q, then navigating to a minimal example org file
> with C-x C-f yields a ~10 second hang on an 8-core/16GB RAM machine with
> nothing else running. Also, scrolling commands like C-v are often laggy
> after the initial hang.
>
> The minimal org file begins:
> """
> #+bibliography: ~/cloud/library/lib.bib
>
> [cite:@tillyPrimaryAnaplasticLargeCell1997]
> """
> with another 80 citations afterwards, one-by-line, but nothing else. As
> mentioned in the title, the lib.bib file is ~10MB - if I swap this out for a
> non-existent or tiny bibtex file the problem goes away, and the in-buffer
> citations are rendered in a red face. Clearly from this and the below profile
> the issue is something to do with checking etc the citations for
> fontification purposes.
>
> # profile
>
> ## cpu
> 1,542,542,267 99% - org-cite-basic-activate
> 787,037,416 50%- org-cite-basic--get-entry
> 4,065,194 0% + org-cite-basic--parse-bibliography
> 754,769,872 48%- org-cite-basic--all-keys
Could you try the following:
1. Execute the following code:
(require 'elp)
(elp-restore-all)
(elp-instrument-function #'org-cite-basic--get-entry)
(elp-instrument-function #'org-cite-basic-activate)
(elp-instrument-function #'org-cite-basic--parse-bibliography)
(elp-instrument-function #'org-cite-basic--all-keys)
2. Open your org file where Emacs hangs
3. Run M-x elp-results
4. Report the ELP buffer contents here
Best,
Ihor
Re: profiling latency in large org-mode buffers (under both main & org-fold feature)
Max Nikulin writes: > It is up to you to choose at which level your prefer to optimize the > code. And it is only my opinion (I do not insist) that benefits from > changes in low level code might be much more significant. I like the > idea of markers, but their current implementation is a source of pain. > >> (note that Nicolas did not use >> markers to store boundaries of org elements). > > E.g. export-related code certainly does need markers. You experienced > enough problems with attempts to properly invalidate cache when lower > level is not supposed to provide appropriate facilities. I understand your argument. However, I feel discouraged to contribute to Emacs devel because, most of Org users will not benefit from such contribution for a long time. Not until next several major versions of Emacs will be released. So, I currently prefer to contribute some backwards-compatible high-level code and leave Emacs core for future. Best, Ihor
Re: [BUG] org-cite: 10 second hang opening a ~4k org file with 10MB bibtex library [9.5.2 (9.5.2-g91681f @ /home/jdm204/.config/emacs/straight/build/org/)]
Jamie Matthews writes: > # Issue > Starting emacs with emacs -Q, then navigating to a minimal example org file > with C-x C-f yields a ~10 second hang on an 8-core/16GB RAM machine with > nothing else running. Also, scrolling commands like C-v are often laggy > after the initial hang. > > The minimal org file begins: > """ > #+bibliography: ~/cloud/library/lib.bib > > [cite:@tillyPrimaryAnaplasticLargeCell1997] > """ > with another 80 citations afterwards, one-by-line, but nothing else. As > mentioned in the title, the lib.bib file is ~10MB - if I swap this out for a > non-existent or tiny bibtex file the problem goes away, and the in-buffer > citations are rendered in a red face. Clearly from this and the below profile > the issue is something to do with checking etc the citations for > fontification purposes. > > # profile > > ## cpu > 1,542,542,267 99% - org-cite-basic-activate > 787,037,416 50%- org-cite-basic--get-entry > 4,065,194 0% + org-cite-basic--parse-bibliography > 754,769,872 48%- org-cite-basic--all-keys Could you try the following: 1. Execute the following code: (require 'elp) (elp-restore-all) (elp-instrument-function #'org-cite-basic--get-entry) (elp-instrument-function #'org-cite-basic-activate) (elp-instrument-function #'org-cite-basic--parse-bibliography) (elp-instrument-function #'org-cite-basic--all-keys) 2. Open your org file where Emacs hangs 3. Run M-x elp-results 4. Report the ELP buffer contents here Best, Ihor
Re: [BUG] Error when using mouse to set TODO state [9.5.2 (9.5.2-g91681f @ /home/garyo/.emacs.d/straight/build/org/)]
Gary Oberbrunner writes: > I'm using org and org-roam. I tried to use the mouse-3 menu to toggle the > state of a TODO item, but I get this error: > > > Debugger entered--Lisp error: (void-variable org-todo) Thanks for reporting! Fixed now on bugfix via af6f1298b. Best, Ihor
