Re: archiving speed [was Re: Tips on maintaining history in Org Mode]
Samuel Wales writes: > i should clarify. bulk archiving slows down even with /nonexistent/ > (have not tried empty) archives. as part of normal and expected > operation, bulk creates the archive for the first entry, and then > subsequent entries are added. those get slower and slower. Hi. I just pushed a change that should solve the archiving speed. Feel free to try it out. Best, Ihor
Re: archiving speed [was Re: Tips on maintaining history in Org Mode]
It is for me. However, this all depends on how you configure things. In my case, I have a standard structure for my org files. I have a different org file for each 'topic' and each org file has headings for * Tasks * Notes * Resources * Comms * General and each of those headings has a property list with an ARCHIVE property which looks like %s_archive:: i.e. %s_archive::* Tasks, so when I archive a tree/subtree is is placed under the heading according to the ARCHIVE property for the tree it comes from. almost all data I enter into org files comes via a capture template. All captured data initially goes into a refile.org file and then I refile to the appropriate topic org file each morning. For completed tasks, I will usually marked them with an ARCHIVE tag soon after they have been completed and then every 6 months or so, I will archive into an archive file (where heading hierarchies are retained). This is most common with completed tasks. I have a custom agenda which shows tasks broken up into "Completed", "In Progress", "Next" and "Backlog". I will archive the DONE items so that the "Completed" list in the agenda does ot grow too large. Every 12 months I move the archive files to an archive folder where they are renamed to include the year in the filename. I don't find archiving terribly slow. this is mainly because few of my org files are particularly large (because they are broken up into topics) and because I move older archives out into an archive directory after 12 months. It is very rare that I need to go digging around in archive files (either current or older year archives). Of course this could all change down the track. My org files are slowly getting larger and I expect at some point, I will hit a tipping point where things become slow. So far, even with the larger files (around 5Mb) performance is fine. I also don't put 'everything' in the org file. If I have another file of data, I will just link to that from the org file rather than have that data actually reside in the org file. SO for example, my org-mode.org file has a lot of links to interesting messages from the org list, but the messages themselves are all in my mu4e maildir folders. My main point was that because configuration like mine exist, simply appending archived items to the archive file simply would not work. I like having my archive records in a similar 'shape' to my normal org files because when I do need to dig into the archive, I don't want to have to go through the whole file looking for something. I generally know if I'm looking for an old task, note, general entry or comms record and it is handly to know I only have to look in that section of the file. This is one of the big challenges for org mode. Because it is so flexible and people take advantage of that flexibility, what may appear like a simple way to solve an issue often ends up being far more complex than it initially seemed. If, for exmaple, you could not archive based on heading, date, etc, just appending entries would probably work fine. However, as the archviing policy might be more complex, org needs to examine/parse the archive file to work out where to insert the archived entry. Tim Samuel Wales writes: > what is the current status of hierarchy in archive files? surely they > don't deal with updating categories and updating hierarchy structure > [sounds brittle and syncy]? i'm thinking it isn't hierarchical at > present, except when you have a doneified task with children? > > > On 8/11/21, Tim Cross wrote: >> I think the problem with just using append to file is that it won't >> preserve the shape of the file. For example, if I had a file with >> >> * Notes >> ** Note 1 >>blah blah >> ** Note 2 blah blah >> >> * Tasks >> ** DONE task 1 >> ** TODO Task 2 >> >> and I decide to archive note 1 and task 1, I would like them to both appear >> under the same headings and with the same level. If the process just uses >> append to file, I can have this for the first archiving i.e. >> >> * Noes >> ** Note 1 >> >> * Tasks >> ** DONE task 1 >> >> but then later, I decide to archive note 2, if append file is used, I will >> end up with >> >> * Notes >> ** Note 1 >> >> * Taks >> ** DONE task 1 >> >> * Notes >> ** Note 2 >> >> which is not what I want. I want >> >> * Notes >> ** Note 1 >> ** Note 2 >> >> * Tasks >> ** DONE Task 1 >> >> So, if we want to preserve hierarchies in our archive files and not have >> everything jumbled up together, the system has to parse the file. If you >> are also using something like Categories, then even more work needs to be >> odne to update the category lists. >> >> What I tend to do is mark items with the ARCHIVE tag and leave them in the >> file and then every few months, move archived data to archive files. It >> can still get slow, but I don't do it often, so it isn't too much of a >> hassle. >> >> >> On Thu, 12 Aug 2021 at 08:23, Samuel Wales wrote: >> >>> thanks for the clarification. are you
Re: archiving speed [was Re: Tips on maintaining history in Org Mode]
what is the current status of hierarchy in archive files? surely they don't deal with updating categories and updating hierarchy structure [sounds brittle and syncy]? i'm thinking it isn't hierarchical at present, except when you have a doneified task with children? On 8/11/21, Tim Cross wrote: > I think the problem with just using append to file is that it won't > preserve the shape of the file. For example, if I had a file with > > * Notes > ** Note 1 >blah blah > ** Note 2 blah blah > > * Tasks > ** DONE task 1 > ** TODO Task 2 > > and I decide to archive note 1 and task 1, I would like them to both appear > under the same headings and with the same level. If the process just uses > append to file, I can have this for the first archiving i.e. > > * Noes > ** Note 1 > > * Tasks > ** DONE task 1 > > but then later, I decide to archive note 2, if append file is used, I will > end up with > > * Notes > ** Note 1 > > * Taks > ** DONE task 1 > > * Notes > ** Note 2 > > which is not what I want. I want > > * Notes > ** Note 1 > ** Note 2 > > * Tasks > ** DONE Task 1 > > So, if we want to preserve hierarchies in our archive files and not have > everything jumbled up together, the system has to parse the file. If you > are also using something like Categories, then even more work needs to be > odne to update the category lists. > > What I tend to do is mark items with the ARCHIVE tag and leave them in the > file and then every few months, move archived data to archive files. It > can still get slow, but I don't do it often, so it isn't too much of a > hassle. > > > On Thu, 12 Aug 2021 at 08:23, Samuel Wales wrote: > >> thanks for the clarification. are you saying that, for every archived >> entry, it calculates teh category property, using the original org >> file, in order to add a category property to just one archived entry? >> >> that would certainly slow down more and more, but it sends me back to >> my question about whether append to file would work. >> i.e. build the single entry in a temporary buffer then write that >> region to a file on disk. >> >> On 8/10/21, Ihor Radchenko wrote: >> > Samuel Wales writes: >> > >> >> i should clarify. bulk archiving slows down even with /nonexistent/ >> >> (have not tried empty) archives. as part of normal and expected >> >> operation, bulk creates the archive for the first entry, and then >> >> subsequent entries are added. those get slower and slower. >> > >> > That's what I suspected. I also see this and my suggestion helped >> > archiving speed in my case. >> > >> >> i use (olpath category itags). i will try (file time) when i can, if >> >> that still applies. my brain needs to be more operational. >> > >> > When you use category, every time you modify the original file (not the >> > archive!), Org mode re-calculates *all* the categories in the original >> > Org file. It happens for every single archived heading. If your >> > original >> > Org file is large, re-calculations make things extremely slow. >> > >> > Best, >> > Ihor >> > >> >> >> -- >> The Kafka Pandemic >> >> Please learn what misopathy is. >> >> https://thekafkapandemic.blogspot.com/2013/10/why-some-diseases-are-wronged.html >> >> > > -- > regards, > > Tim > > -- > Tim Cross > -- The Kafka Pandemic Please learn what misopathy is. https://thekafkapandemic.blogspot.com/2013/10/why-some-diseases-are-wronged.html
Re: archiving speed [was Re: Tips on maintaining history in Org Mode]
thank you. i will give up on archiving categories if needed to make archiving be practical. will ahfe to try it as soon as i can. On 8/11/21, Ihor Radchenko wrote: > Samuel Wales writes: > >> thanks for the clarification. are you saying that, for every archived >> entry, it calculates teh category property, using the original org >> file, in order to add a category property to just one archived entry? > > Nope. It does not just calculate category for the archived entry, but > re-calculates all the category properties in the original Org file > (updating category cache). > >> that would certainly slow down more and more, but it sends me back to >> my question about whether append to file would work. >> i.e. build the single entry in a temporary buffer then write that >> region to a file on disk. > > Appending can indeed work if your archive location is at the end of the > file. However, it is not necessary the performance bottleneck. Certainly > not when the archive file is small. > > Best, > Ihor > -- The Kafka Pandemic Please learn what misopathy is. https://thekafkapandemic.blogspot.com/2013/10/why-some-diseases-are-wronged.html
Re: archiving speed [was Re: Tips on maintaining history in Org Mode]
I think the problem with just using append to file is that it won't preserve the shape of the file. For example, if I had a file with * Notes ** Note 1 blah blah ** Note 2 blah blah * Tasks ** DONE task 1 ** TODO Task 2 and I decide to archive note 1 and task 1, I would like them to both appear under the same headings and with the same level. If the process just uses append to file, I can have this for the first archiving i.e. * Noes ** Note 1 * Tasks ** DONE task 1 but then later, I decide to archive note 2, if append file is used, I will end up with * Notes ** Note 1 * Taks ** DONE task 1 * Notes ** Note 2 which is not what I want. I want * Notes ** Note 1 ** Note 2 * Tasks ** DONE Task 1 So, if we want to preserve hierarchies in our archive files and not have everything jumbled up together, the system has to parse the file. If you are also using something like Categories, then even more work needs to be odne to update the category lists. What I tend to do is mark items with the ARCHIVE tag and leave them in the file and then every few months, move archived data to archive files. It can still get slow, but I don't do it often, so it isn't too much of a hassle. On Thu, 12 Aug 2021 at 08:23, Samuel Wales wrote: > thanks for the clarification. are you saying that, for every archived > entry, it calculates teh category property, using the original org > file, in order to add a category property to just one archived entry? > > that would certainly slow down more and more, but it sends me back to > my question about whether append to file would work. > i.e. build the single entry in a temporary buffer then write that > region to a file on disk. > > On 8/10/21, Ihor Radchenko wrote: > > Samuel Wales writes: > > > >> i should clarify. bulk archiving slows down even with /nonexistent/ > >> (have not tried empty) archives. as part of normal and expected > >> operation, bulk creates the archive for the first entry, and then > >> subsequent entries are added. those get slower and slower. > > > > That's what I suspected. I also see this and my suggestion helped > > archiving speed in my case. > > > >> i use (olpath category itags). i will try (file time) when i can, if > >> that still applies. my brain needs to be more operational. > > > > When you use category, every time you modify the original file (not the > > archive!), Org mode re-calculates *all* the categories in the original > > Org file. It happens for every single archived heading. If your original > > Org file is large, re-calculations make things extremely slow. > > > > Best, > > Ihor > > > > > -- > The Kafka Pandemic > > Please learn what misopathy is. > > https://thekafkapandemic.blogspot.com/2013/10/why-some-diseases-are-wronged.html > > -- regards, Tim -- Tim Cross
Re: archiving speed [was Re: Tips on maintaining history in Org Mode]
Samuel Wales writes: > thanks for the clarification. are you saying that, for every archived > entry, it calculates teh category property, using the original org > file, in order to add a category property to just one archived entry? Nope. It does not just calculate category for the archived entry, but re-calculates all the category properties in the original Org file (updating category cache). > that would certainly slow down more and more, but it sends me back to > my question about whether append to file would work. > i.e. build the single entry in a temporary buffer then write that > region to a file on disk. Appending can indeed work if your archive location is at the end of the file. However, it is not necessary the performance bottleneck. Certainly not when the archive file is small. Best, Ihor
Re: archiving speed [was Re: Tips on maintaining history in Org Mode]
thanks for the clarification. are you saying that, for every archived entry, it calculates teh category property, using the original org file, in order to add a category property to just one archived entry? that would certainly slow down more and more, but it sends me back to my question about whether append to file would work. i.e. build the single entry in a temporary buffer then write that region to a file on disk. On 8/10/21, Ihor Radchenko wrote: > Samuel Wales writes: > >> i should clarify. bulk archiving slows down even with /nonexistent/ >> (have not tried empty) archives. as part of normal and expected >> operation, bulk creates the archive for the first entry, and then >> subsequent entries are added. those get slower and slower. > > That's what I suspected. I also see this and my suggestion helped > archiving speed in my case. > >> i use (olpath category itags). i will try (file time) when i can, if >> that still applies. my brain needs to be more operational. > > When you use category, every time you modify the original file (not the > archive!), Org mode re-calculates *all* the categories in the original > Org file. It happens for every single archived heading. If your original > Org file is large, re-calculations make things extremely slow. > > Best, > Ihor > -- The Kafka Pandemic Please learn what misopathy is. https://thekafkapandemic.blogspot.com/2013/10/why-some-diseases-are-wronged.html
Re: archiving speed [was Re: Tips on maintaining history in Org Mode]
Samuel Wales writes: > i should clarify. bulk archiving slows down even with /nonexistent/ > (have not tried empty) archives. as part of normal and expected > operation, bulk creates the archive for the first entry, and then > subsequent entries are added. those get slower and slower. That's what I suspected. I also see this and my suggestion helped archiving speed in my case. > i use (olpath category itags). i will try (file time) when i can, if > that still applies. my brain needs to be more operational. When you use category, every time you modify the original file (not the archive!), Org mode re-calculates *all* the categories in the original Org file. It happens for every single archived heading. If your original Org file is large, re-calculations make things extremely slow. Best, Ihor
Re: archiving speed [was Re: Tips on maintaining history in Org Mode]
i should clarify. bulk archiving slows down even with /nonexistent/ (have not tried empty) archives. as part of normal and expected operation, bulk creates the archive for the first entry, and then subsequent entries are added. those get slower and slower. i was trying to imply e.g. doing archives by year won't fix it. only a few entries are enough to slow it down. i use (olpath category itags). i will try (file time) when i can, if that still applies. my brain needs to be more operational. i should mention that i did also find a bug, but was not able to narrow it down. it has been a while, but it was something like not killing the original entry for one of the entries. i was unable to figure out what conditions needed to obtain for this to occur. On 8/10/21, Ihor Radchenko wrote: > Samuel Wales writes: > >> [this is the case even with zero-size archive files; after a few >> entries it slows down.] > > Do you get the same behaviour with the following code? > > (setq org-archive-save-context-info '(file time)) > > Best, > Ihor > -- The Kafka Pandemic Please learn what misopathy is. https://thekafkapandemic.blogspot.com/2013/10/why-some-diseases-are-wronged.html
Re: archiving speed [was Re: Tips on maintaining history in Org Mode]
Samuel Wales writes: > [this is the case even with zero-size archive files; after a few > entries it slows down.] Do you get the same behaviour with the following code? (setq org-archive-save-context-info '(file time)) Best, Ihor
Re: archiving speed [was Re: Tips on maintaining history in Org Mode]
i have not bulk archived in a year or two, because it is too slow for me. i am curious why, when appending to an archive file, append-to-file or write-region are not used, to merely put the entries there instead of loading the archive file into emacs? [this is the case even with zero-size archive files; after a few entries it slows down.] On 2/28/21, Samuel Wales wrote: > thank you for your detaild reply. > > more below. > > On 2/28/21, Ihor Radchenko wrote: >> details why). So, many org commands tend to lag on large archives. > > that makes sense. but why would appending to an archive as the result > of bulk archiving lag? if the problem is large archive files, which > i'd bet is the case for a lot of users and not just me, then could org > in principle be changed so that all it does is append? thus not lag? > like, build the entry in a temporary buffer? > > as i see it, having more than one archive file per org file is good > for speed, but doesn't work in existing org, because iirc e.g. v A in > the agenda goes org agenda file -> corresponding archive file and will > miss the archive files that do not have a corresponding org file with > exactly the same basename sans extension. > > i'd be ok with released org either allowing hte user to make > year-based archives by having all of org recognize them, or my just > append thing above. maybe i am missing something. > >> The lags can be solved in several ways: >> 1. Reduce the archive file size > > this implies to me e.g. year-based archives, which would fail the v A > test iiuc. thus needed extra code. > >> 2. Use optimised folding mechanism [1] (this will speed up org-mode in >> general as well) > > i look forward to this filtering down to maint. :] [i used to follow > master but too much for me now for health reasons.] > >> 3. (untested) Put #+STARTUP: showeverything at the beginning of the >>archives, so that nothing is going to be folded > > good idea. my included-by-agenda archive files do seem to be in > showeveryting mode already for some reason. but perhaps not when bulk > archiving. > > would it be a silly idea for an fr that org make this an option for > bulk archiving? hmm or for archive files in general? > >>> i will keep in mind disabling font lock in archive files. any >>> suggested code for that? >> >> Note that it will mostly affect find-file performance. To disable > > if so, then i figure it's a one-time thing per file so no big deal. > but thanks for hte font lock stuff i didnt' know about. > >> Sorry, the config is actually not yet formatted for public use. You can >> search for the code block containing "defun >> org-archive--compute-location". > > firefox find does not seem to find it. > >> >> You will need that code block and the following code block. >> >> [1] https://github.com/yantar92/org >> >> Best, >> Ihor >> >> > > > -- > The Kafka Pandemic > > Please learn what misopathy is. > https://thekafkapandemic.blogspot.com/2013/10/why-some-diseases-are-wronged.html > -- The Kafka Pandemic Please learn what misopathy is. https://thekafkapandemic.blogspot.com/2013/10/why-some-diseases-are-wronged.html
Re: Tips on maintaining history in Org Mode
Ihor Radchenko writes:
> David Masterson writes:
>
>> Interesting, but then how do you get the list? I mean is there an
>> agenda to use?
>
> Generally yes, you can use agenda. Or you can use sparse tree (more manual).
> For agenda, if you customise org-log-done, you can use
> org-agenda-log-mode ("v l" or "v L" in an agenda buffer). You can add
> archived items as well with "v a" or/and "v A".
>
> Just org-agenda-log-mode will show everything, not just calls. Narrowing
> to calls only will depend on how you define a todo, which is a call.
>
> If you use something like PHONE or CALL todo keywords, it might be
> slightly tricky. You will need to customise org-todo-keywords, so that
> your CALL->DONE changes are recorded (see the org-todo-keywords
> docstring). You will also need to filter displayed items in agenda by
> regexp involving the keyword you use to define the call.
>
> An easier way could be marking your calls with a tag. Then, you can
> filter your org-agenda-log by that tag to show only calls.
>
> Hope it helps.
Nice! Thanks.
--
David Masterson
Re: archiving speed [was Re: Tips on maintaining history in Org Mode]
Samuel Wales writes: > that makes sense. but why would appending to an archive as the result > of bulk archiving lag? if the problem is large archive files, which > i'd bet is the case for a lot of users and not just me, then could org > in principle be changed so that all it does is append? thus not lag? > like, build the entry in a temporary buffer? Looking into the source code, the culprit appears to be the call to org-show-all inside org-archive-subtree. The call to org-show-all on master calls org-cycle-hide-drawers, which goes through every single drawer (= every archived heading in archive file) and folds it manually (just to unfold it later). Do not ask. > as i see it, having more than one archive file per org file is good > for speed, but doesn't work in existing org, because iirc e.g. v A in > the agenda goes org agenda file -> corresponding archive file and will > miss the archive files that do not have a corresponding org file with > exactly the same basename sans extension. The code from my config takes care about this. v A will work. I plan to suggest it for master in future, but I am focused on the org-fold branch for now. If you wish, feel free to propose a patch based on this code. >> 3. (untested) Put #+STARTUP: showeverything at the beginning of the >>archives, so that nothing is going to be folded > > good idea. my included-by-agenda archive files do seem to be in > showeveryting mode already for some reason. but perhaps not when bulk > archiving. > > would it be a silly idea for an fr that org make this an option for > bulk archiving? hmm or for archive files in general? Upon reviewing the source of org-archive-subtree, this should not be needed. Actually, org-mode already disables startup visibility and various hooks when opening archives. As you can see, it is not sufficient, since org-archive shoots its own leg later by changing visibility every time you archive a subtree. >> Sorry, the config is actually not yet formatted for public use. You can >> search for the code block containing "defun org-archive--compute-location". > > firefox find does not seem to find it. This is odd. All I can suggest then is cloning the repo and searching using Emacs. Emacs is more reliable when opening org files ;) git clone https://github.com/yantar92/emacs-config emacs emacs-config/config.org Best, Ihor
archiving speed [was Re: Tips on maintaining history in Org Mode]
thank you for your detaild reply. more below. On 2/28/21, Ihor Radchenko wrote: > details why). So, many org commands tend to lag on large archives. that makes sense. but why would appending to an archive as the result of bulk archiving lag? if the problem is large archive files, which i'd bet is the case for a lot of users and not just me, then could org in principle be changed so that all it does is append? thus not lag? like, build the entry in a temporary buffer? as i see it, having more than one archive file per org file is good for speed, but doesn't work in existing org, because iirc e.g. v A in the agenda goes org agenda file -> corresponding archive file and will miss the archive files that do not have a corresponding org file with exactly the same basename sans extension. i'd be ok with released org either allowing hte user to make year-based archives by having all of org recognize them, or my just append thing above. maybe i am missing something. > The lags can be solved in several ways: > 1. Reduce the archive file size this implies to me e.g. year-based archives, which would fail the v A test iiuc. thus needed extra code. > 2. Use optimised folding mechanism [1] (this will speed up org-mode in > general as well) i look forward to this filtering down to maint. :] [i used to follow master but too much for me now for health reasons.] > 3. (untested) Put #+STARTUP: showeverything at the beginning of the >archives, so that nothing is going to be folded good idea. my included-by-agenda archive files do seem to be in showeveryting mode already for some reason. but perhaps not when bulk archiving. would it be a silly idea for an fr that org make this an option for bulk archiving? hmm or for archive files in general? >> i will keep in mind disabling font lock in archive files. any >> suggested code for that? > > Note that it will mostly affect find-file performance. To disable if so, then i figure it's a one-time thing per file so no big deal. but thanks for hte font lock stuff i didnt' know about. > Sorry, the config is actually not yet formatted for public use. You can > search for the code block containing "defun org-archive--compute-location". firefox find does not seem to find it. > > You will need that code block and the following code block. > > [1] https://github.com/yantar92/org > > Best, > Ihor > > -- The Kafka Pandemic Please learn what misopathy is. https://thekafkapandemic.blogspot.com/2013/10/why-some-diseases-are-wronged.html
Re: Tips on maintaining history in Org Mode
Samuel Wales writes: > Hi Ihor, > > it never occurred to me that bulk archiving could be sped up by > changing anything about the archive files. i assumed that archiving > worked by appending to those files, so once the initial find-file was > performed, there should be no additional slowness. Disregarding the initial find-file (which may also be very slow on large archives), the slowness is happening because many org-mode commands are using line-motion functions that respect visibility. Those commands become slow when there are many folded elements (see [1] for technical details why). So, many org commands tend to lag on large archives. The lags can be solved in several ways: 1. Reduce the archive file size 2. Use optimised folding mechanism [1] (this will speed up org-mode in general as well) 3. (untested) Put #+STARTUP: showeverything at the beginning of the archives, so that nothing is going to be folded > i will keep in mind disabling font lock in archive files. any > suggested code for that? Note that it will mostly affect find-file performance. To disable font-lock, you will need to customise font-lock-global-modes, so that it does not auto-start in org-mode. The value should be something like '(not org-mode), but check the current value - you may have other customisation in place. Then, you will need to have a custom org-mode-hook that activates font-lock only when the org file is not the archive file. Something like (defun yant/font-lock-activate-maybe () "Activate font-lock in non-archive org buffers." (when (org-all-archive-files) (font-lock-mode +1))) (add-hook 'org-mode-hook #'yant/font-lock-activate-maybe) > your config is impressively [and for me > dauntingly] comprehensive. the Archiving anchor does not seem to have > a referent. I searched for archiving, but didn't seem to find what > you were probably pointing me to. Sorry, the config is actually not yet formatted for public use. You can search for the code block containing "defun org-archive--compute-location". You will need that code block and the following code block. [1] https://github.com/yantar92/org Best, Ihor
Re: Tips on maintaining history in Org Mode
Hi Ihor, it never occurred to me that bulk archiving could be sped up by changing anything about the archive files. i assumed that archiving worked by appending to those files, so once the initial find-file was performed, there should be no additional slowness. yet i tried a new file with no archive file, and bulk archiving took less than a second for 9 entries. before it would take idk a minute. i will keep in mind disabling font lock in archive files. any suggested code for that? your config is impressively [and for me dauntingly] comprehensive. the Archiving anchor does not seem to have a referent. I searched for archiving, but didn't seem to find what you were probably pointing me to. thanks.
Re: Tips on maintaining history in Org Mode
Samuel Wales writes: > [currently the archiver is so slow i can't use it] Are your existing archives very big (few Mbs)? If so, you may try to speed up the archiving using feature/org-fold branch [1]. If that is not enough, I recommend splitting archives on yearly basis [2] or disabling font-lock in archive files. Best, Ihor [1] https://github.com/yantar92/org [2] https://github.com/yantar92/emacs-config/blob/master/config.org#archiving
Re: Tips on maintaining history in Org Mode
Samuel Wales writes: > perhaps also changing org-archive-file-header-format to allow a format > thingie for a timestamp would allow you to take parts of an archive > file and move them into one per year without having to put the date in > each archived entry. FYI: I have implemented automatic per-year archiving, which is correctly handled by other org commands in my personal config: https://github.com/yantar92/emacs-config/blob/master/config.org#archiving Best, Ihor
Re: Tips on maintaining history in Org Mode
David Masterson writes:
> Interesting, but then how do you get the list? I mean is there an
> agenda to use?
Generally yes, you can use agenda. Or you can use sparse tree (more manual).
For agenda, if you customise org-log-done, you can use
org-agenda-log-mode ("v l" or "v L" in an agenda buffer). You can add
archived items as well with "v a" or/and "v A".
Just org-agenda-log-mode will show everything, not just calls. Narrowing
to calls only will depend on how you define a todo, which is a call.
If you use something like PHONE or CALL todo keywords, it might be
slightly tricky. You will need to customise org-todo-keywords, so that
your CALL->DONE changes are recorded (see the org-todo-keywords
docstring). You will also need to filter displayed items in agenda by
regexp involving the keyword you use to define the call.
An easier way could be marking your calls with a tag. Then, you can
filter your org-agenda-log by that tag to show only calls.
Hope it helps.
Best,
Ihor
Re: Tips on maintaining history in Org Mode
note that there is an issue when you try to name your archive files using years like computer-2000.org_archive. it can take seconds to find-file big files so it is understandable to want to name files like that. however, if you change the name of an archive file, it will not be found by org when you tell it to do certain things with archive files. some code in org derives the basename to search from the basename of the original file like computer,org. in principle, maybe org could allow year suffixes/prefixes or it could search all archive files in dirs that have org files. perhaps also changing org-archive-file-header-format to allow a format thingie for a timestamp would allow you to take parts of an archive file and move them into one per year without having to put the date in each archived entry. On 2/26/21, David Masterson wrote: > Tim Cross writes: > >> David Masterson writes: >> >>> What would you use to then make a list of all meetings you had last >>> year? >> >> For me, archiving is about data I'm unlikely to need again, but just in >> case I do, it is in the archive. I rarely look at my archives. However, >> when I do archive, I will usually archive into a 'year' file. So, to >> find all the meetings held in 2015, I would open that archive file and >> search for entries with the tag MEETING (I also have a tag for PHONE). > > Got it. Thanks. > -- > David Masterson > > -- The Kafka Pandemic Please learn what misopathy is. https://thekafkapandemic.blogspot.com/2013/10/why-some-diseases-are-wronged.html
Re: Tips on maintaining history in Org Mode
Tim Cross writes: > David Masterson writes: > >> What would you use to then make a list of all meetings you had last year? > > For me, archiving is about data I'm unlikely to need again, but just in > case I do, it is in the archive. I rarely look at my archives. However, > when I do archive, I will usually archive into a 'year' file. So, to > find all the meetings held in 2015, I would open that archive file and > search for entries with the tag MEETING (I also have a tag for PHONE). Got it. Thanks. -- David Masterson
Re: Tips on maintaining history in Org Mode
David Masterson writes: > Tim Cross writes: > >> David Masterson writes: >> >> For me, my TODOs are setup so that they record a date stamp for when >> they were added and whenever they change state e.g. started, done, >> delegated etc. > > So, you use progress logging. Yes. > >> For non-TODO items, I will often put an inactive timestamp in the >> heading title. > > Do your headings become busy? > Some would feel they are 'busy'. I always put the timestamp at the beginning of the heading, so there is a regular pattern (not much different from the leading heading stars) and I've just got use to it, so I don't really see it now. > What would you use to then make a list of all meetings you had last year? For me, archiving is about data I'm unlikely to need again, but just in case I do, it is in the archive. I rarely look at my archives. However, when I do archive, I will usually archive into a 'year' file. So, to find all the meetings held in 2015, I would open that archive file and search for entries with the tag MEETING (I also have a tag for PHONE). -- Tim Cross
Re: Tips on maintaining history in Org Mode
Eric S Fraga writes: > My approach is simple. For TODO items, I archive to separate file when > done. That file is easily searchable, e.g. using C-c /. Ah! org-occur! That's something forgot about and looks useful. > I keep both the original file and the archive file under revision > control, just in case. I do that, too. >> The issue (I think) is, when you mark the TODO as DONE, you lose the >> info of what the TODO was originally. > > What info is lost? In fact, if you log state changes, all the > information is there. I see how that workss now with org-occur. > By the way, if you have recurring items, one package that you might find > useful is org-recur which is on ELPA. Thanks -- David Masterson
Re: Tips on maintaining history in Org Mode
Tim Cross writes: > David Masterson writes: > >> There are many ways of maintaining history in a group of Org files: >> 1. Archive within a file >> 2. Archive to a separate (archive) file >> 3. Special TODO types for history >> 4. Special TAG types for history >> 5. etc. >> >> My question is, if you have meetings/phone calls as TODOs, what is the >> preferred way to handle when they move into history so that, *much* >> later, you can easily produce a list of all of the meetings/phone calls >> with dates and times of them? The issue (I think) is, when you mark the >> TODO as DONE, you lose the info of what the TODO was originally. > > A lot will depend on your requirements. > > For me, my TODOs are setup so that they record a date stamp for when > they were added and whenever they change state e.g. started, done, > delegated etc. So, you use progress logging. > For non-TODO items, I will often put an inactive timestamp in the > heading title. Do your headings become busy? What would you use to then make a list of all meetings you had last year? -- David Masterson
Re: Tips on maintaining history in Org Mode
Ihor Radchenko writes: > David Masterson writes: >> My question is, if you have meetings/phone calls as TODOs, what is the >> preferred way to handle when they move into history so that, *much* >> later, you can easily produce a list of all of the meetings/phone calls >> with dates and times of them? The issue (I think) is, when you mark the >> TODO as DONE, you lose the info of what the TODO was originally. > > See Org manual :: 5.3 Progress Logging Interesting, but then how do you get the list? I mean is there an agenda to use? -- David Masterson
Re: Tips on maintaining history in Org Mode
org does indeed have a lot of related features, maybe too many even. here is some of what i do. - if i doneify, it means i will likely not need to search for it. archived to a file. [currently the archiver is so slow i can't use it] CLOSED: [2012-11-08 Thu 19:40] - state logging for repeaters in logbook - your question: notes to keep around like this * LOG [2021-02-26 Fri 14:18] talked to bill gates; he uses linux now these get sorted at bottom by timestamp using user-defined this is the main thing - bare inactive free form as needed -- The Kafka Pandemic Please learn what misopathy is. https://thekafkapandemic.blogspot.com/2013/10/why-some-diseases-are-wronged.html
Re: Tips on maintaining history in Org Mode
On 2021-02-26 01:22, David Masterson wrote: There are many ways of maintaining history in a group of Org files: 1. Archive within a file 2. Archive to a separate (archive) file 3. Special TODO types for history 4. Special TAG types for history 5. etc. My question is, if you have meetings/phone calls as TODOs, what is the preferred way to handle when they move into history so that, *much* later, you can easily produce a list of all of the meetings/phone calls with dates and times of them? The issue (I think) is, when you mark the TODO as DONE, you lose the info of what the TODO was originally. Suggestions I agree with what others already said about logging state changes with timestamps. I do the same and find this an extremely handy place to put "metadata about the task, or reason that it changed" and keep it separate/hidden from the main body of the task (which in my mind should only contain info about the subject of the headline / task itself). However when you say "history" I suspect you mean "archive" and this is something I have been thinking about a long time as well. Because I think the current way that Orgmode handles this is a bit naive / simplistic. Well, at least there is the option to create your own archival function, which I suspect I will do at some point. In fact I have been thinking about the best way to do that for some time already. Cheers, TRS-80
Re: Tips on maintaining history in Org Mode
My approach is simple. For TODO items, I archive to separate file when done. That file is easily searchable, e.g. using C-c /. I keep both the original file and the archive file under revision control, just in case. > The issue (I think) is, when you mark the TODO as DONE, you lose the > info of what the TODO was originally. What info is lost? In fact, if you log state changes, all the information is there. By the way, if you have recurring items, one package that you might find useful is org-recur which is on ELPA. -- : Eric S Fraga via Emacs 28.0.50, Org release_9.4.4-231-gf46925
Re: Tips on maintaining history in Org Mode
David Masterson writes: > There are many ways of maintaining history in a group of Org files: > 1. Archive within a file > 2. Archive to a separate (archive) file > 3. Special TODO types for history > 4. Special TAG types for history > 5. etc. > > My question is, if you have meetings/phone calls as TODOs, what is the > preferred way to handle when they move into history so that, *much* > later, you can easily produce a list of all of the meetings/phone calls > with dates and times of them? The issue (I think) is, when you mark the > TODO as DONE, you lose the info of what the TODO was originally. > A lot will depend on your requirements. For me, my TODOs are setup so that they record a date stamp for when they were added and whenever they change state e.g. started, done, delegated etc. For non-TODO items, I will often put an inactive timestamp in the heading title. I also make extensive use of the ability to add timestamp entries as part of capture templates - for exmaple, my notes and even the file of bookmakrs (RUIs) I have. -- Tim Cross
Re: Tips on maintaining history in Org Mode
David Masterson writes: > My question is, if you have meetings/phone calls as TODOs, what is the > preferred way to handle when they move into history so that, *much* > later, you can easily produce a list of all of the meetings/phone calls > with dates and times of them? The issue (I think) is, when you mark the > TODO as DONE, you lose the info of what the TODO was originally. See Org manual :: 5.3 Progress Logging
