That seems hierarchical, which is ok (as in org-mode itself) but how about implementing a more general graph mechanism, which could be used to do this but more flexibly?
On Sat, 14 Dec 2019, 21:04 Gustav Wikström, <gus...@whil.se> wrote: > Hi list and all honored readers! > > I have an idea. One that I've mentioned before in side notes. And I want > to emphasize that this still only is an idea. But I want to present this > idea to you. As a way to gather feedback and get input. Maybe the idea is > stupid!? Maybe you think it's already solved? Or maybe it's not, and lots > of you resonate with it as well. In any case, please let me know what you > think on the piece below! > > ________________________________ > > ORG MODE: COLLECTIONS/PROJECTS > > Gustav Wikström > ________________________________ > > > Table of Contents > _________________ > > 1. Motivation > 2. Idea > 3. Benefit > .. 1. For the user > .. 2. For the developer > 4. Example use cases > .. 1. Separate actions from reference > .. 2. Work / Personal separation > .. 3. Separated book library > .. 4. More? > 5. Risks and challenges > .. 1. Which configuration to use? > .. 2. Should project config allow local variables? > ..... 1. How to initialize the local variables? > .. 3. Conflict with other customizations > .. 4. Files that belong to multiple collections > .. 5. Dynamic lists of files and folders for a collection? > 6. Alternatives > 7. References > > > 1 Motivation > ============ > > Org mode is more than a major mode for emacs buffers. That has been > clear for quite some time. Org mode can operate on sets of files. > Consolidate TODO's and calendar information into custom views. Publish > sets of files. To do this Org mode assumes that the user has a > configuration for each of those features. Each feature is responsible > for maintaining its own context. And almost all of that context has > to be set globally. So even though Org mode has commands and features > that operate on sets of files and folders it has not yet developed > that in a congruent, extensible and composable way. Thus, for the > sanity of our users and developers I think it's time to ... introduce > another concept! One that hopefully can simplify things both for users > and developers. > > > 2 Idea > ====== > > I propose to introduce `Collection' as a concept in the realm of Org > mode. [1] > > An Org mode collection is defined as the combination of: > 1. A short name and description > 2. A collection of Org mode documents > 3. A collection of files and/or folders called attachments and > attachment-locations for the project > 4. A collection of configurations for the given project > > Globally available collections are defined in a list, > `org-collections'. Org mode should include a safe parameter that can > be set as a folder customization to the local active project, > `org-collections-active'. The default should be to the first entry in > `org-collections' unless customized. This local parameter would be > used to instruct Emacs and Org mode on which collection is active. > Only one collection at a time can be active. > > Org agenda should use `org-collections-active' as default for the > collection of Org mode documents to operate on. Org agenda should get > a new command to switch between active projects. > > I'm thinking that there could be a special Emacs major mode for the > collection as well, called "Org collections mode". Not sure exactly > what to display and how to represent the project there... But > certainly some kind of list of included documents and attachments. > When in that mode there should possibly be single key > keyboard-shortcuts to the most important features that operate on the > collection. And switch between them. > > > 3 Benefit > ========= > > 3.1 For the user > ~~~~~~~~~~~~~~~~ > > A user would gain mainly two benefits as I can see right now: > 1. The ability to clearly define (multiple) collections of files that > belong together across org mode, with unique configurations. > 2. Less global configuration state to manage and worry about! > > The second point might not look like much but is sooo important! Most > programmers know that global state should be avoided. Putting things > in a context most of the time makes things better. And if we can > configure Org mode connected to a context it makes it much more useful > for those who use Org mode for multiple purposes. > > The first point is equally important in my opinion. Today one must > configure Org mode per feature. If you want to configure publishing > you do that globally. If you want to configure the agenda, you have to > do that globally as well. If you want to define a location for > attachments, do it globally! What about custom TODO-keywords? Do it > globally! Track ID-locations? Define a location globally! > > All above adds cognitive load to the user and makes it difficult to > maintain the configuration as the use of Org mode grows (as it should > ;) ). You have to define the context for each and every feature for it > to know what to operate on. I claim that both the human psyche and the > system itself will have a much more easy time if it could configure > these features together, in a given context! > > > 3.2 For the developer > ~~~~~~~~~~~~~~~~~~~~~ > > I claim there will be benefits for developers as well. Today there > exists many packages that extend Org mode functionality. Many work > with the idea of collections. Some that come to mind: > - Org brain (<https://github.com/Kungsgeten/org-brain>) > - Org ql (<https://github.com/alphapapa/org-ql>) > - Ox hugo (<https://ox-hugo.scripter.co/>) > > I think that with the addition of the `collections' concept into Org > mode, package developers get a concept they can easily attach to. Yes, > you can easily define your own package-specific concept for that as > well. But then the user loses out in having to configure another > feature. And yes, today you as a developer can say that Org agenda > will be my collection to operate on. But this is a big limitation > since it limits what your package effectively can only work to a > single list of files. > > Having a collections concept means you as a developer have another > base on which you can extend. No need to define your own concept if > `Org-agenda-files' isn't enough; make it work together with > `org-collections' instead. Org mode users will be happy because what > they have already defined as important for them can be reused for new > things with ease. > > Developing features inside Org mode itself hopefully also can benefit > from this concept. I'm sure there are many people out there with cool > ideas on how to extend and work with Org documents. And I'm equally > sure that the value of developing many of those features will be > bigger if they could naturally attach to an Org collections > definition! > > > 4 Example use cases > =================== > > 4.1 Separate actions from reference > ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ > > One practice promoted by GTD is to separate actionable items from > reference information. While that practice can be overcome by search > etc. some might still value a clear separation. > > Want to look up something related to my general references? Search the > Org collection related to reference-information! Maybe set up custom > views and uses of TODO keywords for reference information for special > agenda views. > > Want to only display not yet finished tasks? Switch to the Org > collection for actionable items and browse away. > > > 4.2 Work / Personal separation > ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ > > The heading says it all. Some like to separate work and personal stuff > out from each other. What more clear way to do that than can there be > than to separate them into their own Org collections? That way you > potentially could let your work-related workflow (I.e. TODO-keywords) > be different than the personal workflow. Without having to think about > a global configuration that has to allow for both. > > > 4.3 Separated book library > ~~~~~~~~~~~~~~~~~~~~~~~~~~ > > Org mode can be used as a media manager of sort. Just define your > conventions for the Org collection using TODO-keywords, categories and > properties. Attach the e-books you have as attachments in an > attachment-scheme special for your book library. Configure export of > the library using maybe a custom HTML/CSS-visual and publish it > somewhere for yourself to look at when on your phone. And do this > without having to think of how changing all these things will affect > the global state of Org mode, potentially messing up your other uses > for task management or other notes and libraries you're trying to > manage! > > Note that one can still have a holistic view on all Org mode documents > as well, if important. It only requires a definition of a collection > as the collection of all other collections! > > > 4.4 More? > ~~~~~~~~~ > > Please add more ideas when you think of them! > > > 5 Risks and challenges > ====================== > > 5.1 Which configuration to use? > ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ > > When I'm visiting a file that belongs to a collection, how should > Emacs resolve configurations for that file? > > There may be configurations in the following places: > - Global in `emacs-custom.el' or `.emacs.d/init.el' > - Directory local variables in the tree > - File local variables > - Local variables for the project definition in which the file > belongs? > > Should visiting a file always have to scan the collections list to see > if the file belongs to any of them, in order to load customizations? > Hmm... Maybe!? Or - maybe not if Emacs can rely on the fact that the > user cares to set the local variable `org-collections-active' (or > whatever it should be called)? In that case, just evaluate the > settings for that project without doing any scan. > > > 5.2 Should project config allow local variables? > ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ > > Should the collections definition allow customization of variables > that apply for Org mode features? Hmm... Maybe!? One thing that comes > to mind is that a project should be able to define a custom attachment > directory... How else would the attachment-feature know what > attachment directory to use for files in that collection? > > Another option could ofc. be that each feature would have to add > support for looking into the collection definition and override the > local variable. But that will add development effort and complexity to > each feature. Not suggested. > > > 5.2.1 How to initialize the local variables? > -------------------------------------------- > > When visiting a file that belongs to a collection, should Emacs at > that point initialize the collection-configuration for that > collection? Ideally some kind of collection-resolution would be made. > Otherwise users will get strange behaviors when the think they are in > one project but Org mode hasn't changed the local variables to match > it. On the other hand, it doesn't sound very performant to have to > check collection-belonging every time an Org mode file is visited! > > Possibly solve this with a variable that can be localized - > `org-collections-active'? > > > 5.3 Conflict with other customizations > ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ > > Maybe I've defined an attachment directory as a directory local > variables in a folder, for all subfolders and files to inherit. Should > collection-customizations override that? Or should the directory local > variables take precedence? > > Maybe could be solved by letting the (advanced) user choose using a > customization itself, something like > `org-collections-precede-local-variables' ? Need a intuitive default > though. Most sane default is probably to let local variables take > precedence. Those are created by the user anyways, so she should be > aware. > > The more I think of it, there shouldn't be a customization for this at > all. I think local definitions always should override the collection > definition. > > > 5.4 Files that belong to multiple collections > ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ > > What if I'm being a clever user and define multiple collections for > the same files (I.e. overlap in the Venn-diagram of files grouped by > collections). Which collection is "active" when I'm visiting the file? > > This depends on if Emacs should evaluate the collection-settings for > each file visit or not. If they are evaluated for each file visit then > the first matching project in the list of collections should apply for > that file. If a cache is created that lists file and collection > relationships then each file should relate to a list of collections > where the first collection in that list should apply. > > If Emacs can rely on `org-collections-active' being set, then the > collection referenced there should be used. > > > 5.5 Dynamic lists of files and folders for a collection? > ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ > > Should the list of files allow for folders with recursion and patterns > should it be required to provide a fixed defined list of files? > > Preferably the same way as `org-agenda-files' work today. Maybe some > kind of caching-mechanism is needed though, for commands that might > have to look for file, collection relations. A cache adds potential > pain for the user though. If a file is added to a folder in a > collection and a "collection-command" is run then the new file might > not show up in the results anyway... So the user will be affected by > caching and will have to know about it. Not good... > > > 6 Alternatives > ============== > > Doing research for this feature made me realize that much of what I'm > proposing already exist! In another form though, as [directory > variables]. That requires customizations to be defined as safe though. > And today some of the things I would consider to define a collection > aren't safe. For example `org-agenda-files', `org-todo-keywords', > `org-publish-project-alist'. > > Some issues with relying on directory variables (Assuming they also > are made safe): > - When invoking Org agenda I will have to first visit a file inside a > specific folder to get the agenda for the correct project > - .... > > > [directory variables] <info:emacs#Directory variables> > > > 7 References > ============ > > I've mentioned this idea the Org mode mailing list previously, but > only as short side notes to other topics: > - < > https://lists.gnu.org/archive/html/emacs-orgmode/2018-11/msg00211.html> > - < > https://lists.gnu.org/archive/html/emacs-orgmode/2019-09/msg00010.html> > > Note that I've talked about it as "project". I think that name still > could be considered instead of "collection". Collection is more > general and less overloaded in terms of productivity software. And it > shifts the focus away from task management a bit, which I think can be > a good thing. Because while Org mode may often start to be used as a > task/project manager software, it's useful in a much wider context > than that! > > > > Footnotes > _________ > > [1] I've previously written about this as "Projects". While Project > was my initial name for this feature I think collection may be a > better option. For the sake of this text both options work just fine. > The idea is the same. >
