Re: One vs many directories
* Ihor Radchenko [2020-11-26 16:29]: > > Sorry for that vague expression. Let us say I open Completions buffer > > I can switch into it, inspect it, ask for defined keys, evaluate with > > M-:, Emacs allows me to remain in the window and go to other > > window. Agenda buffer does not do that, this is probably because it > > just waits for any key and does not allow anything else. That means I > > will open Agenda and I cannot switch to other window, so I will close > > agenda to switch. Maybe I have 10 TODO keywords, I have to open file, > > I open Agenda again, aha, T... then close agenda, open file see > > keyword, then open agenda again. Repetitions without end and user is > > unable to use multiple windows. This really need not be so. > > It appears to me (correct me if I am wrong) that you haven't tried > agenda multi-occur/keyword/etc searches. I know multi-occur and tried it. Functions of org-agenda are useful while initial menu window of org-agenda is so much less useful and I speak of menu, not of individual functions. If it is Org agenda, I see no reason why it could not be displayed in Org mode being read-only and having buffer not killed when user press q. We spoke of standard Emacs way. Normally buffers are not killed. If I press q in Dired buffer is just buried, not killed. I find Org agenda useful and I would be maybe moving to *Agenda Commands* buffer by using (next-buffer) and (prev-buffer) and those are key bindings like hyper key - move-end-of-line and hyper key - move-beginning of line. It is my standard behavior to move to one buffer and go to other buffer. User could have one tab open for Org agenda menu, other tab for work. Why invoke all time C-c a to access org agenda. I am not speaking for me personally, I am giving you clear comparison of that menu to mu4e or org-agenda menu and other pop-up features that do not block user in using Emacs. Org agenda menu blocks the interface! When invoking package-list-packages it does not block me moving to other buffers and also offers complex menu system and key bindings. Buffer is not killed but burried when I press q. Functions of org agenda are fine, this is reference to blocking of Emacs interface while opening Org agenda buffer. It diminishes usability. I guess nobody cares. It is how it is. No need to consult external references such as https://www.nngroup.com/articles/usability-101-introduction-to-usability/ No need even to look how other parts of Emacs are interacting with user and to harmonize the menu. > > You know how agenda is made like the 1985 BASIC menu in Commodore > > C=64, but even back then there was better interface for such menus. > > FYI. Transient.el - menu dispatcher in popular magit package also does > not let you switch buffers. So, apparently many people would not agree > that it is so terrible design. Compare things to things that are better not to things that are worse. You have to read more about usability. To know what is the problem you have to research and not assume that if nobody says anything that it is usable enough. Nobody complains. I did not hear complain, so it must be perfect. Irony. That is not how interfaces get improved and definitely not how developer would discover if something is wrong. If developer is waiting for somebody to complain that will be too late. Thousands of users will have problems that were not told to developer. Was there any methodology to decide what is good user menu? Reference: https://www.nngroup.com/articles/why-you-only-need-to-test-with-5-users/ Was there any survey here to ask about any usability?! Hypothetical question. > P.S. Nothing prevents you from calling, i.e. M-x org-occur or binding it > to a key of your choice. Personally I know that. I speak for general usability. You know that I have my system of doing things with or without Org mode.
Re: One vs many directories
* Ihor Radchenko [2020-11-25 14:48]: > > When I do C-c a it runs (org-agenda) but I do not have "g" and I am on > > development version. The C-c a window is made so that I cannot go with > > cursor inside and that I cannot even expect the key map neither invoke > > command by M-x and I cannot even M-: > > C-c a will first show so-called agenda dispatcher asking you what kind > of agenda view you want to get. You need to press a key according to > the popup window (i.e. `t' to see all not done items). Then, you will > get the proper agenda buffer with all the keymaps set and `g' bound to > refreshing the chosen agenda view in the buffer. > > > All that is wrong and not aligned to Emacs common interface. It is bug > > that bugs. Agenda buffer should allow users those standard Emacs > > features. > > I am wondering what is the common Emacs interface you refer to. I am not > aware about any standard way to prompt user while also showing detailed > description of what to expect from different choices. It is maybe not standard, but I never expected in last 20 years to get blocked when having some window as menu in front of me. Please look how mu4e is showing menu and compare: 1. when I open mu4e the menu does not block me to divide screen in 2 windows 2. org-agenda blocks me, it denies me using Emacs interface. This is personally disturbing and makes accessing Org agenda repetitive Observe how C-x C-f tries to find file: 1. it opens minibuffer and does not disturb user to move from minibuffer to other buffer to find references. I often have file names in other buffers and I move to minibuffer to open specific file 2. org-agenda does not allow any movement It is usability question. Personally I do not mind as I am transitioning and using Org files not anymore for planning, rather for documents. Org agenda window shall simply get a focus and be displayed in read-only Org mode with few key bindings invoking those commands. This way both the minibuffer and other windows remain accessible. I have tried to be nice when describing my experience with it. In very kind way I can say that I do not find it usable. Reference: https://www.nngroup.com/articles/usability-101-introduction-to-usability/ For me it was not easy for reason that org-agenda offers 16 different menues and that I cannot keep open org-agenda window and move to other window for references. It requires me to write notes on paper to be able to use org-agenda. When searching for things I often use other window, I do copy and paste into minibuffer, read info files or other buffers. Consider this a bug, and it is already here on the mailing list. Jean
Re: One vs many directories
Jean Louis writes: > That is great to hear. I wonder how it can be integrated with Emacs as > it is all Javascript based. The server is mostly python-based (see Language stats in https://github.com/hypothesis/h). They provide HTTP API: https://h.readthedocs.io/en/latest/api-reference/. There is even elisp package utilising that API: https://github.com/Kungsgeten/hypothesis/blob/master/hypothesis.el Best, Ihor
Re: One vs many directories
* Ihor Radchenko [2020-12-02 05:53]: > > Jean Louis writes: > > hypothes.is is free software that may be installed locally. > > > > https://github.com/hypothesis > > Thanks. You inspired me to try installing it locally again. > > The main problem with hypothesis is that it is designed to be deployed > to a server and not to a local machine. Thus, the instructions imply > some knowledge of server configuration and skip several important steps, > which were not obvious for me without spending quite a significant time > googling for various issues during and after the installation. > > Anyway, I finally managed to get it working with my local setup. Next > step is Emacs integration... That is great to hear. I wonder how it can be integrated with Emacs as it is all Javascript based. Here is the Org version of: Technology Template Project OHS Framework as Org file http://ix.io/2GdW I suggest when making tools for Emacs that developer of the tool looks into the template for open hyperdocument systems that system are developed that they benefit most people. - "Open" :: implies vendor-independent access to the hyperdocuments within and across work groups, platforms, and applications. If such system is developed with the database backing and generic hyperlinks then database is vendor-independent (mostly, but it is still PostgreSQL, but could be any other because it is SQL), then it can work for any kind of groups, it can be accessed from various platforms by using Emacs, but it can also be accessed by browser, but also through other programming languages would such program be re-written. For Org, to be open hyperdocument system it would mean to spread it onto other editors such as Vim where basic Org mode already exists and other editors, and applications, such as those on Android like Orgzly that supports Org. But all that does not make Org yet open especially in relation to hyperlinks. Hyperlinks should be dynamic not static and centrally managed not user managed to be work over various platforms, applications, for various groups. Using hyperlinks like: file:///something/pdf:page 1 Is not portable for many. Such hyperlink should be dynamic, which means if user accesses the database from remote host such could not access such hyperlink. It would need to convert to ssh:// or scp:// or ftp:// or https:// hyperlink or something else. These types of hyperlinks would be better: u...@hostname.com:PDF:Page:1 or u...@hostname.com:PDF:Query:Title or u...@hostname.com:PDF:Selection:1,2,3,4 but then again they are not enough generic. I am now making hyperlinks whihc are very generic, they can just designate the node number and server from the server list, could be something like: hyperscope:show:1:2 - to get the human designation of the hyperlink, what it is really. hyperscope:activate:1:2 - to go to server 1, node 2 and activate hyperlink. If I am on local file, Dired would open, but if user is accessing it remotely the hyperdocument would open in different way. That is what I mean with dynamic hyperlinks. hyperscope:email:1:2: - it would send it by email to the user. Because system identifies users with their usernames and other attributes hyperscope:email:1:2:u...@example.com - it would send hyperdocument to other user if there are enough permissions In that sense of having generic hyperlinks one should also make annotations. If I make annotation on Emacs than such should be translatable to let us say Hypothes.is but if not with that tool, hyperlink for annotation could open on Emacs with Xpdf or Evince reader on specific page number while showing annotation in Emacs. Systems should work equally well in any browser because of their generic nature. Hypothes.is is now working on browsers, which is for me very limiting. They should be liberated from browsers as well. That means that API should be there that allows access through any interface.
Re: One vs many directories
Jean Louis writes: > hypothes.is is free software that may be installed locally. > > https://github.com/hypothesis Thanks. You inspired me to try installing it locally again. The main problem with hypothesis is that it is designed to be deployed to a server and not to a local machine. Thus, the instructions imply some knowledge of server configuration and skip several important steps, which were not obvious for me without spending quite a significant time googling for various issues during and after the installation. Anyway, I finally managed to get it working with my local setup. Next step is Emacs integration... Best, Ihor
Re: One vs many directories
* Ihor Radchenko [2020-11-30 15:25]: > Jean Louis writes: > > You could record on some of free hostings that respect users' freedom > > that refrain of coercing non-free javascript such as: > > > > Open.tube upload > > https://open.tube/videos/upload > > Thanks for this reference. > > You also mentioned hypothes.is earlier. Do you know if there is any > similar software allowing to store data locally without a need to create > account and trust the service for storing all the personal notes? hypothes.is is free software that may be installed locally. https://github.com/hypothesis
Re: One vs many directories
Jean Louis writes: > You could record on some of free hostings that respect users' freedom > that refrain of coercing non-free javascript such as: > > Open.tube upload > https://open.tube/videos/upload Thanks for this reference. You also mentioned hypothes.is earlier. Do you know if there is any similar software allowing to store data locally without a need to create account and trust the service for storing all the personal notes? Best, Ihor
Re: One vs many directories
Jean Louis writes: > You could record on some of free hostings that respect users' freedom > that refrain of coercing non-free javascript such as: > > Open.tube upload > https://open.tube/videos/upload Thanks for this reference. You also mentioned hypothes.is earlier. Do you know if there is any similar software allowing to store data locally without a need to create account and trust the service for storing all the personal notes? Best, Ihor
Re: One vs many directories
* Texas Cyberthal [2020-11-30 10:36]: > Hi Jean, > > > For that I need video to understand. > > Agreed. I thought the Treefactor gif videos would be enough, but it's > clear that people's imagination cannot extrapolate the utility of > RIITR. > > I developed this skill long ago on the Windows app Brainstorm, and > have forgotten how rare and unintuitive it is. > > This is the key missing concept preventing people from understanding > and adopting Textmind, because Textmind is designed primarily to > exploit this concept. > > So I plan to play AI Dungeon with Treefactor, using RIITR to make > sense and fun of nonsense, and record it on YouTube with live audio > commentary. Monkey see, monkey do. Then people will be able to do > RIITR. You could record on some of free hostings that respect users' freedom that refrain of coercing non-free javascript such as: Open.tube upload https://open.tube/videos/upload
Re: One vs many directories
Ihor> That would be appreciated. I tried to read Treefactor docs at least 3 times and failed to understand its utility. Then I will move AI Dungeon Treefactor demo priority above all but critical Cyborganize documentation, such as broken links. Apparently I've wasted a lot of time documenting Textmind and 10 Bins before Treefactor was adequately explained. To avoid such mistakes in the future, I should focus on instilling the minimum viable behavioral change. Untangling a cognitive knot with an impromptu Treefactor or Brainstormsw session is a good candidate: https://www.brainstormsw.com And recreational use, such as for gaming, is a sticky alternative to undesirable premature production use.
Re: One vs many directories
Texas Cyberthal writes: > Agreed. I thought the Treefactor gif videos would be enough, but it's > clear that people's imagination cannot extrapolate the utility of > RIITR. That would be appreciated. I tried to read Treefactor docs at least 3 times and failed to understand its utility. Best, Ihor
Re: One vs many directories
Hi Jean, > For that I need video to understand. Agreed. I thought the Treefactor gif videos would be enough, but it's clear that people's imagination cannot extrapolate the utility of RIITR. I developed this skill long ago on the Windows app Brainstorm, and have forgotten how rare and unintuitive it is. This is the key missing concept preventing people from understanding and adopting Textmind, because Textmind is designed primarily to exploit this concept. So I plan to play AI Dungeon with Treefactor, using RIITR to make sense and fun of nonsense, and record it on YouTube with live audio commentary. Monkey see, monkey do. Then people will be able to do RIITR.
Re: One vs many directories
* Ihor Radchenko [2020-11-29 10:52]: > Jean Louis writes: > > > When task is assigned to somebody notification of a deadline or alerts > > are definitely useful for me. But they are are useful for those > > conducting the tasks. > > > > Thus integration to remind those *related* people to the assigned > > tasks and their deadlines or schedules would be useful. > > I think that can be done by extending alert.el package (which org-alert > is based on). alert.el allows the user to define custom alert styles, > including sending sms or email. Yes, that is great. We need more integration. There are low level packages but we need integration, high level, user accessible and for users useful features. Emacs is itself useful but there may be long process until user starts doing some programming. Then it becomes trivial to alert self or othre people by SMS, yet majority of users will not have that functionality unless we make it somehow. org-set-reminder by SMS could send it to assigned person. This is for me so important that it deserves menu item where user can assign that reminder. Once assigned it would run in intervals until task has been closed. org-send-task-by-email would find out which user is assigned to (feature is not built-in) and send it to assigned user automatically. It is trivial, but we do not have that. One click and it should go. > >> I agree that org-agenda has many issues that cannot be easily solved > >> because of its complexity. However, everything you describe (including > >> multi-occur) can also be achieved with org-ql > >> (https://github.com/alphapapa/org-ql) - analogue of SQL query language > >> for org-mode (with more optimisations in comparison with org-agenda). > > > > Definitely good only too much low-level. Users need finished functions > > that integrate stuff. To adopt myself to org-ql would mean to read > > documentations, meanings, and starting with some functions, while this > > may be possible for me that approach is not user friendly as users > > need integration and accessible interfaces. > > Not really. Searching in org-ql is made very intuitive - just like you > would search in google. There are multiple interactive commands: > - helm-org-ql :: The user simply types search keywords and the commands > shows all the matching headlines. For example, user can type > "headline:Doe call todo:WAIT tags:invoice" and get all kinds of > matches as he/she types > - org-ql-view :: shows the menu for different pre-customised searches > like "Calendar: This week" or "Overview: NEXT tasks" That is game changer. > > > Example of lack of integration would be to tell to user to simply > > construct the link in the form: [[Here][Link]] and that alone would > > require some thinking and learning. > > There is also (slower) helm-org (https://github.com/emacs-helm/helm-org) > offering similar interface to helm-org-ql, but also adding an option to > do various actions with the selected tasks, like inserting correctly > formatted link. The same can be done in helm-org-ql. I think the author > plans to extend that command in the near future. Or one can add extra > actions as needed - it is trivial to do in helm. > > > Good integration for org-ql would be a wizard that can add to the list > > of various queries and offers users various ways to search such as > > searching by headline, tags, or having both tags involved, date, > > deadlines and so on. > > All these are already available. The user can search tags:tag1,tag2,..., > headline:word1,word2 (or shortly hl:word1,...), scheduled:on=today, > deadline:from=-7,to=2020-12-01 org-ql is indeed a wizard for searching agenda and looks better than default. Jean
Re: One vs many directories
Jean Louis writes: > When task is assigned to somebody notification of a deadline or alerts > are definitely useful for me. But they are are useful for those > conducting the tasks. > > Thus integration to remind those *related* people to the assigned > tasks and their deadlines or schedules would be useful. I think that can be done by extending alert.el package (which org-alert is based on). alert.el allows the user to define custom alert styles, including sending sms or email. >> I agree that org-agenda has many issues that cannot be easily solved >> because of its complexity. However, everything you describe (including >> multi-occur) can also be achieved with org-ql >> (https://github.com/alphapapa/org-ql) - analogue of SQL query language >> for org-mode (with more optimisations in comparison with org-agenda). > > Definitely good only too much low-level. Users need finished functions > that integrate stuff. To adopt myself to org-ql would mean to read > documentations, meanings, and starting with some functions, while this > may be possible for me that approach is not user friendly as users > need integration and accessible interfaces. Not really. Searching in org-ql is made very intuitive - just like you would search in google. There are multiple interactive commands: - helm-org-ql :: The user simply types search keywords and the commands shows all the matching headlines. For example, user can type "headline:Doe call todo:WAIT tags:invoice" and get all kinds of matches as he/she types - org-ql-view :: shows the menu for different pre-customised searches like "Calendar: This week" or "Overview: NEXT tasks" > Example of lack of integration would be to tell to user to simply > construct the link in the form: [[Here][Link]] and that alone would > require some thinking and learning. There is also (slower) helm-org (https://github.com/emacs-helm/helm-org) offering similar interface to helm-org-ql, but also adding an option to do various actions with the selected tasks, like inserting correctly formatted link. The same can be done in helm-org-ql. I think the author plans to extend that command in the near future. Or one can add extra actions as needed - it is trivial to do in helm. > Good integration for org-ql would be a wizard that can add to the list > of various queries and offers users various ways to search such as > searching by headline, tags, or having both tags involved, date, > deadlines and so on. All these are already available. The user can search tags:tag1,tag2,..., headline:word1,word2 (or shortly hl:word1,...), scheduled:on=today, deadline:from=-7,to=2020-12-01 Best, Ihor
Re: One vs many directories
* Texas Cyberthal [2020-11-29 09:20]: > Images should have a creation date in metadata and also some tags, I > agree. Whether to organize them in a Binmind using 10 Bins is a > matter of taste. I prefer to keep them in 10 Bins until their tag > nomenclature is mature. Then, when worthwhile, I would move them to a > dedicated image management app, and add a good set of tags and > metadata to ensure they won't be lost later. > > If the app allows it, I might still organize the pics into a tree, > just to make it easier to navigate the whole collection without > repetition, and understand its primary meaning quickly. This extra > work is profitable only for high-value images. The existence of the > tree permits continued nomenclature evolution without risking some > images getting lost due to tag drift and rot. That gives me idea for Emacs similar to org-noter that one could have bunch of pictures being displayed then just choosed few tags by using mouse and window get replaced with new picture. Process could be used for faster tagging. > Treemind is good at nomenclature evolution via Rapid Inductive > Iterative Tree Refiling. For that I need video to understand.
Re: One vs many directories
Hi Jean, > After a while user will get a subset of highly ranked headings in their > corresponding Org files. That subset then can be used as quick bookmarks or > get bound to keys. This is a higher tier of PIM than Textmind. Textmind is for processing thoughts. For example, I use it to convert these email exchanges into useful notes and documentation. Crystallized knowledge that needs to be optimized for rapid or lateral access belongs in a PKM or CRM app. There are many options, and which is best likely depends on the specific use case. (Assuming it doesn't belong in Postgres or Pubmind.) > From other people's experiences I can see they are thinking different. It is > questionable if there is one algorithm corresponding to many people's natural > thinking. Almost nobody has a holistic natural thought algorithm. An algorithm has a strict definition, whereas biological thought is evolved and messy. > But I cannot access image related to business ABC that is located in 2004 > quickly unless such image is indexed somewhere by its meaning, maybe "boat > purchased" would be meaning related to such picture. Then if I wish to see > boats I have purchased I would type most probably "boat" and from there find > various other attributes such as dates, types and similar. Images should have a creation date in metadata and also some tags, I agree. Whether to organize them in a Binmind using 10 Bins is a matter of taste. I prefer to keep them in 10 Bins until their tag nomenclature is mature. Then, when worthwhile, I would move them to a dedicated image management app, and add a good set of tags and metadata to ensure they won't be lost later. If the app allows it, I might still organize the pics into a tree, just to make it easier to navigate the whole collection without repetition, and understand its primary meaning quickly. This extra work is profitable only for high-value images. The existence of the tree permits continued nomenclature evolution without risking some images getting lost due to tag drift and rot. Treemind is good at nomenclature evolution via Rapid Inductive Iterative Tree Refiling.
Re: One vs many directories
> Sent: Saturday, November 28, 2020 at 5:16 PM > From: "Jean Louis" > To: "Ihor Radchenko" > Cc: "Dr. Arne Babenhauserheide" , "Texas Cyberthal" > , "emacs-orgmode@gnu.org" > Subject: Re: One vs many directories > > * Ihor Radchenko [2020-11-24 10:57]: > > > I find it entertaining for now. Now, what is exomind? > > > > Unless I misunderstood, Jean referred to "external brain" concept: > > - https://beepb00p.xyz/exobrain/ > > The more you send me reference more I discover other set of people > doing same what I am doing. Since I have implemented central meta > level organization it is moving rapidly, everthing gets sorted. It > develops by itself and is rapidly accessible. Believing that only you think a certain way is a big mistake. > That website I have to mirror locally to pick ideas and learn from > others. Mirroring I do with: > > $ wget -Emk http://example.com > > As that command replaces all hyperlinks to local hyperlinks. That > person advanced in organization of things. I stick to few principles > and just design it by principles. > > Design works rapidly. Few Emacs Lisp functions and access to reports > listed in Emacs Buffers and integration with other tools. > > With one function and one PostgreSQL table defined in 3 minutes I get > rudimentary backup and version system for any column values that I am > editing in the database. If I edit note, the note is versioned > (previous version stored) before I start editing it. Principles I am > following are basics what programmers like, to minimize or eliminate > repetitions and efforts to achieve the goal. > > Person above have extracted or exported its own database of hyperlinks > to hyperdocuments. My side I have made for now Org export of any > subtree or the whole dynamic knowledge repository. There are many > things to go. In Emacs development version all kinds of hyperlinks can > get their handlers like gopher:// gemini:// message: tel: sms: and > htat will be very helpful. > > No, I do not use "exobrain" as a term. I rather lean on Engelbart's > terminology and follow his principles as we are very late to implement > what was envisioned back in 1968 and before. It is 52 years already. > > And many more years since Memex has been invented: > > Memex > https://en.wikipedia.org/wiki/Memex > > As author said: "The memex device as described by Bush "would use > microfilm storage, dry photography, and analog computing to give > postwar scholars access to a huge, indexed repository of knowledge any > section of which could be called up with a few keystrokes." > > And that is exactly what I am creating here to have anything called up > with few keystrokes and to be able to share files with individuals or > groups of people without more thinking but just designated what to be > done. > > Have group of 5 people to share notes with? Just find the designated > group and click share. Computer would handle the rest, maybe send > files by emails individually, maybe inform people by SMS, maybe upload > files and share password protected hyperlinks with those people. > > Integration is another keyword I like to follow. Android principle of > sharing is pretty much based on integration. We have all the small > functions around us only not well integrated with their relations that > concern human problems. > > We have files on file system which we cannot easily share with groups > or people we want. Address books are all sparse, one is in this email > client, one is separate, one is on the mobile device, another email > client does not synchronize, and so on. I have forgotten this long > ago and use central address book from where everything derive: > > - no Google, clouds, etc. that is very insecure. Do not give contacts > to Google, there are hundreds of thousands of staff members there > and no guarantee whatsoever that they will not read it. > > - keeping contacts on my computers. I have already spent money for > hard disk, there is enough space > > - exporting contacts from central database and importing to email > clients, mobile devices, this way everything is synchronized. > > How quickly can GNU/Linux user share a file with somebody? > > - locate the file by using hierarchical browsing. If file system is a > mess, this alone may take some time > > - open up email reader > > - find that email address. If it is in the email reader already it is > good. But it could be in the phone. It could be on paper, or on > business card. Where is it? Maybe calling person? But where is the > phone number? On first phone, second phone... if all is
Re: One vs many directories
* Ihor Radchenko [2020-11-24 10:57]: > > I find it entertaining for now. Now, what is exomind? > > Unless I misunderstood, Jean referred to "external brain" concept: > - https://beepb00p.xyz/exobrain/ The more you send me reference more I discover other set of people doing same what I am doing. Since I have implemented central meta level organization it is moving rapidly, everthing gets sorted. It develops by itself and is rapidly accessible. That website I have to mirror locally to pick ideas and learn from others. Mirroring I do with: $ wget -Emk http://example.com As that command replaces all hyperlinks to local hyperlinks. That person advanced in organization of things. I stick to few principles and just design it by principles. Design works rapidly. Few Emacs Lisp functions and access to reports listed in Emacs Buffers and integration with other tools. With one function and one PostgreSQL table defined in 3 minutes I get rudimentary backup and version system for any column values that I am editing in the database. If I edit note, the note is versioned (previous version stored) before I start editing it. Principles I am following are basics what programmers like, to minimize or eliminate repetitions and efforts to achieve the goal. Person above have extracted or exported its own database of hyperlinks to hyperdocuments. My side I have made for now Org export of any subtree or the whole dynamic knowledge repository. There are many things to go. In Emacs development version all kinds of hyperlinks can get their handlers like gopher:// gemini:// message: tel: sms: and htat will be very helpful. No, I do not use "exobrain" as a term. I rather lean on Engelbart's terminology and follow his principles as we are very late to implement what was envisioned back in 1968 and before. It is 52 years already. And many more years since Memex has been invented: Memex https://en.wikipedia.org/wiki/Memex As author said: "The memex device as described by Bush "would use microfilm storage, dry photography, and analog computing to give postwar scholars access to a huge, indexed repository of knowledge any section of which could be called up with a few keystrokes." And that is exactly what I am creating here to have anything called up with few keystrokes and to be able to share files with individuals or groups of people without more thinking but just designated what to be done. Have group of 5 people to share notes with? Just find the designated group and click share. Computer would handle the rest, maybe send files by emails individually, maybe inform people by SMS, maybe upload files and share password protected hyperlinks with those people. Integration is another keyword I like to follow. Android principle of sharing is pretty much based on integration. We have all the small functions around us only not well integrated with their relations that concern human problems. We have files on file system which we cannot easily share with groups or people we want. Address books are all sparse, one is in this email client, one is separate, one is on the mobile device, another email client does not synchronize, and so on. I have forgotten this long ago and use central address book from where everything derive: - no Google, clouds, etc. that is very insecure. Do not give contacts to Google, there are hundreds of thousands of staff members there and no guarantee whatsoever that they will not read it. - keeping contacts on my computers. I have already spent money for hard disk, there is enough space - exporting contacts from central database and importing to email clients, mobile devices, this way everything is synchronized. How quickly can GNU/Linux user share a file with somebody? - locate the file by using hierarchical browsing. If file system is a mess, this alone may take some time - open up email reader - find that email address. If it is in the email reader already it is good. But it could be in the phone. It could be on paper, or on business card. Where is it? Maybe calling person? But where is the phone number? On first phone, second phone... if all is synchronized maybe is easier to find. - attach the file - send the file. But then sending SMS or calling in the same time does not work. The above process is not well integrated. It could work like this: - user just thinks of what has to be shared with other person, types the terms related to the thought - locates the file and press share - locates the user and press enter. FINISHED That would be better integration. Even better it would be if user can choose the automated workflow option: 1. send the file, automatically record that file has been sent to specific user. Tell user automatically how many files are attached and attach annotation belonging to the file as body of the email or any instructions. 2. in the same time inform the user by SMS that file has been sent and record that SMS have been sent.
Re: One vs many directories
* Texas Cyberthal [2020-11-28 11:20]: > Hi Jean, > > > What should it be or do? > > Dbmind does things that Postgres handles better than Org. > > > As you have specific thought order in directory names then maybe > > such could be parsed, maybe slashes / removed to show a full path > > to the file. This becomes long but could be useful in some lists. > I don't intend to do so. Textmind maximizes path dynamism via > Dired+Treefactor. Links shouldn't break that. I enjoy these thoughts. Maybe you refer to browsing the tree. I am switching completely from browsing the tree into index type of access. Index helps to find the thought which in turn finds candidates and file is in front of me. How I think now is little different than past and different than future. If I think of term "Insurance" related to Germany those are two words to search for. Maybe in future I think of "health" then I should be able to find "insurance". There is no fixed pattern on how to think. Repetition in locating the node creates habits that next search is even quicker found. Those most searched nodes could be ranked automatically for even quicker access. And all searches for nodes could be recorded for future as inexpensive and automated log that helps person find what happened in the history and which nodes have been located at certain months, dates, years. org-store-link would get its hook or additional function that each time when Org mode related heading is is stored that it ranks the file the heading in a separate list which can be stored on file system. After a while user will get a subset of highly ranked headings in their corresponding Org files. That subset then can be used as quick bookmarks or get bound to keys. > > Alright and I find that it is the case on my side, and previous work of > > Engelbart, then also within some other information management systems, like > > Semantic Synchrony. > > Some of that might qualify as an algorithm, but not a natural thought > algorithm. A natural thought algorithm must manage substantially all > natural thoughts while satisfying the definition of an algorithm. I wish I could understand definition of "natural though algorithm" in the context how you refer it to. >From other people's experiences I can see they are thinking different. It is questionable if there is one algorithm corresponding to many people's natural thinking. The algorithm in locating specific file on my sides is programming algorithm based on what I find most quick for me personally. There are several such algorithms programmed that help me locate things. I have 19489 nodes, everything is together. There is no visible delay for completion to show me list of nodes. The list of everything is offered for completion and I can type what I think that I need. If I search for PDF related to "mercury" I will locate it as quick as 1-2 seconds and can open it already. Or I could search by word, tag, attribute and get a list of related hyperlinks and hyperdocuments. From there on I can locate the right one. > The things you mentioned are not even as sophisticated and complete as > GTD. And GTD is merely a personal paper-management algorithm, not a > natural thought algorithm. I did look on the hyperlink you gave me. It looks like algorithm deciding what to do with tasks for me. But that is not how we manage tasks in the group. We have processes that are defined from A to Z and tasks are managed with to achieve the overall purpose. A purpose could be project accomplishment such as "bridge built over the water". Then tasks are steps bringing the project closer to accomplishment, such as purchase timber, saw, meter tape, screws. Project planner is the key here as one should not put nothing more and nothing less, and none task shall be defined that is not doable. No algorithm should influence tasks to be pending, or archive them or decide to do task if it is very short as relations to task and from the task to/from other objects are not so simple. There can be 10 tasks each 2 minutes long that cannot be conducted right now as the phone call and subsequent travel may have so much more important. I was referring to this: > After reading appendix IV to Making it all work (another book about > GTD by David Allen) it occured to me that the Process phase of > "Mastering Workflow" would be well-rendered using psuedocode. This > is probably due to the fact that there are several decisions to make > along the way (logic) and this happens for each item in your inbox > (loop). So, without further introduction here's the code: > while length(inbox) > 0: > inbox_item = inbox.pop() > thing = analyze(inbox_item) > if not actionable(thing): > toss(thing) or tickle(thing) or file(thing) > else if less_than_two_minutes(thing): > do(thing) > else if is_single_task(thing): > wait_for(delegate(thing)) or assign_context(thing) > else: >
Re: One vs many directories
Hi Jean, > What should it be or do? Dbmind does things that Postgres handles better than Org. > As you have specific thought order in directory names then maybe such could > be parsed, maybe slashes / removed to show a full path to the file. This > becomes long but could be useful in some lists. I don't intend to do so. Textmind maximizes path dynamism via Dired+Treefactor. Links shouldn't break that. > Alright and I find that it is the case on my side, and previous work of > Engelbart, then also within some other information management systems, like > Semantic Synchrony. Some of that might qualify as an algorithm, but not a natural thought algorithm. A natural thought algorithm must manage substantially all natural thoughts while satisfying the definition of an algorithm. The things you mentioned are not even as sophisticated and complete as GTD. And GTD is merely a personal paper-management algorithm, not a natural thought algorithm. (Textmind manages text only, whereas some people think visually. It would be easy to adapt Textmind principles to Binmind, if needed. Therefore even for visual thinkers, Cyborganize is a natural thought algorithm.) I doubt there are multiple ways to design a natural thought algorithm. For example, all natural thoughts occur in a chronological sequence. This necessitates a ramblog to accurately reflect them. This is the GTD inbox algorithm: https://michaelwhatcott.com/gtd-workflow-processing-algorithm/ GTD is usually called a method. I've started calling Textmind an algorithm to emphasize the finiteness aspect of its design, a key feature. I think I could construct a pseudocode Textmind algorithm. It would of course rely on human judgment for some decisions, and judgment is fuzzy. But the algorithm itself would be unambiguous.
Re: One vs many directories
* Texas Cyberthal [2020-11-27 12:01]: > Hi Jean, > > > does using the 10 Bins and Textmind system gives you personal > > satisfaction of being well organized? > > For what it does, yes, amazingly so. Thank you. I was expecting something like that as we are in similar position of having somewhat personally better or could we say idiosyncratically better organizing system for ourselves. > I still need Dbmind, which I haven't developed yet. What should it be or do? > > did you develop having functions similar to store link that > > quickly obtain the hyperlink in memory to be easier inserted in > > Org files? That is similar to org-capture. I think every system of > > organization and storing objects into X should have automated > > quick hyperlink generation. > I find Emacs Org's native facilities adequate. However, I did a bit > of streamlining: > > > C-c l runs the command treefactor-org-store-link-fold-drawer > > (found in global-map), which is in ‘treefactor.el’. As you have specific thought order in directory names then maybe such could be parsed, maybe slashes / removed to show a full path to the file. This becomes long but could be useful in some lists. > > how does 10 Bins and Textmind enhance what you do with Org files? > > It mind-syncs a natural language thought algorithm, which would > otherwise be impossible. > > https://www.geeksforgeeks.org/introduction-to-algorithms/ > > It is clear and unambiguous, has well-defined inputs and outputs, and > is finite and feasible. Alright and I find that it is the case on my side, and previous work of Engelbart, then also within some other information management systems, like Semantic Synchrony. Would you consider that the system I am using would be or could be said to have natural thought algorithm by considering following: That there are many nodes, they have their names, some properties, and tags and other meta data and searchable lines could look like this as one variety of accessing methods: People / Jean / Computer / Free Software / GNU Emacs / Real-time incremental narrowing completion / Helm now imagine 20,000 or 100,000 such lines but not visible to user, just few would be visible as incremental narrowing search such as helm or ivy or filtering functions could quickly locate particular lines. If I think of "GNU" I would get maybe subset of lines related to that thought, if I think of Emacs, I would get references related to GNU and Emacs. If I think of "completion" I would get references to ivy, selectrym, helm, and so on and by watching the dynamical filter in front of me I get new references displayed in real time which I can then add to the query until I find the matching few or matching tree or matching node. Sounds to me it is also one type of natural thought algorithm. > Unlike Getting Things Done by David Allen, it captures the whole > thought-stream, or at least everything worth typing. I have no idea and by reading basic descriptions I do not find myself there. It seems that systems are pretty much idiosyncratic unless training and well written instructions help in making it applicable for a group. There is one "algorithm" that I am using so that every task CAN BE DONE, and that is that tactical plans are divided into projects only when one step of the plan is too complex to be executed without dividing it or chunking it down. Projects fullfil one step of a larger plan and consists of steps. When one step cannot be fullfiled by its own, it probably means it was not written well chunked, so that step is chunked into tasks or orders. Principle of the algorithm would be to never write tasks that are too complex to be executed and finalized and to have each task in itself doabl so that each step becomes simple step of one bigger complex action. We do projects in real life such as purchasing, construction of machinery, negotiations, and so on, and projects are written on the paper and executed and again reused and executed on other places. By following the above principle our staff members in various countries could get things done just by reading instructions as each step has been simplified down to the atomic doable task.
Re: One vs many directories
Hi Jean, > does using the 10 Bins and Textmind system gives you personal satisfaction of > being well organized? For what it does, yes, amazingly so. I still need Dbmind, which I haven't developed yet. > did you develop having functions similar to store link that quickly obtain > the hyperlink in memory to be easier inserted in Org files? That is similar > to org-capture. I think every system of organization and storing objects into > X should have automated quick hyperlink generation. I find Emacs Org's native facilities adequate. However, I did a bit of streamlining: > C-c l runs the command treefactor-org-store-link-fold-drawer (found in > global-map), which is in ‘treefactor.el’. > how does 10 Bins and Textmind enhance what you do with Org files? It mind-syncs a natural language thought algorithm, which would otherwise be impossible. https://www.geeksforgeeks.org/introduction-to-algorithms/ It is clear and unambiguous, has well-defined inputs and outputs, and is finite and feasible. Unlike Getting Things Done by David Allen, it captures the whole thought-stream, or at least everything worth typing. Man is used to thinking alone, with no internal error-checker. Sloppy conclusions abound since biological memory is ephemeral. Textmind creates a team of past and future selves to collaborate asynchronously. It's quite steadying.
Re: One vs many directories
Hi Jean, Yes, Textmind is a rock tumbler for natural language thoughts. An SME CRM treats people like widgets. The former does many small thoughtful touches, the latter does few robotic touches. Excessive widget volume chokes Textmind. Sure, I will subscribe you when I have a mailing list. For now, readers can only follow the Github repos.
Re: One vs many directories
* Texas Cyberthal [2020-11-26 19:05]: > Hi Jean, > > Yes, Textmind is a rock tumbler for natural language thoughts. An SME > CRM treats people like widgets. The former does many small thoughtful > touches, the latter does few robotic touches. Excessive widget volume > chokes Textmind. > > Sure, I will subscribe you when I have a mailing list. For now, > readers can only follow the Github repos. Alright. I can see there are few people who have got a cognition in organizing things not being only Org notes but also files, and I feel you are one of them. I would like to ask few questions: - does using the 10 Bins and Textmind system gives you personal satisfaction of being well organized? - did you develop having functions similar to store link that quickly obtain the hyperlink in memory to be easier inserted in Org files? That is similar to org-capture. I think every system of organization and storing objects into X should have automated quick hyperlink generation. - how does 10 Bins and Textmind enhance what you do with Org files? Jean
Re: One vs many directories
> Sorry for that vague expression. Let us say I open Completions buffer > I can switch into it, inspect it, ask for defined keys, evaluate with > M-:, Emacs allows me to remain in the window and go to other > window. Agenda buffer does not do that, this is probably because it > just waits for any key and does not allow anything else. That means I > will open Agenda and I cannot switch to other window, so I will close > agenda to switch. Maybe I have 10 TODO keywords, I have to open file, > I open Agenda again, aha, T... then close agenda, open file see > keyword, then open agenda again. Repetitions without end and user is > unable to use multiple windows. This really need not be so. It appears to me (correct me if I am wrong) that you haven't tried agenda multi-occur/keyword/etc searches. The way it works is that you only need to select type of search from agenda dispatcher window (the one you are criticising). Later, when you actually enter the search string, you are left with an ordinary Emacs prompt. You are free to leave the minibuffer and switch to other buffers searching for the keywords you are interested in. In general, there are two types of agenda searches available from agenda dispatcher: 1. Most are free-hand searches where you need to type a search string: - match TAGS/PROP/TODO query - search for keywords - multi-occur - first two searches, but limited only to TODO tasks 2. Pre-defined searches where search string was set in advance (by developers or by user via org-agenda-custom-commands): - agenda listing tasks relevant to current week or day - all TODO entries - FLAGGED entries - stuck projects The first type will let you leave search prompt as soon as you select what type of search you are about to enter. The second type does not really need entering any extra text - it is predefined. > You know how agenda is made like the 1985 BASIC menu in Commodore > C=64, but even back then there was better interface for such menus. FYI. Transient.el - menu dispatcher in popular magit package also does not let you switch buffers. So, apparently many people would not agree that it is so terrible design. P.S. Nothing prevents you from calling, i.e. M-x org-occur or binding it to a key of your choice. Best, Ihor Jean Louis writes: > * Ihor Radchenko [2020-11-25 14:48]: >> > When I do C-c a it runs (org-agenda) but I do not have "g" and I am on >> > development version. The C-c a window is made so that I cannot go with >> > cursor inside and that I cannot even expect the key map neither invoke >> > command by M-x and I cannot even M-: >> >> C-c a will first show so-called agenda dispatcher asking you what kind >> of agenda view you want to get. You need to press a key according to >> the popup window (i.e. `t' to see all not done items). Then, you will >> get the proper agenda buffer with all the keymaps set and `g' bound to >> refreshing the chosen agenda view in the buffer. >> >> > All that is wrong and not aligned to Emacs common interface. It is bug >> > that bugs. Agenda buffer should allow users those standard Emacs >> > features. >> >> I am wondering what is the common Emacs interface you refer to. I am not >> aware about any standard way to prompt user while also showing detailed >> description of what to expect from different choices. > > Sorry for that vague expression. Let us say I open Completions buffer > I can switch into it, inspect it, ask for defined keys, evaluate with > M-:, Emacs allows me to remain in the window and go to other > window. Agenda buffer does not do that, this is probably because it > just waits for any key and does not allow anything else. That means I > will open Agenda and I cannot switch to other window, so I will close > agenda to switch. Maybe I have 10 TODO keywords, I have to open file, > I open Agenda again, aha, T... then close agenda, open file see > keyword, then open agenda again. Repetitions without end and user is > unable to use multiple windows. This really need not be so. > > If I open packages' list I can keep the buffer and move to other > buffer while looking into that list. > > You know how agenda is made like the 1985 BASIC menu in Commodore > C=64, but even back then there was better interface for such menus.
Re: One vs many directories
* Ihor Radchenko [2020-11-25 14:48]: > > When I do C-c a it runs (org-agenda) but I do not have "g" and I am on > > development version. The C-c a window is made so that I cannot go with > > cursor inside and that I cannot even expect the key map neither invoke > > command by M-x and I cannot even M-: > > C-c a will first show so-called agenda dispatcher asking you what kind > of agenda view you want to get. You need to press a key according to > the popup window (i.e. `t' to see all not done items). Then, you will > get the proper agenda buffer with all the keymaps set and `g' bound to > refreshing the chosen agenda view in the buffer. > > > All that is wrong and not aligned to Emacs common interface. It is bug > > that bugs. Agenda buffer should allow users those standard Emacs > > features. > > I am wondering what is the common Emacs interface you refer to. I am not > aware about any standard way to prompt user while also showing detailed > description of what to expect from different choices. Sorry for that vague expression. Let us say I open Completions buffer I can switch into it, inspect it, ask for defined keys, evaluate with M-:, Emacs allows me to remain in the window and go to other window. Agenda buffer does not do that, this is probably because it just waits for any key and does not allow anything else. That means I will open Agenda and I cannot switch to other window, so I will close agenda to switch. Maybe I have 10 TODO keywords, I have to open file, I open Agenda again, aha, T... then close agenda, open file see keyword, then open agenda again. Repetitions without end and user is unable to use multiple windows. This really need not be so. If I open packages' list I can keep the buffer and move to other buffer while looking into that list. You know how agenda is made like the 1985 BASIC menu in Commodore C=64, but even back then there was better interface for such menus.
Re: One vs many directories
* Ihor Radchenko [2020-11-26 06:34]:
> > Can I automated the execution of Babel code upon opening of the Org
> > file?
>
> Adding to other suggestions, you can always add a custom function to
> org-mode-hook instead of playing with file-local variables.
>
> > Then we comes to actual execution of tasks. How do we get
> > reminded?
Yes, that requires customization, file local variables customize
things without user's customization. In the end my use for that slowly
disappears as I am transitioning all tasks to HyperScope. Yesterday I
made simple hard-coded function here that I invoke inside of an Org
heading that captures the heading, date created, and its text and
stores it as a note in the database.
(defun hyperscope-capture-org-heading ()
"Captures Org heading and stores it in the Hyperscope dynamic
knowledge repository"
(interactive)
(let* ((body (org-copy-subtree nil nil nil t))
(body (pop kill-ring))
(body (org-no-properties body))
(heading (org-get-heading))
(created (org-property-values "CREATED"))
(date (if created (substring (car created) 1 11) nil))
(body (with-temp-buffer
(insert body)
(org-mode)
(org-back-to-heading)
(kill-line)
(delete-matching-lines ":PROPERTIES:")
(delete-matching-lines ":CREATED:")
(delete-matching-lines ":ID:")
(delete-matching-lines ":END:")
(buffer-string
(hyperscope-add-note-to-set heading body date)))
The other use I had for users are tables as I have been writing
tables so much.
But then I have to write that Joe Doe have paid money to Jane Dane in
the file of Jane Dane as only so I know how much money Jane Dane keeps
on behalf of business. And then I have to write by hand the same
transaction in the file of Joe Doe, as now he has less money. Tracking
it by head is definitely after some time error prone activity.
Database already has few tables for accounts and businesses, so I can
use database backed accounting which is already implemented as journal
package.
Then if I am doing transaction from Joe Doe I would select Joe Doe
petty cash acccount transferring money to Jane Dane's petty cash
account. I would type less and be less worried about errors. Entries
are stored in the database. If account for Joe Doe is related to Joe
Doe (relations has to be assigned, not just named by person) then I
can invoke the creation of Org table for the Joe Doe's profile.
Majority of times I am creating Org file on the fly for the person
that contains:
* Tasks
* Transactions
Now I will be pushing tasks into meta level and edit them from there
and when Org file is opened that relates to one person then Tasks can
automatically appear in the list ordered by their status or other
attribute.
Transactions I will most probably slowly transition into the database
and they can also automatically appear in the Org table under
Transactions so to be sent easier by using nice Org exports.
While this approach may not be common, it offers me more free time and
less worries.
> > Is the reminder only if I press {C-c a} for org-agenda? Do I need to
> > do action to get reminded?
>
> You can always configure Emacs to run agenda on startup. Just add a
> command to your init file ;)
Thank you.
> For automatic reminders, there is built-in org-notify.el or external
> org-alert package (https://github.com/spegoraro/org-alert).
That is definitely good idea. Just little limited as it relates to
user on computer and not to users to which tasks are assigned to.
Think of that as the next and long forgotten enhancement for Org.
I have this property in Org header:
#+PROPERTY: ASSIGNED_ALL Ezekiel Jean James Victor Dejan TeamTZ
TeamTZ is property of several people, when task is assigned to TeamTZ,
such task get sent to all people involved.
When I am assigning tasks the above list is using one of them to be
assigned.
When task is assigned to somebody notification of a deadline or alerts
are definitely useful for me. But they are are useful for those
conducting the tasks.
Thus integration to remind those *related* people to the assigned
tasks and their deadlines or schedules would be useful.
- org-send-task-by-email (shows some properties, schedules, ID)
- org-send-task-by-email-ascii (not same display in email)
- org-send-task-by-sms (using KDE connect, Termux, gnokii, Twilio, etc)
- org-send-task-by-fax (using email2fax providers or built-in fax modem)
Then think of reminders, they would need to run maybe in Emacs, but
maybe in Emacs that runs from cron job periodically, whatever is more
reliable.
I also think that generic reminding database or one `reminders' table
would be usable on every system so that users may add to it anything,
be it task, birthday, anniversary, deadlines, etc. Such generic
database would be run as system service and remind not only user
Re: Local variables insecurities - Re: One vs many directories
* Detlef Steuer [2020-11-26 14:46]: > Am Thu, 26 Nov 2020 08:31:29 +0300 > schrieb Jean Louis : > > > That is not fair choice. It pushes user to finally ! apply and accept > > it, but does not give chance to permanently ignore it. > > > > > > Do you want to apply it? You can type > > y -- to apply the local variables list. > > n -- to ignore the local variables list. > > ! -- to apply the local variables list, and permanently mark these > > values (*) as safe (in the future, they will be set > > automatically.) > > Well, if you are arguing for a user who chose to use emacs, configured > mutt to invoke emacs etc I do not. Any file opened any how by Emacs is invoking those variables. Please send your opinions to bug: https://debbugs.gnu.org/cgi/bugreport.cgi?bug=44837 by writing to: 44...@debbugs.gnu.org > .but at the same time has no idea he chose a > mighty tool, as in has no clue about programming, variables etc., That is right, I have given references. Do not expect users who obtain Emacs editor to be programmers. When user is asked anything, it should be self-documenting and it should have references to that documentation. That is why majority of actions with Emacs do have ? for help at least. Invoking unsafe variables does not even involve any help or references to manual. > then > this choice won't help neither. If you have no idea about local > variables, you can not make an informed decision about *anything* > involving the concept of local variables. Besides stopping and reading > the manual. That is right, but that will be picked by those better positioned few.
Re: Local variables insecurities - Re: One vs many directories
Am Thu, 26 Nov 2020 08:31:29 +0300 schrieb Jean Louis : > That is not fair choice. It pushes user to finally ! apply and accept > it, but does not give chance to permanently ignore it. > > > Do you want to apply it? You can type > y -- to apply the local variables list. > n -- to ignore the local variables list. > ! -- to apply the local variables list, and permanently mark these > values (*) as safe (in the future, they will be set > automatically.) Well, if you are arguing for a user who chose to use emacs, configured mutt to invoke emacs etc, but at the same time has no idea he chose a mighty tool, as in has no clue about programming, variables etc., then this choice won't help neither. If you have no idea about local variables, you can not make an informed decision about *anything* involving the concept of local variables. Besides stopping and reading the manual. May be I miss something, but then I miss it :-) Furthermore this discussion has imho long left the main topic of this list. Detlef
Re: Local variables insecurities - Re: One vs many directories
* Tom Gillespie [2020-11-26 09:19]: > > As there is the option ! to "apply local variables and permanently > > mark these values" but there is no option "not to apply local > > variables and permanently mark these values". > > I have a longer reply that I will send tomorrow, but wanted to respond > to this. > > Yes exactly! I have the equivalent complaint in the draft extras from > my previous message! I actually implemented a blacklist for file local > variables in orgstrap because it is a critical security feature. The > fact that it is missing is absolutely a bug and is extremely annoying > for some of my current workflows where I have local variables that I > never want to accept. Then maybe just add to the same bug your opinion.
Re: Local variables insecurities - Re: One vs many directories
* Greg Minshall [2020-11-26 08:34]: > Tom, > > > 2. If mutt is launching Emacs, you can pass --eval "(setq > >enable-local-eval nil)" on the command line and all file local > >variables will be ignored and treated as plain text. > > maybe that is one thing that could really help here. possibly mutt and > other emacs-based mail readers, when putting up a received e-mail > message, should by default do just that (and, also, '(setq > enable-local-variables :safe)'?). > > i use mh-e, and it does *not* seem to do that. but, if emacs is niche, > mh-e is the niche-squared. :) I have sent summary of our discussion here to the bug #44837 for further review, we may switch to better Org related subjects here if you people agree. https://debbugs.gnu.org/cgi/bugreport.cgi?bug=44837
Re: Local variables insecurities - Re: One vs many directories
Located Eric's message and can confirm the same for mu4e (no query about setting/evaluating anything upon opening the message). cm Eric S Fraga writes: > On Wednesday, 25 Nov 2020 at 15:38, Jean Louis wrote: >> I have not configured anything. In fact I have opened the email and I >> was surprised that I am getting those dialogues to execute local >> variables. > > Very strange. It was my email that instigated this part of the > thread. I can view my email and I do not get asked to set any locate > variables or, in this case, evaluate the elisp expression. Out of > curiosity, what mail reader did you use that actually interprets file > local variables? Gnus article view does not.
Re: Local variables insecurities - Re: One vs many directories
> As there is the option ! to "apply local variables and permanently > mark these values" but there is no option "not to apply local > variables and permanently mark these values". I have a longer reply that I will send tomorrow, but wanted to respond to this. Yes exactly! I have the equivalent complaint in the draft extras from my previous message! I actually implemented a blacklist for file local variables in orgstrap because it is a critical security feature. The fact that it is missing is absolutely a bug and is extremely annoying for some of my current workflows where I have local variables that I never want to accept.
Re: Local variables insecurities - Re: One vs many directories
Additionally, as a good example of faulty design, user is coerced to ACCEPT local variables rather than is given fair choice. As there is the option ! to "apply local variables and permanently mark these values" but there is no option "not to apply local variables and permanently mark these values". That is not fair choice. It pushes user to finally ! apply and accept it, but does not give chance to permanently ignore it. Do you want to apply it? You can type y -- to apply the local variables list. n -- to ignore the local variables list. ! -- to apply the local variables list, and permanently mark these values (*) as safe (in the future, they will be set automatically.)
Re: Local variables insecurities - Re: One vs many directories
Tom, > 2. If mutt is launching Emacs, you can pass --eval "(setq >enable-local-eval nil)" on the command line and all file local >variables will be ignored and treated as plain text. maybe that is one thing that could really help here. possibly mutt and other emacs-based mail readers, when putting up a received e-mail message, should by default do just that (and, also, '(setq enable-local-variables :safe)'?). i use mh-e, and it does *not* seem to do that. but, if emacs is niche, mh-e is the niche-squared. :) cheers, Greg
Re: Local variables insecurities - Re: One vs many directories
* Tom Gillespie [2020-11-26 05:07]: > Hi Jean, > > Some points in summary before a long email. > 1. Having an accepting default behavior as a user (i.e., saying yes to >all prompts) is bad security practice. The only thing that systems >can do is prompt as infrequently as possible in hopes that people >don't get prompt fatigue. Emacs does this. I know, and do not speak for one person but for those who will not know. To ask users who do not know programming using editor for text editing and to assume that users will know programming terms: variables, values, applying, safe and ANYTHING else that is written in eval: is bad security practice. > 2. If mutt is launching Emacs, you can pass --eval "(setq >enable-local-eval nil)" on the command line and all file local >variables will be ignored and treated as plain text. Sure, I know, but human text editors will not know. They are faced with YES/NO. What has to be improved is the YES/NO dialogue that has no hyperlinks to its corresponding explanations and asks users things with wrong assumption that user will understand them. > 3. If people don't read the manual and don't read the prompt, there >isn't much more that Emacs can do while still empowering the >user. Empowering can take place in the dialogue as such has enhancement space. Do you think it is good idea? > In those cases we also can't assume that the community will be of > much help either, as we are trying to be here. Community is of great help. Users are many and just fraction of users are in this community for example. Only subset of users even while being in community or reading emails or website will be participating in such. > 4. That said, the local variables prompt could be more alarming and >provide a quick way to access the manual about local >variables. I'll look into a patch for that. Yes, that is it. To empower user is to give him straight better reference on what to do. Following places could become buttons: The local variables list in new-concept.org ^^^ Button to A below contains values that may not be safe (*). ^^^ Button to A below Button to A below Template: Do you want to apply it? You can type y -- to apply the local variables list. ^^ Button to A below n -- to ignore the local variables list. ! -- to apply the local variables list, and permanently mark these ^^^ Button to A values (*) as safe (in the future, they will be set automatically.) * eval : (let ((pos (org-babel-find-named-block "stages"))) (when pos (save-excursion (org-with-point-at pos (org-babel-execute-src-block) ^^ Button to A below. The template how it looks originally: The local variables list in new-concept.org contains values that may not be safe (*). Do you want to apply it? You can type y -- to apply the local variables list. n -- to ignore the local variables list. ! -- to apply the local variables list, and permanently mark these values (*) as safe (in the future, they will be set automatically.) * eval : (let ((pos (org-babel-find-named-block "stages"))) (when pos (save-excursion (org-with-point-at pos (org-babel-execute-src-block) Button A should follow here: ^ File: emacs.info, Node: Safe File Variables, Prev: Specifying File Variables, Up: File Variables 49.2.4.2 Safety of File Variables . Right now situation is this: - if user press C-g to abort the file will be loaded, which is good - user will not be able to access any of the visible buttons by using cursor because somebody made cursor invisible in that dialogue. Cursor should be made visible that keys can be used to access buttons - There shall be on minibuffer some key like "? TO READ MANUAL" to go straight to the manual section above which is warning user much better. - IF THE USER press C-g, or tries to escape it or clicks by using keyboard on the button, or even switch to the buffer with keyboard, or uses mouse to activate the button, or uses ? to activate to read manual, in all those cases the dialogue should be interrupted and file loaded. User should not be left facing dialogue if it was clear from user's interaction that there was doubt with the meanings of the dialoue. - Adding references on unsafe local variables to TUTORIAL > Full disclosure. I make use of file local variables every day > and I have spent quite a while writing orgstrap which relies heavily > on them, so I am definitely biased. With such enhancement there is no need to change history of the past, we just change the future and history for the future. > Emacs is a sharp tool. Experts need sharp tools. If someone > complains about the security of prompts but also admits that they > always say yes to
Re: One vs many directories
> For my own setup I run code in a hook to update the agenda whenever I > change a TODO state, clock in or clock out, but that has performance > problems when I do it while the Agenda is shown. You do not have or update the whole agenda view. I use the following code to update the clocking highlights in agenda even when I clock-in/out outside the agenda buffer: https://github.com/yantar92/emacs-config/blob/master/config.org#update-highlight-from-currently-clocked-task-in-agenda-even-if-the-task-was-clocked-inout-from-outside The same can be done for todo state changes using org-agenda-change-all-lines Best, Ihor "Dr. Arne Babenhauserheide" writes: > Jean Louis writes: > >> Some people maybe access multiple Org files through Agenda, me I >> don't. Some items are "non existent" and I do not know how to ask >> agenda to refresh itself. > > Simply press the letter g. > > For my own setup I run code in a hook to update the agenda whenever I > change a TODO state, clock in or clock out, but that has performance > problems when I do it while the Agenda is shown. > > (defvar todo-modified-from-agenda nil "Indicates whether org-mode todo > state changes were triggered from the agenda. Check this to avoid trying to > propagate the change back into the agenda") > ;; continuously update agenda view, from > http://thomasf.github.io/solarized-css/test/org-hacks.html > (defun kiwon/org-agenda-redo-in-other-window () > "Call org-agenda-redo function even in the non-agenda buffer." > (interactive) > (when (not (and (boundp 'todo-modified-from-agenda) > todo-modified-from-agenda)) ; org-state non-nil means we’re coming from the > org-after-todo-state-change-hook, which would throw when changing todo states > from agenda due to a circular action > (let ((agenda-window (get-buffer-window (or (and (boundp > 'org-agenda-buffer-name) org-agenda-buffer-name) "plan.org") t))) > (when agenda-window > (with-selected-window agenda-window > (org-agenda-redo)) > ;; advice agenda todo to avoid redo, thanks to > http://nullprogram.com/blog/2013/01/22/ > (defadvice org-agenda-todo (before org-agenda-disable-redo activate) > (setq todo-modified-from-agenda t)) > (defadvice org-agenda-todo (after org-agenda-enable-redo activate) > (setq todo-modified-from-agenda nil)) > > (add-hook 'org-clock-in-hook 'kiwon/org-agenda-redo-in-other-window) > (add-hook 'org-clock-out-hook 'kiwon/org-agenda-redo-in-other-window) > (add-hook 'org-after-todo-state-change-hook > 'kiwon/org-agenda-redo-in-other-window) > > > Best wishes, > Arne > -- > Unpolitisch sein > heißt politisch sein > ohne es zu merken
Re: One vs many directories
> Can I automated the execution of Babel code upon opening of the Org
> file?
Adding to other suggestions, you can always add a custom function to
org-mode-hook instead of playing with file-local variables.
> Then we comes to actual execution of tasks. How do we get reminded?
>
> Is the reminder only if I press {C-c a} for org-agenda? Do I need to
> do action to get reminded?
You can always configure Emacs to run agenda on startup. Just add a
command to your init file ;)
For automatic reminders, there is built-in org-notify.el or external
org-alert package (https://github.com/spegoraro/org-alert).
> Personal problem is that tasks are sparse and separate in various Org
> files and not centralized. I become dependent of org-agenda to do what
> I need but it never does what I need.
I agree that org-agenda has many issues that cannot be easily solved
because of its complexity. However, everything you describe (including
multi-occur) can also be achieved with org-ql
(https://github.com/alphapapa/org-ql) - analogue of SQL query language
for org-mode (with more optimisations in comparison with org-agenda).
Best,
Ihor
Jean Louis writes:
> * Ihor Radchenko [2020-11-23 08:43]:
>> >> I am wondering what you mean by Org's philosophy. Why would it have
>> >> anything to do with directories?
>> >
>> > Org's philosophy is to have one or a handful of directories without
>> > nesting of directories. Users are not expected to have their Org
>> > files in a deeply nested tree. Org also prefers big files with large
>> > trees rather than lots of little files.
>> >
>> > By philosophy, I mean the dev consensus on the correct way to do
>> > things, and coded configuration and usability biases.
>>
>> I believe that org support all possibilities. The user can decide to
>> keep many (possibly nested) org files, a few large org files, or
>> anywhere in between. There are several parallel feature sets allowing to
>> work in a single file as well as with a bunch of smaller files.
>
> Yes, sure, and I guess you mentioned some people have problems with
> many files. And I have no problem at all with many files as they are
> per subject separated, per person or per subject separated. They are
> not hyperlinked to each other, it is me who make system to hyperlink
> to files.
>
> Searching for Joe Doe, F4 and I am in Org file for Joe Doe. My
> personal TODO list need not really show the tasks assigned to Joe Doe,
> I could show only * TODO Joe Doe and when I click there then I can get
> all tasks for Joe Doe as new Org file.
>
> It means I am accessing hundreds of Org files from the meta level by
> using conceptual location backed by the database.
>
> Some people maybe access multiple Org files through Agenda, me I
> don't. Some items are "non existent" and I do not know how to ask
> agenda to refresh itself. This is not big deal as I do not access
> items throgh Agenda, though I find it very useful.
>
> org-agenda is trying to put all tasks and notes from various files
> into one list and that is of course not so easy task considering that
> files can be anywhere on the file system and that they need to be
> "remembered".
>
>> For a single file, the user can search headings with org-goto (without a
>> need to explicitly travel through all the nesting headline levels),
>> reveal only headings satisfying certain keyword/tag/any other search
>> criteria with org-sparse-tree, or built agenda views restricted to a
>> single file (or even subtree).
>
> M-x org-goto is useful feature to find headlines. And I never use it,
> just standard Emacs search is enough within a file. Meanings I am
> searching are often inside of the headline. And it is not my perosnal
> way locating things.
>
> Personally, I am using parts of Org, like specific headling to export
> it and to send to remote person, or to print the file as project and
> to bind it nicely.
>
> When I see repetitive action, for example that I have to send "Daily
> Report" template to a person by email, than I just bookmark that in
> Emacs with {C-x r m} under something that I think is the meaning of
> it. Then I forget about it. Next time when I need to send report, I am
> {C-x r b} and quickly completing it and then I am exporting and
> inserting into the email.
>
> In general I speak of subsets or sub-lists among lists. List of Org
> files is one list, list of headings within Org file is other list, and
> list of specific subject related headings or bookmarks to such is
> third subset of lists.
>
>> For multiple files located anywhere in the filesystem, there is always
>> org-refile capable of filing the information to proper place
>> searching deeply nested headlines with ease regardless of the file the
>> information is physically located in. Headlines from multiple files can
>> be grouped using agenda views for any given search criteria (showing
>> todo items or items for a single day/week is just a tiny subset of what
>> agenda can do).
>
> That may be useful for
Re: Local variables insecurities - Re: One vs many directories
Hi Jean, Some points in summary before a long email. 1. Having an accepting default behavior as a user (i.e., saying yes to all prompts) is bad security practice. The only thing that systems can do is prompt as infrequently as possible in hopes that people don't get prompt fatigue. Emacs does this. 2. If mutt is launching Emacs, you can pass --eval "(setq enable-local-eval nil)" on the command line and all file local variables will be ignored and treated as plain text. 3. If people don't read the manual and don't read the prompt, there isn't much more that Emacs can do while still empowering the user. In those cases we also can't assume that the community will be of much help either, as we are trying to be here. 4. That said, the local variables prompt could be more alarming and provide a quick way to access the manual about local variables. I'll look into a patch for that. Now on to the rest! Full disclosure. I make use of file local variables every day and I have spent quite a while writing orgstrap which relies heavily on them, so I am definitely biased. Emacs is a sharp tool. Experts need sharp tools. If someone complains about the security of prompts but also admits that they always say yes to prompts without reading them, then no one will take them seriously when they try to argue that it is the prompt that is insecure. It is like complaining that the forest is dangerous while insisting on eating the berries of all the plants and picking up all the snakes. The forest is dangerous, but not merely because the berries and the snakes exist in it. Heck, some snakes even try to warn you! There are some rattlesnakes that have evolved to not rattle because idiot humans go find and kill them. Now everyone suffers because there are silent rattlesnakes that will strike without warning. Degrading usability because some users are irresponsible just reinforces the irresponsible behavior and everyone loses. The endgame for all prompts is a two-man two-key system where the switches are separated by 30 feet and must be turned within 1 second of each other. How else could we be sure that the user really meant to say yes!? Emacs is scary, but not nuclear launch system levels of scary. Furthermore, Emacs does try to account for users who don't read the manual and has sane and safe defaults. Namely it does not blindly execute code but prompts the user and lets them know that something is going on. Further, when it prompts the user it does not provide a default action, it forces them to answer so that they must attend to what they are doing. This provides the user with an opportunity to become aware of a feature of Emacs that is new to them. The only other option is to never execute local variables until a user reads the manual and changes their config. Such a design infantilizes the user and does not empower them. Emacs isn't just a dissociated tool, or at least it tries not to be (lots of people also complain about the splash screen being enabled by default!). The Emacs community often also goes out of its way to share its knowledge when such situations arise (as we are doing here). We can't reach everyone (yet!) but many people donate time and effort to share. Emacs and Org both actively encourage users to participate in the mailing lists because venues like StackOverflow are not guaranteed to be seen by the community and are not officially supported. Finally on this point, if we are willing to open the manual we see that Emacs tries to educate users about this, the first line of the section on file local variables is "File-local variables can be dangerous." With regard to the local variables prompt. Maybe the right thing to do in this case would be to add a "?" option so that users can open the manual? That seems like it would help. That said, the second line of the prompt reads "contains variables that may not be safe (*)" which should be alarming. I guess it could be changed to "THIS FILE COULD PWN YOU. Please review potentially unsafe variables marked with an asterisk (*)." Or just make the second line a bold warning? I have a patch I need to send in to make it possible to use lexical scope in eval local variables, I'll take a look at this while I'm there. It is forgivable to not be utterly terrified of systems that can execute arbitrary code, given that they mostly look like rocks, and we are surrounded by them all day. However, they are utterly terrifying existences! While I think it would be great for the Emacs splash screen to come with a big warning that says "WARNING THIS PROGRAM ENABLES YOU TO EXECUTE ARBITRARY CODE" I admit that it took me nearly a decade to begin to understand what arbitrary code execution means and why I should be scared of it. Those words are meaningless to a 12 year old who at best will think "What is the big deal about arbitrary code? I can already run any code that I want!" This is why I pointed you to orgstrap. The eval: (org-sbe startup)
Re: Local variables insecurities - Re: One vs many directories
* Tim Cross [2020-11-25 23:54]: > I guess this is probably the main point where we disagree. > > Emacs is first and foremost a programmers editor. It was never designed > as a general purpose editor, but rather specifically as an editor for > programmers. Yes. And when I was born as baby I was designed for milk, not for typing, times change. People use GNU/Linux and Emacs is not advertised as programmers or exclusively programmers editor. Some other editors are advertised that way. So think how many hundreds of thousands of users are working with Emacs. Here is how Debian GNU/Linux describes it: https://packages.debian.org/buster/emacs If there are 10 programmers there are probably 100 if not 500 non-programmers. > If you jump into a formula 1 race car, you would find it almost > impossible to drive. The gearbox would be unfamiliar and difficult to > use, the clutch would be difficult to use etc. If you got it going, you > would have a high likelihood of crashing. Luckily, you would probably > just stall and get nowhere. > > Is this the fault of the design of the race car or the driver? Race cars are not distributed through GNU/Linux operating systems and are not easily downloadable by everybody, in general, they are also expensive. While it all sounds entertaining, Emacs is not a race car. And we cannot say to users not to use it if they are not Formula One Drivers. > With respect to your email example, the number of people who are exposed > is even less - it is really only those who are using it in the same > manner as you. That is, where they have configured their mail client > (such as Mutt) to use Emacs as the external editor. None of the Emacs > mail clients I have used do this (this includes VM, mu4e, gnus, > wonderlust and mew). I do not need to use Emacs with Mutt to invoke local variables. I can get files by any means and by any opening of the file with Emacs it will be invoked. Somebody could send me file to download and open. File can come from anywhere, it is not Mutt related really. Gnus buffers and email clients do not invoke local variables and that is fine. But security issue is not email centric, but file centric. > anyone who has gone to the effort to configure their mail system to use > an external editor and who then answers yes to the statement > "...contains values that may not be safe. Do you want to apply it?" is > someone with inherently unsafe practices. That is very rigid assumption. People set editors for various email clients since decades. Try to think from another people's view points. Here is example: https://stackoverflow.com/questions/15865495/opening-a-file-in-emacs-values-that-are-not-safe That person has quite different view point. Person asks "Why it would not be safe?" and one should know when one person writes there for an answer there are probably thousand other persons who did not write for the answer. Other person asked: "Thanks, that's very helpful. Why would a file (i.e. the author of the file) require or ask Emacs to apply configuration values when just opening/visiting the file? – Amelio Vazquez-Reina" I know why, but people using Emacs are asking why. Many will not ask and will say, damn YES, as I feel safe! Denial of Service Attacks possible: https://github.com/aquamacs-emacs/aquamacs-emacs/issues/147 https://gitmemory.com/issue/davidswelt/aquamacs-emacs/147/478196367 .emacs considered not safe: https://www.cs.ait.ac.th/~on/O/oreilly/tcpip/puis/ch11_05.htm OK then now back to Org discussions. Jean
Re: Local variables insecurities - Re: One vs many directories
Jean Louis writes: > * Eric S Fraga [2020-11-25 16:58]: >> On Wednesday, 25 Nov 2020 at 16:13, Jean Louis wrote: >> > I use Mutt. >> > The message is opened in Emacs in mail-mode >> >> Ah, so mutt saves content in a file which is then opened by >> Emacs. Okay, that makes sense. Gnus does things the other way around: >> opens the buffer (associated with a file in the draft directory), >> inserts the content, and then puts the user in control. File local >> variables don't get a chance to be interpreted then. > > That is specific to Gnus. Any file opened by Emacs any will still > invoke the dialogue for local variables. > >> > Then I have been testing and even text files invoke local variables. >> >> Yes, of course. That's the whole point? > > You know that point is bad design and assumption that every user is > programmer who knows what are local variables and what are definitions > of Emacs functions, it is incredible situation. I guess this is probably the main point where we disagree. Emacs is first and foremost a programmers editor. It was never designed as a general purpose editor, but rather specifically as an editor for programmers. If you jump into a formula 1 race car, you would find it almost impossible to drive. The gearbox would be unfamiliar and difficult to use, the clutch would be difficult to use etc. If you got it going, you would have a high likelihood of crashing. Luckily, you would probably just stall and get nowhere. Is this the fault of the design of the race car or the driver? Would it make sense to change the design of a race car to use a standard gearbox and clutch just because at some point someone who is not a race car driver might want to drive it? Are there risks associated with local variables. Yes. Is there sufficient protection against these variables being used for malicious purposes in Emacs. I think the answer is yes. Is there any evidence of these variables being used for malicious purpose. None that I am aware of. Has there been bugs in the implementation of this facility - yes. Have these bugs been addressed once identified - yes. With respect to your email example, the number of people who are exposed is even less - it is really only those who are using it in the same manner as you. That is, where they have configured their mail client (such as Mutt) to use Emacs as the external editor. None of the Emacs mail clients I have used do this (this includes VM, mu4e, gnus, wonderlust and mew). anyone who has gone to the effort to configure their mail system to use an external editor and who then answers yes to the statement "...contains values that may not be safe. Do you want to apply it?" is someone with inherently unsafe practices. I doubt any change in wording or phrasing would be of any help for them. However, the correct way to deal with this would be to offer up a patch to the Emacs developers which improve this wording. I would suggest the set of people who are technically aware enough or have sufficient technical interest to have adopted emacs as their email viewer and who would still answer yes to any dialogue warning them of unsafe actions when opening content from an unknown source is very small. Local variables is a powerful and useful feature. Like many powerful features, it can be abused. We differ in our opinions on whether those safe guards are sufficient. I believe they are and I believe you are over stating the risks. I don't believe we will arrive at any consensus and I feel this thread has run its course. You are of course free to respond, but I will refrain from further participation as this has wondered off topic for org mode and I see little to be gained from further back and forth. -- Tim Cross
Re: Local variables insecurities - Re: One vs many directories
* Eric S Fraga [2020-11-25 16:58]: > On Wednesday, 25 Nov 2020 at 16:13, Jean Louis wrote: > > I use Mutt. > > The message is opened in Emacs in mail-mode > > Ah, so mutt saves content in a file which is then opened by > Emacs. Okay, that makes sense. Gnus does things the other way around: > opens the buffer (associated with a file in the draft directory), > inserts the content, and then puts the user in control. File local > variables don't get a chance to be interpreted then. That is specific to Gnus. Any file opened by Emacs any will still invoke the dialogue for local variables. > > Then I have been testing and even text files invoke local variables. > > Yes, of course. That's the whole point? You know that point is bad design and assumption that every user is programmer who knows what are local variables and what are definitions of Emacs functions, it is incredible situation.
Re: Local variables insecurities - Re: One vs many directories
On Wednesday, 25 Nov 2020 at 16:13, Jean Louis wrote: > I use Mutt. > The message is opened in Emacs in mail-mode Ah, so mutt saves content in a file which is then opened by Emacs. Okay, that makes sense. Gnus does things the other way around: opens the buffer (associated with a file in the draft directory), inserts the content, and then puts the user in control. File local variables don't get a chance to be interpreted then. > Then I have been testing and even text files invoke local variables. Yes, of course. That's the whole point? (and, yes, I've been reading the thread so I know the concerns about security etc.) -- : Eric S Fraga via Emacs 28.0.50, Org release_9.4-118-g2a4578.dirty
Re: Local variables insecurities - Re: One vs many directories
* Eric S Fraga [2020-11-25 16:06]: > On Wednesday, 25 Nov 2020 at 15:38, Jean Louis wrote: > > I have not configured anything. In fact I have opened the email and I > > was surprised that I am getting those dialogues to execute local > > variables. > > Very strange. It was my email that instigated this part of the > thread. I can view my email and I do not get asked to set any locate > variables or, in this case, evaluate the elisp expression. Out of > curiosity, what mail reader did you use that actually interprets file > local variables? Gnus article view does not. I use Mutt. The message is opened in Emacs in mail-mode Then I have been testing and even text files invoke local variables. When I send messages I often send it from Emacs, but that is different subject.
Re: One vs many directories
* Ihor Radchenko [2020-11-25 16:16]: > > > ... It does > > evaluates and I get the result in the message buffer, but it does not > > expands in the Org buffer. > > It is expected behaviour. According to the docstring of org-sbe, it only > returns the value, but does not actually change buffer. > > If you want to replace the RESULTS, you need to use the following: > > # Local Variables: > # eval: (let ((pos (org-babel-find-named-block "stages"))) (when pos > (save-excursion (org-with-point-at pos (org-babel-execute-src-block) > # End: That works well, thank you for the tip. Of course I will not be writing all that by hand, program would simply invoke the creation and generation of Org file and write headings and the local variables. Next time I open the Org file related to that person I can see all the pending tasks or tasks done with hyperlinks to their corresponding actual tracking places in the database. I have already made the function to capture Org tasks into the database, concept is here: (defun hyperscope-capture-org-heading () "Captures Org heading and stores it in the Hyperscope dynamic knowledge repository" (interactive) (let* ((body (org-copy-subtree nil nil nil t)) (body (pop kill-ring)) (body (org-no-properties body)) (heading (org-get-heading)) (created (org-property-values "CREATED")) (date (if created (substring (car created) 1 11) nil)) (body (with-temp-buffer (insert body) (org-mode) (org-back-to-heading) (kill-line) (delete-matching-lines ":PROPERTIES:") (delete-matching-lines ":CREATED:") (delete-matching-lines ":ID:") (delete-matching-lines ":END:") (buffer-string (hyperscope-add-note-to-set heading body date))) So I am now transitioning all Org tasks into little bit different and more streamlined system for me personally. Nevertheless, when I use a browser I can still use org-capture and later just parse all headings automatically and insert into the database.
Re: One vs many directories
* Eric S Fraga [2020-11-25 16:08]: > On Wednesday, 25 Nov 2020 at 14:46, Jean Louis wrote: > > I have got it to work as I had to name the source block. > > Yes, org-sbe looks for the first source block with the name > given. Nothing to do with headings etc. > > > It does evaluates and I get the result in the message buffer, but it > > does not expands in the Org buffer. That is what I wish to find out > > how. > > What do you get on the same src block if you explicitly C-c C-c there? I get results in the buffer. That is what I would like, to get expanded results in the buffer upon opening of the Org file.
Re: Local variables insecurities - Re: One vs many directories
On Wednesday, 25 Nov 2020 at 15:38, Jean Louis wrote: > I have not configured anything. In fact I have opened the email and I > was surprised that I am getting those dialogues to execute local > variables. Very strange. It was my email that instigated this part of the thread. I can view my email and I do not get asked to set any locate variables or, in this case, evaluate the elisp expression. Out of curiosity, what mail reader did you use that actually interprets file local variables? Gnus article view does not. -- : Eric S Fraga via Emacs 28.0.50, Org release_9.4-118-g2a4578.dirty
Re: One vs many directories
> ... It does > evaluates and I get the result in the message buffer, but it does not > expands in the Org buffer. It is expected behaviour. According to the docstring of org-sbe, it only returns the value, but does not actually change buffer. If you want to replace the RESULTS, you need to use the following: # Local Variables: # eval: (let ((pos (org-babel-find-named-block "stages"))) (when pos (save-excursion (org-with-point-at pos (org-babel-execute-src-block) # End: Best, Ihor Jean Louis writes: > * Eric S Fraga [2020-11-24 12:46]: >> On Tuesday, 24 Nov 2020 at 12:00, Jean Louis wrote: >> > Can I automated the execution of Babel code upon opening of the Org >> > file? >> >> You can, by using file local variables. For instance, for some files, I >> do this: >> >> #+begin_src org >> ,* local variables :noexport: >> # Local Variables: >> # eval: (org-sbe "startup") >> # End: >> #+end_src > > I have got it to work as I had to name the source block. It does > evaluates and I get the result in the message buffer, but it does not > expands in the Org buffer. That is what I wish to find out how. > > ** Stages > #+NAME: stages > #+BEGIN_SRC sql :engine postgresql :exports results :results replace > SELECT 1 AS table; > #+END_SRC > > # Local Variables: > # eval: (org-sbe "stages") > # End:
Re: One vs many directories
On Wednesday, 25 Nov 2020 at 14:46, Jean Louis wrote: > I have got it to work as I had to name the source block. Yes, org-sbe looks for the first source block with the name given. Nothing to do with headings etc. > It does evaluates and I get the result in the message buffer, but it > does not expands in the Org buffer. That is what I wish to find out > how. What do you get on the same src block if you explicitly C-c C-c there? -- : Eric S Fraga via Emacs 28.0.50, Org release_9.4-118-g2a4578.dirty
Local variables insecurities - Re: One vs many directories
* Tim Cross [2020-11-25 09:41]: > >> Why is it a security issue? The variables do need to be close to the end > >> — 3000 characters is only about 50 lines. > > > > Emacs users, Org users on our mailing lists are not so private. Their > > names and email addresses are in the public database. Spammer can > > construct phishing type of an email, including something like Org news > > or something and send such email to users. Among let us say 3000 > > people there will be percentage of users that will say Y to invoke the > > local variables due to lack of knowing what is it doing to computer. > > > > After that, anything becomes possible, including intrusion into > > computer, capturing all email addresses, passwords, sending spam > > emails from computer and so on. > > this is just baseless fear mongering based on nothing but > speculation. My experience with such people come from last more than 25 years. CVE list is the reference I already quotes. Some thing were improved like this: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=684695 So one should know how dangerous it was to introduce local variables with possibility to automatically eval anything there. > Of your suggested 3000 users, only a very small percentage use Emacs as > their mail client. Those which I know through Emacs list mostly use it. I am using it. Is there any way to know who use and who not? In general I am reading emails with Mutt, but I am answering with Emacs. Sending emails is often by Emacs or by Mutt. I use sometimes M-x rmail as well > Of those, only an even smaller number will have their mail client > configured to render messages with a mode that supports local I have not configured anything. In fact I have opened the email and I was surprised that I am getting those dialogues to execute local variables. And I an in mail-mode now. Mutt opens new emacsclient and I edit the file. Obviously user does not need to configure anything. You maybe refer to specific mode where it does not execute. It will try to execute even if I open .TXT file. The very design of Local variables I do not find trustful for these explained reasons. I do not protest against it as now is too late, but as I mentioned more educational references could be provided in the dialogue that asks user to execute local variables or not. > variables and even a smaller number of those would say yes to > executing the code. In fact, anyone who has gone to the extent of > configuring their Emacs email client to open messages with a mime > type of x-org (or even just based on an attachment with *.org in the > file name) is more than likely to be sufficiently technically aware > not to say yes when asked. I do not need to configure emacs to anything to get the local variable execution dialogue, verify it on your side. I can basically get any attachments by email, try to view them in Emacs and it will execute. > Few, if any user, is going to download some random attachment in an > email message Sorry, but I have no choice but to download all attachments. Majority of email clients do not offer choice to download specific attachments they download whole message as one. > open it in emacs and then say yes to a message warning them that > doing so might be dangerous. It does not warn to be dangerous. There is no word of danger in the dialogue. I would rather like the dialogue to does what you say but it does not, I would prefer it like this: === DANGEROUS === But it is not like that, and I have elaborated why it is not like that. Text writers are not programmers. You assume that every user starts from your viewpoint of 30 years experienced Emacs wizard. And I am speaking of design view point. Emacs is still called "editor" today. The description of Emacs I get in Hyperbola GNU/Linux-libre is: The extensible, customizable, self-documenting real-time display editor, without D-Bus support Nowhere it says in the description that it can potentially expose me to malicious evaluations of software just by opening a text file. That you know what Emacs is really and me too, it does not make it more secure. We make assumptions that we will know that users will be safe, but that is wrong assumption and there would be no CVE as I have already referenced it it it would be so. > Such ill-informed users have been pretty much weeded out by all the > other scam phishing out there by now. I would not say so. As non-native English speaker to say for users that they are ill-informed sounds to me not appropriate. We invite users to use Emacs, they will download, open, and may be offered to read the ebook or other interesting text, and text will ask them to evaluate the variables. You say that the subset of those users who will know what is "variable", "value", "evaluation", "safe" is small and not important, and I think this is most important for the future of next 100 years for Emacs being useful to
Re: One vs many directories
* Eric S Fraga [2020-11-24 12:46]: > On Tuesday, 24 Nov 2020 at 12:00, Jean Louis wrote: > > Can I automated the execution of Babel code upon opening of the Org > > file? > > You can, by using file local variables. For instance, for some files, I > do this: > > #+begin_src org > ,* local variables :noexport: > # Local Variables: > # eval: (org-sbe "startup") > # End: > #+end_src I have got it to work as I had to name the source block. It does evaluates and I get the result in the message buffer, but it does not expands in the Org buffer. That is what I wish to find out how. ** Stages #+NAME: stages #+BEGIN_SRC sql :engine postgresql :exports results :results replace SELECT 1 AS table; #+END_SRC # Local Variables: # eval: (org-sbe "stages") # End:
Re: One vs many directories
> When I do C-c a it runs (org-agenda) but I do not have "g" and I am on > development version. The C-c a window is made so that I cannot go with > cursor inside and that I cannot even expect the key map neither invoke > command by M-x and I cannot even M-: C-c a will first show so-called agenda dispatcher asking you what kind of agenda view you want to get. You need to press a key according to the popup window (i.e. `t' to see all not done items). Then, you will get the proper agenda buffer with all the keymaps set and `g' bound to refreshing the chosen agenda view in the buffer. > All that is wrong and not aligned to Emacs common interface. It is bug > that bugs. Agenda buffer should allow users those standard Emacs > features. I am wondering what is the common Emacs interface you refer to. I am not aware about any standard way to prompt user while also showing detailed description of what to expect from different choices. Best, Ihor Jean Louis writes: > * Dr. Arne Babenhauserheide [2020-11-25 11:14]: >> >> Jean Louis writes: >> >> > * Dr. Arne Babenhauserheide [2020-11-24 21:48]: >> >> >> >> Jean Louis writes: >> >> >> >> > Some people maybe access multiple Org files through Agenda, me I >> >> > don't. Some items are "non existent" and I do not know how to ask >> >> > agenda to refresh itself. >> >> >> >> Simply press the letter g. >> > >> > What function is on g on your side? >> >> (org-agenda-redo-all EXHAUSTIVE) > > When I do C-c a it runs (org-agenda) but I do not have "g" and I am on > development version. The C-c a window is made so that I cannot go with > cursor inside and that I cannot even expect the key map neither invoke > command by M-x and I cannot even M-: > > All that is wrong and not aligned to Emacs common interface. It is bug > that bugs. Agenda buffer should allow users those standard Emacs > features. > >> > Thank you. I have plan to switch anything action related to >> > database system and use Org to view tasks, not to handle or store >> > them.
Re: One vs many directories
* Texas Cyberthal [2020-11-25 13:40]: > > You also spoke of device, do you really mean physical device? > > Brain-Computer Interface is a term that usually means an > electromagnetic connection between nerves and electronics. However, > really a keyboard is a superior version of that for non-motor tasks, > despite being insulated on both ends with skin and plastic. hahahha you are right > Because you built your own SME CRM. My point is that an SME CRM > should probably be a separate entity from a Textmind. Their > purposes conflict. Really, but for me those similarities extend each other. Not that I am sorting files by Textmind but I do have similar method in sorting files. You sort files by tags in a hierachy, that is the concept of Textmind. How tags are called or how they relate to each other is then rather implementation. Anyway I wish to know more of that, subscribe me to your mailing list. In last days I got more ideas than I was thinking I will, all from this Org mailing list. And now slowly I start implementing. I like Org but I also like more structured data.
Re: One vs many directories
Hi Jean, > There are those who die and come back and view things from above and can > think and use their mind even though brain was turned off temporarily. I didn't say that the mind always turns off when the brain is damaged. > You also spoke of device, do you really mean physical device? Brain-Computer Interface is a term that usually means an electromagnetic connection between nerves and electronics. However, really a keyboard is a superior version of that for non-motor tasks, despite being insulated on both ends with skin and plastic. Human fingers are amazingly dextrous, so their bandwidth is high. Eventually mammalian brain plasticity will permit implants to surpass keyboards, but that will take a while and might require implantation as a child to achieve maximum results. The generational technology gap moves ever inward, towards our very genes. Obsolete thyself to perpetuate thy meme-ery. > I cannot get it as I do not know what is ADD. Attention Deficit Disorder > But I have M-x already that works. Because you built your own SME CRM. My point is that an SME CRM should probably be a separate entity from a Textmind. Their purposes conflict.
Re: One vs many directories
* Texas Cyberthal [2020-11-25 09:58]: > Hi Jean, > > > Now, what is exomind? > > https://cyberthal-docs.nfshost.com/cyborganize/exomind/ How I like those thoughts. > Mind vs brain > Your brain is the squishy jelly between your ears. > Your mind is what disappears when that jelly gets shaken too hard by > a punch. There are those who die and come back and view things from above and can think and use their mind even though brain was turned off temporarily. > Exomind — an individual's PIMS and the personal info it contains. One > can wax philosophical about how far the exomind extends into situated > cognition. Perhaps the Internet is a shared global exomind. Anyway, > Cyborganize is a personal exomind. I just get idea how exoskelet does that. You also spoke of device, do you really mean physical device? > What you described is not how you think, it is how you wish your CRM > info retrieval system to perform conveniently. Almost nobody has a > formal thought algorithm, because brains have ADD compared to > computers. I cannot get it as I do not know what is ADD. > If you have a huge number of incoming text of a special type, such > as customer leads, who are just cogs that you don't genuinely > ponder, then Textmind is the wrong tool for that job. You need SME > CRM software. But I have M-x already that works. Thank you for examples of file sorting with 10 Bins. > https://cyberthal-docs.nfshost.com/cyborganize/pubmind/ Thank you, I was reading.
Re: One vs many directories
* Dr. Arne Babenhauserheide [2020-11-25 11:14]: > > Jean Louis writes: > > > * Dr. Arne Babenhauserheide [2020-11-24 21:48]: > >> > >> Jean Louis writes: > >> > >> > Some people maybe access multiple Org files through Agenda, me I > >> > don't. Some items are "non existent" and I do not know how to ask > >> > agenda to refresh itself. > >> > >> Simply press the letter g. > > > > What function is on g on your side? > > (org-agenda-redo-all EXHAUSTIVE) When I do C-c a it runs (org-agenda) but I do not have "g" and I am on development version. The C-c a window is made so that I cannot go with cursor inside and that I cannot even expect the key map neither invoke command by M-x and I cannot even M-: All that is wrong and not aligned to Emacs common interface. It is bug that bugs. Agenda buffer should allow users those standard Emacs features. > > Thank you. I have plan to switch anything action related to > > database system and use Org to view tasks, not to handle or store > > them. > > That might cost you reaction-time in the end, because org then cannot > just keep the file open. Do you mean my reaction time or Org reaction time? I did not understand about to keep file open, how do you mean? You remember I said Org files with tasks are on my side almost always related to some people, among them I am included. So I find Org file per person in the directory wherever it is by clicking F4. Then task list in the Org file would update itself as I have got the tip on how to execute the SQL block. Do you mean that SQL block would cost me reaction time? If so, that can be. In the Hyperscope database managed by Emacs and displayed within tabulated-list-mode on this T410 Thinkpad the list of entries looks almost instant, I just press left or right and I get new list. Each time there is SQL query. Without you telling me I would not be thinking on that, but it is true, the larger or more complex the query then it could be processing. But if there is large list of tasks not done, that means anyway that something is terribly wrong with it. Those tasks done, can appear under different heading. But then I cannot selectively automatically execute block under one heading at file opening time while not executing block under different heading.
Re: One vs many directories
Jean Louis writes: > * Dr. Arne Babenhauserheide [2020-11-24 21:48]: >> >> Jean Louis writes: >> >> > Some people maybe access multiple Org files through Agenda, me I >> > don't. Some items are "non existent" and I do not know how to ask >> > agenda to refresh itself. >> >> Simply press the letter g. > > What function is on g on your side? (org-agenda-redo-all EXHAUSTIVE) > I press g and I get error: invalid key g > >> For my own setup I run code in a hook to update the agenda whenever I >> change a TODO state, clock in or clock out, but that has performance >> problems when I do it while the Agenda is shown. > > That sounds that it will use computing power. Yes, it does, but it gives me the interaction I want. > Thank you. I have plan > to switch anything action related to database system and use Org to > view tasks, not to handle or store them. That might cost you reaction-time in the end, because org then cannot just keep the file open. Best wishes, Arne -- Unpolitisch sein heißt politisch sein ohne es zu merken signature.asc Description: PGP signature
Re: One vs many directories
Jean Louis writes: > * Dr. Arne Babenhauserheide [2020-11-24 21:51]: >> >> Jean Louis writes: >> >> The start of the local variables list should be no more than 3000 >> >> > characters from the end of the file >> >> >> >> >> >> Given the length of the email, I guess this is why Emacs saw the variables >> >> as being within the correct range. >> > >> > Yes thank you. I was thinking Emacs will do that only in files where >> > it recognizes some comments or no comments and that variables need >> > to be pretty down in the file, on the bottom. Now I learn it is not >> > so. >> > >> > That is security issue. >> >> Why is it a security issue? The variables do need to be close to the end >> — 3000 characters is only about 50 lines. > > Emacs users, Org users on our mailing lists are not so private. Their > names and email addresses are in the public database. Spammer can > construct phishing type of an email, including something like Org news > or something and send such email to users. Among let us say 3000 > people there will be percentage of users that will say Y to invoke the > local variables due to lack of knowing what is it doing to computer. That isn’t what I meant with my question. What I meant is: Why is it a security issue that the variable cannot only be at the exact end of the file but instead can be in the last 3000 characters of the file? Best wishes, Arne -- Unpolitisch sein heißt politisch sein ohne es zu merken signature.asc Description: PGP signature
Local variables issue - Re: One vs many directories
* Tim Cross [2020-11-25 08:54]: > > Jean Louis writes: > > > Observing users who are asked questions upon invokation of other > > software I can say that many times users just click one of the > > options, either YES or NO, but without real regard to the > > meanings. The purpose to click either YES or NO is to continue one > > step forward and randomity decides later what happens. > > You cannot prevent people from making bad decisions. Saying yes to > something when you don't know what it means is like using the same > weak password for everything. There has been massive amounts of > communication and education out there to let people know about the > risks. If they choose to ignore it or follow practices which are unsafe, > it is their tough luck. I do agree only that it is too general to apply here. That is specific case of showing a dialogue with potential dangers to users who did not specify those local variables and probably do not need such! If you personally specify local variables you need them and know what it is. That is fine. It is general design that was meant for hackers in beginning stages of Emacs development. Today many people use Emacs who may not be hackers. Subset of those people will say YES to anything. It is possible to prevent people to make bad decisions and it is easy, simply don't ask them with the option to make bad decision. Disabling local variables by default would be better decision. I think that it is not possible today to change the default. Design is flawed. From a view point of text editor should not ask user to execute anything by default. User should enable "Local variables detection" when user wish to get asked about executions. > We need to encourage people to take more responsibility for their > actions, not less. I do fully agree on that statement, only it is too general and not specific. I have presented my specific case how I have been answering YES to that dialogue by thinking it is something necessary to read the file properly. I had misunderstandings. Since some time I have the general rule to answer NO on such dialogues. Would there be some malicious intention in those years the intruder would be quite successful. One could fetch the external program and call it "general Emacs enhancement" and open up a backdoor shell into the system. To encourage people to take more responsibility is not necessarily general principle for GNU Emacs. And to encourage them alone will never work. To gain any responsibility person needs knowledge or information. Driving car without knowledge is impossible. Thus pushing some kinds of responsibilities to user, coercing user to decide on something that user does not understand itself lacks one part of responsibility. If user is informed what are local variables or is using them already or reads manual and understands dangers, then user has acquired knowledge to be able to reason when such dialogue pops up with YES or NO. In general the answer YES can ruin your data and computing, and users are not aware of it. When somebody is not aware of what is doing that is not condition of encouragement, rather condition of lack of responsibility. > One important component of this is allowing the consequences for bad > decisions to occur and not spend endless resources protecting people > from themselves. The YES/NO dialogue was invented by programmers and not by users. So it was never an initial decision of the user who reads file from other person, to be asked in the first place if something in that text file should get executed. It may also negatively impact Emacs's image as software: https://cve.mitre.org/cgi-bin/cvekey.cgi?keyword=emacs > If your asked to do something and your clearly told that doing so might > be unsafe and your given an option not to do it which is just as easy to > perform and you still decide to do it, that decision is on you. Problem is with the assumption that user is "clearly told". I have used Emacs since 1999 and I was all the time in scratch buffers and did not really know it is for Emacs Lisp. I have been putting my notes there. And I have ignored many functions of Emacs and used it to program in other programming languages. Despite that I did read the manual I did not clearly get information that there is programming language built-in and it could execute any code. Despite knowing about packages and installing them I did not know how it works. That is my personal reality. I was playing tetris and I did not know it is program that is loaded and executed. For me that was a built-in feature, something that Richard Stallman and others invented and built-in. > The alternative is to remove extremely useful functionality from > many responsible users because of an unknown number of others who > make poor decisions. Review that statement of yours again as you said what I am saying: unknown number of others who make poor decisions. Functionality need not be removed from Emacs to have or to
Re: One vs many directories
Jean Louis writes: > * Tim Cross [2020-11-24 23:40]: >> If people are really concerned about security, they should look first at >> their use of repositories like MELPA. There is no formal review or >> analysis of packages in these repositories, yet people will happily >> select some package and install it. > > Interesting that you are one who mentions that. There are just few > people ever mentioned it. > > I am still in process of the review of MELPA packages and its > system. There are many security issues. > > Package signing is one example. It does not offer much of security > when packages are signed automatically, but it raises level of > security. > > MELPA packages and archive-contents are not PGP signed, while GNU ELPA > packages are signed. > IMO signing of packages is irrelevant when there is no formal review process or even any formal process to verify the credentials of signatures. In fact, just adding signing would likely be coutner-productive as it would give the impression of some sort of security where there is none. Basically, anyone can upload anything to MELPA. The only way anyone would find out that an uploaded package has malicious code is if someone does a code review and spots the malicious payload. Even once they find that, there is little chance of being able to attribute the actions to any individual because no real identity vetting is conducted. MELPA is the wild west. The new non-GNU repository has bene setup precisely due to both the licensing issue and the fact many MELPA packages recommend/encourage the use of non-free software/services. While non-GNU will improve this situation, I don't believe there are any plans to actively review the code in the packages. So, like MELPA, all you really have to go on is package reputation. You cannot have any high level of confidence a package does not contain malicious code other than an expectation that if it is used by a sufficiently large enough number of users, it is unlikely. this is not an issue unique to Emacs. You only have to look at the issues both Google's play store and Apples app store have had in the past to see what the risks are. Both Google and Apple have put large amounts of resources into trying to ensure their repository content is safe and yet they still have failures. Something like GNU Emacs has nowhere near the same resources, so is unlikely to come even close to the same level of security.
Re: One vs many directories
Hi Jean, > Now, what is exomind? https://cyberthal-docs.nfshost.com/cyborganize/exomind/ What you described is not how you think, it is how you wish your CRM info retrieval system to perform conveniently. Almost nobody has a formal thought algorithm, because brains have ADD compared to computers. Textmind is a tool for general human text thoughts. It's unspecialized. If you have a huge number of incoming text of a special type, such as customer leads, who are just cogs that you don't genuinely ponder, then Textmind is the wrong tool for that job. You need SME CRM software. Just common sense. Combine harvester works "better" than a push lawnmower, but everyone still needs a push lawnmower. Ok, Textmind feels more like a riding lawnmower to me, because so comfy, but also better detail work, like a wheedwhacker... and the analogy falls apart. > Contract signed between your company 123 and person ABC? I might like a separate Textmind for a company. If separate, then file under ~/1-Mansort/1-Textmind/2-Linked/7-Name/2-Flat/Doe-John~. Otherwise, sounds like something a relational database should handle. > Image on which there is only your family member, not you, which has its date? Same path, but ~/Surname-/Given-name~, and binaries are stored in Binmind. > Image of you, with the date? Same path, but ~/2-Me~. - Image without date in the file name and not embedded? Depends on image content. - Software git tree related to mailing things? ~/1-Mansort/1-Textmind/2-Linked/2-Codex/~ Large physical keyboards are the best human to computer input device for work and nothing on the horizon challenges that. Keyboards are spreading to more enterprise contexts, such as keyboards in cop cars, as software eats everything. > The other complete thought algorithm is Pubmind, for longform content. > But it doesn't work without Textmind. https://cyberthal-docs.nfshost.com/cyborganize/pubmind/ I suppose it takes Textmind+Pubmind to make a complete thought algorithm. Pubmind terminates in publications rather than action. Pubmind is only necessary once Textmind starts growing clogged with publications that no longer belong inside one's personal exomind.
Re: One vs many directories
Jean Louis writes: > * Dr. Arne Babenhauserheide [2020-11-24 21:51]: >> >> Jean Louis writes: >> >> The start of the local variables list should be no more than 3000 >> >> > characters from the end of the file >> >> >> >> >> >> Given the length of the email, I guess this is why Emacs saw the variables >> >> as being within the correct range. >> > >> > Yes thank you. I was thinking Emacs will do that only in files where >> > it recognizes some comments or no comments and that variables need >> > to be pretty down in the file, on the bottom. Now I learn it is not >> > so. >> > >> > That is security issue. >> >> Why is it a security issue? The variables do need to be close to the end >> — 3000 characters is only about 50 lines. > > Emacs users, Org users on our mailing lists are not so private. Their > names and email addresses are in the public database. Spammer can > construct phishing type of an email, including something like Org news > or something and send such email to users. Among let us say 3000 > people there will be percentage of users that will say Y to invoke the > local variables due to lack of knowing what is it doing to computer. > > After that, anything becomes possible, including intrusion into > computer, capturing all email addresses, passwords, sending spam > emails from computer and so on. this is just baseless fear mongering based on nothing but speculation. Of your suggested 3000 users, only a very small percentage use Emacs as their mail client. Of those, only an even smaller number will have their mail client configured to render messages with a mode that supports local variables and even a smaller number of those would say yes to executing the code. In fact, anyone who has gone to the extent of configuring their Emacs email client to open messages with a mime type of x-org (or even just based on an attachment with *.org in the file name) is more than likely to be sufficiently technically aware not to say yes when asked. Few, if any user, is going to download some random attachment in an email message, open it in emacs and then say yes to a message warning them that doing so might be dangerous. Such ill-informed users have been pretty much weeded out by all the other scam phishing out there by now. To convince them to go through such steps would require some pretty convincing content - a simple org news attachment or similar is unlikely to do it. Even if you do get them to say yes, they are still a long way from being able to compromise the computer Emacs is running on. Gaining some level of access is one thing, actually being able to do something with that access is another. Trying to compromise a computer, which these days typically involves privilege escalation, would be extremely difficult to achieve with elisp. The best you could hope for would be to install a trojan or back door what would allow the attacker to install other tools. Could be possible, but is definitely non-trivial. This ability has been around in Emacs for a very long time - more than 30 years, possibly even longer. There has not been a single recorded incident of large number of users being compromised as a result of this feature. I've not even heard of small numbers being compromised as a result of this feature. I'm sure you will disagree. My suggestion is that if you believe this is a security issue, you put together a proof of concept to demonstrate the vulnerability - this is how such security issues get resolved. Demonstrate how the security issue can be exploited with actual proof of concept code rather than mere speculation and that will provide something concrete which can be dealt with. I suspect you will find it much harder to achieve once you actually try to make it work. -- Tim Cross
Re: One vs many directories
Jean Louis writes: > Observing users who are asked questions upon invokation of other > software I can say that many times users just click one of the > options, either YES or NO, but without real regard to the > meanings. The purpose to click either YES or NO is to continue one > step forward and randomity decides later what happens. You cannot prevent people from making bad decisions. Saying yes to something when you don't know what it means is like using the same weak password for everything. There has been massive amounts of communication and education out there to let people know about the risks. If they choose to ignore it or follow practices which are unsafe, it is their tough luck. We need to encourage people to take more responsibility for their actions, not less. One important component of this is allowing the consequences for bad decisions to occur and not spend endless resources protecting people from themselves. If your asked to do something and your clearly told that doing so might be unsafe and your given an option not to do it which is just as easy to perform and you still decide to do it, that decision is on you. The alternative is to remove extremely useful functionality from many responsible users because of an unknown number of others who make poor decisions. Furthermore, keep in mind that this ability in Emacs has been around for longer than many users have been alive. I've been using it for nearly 30 years and have participated in many forums over that time. I have yet to hear of a single security incident occurring because of local variables. That doesn't mean such incidents have not occurred, but it does likely mean they are rare. -- Tim Cross
Re: One vs many directories
* Tim Cross [2020-11-24 23:40]: > If people are really concerned about security, they should look first at > their use of repositories like MELPA. There is no formal review or > analysis of packages in these repositories, yet people will happily > select some package and install it. Interesting that you are one who mentions that. There are just few people ever mentioned it. I am still in process of the review of MELPA packages and its system. There are many security issues. Package signing is one example. It does not offer much of security when packages are signed automatically, but it raises level of security. MELPA packages and archive-contents are not PGP signed, while GNU ELPA packages are signed. Licensing issues are also a problem with MELPA as it becomes unclear if I have got the license or not when author does not have a proper name. It is not relevant if majority of people do not think or are not aware of licensing as I have to think of it for software that I may re-use, distribute, modify. Did I really get the license if user is named "nick-abc" and have no possible contact information? In some cases for subset of MELPA packages there is no way to verify who really wrote piece of software and if I have received the license legally. Due diligence is on my side. I cannot just claim "But he gave me license" will not help if I have not done proper due diligence, court would not be on my side. Other issue is that MELPA philosophy is to accept any kind of software even if software has been made to drive or control proprietary software. For that reason there is now non-GNU ELPA being developed where useful packages will be distributed from.
Re: One vs many directories
* Tim Cross [2020-11-24 23:40]: > > Thus it is only a security issue if you permanently accept that eval > > file local variable and then open random org files that use it with a > > malicious startup block. An eval file local variable like that which > > blindly executes an org babel block should never be permanently > > accepted > > > > Quite right Tom. > > If people are really concerned about security, they should look first at > their use of repositories like MELPA. There is no formal review or > analysis of packages in these repositories, yet people will happily > select some package and install it. That is analogous to enabling local variables because user has been asked. Popping up a window with question is often a dialogue that users are asked in other software. Dialogues are often not read, just as I was not reading it for years and I did click YES many times. Using such variables is unsafe and the default should be not to execute it without any question. Only when user enables local variables then user should be asked to execute it. That would mean that aware user knows why that is needed. Such will be able to answer questions YES or NO. Unaware users must answer something. To be aware one has to know Emacs Lisp and deeper functions of Emacs. In beginning years it was just fine to assume so due to general computing interests and people being interested in every detail, today there are even more users of Emacs who will not know what is going on. I do not know for you, but when computer asks me anything YES or NO, my tendency is to answer YES regardless if I have read it or not. This same tendency may be with thousands of other users. If I have invoked something on computer and I get asked anything, I have tendency to approve whatever comes on me as I approved it by invoking some action. Not that I am doing it every time yet I have the tendency of doing it. Observing users who are asked questions upon invokation of other software I can say that many times users just click one of the options, either YES or NO, but without real regard to the meanings. The purpose to click either YES or NO is to continue one step forward and randomity decides later what happens.
Re: One vs many directories
* Tom Gillespie [2020-11-24 23:11]: > > > That is security issue. > > > > Why is it a security issue? The variables do need to be close to the end > > — 3000 characters is only about 50 lines. > > It isn't a security issue by itself. Emacs never automatically runs > eval file local variables unless you have tampered with > enable-local-eval, in which case the tamperin is the security issue > not the existence of the local variables list. > > Thus it is only a security issue if you permanently accept that eval > file local variable and then open random org files that use it with a > malicious startup block. An eval file local variable like that which > blindly executes an org babel block should never be permanently > accepted I do understand conditions. But I can say that I did not understand conditions for one decade and a half, as I was not aware that Emacs has a "real programming language " built-in, and I have been spending my time with outside languages that I was invoking from Emacs. Yes, I did read that Emacs has Emacs Lisp. I was configuring Emacs but I have not been thinkin that it is Lisp. I could figure out those settings without reading manual. As I am programming in Emacs Lisp for years I am aware of it. Before I was thinking that local variables belong somewhere and that I should enable it, despite all the warnings. There was lack of understanding despite the information in front of me. Some files opened asked me to enable local variables, so many times I did so without thinking. My personal behavior to enable local variables that other authors have written is probable not isolated case. So that is security issue as number of users among thousands are weak on this. When I say security issue I do not think myself, you or majority of people currently, but that there are probably millions of people who can be affected by this. I also know spammers are harvesting mailing lists.
Re: One vs many directories
* Dr. Arne Babenhauserheide [2020-11-24 21:48]: > > Jean Louis writes: > > > Some people maybe access multiple Org files through Agenda, me I > > don't. Some items are "non existent" and I do not know how to ask > > agenda to refresh itself. > > Simply press the letter g. What function is on g on your side? I press g and I get error: invalid key g > For my own setup I run code in a hook to update the agenda whenever I > change a TODO state, clock in or clock out, but that has performance > problems when I do it while the Agenda is shown. That sounds that it will use computing power. Thank you. I have plan to switch anything action related to database system and use Org to view tasks, not to handle or store them.
Re: One vs many directories
* Dr. Arne Babenhauserheide [2020-11-24 21:51]: > > Jean Louis writes: > >> The start of the local variables list should be no more than 3000 > >> > characters from the end of the file > >> > >> > >> Given the length of the email, I guess this is why Emacs saw the variables > >> as being within the correct range. > > > > Yes thank you. I was thinking Emacs will do that only in files where > > it recognizes some comments or no comments and that variables need > > to be pretty down in the file, on the bottom. Now I learn it is not > > so. > > > > That is security issue. > > Why is it a security issue? The variables do need to be close to the end > — 3000 characters is only about 50 lines. Emacs users, Org users on our mailing lists are not so private. Their names and email addresses are in the public database. Spammer can construct phishing type of an email, including something like Org news or something and send such email to users. Among let us say 3000 people there will be percentage of users that will say Y to invoke the local variables due to lack of knowing what is it doing to computer. After that, anything becomes possible, including intrusion into computer, capturing all email addresses, passwords, sending spam emails from computer and so on.
Re: One vs many directories
Tom Gillespie writes: >> > That is security issue. >> >> Why is it a security issue? The variables do need to be close to the end >> — 3000 characters is only about 50 lines. > > It isn't a security issue by itself. Emacs never automatically runs > eval file local variables unless you have tampered with > enable-local-eval, in which case the tamperin is the security issue > not the existence of the local variables list. > > Thus it is only a security issue if you permanently accept that eval > file local variable and then open random org files that use it with a > malicious startup block. An eval file local variable like that which > blindly executes an org babel block should never be permanently > accepted > Quite right Tom. If people are really concerned about security, they should look first at their use of repositories like MELPA. There is no formal review or analysis of packages in these repositories, yet people will happily select some package and install it. -- Tim Cross
Re: One vs many directories
> > That is security issue. > > Why is it a security issue? The variables do need to be close to the end > — 3000 characters is only about 50 lines. It isn't a security issue by itself. Emacs never automatically runs eval file local variables unless you have tampered with enable-local-eval, in which case the tamperin is the security issue not the existence of the local variables list. Thus it is only a security issue if you permanently accept that eval file local variable and then open random org files that use it with a malicious startup block. An eval file local variable like that which blindly executes an org babel block should never be permanently accepted Best, Tom
Re: One vs many directories
Jean Louis writes: >> The start of the local variables list should be no more than 3000 >> > characters from the end of the file >> >> >> Given the length of the email, I guess this is why Emacs saw the variables >> as being within the correct range. > > Yes thank you. I was thinking Emacs will do that only in files where > it recognizes some comments or no comments and that variables need > to be pretty down in the file, on the bottom. Now I learn it is not > so. > > That is security issue. Why is it a security issue? The variables do need to be close to the end — 3000 characters is only about 50 lines. Best wishes, Arne -- Unpolitisch sein heißt politisch sein ohne es zu merken signature.asc Description: PGP signature
Re: One vs many directories
Jean Louis writes: > Some people maybe access multiple Org files through Agenda, me I > don't. Some items are "non existent" and I do not know how to ask > agenda to refresh itself. Simply press the letter g. For my own setup I run code in a hook to update the agenda whenever I change a TODO state, clock in or clock out, but that has performance problems when I do it while the Agenda is shown. (defvar todo-modified-from-agenda nil "Indicates whether org-mode todo state changes were triggered from the agenda. Check this to avoid trying to propagate the change back into the agenda") ;; continuously update agenda view, from http://thomasf.github.io/solarized-css/test/org-hacks.html (defun kiwon/org-agenda-redo-in-other-window () "Call org-agenda-redo function even in the non-agenda buffer." (interactive) (when (not (and (boundp 'todo-modified-from-agenda) todo-modified-from-agenda)) ; org-state non-nil means we’re coming from the org-after-todo-state-change-hook, which would throw when changing todo states from agenda due to a circular action (let ((agenda-window (get-buffer-window (or (and (boundp 'org-agenda-buffer-name) org-agenda-buffer-name) "plan.org") t))) (when agenda-window (with-selected-window agenda-window (org-agenda-redo)) ;; advice agenda todo to avoid redo, thanks to http://nullprogram.com/blog/2013/01/22/ (defadvice org-agenda-todo (before org-agenda-disable-redo activate) (setq todo-modified-from-agenda t)) (defadvice org-agenda-todo (after org-agenda-enable-redo activate) (setq todo-modified-from-agenda nil)) (add-hook 'org-clock-in-hook 'kiwon/org-agenda-redo-in-other-window) (add-hook 'org-clock-out-hook 'kiwon/org-agenda-redo-in-other-window) (add-hook 'org-after-todo-state-change-hook 'kiwon/org-agenda-redo-in-other-window) Best wishes, Arne -- Unpolitisch sein heißt politisch sein ohne es zu merken signature.asc Description: PGP signature
Re: One vs many directories
* Diego Zamboni [2020-11-24 16:15]: > > > > So I think this is bug in Emacs as Local-variables should be on the > > end of the file. > > > According to the manual ( > https://www.gnu.org/software/emacs/manual/html_node/emacs/Specifying-File-Variables.html#Specifying-File-Variables > ): > > The start of the local variables list should be no more than 3000 > > characters from the end of the file > > > Given the length of the email, I guess this is why Emacs saw the variables > as being within the correct range. Yes thank you. I was thinking Emacs will do that only in files where it recognizes some comments or no comments and that variables need to be pretty down in the file, on the bottom. Now I learn it is not so. That is security issue.
Re: One vs many directories
* Diego Zamboni [2020-11-24 16:13]: > > > > So I think this is bug in Emacs as Local-variables should be on the > > end of the file. > > > According to the manual ( > https://www.gnu.org/software/emacs/manual/html_node/emacs/Specifying-File-Variables.html#Specifying-File-Variables > ): > > The start of the local variables list should be no more than 3000 > > characters from the end of the file I see and I wonder. I was thinking that not every prefix or comment will be considered and that such must be really on the end of the file.
Re: One vs many directories
> > So I think this is bug in Emacs as Local-variables should be on the > end of the file. According to the manual ( https://www.gnu.org/software/emacs/manual/html_node/emacs/Specifying-File-Variables.html#Specifying-File-Variables ): The start of the local variables list should be no more than 3000 > characters from the end of the file Given the length of the email, I guess this is why Emacs saw the variables as being within the correct range. --Diego On Tue, Nov 24, 2020 at 12:57 PM Eric S Fraga wrote: > On Tuesday, 24 Nov 2020 at 12:51, Jean Louis wrote: > > So I think this is bug in Emacs as Local-variables should be on the > > end of the file. > > "end of file" is a rather loose term when it comes to local variables... > -- > : Eric S Fraga via Emacs 28.0.50, Org release_9.4-118-g2a4578.dirty > >
Re: One vs many directories
On Tuesday, 24 Nov 2020 at 12:51, Jean Louis wrote: > So I think this is bug in Emacs as Local-variables should be on the > end of the file. "end of file" is a rather loose term when it comes to local variables... -- : Eric S Fraga via Emacs 28.0.50, Org release_9.4-118-g2a4578.dirty
Re: One vs many directories
* Eric S Fraga [2020-11-24 12:46]: > On Tuesday, 24 Nov 2020 at 12:00, Jean Louis wrote: > > Can I automated the execution of Babel code upon opening of the Org > > file? > > You can, by using file local variables. For instance, for some files, I > do this: > > #+begin_src org > ,* local variables :noexport: > # Local Variables: > # eval: (org-sbe "startup") > # End: > #+end_src > > which will evaluate the named src block "startup" when file is opened. > > Note that this is a potential security hole so only do this for files > you trust! For me is fine, as I do that for files I create. When I have opened this email i was also asked to set local variables, imagine. So that could maybe also mean that one could send email that is constructed as Org file and if user answers YES, one could inject malicious stuff. --- the text above --- still asks me if I like to allow eval: (org-sbe "startup") So I think this is bug in Emacs as Local-variables should be on the end of the file.
Re: One vs many directories
On Tuesday, 24 Nov 2020 at 12:00, Jean Louis wrote: > Can I automated the execution of Babel code upon opening of the Org > file? You can, by using file local variables. For instance, for some files, I do this: #+begin_src org ,* local variables :noexport: # Local Variables: # eval: (org-sbe "startup") # End: #+end_src which will evaluate the named src block "startup" when file is opened. Note that this is a potential security hole so only do this for files you trust! -- : Eric S Fraga via Emacs 28.0.50, Org release_9.4-118-g2a4578.dirty
Re: One vs many directories
* Ihor Radchenko [2020-11-23 08:43]:
> >> I am wondering what you mean by Org's philosophy. Why would it have
> >> anything to do with directories?
> >
> > Org's philosophy is to have one or a handful of directories without
> > nesting of directories. Users are not expected to have their Org
> > files in a deeply nested tree. Org also prefers big files with large
> > trees rather than lots of little files.
> >
> > By philosophy, I mean the dev consensus on the correct way to do
> > things, and coded configuration and usability biases.
>
> I believe that org support all possibilities. The user can decide to
> keep many (possibly nested) org files, a few large org files, or
> anywhere in between. There are several parallel feature sets allowing to
> work in a single file as well as with a bunch of smaller files.
Yes, sure, and I guess you mentioned some people have problems with
many files. And I have no problem at all with many files as they are
per subject separated, per person or per subject separated. They are
not hyperlinked to each other, it is me who make system to hyperlink
to files.
Searching for Joe Doe, F4 and I am in Org file for Joe Doe. My
personal TODO list need not really show the tasks assigned to Joe Doe,
I could show only * TODO Joe Doe and when I click there then I can get
all tasks for Joe Doe as new Org file.
It means I am accessing hundreds of Org files from the meta level by
using conceptual location backed by the database.
Some people maybe access multiple Org files through Agenda, me I
don't. Some items are "non existent" and I do not know how to ask
agenda to refresh itself. This is not big deal as I do not access
items throgh Agenda, though I find it very useful.
org-agenda is trying to put all tasks and notes from various files
into one list and that is of course not so easy task considering that
files can be anywhere on the file system and that they need to be
"remembered".
> For a single file, the user can search headings with org-goto (without a
> need to explicitly travel through all the nesting headline levels),
> reveal only headings satisfying certain keyword/tag/any other search
> criteria with org-sparse-tree, or built agenda views restricted to a
> single file (or even subtree).
M-x org-goto is useful feature to find headlines. And I never use it,
just standard Emacs search is enough within a file. Meanings I am
searching are often inside of the headline. And it is not my perosnal
way locating things.
Personally, I am using parts of Org, like specific headling to export
it and to send to remote person, or to print the file as project and
to bind it nicely.
When I see repetitive action, for example that I have to send "Daily
Report" template to a person by email, than I just bookmark that in
Emacs with {C-x r m} under something that I think is the meaning of
it. Then I forget about it. Next time when I need to send report, I am
{C-x r b} and quickly completing it and then I am exporting and
inserting into the email.
In general I speak of subsets or sub-lists among lists. List of Org
files is one list, list of headings within Org file is other list, and
list of specific subject related headings or bookmarks to such is
third subset of lists.
> For multiple files located anywhere in the filesystem, there is always
> org-refile capable of filing the information to proper place
> searching deeply nested headlines with ease regardless of the file the
> information is physically located in. Headlines from multiple files can
> be grouped using agenda views for any given search criteria (showing
> todo items or items for a single day/week is just a tiny subset of what
> agenda can do).
That may be useful for those who find it while my use case is
different, here is how it is for me:
** TODO Heading [1/2] [50%]
1) [X] Do this
2) [ ] Do that
that is my personal use case. I do not do things like:
** TODO Do this
Description of the task
And so far I know org-refile works on headings. It does not work on
list items.
Sometimes the task describes something that belongs to other file, I
just kill and yank to other file. And I keep RCS revision control
system of files.
As user may have many various sparse tasks to do or notes that require
action and attention in soonest future it is best to consolidate tasks
into one centralized system.
Such system should encompass all tasks or notes that require attention
or action in soonest future and should offer constant reminders to
user on what has to be done and when and which people are related to
the task.
When I mentioned "sparse tasks" I refer to my usage and handling of
mess:
1) Bunch of Org files, org-agenda and Org mode tries to accommodate me
by consolidating everything into lists
2) There are hundreds of such tasks all over, Org tries to consolidate
it.
3) There are various tasks and actions to do that are not recorded in
Org files, those cannot be handled by Org.
4) There are database based
Re: One vs many directories
> I find it entertaining for now. Now, what is exomind? Unless I misunderstood, Jean referred to "external brain" concept: - https://beepb00p.xyz/exobrain/ - https://zettelkasten.de/posts/extend-your-mind-and-memory-with-a-zettelkasten/ - https://github.com/novoid/Memacs - https://blog.jethro.dev/posts/org_mode_workflow_preview/ Best, Ihor Jean Louis writes: > * Texas Cyberthal [2020-11-23 19:08]: >> Hi Jean, >> >> > I have tried your solution and could not find the mental concept to relate >> > to my thinking. >> >> I forgot this inductive sorting skill must be learned gradually, like >> touch typing, at small scale before exomind conversion. > > I find it entertaining for now. Now, what is exomind?
Re: One vs many directories
* Texas Cyberthal [2020-11-23 19:08]: > Hi Jean, > > > I have tried your solution and could not find the mental concept to relate > > to my thinking. > > I forgot this inductive sorting skill must be learned gradually, like > touch typing, at small scale before exomind conversion. I find it entertaining for now. Now, what is exomind? > > Do we think of a tree of knowledge first? I do not think so. And there are > > memory systems that DO think of plethora of various things and increase > > human memory capabilities. > > Yes, Textmind becomes a mnemonic system. The tree associates all > one's info together, making explicit one's personal implicit > prioritization of info. Doing so systematically is only possible with > computer plus user algorithm. I do agree that various systems like 10 Bins for filing files by how human thinks are useful. Only that "how human thinks" varies by humans and not even humans may know how to explain how they think. My concept is that I think what I want, like "I wish to see history of interactions with this person" and then I click F3 or something, I can see SMS, emails, if there were any negotiations, contracts, if there are licenses submitted for partnership. That all has some practical meanings. As for now and probably due to misunderstanding I cannot find 10 Bin practical for me and this may be due to different way of thinking. Could you give me reference describing how you would file: - Contract signed between your company 123 and person ABC? - Image on which there is only your family member, not you, which has its date? - Image of you, with the date? - Image without date in the file name and not embedded? - Software git tree related to mailing things? > > How human think -- is nowhere defined and is vague. Human thinks how they > > think and there may be as many versions as humans. > > The brain is plastic. It adapts easily to sync with a Textmind tree. > This tree's complete thought algorithm is an improvement over native > thought pattern. Computer and brain meet in the middle. That is > cognitive cyborg first stage. Keyboard+screen is Brain-Computer > Interface. OK while I do try to adapt for now I do not find it easy. While I do not adore hierarchies, file system is easier to adapt, as user can just something like "Brother" and file it there. It is entertaining as it sounds futuristic. But I do not know thought algorithm, and what would be native thought pattern, and I would not like to become cognitive cyborg, not in this stage of Emacs development, as nobody likes to get killed. Maybe in some future when I can load it into my personal memory and run on it even during sleep time. Keyboards will soon expire, they partially already expired as billions of people use computers without keyboards including that term "computer" became dilluted and hidden so that people do not even know they carry one or two in their pockets. Screen will expire too. Keyboards will become sensitive lights and screens holograms. > The other complete thought algorithm is Pubmind, for longform content. > But it doesn't work without Textmind. Need practical example.
Re: One vs many directories
Hi Jean, > I have tried your solution and could not find the mental concept to relate to > my thinking. I forgot this inductive sorting skill must be learned gradually, like touch typing, at small scale before exomind conversion. > Do we think of a tree of knowledge first? I do not think so. And there are > memory systems that DO think of plethora of various things and increase human > memory capabilities. Yes, Textmind becomes a mnemonic system. The tree associates all one's info together, making explicit one's personal implicit prioritization of info. Doing so systematically is only possible with computer plus user algorithm. > How human think -- is nowhere defined and is vague. Human thinks how they > think and there may be as many versions as humans. The brain is plastic. It adapts easily to sync with a Textmind tree. This tree's complete thought algorithm is an improvement over native thought pattern. Computer and brain meet in the middle. That is cognitive cyborg first stage. Keyboard+screen is Brain-Computer Interface. The other complete thought algorithm is Pubmind, for longform content. But it doesn't work without Textmind. Other thought methods are even less complete, and thus more dependent on the Textmind foundation. For example, pure association and search retrieval benefits greatly from Textmind de facto spaced repetition and directory scoping. > Doug Engelbart has already envisioned how files could be stored, accessed, > hyperlinked, referenced and we do not use it in that sense today after how > many years? Exactly. To the extent he was correct, his ideas have been adopted. To the extent wrong, ignored. The main problem is sustaining long-term hybrid-intelligence text-mind sync. It requires a complete OODA algorithm. Engelbart doesn't think on this scale. David Allen's GTD tries to, but is limited by paper. One can always improve the crystallized knowledge of a PIMS by adding more metadata and links. That misses the point: fluid intelligence is more important. It tells which info is worth promoting to more expensive representations.
Re: One vs many directories
Dear Jean Louis, Your description of the database reminds me how org-roam handles the files - it also uses an external database for linking and allows quick incremental search that does not really depend on where the files/headings are stored. However, what you are talking about is against org-mode philosophy, as I know it. Currently, the dev's stance is that org files must be self-sufficient. Org-mode should not depend on external database to manage the org files and operations with them. Everything must be plain text! Moreover, some devs are even reluctant to hide metadata (like unique ID implemented in org-id module) from users (which is possible and also partially implemented). Best, Ihor Jean Louis writes: > * Texas Cyberthal [2020-11-23 12:51]: >> Hi Dr. Arne, >> >> > The only part that hits performance limits is the agenda. >> >> Well, IIRC your Org Textmind is much smaller than mine. >> >> > My current guess is that the agenta is slow because it has to parse all my >> > 7500 clock entries, and it has to check the Todo states of around 1200 >> > headings. >> >> Ouch. I'd rather keep a "ramble log" so I can reconstruct an exactly >> honest time accounting, with discounts for partial attention, without >> worrying about fiddly clockin/outs. At least when working from home. >> If clocking into a work site, that's different, because one can >> reasonably bill for the entire time, with minimal clock toggling. >> > >> > Did you check against filesystem limits? At 10k entries in a >> directory typical filesystems start becoming slow. That's the main >> reason I see for adding hierarchies. > > From ext4 manual: > > dir_index > Use hashed b-trees to speed up name lookups in large > directories. This feature is supported by ext3 and > ext4 file systems, and is ignored by ext2 file systems. > > dir_nlink > This ext4 feature allows more than 65000 subdirectories > per directory. > > I think that file systems should be unlimited and fast in relation to > that. I have ~/Maildir with over 5 subdirectories, direct access > is very easy and fast while listing takes some time. > > If file system does not allow fast access it is time to replace it > with one that does allow it. > > Now I wonder of HAMMER in DragonflyBSD is also slow with 5 > directories. > > My PostgreSQL database is not huge, it is when packed about 50 MB. On > the file system it is 810 MB. > > To select 2469 contacts as subset of 204048 contacts that belong in > certain group does not give (usually) feeling of any delay, it looks > instant for human. > > My Org work is on meta-level so my truly important headings or subtree > names are in the database. Subtrees have their various properties, > like I can place any tags there inside, like TODO or designate type of > TODO. My work is intertwined with text and Org mode mostly, but I > could use any kind of mime type or any kind of Emacs mode. Some nodes > are on file system while some are in the database. > > Nodes within subtree are hyperdocuments, they are all linkable and > could be on file system or not on file system. > > Everything is together in one tree and it does not matter as access to > the nodes does not go over the tree necessary. There are 19197 nodes. > To find 76 that are tagged with TODO does not give me any slight or > visible delay, definitely not even 0.2 seconds. When I press enter it > is right there. > > From the system I am using personally I am thinking that Org mode > could get its database connection so that headings and properties are > managed in the database fully while text could be managed in files. It > seems very possible. > > The only thing that would be needed to add to Org in that case is some > heading tag that would uniquely designate where in the database that > heading is managed. It could be very lightly displayed on the screen > and would not be exported by default. > > Something like > > *** TODO Heading :ID-123: > > That would be all. All other meta data belonging to the heading could > be managed in the database. If heading is deleted it need not be > deleted in the database. Text belonging to heading could be managed in > the text file. Properties in the database. It can be simple database > such as GDBM if such is fast enough. > > Meta data for the heading would or could be updated automatically from > time to time. > > User could easily decide to show the properties in the Org file or not > to show. It does not matter much as long as :ID-123: tag is there. > > All things like tags, properties, clock-in and out, schedule, > deadlines, custom_id and everything else as heading meta data could be > manageable in the database. It could be copied into new headings. > Creation of heading like this: > > *** TitleRET > > would automatically invoke creation of heading 124 in the database and it > would appear as: > > *** Title :ID-124; > >
Re: One vs many directories
* Texas Cyberthal [2020-11-23 12:51]: > Hi Dr. Arne, > > > The only part that hits performance limits is the agenda. > > Well, IIRC your Org Textmind is much smaller than mine. > > > My current guess is that the agenta is slow because it has to parse all my > > 7500 clock entries, and it has to check the Todo states of around 1200 > > headings. > > Ouch. I'd rather keep a "ramble log" so I can reconstruct an exactly > honest time accounting, with discounts for partial attention, without > worrying about fiddly clockin/outs. At least when working from home. > If clocking into a work site, that's different, because one can > reasonably bill for the entire time, with minimal clock toggling. > > > Did you check against filesystem limits? At 10k entries in a > directory typical filesystems start becoming slow. That's the main > reason I see for adding hierarchies. >From ext4 manual: dir_index Use hashed b-trees to speed up name lookups in large directories. This feature is supported by ext3 and ext4 file systems, and is ignored by ext2 file systems. dir_nlink This ext4 feature allows more than 65000 subdirectories per directory. I think that file systems should be unlimited and fast in relation to that. I have ~/Maildir with over 5 subdirectories, direct access is very easy and fast while listing takes some time. If file system does not allow fast access it is time to replace it with one that does allow it. Now I wonder of HAMMER in DragonflyBSD is also slow with 5 directories. My PostgreSQL database is not huge, it is when packed about 50 MB. On the file system it is 810 MB. To select 2469 contacts as subset of 204048 contacts that belong in certain group does not give (usually) feeling of any delay, it looks instant for human. My Org work is on meta-level so my truly important headings or subtree names are in the database. Subtrees have their various properties, like I can place any tags there inside, like TODO or designate type of TODO. My work is intertwined with text and Org mode mostly, but I could use any kind of mime type or any kind of Emacs mode. Some nodes are on file system while some are in the database. Nodes within subtree are hyperdocuments, they are all linkable and could be on file system or not on file system. Everything is together in one tree and it does not matter as access to the nodes does not go over the tree necessary. There are 19197 nodes. To find 76 that are tagged with TODO does not give me any slight or visible delay, definitely not even 0.2 seconds. When I press enter it is right there. >From the system I am using personally I am thinking that Org mode could get its database connection so that headings and properties are managed in the database fully while text could be managed in files. It seems very possible. The only thing that would be needed to add to Org in that case is some heading tag that would uniquely designate where in the database that heading is managed. It could be very lightly displayed on the screen and would not be exported by default. Something like *** TODO Heading :ID-123: That would be all. All other meta data belonging to the heading could be managed in the database. If heading is deleted it need not be deleted in the database. Text belonging to heading could be managed in the text file. Properties in the database. It can be simple database such as GDBM if such is fast enough. Meta data for the heading would or could be updated automatically from time to time. User could easily decide to show the properties in the Org file or not to show. It does not matter much as long as :ID-123: tag is there. All things like tags, properties, clock-in and out, schedule, deadlines, custom_id and everything else as heading meta data could be manageable in the database. It could be copied into new headings. Creation of heading like this: *** TitleRET would automatically invoke creation of heading 124 in the database and it would appear as: *** Title :ID-124; >From there on user would be doing anything as usual in the Org mode with the difference that properties would be displayed in the updated manner and would not be really in the Org file. They would be displayed on the fly. Any properties and plethora of other new properties could be included. System would recognize automatically by saving the Org file or by opening it: - If headings are in the right file, if file changed its place it would be automatically updated in the database. - the heading ID would always remain unique no matter what, so users linking to any heading would not need to worry of title remaining. The unique ID that links to heading would basically link to the database entry. Opening the link would ask database where the entry is located and it would open up proper Org file at proper location without parsing the Org file in usual manner. Org file would then
Re: One vs many directories
Hi Dr. Arne, > The only part that hits performance limits is the agenda. Well, IIRC your Org Textmind is much smaller than mine. > My current guess is that the agenta is slow because it has to parse all my > 7500 clock entries, and it has to check the Todo states of around 1200 > headings. Ouch. I'd rather keep a "ramble log" so I can reconstruct an exactly honest time accounting, with discounts for partial attention, without worrying about fiddly clockin/outs. At least when working from home. If clocking into a work site, that's different, because one can reasonably bill for the entire time, with minimal clock toggling. > Did you check against filesystem limits? At 10k entries in a directory > typical filesystems start becoming slow. That’s the main reason I see for > adding hierarchies. 10k entries in a directory sounds inhumanely unergonomic. I guess my biggest flat name directory might eventually reach that size? In which case I could just split it in the middle of the alphabet, or similar solution. Right now it's only 600. If I guess a generous growth rate of 2 per day, times 30 years, that would be an additional 22k. Sounds manageable. Remember there are ways to consolidate entries even in flat "solid names" directory. It's advantageous to do so to facilitate isearch matching. For example, everyone with the same last name is one directory. Ditto everything that starts with the same word or even prefix. For example I have a directory called ~Wiki-~ and another called ~Tru-~ which contains truth, Trudeau and Trump. Most adults know 20-35k words. That's not the same as "solid names" known, but gives a ballpark on human memory size for a similar name type. I suspect computers will advance faster than anyone's Textmind reaches the Dired lag limit. No, if we are talking about scaling limits, then limits such as buffer size and Agenda search speed are orders of magnitude more relevant. Which problems deep tree nesting fixes. A 10k entry directory is getting into enterprise territory, and I'm sure enterprise has tech tricks that become worthwhile at that scale. > There are scaling problems in every direction: Too many files per directory, > too large files, too much content per heading, too many headings. There are scaling problems from too much deep tree nesting, namely too much fiddly ambiguous manual refiling. Solution is flat "solid name" directories just below feasible 10 Bins. Work fine. > I would have to build lots of additional tooling to make that work as well. > Many of the tools in Emacs work better on large files than on many files — I > will switch to more files when performance on large files reaches its limits. Nah, my 100 mb (non archived) Textmind works fine. I just separated Agenda metadata from bulk prose. I am curious how many headings I have, how would I count that recursively? On Sun, Nov 22, 2020 at 8:04 PM Dr. Arne Babenhauserheide wrote: > > > Texas Cyberthal writes: > > >> I need instant search in the knowledge database and quick filing of tasks. > >> Also I need the agenda to create a clocktable (that’s on the limit of > >> being too slow) and the calendar and tasks of the week. > > > >> Also I need quick filing of notes and quotes (in specific files, not part > >> of the agenda) and of long-form articles, one file per article (using > >> journal.el, also outside the agenda, searched using M-x deft), and quick > >> creation of website entries for a given category within the site (i.e. M-x > >> draketo-software). > > > > So your Org usage style quickly hits critical performance problems at scale. > > The only part that hits performance limits is the agenda. All the rest > scales nicely. My current guess is that the agenta is slow because it > has to parse all my 7500 clock entries, and it has to check the TODO > states of around 1200 headings. Having multiple files would only add to > that. > > > I don't have these problems. Treefactor refiling is immune to scale. > > Did you check against filesystem limits? At 10k entries in a directory > typical filesystems start becoming slow. That’s the main reason I see > for adding hierarchies. > > > Org's many tools and tricks are still handy in niche cases, but they > > don't cause scaling problems because they don't handle bulk info > > management. For example Org's refile tools are useful when writing > > advanced documentation with large single-file outlines. Most info > > doesn't require that much organization. It works fine as flat lists > > of headings in a detailed directory tree. > > Or as sub-headings in a large outline. > > There are scaling problems in every direction: Too many files per > directory, too large files, too much content per heading, too many > headings. > > I would have to build lots of additional tooling to make that work as > well. Many of the tools in Emacs work better on large files than on many > files — I will switch to more files when performance on large
Re: One vs many directories
>> I am wondering what you mean by Org's philosophy. Why would it have anything >> to do with directories? > > Org's philosophy is to have one or a handful of directories without > nesting of directories. Users are not expected to have their Org > files in a deeply nested tree. Org also prefers big files with large > trees rather than lots of little files. > > By philosophy, I mean the dev consensus on the correct way to do > things, and coded configuration and usability biases. I believe that org support all possibilities. The user can decide to keep many (possibly nested) org files, a few large org files, or anywhere in between. There are several parallel feature sets allowing to work in a single file as well as with a bunch of smaller files. For a single file, the user can search headings with org-goto (without a need to explicitly travel through all the nesting headline levels), reveal only headings satisfying certain keyword/tag/any other search criteria with org-sparse-tree, or built agenda views restricted to a single file (or even subtree). For multiple files located anywhere in the filesystem, there is always org-refile capable of filing the information to proper place or searching deeply nested headlines with ease regardless of the file the information is physically located in. Headlines from multiple files can be grouped using agenda views for any given search criteria (showing todo items or items for a single day/week is just a tiny subset of what agenda can do). Best, Ihor Texas Cyberthal writes: > * Hi Ihor Radchenko, > >> I am wondering what you mean by Org's philosophy. Why would it have anything >> to do with directories? > > Org's philosophy is to have one or a handful of directories without > nesting of directories. Users are not expected to have their Org > files in a deeply nested tree. Org also prefers big files with large > trees rather than lots of little files. > > By philosophy, I mean the dev consensus on the correct way to do > things, and coded configuration and usability biases. > > I know this is Org's philosophy because I violated it thoroughly when > writing Treefactor documentation, and was told as much. I can see how > it wouldn't be obvious to casual users. > > Good idea, I'll comment on Voit's article, thanks. > > * Hi Palak Mathur, > >> it seems overwhelming to have 10 directories. I am not saying it is not good >> just that I will not be able to handle those. > > I didn't anticipate this problem. Maybe practicing with Treefactor > and Dired would build this muscle over time. > > The rules are written to be straightforward at filing time. One need > only consider one object at a time. Cascade filing means one need > only compare the object with one directory at a time. The first match > wins. I should emphasize that in the docs. > > Having all your headings jumbled into three huge files sounds like a > state of permanent intractable overwhelm to me. > > * Hi Jean Louis, > > You are using Hyperscope as your primary PIM but integrating it with > Org, and it sounds like you're using PostGreSQL and the directory tree > together somehow. I don't fully understand it. > > Clearly a database can do more than a directory tree alone. The cost > is that a database is more complex to use and maintain. So that which > can be handled by directory tree alone, should be. > >> I can find a mining engineer in country Senegal if I wish so, that has some >> work experience and I can see files pertaining to this person. But not that >> I would make file system directory Senegal to find the files for this person > > Of course not. You would find a person under his name, not his > country. The person can move to a different country, after all. At > most you might link him to the country, as part of a list of people > from X country. But that is something better handled by a real > database. > > To clarify, Treefactor is just an Emacs package with some minimal > opinions. 10 Bins is an opinionated directory tree template. Neither > requires the other, but they're both part of Cyborganize, my overall > PIM and publishing system.
Re: One vs many directories
* briangpowell [2020-11-22 05:48]:
> Emacs, believe it or not, has the FASTEST ENGINE available, without
> augmentation in any way, for INTERACTIVE SEARCH--since the whole engine is
> designed to be optimized to search-while-editing
Interesting, I did not know about that.
> But for many other searches, more elaborate searches, fancier GREP
> searches, it's a VERY BAD choice of ways or programs to use for
> searching
Do you mean to run grep on file or buffer while editing?
Or outside of editor?
> What I mean is, say you're editing a file, and you search for your
> "ProviderBuilderFactory"
>
> Suggest you try opening a huge file--even MULTI-GIGABYTE FILES--huge files
> in Emacs VLF-Mode--Very Large File-mode {which I believe can be done as a
> sub-mode to/with Org-Mode}
>
> And you can do this fearlessly since vlf-mode works by dividing the files
> up for you in the background, etc.--while you're editing--but uses the same
> built-in Emacs engine, optimized for such searches
>
> And then you type:
>
> Control-s
>
> And start to type the first letters of "ProviderBuilderFactory"
>
> This will interactive-search HUGE files, very quickly, and in near "Real
> Time"--since this is what Emacs (implemented in C) is optimized to do--its
> optimized for initial-character-searching "as you type them"--most other
> engines are NOT
Does occur also uses that built-in speedy search? Would it work with
your library? Now I am researching it and I see vlf-occur
I have tried using vlf-occur in this email and what happened seems to
be error:
- M-x vlf-occur
- Tried with word "has"
- found many "has" in vlf-occur
- I press RET on the line there
- I get asked "Chunk modified, are you sure?" I say yes.
- my email buffer is erased at that moment and cursor switches to erased buffer
- then to undo stuff I press undo
So that is bug. User should never get buffer erased.
> IF FOR NO OTHER REASON THAN IT SOUNDS LIKE FUN! And you might use
> vlf-mode for other tasks you may face in the future.
I am interested to research the possibility of expanding large number
of database items into the file which would then be hyperlinked. Would
some mode like Org mode work with it? I would use Hyperbole or Org
links or goto-address-mode
The file would be one liner expansion of database entries.
By using quick vlf-occur (at this moment I just imagine it is quick,
but do not have a feeling) I would quickly locate some entries.
I would then click or activate the button to move to specific other
database entries or outside hyperlinks.
> Lastly, say you want to search for things without opening a file, you can
> still use Emacs in batch-mode, at the command line, without opening a full
> emacs session
Provide the one line to understand it.
Side thought: I remember I was making website search engines back in
time with grep only. Each HTML file was converted to one line of text,
something like:
LOCATION:::TITLE:::KEYWORDS:::DESCRIPTION:::HERE COMES LONG LINE OF BODY
Then grep was used to quickly find results which are formatted on the
website page. And yet we can just find complex software for website
searching on Internet these days.
Re: One vs many directories
* Ihor Radchenko [2020-11-22 11:33]:
> > * Full contact information is required
> > :PROPERTIES:
> > :CREATED: [2018-10-08 Mon 21:34]
> > :ID: 06781e66-0382-4833-a61e-0d76e317593f
> > :END:
> >
> > Thank you. Am I supposed to declare these properties in
> > org-custom-properties for it not to be visible?
>
> Yes. I use
>
> (setq org-custom-properties '("ID" "STYLE" "REPEAT_TO_STATE"
> "CREATED"))
I've tried then :PROPERTIES and :END: still remain. It is possible
even not to show PROPERTIES but :END remains.
> Then, you will need to run M-x org-toggle-custom-properties-visibility
> or add it to org-mode-hook.
Thank you, I will rather give up on that. It is not designed for it.
> Hiding the emptied property drawer is currently not possible, unless you
> use my WIP patch [1]. The branch also mitigates the issue with large org
> files. However, this particular functionality was opposed by other devs
> earlier. Further discussion will follow once more important parts of the
> patch are resolved.
>
> If you use the patch, you can also set
> org-custom-properties-hide-emptied-drawers to 't. Then, your example
> should look like
>
> * Full contact information is required
Thank you, I will choose to give up on that as it has not get much
priority and maybe I stop creating unique IDs.
I will explore how to re-file tasks into the SQL database.
Re: One vs many directories
> * Full contact information is required
> :PROPERTIES:
> :CREATED: [2018-10-08 Mon 21:34]
> :ID: 06781e66-0382-4833-a61e-0d76e317593f
> :END:
>
> Thank you. Am I supposed to declare these properties in
> org-custom-properties for it not to be visible?
Yes. I use
(setq org-custom-properties '("ID" "STYLE" "REPEAT_TO_STATE" "CREATED"))
Then, you will need to run M-x org-toggle-custom-properties-visibility
or add it to org-mode-hook.
Your example will then look like
* Full contact information is required
:PROPERTIES:
:END:
Beware of using this on large org files. The command creates a huge
number of overlays (one overlay per property line), which may slow down
Emacs.
Hiding the emptied property drawer is currently not possible, unless you
use my WIP patch [1]. The branch also mitigates the issue with large org
files. However, this particular functionality was opposed by other devs
earlier. Further discussion will follow once more important parts of the
patch are resolved.
If you use the patch, you can also set
org-custom-properties-hide-emptied-drawers to 't. Then, your example
should look like
* Full contact information is required
[1] https://orgmode.org/list/87lfh5vvrp.fsf@localhost/
Best,
Ihor
Jean Louis writes:
> * Ihor Radchenko [2020-11-22 09:37]:
>> > In fact the properties, custom ID and else inside is mostly visually
>> > disturbing me though it is necessary.
>>
>> FYI: org-custom-properties and
>> org-toggle-custom-properties-visibility
>
> * Full contact information is required
> :PROPERTIES:
> :CREATED: [2018-10-08 Mon 21:34]
> :ID: 06781e66-0382-4833-a61e-0d76e317593f
> :END:
>
> Thank you. Am I supposed to declare these properties in
> org-custom-properties for it not to be visible?
Re: One vs many directories
* Ihor Radchenko [2020-11-22 09:37]: > > In fact the properties, custom ID and else inside is mostly visually > > disturbing me though it is necessary. > > FYI: org-custom-properties and > org-toggle-custom-properties-visibility * Full contact information is required :PROPERTIES: :CREATED: [2018-10-08 Mon 21:34] :ID: 06781e66-0382-4833-a61e-0d76e317593f :END: Thank you. Am I supposed to declare these properties in org-custom-properties for it not to be visible?
Re: One vs many directories
> In fact the properties, custom ID and else inside is mostly visually > disturbing me though it is necessary. FYI: org-custom-properties and org-toggle-custom-properties-visibility Jean Louis writes: > * Texas Cyberthal [2020-11-21 18:46]: >> I guess I avoid the problem you're talking about by mostly excluding >> bulk prose from the Agenda directory. They're fundamentally different >> and should be handled differently. > > Well said. > >> One is about human language, the other is about database metadata. > > That is so. > > In fact the properties, custom ID and else inside is mostly visually > disturbing me though it is necessary. Before knowing about Org I have > already had a system of planning, projects, tasks, and all that could > be related to people, groups and what other else things. This part of > Org that is dependant on meta data is better managed through > dedicated databases. > >> The temptation to do everything inline just because Org supports it >> is one of Org's biggest traps. It's like the trap of trying to >> outline strictly when you should be rambling. > > Temptation is brought through the usage of Org as there are not so > many choices beyond it. Org is meant to keep simple things > simple. Data being managed may easily grow into complex thing.
Re: One vs many directories
* Strongly suggest looking into Emacs' vlf-mode and the newer vlfi-mode
** That is Very-Large-File-Mode & Very-Large-File-Improved-Mode for issues
you're experiencing & if not, simply because they're very useful &
interesting & fun Emacs Modes to explore & put into your toolbox
https://www.emacswiki.org/emacs/VLF
https://github.com/m00natic/vlfi
* You mentioned other types of GREP, I second that, and the indexing
suggestion--I remember long ago using SGREP which is Simple-GREP, used
indexing & was much faster than the usual grep implementations for some
things; but, this is at the expense of the fancier & more elaborate GREP
functions
** You mention RipGrep--thanks for that, very interesting
* Which brings me to my main suggestion to you & why:
Emacs, believe it or not, has the FASTEST ENGINE available, without
augmentation in any way, for INTERACTIVE SEARCH--since the whole engine is
designed to be optimized to search-while-editing
But for many other searches, more elaborate searches, fancier GREP
searches, it's a VERY BAD choice of ways or programs to use for searching
What I mean is, say you're editing a file, and you search for your
"ProviderBuilderFactory"
Suggest you try opening a huge file--even MULTI-GIGABYTE FILES--huge files
in Emacs VLF-Mode--Very Large File-mode {which I believe can be done as a
sub-mode to/with Org-Mode}
And you can do this fearlessly since vlf-mode works by dividing the files
up for you in the background, etc.--while you're editing--but uses the same
built-in Emacs engine, optimized for such searches
And then you type:
Control-s
And start to type the first letters of "ProviderBuilderFactory"
This will interactive-search HUGE files, very quickly, and in near "Real
Time"--since this is what Emacs (implemented in C) is optimized to do--its
optimized for initial-character-searching "as you type them"--most other
engines are NOT
IF FOR NO OTHER REASON THAN IT SOUNDS LIKE FUN! And you might use vlf-mode
for other tasks you may face in the future.
Please try it out & tell how you like it--you'll never avoid opening huge
files again is one benefit
Beyond that, suggest you look into using LEX, it's as fast as you can get
for some things too.
Everything has its niche in the *nix world--which is where GREP came from,
Bell Labs, etc.--that's the Unix philosophy, Emacs & LEX tools came from
that world & the work of thousands of contributors--suggest you try these
tools too for these issues
Lastly, say you want to search for things without opening a file, you can
still use Emacs in batch-mode, at the command line, without opening a full
emacs session
On Sat, Nov 21, 2020 at 8:34 PM Jean Louis wrote:
> * Dr. Arne Babenhauserheide [2020-11-22 01:48]:
> > > So in general I never need to use some general search through Org
> > > files or any other files as my way of thinking begins with People or
> > > Groups and that narrows what has to be searched.
> >
> > How do you deal with stuff that applies to several people?
>
> From database viewpoint there are
>
> - accounts (which means companies, groups, entities, like "People who
> wish to get employed in Europe")
>
> - there are contacts, that may belong to account, additionally belong
> to company (also account), additionally be member of account, so
> there are 3 groupings for each contact how that contact may be
> related to account. If it is main account such as "Welders" or if
> maybe under "Company" is written "welders" (not quite correct) in
> reality it does not matter.
>
> - then there are lists to which other lists belong. Account A and
> account B, C, D can belong to list 01. Various accounts can be put
> together in uniting lists. Those lists are encompassing other lists,
> not individual people but people in the list (account) usually
> unless there is only one in the account. Those lists I am using for
> mailing them or informing them by letter, SMS, etc. Geologists and
> mining engineers and metallurgists are 3 different accounts but if
> all of them speak Swahili both in Kenya and Tanzania and are in the
> related branch of economy so they can be sent same type of
> information.
>
> Then there are groups, which is just another name for a new list. Then
> there are tags. I can freely tag account, contact or anything else. By
> tags I can finely select specific people belonging to specific group.
>
> There are account types and group types.
>
> Tags by itself have its own description or purpose to name it type.
>
> Some people introduce other people, few of them introduced
> thousands. So contacts have a column "introduced by". That becomes
> very handy when talking to somebody and it also helps in awarding
> introduces. It helps when people place their hyperlinks and become
> automated introducers (lead generation).
>
> When I know that person belongs to some group of people and I have to
> write email and I know it is better to inform everybody, then there is
Re: One vs many directories
* Dr. Arne Babenhauserheide [2020-11-22 01:48]: > > So in general I never need to use some general search through Org > > files or any other files as my way of thinking begins with People or > > Groups and that narrows what has to be searched. > > How do you deal with stuff that applies to several people? >From database viewpoint there are - accounts (which means companies, groups, entities, like "People who wish to get employed in Europe") - there are contacts, that may belong to account, additionally belong to company (also account), additionally be member of account, so there are 3 groupings for each contact how that contact may be related to account. If it is main account such as "Welders" or if maybe under "Company" is written "welders" (not quite correct) in reality it does not matter. - then there are lists to which other lists belong. Account A and account B, C, D can belong to list 01. Various accounts can be put together in uniting lists. Those lists are encompassing other lists, not individual people but people in the list (account) usually unless there is only one in the account. Those lists I am using for mailing them or informing them by letter, SMS, etc. Geologists and mining engineers and metallurgists are 3 different accounts but if all of them speak Swahili both in Kenya and Tanzania and are in the related branch of economy so they can be sent same type of information. Then there are groups, which is just another name for a new list. Then there are tags. I can freely tag account, contact or anything else. By tags I can finely select specific people belonging to specific group. There are account types and group types. Tags by itself have its own description or purpose to name it type. Some people introduce other people, few of them introduced thousands. So contacts have a column "introduced by". That becomes very handy when talking to somebody and it also helps in awarding introduces. It helps when people place their hyperlinks and become automated introducers (lead generation). When I know that person belongs to some group of people and I have to write email and I know it is better to inform everybody, then there is action to assign identity from which I am sending email. It could be public or internal identity with different email address. Emails to that person would always go from designated email address without me thinking about it. Then there are Cc and Bcc fields and in those fields I could designate: to inform same contact to each of email addresses with same message, and to inform group of people each time. Thus if writing to one contact all others get informed through same message. But I am not thinking about who belongs to the group, and what are their email addresses, that gets inserted automatically. Email composition is usually inside of Emacs. Sending files? If contact is in the group and others need to see the file, then Cc and Bcc fields work in the same way, file sent to one person is sent to other members of the group. Sometimes contact tells me to please exclude some people from Cc, so I just go into definition of identities for contact and exclude such. SMS I am sending by using kdeconnect.el package so any SMS I send this way it is getting recorded to the contact. If I need to send SMS to the group, then same SMS could be sent to whole group. If the note on one contact is related to other people then is trivial to automate to inform other people on that particular note. If any group of people is there, filing files under group does not make much sense to me. I would not file it as Group/ABC/file-01.pdf I would file it under By-ID/Contact-ID-123/Group/file-01.pdf as files are normally sent by one email address of one person or under one name. I deal with people not with empty groups that do not communicate. But group is important, so there can be /Group/ directory on the file system where all contacts of one group can be symlinked automatically if it happens that I need to search by Group on file system, but I don't. I search for groups in the database and see list of contacts there and then jump to contact. Same thing with invoices, they are written either to company or to individual or some individual in some company. I will not file or act based on invoice. I have to act based on authorirative or maybe hierarchically higher object first. In general is always good to make a list of things and then lists are foundation for dealing easier with whatever groups of anything. > > it comfortable. My way of thinking is always People or Groups, and > > from there various searches are performed and that narrows drastically > > the subject that has to be searched. > > That does sound like it should speed up searching by directory. You remember that I do not search by directory. Computer stuff I search by directory, like Work/Computer/Devices/Dictaphone or Work/Computer/Graphics/shadow Stuff related to any entity,
Re: One vs many directories
Jean Louis writes: > So in general I never need to use some general search through Org > files or any other files as my way of thinking begins with People or > Groups and that narrows what has to be searched. How do you deal with stuff that applies to several people? > it comfortable. My way of thinking is always People or Groups, and > from there various searches are performed and that narrows drastically > the subject that has to be searched. That does sound like it should speed up searching by directory. My mind works differently here: I remember some concept and need to find stuff connected to that, including people, but also additional ideas, instructions, and just bullet points with info I might need again later (which multiple times saved many hours of searching already). The one thing that keeps me from that is that I often file under specific projects, and the active projects are shifting constantly. For that I enjoy it a lot that I only need to customize the capture templates to add a project. > On my side I almost never put notes in Org files. As by definition > from Wordnet, note is "brief written record". With note I just mean "something". Mostly its bullet points with tasks, some links and references into source-code which allows me to quickly take up a tasks after some downtime. > Overall from this discussion I hope that people find some useful ways > of using Org, like org-rifle, semantic organization of stuff and > similar. I hope so, too. Thank you for describing the tools you use! I for one am still working on my workflow, and I guess that 10 years from now it won’t be the same as today. I hope that learning about other ways to work with org will help me a lot in future. Best wishes, Arne -- Unpolitisch sein heißt politisch sein ohne es zu merken signature.asc Description: PGP signature
Re: One vs many directories
Hi Texas, > Grepping my 94 Mb 6562 files (excluding archive) Textmind for > "elephantine" takes a few seconds, which is fine. For the sake of ruining my argument ( :-) ), you might want to check ripgrep. Searching within 30k files of in total around 150 MiB for ProviderBuilderFactory (guess what type the files are :-) ) takes 0.4s with ripgrep. (that’s on an nvme (M.2) disk) It’s still too slow for interactive use. Within 1k files of in tootal 7 MiB it is fast enough. > Org searching for a nonexistent UID link takes a few minutes, which is > why I run that search nightly, to refresh the index. My Org Agenda > search scope is only 252k in 58 files and is effectively lagless. That lagless is what I see as being required for actual operation. > I'm not sure what kind of lagless Org database operations are required > in your workflow, but I suspect they could be mostly replaced by a > proper Textmind workflow and 10 Bins structure. I need instant search in the knowledge database and quick filing of tasks. Also I need the agenda to create a clocktable (that’s on the limit of being too slow) and the calendar and tasks of the week. Also I need quick filing of notes and quotes (in specific files, not part of the agenda) and of long-form articles, one file per article (using journal.el, also outside the agenda, searched using M-x deft), and quick creation of website entries for a given category within the site (i.e. M-x draketo-software). > I guess I avoid the problem you're talking about by mostly excluding > bulk prose from the Agenda directory. They're fundamentally different > and should be handled differently. I do that, too. One is source code, one is organisation tasks. I link from org into the source code, though (but never check the targets). I do use org-planning within prose, but there the scope is only the one org-document. Best wishes, Arne -- Unpolitisch sein heißt politisch sein ohne es zu merken signature.asc Description: PGP signature
Re: One vs many directories
* Texas Cyberthal [2020-11-21 21:02]:
> Hi Jean,
>
> > That is good and isn't it general way of sorting things? I guess
> > that general computer users may not be aware that they could make
> > nice hierarchical tree of directories.
>
> It's not that they're unaware. Everybody with a mouse and Windows
> Explorer tries to make good directories.
People try and people fail. I know hackers only on distance, I do not
know them personally face to face. Often computer users that I see in
offices like stationary, even sometimes legal office, have their
desktops overloaded of files on top of files on top of files where
majority of them are named something like unknown or whatever new-file
and similar nonsense. All sorting is taking place under Desktop on
Desktop space and it becomes mess.
> It's just that Dired+Treefactor's order of magnitude improvement in
> speed and fluency means that the directory tree can be mind-synced
Good term, mind synced. That is how it should be. In my opinion
various paradigms of file sorting should be sorted in a list similar
to those Awesome lists. Sorting files how mind thinks is how it should
be. Computer should help user to complement the mind and help in
execution of actions and not bother the mind or make mind confused
what is really the case with majority of users.
Side note: I was always sorting files, people, stuff from the command
line or from web browser into their places. And I mostly used readline
in bash for completion or finding of the selection candidate.
In Emacs I am using `ivy-mode` or `helm`, `selectrum` and similar,
Emacs offers quite good interface for completions.
And often I am completing file locations, maybe I wish to file into
this or the other subject. So the list of subjects has to be brought
up.
In X and outside of Emacs there is dmenu and I could use it even from
Emacs inside out.
(defun dmenu-completing-read (prompt collection)
;; predicate require-match initial-input history def
inherit-input-method)
"Uses external `dmenu' command for Emacs completions."
(let* ((collection (concat (string-join collection "\n") "\n"))
(file (string-to-file-force collection "/dev/shm/collection"))
(dmenu "dmenu"))
(with-temp-buffer
(call-process dmenu file (current-buffer) nil "-fn" "DejaVu:pixelsize=30"
"-l" "10" "-i" "-b" "-p" prompt "-nb" "dark goldenrod" "-nb" "black")
(string-trim (buffer-string)
(dmenu-completing-read "Subject: " '("People" "Work" "Group"))
I find dmenu as excellent tool to provide it for functions that are used within
Emacs and that may be used outside of Emacs to complement Emacs filing of files.
For example Rox filer file manager or Nautilus can invoke external command that
files the file into specific directory by using dmenu. Choose the directory by
few words, and if directory is semnatically written, file may be filed easily.
Three interfaces for selection among large line-based items were needed:
- [X] Emacs
- [X] X interface: Dmenu
- [ ] Console. I was using readline with TAB, it is not visual enough
Now I found finally `fzf`.
$ mv courier-crm114 `find Work/Computer -type d | fzf`
Then in the next step I can type "mail" or "spam" subdirectories and file it
there.
fzf is great tool for console and terminal emulators. It similar
things on console what helm does on Emacs and dmenu on X.
- [X] Console: fzf
fzf is great tool that may be used for finding stuff, filing, retrieval of
stuff, launching or opening files. Example:
$ mimeopen `locate *.org | fzf`
and similar with dmenu:
$ mimeopen `locate -d .locate.database -A Documents .org | dmenu -fn
DejaVu:pixelsize=20 -l 10 -i -b -p "Org: " -nb "dark goldenrod" -nb black `
As my files are in ~/Documents/Org/ I would find at least 185
files ending with .org there by its name and quickly open it with
emacs. Instead of "mimeopen" I could as well use emacsclient.
If you have set your mimeopen to open it by Emacs, you will
quickly locate the file and open it. Provided that Org file names
have some built-in semantics.
Re: One vs many directories
Hi Jean, > That is good and isn't it general way of sorting things? I guess that general > computer users may not be aware that they could make nice hierarchical tree > of directories. It's not that they're unaware. Everybody with a mouse and Windows Explorer tries to make good directories. It's just that Dired+Treefactor's order of magnitude improvement in speed and fluency means that the directory tree can be mind-synced as never before, making walking the tree an education in itself. With so much intelligence embedded in each level, bypassing it makes little sense. Also one should check during the walk whether key info is waiting in transit for batch refiling. For classic database tasks, I'd rather use Postgres than symlinks. I already use symlinks enough for conveniences such as connecting synonyms. Too many symlinks make paths unpredictably brittle, chilling sync dynamism, creating rigidity. Textmind is documented at http://cyberthal-docs.nfshost.com I neglected Dr. Arne's honorific.
Re: One vs many directories
* Texas Cyberthal [2020-11-21 18:46]: > I guess I avoid the problem you're talking about by mostly excluding > bulk prose from the Agenda directory. They're fundamentally different > and should be handled differently. Well said. > One is about human language, the other is about database metadata. That is so. In fact the properties, custom ID and else inside is mostly visually disturbing me though it is necessary. Before knowing about Org I have already had a system of planning, projects, tasks, and all that could be related to people, groups and what other else things. This part of Org that is dependant on meta data is better managed through dedicated databases. > The temptation to do everything inline just because Org supports it > is one of Org's biggest traps. It's like the trap of trying to > outline strictly when you should be rambling. Temptation is brought through the usage of Org as there are not so many choices beyond it. Org is meant to keep simple things simple. Data being managed may easily grow into complex thing.
Re: One vs many directories
* Dr. Arne Babenhauserheide [2020-11-21 18:04]: > > Jean Louis writes: > > > When there are more than 2000 people related notes, tasks, > > calculations, questions arise if such better be kept in one Org file > > or multiple Org files in one directory or multiple directories for > > multiple Org files?! > > This came up multiple times in discussions. I think that it is wrong, > and the reason comes down to efficiency. If you want to work > efficiently, then sub-second delays matter, and having 4 files with > 500 entries means that to search in them you need to open 4 files. Hallo Dr. Arne, It maybe wrong and it depends of the approach. My approach is that I think with people and subjects, not with notes only. Subject can be special plan like ABC.org and I do not need to search notes related to that plan outside of ABC, because I do not mix things. I am searching within one file only. Things TODO are per subject or per person. Files pertaining to any person are filed in the person's folder. Somebody else deals only with personal notes and they maybe put such in various files and of course they need to search. I am thinking of "Joe Doe" and here is the flow: - press s-c (for contacts) - enter Joe Doe or Joe Doe, Berlin, etc. - among many Joe Doe, I may narrow down to right one - click F4 there is Org file for Joe Doe, enter Tasks, Transactions and whatever else, send Tasks, Notes to Joe Doe, collaborate or make agreements. I never construct or open file for a person, function is doing that. It makes the file ~/Work/People/By-ID/320431/320431.org If I need to search, I search inside of the file. - click F5 and find all other files for Joe Doe. For example contracts and similar. If I need to search there then I use find and grep and similar tools. No need for indexing. Files are mostly sorted by data how they come. There is same flow if I think of a group of people with the difference that if I need a person I still need to find the person in the list of people. So in general I never need to use some general search through Org files or any other files as my way of thinking begins with People or Groups and that narrows what has to be searched. > If you have 100 files with 20 notes each, you have to open 100 > files. You maybe mean opening automatically files and searching through such. I do not find Org system comfortable for that. I see it tries to remember files, IDs, and agenda among various files. Not that I find it comfortable. My way of thinking is always People or Groups, and from there various searches are performed and that narrows drastically the subject that has to be searched. > My current setup has around 1200 notes in 10 files (most of them in the > two main files, some of the notes are several pages long, but most take > up around half a page). People are all over the world using Org in various manners and every day I find different ways of using Org mode. On my side I almost never put notes in Org files. As by definition from Wordnet, note is "brief written record". If it is brief written record I do record it in the database under Notes related to person, or group or opportunity or some case, or it can be related to anything else. Then again I think of person and I can get all notes for the person. Org files I am using mostly for planning and project administration. There are almost no notes, just instructions on how to execute specific steps and there are headings with articles or instructions that do not need execution. There are no records that are saved for later or that do not need any execution or learning. Org files on my side thus offer: - hierarchical knowledge database that may be shared with other people, and is almost always directed to sharing with other people - plan and project administration with tasks, whereby such subtrees can be shared with other people and for multiple times executed If those are called notes by other people, alright fine. On my side those are not just notes. Notes I relate to objects like People, Groups, Opportunities, Cases, so I put some notes there. But general dynamical knowledge repository is better, that is where I mention HyperScope. It is like database of hyperlinks that hyperlink to anything, it is more abstract and I find that approach also versatile. No need to define specific database for notes, all I do is defining hyperdocument type to be "Note" and I can link it to anything else. Semantic Synchrony https://github.com/synchrony/smsn Semantic Synchrony is using maybe better type of a database I do not know, I am using SQL, SMSN uses graph database. > Using org-rifle (https://github.com/alphapapa/org-rifle) I can > full-text-search them with barely perceptible delay on a system > clocked down to 1 GHz. That is great tool for many. Org files are for me to write complex documents like 850 kb something like a organizational knowledge, training for each staff member, plans, projects, tasks,
Re: One vs many directories
* Texas Cyberthal [2020-11-21 18:01]: > Hi Jean, > > > Navigating does not necessarily contribute to production. Productivity may > > say what it wants but it may not reach those who are actually more > > productive without using the navigation. So studies may not tell us what is > > more productive, such may only tell what is currently used within the > > subject of being productive. > > Outside 10 Bins, navigation is often negatively productive. Whenever > the user navigates bad tree structure without correcting it on the > fly, he suffers profitless friction. That's why Treefactor combines > with JiT sorting to make navigation and sorting a single activity. > > Frankly I was surprised people prefer navigation despite being > generally so bad at tree structure. In the absence of good structure > and tools, search is much better. Do you really think people prefer? Or they simple have no other option? If I have no other option to drink a juice I will drink water, not necessarily I prefer drinking water in hot sunshine. I like Apfelschörle. Searching file contents is database search. I am not any more fan of that. Tools like Beagle or Tracker are making basically double database of files only for me to find or search files. Most of time I am only searching for meta data, not for contents. With 1000 GB one would need to have maybe that much indexed database and constant updating of files and their positions. That is why I prefer relational pinpointing or relational access and actions or locating files by using indexes. Relational access would mean when you inspect the object to quickly jump to all relational pieces of information relating to that object. If I look on person's name there need not be any note or contact information but I should be able to quickly access such information. And each object should have general actions like actions relating to email object could be to send email, delete email, forward, etc. that is what mail readers do. Action on file relating to user should be to quickly see emails of the user, social networks, websites, to call user, SMS, send information, jump into XMPP chat, share the file or request new version of file and similar. Locating files by indexes would be to index only meta data and then to use searches to find meta data such as title, date, author, or various attributes. Tools like locate and similar do that. > I agree, email should be database-centric. Manually organizing > emails into folders (or worse, trees) is therefore wrong except in > tiny doses. You missed this: - I am reading email, answering and if I wish to keep it all I do is `s` to save it into the mailbox related to the email address: ~/Maildir/te...@example.com Emails that I send to user are saved to same mailbox. That is all. No thinking, nothing, saving goes automatic. This single plan of filing emails enables me to see all conversations related to that email address. Database keeps all email addresses of specific person $ emailsof texas@examp would give me 3 views if there are 3 email addresses, I could basically review all conversations of that person. There need not be any true database like `mu` or `notmuch` as this way I find most of times what I need. I can search inside of mailboxes easily. I would not keep emails in the database like PostgreSQL, no. Emails have to be accessed by mail readers and there are just few that would be supporting databases. > > Take care of duplicates. When marketing contact database is > > growing fast, some times 1000 people per day or more. People have > > same names. Often one cannot even know what is first and what is > > last name. You may know it for your country, in other countries is > > not so. Then those people engage in a course on distance. They are > > sending me images and files as results of their course > > assignments. I have to file the files in proper folder. Because > > names are not always unique I better file it under unique IDs, and > > keep separate symlinks with names of people to such unique IDs > > whereby symlinks can be automatically updated. > > This is clearly a CRM database use case. In this situation, the CRM > should define the unique ID, and then Textmind should accept it. You > can still use the directory tree, though. Just file it under > ~/Surname-given-name/ID-number/~ for the non-unique name. Recursively > searching ~/1-Textmind/2-Linked/7-Name/2-Flat/~ for a directory named > ~/ID-number/~ will find the target even if his name changes slightly. > So you can save time by not inputting every scrap of text and files > into the CRM. That is right, good idea to file under surname/ID. I would rather prefer the approach ID/Full-Name as if directory is automatically created by its ID, so I do not need to think about it. CRM is for me not the database, it is method of management of anything related to people and I call it Central Files. I do not put text files
