Re: Security issues in Emacs packages

2020-11-25 Thread Tim Cross


Jean Louis  writes:

> * Tim Cross  [2020-11-26 02:40]:
>> > OK it is great that it is so. Are you maybe author doing it? Is there
>> > any reference that authors are doing so? I have MELPA downloaded you
>> > could tell me how do I see that author is deciding if package is for
>> > release?
>> >
>>
>> You can clone the melpa repository and see the recipes for each
>> package.
>
> I did before some time.
>
>> It depends on how the author specifies their MELPA recipe. They can
>> define their recipe based on a specific commit (SHA). If they do this,
>> it doesn't matter how often or when MELPA pulls from the repository as
>> they will always get the same commit.
>
> I have not seen that, and I have assumed you would know better and
> wanted to see how authors are reporting that package is ready for
> release and I do not see that.
>
> Recipes are like this:
>
> (0blayout :repo "etu/0blayout-mode" :fetcher github)
>
> (0x0 :url "https://git.sr.ht/~zge/nullpointer-emacs; :fetcher git)
>
> (0xc :fetcher github :repo "AdamNiederer/0xc")
>
> So that recipe alone does not tell me that author reports that new
> package is ready, it is fetched from git, but there are parts of code
> that I did not see that is why I am assuming you know it better.
>
>> Your model is flawed. You can have both automatic pulling AND author
>> control over when a new package is issued.
>
> To make it practical tell me where is that author's control?
>
> I have quick view of files and any recipe files in directory
> melpa/recipes do not give me any pointers, it is all automated and
> fetched from git.
>
>> If author defines their MELPA recipe to use a SHA a new package will not
>> be issued until they update their recipe with a new SHA.
>
> You seem to be very confident and for this reason I assume you know it
> better, but due to contradictions please show one practical recipe or
> package where author has control on when is package ready to be
> released.
>
> $ grep sha *
>
> on recipes does not give any reference.
>
> $ grep commit *
>
> eval-in-repl:  :commit  "origin/master")
> git-auto-commit-mode:(git-auto-commit-mode :fetcher github :repo 
> "ryuslash/git-auto-commit-mode")
> git-commit:(git-commit :fetcher github
> git-commit::files ("lisp/git-commit.el")
> git-commit::old-names (git-commit-mode))
> git-commit-insert-issue:(git-commit-insert-issue :fetcher gitlab :repo 
> "emacs-stuff/git-commit-insert-issue")
> vc-auto-commit:(vc-auto-commit :fetcher github :repo "thisirs/vc-auto-commit")
> what-the-commit:(what-the-commit :fetcher github
> what-the-commit: :repo "danielbarbarito/what-the-commit.el")
>
> So there is nothing I can find that points or references to what you
> say.
>
>> If author defines their MELPA recipe to pull from a release branch, a
>> new package will not be issued until they update the release branch and
>> version tag.
>
> I am sorry I do not see reference to it. You are convincing but I do
> not see reference.
>
> Recipe for bar-cursor:
> (bar-cursor :repo "ajsquared/bar-cursor"
> :fetcher github)
>
> Recipe for magit:
>
> (magit :fetcher github
>:repo "magit/magit"
>:files ("lisp/magit"
>"lisp/magit*.el"
>"lisp/git-rebase.el"
>"Documentation/magit.texi"
>"Documentation/AUTHORS.md"
>"LICENSE"
>  (:exclude "lisp/magit-libgit.el"
>;; Cannot remove this yet because it would
>;; also be removed from the stable version.
>;; "lisp/magit-section.el"
>)))
>
> Repo magit/magit:
> https://github.com/magit/magit
>
> I have given you references, maybe I cannot read that well, so you can
> give me references to show if authors have participation in decision.
>

The available recipe options are all clearly documented in the README
for the melpa repository. Most maintainers don't use the :commit option
because it is extremely inconvenient, but it is there if they want it.
It is inconvenient because it means the recipe has to be updated, which
means a pull request has to be accepted before the package can be released.

Most maintainers will maintain a specific branch for releases. This is
normal practice in version control. Often, this is the master branch,
but 'release' and 'melpa' are also commonly used. Code is not pushed
onto these branches until it is ready for release. The package
maintainer has full control of this branch and therefore has full
control over when new code is released. This is also the model used by
GNU ELPA for external packages.

This is not the model you imply, where MELPA just grabs the data
whenever it wants and releases new version without management by the
package maintainer, resulting in the release of code that is not ready
for release.

--
Tim Cross



Re: Security issues in Emacs packages

2020-11-25 Thread Tim Cross


Greg Minshall  writes:

> Tim,
>
>> I think you missed my point. There is no benefit in MELPA adopting
>> signed packages because there is no formal code review and no vetting
>> of the individuals who submit the code.
>
> it occurs to me there might be one benefit: if George, whom you trust,
> says, "I've been running version 1.2.3 of package xYandZ from MELPA and
> i have a lot of confidence in it", then if you find that version of that
> package with a trusted MELPA signature, you maybe know that you and
> George are running the same software.  i.e., it helps with the "web of
> trust" (if people still talk of that).
>
> (so, the requirement for this is not audited packages, but a solid,
> "secure", release procedure by MELPA.)
>

It could, but to get that level of assurance, you not only have to
verify the signature is valid (something which is automated if enabled),
you also need to verify that both packages have the exact same
signature, which is pretty much a manual process. So in addition to
telling you the version number, George would also need to communicate
the signature and that would need to be compared to the signature you
have in the package you downloaded to know that the packages are in fact
the same (you cannot rely on version numbers for any real verification).

Signatures are a good thing and MELPA should implement them. However,
what they are really useful for is ensuring the package you have
downloaded has not been modified since it was created and signed.

--
Tim Cross



Re: Local variables insecurities - Re: One vs many directories

2020-11-25 Thread Tom Gillespie
> As there is the option ! to "apply local variables and permanently
> mark these values" but there is no option "not to apply local
> variables and permanently mark these values".

I have a longer reply that I will send tomorrow, but wanted to respond
to this.

Yes exactly! I have the equivalent complaint in the draft extras from
my previous message! I actually implemented a blacklist for file local
variables in orgstrap because it is a critical security feature. The
fact that it is missing is absolutely a bug and is extremely annoying
for some of my current workflows where I have local variables that I
never want to accept.



Re: Local variables insecurities - Re: One vs many directories

2020-11-25 Thread Jean Louis
Additionally, as a good example of faulty design, user is coerced to
ACCEPT local variables rather than is given fair choice.

As there is the option ! to "apply local variables and permanently
mark these values" but there is no option "not to apply local
variables and permanently mark these values".

That is not fair choice. It pushes user to finally ! apply and accept it, but
does not give chance to permanently ignore it.


Do you want to apply it?  You can type
y  -- to apply the local variables list.
n  -- to ignore the local variables list.
!  -- to apply the local variables list, and permanently mark these
  values (*) as safe (in the future, they will be set automatically.)



Re: Local variables insecurities - Re: One vs many directories

2020-11-25 Thread Greg Minshall
Tom,

> 2. If mutt is launching Emacs, you can pass --eval "(setq
>enable-local-eval nil)" on the command line and all file local
>variables will be ignored and treated as plain text.

maybe that is one thing that could really help here.  possibly mutt and
other emacs-based mail readers, when putting up a received e-mail
message, should by default do just that (and, also, '(setq
enable-local-variables :safe)'?).

i use mh-e, and it does *not* seem to do that.  but, if emacs is niche,
mh-e is the niche-squared. :)

cheers, Greg



Re: Security issues in Emacs packages

2020-11-25 Thread Greg Minshall
Tim,

> I think you missed my point. There is no benefit in MELPA adopting
> signed packages because there is no formal code review and no vetting
> of the individuals who submit the code.

it occurs to me there might be one benefit: if George, whom you trust,
says, "I've been running version 1.2.3 of package xYandZ from MELPA and
i have a lot of confidence in it", then if you find that version of that
package with a trusted MELPA signature, you maybe know that you and
George are running the same software.  i.e., it helps with the "web of
trust" (if people still talk of that).

(so, the requirement for this is not audited packages, but a solid,
"secure", release procedure by MELPA.)

cheers, Greg



Re: Security issues in Emacs packages

2020-11-25 Thread Jean Louis
* Tim Cross  [2020-11-26 02:40]:
> > OK it is great that it is so. Are you maybe author doing it? Is there
> > any reference that authors are doing so? I have MELPA downloaded you
> > could tell me how do I see that author is deciding if package is for
> > release?
> >
> 
> You can clone the melpa repository and see the recipes for each
> package.

I did before some time.

> It depends on how the author specifies their MELPA recipe. They can
> define their recipe based on a specific commit (SHA). If they do this,
> it doesn't matter how often or when MELPA pulls from the repository as
> they will always get the same commit.

I have not seen that, and I have assumed you would know better and
wanted to see how authors are reporting that package is ready for
release and I do not see that.

Recipes are like this:

(0blayout :repo "etu/0blayout-mode" :fetcher github)

(0x0 :url "https://git.sr.ht/~zge/nullpointer-emacs; :fetcher git)

(0xc :fetcher github :repo "AdamNiederer/0xc")

So that recipe alone does not tell me that author reports that new
package is ready, it is fetched from git, but there are parts of code
that I did not see that is why I am assuming you know it better.

> Your model is flawed. You can have both automatic pulling AND author
> control over when a new package is issued.

To make it practical tell me where is that author's control?

I have quick view of files and any recipe files in directory
melpa/recipes do not give me any pointers, it is all automated and
fetched from git.

> If author defines their MELPA recipe to use a SHA a new package will not
> be issued until they update their recipe with a new SHA.

You seem to be very confident and for this reason I assume you know it
better, but due to contradictions please show one practical recipe or
package where author has control on when is package ready to be
released.

$ grep sha *

on recipes does not give any reference.

$ grep commit *

eval-in-repl:  :commit  "origin/master")
git-auto-commit-mode:(git-auto-commit-mode :fetcher github :repo 
"ryuslash/git-auto-commit-mode")
git-commit:(git-commit :fetcher github
git-commit::files ("lisp/git-commit.el")
git-commit::old-names (git-commit-mode))
git-commit-insert-issue:(git-commit-insert-issue :fetcher gitlab :repo 
"emacs-stuff/git-commit-insert-issue")
vc-auto-commit:(vc-auto-commit :fetcher github :repo "thisirs/vc-auto-commit")
what-the-commit:(what-the-commit :fetcher github
what-the-commit: :repo "danielbarbarito/what-the-commit.el")

So there is nothing I can find that points or references to what you
say.

> If author defines their MELPA recipe to pull from a release branch, a
> new package will not be issued until they update the release branch and
> version tag.

I am sorry I do not see reference to it. You are convincing but I do
not see reference.

Recipe for bar-cursor:
(bar-cursor :repo "ajsquared/bar-cursor"
:fetcher github)

Recipe for magit:

(magit :fetcher github
   :repo "magit/magit"
   :files ("lisp/magit"
   "lisp/magit*.el"
   "lisp/git-rebase.el"
   "Documentation/magit.texi"
   "Documentation/AUTHORS.md"
   "LICENSE"
   (:exclude "lisp/magit-libgit.el"
 ;; Cannot remove this yet because it would
 ;; also be removed from the stable version.
 ;; "lisp/magit-section.el"
 )))

Repo magit/magit:
https://github.com/magit/magit

I have given you references, maybe I cannot read that well, so you can
give me references to show if authors have participation in decision.

Jean



Re: Local variables insecurities - Re: One vs many directories

2020-11-25 Thread Jean Louis
* Tom Gillespie  [2020-11-26 05:07]:
> Hi Jean,
> 
> Some points in summary before a long email.
> 1. Having an accepting default behavior as a user (i.e., saying yes to
>all prompts) is bad security practice. The only thing that systems
>can do is prompt as infrequently as possible in hopes that people
>don't get prompt fatigue. Emacs does this.

I know, and do not speak for one person but for those who will not
know. To ask users who do not know programming using editor for text
editing and to assume that users will know programming terms:
variables, values, applying, safe and ANYTHING else that is written in
eval: is bad security practice.

> 2. If mutt is launching Emacs, you can pass --eval "(setq
>enable-local-eval nil)" on the command line and all file local
>variables will be ignored and treated as plain text.

Sure, I know, but human text editors will not know. They are faced
with YES/NO. What has to be improved is the YES/NO dialogue that has
no hyperlinks to its corresponding explanations and asks users things
with wrong assumption that user will understand them.

> 3. If people don't read the manual and don't read the prompt, there
>isn't much more that Emacs can do while still empowering the
>user.

Empowering can take place in the dialogue as such has enhancement
space. Do you think it is good idea?

>   In those cases we also can't assume that the community will be of
>   much help either, as we are trying to be here.

Community is of great help. Users are many and just fraction of users
are in this community for example. Only subset of users even while
being in community or reading emails or website will be participating
in such.

> 4. That said, the local variables prompt could be more alarming and
>provide a quick way to access the manual about local
>variables. I'll look into a patch for that.

Yes, that is it. To empower user is to give him straight better
reference on what to do.

Following places could become buttons:

The local variables list in new-concept.org
^^^
Button to A below

contains values that may not be safe (*).
^^^ 
Button to A below   Button to A below

Template:

Do you want to apply it?  You can type
   
y  -- to apply the local variables list.
 ^^
 Button to A below
 
n  -- to ignore the local variables list.

!  -- to apply the local variables list, and permanently mark these
 ^^^
 Button to A
  values (*) as safe (in the future, they will be set automatically.)

  * eval : (let ((pos (org-babel-find-named-block "stages"))) (when
pos (save-excursion (org-with-point-at pos
(org-babel-execute-src-block)
^^
Button to A below.

The template how it looks originally:

The local variables list in new-concept.org
contains values that may not be safe (*).

Do you want to apply it?  You can type
y  -- to apply the local variables list.
n  -- to ignore the local variables list.
!  -- to apply the local variables list, and permanently mark these
  values (*) as safe (in the future, they will be set automatically.)

  * eval : (let ((pos (org-babel-find-named-block "stages"))) (when pos 
(save-excursion (org-with-point-at pos (org-babel-execute-src-block)

Button A should follow here:
^
File: emacs.info,  Node: Safe File Variables,  Prev: Specifying File Variables, 
 Up: File Variables

49.2.4.2 Safety of File Variables
.


Right now situation is this:

- if user press C-g to abort the file will be loaded, which is good

- user will not be able to access any of the visible buttons by using
  cursor because somebody made cursor invisible in that
  dialogue. Cursor should be made visible that keys can be used to
  access buttons

- There shall be on minibuffer some key like "? TO READ MANUAL" to go
  straight to the manual section above which is warning user much
  better.

- IF THE USER press C-g, or tries to escape it or clicks by using
  keyboard on the button, or even switch to the buffer with keyboard,
  or uses mouse to activate the button, or uses ? to activate to read
  manual, in all those cases the dialogue should be interrupted and
  file loaded. User should not be left facing dialogue if it was clear
  from user's interaction that there was doubt with the meanings of
  the dialoue.

- Adding references on unsafe local variables to TUTORIAL

> Full disclosure. I make use of file local variables every day
> and I have spent quite a while writing orgstrap which relies heavily
> on them, so I am definitely biased.

With such enhancement there is no need to change history of the past,
we just change the future and history for the future.

> Emacs is a sharp tool. Experts need sharp tools. If someone
> complains about the security of prompts but also admits that they
> always say yes to 

Re: One vs many directories

2020-11-25 Thread Ihor Radchenko
> For my own setup I run code in a hook to update the agenda whenever I
> change a TODO state, clock in or clock out, but that has performance
> problems when I do it while the Agenda is shown.

You do not have or update the whole agenda view.

I use the following code to update the clocking highlights in agenda
even when I clock-in/out outside the agenda buffer:

https://github.com/yantar92/emacs-config/blob/master/config.org#update-highlight-from-currently-clocked-task-in-agenda-even-if-the-task-was-clocked-inout-from-outside

The same can be done for todo state changes using
org-agenda-change-all-lines

Best,
Ihor

"Dr. Arne Babenhauserheide"  writes:

> Jean Louis  writes:
>
>> Some people maybe access multiple Org files through Agenda, me I
>> don't. Some items are "non existent" and I do not know how to ask
>> agenda to refresh itself.
>
> Simply press the letter g.
>
> For my own setup I run code in a hook to update the agenda whenever I
> change a TODO state, clock in or clock out, but that has performance
> problems when I do it while the Agenda is shown.
>
> (defvar todo-modified-from-agenda nil "Indicates whether org-mode todo 
> state changes were triggered from the agenda. Check this to avoid trying to 
> propagate the change back into the agenda")
> ;; continuously update agenda view, from 
> http://thomasf.github.io/solarized-css/test/org-hacks.html
> (defun kiwon/org-agenda-redo-in-other-window ()
>   "Call org-agenda-redo function even in the non-agenda buffer."
>   (interactive)
>   (when (not (and (boundp 'todo-modified-from-agenda) 
> todo-modified-from-agenda)) ; org-state non-nil means we’re coming from the 
> org-after-todo-state-change-hook, which would throw when changing todo states 
> from agenda due to a circular action
> (let ((agenda-window (get-buffer-window (or (and (boundp 
> 'org-agenda-buffer-name) org-agenda-buffer-name) "plan.org") t)))
>   (when agenda-window
>   (with-selected-window agenda-window
> (org-agenda-redo))
> ;; advice agenda todo to avoid redo, thanks to 
> http://nullprogram.com/blog/2013/01/22/
> (defadvice org-agenda-todo (before org-agenda-disable-redo activate)
>   (setq todo-modified-from-agenda t))
> (defadvice org-agenda-todo (after org-agenda-enable-redo activate)
>   (setq todo-modified-from-agenda nil))
> 
> (add-hook 'org-clock-in-hook 'kiwon/org-agenda-redo-in-other-window)
> (add-hook 'org-clock-out-hook 'kiwon/org-agenda-redo-in-other-window)
> (add-hook 'org-after-todo-state-change-hook 
> 'kiwon/org-agenda-redo-in-other-window)
>
>
> Best wishes,
> Arne
> -- 
> Unpolitisch sein
> heißt politisch sein
> ohne es zu merken



Re: consistent behavior across babel languages

2020-11-25 Thread ian martins
On Wed, Nov 25, 2020 at 2:06 AM Tim Cross  wrote:
>
> I will try to allocate time on the weekend to
> review what you have and see if there are any I know of which you have
> not included.

That would be great.

On Wed, Nov 25, 2020 at 2:39 PM Tom Gillespie  wrote:
>
> Since there are so many different features that
> a babel language implementation can support I don't want to try to put
> them all in one table quite yet but I think it may be helpful to do so
> eventually. Until that time does it make sense to add new sections to
> the file that cover specific features? For example I have tests on
> TRAMP support and/or support for execution via a remote session that I
> would like to add from my internal notes. Can I add it as a new
> section?

Yes, definitely. My plan is to add sections for major features or
feature groups and have a table for each section, and when that stops
working to think of something else. I am planning a section for
variables focusing on passing lists and tables which will probably
look a lot like the section on returning list and tables. There should
definitely be a section on TRAMP.



Re: One vs many directories

2020-11-25 Thread Ihor Radchenko
> Can I automated the execution of Babel code upon opening of the Org
> file?

Adding to other suggestions, you can always add a custom function to
org-mode-hook instead of playing with file-local variables.

> Then we comes to actual execution of tasks. How do we get reminded?
>
> Is the reminder only if I press {C-c a} for org-agenda? Do I need to
> do action to get reminded?

You can always configure Emacs to run agenda on startup. Just add a
command to your init file ;)

For automatic reminders, there is built-in org-notify.el or external
org-alert package (https://github.com/spegoraro/org-alert).

> Personal problem is that tasks are sparse and separate in various Org
> files and not centralized. I become dependent of org-agenda to do what
> I need but it never does what I need.

I agree that org-agenda has many issues that cannot be easily solved
because of its complexity. However, everything you describe (including
multi-occur) can also be achieved with org-ql
(https://github.com/alphapapa/org-ql) - analogue of SQL query language
for org-mode (with more optimisations in comparison with org-agenda). 

Best,
Ihor

Jean Louis  writes:

> * Ihor Radchenko  [2020-11-23 08:43]:
>> >> I am wondering what you mean by Org's philosophy. Why would it have 
>> >> anything to do with directories?
>> >
>> > Org's philosophy is to have one or a handful of directories without
>> > nesting of directories.  Users are not expected to have their Org
>> > files in a deeply nested tree.  Org also prefers big files with large
>> > trees rather than lots of little files.
>> >
>> > By philosophy, I mean the dev consensus on the correct way to do
>> > things, and coded configuration and usability biases.
>> 
>> I believe that org support all possibilities. The user can decide to
>> keep many (possibly nested) org files, a few large org files, or
>> anywhere in between. There are several parallel feature sets allowing to
>> work in a single file as well as with a bunch of smaller files.
>
> Yes, sure, and I guess you mentioned some people have problems with
> many files. And I have no problem at all with many files as they are
> per subject separated, per person or per subject separated. They are
> not hyperlinked to each other, it is me who make system to hyperlink
> to files.
>
> Searching for Joe Doe, F4 and I am in Org file for Joe Doe. My
> personal TODO list need not really show the tasks assigned to Joe Doe,
> I could show only * TODO Joe Doe and when I click there then I can get
> all tasks for Joe Doe as new Org file.
>
> It means I am accessing hundreds of Org files from the meta level by
> using conceptual location backed by the database.
>
> Some people maybe access multiple Org files through Agenda, me I
> don't. Some items are "non existent" and I do not know how to ask
> agenda to refresh itself. This is not big deal as I do not access
> items throgh Agenda, though I find it very useful. 
>
> org-agenda is trying to put all tasks and notes from various files
> into one list and that is of course not so easy task considering that
> files can be anywhere on the file system and that they need to be
> "remembered". 
>
>> For a single file, the user can search headings with org-goto (without a
>> need to explicitly travel through all the nesting headline levels),
>> reveal only headings satisfying certain keyword/tag/any other search
>> criteria with org-sparse-tree, or built agenda views restricted to a
>> single file (or even subtree).
>
> M-x org-goto is useful feature to find headlines. And I never use it,
> just standard Emacs search is enough within a file. Meanings I am
> searching are often inside of the headline. And it is not my perosnal
> way locating things.
>
> Personally, I am using parts of Org, like specific headling to export
> it and to send to remote person, or to print the file as project and
> to bind it nicely.
>
> When I see repetitive action, for example that I have to send "Daily
> Report" template to a person by email, than I just bookmark that in
> Emacs with {C-x r m} under something that I think is the meaning of
> it. Then I forget about it. Next time when I need to send report, I am
> {C-x r b} and quickly completing it and then I am exporting and
> inserting into the email.
>
> In general I speak of subsets or sub-lists among lists. List of Org
> files is one list, list of headings within Org file is other list, and
> list of specific subject related headings or bookmarks to such is
> third subset of lists.
>
>> For multiple files located anywhere in the filesystem, there is always
>> org-refile capable of filing the information to proper place
>> searching deeply nested headlines with ease regardless of the file the
>> information is physically located in. Headlines from multiple files can
>> be grouped using agenda views for any given search criteria (showing
>> todo items or items for a single day/week is just a tiny subset of what
>> agenda can do).
>
> That may be useful for 

Re: Is Org really so simple?

2020-11-25 Thread Ihor Radchenko
> Only philosophy I know is that it is plain text. Is there any official
> philosophy? I have no idea, at least manual does not give me
> references. I cannot find "philosophy", send me references.

You are right. There is no official "philosophy" in org. In my reply I
tried to follow the topic starter's view:

 Texas Cyberthal :
> By philosophy, I mean the dev consensus on the correct way to do
> things, and coded configuration and usability biases.

According to my experience with org-mode development (I am not talking
about third-party packages here), it is discouraged to change org-mode
towards hiding metadata in org files or store *unique* data (that cannot
be derived from the contents of the org files) related to org-mode
externally (not in org files). It is not official statement, but rather
my impression so far.

> Question is what is meant by database, it can be anything. One can
> save LISP data. Recent files, desktop, eww bookmarks, init.el or
> .emacs file are also all similar databases, there is the underused
> EIEIO with persistent stuff that represent built-in database
> functionality.

Org-mode does not limit user customisations. It does not limit
third-party packages either. Users are free to use any other tools,
store their data anywhere other than org files (why would org force
users to do otherwise?). What I referred to earlier is just the core
org-mode development.

> And people try to do exactly that, people are developing Org in the
> manner to relate objects in Org file to anything else. And they do
> hard work as they do it manually. Relational database speeds up and
> does not tell user to manually hyperlink various relations.

Could you provide some more examples? I do not see how relational
database is different from creating hyperlinks in org. Either way, the
user needs to file an object/headline somewhere into org file, which is
inherently manual.

> I see hard work by many people who try to enhance Org as hierarchical
> knowledge of data because people want to have feature X, but group of
> those enhancements in reality belong to relational databases and not
> to text files.

It is an interesting point. I would be happy if some existing tools
could be reused instead of re-inventing the wheel for org. Do you have
concrete examples where it can be useful? If you have, I encourage you
to bring up a feature request to discuss this possibility with org-mode
devs.

Best,
Ihor

Jean Louis  writes:

> * Ihor Radchenko  [2020-11-23 17:18]:
>   :PROPERTIES:
>   :CREATED:  [2020-11-23 Mon 18:42]
>   :ID:   edebb3e7-e755-4ecc-a5e8-a3353a3f5fd0
>   :END:
>> Dear Jean Louis,
>> 
>> Your description of the database reminds me how org-roam handles the
>> files - it also uses an external database for linking and allows quick
>> incremental search that does not really depend on where the
>> files/headings are stored.
>
> Sounds good, I can see there is graph database used.
>
>> However, what you are talking about is against org-mode philosophy,
>> as I know it.
>
> Only philosophy I know is that it is plain text. Is there any official
> philosophy? I have no idea, at least manual does not give me
> references. I cannot find "philosophy", send me references.
>
> It says "to keep simple things simple". But Org is far far from being
> simple any more. It offers good principles, paradigms and people built
> many enhancements upon those. Speedily it becomes way much more than
> simple.
>
> Headings do not look any more as headings, it looks like pieces of
> code to a person that is new. Properties, tags, clocks, schedule,
> deadline, all that tries to store itself under specific heading. There
> is easily too much of it and things are not simple any more.
>
>> Currently, the dev's stance is that org files must be
>> self-sufficient.
>
> There is no compact principle there practically. Anything is
> possible. That Org files are not practically self-sufficient shows the
> fact that there are 129 Emacs packages in one Org distribution. 
>
>> Org-mode should not depend on external database to manage the org
>> files and operations with them. Everything must be plain text!
>
> Question is what is meant by database, it can be anything. One can
> save LISP data. Recent files, desktop, eww bookmarks, init.el or
> .emacs file are also all similar databases, there is the underused
> EIEIO with persistent stuff that represent built-in database
> functionality.
>
> That Org files are not self-sufficient shows the demand that there is
> almost no Org user who does not have add-ons, packages, modifications,
> configurations.
>
> Would it be really self-sufficient there would be no development going
> on, right?
>
> Babel executions clearly show that Org is not self sufficient and
> depends on number of external software.
>
> I don't mind of philosophy, in fact I would like that philosophy is
> really that what it wanted to be, but that time is over.
>
> I am just pointing out that it is by many means not 

Re: Bug: Font highlighting with in-line latex math broken [9.4 (9.4-dist)]

2020-11-25 Thread Vladimir Lomov
Hello,
** Edmund Christian Herenz  [2020-11-25 16:50:46 -0300]:

> Hi,

> me again...

> On Wed, Nov 25, 2020 at 04:32:28PM -0300, Edmund Christian Herenz wrote:
>> This is only cosmetical, but a bit annoying.

> Ah, and while we are at it

> Entering a line break in $math
> mode$ is also valid and should not break the font highlighting.

Did you try to use \(y=\sin(x)\) instead of $y=\sin(x)$ ?

> CH.

---
WBR, Vladimir Lomov

-- 
Virtue is not left to stand alone.  He who practices it will have neighbors.
-- Confucius


signature.asc
Description: PGP signature


Re: Local variables insecurities - Re: One vs many directories

2020-11-25 Thread Tom Gillespie
Hi Jean,

Some points in summary before a long email.
1. Having an accepting default behavior as a user (i.e., saying yes to
   all prompts) is bad security practice. The only thing that systems
   can do is prompt as infrequently as possible in hopes that people
   don't get prompt fatigue. Emacs does this.
2. If mutt is launching Emacs, you can pass --eval "(setq
   enable-local-eval nil)" on the command line and all file local
   variables will be ignored and treated as plain text.
3. If people don't read the manual and don't read the prompt, there
   isn't much more that Emacs can do while still empowering the
   user. In those cases we also can't assume that the community will
   be of much help either, as we are trying to be here.
4. That said, the local variables prompt could be more alarming and
   provide a quick way to access the manual about local
   variables. I'll look into a patch for that.

Now on to the rest!

Full disclosure. I make use of file local variables every day
and I have spent quite a while writing orgstrap which relies heavily
on them, so I am definitely biased.

Emacs is a sharp tool. Experts need sharp tools. If someone complains
about the security of prompts but also admits that they always say yes
to prompts without reading them, then no one will take them seriously
when they try to argue that it is the prompt that is insecure. It is
like complaining that the forest is dangerous while insisting on
eating the berries of all the plants and picking up all the snakes.
The forest is dangerous, but not merely because the berries and the
snakes exist in it. Heck, some snakes even try to warn you! There are
some rattlesnakes that have evolved to not rattle because idiot humans
go find and kill them. Now everyone suffers because there are silent
rattlesnakes that will strike without warning.

Degrading usability because some users are irresponsible just
reinforces the irresponsible behavior and everyone loses. The endgame
for all prompts is a two-man two-key system where the switches are
separated by 30 feet and must be turned within 1 second of each
other. How else could we be sure that the user really meant to say
yes!? Emacs is scary, but not nuclear launch system levels of scary.

Furthermore, Emacs does try to account for users who don't read the
manual and has sane and safe defaults. Namely it does not blindly
execute code but prompts the user and lets them know that something is
going on. Further, when it prompts the user it does not provide a
default action, it forces them to answer so that they must attend to
what they are doing. This provides the user with an opportunity to
become aware of a feature of Emacs that is new to them. The only other
option is to never execute local variables until a user reads the
manual and changes their config. Such a design infantilizes the user
and does not empower them.

Emacs isn't just a dissociated tool, or at least it tries not to be
(lots of people also complain about the splash screen being enabled by
default!). The Emacs community often also goes out of its way to share
its knowledge when such situations arise (as we are doing here). We
can't reach everyone (yet!) but many people donate time and effort to
share. Emacs and Org both actively encourage users to participate in
the mailing lists because venues like StackOverflow are not guaranteed
to be seen by the community and are not officially supported.

Finally on this point, if we are willing to open the manual we see
that Emacs tries to educate users about this, the first line of the
section on file local variables is "File-local variables can be
dangerous."

With regard to the local variables prompt. Maybe the right thing to do
in this case would be to add a "?" option so that users can open the
manual? That seems like it would help. That said, the second line of
the prompt reads "contains variables that may not be safe (*)" which
should be alarming. I guess it could be changed to "THIS FILE COULD
PWN YOU. Please review potentially unsafe variables marked with an
asterisk (*)." Or just make the second line a bold warning? I have a
patch I need to send in to make it possible to use lexical scope in
eval local variables, I'll take a look at this while I'm there.

It is forgivable to not be utterly terrified of systems that can
execute arbitrary code, given that they mostly look like rocks, and we
are surrounded by them all day. However, they are utterly terrifying
existences! While I think it would be great for the Emacs splash
screen to come with a big warning that says "WARNING THIS PROGRAM
ENABLES YOU TO EXECUTE ARBITRARY CODE" I admit that it took me nearly
a decade to begin to understand what arbitrary code execution means
and why I should be scared of it. Those words are meaningless to a 12
year old who at best will think "What is the big deal about arbitrary
code? I can already run any code that I want!"

This is why I pointed you to orgstrap. The eval: (org-sbe startup)

Re: Security issues in Emacs packages

2020-11-25 Thread Tim Cross


Jean Louis  writes:

>>
>> this is wrong. In melpa you specify either a commit (SHA) or a branch or
>> both. The repository owner has control over this. MELPA doesn't just
>> pull data from the repository because there has bene an update. You can
>> configure things so that whenever data is committed to a release branch,
>> it is pulled, but this is under the control of the repository owner. It
>> isn't that different to ELPA where the maintainer will either push new
>> data to the ELPA repository (or ask someone with write permission to
>> pull it from their repository).
>
> OK it is great that it is so. Are you maybe author doing it? Is there
> any reference that authors are doing so? I have MELPA downloaded you
> could tell me how do I see that author is deciding if package is for
> release?
>

You can clone the melpa repository and see the recipes for each package.

It depends on how the author specifies their MELPA recipe. They can
define their recipe based on a specific commit (SHA). If they do this,
it doesn't matter how often or when MELPA pulls from the repository as
they will always get the same commit.

They can also specify a branch rather than a commit SHA. In this case,
MELPA will retrieve updates from that branch, so when that branch is
updated, it will pull new data. In this case, it is up to the developer
to manage their 'release' branch appropriately. when they are ready for
a new release, they push their updates to the release branch and update
the version tag. This is pretty much the same as ELPA works for external
packages (those which don't manage their code within the GNU ELPA repository 
itself)


>
> So is there automatic pulling?
>
> I compare automatic pulling and building to author's decision on when
> a package would be issued.
>

Your model is flawed. You can have both automatic pulling AND author
control over when a new package is issued.

If author defines their MELPA recipe to use a SHA a new package will not
be issued until they update their recipe with a new SHA.

If author defines their MELPA recipe to pull from a release branch, a
new package will not be issued until they update the release branch and
version tag.

MELPA does not automatically generate a new package just because
something has changed within the git repository. It has to be a change
to a specified branch and update to the version tag or it has to be a
change in the recipe with an update to the commit SHA.

--
Tim Cross



Re: Security issues in Emacs packages

2020-11-25 Thread Jean Louis
* Tim Cross  [2020-11-26 01:47]:
> I think you missed my point. There is no benefit in MELPA adopting
> signed packages because there is no formal code review and no vetting of
> the individuals who submit the code.

Do you think it is really their reason? Or maybe you are developer in
MELPA?

There is still difference if package comes from MELPA or from third
party archive, definitely signing would say this package was signed by
MELPA and other package was not signed by trusted key, so who is that?
Is the MD5SUM same as original? It would give some initiative to
investigate.

Packages are not audited that is so. I think audit can be quick by
using grep for some dangerous commands, I have already found rm -rf as
example command in one of packages, not as malicious one. One could
search for (shell-command and verify such and similar functions.

> If you have no controls in place over the contents of what is being
> signed, the value of the signatures as a security measure is
> drastically reduced. Yes, the valid signature may provide some
> assurances as to where the package originated, but that means little
> if the contents could be anything.

What you explain is logical to me, users though need better
information. One big DANGER should be given to users.

> As it stands, the signing of ELPA packages only provides assurance that
> they are packages assembled by GNU. These signatures do not provide any
> real assurance regarding the content of the packages other than they are
> GPL's and do not recommend or encourage the use of non-free
> software.

There is no absolute. Signing says about origins. Mirrors are placed
anywhere in the world, including behind Internet. It is one way for
users to verify origin and if source has been changed.

> The question is what level of trust should you assume. With ELPA, all
> you can really trust is that the package has a GPL license and does not
> recommend/require the use of non-free software. There is little trust
> that the package does not do something malicious or includes code which
> may compromise the user's security. to provide that level of assurance,
> you would need formal code reviews, which is not feasible given
> available resources.

Last month I could see that several packages were here and there
improved by developers so they do look into code and how much they do
I cannot know.

> I think it is important users are aware of this
> limitation. Furthermore, I ask the question "Does having signed
> packages imply a level of expected assurance which is higher than it
> really is?"  In other words, do users expect that a package is
> completely safe just because it was downloaded from an official GNU
> ELPA repository?

Download numbers on MELPA tells me that answer should be rather
positive, any package is safe to be installed. See
numbers. Information is no enough to teach users. More attention is
necessary. 

> Last time I looked, ELPA also supported 'external' packages where the
> data is retrieved from an external git repository. I think org is one
> such package.

Majority of GNU ELPA packages are external how I know about it, but
authors decide WHEN to upload them.

> > The point number (1) is human, not automated. Author decides when is
> > the package ripe for distribution and what is "release".
> >
> > Git repository is never release and not meant to be "release". Git is
> > for collaborative development and users are made blind that it is some
> > kind of release while it is not. One shall always assume that Git
> > repository contains development versions not ready for public.
> >
> 
> Why? This is not normal. Git repositories contain all versions, both
> production and development. What is production and what is development
> is managed through branches and tags. Anyone who wants can clone the
> ELPA git repository.

How I see practically, people hack on git master branches and main
branch need not be considered release ready. Git hosting websites then
have special section for releases. Git branch is not a release
according to what I know, it is revision control system or version
control system. Git often looks pretty different than release as
package. Of course everybody can clone. Point is that software is no
ripe. Maybe somebody else knows if Git can tell that software is ripe,
what I know it is not so. Author has to say when it is ripe for
release. 

> > MELPA pulls those packages, correct me if I am wrong, automatically
> > from Git repositories without regard if the package is actually
> > release. That does not align or respect the established Emacs
> > conventions how packages should be released, if they are multi files
> > they should be in .tar file otherwise .el and there are version
> > numbers that MELPA fiddles with and makes possible conflicts and
> > introduces confusions.
> 
> this is wrong. In melpa you specify either a commit (SHA) or a branch or
> both. The repository owner has control over this. MELPA doesn't just
> pull 

Re: Security issues in Emacs packages

2020-11-25 Thread Tim Cross


Jean Louis  writes:

> * Tim Cross  [2020-11-25 10:01]:
>>
>> Jean Louis  writes:
>>
>> > * Tim Cross  [2020-11-24 23:40]:
>> >> If people are really concerned about security, they should look first at
>> >> their use of repositories like MELPA. There is no formal review or
>> >> analysis of packages in these repositories, yet people will happily
>> >> select some package and install it.
>> >
>> > Interesting that you are one who mentions that. There are just few
>> > people ever mentioned it.
>> >
>> > I am still in process of the review of MELPA packages and its
>> > system. There are many security issues.
>> >
>> > Package signing is one example. It does not offer much of security
>> > when packages are signed automatically, but it raises level of
>> > security.
>> >
>> > MELPA packages and archive-contents are not PGP signed, while GNU ELPA
>> > packages are signed.
>> >
>>
>> IMO signing of packages is irrelevant when there is no formal review
>> process or even any formal process to verify the credentials of
>> signatures. In fact, just adding signing would likely be
>> coutner-productive as it would give the impression of some sort of
>> security where there is none.
>
> When user receives a signed package from GNU ELPA, that means it comes
> from GNU ELPA. If signed package is then distributed by mirrors or
> other websites users who enable package signature verifications will
> know that package still comes from GNU ELPA, and not from Chinese
> distributor. If the package is tampered the signature verification
> will not allow installation of the package.
>

I think you missed my point. There is no benefit in MELPA adopting
signed packages because there is no formal code review and no vetting of
the individuals who submit the code. If you have no controls in place
over the contents of what is being signed, the value of the signatures
as a security measure is drastically reduced. Yes, the valid signature
may provide some assurances as to where the package originated, but that
means little if the contents could be anything.

The situation with ELPA is a little better because those who maintain
the code are required to assign legal copyright to GNU. However, I'm not
sure how much checking is done to verify the information in those
assignments. As far as I know, there is no formal code review. A number
of the Emacs developers do perform some informal review, but as we all
know from the issues with openssl, informal reviews provide little
assurance. This is not a criticism of GNU or emacs developers. The
amount of resources necessary to perform formal review is much larger
than the available resources. On the whole, the emacs user community
appears to be happy with the current situation. If they were not, it
would be on the community to step up and do something about it.

As it stands, the signing of ELPA packages only provides assurance that
they are packages assembled by GNU. These signatures do not provide any
real assurance regarding the content of the packages other than they are
GPL's and do not recommend or encourage the use of non-free software.


> That it does add to security shows the fact that GNU/Linux
> distributions do sign packages. There is difference if the package
> comes from trusted source or not trusted source.
>

The question is what level of trust should you assume. With ELPA, all
you can really trust is that the package has a GPL license and does not
recommend/require the use of non-free software. There is little trust
that the package does not do something malicious or includes code which
may compromise the user's security. to provide that level of assurance,
you would need formal code reviews, which is not feasible given
available resources. I think it is important users are aware of this
limitation. Furthermore, I ask the question "Does having signed packages
imply a level of expected assurance which is higher than it really is?"
In other words, do users expect that a package is completely safe just
because it was downloaded from an official GNU ELPA repository?

>> Basically, anyone can upload anything to MELPA.
>
> Maintainers verifies the package initially for certain conventions,
> after that, how I have understood it, packages are automatically
> pulled from Git. The more authors give packages to MELPA the more
> insecurity probability is raised.
>
> GNU ELPA how I understand it, I may be wrong, works like this:
>
> 1) packages are uploaded to GNU ELPA
> 2) then automatically signed by GNU ELPA PGP keys
> 3) offered for distribution
>
Last time I looked, ELPA also supported 'external' packages where the
data is retrieved from an external git repository. I think org is one
such package.

> The point number (1) is human, not automated. Author decides when is
> the package ripe for distribution and what is "release".
>
> Git repository is never release and not meant to be "release". Git is
> for collaborative development and users are made blind that it is some
> kind 

Re: Local variables insecurities - Re: One vs many directories

2020-11-25 Thread Jean Louis
* Tim Cross  [2020-11-25 23:54]:
> I guess this is probably the main point where we disagree.
> 
> Emacs is first and foremost a programmers editor. It was never designed
> as a general purpose editor, but rather specifically as an editor for
> programmers.

Yes. And when I was born as baby I was designed for milk, not for
typing, times change. People use GNU/Linux and Emacs is not advertised
as programmers or exclusively programmers editor. Some other editors
are advertised that way. So think how many hundreds of thousands of
users are working with Emacs.

Here is how Debian GNU/Linux describes it:
https://packages.debian.org/buster/emacs

If there are 10 programmers there are probably 100 if not 500
non-programmers.

> If you jump into a formula 1 race car, you would find it almost
> impossible to drive. The gearbox would be unfamiliar and difficult to
> use, the clutch would be difficult to use etc. If you got it going, you
> would have a high likelihood of crashing. Luckily, you would probably
> just stall and get nowhere.
> 
> Is this the fault of the design of the race car or the driver?

Race cars are not distributed through GNU/Linux operating systems and
are not easily downloadable by everybody, in general, they are also
expensive. While it all sounds entertaining, Emacs is not a race
car. And we cannot say to users not to use it if they are not Formula
One Drivers.

> With respect to your email example, the number of people who are exposed
> is even less - it is really only those who are using it in the same
> manner as you. That is, where they have configured their mail client
> (such as Mutt) to use Emacs as the external editor. None of the Emacs
> mail clients I have used do this (this includes VM, mu4e, gnus,
> wonderlust and mew).

I do not need to use Emacs with Mutt to invoke local variables. I can
get files by any means and by any opening of the file with Emacs it
will be invoked. Somebody could send me file to download and
open. File can come from anywhere, it is not Mutt related really.

Gnus buffers and email clients do not invoke local variables and that
is fine. But security issue is not email centric, but file centric.

> anyone who has gone to the effort to configure their mail system to use
> an external editor and who then answers yes to the statement
> "...contains values that may not be safe. Do you want to apply it?" is
> someone with inherently unsafe practices.

That is very rigid assumption. People set editors for various email
clients since decades. Try to think from another people's view points.

Here is example:
https://stackoverflow.com/questions/15865495/opening-a-file-in-emacs-values-that-are-not-safe

That person has quite different view point. Person asks "Why it would
not be safe?" and one should know when one person writes there for an
answer there are probably thousand other persons who did not write for
the answer.

Other person asked:

"Thanks, that's very helpful. Why would a file (i.e. the author of the
file) require or ask Emacs to apply configuration values when just
opening/visiting the file? – Amelio Vazquez-Reina"

I know why, but people using Emacs are asking why. Many will not ask
and will say, damn YES, as I feel safe!

Denial of Service Attacks possible:
https://github.com/aquamacs-emacs/aquamacs-emacs/issues/147
https://gitmemory.com/issue/davidswelt/aquamacs-emacs/147/478196367

.emacs considered not safe:
https://www.cs.ait.ac.th/~on/O/oreilly/tcpip/puis/ch11_05.htm

OK then now back to Org discussions.

Jean



Re: Local variables insecurities - Re: One vs many directories

2020-11-25 Thread Tim Cross


Jean Louis  writes:

> * Eric S Fraga  [2020-11-25 16:58]:
>> On Wednesday, 25 Nov 2020 at 16:13, Jean Louis wrote:
>> > I use Mutt.
>> > The message is opened in Emacs in mail-mode
>>
>> Ah, so mutt saves content in a file which is then opened by
>> Emacs.  Okay, that makes sense.  Gnus does things the other way around:
>> opens the buffer (associated with a file in the draft directory),
>> inserts the content, and then puts the user in control.  File local
>> variables don't get a chance to be interpreted then.
>
> That is specific to Gnus. Any file opened by Emacs any will still
> invoke the dialogue for local variables.
>
>> > Then I have been testing and even text files invoke local variables.
>>
>> Yes, of course.  That's the whole point?
>
> You know that point is bad design and assumption that every user is
> programmer who knows what are local variables and what are definitions
> of Emacs functions, it is incredible situation.

I guess this is probably the main point where we disagree.

Emacs is first and foremost a programmers editor. It was never designed
as a general purpose editor, but rather specifically as an editor for
programmers.

If you jump into a formula 1 race car, you would find it almost
impossible to drive. The gearbox would be unfamiliar and difficult to
use, the clutch would be difficult to use etc. If you got it going, you
would have a high likelihood of crashing. Luckily, you would probably
just stall and get nowhere.

Is this the fault of the design of the race car or the driver? Would it
make sense to change the design of a race car to use a standard gearbox
and clutch just because at some point someone who is not a race car
driver might want to drive it?

Are there risks associated with local variables. Yes. Is there
sufficient protection against these variables being used for malicious
purposes in Emacs. I think the answer is yes. Is there any evidence of
these variables being used for malicious purpose. None that I am aware
of. Has there been bugs in the implementation of this facility - yes.
Have these bugs been addressed once identified - yes.

With respect to your email example, the number of people who are exposed
is even less - it is really only those who are using it in the same
manner as you. That is, where they have configured their mail client
(such as Mutt) to use Emacs as the external editor. None of the Emacs
mail clients I have used do this (this includes VM, mu4e, gnus,
wonderlust and mew).

anyone who has gone to the effort to configure their mail system to use
an external editor and who then answers yes to the statement
"...contains values that may not be safe. Do you want to apply it?" is
someone with inherently unsafe practices. I doubt any change in wording
or phrasing would be of any help for them. However, the correct way to
deal with this would be to offer up a patch to the Emacs developers
which improve this wording. I would suggest the set of people who are
technically aware enough or have sufficient technical interest to have
adopted emacs as their email viewer and who would still answer yes to
any dialogue warning them of unsafe actions when opening content from an
unknown source is very small.

Local variables is a powerful and useful feature. Like many powerful
features, it can be abused. We differ in our opinions on whether those
safe guards are sufficient. I believe they are and I believe you are
over stating the risks. I don't believe we will arrive at any consensus
and I feel this thread has run its course. You are of course free to
respond, but I will refrain from further participation as this has
wondered off topic for org mode and I see little to be gained from
further back and forth.


--
Tim Cross



Re: [PATCH] doc/org-manual.org: Extend table formulas Lisp form documentation

2020-11-25 Thread Daniele Nicolodi
On 25/11/2020 05:37, Kyle Meyer wrote:
> Daniele Nicolodi writes:
> 
>> Hello,
>>
>> I always found the description of Lisp forms in Org table formulas not
>> extremely clear, especially in regard to the use of mode flags. The
>> attached patch tries to clarify the manual a bit.
> 
> Thanks.

Thank you for the review, Kyle. An updated patch is attached.

>> Would it be worth to mention org-sbe in the same section of the manual?
> 
> Yeah, it looks like there's no mention of org-sbe in the manual, so I
> think so (as a separate patch).

After playing a bit with org-sbe, I came to the conclusion that it is
broken beyond repair, at least without breaking it for the people that
managed to make it work for them.

I think that adding mention of it in the manual and explain all the
quirks of the macro is much more work than replace it with something
better. I tried to write a better macro, please have a look here:

https://orgmode.org/list/d429d29b-42fa-7d7b-6f3a-9fe692fd6...@grinta.net/

and the parent message for an explanation of what I think is broken in
org-sbe. Would you support adding org-sbx (for a lack of a better name)
to ob-table.el and mention it in the manual instead of org-sbe? I would
not go as far as deprecating org-sbe, just yet, but maybe soon...

Cheers,
Dan
From d39ec4465605f56d5f53a36faf4e419ae1b862f0 Mon Sep 17 00:00:00 2001
From: Daniele Nicolodi 
Date: Sat, 14 Nov 2020 18:33:41 +0100
Subject: [PATCH] doc/org-manual.org: Extend table formulas Lisp form
 documentation

doc/org-manual.org (Emacs Lisp forms as formulas): Be more
explicit about how fields are interpolated into the Lisp forms,
clarify the use of mode flags, and add a couple more examples.
---
 doc/org-manual.org | 64 +-
 1 file changed, 40 insertions(+), 24 deletions(-)

diff --git a/doc/org-manual.org b/doc/org-manual.org
index 2f7f5f847..97018d075 100644
--- a/doc/org-manual.org
+++ b/doc/org-manual.org
@@ -2178,38 +2178,54 @@ It is also possible to write a formula in Emacs Lisp.  
This can be
 useful for string manipulation and control structures, if Calc's
 functionality is not enough.
 
-If a formula starts with a single-quote followed by an opening
-parenthesis, then it is evaluated as a Lisp form.  The evaluation
-should return either a string or a number.  Just as with Calc
-formulas, you can specify modes and a ~printf~ format after
-a semicolon.
+A formula is evaluated as a Lisp form when it starts with a
+single-quote followed by an opening parenthesis.  Cell table
+references are interpolated into the Lisp form before execution.  The
+evaluation should return either a string or a number.  Evaluation
+modes and a ~printf~ format used to render the returned values can be
+specified after a semicolon.
 
-With Emacs Lisp forms, you need to be conscious about the way field
-references are interpolated into the form.  By default, a reference is
-interpolated as a Lisp string (in double-quotes) containing the field.
-If you provide the =N= mode switch, all referenced elements are
-numbers---non-number fields will be zero---and interpolated as Lisp
-numbers, without quotes.  If you provide the =L= flag, all fields are
-interpolated literally, without quotes.  For example, if you want a
-reference to be interpreted as a string by the Lisp form, enclose the
-reference operator itself in double-quotes, like ="$3"=.  Ranges are
-inserted as space-separated fields, so you can embed them in list or
-vector syntax.
+By default, references are interpolated as literal Lisp strings: the
+field content is replaced in the Lisp form stripped of leading and
+trailing white space and surrounded in double-quotes.  For example:
 
-Here are a few examples---note how the =N= mode is used when we do
-computations in Lisp:
+: '(concat $1 $2)
 
-- ='(concat (substring $1 1 2) (substring $1 0 1) (substring $1 2))= ::
+#+texinfo: @noindent
+concatenates the content of columns 1 and column 2.
+
+When the =N= flag is used, all referenced elements are parsed as
+numbers and interpolated as Lisp numbers, without quotes. Fields that
+cannot be parsed as numbers are interpolated as zeros.  For example:
+
+: '(+ $1 $2);N
+
+#+texinfo: @noindent
+adds columns 1 and 2, equivalent to Calc's =$1+$2=.  Ranges are
+inserted as space-separated fields, so they can be embedded in list or
+vector syntax. For example:
 
-  Swap the first two characters of the content of column 1.
+: '(apply '+ '($1..$4));N
 
-- ='(+ $1 $2);N= ::
+#+texinfo: @noindent
+computes the sum of columns 1 to 4, like Calc's =vsum($1..$4)=.
+
+When the =L= flag is used, all fields are interpolated literally: the
+cell content is replaced in the Lisp form stripped of leading and
+trailing white space and without quotes.  If a reference is intended
+to be interpreted as a string by the Lisp form, the reference operator
+itself should be enclosed in double-quotes, like ="$3"=.  The =L= flag
+is useful when strings and numbers are used in the same Lisp 

Re: Bug: Font highlighting with in-line latex math broken [9.4 (9.4-dist)]

2020-11-25 Thread Edmund Christian Herenz
Hi,

me again...

On Wed, Nov 25, 2020 at 04:32:28PM -0300, Edmund Christian Herenz wrote:
> This is only cosmetical, but a bit annoying.

Ah, and while we are at it

Entering a line break in $math
mode$ is also valid and should not break the font highlighting.

CH.




Re: [PATCH] ob-ruby.el: allow specification of ruby command w/header arg

2020-11-25 Thread Juri Linkov
>> ob-ruby.el: allow specification of session ruby command w/header arg
>>
>> * lisp/ob-ruby.el (org-babel-ruby-initiate-session): Use a header argument
>> to get ruby command.
>
> I've ported it to the Org repo, but please avoid doing this in the
> future.  The preferred method is for development to happen through
> Org's list and repo.

Thanks, point taken.

Now in https://github.com/nonsequitur/inf-ruby/issues/121
we came to the conclusion that another patch is needed for ob-ruby.el.

This patch should allow sharing the same :session among code blocks
for the same root dir only when :session is nil.  But in case of
a named :session, its commands always should be encapsulated
within the buffer with the same name as :session.

Therefore I'm submitting the patch here for inclusion
to the Org repo.  Thanks in advance.

>From a20049fc53f4474b3c6d50da08369dbb67af9c38 Mon Sep 17 00:00:00 2001
From: Juri Linkov 
Date: Wed, 25 Nov 2020 21:35:01 +0200
Subject: [PATCH] ob-ruby.el: Don't reuse the same buffer among different named
 sessions

* lisp/ob-ruby.el (org-babel-ruby-initiate-session): Instead of
run-ruby that always insists on using the existing buffer, use
run-ruby-or-pop-to-buffer that allows using the buffer with the
same name as :session header arg.  Use the existing buffer
returned by inf-ruby-buffer only when :session header arg is nil.
https://github.com/nonsequitur/inf-ruby/issues/121
---
 lisp/ob-ruby.el | 7 ++-
 1 file changed, 6 insertions(+), 1 deletion(-)

diff --git a/lisp/ob-ruby.el b/lisp/ob-ruby.el
index 5ad006429..47548ff7b 100644
--- a/lisp/ob-ruby.el
+++ b/lisp/ob-ruby.el
@@ -40,6 +40,8 @@
 (require 'org-macs)
 
 (declare-function run-ruby "ext:inf-ruby" ( command name))
+(declare-function run-ruby-or-pop-to-buffer "ext:inf-ruby" (command  name buffer))
+(declare-function inf-ruby-buffer "ext:inf-ruby" ())
 (declare-function xmp "ext:rcodetools" ( option))
 
 (defvar inf-ruby-default-implementation)
@@ -163,7 +165,10 @@ org-babel-ruby-initiate-session
 inf-ruby-implementations
 	   (buffer (get-buffer (format "*%s*" session)))
 	   (session-buffer (or buffer (save-window-excursion
-	(run-ruby cmd session)
+	(run-ruby-or-pop-to-buffer
+	 cmd (or session "ruby")
+	 (unless session
+	   (inf-ruby-buffer)))
 	(current-buffer)
   (if (org-babel-comint-buffer-livep session-buffer)
 	  (progn (sit-for .25) session-buffer)
-- 
2.25.1



Re: consistent behavior across babel languages

2020-11-25 Thread Tom Gillespie
Hi Ian,
Thanks for getting this started. I have been collecting a list of
org babel issues and worg is definitely a better place to put them
than in one big email. Since there are so many different features that
a babel language implementation can support I don't want to try to put
them all in one table quite yet but I think it may be helpful to do so
eventually. Until that time does it make sense to add new sections to
the file that cover specific features? For example I have tests on
TRAMP support and/or support for execution via a remote session that I
would like to add from my internal notes. Can I add it as a new
section? Best!
Tom



On Wed, Nov 25, 2020 at 2:06 AM Tim Cross  wrote:
>
>
> ian martins  writes:
>
> > Something I've found challenging is the inconsistency between babel
> > languages. It makes it difficult for a babel user to get a source
> > block to do what they want, or for a babel developer to even know what
> > correct behavior is.
> >
> > I'm not sure if anything can be done since changes will likely break
> > existing behavior, but it's good to at least know what the rule is and
> > where the exceptions to the rule are. To that end I started a page on
> > worg [1] to document current behavior for actions taken across babel
> > languages.
> >
> > [1] https://orgmode.org/worg/org-contrib/babel/languages/lang-compat.html
>
>
> this is a great initiative Ian. First step in addressing inconsistencies
> is documenting them. I will try to allocate time on the weekend to
> review what you have and see if there are any I know of which you have
> not included.
>
>
> --
> Tim Cross
>



Bug: Font highlighting with in-line latex math broken [9.4 (9.4-dist)]

2020-11-25 Thread Edmund Christian Herenz
Hi,

In AucTeX inserting

   $1\times10^{-18}$erg\,s\,cm$^{-2}$

Will correctly highlight $1\times10^{-18}$ and $^{-2}$ as math.
However, in org mode I get erg\,s\,cm highlighted.

This is only cosmetical, but a bit annoying.

With best regards,
Christian Herenz



Re: org-sbe to automate some source block executions

2020-11-25 Thread Jean Louis
* Ihor Radchenko  [2020-11-25 14:40]:
> > I am trying to implement it, do you see here anything below that I am
> > doing wrong? Error is that it cannot find source block stages
> 
> You should assign a name to your source block. Also, I am clueless why
> you need a variable ":var results=stages". You should probably do
> something like the following:

Because I did not know #+NAME: is there for source block names. Now I
have solved it all.

As I am transitioning with tasks it becomes also little questionable
how to structure the rest of files.



Re: Browser org-capture bookmarklet

2020-11-25 Thread Jean Louis
* Ihor Radchenko  [2020-11-25 15:53]:
> Probably, you did not define the capture template your browser
> extension/bookmarklet expects (it's usually a single letter). You may
> look at https://github.com/sprig/org-capture-extension/issues/1 for
> hints how to solve the problem.

Now I have transitioned all the captured Org notes to Hyperscope and
minimized the org-capture-templates to this below, and now it works
again just fine!

(("p" "Protocol" entry
  (file+headline "~/Documents/Orgnotes.org" "Inbox")
  "* %^{Title}
Source: %u, %c
 #+BEGIN_QUOTE
%i
#+END_QUOTE


%?")
 ("L" "Protocol Link" entry
  (file+headline "~/Documents/Orgnotes.org" "Inbox")
  "* %:description
%:link"))



Re: Browser org-capture bookmarklet

2020-11-25 Thread Jean Louis
* Ihor Radchenko  [2020-11-25 15:53]:
> Probably, you did not define the capture template your browser
> extension/bookmarklet expects (it's usually a single letter). You may
> look at https://github.com/sprig/org-capture-extension/issues/1 for
> hints how to solve the problem.

I got it working well, everything worked and I did not change
anything. Templates are defined and yesterday I started collecting.





Re: Local variables insecurities - Re: One vs many directories

2020-11-25 Thread Jean Louis
* Eric S Fraga  [2020-11-25 16:58]:
> On Wednesday, 25 Nov 2020 at 16:13, Jean Louis wrote:
> > I use Mutt.
> > The message is opened in Emacs in mail-mode
> 
> Ah, so mutt saves content in a file which is then opened by
> Emacs.  Okay, that makes sense.  Gnus does things the other way around:
> opens the buffer (associated with a file in the draft directory),
> inserts the content, and then puts the user in control.  File local
> variables don't get a chance to be interpreted then.

That is specific to Gnus. Any file opened by Emacs any will still
invoke the dialogue for local variables.

> > Then I have been testing and even text files invoke local variables.
> 
> Yes, of course.  That's the whole point?

You know that point is bad design and assumption that every user is
programmer who knows what are local variables and what are definitions
of Emacs functions, it is incredible situation.



Re: Local variables insecurities - Re: One vs many directories

2020-11-25 Thread Eric S Fraga
On Wednesday, 25 Nov 2020 at 16:13, Jean Louis wrote:
> I use Mutt.
> The message is opened in Emacs in mail-mode

Ah, so mutt saves content in a file which is then opened by
Emacs.  Okay, that makes sense.  Gnus does things the other way around:
opens the buffer (associated with a file in the draft directory),
inserts the content, and then puts the user in control.  File local
variables don't get a chance to be interpreted then.

> Then I have been testing and even text files invoke local variables.

Yes, of course.  That's the whole point?

(and, yes, I've been reading the thread so I know the concerns about
security etc.)

-- 
: Eric S Fraga via Emacs 28.0.50, Org release_9.4-118-g2a4578.dirty



Re: Local variables insecurities - Re: One vs many directories

2020-11-25 Thread Jean Louis
* Eric S Fraga  [2020-11-25 16:06]:
> On Wednesday, 25 Nov 2020 at 15:38, Jean Louis wrote:
> > I have not configured anything. In fact I have opened the email and I
> > was surprised that I am getting those dialogues to execute local
> > variables. 
> 
> Very strange.  It was my email that instigated this part of the
> thread.  I can view my email and I do not get asked to set any locate
> variables or, in this case, evaluate the elisp expression.  Out of
> curiosity, what mail reader did you use that actually interprets file
> local variables?  Gnus article view does not.

I use Mutt.

The message is opened in Emacs in mail-mode

Then I have been testing and even text files invoke local variables.

When I send messages I often send it from Emacs, but that is different
subject.



Re: One vs many directories

2020-11-25 Thread Jean Louis
* Ihor Radchenko  [2020-11-25 16:16]:
> 
> > ... It does
> > evaluates and I get the result in the message buffer, but it does not
> > expands in the Org buffer.
> 
> It is expected behaviour. According to the docstring of org-sbe, it only
> returns the value, but does not actually change buffer.
> 
> If you want to replace the RESULTS, you need to use the following:
> 
> # Local Variables:
> # eval: (let ((pos (org-babel-find-named-block "stages"))) (when pos 
> (save-excursion (org-with-point-at pos (org-babel-execute-src-block)
> # End:

That works well, thank you for the tip.

Of course I will not be writing all that by hand, program would simply
invoke the creation and generation of Org file and write headings and
the local variables. Next time I open the Org file related to that
person I can see all the pending tasks or tasks done with hyperlinks
to their corresponding actual tracking places in the database.

I have already made the function to capture Org tasks into the
database, concept is here:

(defun hyperscope-capture-org-heading ()
  "Captures Org heading and stores it in the Hyperscope dynamic
  knowledge repository"
  (interactive)
  (let* ((body (org-copy-subtree nil nil nil t))
 (body (pop kill-ring))
 (body (org-no-properties body))
 (heading (org-get-heading))
 (created (org-property-values "CREATED"))
 (date (if created (substring (car created) 1 11) nil))
 (body (with-temp-buffer
 (insert body)
 (org-mode)
 (org-back-to-heading)
 (kill-line)
 (delete-matching-lines ":PROPERTIES:")
 (delete-matching-lines ":CREATED:")
 (delete-matching-lines ":ID:")
 (delete-matching-lines ":END:")
 (buffer-string
(hyperscope-add-note-to-set heading body date)))

So I am now transitioning all Org tasks into little bit different and
more streamlined system for me personally.

Nevertheless, when I use a browser I can still use org-capture and
later just parse all headings automatically and insert into the
database.



Re: One vs many directories

2020-11-25 Thread Jean Louis
* Eric S Fraga  [2020-11-25 16:08]:
> On Wednesday, 25 Nov 2020 at 14:46, Jean Louis wrote:
> > I have got it to work as I had to name the source block. 
> 
> Yes, org-sbe looks for the first source block with the name
> given.  Nothing to do with headings etc.
> 
> > It does evaluates and I get the result in the message buffer, but it
> > does not expands in the Org buffer. That is what I wish to find out
> > how.
> 
> What do you get on the same src block if you explicitly C-c C-c there?

I get results in the buffer. That is what I would like, to get
expanded results in the buffer upon opening of the Org file.




org-indent-line of lines ending blocks

2020-11-25 Thread Jarmo Hurri


Greetings.

# 
---
* Indentation of lines ending blocks
  #+begin_example
My wish is that when the cursor is on the line indicating end of this block 
(end_example), 
and (org-indent-line) is evaluated, the line would be indented at the level 
of the 
begin_example which begins the block. Is this feasible / possible?
#+end_example
# 
---

All the best, and stay safe.

Jarmo




Font lock in inner blocks [was: Re: Differentiate source blocks in export?]

2020-11-25 Thread Joost Kremers


On Wed, Nov 25 2020, Eric S Fraga wrote:
> On Wednesday, 25 Nov 2020 at 09:37, Joost Kremers wrote:
>> I like this solution for the "Org-ness" of the syntax, and yes, =C-c
>> '= works, but font lock is gone...
>
> Yes, font lock is gone unfortunately but I am not sure why that
> is.  Something for somebody who understands the syntax highlighting code
> to investigate?

I guess font lock is based on the outer block, which in this case doesn't
correspond to any major mode, so there is no font lock. Or at least no
major-mode-based font lock. Org markup such as for =code= *is* font locked.

One could argue that this isn't entirely consistent with the fact that
=org-edit-special= *does* recognise the inner code block. It certainly would be
nice if font lock did, as well.

-- 
Joost Kremers
Life has its moments



Re: Local variables insecurities - Re: One vs many directories

2020-11-25 Thread Eric S Fraga
On Wednesday, 25 Nov 2020 at 15:38, Jean Louis wrote:
> I have not configured anything. In fact I have opened the email and I
> was surprised that I am getting those dialogues to execute local
> variables. 

Very strange.  It was my email that instigated this part of the
thread.  I can view my email and I do not get asked to set any locate
variables or, in this case, evaluate the elisp expression.  Out of
curiosity, what mail reader did you use that actually interprets file
local variables?  Gnus article view does not.

-- 
: Eric S Fraga via Emacs 28.0.50, Org release_9.4-118-g2a4578.dirty



Re: One vs many directories

2020-11-25 Thread Ihor Radchenko


> ... It does
> evaluates and I get the result in the message buffer, but it does not
> expands in the Org buffer.

It is expected behaviour. According to the docstring of org-sbe, it only
returns the value, but does not actually change buffer.

If you want to replace the RESULTS, you need to use the following:

# Local Variables:
# eval: (let ((pos (org-babel-find-named-block "stages"))) (when pos 
(save-excursion (org-with-point-at pos (org-babel-execute-src-block)
# End:

Best,
Ihor


Jean Louis  writes:

> * Eric S Fraga  [2020-11-24 12:46]:
>> On Tuesday, 24 Nov 2020 at 12:00, Jean Louis wrote:
>> > Can I automated the execution of Babel code upon opening of the Org
>> > file?
>> 
>> You can, by using file local variables.  For instance, for some files, I
>> do this:
>> 
>> #+begin_src org
>>   ,* local variables :noexport:
>>   # Local Variables:
>>   # eval: (org-sbe "startup")
>>   # End:
>> #+end_src
>
> I have got it to work as I had to name the source block. It does
> evaluates and I get the result in the message buffer, but it does not
> expands in the Org buffer. That is what I wish to find out how.
>
> ** Stages
> #+NAME: stages   
> #+BEGIN_SRC sql :engine postgresql :exports results :results replace
> SELECT 1 AS table;
> #+END_SRC
>
> # Local Variables:
> # eval: (org-sbe "stages")
> # End:



Re: One vs many directories

2020-11-25 Thread Eric S Fraga
On Wednesday, 25 Nov 2020 at 14:46, Jean Louis wrote:
> I have got it to work as I had to name the source block. 

Yes, org-sbe looks for the first source block with the name
given.  Nothing to do with headings etc.

> It does evaluates and I get the result in the message buffer, but it
> does not expands in the Org buffer. That is what I wish to find out
> how.

What do you get on the same src block if you explicitly C-c C-c there?

-- 
: Eric S Fraga via Emacs 28.0.50, Org release_9.4-118-g2a4578.dirty



Re: ob-python: import local package into a session

2020-11-25 Thread Joost Kremers


On Tue, Nov 24 2020, Jack Kamm wrote:
> If you install the package using either "python setup.py develop", or
> "pip install -e", then Python will install your code via symlinks
> instead of copying, so then you don't have to worry about reinstalling
> every time you make an edit.

Hey, that's good to know, thanks.

> To switch between venv's in emacs, I use pyvenv:
> https://github.com/jorgenschaefer/pyvenv

Yes, that's what I use, as well.

-- 
Joost Kremers
Life has its moments



Re: Browser org-capture bookmarklet

2020-11-25 Thread Ihor Radchenko
Probably, you did not define the capture template your browser
extension/bookmarklet expects (it's usually a single letter). You may
look at https://github.com/sprig/org-capture-extension/issues/1 for
hints how to solve the problem.

Best,
Ihor

Jean Louis  writes:

> I have started using Org-capture but now without changing Emacs
> session it stopped working with error:
>
> Greedy org-protocol handler.  Killing client.
> No server buffers remain to edit
>
> Do you know any how why this happens?



Browser org-capture bookmarklet

2020-11-25 Thread Jean Louis
I have started using Org-capture but now without changing Emacs
session it stopped working with error:

Greedy org-protocol handler.  Killing client.
No server buffers remain to edit

Do you know any how why this happens?




Local variables insecurities - Re: One vs many directories

2020-11-25 Thread Jean Louis
* Tim Cross  [2020-11-25 09:41]:
> >> Why is it a security issue? The variables do need to be close to the end
> >> — 3000 characters is only about 50 lines.
> >
> > Emacs users, Org users on our mailing lists are not so private. Their
> > names and email addresses are in the public database. Spammer can
> > construct phishing type of an email, including something like Org news
> > or something and send such email to users. Among let us say 3000
> > people there will be percentage of users that will say Y to invoke the
> > local variables due to lack of knowing what is it doing to computer.
> >
> > After that, anything becomes possible, including intrusion into
> > computer, capturing all email addresses, passwords, sending spam
> > emails from computer and so on.
> 
> this is just baseless fear mongering based on nothing but
> speculation.

My experience with such people come from last more than 25
years. CVE list is the reference I already quotes. Some thing were
improved like this:

https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=684695

So one should know how dangerous it was to introduce local variables
with possibility to automatically eval anything there.

> Of your suggested 3000 users, only a very small percentage use Emacs as
> their mail client.

Those which I know through Emacs list mostly use it. I am using it. Is
there any way to know who use and who not?

In general I am reading emails with Mutt, but I am answering with
Emacs. Sending emails is often by Emacs or by Mutt. I use sometimes
M-x rmail as well

> Of those, only an even smaller number will have their mail client
> configured to render messages with a mode that supports local

I have not configured anything. In fact I have opened the email and I
was surprised that I am getting those dialogues to execute local
variables. And I an in mail-mode now. Mutt opens new emacsclient and I
edit the file.

Obviously user does not need to configure anything. You maybe refer to
specific mode where it does not execute.

It will try to execute even if I open .TXT file.

The very design of Local variables I do not find trustful for these
explained reasons. I do not protest against it as now is too late, but
as I mentioned more educational references could be provided in the
dialogue that asks user to execute local variables or not.

> variables and even a smaller number of those would say yes to
> executing the code. In fact, anyone who has gone to the extent of
> configuring their Emacs email client to open messages with a mime
> type of x-org (or even just based on an attachment with *.org in the
> file name) is more than likely to be sufficiently technically aware
> not to say yes when asked.

I do not need to configure emacs to anything to get the local variable
execution dialogue, verify it on your side.

I can basically get any attachments by email, try to view them in
Emacs and it will execute.

> Few, if any user, is going to download some random attachment in an
> email message

Sorry, but I have no choice but to download all attachments. Majority
of email clients do not offer choice to download specific attachments
they download whole message as one.

> open it in emacs and then say yes to a message warning them that
> doing so might be dangerous.

It does not warn to be dangerous. There is no word of danger in the
dialogue.

I would rather like the dialogue to does what you say but it does not,
I would prefer it like this:

 ===
  DANGEROUS
 ===

But it is not like that, and I have elaborated why it is not like
that. Text writers are not programmers. You assume that every user
starts from your viewpoint of 30 years experienced Emacs wizard. And I
am speaking of design view point. Emacs is still called "editor"
today.

The description of Emacs I get in Hyperbola GNU/Linux-libre is:

The extensible, customizable, self-documenting real-time display
editor, without D-Bus support

Nowhere it says in the description that it can potentially expose me
to malicious evaluations of software just by opening a text file. That
you know what Emacs is really and me too, it does not make it more
secure.

We make assumptions that we will know that users will be safe, but
that is wrong assumption and there would be no CVE as I have already
referenced it it it would be so.

> Such ill-informed users have been pretty much weeded out by all the
> other scam phishing out there by now.

I would not say so. As non-native English speaker to say for users
that they are ill-informed sounds to me not appropriate. We invite
users to use Emacs, they will download, open, and may be offered to
read the ebook or other interesting text, and text will ask them to
evaluate the variables. You say that the subset of those users who
will know what is "variable", "value", "evaluation", "safe" is small
and not important, and I think this is most important for the future
of next 100 years for Emacs being useful to 

Re: One vs many directories

2020-11-25 Thread Jean Louis
* Eric S Fraga  [2020-11-24 12:46]:
> On Tuesday, 24 Nov 2020 at 12:00, Jean Louis wrote:
> > Can I automated the execution of Babel code upon opening of the Org
> > file?
> 
> You can, by using file local variables.  For instance, for some files, I
> do this:
> 
> #+begin_src org
>   ,* local variables :noexport:
>   # Local Variables:
>   # eval: (org-sbe "startup")
>   # End:
> #+end_src

I have got it to work as I had to name the source block. It does
evaluates and I get the result in the message buffer, but it does not
expands in the Org buffer. That is what I wish to find out how.

** Stages
#+NAME: stages   
#+BEGIN_SRC sql :engine postgresql :exports results :results replace
SELECT 1 AS table;
#+END_SRC

# Local Variables:
# eval: (org-sbe "stages")
# End:



Re: One vs many directories

2020-11-25 Thread Ihor Radchenko
> When I do C-c a it runs (org-agenda) but I do not have "g" and I am on
> development version. The C-c a window is made so that I cannot go with
> cursor inside and that I cannot even expect the key map neither invoke
> command by M-x and I cannot even M-:

C-c a will first show so-called agenda dispatcher asking you what kind
of agenda view you want to get. You need to press a key according to
the popup window (i.e. `t' to see all not done items). Then, you will
get the proper agenda buffer with all the keymaps set and `g' bound to
refreshing the chosen agenda view in the buffer.

> All that is wrong and not aligned to Emacs common interface. It is bug
> that bugs. Agenda buffer should allow users those standard Emacs
> features.

I am wondering what is the common Emacs interface you refer to. I am not
aware about any standard way to prompt user while also showing detailed
description of what to expect from different choices.

Best,
Ihor

Jean Louis  writes:

> * Dr. Arne Babenhauserheide  [2020-11-25 11:14]:
>> 
>> Jean Louis  writes:
>> 
>> > * Dr. Arne Babenhauserheide  [2020-11-24 21:48]:
>> >> 
>> >> Jean Louis  writes:
>> >> 
>> >> > Some people maybe access multiple Org files through Agenda, me I
>> >> > don't. Some items are "non existent" and I do not know how to ask
>> >> > agenda to refresh itself.
>> >> 
>> >> Simply press the letter g.
>> >
>> > What function is on g on your side?
>> 
>> (org-agenda-redo-all  EXHAUSTIVE)
>
> When I do C-c a it runs (org-agenda) but I do not have "g" and I am on
> development version. The C-c a window is made so that I cannot go with
> cursor inside and that I cannot even expect the key map neither invoke
> command by M-x and I cannot even M-:
>
> All that is wrong and not aligned to Emacs common interface. It is bug
> that bugs. Agenda buffer should allow users those standard Emacs
> features.
>
>> > Thank you. I have plan to switch anything action related to
>> > database system and use Org to view tasks, not to handle or store
>> > them.



Re: org-sbe to automate some source block executions

2020-11-25 Thread Ihor Radchenko
> I am trying to implement it, do you see here anything below that I am
> doing wrong? Error is that it cannot find source block stages

You should assign a name to your source block. Also, I am clueless why
you need a variable ":var results=stages". You should probably do
something like the following:

** Stages

#+name: stages   
#+BEGIN_SRC sql :engine postgresql :exports results
SELECT -- salesflowstages_priority AS "Priority", 
   salesflowstages_publicstage AS "Development Stages" --,
--   salesflowstages_description AS "Description" 
FROM salesflowstages
WHERE salesflowstages_salesflows = 1 
ORDER BY salesflowstages_priority ASC
#+END_SRC

# Local Variables:
# eval: (org-sbe "stages")
# End:

Best,
Ihor


Jean Louis  writes:

> * Eric S Fraga  [2020-11-24 12:46]:
>> On Tuesday, 24 Nov 2020 at 12:00, Jean Louis wrote:
>> > Can I automated the execution of Babel code upon opening of the Org
>> > file?
>> 
>> You can, by using file local variables.  For instance, for some files, I
>> do this:
>> 
>> #+begin_src org
>>   ,* local variables :noexport:
>>   # Local Variables:
>>   # eval: (org-sbe "startup")
>>   # End:
>> #+end_src
>> 
>> which will evaluate the named src block "startup" when file is
>> opened.
>
> I am trying to implement it, do you see here anything below that I am
> doing wrong? Error is that it cannot find source block stages
>
> ** Stages
>
> #+BEGIN_SRC sql :var results=stages :engine postgresql :exports results
> SELECT -- salesflowstages_priority AS "Priority", 
>salesflowstages_publicstage AS "Development Stages" --,
> --   salesflowstages_description AS "Description" 
> FROM salesflowstages
> WHERE salesflowstages_salesflows = 1 
> ORDER BY salesflowstages_priority ASC
> #+END_SRC
>
> # Local Variables:
> # eval: (org-sbe "stages")
> # End:



Re: One vs many directories

2020-11-25 Thread Jean Louis
* Texas Cyberthal  [2020-11-25 13:40]:
> > You also spoke of device, do you really mean physical device?
> 
> Brain-Computer Interface is a term that usually means an
> electromagnetic connection between nerves and electronics.  However,
> really a keyboard is a superior version of that for non-motor tasks,
> despite being insulated on both ends with skin and plastic.

hahahha you are right

> Because you built your own SME CRM.  My point is that an SME CRM
> should probably be a separate entity from a Textmind.  Their
> purposes conflict.

Really, but for me those similarities extend each other. Not that I am
sorting files by Textmind but I do have similar method in sorting
files. You sort files by tags in a hierachy, that is the concept of
Textmind. How tags are called or how they relate to each other is then
rather implementation.

Anyway I wish to know more of that, subscribe me to your mailing
list. In last days I got more ideas than I was thinking I will, all
from this Org mailing list.

And now slowly I start implementing.

I like Org but I also like more structured data.



[PATCH] add :tree-type 'month' in org-capture-templates description (and fix documentation about that)

2020-11-25 Thread Florian Dufour
Hi!

I noticed one typo in org-manual.org: 'month' value for 'tree-type'
property should not be @code{month} but only 'month'.

Moreover, I added this 'month' value to org-capture-templates
description in org-capture.el.

This is just documentation, but I had to read to code in order to use
this new value (which is awesome).

Feel free to merge this patch if you want!

Cheers,
-- 
Florian
From eabd16e7530aed5ecdd636b7c55d03d644b02df9 Mon Sep 17 00:00:00 2001
From: Florian Dufour 
Date: Wed, 25 Nov 2020 11:41:48 +0100
Subject: [PATCH] org-capture.el: Add :tree-type 'month' in
 org-capture-templates description

* lisp/org-capture.el (org-capture-templates): Enhance description
(org-capture-templates): add 'month' value for :tree-type in description

* doc/org-manual.org: Fix typo for displaying 'month' in :tree-type property
---
 doc/org-manual.org  | 2 +-
 lisp/org-capture.el | 4 +++-
 2 files changed, 4 insertions(+), 2 deletions(-)

diff --git a/doc/org-manual.org b/doc/org-manual.org
index 2f7f5f847..a0a82cac6 100644
--- a/doc/org-manual.org
+++ b/doc/org-manual.org
@@ -7730,7 +7730,7 @@ Now lets look at the elements of a template definition.  
Each entry in
 
 Use ~week~ to make a week tree instead of the month-day tree,
 i.e., place the headings for each day under a heading with the
-current ISO week.  Use @code{month} to group entries by month
+current ISO week.  Use ~month~ to group entries by month
 only.  Default is to group entries by day.
 
   - ~:unnarrowed~ ::
diff --git a/lisp/org-capture.el b/lisp/org-capture.el
index 020feb4d6..8bd233d15 100644
--- a/lisp/org-capture.el
+++ b/lisp/org-capture.el
@@ -270,7 +270,9 @@ properties are:
  :time-promptPrompt for a date/time to be used for date/week trees
  and when filling the template.
 
- :tree-type  When `week', make a week tree instead of the month tree.
+ :tree-type  When `week', make a week tree instead of the month-day
+ tree.  When `month', make a month tree instead of the
+ month-day tree.
 
  :unnarrowed Do not narrow the target buffer, simply show the
  full buffer.  Default is to narrow it so that you
-- 
2.29.0



signature.asc
Description: OpenPGP digital signature


Re: Security issues in Emacs packages

2020-11-25 Thread tomas
On Wed, Nov 25, 2020 at 12:26:11PM +0300, Jean Louis wrote:
> * to...@tuxteam.de  [2020-11-25 12:08]:
> > On Wed, Nov 25, 2020 at 11:23:27AM +0300, Jean Louis wrote:
> > 
> > [...]
> > 
> > > [...] and not from Chinese distributor [...]
> > 
> > I think this was an unnecessary slur.
> 
> Why, there is legitimate mirror in China.
> 
> I did not mean nothing wrong with it. I hope nobody gets offended.

I'm not. I don't assume (even suspect) bad intention at all.
And I don't want to make a state affair of it. I just wanted
to mirror the metaphor you used ("Chinese distributor" as
"untrusted instance") which seems somewhat problematic. We
all do this, myself not the least. I'm happy whenever someone
points that out to me.

That's all :-)

Cheers
 - t


signature.asc
Description: Digital signature


Re: One vs many directories

2020-11-25 Thread Texas Cyberthal
Hi Jean,

> There are those who die and come back and view things from above and can 
> think and use their mind even though brain was turned off temporarily.

I didn't say that the mind always turns off when the brain is damaged.

> You also spoke of device, do you really mean physical device?

Brain-Computer Interface is a term that usually means an
electromagnetic connection between nerves and electronics.  However,
really a keyboard is a superior version of that for non-motor tasks,
despite being insulated on both ends with skin and plastic.  Human
fingers are amazingly dextrous, so their bandwidth is high.
Eventually mammalian brain plasticity will permit implants to surpass
keyboards, but that will take a while and might require implantation
as a child to achieve maximum results.  The generational technology
gap moves ever inward, towards our very genes.  Obsolete thyself to
perpetuate thy meme-ery.

> I cannot get it as I do not know what is ADD.

Attention Deficit Disorder

> But I have M-x already that works.

Because you built your own SME CRM.  My point is that an SME CRM
should probably be a separate entity from a Textmind.  Their purposes
conflict.



Minor fix in org-shifttab on master

2020-11-25 Thread Gustav Wikström
Thought to mention that commit 9a154910ed is pushed to master,

It fixes an issue that drawers were opened when cycling global visibility using 
org-shifttab with a numeric argument (to only show outline levels up to that 
level).

/Gustav


org-sbe to automate some source block executions

2020-11-25 Thread Jean Louis
* Eric S Fraga  [2020-11-24 12:46]:
> On Tuesday, 24 Nov 2020 at 12:00, Jean Louis wrote:
> > Can I automated the execution of Babel code upon opening of the Org
> > file?
> 
> You can, by using file local variables.  For instance, for some files, I
> do this:
> 
> #+begin_src org
>   ,* local variables :noexport:
>   # Local Variables:
>   # eval: (org-sbe "startup")
>   # End:
> #+end_src
> 
> which will evaluate the named src block "startup" when file is
> opened.

I am trying to implement it, do you see here anything below that I am
doing wrong? Error is that it cannot find source block stages

** Stages
   
#+BEGIN_SRC sql :var results=stages :engine postgresql :exports results
SELECT -- salesflowstages_priority AS "Priority", 
   salesflowstages_publicstage AS "Development Stages" --,
--   salesflowstages_description AS "Description" 
FROM salesflowstages
WHERE salesflowstages_salesflows = 1 
ORDER BY salesflowstages_priority ASC
#+END_SRC

# Local Variables:
# eval: (org-sbe "stages")
# End:



Re: One vs many directories

2020-11-25 Thread Jean Louis
* Texas Cyberthal  [2020-11-25 09:58]:
> Hi Jean,
> 
> > Now, what is exomind?
> 
> https://cyberthal-docs.nfshost.com/cyborganize/exomind/

How I like those thoughts. 

> Mind vs brain 
> Your brain is the squishy jelly between your ears. 
> Your mind is what disappears when that jelly gets shaken too hard by
> a punch.

There are those who die and come back and view things from above and
can think and use their mind even though brain was turned off
temporarily.

> Exomind — an individual's PIMS and the personal info it contains. One
> can wax philosophical about how far the exomind extends into situated
> cognition. Perhaps the Internet is a shared global exomind. Anyway,
> Cyborganize is a personal exomind.

I just get idea how exoskelet does that.

You also spoke of device, do you really mean physical device?

> What you described is not how you think, it is how you wish your CRM
> info retrieval system to perform conveniently.  Almost nobody has a
> formal thought algorithm, because brains have ADD compared to
> computers.

I cannot get it as I do not know what is ADD.

> If you have a huge number of incoming text of a special type, such
> as customer leads, who are just cogs that you don't genuinely
> ponder, then Textmind is the wrong tool for that job.  You need SME
> CRM software.

But I have M-x already that works.

Thank you for examples of file sorting with 10 Bins.

> https://cyberthal-docs.nfshost.com/cyborganize/pubmind/

Thank you, I was reading.




Re: Security issues in Emacs packages

2020-11-25 Thread Jean Louis
* to...@tuxteam.de  [2020-11-25 12:08]:
> On Wed, Nov 25, 2020 at 11:23:27AM +0300, Jean Louis wrote:
> 
> [...]
> 
> > [...] and not from Chinese distributor [...]
> 
> I think this was an unnecessary slur.

Why, there is legitimate mirror in China.

I did not mean nothing wrong with it. I hope nobody gets offended.

I was just thinking on mirrors like when using /etc/apt/sources.list
on Debian GNU/Linux and sources come from let us say "Digital Ocean"
servers. It is not official mirror of Debian, but if packages are
signed by Debian keys then I do not mind as I know they are not
tampered.

I hope I have expressed myself now more clear.



columnview: could «id» be a list of values?

2020-11-25 Thread Uwe Brauer


Hi

I have the following structure in a document. I will include the
document at the end of the message for better readability.

The structure is as follows

* Sheet 1
** G1/S1/E1 (id G1-S1-E1)
** G2/S1/E2 (id G2-S1-E2)
** Table generated via coumnview containing the tree

* Sheet 2
** G1/S2/E1 (id G1-S2-E1)
** G2/S2/E2 (id G2-S2-E1)
** Table generated via coumnview containing the tree

But I would also have tables which are generated by a reordering of the
file such as

* Group1
** G1/S1/E1
** G1/S2/E1
** Table generated via coumnview containing the tree

* Group 2
** G2/S1/E2
** G2/S2/E2
** Table generated via coumnview containing the tree

Now it is a hassle to reorder the file in this way. 
Each subheader has a different id

So the best would be if I could do the following 

#+BEGIN: columnview  :hlines 1 :indent nil :format 
"%50id[G1-S1-E1,G1-S2-E2](Sheet) %7Status(Status){X/}"
| Sheet| Status |
|--+|
| Sheet1   | [2/2]  |
| G1-S1-E1 | [X]|
| G2-S1-E2 | [X]|
|  ||
#+END

But this seems not to be possible.

Does anybody has an idea how to achieve my goal, producing tables via
columnview form sub-header that posses a distinct id?

Thanks and regards

Uwe Brauer

File:

* Sheet 1
   :PROPERTIES:
   :COLUMNS:  %10Sheet(Sheet Nr) %10Ub(Exercise Nr)  %7STATUS(Status){X/} 
   :ID:   Sheet1
   :END:
** G1/S1/E1
   :PROPERTIES:
   :ID: G1-S1-E1
   :STATUS:   [X]
   :END:

** G2/S1/E2
   :PROPERTIES:
   :Id: G2-S1-E2
   :STATUS:   [X]
   :END:

** DONE

#+BEGIN: columnview  :hlines 1 :indent nil :format "%50id(Sheet) 
%7Status(Status){X/}"
| Sheet| Status |
|--+|
| Sheet1   | [2/2]  |
| G1-S1-E1 | [X]|
| G2-S1-E2 | [X]|
|  ||
#+END
* Sheet 2
   :PROPERTIES:
   :COLUMNS:  %10Sheet(Sheet Nr) %10Ub(Exercise Nr)  %7STATUS(Status){X/} 
   :ID:   Sheet2
   :END:
** G1/S2/E1
   :PROPERTIES:
   :ID: G1-S2-E1
   :STATUS:   [X]
   :END:

** G2/S2/E2
   :PROPERTIES:
   :Id: G2-S2-E2
   :STATUS:   [X]
   :END:

** DONE

#+BEGIN: columnview  :hlines 1 :indent nil :format "%50id(Sheet) 
%7Status(Status){X/}"
| Sheet| Status |
|--+|
| Sheet2   | [2/2]  |
| G1-S2-E1 | [X]|
| G2-S2-E2 | [X]|
|  ||
#+END



* Group 1
   :PROPERTIES:
   :COLUMNS:  %10Group(Sheet Nr) %10Ub(Exercise Nr)  %7STATUS(Status){X/} 
   :ID:   Group1
   :END:
** G1/S1/E1
   :PROPERTIES:
   :ID: G1-S1-E1
   :STATUS:   [X]
   :END:
** G1/S2/E1
   :PROPERTIES:
   :ID: G1-S2-E1
   :STATUS:   [X]
   :END:
** DONE

#+BEGIN: columnview  :hlines 1 :indent nil :format "%50id(Group) 
%7Status(Status){X/}"
| Group| Status |
|--+|
| Group1   | [2/2]  |
| G1-S1-E1 | [X]|
| G1-S2-E1 | [X]|
|  ||
#+END
* Group 2
   :PROPERTIES:
   :COLUMNS:  %10Group(Sheet Nr) %10Ub(Exercise Nr)  %7STATUS(Status){X/} 
   :ID:   Group2
   :END:

** G2/S1/E2
   :PROPERTIES:
   :Id: G2-S1-E2
   :STATUS:   [X]
   :END:
** G2/S2/E2
   :PROPERTIES:
   :Id: G2-S2-E2
   :STATUS:   [X]
   :END:
** DONE

#+BEGIN: columnview  :hlines 1 :indent nil :format "%50id(Group) 
%7Status(Status){X/}"
| Group| Status |
|--+|
| Group2   | [2/2]  |
| G2-S1-E2 | [X]|
| G2-S2-E2 | [X]|
|  ||
#+END




issues and limitations with macros and export process

2020-11-25 Thread Dauer, Michael
Hi,

My use case:
I export various sub-branches with different exporters (html, latex,
ox-taskjuggler) via dispatcher and more via custom code.

The issues:
1. Include files about the current branch and on file level are not
considered.
2. Macros that are evaluated during expansion only see the sub-branch and
cannot access e.g. properties on file level or in parent nodes.
3. Interestingly static macros created on file level are
resolved correctly. Where/when are they collected?

Guessed root cause:
During the export process as a first step the sub-branch is copied into a
"temp" buffer, and all macro expansions happen there. In this buffer is no
information left from file level or parent nodes.

Possible solutions / work-arounds:
1. Solution:
a. First copy whole buffer in first temp buffer.
b. Insert include files in sub-branch, nodes above, and file level (at
least before the sub-branch). Include files can hold macro definitions too.
c. Expand macros in sub-branch. (wider scope needed if recursive macros are
expected to work)
d. Do further export processing of sub-branch maybe in a second temp buffer
2. Work-around A:
a. Save reference to original buffer before copying into temp buffer.
b. Do macro evaluation during expansion while temporarily switching back to
original buffer
This does not resolve the limitations of the include files.
3. Work-around B:
a. Have a means to know what the original buffer was.
b. Switch back to the original buffer explicitly in the code of the eval
macros.
This is a quite limited improvement, which also increases the minimal
complexity of eval macros.

The solution and work-around A would imply changes in the code base with
potential incompatibilities to existing custom code.

For work-around B I haven't found a way to temporarily switch to the
original buffer.
1. There is no hook for before the switch to the temp buffer, in which I
could save the reference to the original buffer.
2. The change-buffer-list hook is not reliable to save this reference
either.
3. Currently I'm trying to derive the original buffer from the name of the
temp buffer (removing the <2> suffix) but there seems to be a narrowing to
the sub-tree even in the original buffer at that point of time. If I just
widen then this would cause side effects on the narrowings done by the user
before, right?

Please comment on my thoughts. Tell me if I missed something relevant. And
direct me to a better solution if you know.

Thanks


Re: Differentiate source blocks in export?

2020-11-25 Thread Eric S Fraga
On Wednesday, 25 Nov 2020 at 09:37, Joost Kremers wrote:
> I like this solution for the "Org-ness" of the syntax, and yes, =C-c
> '= works, but font lock is gone...

Yes, font lock is gone unfortunately but I am not sure why that
is.  Something for somebody who understands the syntax highlighting code
to investigate?

-- 
: Eric S Fraga via Emacs 28.0.50, Org release_9.4-118-g2a4578.dirty



Re: Security issues in Emacs packages

2020-11-25 Thread tomas
On Wed, Nov 25, 2020 at 11:23:27AM +0300, Jean Louis wrote:

[...]

> [...] and not from Chinese distributor [...]

I think this was an unnecessary slur.

Cheers
 - t


signature.asc
Description: Digital signature


Re: One vs many directories

2020-11-25 Thread Jean Louis
* Dr. Arne Babenhauserheide  [2020-11-25 11:14]:
> 
> Jean Louis  writes:
> 
> > * Dr. Arne Babenhauserheide  [2020-11-24 21:48]:
> >> 
> >> Jean Louis  writes:
> >> 
> >> > Some people maybe access multiple Org files through Agenda, me I
> >> > don't. Some items are "non existent" and I do not know how to ask
> >> > agenda to refresh itself.
> >> 
> >> Simply press the letter g.
> >
> > What function is on g on your side?
> 
> (org-agenda-redo-all  EXHAUSTIVE)

When I do C-c a it runs (org-agenda) but I do not have "g" and I am on
development version. The C-c a window is made so that I cannot go with
cursor inside and that I cannot even expect the key map neither invoke
command by M-x and I cannot even M-:

All that is wrong and not aligned to Emacs common interface. It is bug
that bugs. Agenda buffer should allow users those standard Emacs
features.

> > Thank you. I have plan to switch anything action related to
> > database system and use Org to view tasks, not to handle or store
> > them.
> 
> That might cost you reaction-time in the end, because org then cannot
> just keep the file open.

Do you mean my reaction time or Org reaction time? I did not
understand about to keep file open, how do you mean?

You remember I said Org files with tasks are on my side almost always
related to some people, among them I am included. So I find Org file
per person in the directory wherever it is by clicking F4. Then task
list in the Org file would update itself as I have got the tip on how
to execute the SQL block. Do you mean that SQL block would cost me
reaction time? If so, that can be. In the Hyperscope database managed
by Emacs and displayed within tabulated-list-mode on this T410
Thinkpad the list of entries looks almost instant, I just press left
or right and I get new list. Each time there is SQL query.

Without you telling me I would not be thinking on that, but it is
true, the larger or more complex the query then it could be
processing.

But if there is large list of tasks not done, that means anyway that
something is terribly wrong with it.

Those tasks done, can appear under different heading. But then I
cannot selectively automatically execute block under one heading at
file opening time while not executing block under different heading.




Local variables liberties

2020-11-25 Thread Jean Louis
* Dr. Arne Babenhauserheide  [2020-11-25 11:11]:
> 
> Jean Louis  writes:
> 
> > * Dr. Arne Babenhauserheide  [2020-11-24 21:51]:
> >> 
> >> Jean Louis  writes:
> >> >> The start of the local variables list should be no more than 3000
> >> >> > characters from the end of the file
> >> >> 
> >> >> 
> >> >> Given the length of the email, I guess this is why Emacs saw the 
> >> >> variables
> >> >> as being within the correct range.
> >> >
> >> > Yes thank you. I was thinking Emacs will do that only in files where
> >> > it recognizes some comments or no comments and that variables need
> >> > to be pretty down in the file, on the bottom. Now I learn it is not
> >> > so.
> >> >
> >> > That is security issue.
> >> 
> >> Why is it a security issue? The variables do need to be close to the end
> >> — 3000 characters is only about 50 lines.
> >
> > Emacs users, Org users on our mailing lists are not so private. Their
> > names and email addresses are in the public database. Spammer can
> > construct phishing type of an email, including something like Org news
> > or something and send such email to users. Among let us say 3000
> > people there will be percentage of users that will say Y to invoke the
> > local variables due to lack of knowing what is it doing to computer.
> 
> That isn’t what I meant with my question. What I meant is: Why is it a
> security issue that the variable cannot only be at the exact end of the
> file but instead can be in the last 3000 characters of the file?

Not that I really answered on that meaning. In that meaning I find it
troublesome that Local variables will work by formatting of any kind

Example would be when there are OTHER things intertwined with local
variables. The file with this text below still works, and it is not
necessary that it works in that way. My eye goes to the end of
file. My expectation is not that local variables work anywhere in the
file just by counting number of characters as such ends of files can
look any how, formatting is too free and I would make it rigid that
users can look with their eyes easily to where those local variables
really are.

I would rather support that Local variables cannot start in the column
like 10, maybe they can start in column up to 5th or something similar
and that their End: cannot be longer then few lines from the bottom
and to be highlighted by default.


Some file

Lorem ipsum dolor sit amet, consectetur adipiscing elit. Donec a diam
lectus. Sed sit amet ipsum mauris. Maecenas congue ligula ac quam
viverra nec consectetur ante hendrerit. Donec et mollis
dolor. Praesent et diam eget libero egestas mattis sit amet vitae
augue. Nam tincidunt congue enim, ut porta lorem lacinia
consectetur. Donec ut libero sed arcu vehicula ultricies a non  
Local Variables:
consectetur. Donec ut libero sed arcu vehicula ultricies a non  
ivy-mode: 1
consectetur. Donec ut libero sed arcu vehicula ultricies a non  
End:
tortor. Lorem ipsum dolor sit amet, consectetur adipiscing  
ivy-mode: 1
elit. Aenean ut gravida lorem. Ut turpis felis, pulvinar a semper sed,  
End:
adipiscing id dolor. Pellentesque auctor nisi id magna consequat
  
sagittis. Curabitur dapibus enim sit amet elit pharetra tincidunt
feugiat nisl imperdiet. Ut convallis libero in urna ultrices
accumsan. Donec sed odio eros. Donec viverra mi quis quam pulvinar at
malesuada arcu rhoncus. Cum sociis natoque penatibus et magnis dis
parturient montes, nascetur ridiculus mus. In rutrum accumsan
ultricies. Mauris vitae nisi at sem facilisis semper ac in est.




Security issues in Emacs packages

2020-11-25 Thread Jean Louis
* Tim Cross  [2020-11-25 10:01]:
> 
> Jean Louis  writes:
> 
> > * Tim Cross  [2020-11-24 23:40]:
> >> If people are really concerned about security, they should look first at
> >> their use of repositories like MELPA. There is no formal review or
> >> analysis of packages in these repositories, yet people will happily
> >> select some package and install it.
> >
> > Interesting that you are one who mentions that. There are just few
> > people ever mentioned it.
> >
> > I am still in process of the review of MELPA packages and its
> > system. There are many security issues.
> >
> > Package signing is one example. It does not offer much of security
> > when packages are signed automatically, but it raises level of
> > security.
> >
> > MELPA packages and archive-contents are not PGP signed, while GNU ELPA
> > packages are signed.
> >
> 
> IMO signing of packages is irrelevant when there is no formal review
> process or even any formal process to verify the credentials of
> signatures. In fact, just adding signing would likely be
> coutner-productive as it would give the impression of some sort of
> security where there is none.

When user receives a signed package from GNU ELPA, that means it comes
from GNU ELPA. If signed package is then distributed by mirrors or
other websites users who enable package signature verifications will
know that package still comes from GNU ELPA, and not from Chinese
distributor. If the package is tampered the signature verification
will not allow installation of the package.

Variable package-check-signature is not enabled by default and it is
due to similar issue just as local variables.

Users who seek to verify the credential of the signatures can do so in
human non-automated way as that is also how GnuPG instructions advise
users to do so. The GPG/PGP keys are in ~/.emacs.d/elpa/gnupg/ and
users can contact GNU ELPA maintainers and verify the signature. M-x
package-install will automatically verify signatures if the option
package-check-signature has been enabled.

That it does add to security shows the fact that GNU/Linux
distributions do sign packages. There is difference if the package
comes from trusted source or not trusted source. 

> Basically, anyone can upload anything to MELPA.

Maintainers verifies the package initially for certain conventions,
after that, how I have understood it, packages are automatically
pulled from Git. The more authors give packages to MELPA the more
insecurity probability is raised.

GNU ELPA how I understand it, I may be wrong, works like this:

1) packages are uploaded to GNU ELPA
2) then automatically signed by GNU ELPA PGP keys
3) offered for distribution

The point number (1) is human, not automated. Author decides when is
the package ripe for distribution and what is "release".

Git repository is never release and not meant to be "release". Git is
for collaborative development and users are made blind that it is some
kind of release while it is not. One shall always assume that Git
repository contains development versions not ready for public.

MELPA pulls those packages, correct me if I am wrong, automatically
from Git repositories without regard if the package is actually
release. That does not align or respect the established Emacs
conventions how packages should be released, if they are multi files
they should be in .tar file otherwise .el and there are version
numbers that MELPA fiddles with and makes possible conflicts and
introduces confusions.

In GNU ELPA authors decide when package is release ready and when
it should be released.

In MELPA authors only apply for their packages to be pulled out
automatically and offered for distribution.

Both repositories could be compromised but probability is becoming
larger and larger that by automatic pulling of packages something
worse happens.

MELPA cannot know possibly who is behind authors who offer those
packages for distribution and who has access or who may do something
malicious.

Some new similar package like angry-police-captain could serve for
potential attacks.

#+TITLE: <2020-10-23 Fri 18:28>  WTF angry-police-captain
#+AUTHOR: Jean Louis
- This should scrap information from a third party unknown website and
  show it in minibuffer. Function does not work, and yes, it is just
  one function inside. Good example of nonsensical
  "packages". *Deleted*

While similar packages can be made for entertainment they can be also
used to track users and save data that should not be saved. Update to
this package would not be checked by MELPA, and users who have enabled
it would continue using it. And package could suddenly start doing
something else. Author of the package could know how many users are
using it as package is actually fetching from their website. By
fetching the information from website the website can know many things
about those users such as their locations, operating system and
versions, etc. and could invoke specific malicious stuff for those
specific users 

Re: Differentiate source blocks in export?

2020-11-25 Thread Joost Kremers


On Wed, Nov 25 2020, Eric S Fraga wrote:
> On Tuesday, 24 Nov 2020 at 23:02, Joost Kremers wrote:
>> That, unfortunately, seems to make it impossible to edit the source block as
>> Octave (or in my case Python) code. Pressing =C-c '= in this source block 
>> gives
>> me an Org buffer.
>
> Take away the begin...end org block itself which I only put to protect
> the org code for the email.  You should then be able to edit the src
> block with no problem.

Oh, sorry, I didn't realise that... 

>   #+begin_myclass
>   #+begin_src octave
>   y = 3 * x + 5
>   #+end_src
>   #+end_myclass
>
> If I have point within the octave src block, C-c ' works fine for me.

I like this solution for the "Org-ness" of the syntax, and yes, =C-c '= works,
but font lock is gone...

-- 
Joost Kremers
Life has its moments



Re: One vs many directories

2020-11-25 Thread Dr. Arne Babenhauserheide

Jean Louis  writes:

> * Dr. Arne Babenhauserheide  [2020-11-24 21:48]:
>> 
>> Jean Louis  writes:
>> 
>> > Some people maybe access multiple Org files through Agenda, me I
>> > don't. Some items are "non existent" and I do not know how to ask
>> > agenda to refresh itself.
>> 
>> Simply press the letter g.
>
> What function is on g on your side?

(org-agenda-redo-all  EXHAUSTIVE)

> I press g and I get error: invalid key g
>
>> For my own setup I run code in a hook to update the agenda whenever I
>> change a TODO state, clock in or clock out, but that has performance
>> problems when I do it while the Agenda is shown.
>
> That sounds that it will use computing power. 

Yes, it does, but it gives me the interaction I want.

> Thank you. I have plan
> to switch anything action related to database system and use Org to
> view tasks, not to handle or store them.

That might cost you reaction-time in the end, because org then cannot
just keep the file open.

Best wishes,
Arne
-- 
Unpolitisch sein
heißt politisch sein
ohne es zu merken


signature.asc
Description: PGP signature


Re: One vs many directories

2020-11-25 Thread Dr. Arne Babenhauserheide

Jean Louis  writes:

> * Dr. Arne Babenhauserheide  [2020-11-24 21:51]:
>> 
>> Jean Louis  writes:
>> >> The start of the local variables list should be no more than 3000
>> >> > characters from the end of the file
>> >> 
>> >> 
>> >> Given the length of the email, I guess this is why Emacs saw the variables
>> >> as being within the correct range.
>> >
>> > Yes thank you. I was thinking Emacs will do that only in files where
>> > it recognizes some comments or no comments and that variables need
>> > to be pretty down in the file, on the bottom. Now I learn it is not
>> > so.
>> >
>> > That is security issue.
>> 
>> Why is it a security issue? The variables do need to be close to the end
>> — 3000 characters is only about 50 lines.
>
> Emacs users, Org users on our mailing lists are not so private. Their
> names and email addresses are in the public database. Spammer can
> construct phishing type of an email, including something like Org news
> or something and send such email to users. Among let us say 3000
> people there will be percentage of users that will say Y to invoke the
> local variables due to lack of knowing what is it doing to computer.

That isn’t what I meant with my question. What I meant is: Why is it a
security issue that the variable cannot only be at the exact end of the
file but instead can be in the last 3000 characters of the file?

Best wishes,
Arne
-- 
Unpolitisch sein
heißt politisch sein
ohne es zu merken


signature.asc
Description: PGP signature


Re: Differentiate source blocks in export?

2020-11-25 Thread Eric S Fraga
On Tuesday, 24 Nov 2020 at 23:02, Joost Kremers wrote:
> That, unfortunately, seems to make it impossible to edit the source block as
> Octave (or in my case Python) code. Pressing =C-c '= in this source block 
> gives
> me an Org buffer.

Take away the begin...end org block itself which I only put to protect
the org code for the email.  You should then be able to edit the src
block with no problem.

  #+begin_myclass
  #+begin_src octave
  y = 3 * x + 5
  #+end_src
  #+end_myclass

If I have point within the octave src block, C-c ' works fine for me.

-- 
: Eric S Fraga via Emacs 28.0.50, Org release_9.4-118-g2a4578.dirty