Re: [BUG] Org requested me to send this after doing org-cycle [9.5.1 (release_9.5.1-205-g20ed79 @ /home/jay/Builds/org-mode/lisp/)]

[Jay Dresser]

Here is Warnings and Backtrace

> From: Ihor Radchenko 
> To: Jay Dresser 
> Cc: emacs-orgmode@gnu.org
> Subject: Re: [BUG] Org requested me to send this after doing org-cycle
>  [9.5.1 (release_9.5.1-205-g20ed79 @ /home/jay/Builds/org-mode/lisp/)]
> Date: Sat, 04 Dec 2021 21:56:25 +0800
> 
> Jay Dresser  writes:
> >
> > I did org-cycle, via S-TAB, and an error occurred. I don't know why.
> 
> Thanks for reporting!
> 
> Could you also post the warning text next time you encounter it? The
> text contains important clues that can help us to understand what went
> wrong.
> 
> Best,
> Ihor

== Backtrace ==
Debugger entered--Lisp error: (error "rx ‘**’ range error")
  signal(error ("rx ‘**’ range error"))
  error("rx `%s' range error" **)
  rx--translate-bounded-repetition(** (1 0 "*"))
  rx--translate-**((1 0 "*"))
  rx--translate-form((** 1 0 "*"))
  rx--translate((** 1 0 "*"))
  mapcar(rx--translate (line-start (** 1 0 "*") " "))
  rx--translate-seq((line-start (** 1 0 "*") " "))
  rx--translate-form((seq line-start (** 1 0 "*") " "))
  rx--translate((seq line-start (** 1 0 "*") " "))
  rx-to-string((seq line-start (** 1 0 "*") " "))
  (let ((re (rx-to-string (cons 'seq (cons 'line-start (cons (cons ... ...) 
'...)) (if (re-search-forward re nil t) (line-beginning-position) 
(point-max)))
  (save-excursion (let ((re (rx-to-string (cons 'seq (cons 'line-start (cons 
... ...)) (if (re-search-forward re nil t) (line-beginning-position) 
(point-max
  (let* ((begin (point)) (true-level (prog1 (skip-chars-forward "*") 
(skip-chars-forward " \11"))) (level (org-reduced-level true-level)) (todo (and 
org-todo-regexp (let (case-fold-search) (looking-at (concat org-todo-regexp " 
"))) (progn (goto-char (match-end 0)) (skip-chars-forward " \11") (match-string 
1 (todo-type (and todo (if (member todo org-done-keywords) 'done 'todo))) 
(priority (and (looking-at "\\[#.\\][ \11]*") (progn (goto-char (match-end 0)) 
(aref (match-string 0) 2 (commentedp (and (let ((case-fold-search nil)) 
(looking-at org-element-comment-string)) (goto-char (match-end 0 
(title-start (prog1 (point) (if (or todo priority commentedp) nil 
(skip-chars-backward " \11" (tags (if (re-search-forward "[ 
\11]+\\(:[[:alnum:]_@#%:]+:\\)[ \11]*$" (line-end-position) 'move) (progn 
(goto-char (match-beginning 0)) (org-split-string (match-string 1) ":" 
(title-end (point)) (raw-value (org-trim (buffer-substring-no-properties 
title-start title-end))) (archivedp (member org-element-archive-tag tags)) 
(footnote-section-p (and org-footnote-section (string= org-footnote-section 
raw-value))) (standard-props (org-element--get-node-properties)) (time-props 
(org-element--get-time-properties)) (end (save-excursion (let ((re 
(rx-to-string ...))) (if (re-search-forward re nil t) (line-beginning-position) 
(point-max) (contents-begin (save-excursion (forward-line) 
(skip-chars-forward " \15\11\n" end) (and (/= (point) end) 
(line-beginning-position (contents-end (and contents-begin (progn 
(goto-char end) (skip-chars-backward " \15\11\n") (line-beginning-position 
2 (robust-begin (and contents-begin (progn (goto-char contents-begin) (if 
(looking-at-p org-element-planning-line-re) (progn (forward-line))) (if 
(looking-at org-property-drawer-re) (progn (goto-char ...))) (max (if (< ... 
contents-end) (+ 2 contents-begin) 0) (point) (robust-end (and robust-begin 
(if (> (- contents-end 2) robust-begin) (progn (- contents-end 2)) (if 
robust-end nil (setq robust-begin nil)) (let ((headline (list 'headline (nconc 
(list :raw-value raw-value :begin begin :end end :pre-blank (if ... 0 ...) 
:contents-begin contents-begin :contents-end contents-end :robust-begin 
robust-begin :robust-end robust-end :level level :priority priority :tags tags 
:todo-keyword todo :todo-type todo-type :post-blank (if contents-end ... ...) 
:footnote-section-p footnote-section-p :archivedp archivedp :commentedp 
commentedp :post-affiliated begin) time-props standard-props 
(org-element-put-property headline :title (if raw-secondary-p raw-value 
(org-element--parse-objects (progn (goto-char title-start) (skip-chars-forward 
" \11") (point)) (progn (goto-char title-end) (skip-chars-backward " \11") 
(point)) nil (org-element-restriction 'headline) headline)
  (save-excursion (let* ((begin (point)) (true-level (prog1 (skip-chars-forward 
"*") (skip-chars-forward " \11"))) (level (org-reduced-level true-level)) (todo 
(and org-todo-regexp (let (case-fold-search) (looking-at (concat 
org-todo-regexp " "))) (progn (goto-char (match-end 0)) (skip-chars-forward " 
\11") (match-string 1 (todo-type (and todo (if (member todo 
org-done-keywords) 'done 'todo))) (priority (and (looking-at "\\[#.\\][ \11]*") 
(progn (goto-char (match-end 0)) (aref (match-string 0) 2 (commentedp (and 
(let ((case-fold-search nil)) (looking-at org-element-comment-string)) 
(goto-char (match-end 0 

Concrete suggestions to improve Org mode third-party integration :: an afterthought following Karl Voit's Orgdown proposal

[Ihor Radchenko]

Dear Fellow Orgers,

The recent spike of discussions following Karl's presentation in
Emacsconf 2021 revealed a lot of controversy among Org and Emacs
enthusiasts. Yet, Karl named a number of very real problems surrounding
Org mode usage outside Emacs.

>From the narrow perspective of this mailing list, I would like to list
some of the problems and possible solutions to them on our (Org dev)
side.

1. Org mode is almost impossible to separate from Emacs in its full strength
   - Yet, a number of people seems to be interested in using Org mode
 outside Emacs
 + Most notably, mobile users
 + A number of websites, like Github/Gitlab
   - The existing interest gave a rise to a number of third-party
 Org syntax parsers
 + None of the parsers support all the Org features, and even not
  all the grammar!
 + The parsers often do not even try to support all the features.
   They are merely looking at Org as a lightweight markup format.
   
2. Despite user interest, we lack a clear definition of Org grammar with
   examples and concrete guidelines for third-party parser developers

3. Many elements of the grammar are excessive for simple cases not
   involving document export, babel, and other powerful Org mode
   features

4. "Org mode" is an ambiguous word combination for search engines and
   people may not be able to find relevant information.
   - This one is not 100% true from my quick search. Try the following
 links:
 + https://duckduckgo.com/?q=org-mode=web
 + https://duckduckgo.com/?q=org+mode=web
 + https://duckduckgo.com/?q=org+mode+syntax=web
 + https://duckduckgo.com/?q=org+mode+markup=web
 The results are extremely relevant to Org, though orgmode.org
 search result looks slightly confusing (more below).
  
--
| My suggestions how we can address the above points |
--

1. Despite webengines delivering fairly good results for "Org mode"
   search term, I am a bit concerned about the first search hit, which
   is our flagship "https://orgmode.org; website.

   The website title is "Org mode for Emacs", repelling users who _do
   not want_ to use Org inside Emacs. Maybe we can do better? Something
   with less accent on Emacs like "Org mode: your life in plain text"

   The "abstract" in the search result is also not fully relevant:
   > Org and Org-mode have so many use cases that it is simply not
   > possible to easily document them, let alone show them all off on a
   > single page. As a result, Worg serves as a community wiki and
   > provides a place to document and share information about all aspects
   > of using and working with Org. For example, Worg contains:
   Again, we can make a simple change revealing the paragraph shown the
   at our front page:
   > Org is a highly flexible structured plain text file format,
   > composed of a few simple, yet versatile, structures — constructed
   > to be both simple enough for the novice and powerful enough for the
   > expert. 
   > 
   > Org mode is also a GNU Emacs major mode for keeping notes,
   > authoring documents, computational notebooks, literate programming,
   > maintaining to-do lists, planning projects, and more — in a fast
   > and effective plain text system.

2. Our front pages gives an impression that user must install Org
   I refer to the big image links "Features Install Quickstart Contribute"

   Maybe we can add "Try in browser" linking to our own instance of
   https://organice.200ok.ch/sample

3. We can provide a "source of truth" for Org syntax for third-party
   parser developers. Something easily reachable from the front page:
   "Org-Mode Logo Org Mode
   Features
   Releases
   ...
   --> Add Org support in third-party apps"

   The page should give a nice summary of existing third-party
   libraries, official _technical_ Org syntax, and tools for developers.

   3.1. In particular, I suggest to link
https://orgmode.org/worg/dev/org-syntax.html (it will be ready
eventually)
   
   3.2. Also, we may add a simplified Org syntax, as Karl suggested
(similar to Basic and Extended syntax in
https://www.markdownguide.org/, but more technical)
   
   3.3. I strongly suggest to add a community test set with example Org
files. The files should be a source of tests for Org parsers
with the true parsed representations in sexp format (possibly
also converted to json).

The example files can live in a separate repo for easy
contributions (possibly with Github/Gitlab mirrors is someone is
willing to maintain those).

The example files will be used by Org mode itself in our test
suite and will serve as a benchmark for external parser quality.

   3.4. Finally, we can have a separate page listing recommended
features for editors interacting with Org files. Something 

Re: Some commentary on the Org Syntax document

[Ihor Radchenko]

Nicolas Goaziou  writes:

>> Maybe we can just say "... lesser elements" that cannot contain other
>> elements."? Then, we mention that some elements cannot contain objects
>> in the description of those elements.
>
> But then, you do not remove the ambiguity that is condemned in this
> thread. The greater element/element and greater element/lesser element
> distinctions are equivalent, albeit not identical.

AFAIU, elements = greater-elements ∪ lesser-elements
The current syntax draft contains section "Greater elements" defining
all the greater-elements and section "Elements" defining lesser-elements
However, the word "elements" also refers to all possible elements in
some parts of the draft.
I propose to remove the ambiguity by referring to members of
org-element-greater-elements as "greater elements"; to
org-element-all-elements - org-element-greater-elements as "lesser
elements"; and to org-element-all-elements as just "elements".

> IIUC, you want three terms for elements (I am not even talking about
> secondary strings, which can hold objects that are not part of
> contents),

Yep.

> ... and probably two for objects: terminal and non-terminal.

Sorry, I do not understand what you refer to here.

Best,
Ihor

Re: [BUG] C-c C-* causes "org-element--cache: Unregistered buffer modifications detected."

[Ihor Radchenko]

Max Nikulin  writes:

>  >8 
> * A
> B
>  8< 
>
> C-c C-* while cursor is at the end of second line

Thanks! I was able to reproduce. And it is bad news.

>   Current command: (nil 26 30)
>   Chars modified: 26
>   Buffer modified: 30

This has the same footprint with
(let ((inhibit-modification-hooks t))
 (insert "* This is going to break the cache\n"))

So, I pushed a change that suppresses the warning completely for
Emacs <28 and silently resets the cache. I see no other way around it.

Fortunately, all the problems you reported in this thread do not happen
in newer Emacs versions.

Best,
Ihor

bug#44833: 27.1; Org org-docview-open (wrong-number-of-arguments (2 . 2) 1)

[Kyle Meyer]

Michael 1 writes:

> Debugger entered--Lisp error: (wrong-number-of-arguments (2 . 2) 1)
>   org-docview-open("file.odt::1")
>   org-link-open((link (:type "docview" :path "file.odt::1" :format bracket 
> :raw-link "docview:file.odt::1" :application nil :search-option nil :begin 
> 364 :end 453 :contents-begin 415 :contents-end 451 :post-blank 0 :parent 
> (paragraph (:begin 364 :end 454 :contents-begin 364 :contents-end 454 
> :post-blank 0 :post-affiliated 364 :parent nil nil)
>   org-open-at-point(nil)
>   funcall-interactively(org-open-at-point nil)
>   call-interactively(org-open-at-point nil nil)
[...]
> Can you reproduce the error?
> Is there any way to fix this behavior?

I'm sorry you didn't get a response at the time, but it seems unlikely
that you still have this issue because, as of Org's afd3b04ec (ol:
Extend open tooling in link parameters, 2020-02-19), org-link-open
handles the above error:

;; Function defined in `:follow' parameter may use a single
;; argument, as it was mandatory before Org 9.4.  This is
;; deprecated, but support it for now.
(condition-case nil
(funcall (org-link-get-parameter type :follow) path arg)
  (wrong-number-of-arguments
   (funcall (org-link-get-parameter type :follow) path)))

And that change made its way into the Emacs repo with f22856a5c54
(Update to Org 9.4.1, 2020-12-13), which was a part of the 27.2 release.

Can you confirm that the issue is resolved for you?

Re: [BUG] make test is extremely slow since b3cc2f793 [9.5.1 (9.5.1-g984367 @ /home/yantar92/.emacs.d/straight/build/org/)]

[Ihor Radchenko]

Nicolas Goaziou  writes:

>> I noticed that make test became extremely slow recently.
>> The first bottleneck can be seen in
>>
>> make BTEST_RE="^test-org-cite/adjust-note" test
>>
>> passed  1/1  test-org-cite/adjust-note (12.321278 sec)
>>
>> The test takes 12sec!
>
> Fixed. Thank you.

Sorry, but tests are still slow for me after the update.
If I run the test from inside Emacs with ert, things got better, but
make BTEST_RE="^test-org-cite/adjust-note" test
still takes 12sec.

Best,
Ihor

Re: [PATCH] Fix org-comment-line-break-function

[Kaushal Modi]

On Sat, Dec 4, 2021, 5:25 PM Tim Cross  wrote:

>
> Given that Nicholas cannot remember the reason for the original function
> and suspects it was meant to be an internal only function, I think this
> patch is probably the best way forward and should be applied.
>

Thanks. Nicolas asked me to add tests for this patch. But I need to look
into how to add tests for behavior of bindings. Need to add tests for M-j
binding behavior when cursor is within a comment or outside.

>

Re: [fyi] extensible syntax, syntax features, parsing risk

[John Kitchin]

Is it related to
https://lists.gnu.org/archive/html/emacs-orgmode/2010-08/msg00404.html

I implemented that idea for fun once:
https://kitchingroup.cheme.cmu.edu/blog/2015/02/05/Extending-the-org-mode-link-syntax-with-attributes/

John

---
Professor John Kitchin (he/him/his)
Doherty Hall A207F
Department of Chemical Engineering
Carnegie Mellon University
Pittsburgh, PA 15213
412-268-7803
@johnkitchin
http://kitchingroup.cheme.cmu.edu



On Sat, Dec 4, 2021 at 5:10 PM Samuel Wales  wrote:

> the quoted post below, which had message id
> 20524da70901041233g105f372fv175a47dc9884f...@mail.gmail.com , starts a
> thread relevant to the current discussion of syntax, at least
> historically, maybe practically.  could not find it online.
>
> there were succeeding threads with examples and other stuff for id
> markers and graph-theoretic things and other examples, where you as
> user could always use a cl-like syntax [i.e. something like
> "$[functionality-name arg :keyword arg]"].  my main concern was the
> proliferation of syntax, and the risks of that [e.g. parsing] and
> wanting the ability of factoring of syntax features.
>
> display, parsing and so on would be shared [factored] among all the
> different such features; org would always handle that the same.  thus
> as a user even, you could add some new feature in lisp, and write it
> in org using this syntax.  it would already work.
>
> i have more in notes.  idk if it is still relevant, but i have
> included the thread starter for the earliest thread [carsten/myself].
>
> On 1/4/09, Samuel Wales  wrote:
> > A general idea, which might or might not be useful.
> >
> > There are occasionally questions about syntax, like this:
> >
> >   Also, I'm afraid definition matching regexp won't play
> >   nicely with text indentation, ... -- Paul
> >
> > Or this:
> >
> >   What would be safer?  -- Carsten
> >
> > I like the footnote implementation, so this is for future
> > features, not necessarily footnotes.
> >
> > One issue when implementing new syntax (or changing existing
> > syntax or cleaning up code) is parsing risk, which I will
> > define as the risk that the syntax and the regexp or
> > matching code:
> >
> >   1) conflicts with user text
> >   2) conflicts with existing features
> >   3) will be hard to maintain
> >   4) constrains future features by making them conflict
> >  syntactically
> >   5) makes you run out of syntax to use in the future
> >   6) will require complicated regexps
> >   7) doesn't readily handle stuff you might want in the
> >  future, like being combined with another feature
> >   8) will be hard to quote, escape, comment, *boldify*, etc.
> >   9) doesn't handle nestability, print-readability,
> >  pretty-printability, syntax coloring, etc.
> >   10) will be inefficient when called in a loop
> >   11) isn't factored out
> >   12) etc.
> >
> > For example, one of the many reasons for using org-IDs (:))
> > in the conversation manager (as proposed) is that there are
> > already functions to parse org-IDs, so a new syntax is not
> > necessary and therefore parsing risk is minimized.
> >
> > In case parsing risk is a concern when adding a new feature
> > to org, here is one idea: have a generic syntax that is
> > extensible with keywords.
> >
> > The idea is to have a bracketing syntax with a reserved
> > keyword as the first element that says what you are doing.
> >
> > To use footnotes as an example (this is not a suggestion to
> > change footnote syntax, just an example that can be used for
> > future syntax):
> >
> > You might use something like "here is some text to be
> > footnoted $[fn apple]" to specify the footnote link, and
> > "$[fn-target apples are delicious]" to specify the target.
> >
> > The general point I want to make is that once such a syntax
> > is decided on, many future features are possible without
> > introducing parsing risk.
> >
> > For example, you can implement a new feature as
> > "$[my-new-feature ...]".  Then there is no parsing risk,
> > even though you have just added a new feature.
> >
> > For modifications of features, you can use keywords:
> > "$[my-new-feature ... :myparameter ...]".  These are easily
> > parsed by standard functions for parsing lists.
> >
> > You can develop ways to boldify, quote, nest, prettily
> > print, etc. this syntax, and those ways will work well with
> > all future features that use it.
> >
> > Of course, the dollar sign and brackets are not the only
> > possibility; it could be percent sign and parentheses, for
> > example.
> >
> > You will not be starting from scratch.  Lisp has already
> > worked out many of these issues.
> >
> > I will leave it to those who write massive amounts of org
> > code to decide whether an extensible syntax might be useful
> > to reduce parsing risk for future features.
> >
> > But I thought that I would propose the idea in case it is of
> > interest.
> >
> > --
> > For personal gain, myalgic 

Re: [PATCH] Fix org-comment-line-break-function

[Tim Cross]

Given that Nicholas cannot remember the reason for the original function
and suspects it was meant to be an internal only function, I think this
patch is probably the best way forward and should be applied.

Kaushal Modi  writes:

> On Tue, Nov 30, 2021 at 6:29 PM Tim Cross  wrote:
>
>  It would be good to get Nicholas' input here as I think he wrote the
>  original function back in 2012. 
>
> Just to see what happens, I tried this:
>
> M-: (setq-local comment-line-break-function #'comment-indent-new-line)
>
> .. and then M-j started working perfectly! It worked fine both: in Org 
> comment lines and out of comment lines.
>
> I see that comment-indent-new-line was added to emacs in newcomment.el back 
> in 2000. So I don't know why
> org-comment-line-break-function was added. May be Nicolas can comment more on 
> that.
>
> So would this patch work?
>
> =
>
> From 1a9187b82ed8d4e8d54ddd369a44d34295281fc3 Mon Sep 17 00:00:00 2001
> From: Kaushal Modi 
> Date: Tue, 30 Nov 2021 20:37:10 -0500
> Subject: [PATCH] org: Remove `org-comment-line-break-function'
>
> * lisp/org.el: Remove `org-comment-line-break-function' and let
> `comment-line-break-function' be the default value.
>
> This fixes the `M-j' binding issue reported by Richard Lawrence in
> .
> ---
>  lisp/org.el | 17 ++---
>  1 file changed, 2 insertions(+), 15 deletions(-)
>
> diff --git a/lisp/org.el b/lisp/org.el
> index ec59ddf44..ee8ca1f03 100644
> --- a/lisp/org.el
> +++ b/lisp/org.el
> @@ -19624,8 +19624,7 @@ assumed to be significant there."
>  
>  ;; `org-auto-fill-function' takes care of auto-filling.  It calls
>  ;; `do-auto-fill' only on valid areas with `fill-prefix' shadowed with
> -;; `org-adaptive-fill-function' value.  Internally,
> -;; `org-comment-line-break-function' breaks the line.
> +;; `org-adaptive-fill-function' value.  
>  
>  ;; `org-setup-filling' installs filling and auto-filling related
>  ;; variables during `org-mode' initialization.
> @@ -19647,8 +19646,7 @@ assumed to be significant there."
>(setq-local fill-paragraph-function 'org-fill-paragraph)
>(setq-local auto-fill-inhibit-regexp nil)
>(setq-local adaptive-fill-function 'org-adaptive-fill-function)
> -  (setq-local normal-auto-fill-function 'org-auto-fill-function)
> -  (setq-local comment-line-break-function 'org-comment-line-break-function))
> +  (setq-local normal-auto-fill-function 'org-auto-fill-function))
>  
>  (defun org-fill-line-break-nobreak-p ()
>"Non-nil when a new line at point would create an Org line break."
> @@ -19903,17 +19901,6 @@ filling the current element."
>   (adaptive-fill-mode (not (equal fill-prefix ""
>   (when fill-prefix (do-auto-fill))
>  
> -(defun org-comment-line-break-function ( soft)
> -  "Break line at point and indent, continuing comment if within one.
> -The inserted newline is marked hard if variable
> -`use-hard-newlines' is true, unless optional argument SOFT is
> -non-nil."
> -  (if soft (insert-and-inherit ?\n) (newline 1))
> -  (save-excursion (forward-char -1) (delete-horizontal-space))
> -  (delete-horizontal-space)
> -  (indent-to-left-margin)
> -  (insert-before-markers-and-inherit fill-prefix))
> -
>  
>  ;;; Fixed Width Areas

Re: Removing obsolete function `org-truely-invisible-p'.

[Samuel Wales]

idk and will go along with whatever developers decide, but your note
was a good reminder that visible mode can be checked.

also made me wonder if it is possible to move anything from org-macs
out to topic-specific files like org-visibility.el or so?  no?

[idk what org visibility functions do what.  although i suspect org
can do it, i still use htmlize-buffer-substring-no-invisible.]


On 12/4/21, Karl Fogel  wrote:
> The function `org-truely-invisible-p' is defined in
> 'lisp/org-macs.el', but it is not used anywhere anymore in Org
> Mode, nor is it used anywhere in GNU Emacs (I checked on both
> 'emacs-28' branch and 'master' branch).
>
> The last (and possibly only?) call to that function was removed
> from `org-beginning-of-line' in this commit:
>
>   commit 3baf246f4f73005a4eacd7c368f7222f95d50243
>   Author: Nicolas Goaziou 
>   AuthorDate: Thu Apr 28 23:28:15 2016 +0200
>   Commit: Nicolas Goaziou 
>   CommitDate: Thu Apr 28 23:28:15 2016 +0200
>
>   Handle correctly `shift-select-mode'
>
>   * lisp/org.el (org-beginning-of-line): Handle correctly
>   `shift-select-mode'.
>
>   Reported-by: Mathieu Marques 
>   
>
> Then, a few months later in commit 87116700e6e, Nicholas moved the
> function to 'org-macs.el', where it has been sitting unused ever
> since.
>
> Should we just remove `org-truely-invisible-p'?  Or at least
> correct the spelling of its name ("truely" should be "truly")?
>
> It seems fairly unlikely to me that people are using it in their
> own code, although of course I cannot be sure of that.  Partly, I
> believe that because surely someone else would have noticed the
> misspelling by now, if enough people were using it :-).
>
> Best regards,
> -Karl
>
>


-- 
The Kafka Pandemic

A blog about science, health, human rights, and misopathy:
https://thekafkapandemic.blogspot.com

[fyi] extensible syntax, syntax features, parsing risk

[Samuel Wales]

the quoted post below, which had message id
20524da70901041233g105f372fv175a47dc9884f...@mail.gmail.com , starts a
thread relevant to the current discussion of syntax, at least
historically, maybe practically.  could not find it online.

there were succeeding threads with examples and other stuff for id
markers and graph-theoretic things and other examples, where you as
user could always use a cl-like syntax [i.e. something like
"$[functionality-name arg :keyword arg]"].  my main concern was the
proliferation of syntax, and the risks of that [e.g. parsing] and
wanting the ability of factoring of syntax features.

display, parsing and so on would be shared [factored] among all the
different such features; org would always handle that the same.  thus
as a user even, you could add some new feature in lisp, and write it
in org using this syntax.  it would already work.

i have more in notes.  idk if it is still relevant, but i have
included the thread starter for the earliest thread [carsten/myself].

On 1/4/09, Samuel Wales  wrote:
> A general idea, which might or might not be useful.
>
> There are occasionally questions about syntax, like this:
>
>   Also, I'm afraid definition matching regexp won't play
>   nicely with text indentation, ... -- Paul
>
> Or this:
>
>   What would be safer?  -- Carsten
>
> I like the footnote implementation, so this is for future
> features, not necessarily footnotes.
>
> One issue when implementing new syntax (or changing existing
> syntax or cleaning up code) is parsing risk, which I will
> define as the risk that the syntax and the regexp or
> matching code:
>
>   1) conflicts with user text
>   2) conflicts with existing features
>   3) will be hard to maintain
>   4) constrains future features by making them conflict
>  syntactically
>   5) makes you run out of syntax to use in the future
>   6) will require complicated regexps
>   7) doesn't readily handle stuff you might want in the
>  future, like being combined with another feature
>   8) will be hard to quote, escape, comment, *boldify*, etc.
>   9) doesn't handle nestability, print-readability,
>  pretty-printability, syntax coloring, etc.
>   10) will be inefficient when called in a loop
>   11) isn't factored out
>   12) etc.
>
> For example, one of the many reasons for using org-IDs (:))
> in the conversation manager (as proposed) is that there are
> already functions to parse org-IDs, so a new syntax is not
> necessary and therefore parsing risk is minimized.
>
> In case parsing risk is a concern when adding a new feature
> to org, here is one idea: have a generic syntax that is
> extensible with keywords.
>
> The idea is to have a bracketing syntax with a reserved
> keyword as the first element that says what you are doing.
>
> To use footnotes as an example (this is not a suggestion to
> change footnote syntax, just an example that can be used for
> future syntax):
>
> You might use something like "here is some text to be
> footnoted $[fn apple]" to specify the footnote link, and
> "$[fn-target apples are delicious]" to specify the target.
>
> The general point I want to make is that once such a syntax
> is decided on, many future features are possible without
> introducing parsing risk.
>
> For example, you can implement a new feature as
> "$[my-new-feature ...]".  Then there is no parsing risk,
> even though you have just added a new feature.
>
> For modifications of features, you can use keywords:
> "$[my-new-feature ... :myparameter ...]".  These are easily
> parsed by standard functions for parsing lists.
>
> You can develop ways to boldify, quote, nest, prettily
> print, etc. this syntax, and those ways will work well with
> all future features that use it.
>
> Of course, the dollar sign and brackets are not the only
> possibility; it could be percent sign and parentheses, for
> example.
>
> You will not be starting from scratch.  Lisp has already
> worked out many of these issues.
>
> I will leave it to those who write massive amounts of org
> code to decide whether an extensible syntax might be useful
> to reduce parsing risk for future features.
>
> But I thought that I would propose the idea in case it is of
> interest.
>
> --
> For personal gain, myalgic encephalomyelitis denialists are knowingly
> causing further suffering and death by grossly corrupting science.  Do
> you care about the world?
> http://www.meactionuk.org.uk/What_Is_ME_What_Is_CFS.htm
>


-- 
The Kafka Pandemic

A blog about science, health, human rights, and misopathy:
https://thekafkapandemic.blogspot.com

Re: [PATCH] org-src.el: add option `org-src-native-defun-movements'

[Tim Cross]

Sébastien Miquel  writes:

> Hi,
>
> Thank you for having a look.
>
> Tim Cross writes:
>> This also seems like an edge case and I'm not convinced yet another
>> option is justified. Why have eilisp in org blocks rather than an
>> emacs-lisp block?
>
> By org src blocks I meant an org emacs-lisp src block. The point of
> the patch is to be able to eval-defun some lisp code in an emacs-lisp
> src block from the org-buffer.
>

OK, that makes it clearer. However, I'm not convinced this is something
we need or want. You can evaluate emacs lisp blocks and you can use
edit-special buffers to evaluate individual lines in a source block.
Being able to execute arbitrary lisp forms at a top level inside an org
buffer could be considered dangerous. I don't think it should be enabled
by default. 

>> As this is a breaking change, it should not be on by default.
> Currently eval-defun errors out, and fixing that will break things
> sooner or later, I think.
>
> I do not mind updating the patch to set the new option to nil by
> default, although I'll wait for a second opinion on this.
>

That is fine. However, note that this would mean your patch can only be
applied to the next version (development version) of org and not to the
current maintenance branch because you cannot add a breaking change to
an already released version.

I think you are making it harder to get the patch applied by enabling
the option. There is a (rightly) conservative stance on breaking
changes. Adding a new option which is enabled by default and which
breaks existing functionality has almost no chance of being applied.
Adding a new option which is a breaking change that needs to be enabled
by the user is far more likely to be considered.

Re: Org Mode "Contribute", "Maintenance" pages don't give repository.

[Karl Fogel]

On 04 Dec 2021, Daniel Fleischer wrote:
The "Contribute" page at 
https://orgmode.org/worg/org-contribute.html does not mention
git://git.sv.gnu.org/emacs/org-mode.git .  Actually, now that I 
look around, the "Maintenance" page at
https://orgmode.org/worg/org-maintenance.html doesn't mention 
the repository address either.  It does mention the word

"repository" a few times, but never actually links to it.


I've added a link to the org-maintenance page but there was a 
link in
the org-contribute, in the "Your first commit as an Org 
maintainer"
section. The git.sv.gnu.org is a shorthand for 
git.savannah.gnu.org.


Thanks for reporting this,


Thanks, Daniel!

You're right about repository link on the Contribute page.  It 
might still be good to mention it earlier, like in the "Ways that 
involve programming" section or even just in the first paragraph 
of the page.


The reason I suggest this is that often when someone is hot on the 
trail of debugging something, they haven't yet decided whether 
they want to become a contributor or not.  They just know they 
want to ook at the latest sources so they can first figure out 
more about whatever ${technical_thing} they encountered.  So 
making the latest sources very easy to find is a way of making it 
more likely that someone becomes a contributor, by removing a 
commonly-encountered barrier.


Best regards,
-Karl

Re: Org-syntax: Intra-word markup

[Tom Gillespie]

> Since org is a valid export backend though, perhaps this behaviour should be
> reserved for @@:…@@, i.e. no export backend, which I think semantically fits
> fairly nicely.

This ends up being even more convenient than I initially realized.
The current spec for export snippets is ambiguous when it says
"NAME can contain any alpha-numeric character and hyphens"
but the implementation behavior requires that "any" means "at
least one" and is implemented using the + regex operator.

What this means is that @@:...@@ syntax is not actually used
in Org at all at the moment and renders as plain text. I agree that
we need to avoid @@org:..@@ because it has legitimate uses.
Making a back-end of empty string valid for parse separately
syntax thus makes @@ syntax more regular overall, and allows
@@:...@@ to be processed separately because it currently
never enters the export snippet processing.

This is important because export snippets do not seem to be easily
accessible to earlier phases of the org-export machinery, i.e. there
isn't a nice centralized place to preprocess @@org:...@@ even
if we wanted to. On the other hand @@:...@@ isn't processed
at all. I could be missing something in the org export code though.

It will take a bit of work to get this behavior implemented I think,
but it doesn't seem to have any conflicts. Some users may have
set the empty backend to expand manually via
org-export-snippet-translation-alist, but as long as we give
org-export-snippet-translation-alist priority and warn people
that setting "" manually will disable the new functionality
then there shouldn't be any disruption. The behavior also sort
of matches what we would want the empty string to be in this
case, which is "all backends" and of course the only markup
that makes sense for "all backends" is org itself!

Best,
Tom

Re: quotation marks in table cell vs. org-babel-ref-resolve

[Tim Cross]

Greg Minshall  writes:

> hi, Tim,
>
>> The key question is what is the use case for having this 'mixed' content
>> in a table cell?
>
> in my case, i am putting RFC822('ish) e-mail addresses in a column of an
> org-mode table.  and, i want to extract them.
> 
> | oxymo...@example.com  |
> | Greg Oxymoron   |
> | "Greg G. Oxymoron"  |
> 
> for the third row returns =Greg G. Oxymoron=, rather than my desired
> ="Greg G. Oxymoron" =.
>
> by the way, do you know the use case for the current behavior for
> strings that start with a ="=?  i couldn't find anything in the manual.
>
> i wonder if maybe the existing parameter =inhibit-lisp-eval= (which, in
> the path i am exercising, is non-nil) could also be used to not do the
> check for a ="=?  (maybe that's also a hack, but i think it would solve
> my problem. :)
>

I don't know. It could be related to the spreadsheet capabilities or it
could simply be an oversight in how the code extracts values from
tables. 

I tend to use the function org-table-to-list to extract the data from a
table. It gives me a nested list which I can then process with elisp in
any way I want. I don't know if that would help or how it will interpret
a cell whic contains both quoted and unquoted data.

Re: On zero width spaces and Org syntax

[Tim Cross]

Max Nikulin  writes:

> On 04/12/2021 04:48, Tim Cross wrote:
>> My vote is to simply maintain the status quo. Don't modify the syntax,
>> don't make the zero space character somewhat special or processed in any
>> special way during export. In short, accept that inner word markup has
>> only limited support and if that is a requirement which is critical to
>> your use case, accept that org mode may not be the right solution for
>> your requirements.
>
> Tim, you are skeptical concerning usage of Org markup outside of Emacs. Though
> some subscribers of this list support such idea with hope for collaboration 
> with
> colleagues and for other reasons. Status quo in respect to similar questions
> increases risk that other tools will adapt different workarounds and
> incompatible dialects will appear.

This is a misrepresentation of my position. I've never stated I'm
sceptical or org markup outside of Emacs. I'm sceptical of org mode
outside of Emacs, but have never expressed an opinion of org markup
outside of Emacs.  

However, I will now

Org markup outside Emacs is very much a secondary concern that would be
a nice to have for some workflows, but should be achieved with zero
impact on Emacs users. Org mode and the markup it uses is primarily an
Emacs mode. In fact, making it easier for non-Emacs users to use org
mode is almost certainly working against the FSF philosophy. I'm pretty
certain RMS would be very unhappy of any efforts to allow users to use
org mode in products like MS Visual Code. While it is fine for 3rd party
systems to try and mimic org mode, it is totally contrary to GNU
philosophy for a GNU project to actively support or enable such
functionality in non-free solutions. Any decisions to make changes to
org mode must be primarily for the benefit of Emacs users. When such
decisions also have benefit for non-Emacs users, that is great, but it
should not be a driving factor in making decisions regarding change or
extensions to org mode.

>
> From the point of view of popularizing Org it is better to make some decision:
> either zero-width space should become a part of syntax or some other printable
> marker should be chosen to suppress effect of Org markup or vice versa to
> activate some construct.

Chasing popularity is always a mistake and should never be used as an
argument for change. We are also talking about something where there is
little evidence of demand. We have a single post from someone asking how
to support inner word emphasis and suddenly, threads about modifying
syntax, modifying back ends and a dozen proposals on how to support this
'feature'.

A question I would ask is that if extending and adding broader support
for emphasis is so straight-forward, why do we already have so many
issues reported about incorrect application of markup? We have not been
successful in eliminating existing ambiguities with the markup and yet
some would have us charge off and add even more complexity.

Rather than extending markup syntax, lets focus on fixing the real issues we
already have. There have been far more posts to this list about that
than about inner word emphasis. For example, the many posts about markup
and links. 

With respect to the status of zero width space, I'm not convinced we
need to do anything. Would it be classified as a kludge, probably. Does
it provide an escape hatch for some situations, yes. Does that mean it
needs to be formally recognised and added to the syntax, no. Does the
existence of this kludge make implementation of org mode markup for
other tools more difficult or less clear, probably. Should that be a
primary concern for Emacs org-mode, no. Should it be something we
consider when making decisions, sure, but only as a secondary
consideration. 

What the need for the zero width space kludge really means is that in
some situations, we have some ambiguity in the existing syntax. Can we
fix those ambiguities? I don't know - so far, I've not seen a proposal
which doesn't introduce as many problems as it solves, (though Tomp's @@
proposal looks interesting, but lots more analysis is required).

The zero width kludge is certainly a symptom of limitations in the
existing syntax definition. However, I don't think it is the cure and I
don't agree it needs to be formally recognised as part of the syntax -
it is not the cure. If we can find the correct cure, the zero width
kludge will not be necessary (or will only be necessary in extreme and
rare edge cases). 

Re: Org Mode "Contribute", "Maintenance" pages don't give repository.

[Daniel Fleischer]

Karl Fogel [2021-12-04 Sat 14:59] wrote:

> The "Contribute" page at https://orgmode.org/worg/org-contribute.html does 
> not mention
> git://git.sv.gnu.org/emacs/org-mode.git .  Actually, now that I look around, 
> the "Maintenance" page at
> https://orgmode.org/worg/org-maintenance.html doesn't mention the repository 
> address either.  It does mention the word
> "repository" a few times, but never actually links to it.

I've added a link to the org-maintenance page but there was a link in
the org-contribute, in the "Your first commit as an Org maintainer"
section. The git.sv.gnu.org is a shorthand for git.savannah.gnu.org.

Thanks for reporting this,

-- 

Daniel Fleischer

Re: Org-syntax: Intra-word markup

[Juan Manuel Macías]

Hi John,

John Kitchin writes:

> Along these lines (and combining the s-exp suggestion from Max) , you
> can achieve something like this with links. 

I like this idea of merging the Maxim's proposal with the power of links.

In any case, this and other workarounds provided here make it clear that
in Org we do not lack of good and useful resources. I usually use macros
(taking advantage of the fact that macros expand soon). For example
(only in this case with the LaTeX backend):

#+MACRO: emph (eval (when (org-export-derived-backend-p 
org-export-current-backend 'latex) (concat "@@latex:\\emph{@@" $1 
"@@latex:}@@")))

Defined the macro this way, it allows me also to introduce nested
emphases by both ways:

#+begin_src example
{{{emph(lorem *ipsum* /dolor/ {{{emph(sit)}}} amet)}}}
#+end_src

==> \emph{lorem \textbf{ipsum} \emph{dolor} \emph{sit} amet}

Best regards,

Juan Manuel

Org Mode "Contribute", "Maintenance" pages don't give repository.

[Karl Fogel]

The "Contribute" page at 
https://orgmode.org/worg/org-contribute.html does not mention 
git://git.sv.gnu.org/emacs/org-mode.git .  Actually, now that I 
look around, the "Maintenance" page at 
https://orgmode.org/worg/org-maintenance.html doesn't mention the 
repository address either.  It does mention the word "repository" 
a few times, but never actually links to it.


These oversights would be pretty easy to fix (if there is 
agreement that they shoud be fixed), so I haven't attached a patch 
here.  It would be quick for someone who already maintains those 
pages to make the necessary changes.


The only place I see the repository is at the top of the home page 
of
https://orgmode.org/.  That's nice and prominent :-), but 
developers would probably expect to find the address on those 
other two pages as well.


Best regards,
-Karl

Removing obsolete function `org-truely-invisible-p'.

[Karl Fogel]

The function `org-truely-invisible-p' is defined in 
'lisp/org-macs.el', but it is not used anywhere anymore in Org 
Mode, nor is it used anywhere in GNU Emacs (I checked on both 
'emacs-28' branch and 'master' branch).


The last (and possibly only?) call to that function was removed 
from `org-beginning-of-line' in this commit:


 commit 3baf246f4f73005a4eacd7c368f7222f95d50243
 Author: Nicolas Goaziou 
 AuthorDate: Thu Apr 28 23:28:15 2016 +0200
 Commit: Nicolas Goaziou 
 CommitDate: Thu Apr 28 23:28:15 2016 +0200
 
 Handle correctly `shift-select-mode'
 
 * lisp/org.el (org-beginning-of-line): Handle correctly 
 `shift-select-mode'.
 
 Reported-by: Mathieu Marques 

 

Then, a few months later in commit 87116700e6e, Nicholas moved the 
function to 'org-macs.el', where it has been sitting unused ever 
since.


Should we just remove `org-truely-invisible-p'?  Or at least 
correct the spelling of its name ("truely" should be "truly")?


It seems fairly unlikely to me that people are using it in their 
own code, although of course I cannot be sure of that.  Partly, I 
believe that because surely someone else would have noticed the 
misspelling by now, if enough people were using it :-).


Best regards,
-Karl

Re: Org-syntax: Intra-word markup

[Timothy]

Hi Tom,

> After a bunch of rambling (see below if interested), I think I have
> a solution that should work for everyone. The key realization is that
> what we really want is the ability to have a “parse me separately”
> type of syntax. This meets the intra-word syntax needs and might
> meet some other needs as well.
>
> The solution is to make  “parse me separately”
> block! It nearly works that way already too! To minimize typing
> we could have @@:…@@ the empty type default to org.
>
> Thoughts?

This isn’t quite as succinct as the ascii-doc inspired suggestions, but it’s
barely an extension on the current syntax — I like it!

Since org is a valid export backend though, perhaps this behaviour should be
reserved for @@:…@@, i.e. no export backend, which I think semantically fits
fairly nicely.

All the best,
Timothy

Re: Org-syntax: Intra-word markup

[John Kitchin]

Along these lines (and combining the s-exp suggestion from Max) , you can
achieve something like this with links.

This is lightly tested, and I am not thrilled with the eval for exporting,
but I couldn't get a macro to work on the export function to avoid it, and
this is just a proof of concept idea. This might only be suitable for
individual solutions, since you have to define this markup yourself.

#+BEGIN_SRC emacs-lisp :results silent
(defun italic (s)
  (pcase backend ;; lexical
('latex (format "{\\textit{%s}}" s))
('html (format "%s" s))
(_ s)))

(defun @@-export (path desc backend)
  (eval `(concat ,@(read path

(org-link-set-parameters
 "@@"
 :export #'@@-export)
#+END_SRC

In org, it would look like Here is a [[@@:((italic "part") "ial")]] markup.
And in exports this is what this implementation does.

#+BEGIN_SRC emacs-lisp
(org-export-string-as "Here is a [[@@:((italic \"part\") \"ial\")]]
markup." 'latex t)
#+END_SRC

#+RESULTS:
: Here is a {\textit{part}}ial markup.


#+BEGIN_SRC emacs-lisp
(org-export-string-as "Here is a [[@@:((italic \"part\") \"ial\")]]
markup." 'html t)
#+END_SRC

#+RESULTS:
: 
: Here is a partial markup.

#+BEGIN_SRC emacs-lisp
(org-export-string-as "Here is a [[@@:((italic \"part\") \"ial\")]]
markup." 'ascii t)
#+END_SRC

#+RESULTS:
: Here is a partial markup.

Of course, you are free to do what you want with the path, including parse
it yourself to generate the output, and since it is a link, you could do
all kinds of things to make it look the way you want with faces, overlays,
etc.



John

---
Professor John Kitchin (he/him/his)
Doherty Hall A207F
Department of Chemical Engineering
Carnegie Mellon University
Pittsburgh, PA 15213
412-268-7803
@johnkitchin
http://kitchingroup.cheme.cmu.edu



On Sat, Dec 4, 2021 at 12:54 PM Tom Gillespie  wrote:

> Hi all,
> After a bunch of rambling (see below if interested), I think I have
> a solution that should work for everyone. The key realization is that
> what we really want is the ability to have a "parse me separately"
> type of syntax. This meets the intra-word syntax needs and might
> meet some other needs as well.
>
> The solution is to make @@org:...@@ "parse me separately"
> block! It nearly works that way already too! To minimize typing
> we could have @@:...@@ the empty type default to org.
>
> This seems like a winner to me. The syntax for it already exists
> and won't conflict. It requires relatively minimal additional typing
> the implication is clear, and there are other places where such
> behavior could be useful.
>
> This syntax seems like a winner to me
> @@org:/hello/@@world
> @@:/hello/@@world
>
> You can also do things like
> #+begin_src org
> I want a number in this number@@org:src_elisp{(+ 1 2)}@@word!
> #+end_src
>
> Which would render to
> #+begin_src org
> I want a number in this number3word!
> #+end_src
>
> Thoughts?
>
> Best!
> Tom
>
> --- rambling below -
>
>
> > This idea reminds me a bit of Scribble/Racket where every document is
> > just inverted code, which makes it possible to insert arbitrary Racket
> > code in your prose...
>
> I will say, despite some of my comments elsewhere, that I think
> exploring certain features of Scribble syntax for use in Org mode
> would simplify certain parts of the syntax immensely.
>
> For example
> various inline blocks are an absolute pain to parse because they
> allow nested delimiters /if they are matched/. The implementation
> of the /if they are matched/ clause is currently a nasty hack which
> generates a regular expression that can only actually handle nesting
> to depth 3. Actually implementing the recursive grammar add a lot
> of complexity to the syntax and is hard to get right.
>
> It would be vastly simpler to use Scribble's |<{hello }} world}>|
> style syntax and always terminate at the first matching delimiter.
> I'm sure that this would break some Org files, but it would make
> dealing with latex fragments and inline source blocks and inline
> footnotes SO much simpler. Matching an arbitrary number of
> angle brackets does add some complexity, but it is tiny compared
> to the complexity of enforcing matched parens and their failure cases
> especially because many of the places where nesting is required
> probably only see use of the nesting feature in a tiny fraction of
> all cases.
>
> One other reason why this is attractive is that all the instances
> where nested delimiters can appear on a line are preceded by
> some non-whitespace character. This means that using the
> pipe syntax does not conflict with table syntax!
>
> Now the question comes. If we could implement this for
> delimiters, could we also implement something similar
> for markup? The issue with the proposed markup outside
> delimiter inside approach is that it will change existing
> behavior for files that want the delimiters to be included
> in the markup, i.e. /{oops}/ becoming /oops/ is bad. A
> 

Re: Org-syntax: Intra-word markup

[Tom Gillespie]

Hi all,
After a bunch of rambling (see below if interested), I think I have
a solution that should work for everyone. The key realization is that
what we really want is the ability to have a "parse me separately"
type of syntax. This meets the intra-word syntax needs and might
meet some other needs as well.

The solution is to make @@org:...@@ "parse me separately"
block! It nearly works that way already too! To minimize typing
we could have @@:...@@ the empty type default to org.

This seems like a winner to me. The syntax for it already exists
and won't conflict. It requires relatively minimal additional typing
the implication is clear, and there are other places where such
behavior could be useful.

This syntax seems like a winner to me
@@org:/hello/@@world
@@:/hello/@@world

You can also do things like
#+begin_src org
I want a number in this number@@org:src_elisp{(+ 1 2)}@@word!
#+end_src

Which would render to
#+begin_src org
I want a number in this number3word!
#+end_src

Thoughts?

Best!
Tom

--- rambling below -


> This idea reminds me a bit of Scribble/Racket where every document is
> just inverted code, which makes it possible to insert arbitrary Racket
> code in your prose...

I will say, despite some of my comments elsewhere, that I think
exploring certain features of Scribble syntax for use in Org mode
would simplify certain parts of the syntax immensely.

For example
various inline blocks are an absolute pain to parse because they
allow nested delimiters /if they are matched/. The implementation
of the /if they are matched/ clause is currently a nasty hack which
generates a regular expression that can only actually handle nesting
to depth 3. Actually implementing the recursive grammar add a lot
of complexity to the syntax and is hard to get right.

It would be vastly simpler to use Scribble's |<{hello }} world}>|
style syntax and always terminate at the first matching delimiter.
I'm sure that this would break some Org files, but it would make
dealing with latex fragments and inline source blocks and inline
footnotes SO much simpler. Matching an arbitrary number of
angle brackets does add some complexity, but it is tiny compared
to the complexity of enforcing matched parens and their failure cases
especially because many of the places where nesting is required
probably only see use of the nesting feature in a tiny fraction of
all cases.

One other reason why this is attractive is that all the instances
where nested delimiters can appear on a line are preceded by
some non-whitespace character. This means that using the
pipe syntax does not conflict with table syntax!

Now the question comes. If we could implement this for
delimiters, could we also implement something similar
for markup? The issue with the proposed markup outside
delimiter inside approach is that it will change existing
behavior for files that want the delimiters to be included
in the markup, i.e. /{oops}/ becoming /oops/ is bad. A
second issue is that putting the delimiter inside the markup
cannot work for verbatim and code ={oops}= is ={oops}= no
matter what. Therefore the solution is not uniform across all
types of markup. We need another solution that works for
all types of markup.

What if we put the "start arbitrary markup" char outside
the markup? Say something like |/ital/|icks? Or what if
we went whole hog and used |{/ital/}|ics and made the
|{...}| syntax trigger a generalized feature where the
contents of the |{...}| block are parsed by themselves
and can abutt any other text? This would be generally
useful in a variety of situations beyond just intra-word
markup.

What are the issues with this approach? The first issue
is that there is a conflict with table syntax if we were to
use the pipe character because markup can appear at
the start of a line. The second issue is that it might be
confusing for users if |{}| also worked like {} when in the
context of latex elements or inline src blocks, or maybe
that is ok because |{}| never renders as text. Hrm. Ok.
Second issue resolved, but what to do about the first?

If we want generalized "parse this by itself" syntax so
that we can write hello|{/world/}|ok, then we need a
solution that can appear at the start of a line. So we
can't use pipe because that is always a table line even
if a zero width space is put before it ;). What other
options do we have? How about #+|{/hello/}|world for
the start of a line? As long as there is no trailing colon
it isn't a keyword, so it could work ... except that if
someone reflows the text and it is no longer a the
start of a line then the syntax breaks. That is to say
using #+| at the start of a line is not uniform, so we
can't take that approach.

What other chars to we have at our disposal? Hrm.
How about @@? Could we use that? What happens
if we use @@org:/hello/@@world? Or maybe if we
want to minimize the number of chars we could do
@@:/hello/@@world and have the empty prefix in
@@ blocks mean org?

Re: On zero width spaces and Org syntax

[Marcin Borkowski]

On 2021-12-04, at 08:22, Ihor Radchenko  wrote:

> Marcin Borkowski  writes:
>> 2. We modify Emacs itself to somehow highlight the ZWS.  There is (kind
>> of) a precedent – a no-breaking space is already fontified with
>> =nobreak-space= face.  At the very least, make whitespace-mode somehow
>> show ZWSs (which it doesn't now, and I'd probably say it's a bug).
>>
>> I know that my point 2. is a bit controversial, since it could lead to
>> alignment issues where a ZWS is displayed as something with a positive
>> width. OTOH, even now changing the face of a ZWS leads to a narrow
>> (1-pixel wide) line of a different color.  Is there a way to make it
>> a bit stronger?
>
> We can try to create an accent. Try the following:
> 1. Open new empty org buffer
> 2. Disable font-lock-mode
> 3. M-: (insert (compose-string "a​" nil nil (list ?a '(bl . tl) ?␣)))
>
> The result will look like on the attached image.

I'm not sure if I like that idea - looks great, but I'd be a bit afraid
of unintended consequences.

Either way, personally I can live with ZWSs in my Org files, so whatever
is decided, it's fine with me.

Best,

-- 
Marcin Borkowski
http://mbork.pl

Re: [BUG] make test is extremely slow since b3cc2f793 [9.5.1 (9.5.1-g984367 @ /home/yantar92/.emacs.d/straight/build/org/)]

[Nicolas Goaziou]

Hello,

Ihor Radchenko  writes:

> I noticed that make test became extremely slow recently.
> The first bottleneck can be seen in
>
> make BTEST_RE="^test-org-cite/adjust-note" test
>
> passed  1/1  test-org-cite/adjust-note (12.321278 sec)
>
> The test takes 12sec!

Fixed. Thank you.

Regards,
-- 
Nicolas Goaziou

Re: Org-syntax: emphasis and not English punctuation

[Juan Manuel Macías]

Max Nikulin writes:

> Maybe this issue should be considered independently of itra-word emphasis.

Yes I agree. Apologies for mixing up this topic in the discussion about
intra-word emphasis...

> Second and third examples looks like they should be supported. Ihor
> mentioned treating punctuation in a more general way. It requires rich 
> test set to estimate changes in heuristics. I suspect some problems
> since start and end patterns are not symmetric and I have not found a 
> way to specify in regexp only punctuation marks that normally appears
> in front of words. Square brackets likely should be excluded somehow
> as well since they are part of Org syntax. I am unsure if it is
> possible to use just regexp without additional checks of candidates.

Ihor's idea seems interesting to me, although I understand the possible
problems you mention. By the way, I'm afraid of initial inverted
punctuation (¡¿) are only used in Castilian Spanish and other languages
of Spain, such as Galician or Asturian, due to the Castilian influence
(we go backwards from the rest of the world ;-):
https://en.wikipedia.org/wiki/Inverted_question_and_exclamation_marks

> Ihor Radchenko. [PATCH] Re: c47b535bb origin/main org-element: Remove
> dependency on ‘org-emphasis-regexp-components’
> Sun, 21 Nov 2021 17:28:57 +0800.
> https://list.orgmode.org/87v90lzwkm.fsf@localhost

I see. I believe it's a sensible decision to get rid of the dependency
on org-emphasis-regexp-components. I understand that now everything
related to the structure of emphases is the competence of org-element?

Best regards,

Juan Manuel 

Re: Org-syntax: Intra-word markup

[Denis Maier]

Am 03.12.2021 um 15:24 schrieb Max Nikulin:

On 03/12/2021 02:03, Nicolas Goaziou wrote:

Denis Maier writes:


As for suggestions: If just using /intra/word creates ambiguities, what
about the asciidoc solution? So //intra//word?


I sympathize to the idea of intra-word emphasis, but the syntax above is
going to cause some ambiguous situations.


I suppose, some more general solution is required.


I do think the marker + zero-width space is one way to go. We could, as
an improvement, consider zero-width spaces around emphasis markers to be
part of the markup, and replace them along during export.


Zero-space characters adjacent to emphasis markers is a better idea than 
replacing any zero space. However I agree with Juan Manuel that white 
space characters, especially completely invisible (I am not Eli who sees 
such special characters by moving cursor through them) should not be 
overloaded. From my point of view, it is acceptable to use zero width 
spaces as a workaround but they should not become official part of Org 
syntax.



Another solution is to introduce a less-subtle, but less prone to
ambiguity, syntax, e.g.,

   /{bold}/markup   or   /|bold|/markup

where /{ }/  or /|  |/ become "extended" markers.


More explicit markup leaves less room for ambiguities, and I like the 
idea due to this reason. On the other hand it diverges from principle of 
lightweight markup. The almost only special character in TeX is "\", 
HTML has three ones "&<>" with simple escape rules. Org uses many 
special characters to avoid verbosity and requires some tricks to escape 
them. Markers like "\{" make Org more verbose but do not make it more 
strict, a lot of things still rely on heuristics.


I have an idea what can be done when some special markup is required 
that is not fit into current syntax. Unfortunately some new constructs 
should be introduced anyway: inline objects and multiline elements that 
represent simplified result of parsed Org structures:


     ((italic "intra") "word")

wrapped with some markup. It should satisfy any special needs (and even 
should allow to create invalid impossible constructs). Maybe idea of 
combination of lightweight markup and low-level blocks better suits for 
some other project with more expressive internal representation. In Org 
it may become the most hated feature.


I have to admit I like this idea. That brings a lot of flexibility to 
accomodate even the most obscure needs, yet it makes the discussion 
about escape characters or new symbols much less pressing. After all, 
most markup languages face the same problem, i.e., special characters 
are limited, and beyond the usual /*_ the meaning of characters becomes 
much less obvious.


This idea reminds me a bit of Scribble/Racket where every document is 
just inverted code, which makes it possible to insert arbitrary Racket 
code in your prose...


Denis

Re: citeproc-org and org-ref 3

[Joseph Vidal-Rosset]

Dear John,

It is in fact an org cite issue. With org-ref 2, I used to write:
something like [[cite:key][page  n]]  with C-l + description .

With org-ref 3, I see that I must write now: [[cite:key page n]] and it
works with ieee csl style also.

(I still do not know how to quote directly references of pages in the
cite key when I need to quote pages.)

Best wishes, and thanks !

Jo.

Le 04/12/2021 à 16:07, John Kitchin a écrit :
> AFAICT this is also a style issue. This document:
>
> #+csl-style: apa-5th-edition.csl
>
> See [[cite: pg. 127]].
>
> bibliography:~/Dropbox/emacs/bibliography/references.bib
>
> exports to:
> image.png.
>
> This one:
> #+csl-style: apa-numeric-superscript-brackets.csl
>
> See [[cite: pg. 127]].
>
> bibliography:~/Dropbox/emacs/bibliography/references.bib
> exports to
> image.png
>
> and this one (note the extra HTML_HEAD lines which fixes the separate
> lines due to the div elements (Thank you for this tip Max!).
>
> #+csl-style: biochimica-et-biophysica-acta.csl
> #+HTML_HEAD: 
> #+HTML_HEAD:   .csl-left-margin, .csl-right-inline { display: inline; }
> #+HTML_HEAD: 
>
> See [[cite: pg. 127]].
>
> bibliography:~/Dropbox/emacs/bibliography/references.bib
>
>
> leads to
> image.png
>
>
> You can see that the first two styles show the locators, but the last
> one you use does not. I don't think that is something org-ref can or
> should fix. That would be something to fix in the styles themselves.
>
> John
>
> ---
> Professor John Kitchin (he/him/his)
> Doherty Hall A207F
> Department of Chemical Engineering
> Carnegie Mellon University
> Pittsburgh, PA 15213
> 412-268-7803
> @johnkitchin
> http://kitchingroup.cheme.cmu.edu 
>
>
>
> On Sat, Dec 4, 2021 at 9:45 AM Joseph Vidal-Rosset
> mailto:jos...@vidal-rosset.net>> wrote:
>
> Dear John,
>
> Another more important point.  It is now more probably a point where
> org-ref 3  needs to be improved: it seems to me that the format
> [numerical_reference, page] or (name_reference, page) cannot be produced
> for the html export.
>
> (The format [numerical_reference, page]  was produced with org-ref2 +
> citeproc-org , see for example:
>
> 
> https://www.vidal-rosset.net/a_logical_remark_on_swinburnes_cartesian_argument_for_substance_dualism.html
> 
> 
>
> Swinburne’s argument is an amended version of Descartes’s in Discourse
> on the Method [1, p. 127]:
> )
>
> I hope that there is a solution.
>
> Best wishes,
>
> Jo.
>
> Le 03/12/2021 à 18:13, John Kitchin a écrit :
>  > I think this is caused by something in the style, or in how citeproc
>  > uses the style.
>  >
>  > This document:
>  >
>  > #+csl-style: biochimica-et-biophysica-acta.csl
>  >
>  > See [[cite:]].
>  >
>  > bibliography:~/Dropbox/emacs/bibliography/references.bib
>  >
>  > Leads to (as you have seen):
>  >
>  > image.png
>  >
>  > The html for that reference looks like:
>  > 
>  >    
>  >      [1]  > class="csl-right-inline">G. Scalia, C.A. Grambow, B. Pernici,
> Y.-P. Li,
>  > W.H. Green, https://doi.org/10.1021/acs.jcim.9b00975
> 
>  >  >">Evaluating Scalable
>  > Uncertainty Estimation Methods for Deep Learning-Based Molecular
>  > Property Prediction, Journal of Chemical Information and
> Modeling,
>  > 60 (2020) 2697-2717.
>  >    
>  > 
>  > 
>  >
>  > I don't see why there is an extra line there, I guess it is some CSS
>  > styling.
>  >
>  > if you change the csl style to apa-numeric-superscript-brackets.csl
>  >
>  > you get
>  > image.png
>  >
>  > which I think is closer to what you want. The html for this looks
> like
>  > this, and does not have some of the div elements seen above. I guess
>  > this is something in the style files themselves.
>  >
>  > 
>  >    1.
> Scalia, G.,
>  > Grambow, C. A., Pernici, B., Li, Y.-P.,  Green, W. H. (2020).
>  > Evaluating Scalable Uncertainty Estimation Methods for Deep
>  > Learning-Based Molecular Property Prediction. Journal of Chemical
>  > Information and Modeling, 60(6), 2697–2717.   > href="https://doi.org/10.1021/acs.jcim.9b00975
> 
>  >  
> >">https://doi.org/10.1021/acs.jcim.9b00975
> 
>  >  >
>  > 
>  > 
>  >
>  >
>  >
>  > John
>  >
> 

Re: On zero width spaces and Org syntax

[Max Nikulin]

On 04/12/2021 04:48, Tim Cross wrote:


My vote is to simply maintain the status quo. Don't modify the syntax,
don't make the zero space character somewhat special or processed in any
special way during export. In short, accept that inner word markup has
only limited support and if that is a requirement which is critical to
your use case, accept that org mode may not be the right solution for
your requirements.


Tim, you are skeptical concerning usage of Org markup outside of Emacs. 
Though some subscribers of this list support such idea with hope for 
collaboration with colleagues and for other reasons. Status quo in 
respect to similar questions increases risk that other tools will adapt 
different workarounds and incompatible dialects will appear.


From the point of view of popularizing Org it is better to make some 
decision: either zero-width space should become a part of syntax or some 
other printable marker should be chosen to suppress effect of Org markup 
or vice versa to activate some construct.

Re: Org-syntax: Intra-word markup

[Max Nikulin]

On 04/12/2021 06:51, Tim Cross wrote:


Please, please can we stop trying to satisfy every edge case or extend
the markup to satisfy every possible scenario.

Org's big strength is in its simplicity. This comes at a price -
limitations in what can be done. If those limitations are unacceptable,
then use a richer markup format like Latex, XML, HTML etc.


It is ridiculous to throw away a nice tool and start to struggle with 
another bunch of problems when a small missed feature is really required.



The point about back end exporter support is very relevant.


Notice that this particular feature does not require extending of 
underlying intermediate representation. There may be some subtle points 
but generally export backends are ready to intra-word markup.



In 18 years, I've seen requests for inner word markup less than 4 times.
this is not a feature we should even be considering adding to the markup
syntax.

Org provides a light weight markup, not a fully flexible rich markup
designed to meet any need. It makes the easy stuff simple.


Different users wish to have different minor features. It would be great 
to have a way to include a fragment with more verbose markup that allows 
to express special needs unsupported by lightweight markup. I am 
discussing a more general solution, not syntax extension namely for 
intra-word markup.

Re: citeproc-org and org-ref 3

[Joseph Vidal-Rosset]

Dear John,

Another more important point.  It is now more probably a point where
org-ref 3  needs to be improved: it seems to me that the format
[numerical_reference, page] or (name_reference, page) cannot be produced
for the html export.

(The format [numerical_reference, page]  was produced with org-ref2 +
citeproc-org , see for example:

https://www.vidal-rosset.net/a_logical_remark_on_swinburnes_cartesian_argument_for_substance_dualism.html

Swinburne’s argument is an amended version of Descartes’s in Discourse
on the Method [1, p. 127]:
)

I hope that there is a solution.

Best wishes,

Jo.

Le 03/12/2021 à 18:13, John Kitchin a écrit :
> I think this is caused by something in the style, or in how citeproc
> uses the style.
>
> This document:
>
> #+csl-style: biochimica-et-biophysica-acta.csl
>
> See [[cite:]].
>
> bibliography:~/Dropbox/emacs/bibliography/references.bib
>
> Leads to (as you have seen):
>
> image.png
>
> The html for that reference looks like:
> 
>    
>      [1] class="csl-right-inline">G. Scalia, C.A. Grambow, B. Pernici, Y.-P. Li,
> W.H. Green, https://doi.org/10.1021/acs.jcim.9b00975
> ">Evaluating Scalable
> Uncertainty Estimation Methods for Deep Learning-Based Molecular
> Property Prediction, Journal of Chemical Information and Modeling,
> 60 (2020) 2697-2717.
>    
> 
> 
>
> I don't see why there is an extra line there, I guess it is some CSS
> styling.
>
> if you change the csl style to apa-numeric-superscript-brackets.csl
>
> you get
> image.png
>
> which I think is closer to what you want. The html for this looks like
> this, and does not have some of the div elements seen above. I guess
> this is something in the style files themselves.
>
> 
>    1. Scalia, G.,
> Grambow, C. A., Pernici, B., Li, Y.-P.,  Green, W. H. (2020).
> Evaluating Scalable Uncertainty Estimation Methods for Deep
> Learning-Based Molecular Property Prediction. Journal of Chemical
> Information and Modeling, 60(6), 2697–2717.  href="https://doi.org/10.1021/acs.jcim.9b00975
> ">https://doi.org/10.1021/acs.jcim.9b00975
> 
> 
> 
>
>
>
> John
>
> ---
> Professor John Kitchin (he/him/his)
> Doherty Hall A207F
> Department of Chemical Engineering
> Carnegie Mellon University
> Pittsburgh, PA 15213
> 412-268-7803
> @johnkitchin
> http://kitchingroup.cheme.cmu.edu 
>
>
>
> On Fri, Dec 3, 2021 at 11:49 AM Joseph Vidal-Rosset
> mailto:jos...@vidal-rosset.net>> wrote:
>
> Le 03/12/2021 à 16:24, John Kitchin a écrit : > I have seen this
> happen at times, and I think it is style and maybe > browser
> dependent. > > Could you send me a small example (including the csl
> file you use) that > I could look at? Dear John, In attachment, two
> small examples, the same text exported with
> biochimica-et-biophysica-acta.csl and with ieee-with-url.csl Best
> wishes, Jo.
>

Re: Some commentary on the Org Syntax document

[Nicolas Goaziou]

Ihor Radchenko  writes:

>> There are actually three types of elements: not all elements can contain
>> objects.
>
> You are right. However, I am not sure if it is a good idea to mention
> this in the introduction part of the syntax document.
>
> Maybe we can just say "... lesser elements" that cannot contain other
> elements."? Then, we mention that some elements cannot contain objects
> in the description of those elements.

But then, you do not remove the ambiguity that is condemned in this
thread. The greater element/element and greater element/lesser element
distinctions are equivalent, albeit not identical.

IIUC, you want three terms for elements (I am not even talking about
secondary strings, which can hold objects that are not part of
contents), and probably two for objects: terminal and non-terminal.

Regards,

Re: citeproc-org and org-ref 3

[Max Nikulin]

On 04/12/2021 00:13, John Kitchin wrote:


     [1]class="csl-right-inline">G. Scalia, C.A. Grambow, B. Pernici, Y.-P. Li, 
W.H. Green, https://doi.org/10.1021/acs.jcim.9b00975 

I don't see why there is an extra line there, I guess it is some CSS 
styling.


 by default is a block-level element, so "[1]" is shown on a 
separate line. Either csl-left-margin and csl-right-inline should be 
e.g.  elements or CSS should have something like


.csl-left-margin, .csl-right-inline { display: inline; }

or more versatile separate styles for proper indentation.

   1. Scalia, G., 
Grambow, C. A., Pernici, B., Li, Y.-P.,  Green, W. H. (2020). 


Notice that there is no separate  elements for the number and for 
the content.

Re: Some commentary on the Org Syntax document

[Ihor Radchenko]

Nicolas Goaziou  writes:

>> This sounds reasonable. We can change
>>
>> - Three categories are used to classify these environments: “Greater
>>   elements”, “elements”, and “objects”, from the broadest scope to the
>>   narrowest. The word “element” is used for both Greater and non-Greater
>>   elements, the context should make that clear.
>> + Two main categories are used to classify these environments:
>>   "elements" and "objects", from the broadest scope to the narrowest.
>>   "Elements" consist of "greater elements" that can contain other
>>   elements and objects and "lesser elements" that can only contain
>>   objects.
>
> There are actually three types of elements: not all elements can contain
> objects.

You are right. However, I am not sure if it is a good idea to mention
this in the introduction part of the syntax document.

Maybe we can just say "... lesser elements" that cannot contain other
elements."? Then, we mention that some elements cannot contain objects
in the description of those elements.

Best,
Ihor

Re: [BUG] Org requested me to send this after doing org-cycle [9.5.1 (release_9.5.1-205-g20ed79 @ /home/jay/Builds/org-mode/lisp/)]

[Ihor Radchenko]

Jay Dresser  writes:
>
> I did org-cycle, via S-TAB, and an error occurred. I don't know why.

Thanks for reporting!

Could you also post the warning text next time you encounter it? The
text contains important clues that can help us to understand what went
wrong.

Best,
Ihor

Re: Org-syntax: emphasis and not English punctuation

[Max Nikulin]

On 03/12/2021 02:09, Juan Manuel Macías wrote:


I believe, that emphasis marks are a part of Org that can be very
shocking to new users. I mean, there is a series of behaviors that seem
obvious and trivial in the emphasized text, but that in Org are not
possible out of the box, unless you configure
`org-emphasis-regexp-components'. Three quick examples. This in Org is
not possible out of the box:

#+begin_example
[/emphasis/]
¡/emphasis/!
¿/Emphasis/?
#+end_example


Maybe this issue should be considered independently of itra-word emphasis.

Second and third examples looks like they should be supported. Ihor 
mentioned treating punctuation in a more general way. It requires rich 
test set to estimate changes in heuristics. I suspect some problems 
since start and end patterns are not symmetric and I have not found a 
way to specify in regexp only punctuation marks that normally appears in 
front of words. Square brackets likely should be excluded somehow as 
well since they are part of Org syntax. I am unsure if it is possible to 
use just regexp without additional checks of candidates.


Ihor Radchenko. [PATCH] Re: c47b535bb origin/main org-element: Remove 
dependency on ‘org-emphasis-regexp-components’

Sun, 21 Nov 2021 17:28:57 +0800.
https://list.orgmode.org/87v90lzwkm.fsf@localhost

[BUG] Org requested me to send this after doing org-cycle [9.5.1 (release_9.5.1-205-g20ed79 @ /home/jay/Builds/org-mode/lisp/)]

[Jay Dresser]

Remember to cover the basics, that is, what you expected to happen and
what in fact did happen.  You don't know how to make a good report?  See

 https://orgmode.org/manual/Feedback.html#Feedback

Your bug report will be posted to the Org mailing list.


I did org-cycle, via S-TAB, and an error occurred. I don't know why.

Emacs  : GNU Emacs 27.2 (build 1, x86_64-pc-linux-gnu, GTK+ Version 3.24.27, 
cairo version 1.17.4)
 of 2021-03-26
Package: Org mode version 9.5.1 (release_9.5.1-205-g20ed79 @ 
/home/jay/Builds/org-mode/lisp/)

Re: Some commentary on the Org Syntax document

[Nicolas Goaziou]

Hello,

Ihor Radchenko  writes:

> Timothy  writes:
>
>> ⁃ Elements
>>   • Greater Elements
>>   • (other) Elements
>>
>> to
>>
>> ⁃ Elements
>>   • Greater Elements
>>   • Lesser Elements
>
> This sounds reasonable. We can change
>
> - Three categories are used to classify these environments: “Greater
>   elements”, “elements”, and “objects”, from the broadest scope to the
>   narrowest. The word “element” is used for both Greater and non-Greater
>   elements, the context should make that clear.
> + Two main categories are used to classify these environments:
>   "elements" and "objects", from the broadest scope to the narrowest.
>   "Elements" consist of "greater elements" that can contain other
>   elements and objects and "lesser elements" that can only contain
>   objects.

There are actually three types of elements: not all elements can contain
objects.

Regards,
-- 
Nicolas Goaziou

Re: Some commentary on the Org Syntax document

[Timothy]

Hi Ihor,

> This sounds reasonable. We can change
> [snip]

 I’ll make a note in my draft then.

>>> [Comments on headings and sections]
>>
>> This accords with my reading of the document and the way I’ve implemented 
>> things
>> in OrgMode.jl (see 
>> ).
>
> One small clarification. The headline structure is actually
> (headline (optional whitespace) (optional section) (optional repeat 
> nester-headlines))

You may be happy to hear that your example seems to be interpreted correctly by
OrgMode.jl, here’s the parse tree:

┌
│ Org Parse Tree
│ Heading (This is a headline _without_ section, even though it contains 
some newlines) (empty)
│ Heading (Another headline)
│ Section
│ Paragraph
│ TextPlain
│ Heading (Next headline) (empty)
└

All the best,
Timothy