Re: Minimal Gnus setup for test purposes

[Kévin Le Gouguec]

Jens Schmidt  writes:

> More notes:

Don't know if this is well-suited to ol-gnus hacking, but the gnus-mock
package on GNU ELPA might be of interest?  I have faint memories of
using it once or twice to debug & fix Gnus problems.

Re: [RFC] If you use Org 9.6, please share the output of M-x org-element-cache-hash-show-statistics

[Kévin Le Gouguec]

Kévin Le Gouguec  writes:

> Here are the stats for today:

And today's session:

> org-element-cache-hash-show-statistics:
>> 19.32% of cache searches hashed, 11.88% non-hashable.

23.65% of cache searches hashed, 17.88% non-hashable.

> emacs-uptime:
>> 10 hours, 50 minutes, 12 seconds

13 hours, 5 minutes, 31 seconds

> emacs-version:
>> GNU Emacs 30.0.50 (build 1, x86_64-pc-linux-gnu, GTK+ Version 3.24.20,
>> cairo version 1.16.0) of 2023-02-06
>
> emacs-repository-version:
>> 907fd1f7ff402f9d226ebb3b891ea5b54fac1d1c
>
> org-version:
>> Org mode version 9.6.1 (release_9.6.1-23-gc45a05 @
>> /usr/local/share/emacs/30.0.50/lisp/org/)
>
> FWIW, this was with 5 Org files open, clocking in and out of 18 headings
> out of a little under 2000 headings.

7 Org files open, clocking in and out of 19 headings.

Re: [RFC] If you use Org 9.6, please share the output of M-x org-element-cache-hash-show-statistics

[Kévin Le Gouguec]

Hey Ihor,

Ihor Radchenko  writes:

> If you are ok with sharing the statistics, and you are running Emacs
> session for at least few hours (using Org mode, obviously), please reply
> sharing the output of
>  M-x org-element-cache-hash-show-statistics 
>  M-x emacs-uptime 

Here are the stats for today:

org-element-cache-hash-show-statistics:
> 19.32% of cache searches hashed, 11.88% non-hashable.

Evolution throughout the day: (don't know if *Messages* can be told to
add timestamps)

> 28.02% of cache searches hashed, 4.20% non-hashable.
> 19.79% of cache searches hashed, 13.80% non-hashable.
> 19.57% of cache searches hashed, 12.10% non-hashable.
> 19.25% of cache searches hashed, 11.93% non-hashable.
> 19.32% of cache searches hashed, 11.88% non-hashable.

emacs-uptime:
> 10 hours, 50 minutes, 12 seconds

emacs-version:
> GNU Emacs 30.0.50 (build 1, x86_64-pc-linux-gnu, GTK+ Version 3.24.20,
> cairo version 1.16.0) of 2023-02-06

emacs-repository-version:
> 907fd1f7ff402f9d226ebb3b891ea5b54fac1d1c

org-version:
> Org mode version 9.6.1 (release_9.6.1-23-gc45a05 @
> /usr/local/share/emacs/30.0.50/lisp/org/)

FWIW, this was with 5 Org files open, clocking in and out of 18 headings
out of a little under 2000 headings.

Hope that helps; let me know if there is more info I can collect
tomorrow.

Re: 9.6 - hide-drawer-startup option(s?) [9.5.5 (release_9.5.5 @ /usr/local/share/emacs/29.0.50/lisp/org/)]

[Kévin Le Gouguec]

Ihor Radchenko  writes:

> Kévin Le Gouguec  writes:
>
>> Just to make sure I understand this new setting correctly: by default,
>> despite org-cycle-hide-drawer-startup being t, it's ineffectual, because
>> org-startup-folded defaults to 'showeverything, right?
>
> Yes

Gotcha 

>> (Asking because when reading the ORG-NEWS entry for the new option in
>> isolation…
>
> You can try to play around with
>
> #+startup: nofold hidedrawers hideblocks

Yup, all makes sense now.  I think customizing org-startup-folded to
'nofold ≡ 'showall is the way to go for me; that will take care of
automatically hiding humongous LOGBOOKs.

Thanks again for the help!

Re: 9.6 - hide-drawer-startup option(s?) [9.5.5 (release_9.5.5 @ /usr/local/share/emacs/29.0.50/lisp/org/)]

[Kévin Le Gouguec]

Ihor Radchenko  writes:

> Thanks for the heads-up!
> Fixed on bugfix.
> https://git.savannah.gnu.org/cgit/emacs/org-mode.git/commit/?id=4323a19b1
> https://git.savannah.gnu.org/cgit/emacs/org-mode.git/commit/?id=47bcdce19

Fixes duly noted, thanks 

Just to make sure I understand this new setting correctly: by default,
despite org-cycle-hide-drawer-startup being t, it's ineffectual, because
org-startup-folded defaults to 'showeverything, right?

(Asking because when reading the ORG-NEWS entry for the new option in
isolation…

> The new custom setting gives more control over initial folding state
> of the drawers.  When set to =nil= (default is =t=), the drawers are
> not folded on startup.

… my initial reaction was "oh neat so LOGBOOKs will be folded on startup
by default", until I tried, got surprised they remained open, frowned
for a moment, then remembered this other bit…

>  Folding state of the drawers is now preserved when cycling headline 
> visibility
> 
> In the past drawers were folded every time a headline is unfolded.
> 
> Now, it is not the case anymore.  The drawer folding state is
> preserved.  The initial folding state of all the drawers in buffer is
> set according to the startup visibility settings.

… then it clicked)

9.6 - hide-drawer-startup option(s?) [9.5.5 (release_9.5.5 @ /usr/local/share/emacs/29.0.50/lisp/org/)]

[Kévin Le Gouguec]

Hey Org,

Trying out Emacs 29 and going over ORG-NEWS, this item stands out to me:

> *** A new custom setting =org-hide-drawer-startup= to control initial folding 
> state of drawers

A quick grep for org(-cycle)?-hide-drawer-startup turns this up:

File: etc/ORG-NEWS
 459  27 *** A new custom setting =org-hide-drawer-startup= to control initial 
folding state of drawers

File: lisp/org-cycle.el
 128  12 (defcustom org-cycle-hide-drawer-startup t
 630  11 (when org-cycle-hide-drawer-startup (org-cycle-hide-drawers 'all))

File: lisp/org.el
4074  20 ("hidedrawers" org-hide-drawer-startup t)
4075  22 ("nohidedrawers" org-hide-drawer-startup nil)

AFAICT those were all introduced in 2022-06-25 "org-cycle.el: New custom
setting `org-cycle-hide-drawer-startup'" (bcfed0f34).  Is the patch
below in order, or have I missed something that makes this all work?

(Haven't actually tried the feature and won't be able to before this
evening; sending this speculatively in case things are as simple as they
look.  Apologies for the noise if not)


diff --git a/etc/ORG-NEWS b/etc/ORG-NEWS
index 67889c0b1..e3b3f802d 100644
--- a/etc/ORG-NEWS
+++ b/etc/ORG-NEWS
@@ -456,7 +456,7 @@ options:
 =:icalendar-scheduled-summary-prefix=,
 =:icalendar-deadline-summary-prefix=
 
-*** A new custom setting =org-hide-drawer-startup= to control initial folding 
state of drawers
+*** A new custom setting =org-cycle-hide-drawer-startup= to control initial 
folding state of drawers
 
 Previously, all the drawers were always folded when opening an Org
 file.  This only had an effect on the drawers outside folded
diff --git a/lisp/org.el b/lisp/org.el
index 3018e4d6f..80e44d85a 100644
--- a/lisp/org.el
+++ b/lisp/org.el
@@ -4071,8 +4071,8 @@ After a match, the following groups carry important 
information:
 ("noptag" org-tag-persistent-alist nil)
 ("hideblocks" org-hide-block-startup t)
 ("nohideblocks" org-hide-block-startup nil)
-("hidedrawers" org-hide-drawer-startup t)
-("nohidedrawers" org-hide-drawer-startup nil)
+("hidedrawers" org-cycle-hide-drawer-startup t)
+("nohidedrawers" org-cycle-hide-drawer-startup nil)
 ("beamer" org-startup-with-beamer-mode t)
 ("entitiespretty" org-pretty-entities t)
 ("entitiesplain" org-pretty-entities nil))

Re: Setting org-todo-keywords through directory-local variables

[Kévin Le Gouguec]

Ihor Radchenko  writes:

> [sending to Org ML in-reply to the relevant thread]

[Thanks!]

> Kévin Le Gouguec  writes:
>
> https://debbugs.gnu.org/cgi/bugreport.cgi?bug=57003
> 28.1.90; Can local variables be loaded before loading major mode?
>
>> … reminded me of a patch I submitted to the Org ML… some time ago 
>> (sorry for not following up) to set TODO keywords via .dir-locals.el:
>>
>> https://list.orgmode.org/87a70stkmv@gmail.com/
>
> Your patch is not listed on https://updates.orgmode.org/
> It is also not in my records (I am only following patches closely since
> the beginning of this year).
> So, it slipped through the cracks.

(Right, that's entirely on me, I posted it knowing that an Org release
was pending and figuring I'd come back later…  well, better late than
never)

>> * could do so piecemeal, adding support for variables one at a time as
>>   people chime in the ML to express a need.
>
>> E.g. my patch only added support for org-todo-keywords and
>> org-todo-keyword-faces, but it laid the foundation for adding support
>> for other variables later.
>
> I'd prefer to solve it once and for all. I tried early loading of
> file-local variables in the past, but had to revert the commit because
> of major issues. See
> https://list.orgmode.org/87r11wkmew@ucl.ac.uk/T/#mab6359ed2107d5515c6bb6b266551f0c5049ceca
>
> Maybe the hook approach can work better. But I'd prefer to discuss all
> the possible caveats first.

My reasoning for keeping the current initialization code untouched and
_re_computing stuff in hack-local-variables-hook hinged on…

* refactoring being fraught; since Org already has a "blessed" way to do
  more or less what file/dir-locals do (SETUPFILEs), I figured it wasn't
  worth rocking everyone's boat for the benefit of the few,

* the prior art in other markup modes (markdown-handle-local-variables,
  font-latex-after-hacking-local-variables).

>> Also to try to reduce the risk of breakage, it went for "compute Org
>> settings normally; then selectively recompute some if relevant variables
>> are found in dir/file-locals".  That way "regular" Org users who rely
>> rather on SETUPFILEs wouldn't be impacted, only "early adopters" of
>> dir/file-locals might shoot themselves in the foot.
>
> I am not sure what is the problem with SETUPFILE.
> We can simply load it in the hook. 

I wasn't suggesting there's a problem with SETUPFILEs; my point was that
I considered two categories of users:

* those who are happy with SETUPFILEs: my implementation goal was to
  "guarantee" that my patch would have zero impact on them,

* those who want to play with dir/file-locals (): conversely, I wanted
  to make sure that only them would get to "pick up the pieces" when
  something would inevitably break.

This patch might have been my first foray into Org's init code, so it
felt too risky to go with any approach other than "keep the
implementation for the established features _exactly_ _as_ _now_; stuff
all the experimental stuff in hack-local-variables-hook".

>Though the priority of SETUPFILE vs.
> local variables should be discussed. Probably, local variables should
> take precedence to keep things consistent with the rest of Emacs.

No strong opinions there.  I'm not even sure how my patch handled that?
樂

> +(defun org--process-local-variables ()
> +  "Refresh settings affected by file-local or directory-local variables."
> +  (when
> +  (let ((local-vars (mapcar #'car file-local-variables-alist)))
> + (or (memq 'org-todo-keywords local-vars)
> + (memq 'org-todo-keyword-faces local-vars)))
> +(org-set-regexps-and-options)
> +(org-set-font-lock-defaults)))

IIUC the logic goes org-set-regexps-and-options ⇒ org-collect-keywords ⇒
org--collect-keywords-1, and that's where SETUPFILE is processed?  So
currently the SETUPFILE would have priority  (unless I'm misreading
the code)

Re: [FR] [Revived] Human readable / customizable link anchors during export

[Kévin Le Gouguec]

Ihor Radchenko  writes:

> Kévin Le Gouguec  writes:
>
>>> 3. Instead of trying to find a silver bullet for human-readable anchor
>>>generator, we allow users to customize it. The default will be
>>>constant "org" yielding "org-Ajjq"-type anchors, just like we have
>>>now. But we can also provide other generators, like the one Timothy
>>>proposed, or better versions contributed in future if there is
>>>demand.
>>
>> Personally, I'd be wholly satisfied with a customization option; the
>> grail for me would be [pandoc's algorithm], which doesn't look too hard
>> to reimplement in one's config.
>>
>> (Although the collision-handling logic could be tricky: since it
>> revolves around appending a counter to the ID, it needs to keep track of
>> how many times the same ID has been generated for the whole document.
>> If the generator API is just {heading ↦ ID}, i.e. there is no extra
>> "context" argument, one will need to do some bookkeeping "on the
>> side"…)
>
> The collision-handling should not concern the anchor generator code. If
> an anchor generator returns a duplicate ID, we can just append some
> symbols to make it unique. All the bookkeeping has to be made in
> centralized way on ox.el side.

Well, it depends on what "some symbols" are.  Pandoc's counter-based
suffixes, for example, are reproducible & stable for a given document
outline.

That's why I'd personally like to have those.  Other suffix-generation
schemes (e.g.  hashing the section content) might produce something
reproducible as well, but not necessarily so stable wrt the outline
(e.g. hashing the section content will produce different results
everytime I add or remove a comma).

Re: [FR] [Revived] Human readable / customizable link anchors during export

[Kévin Le Gouguec]

Ihor Radchenko  writes:

> 3. Instead of trying to find a silver bullet for human-readable anchor
>generator, we allow users to customize it. The default will be
>constant "org" yielding "org-Ajjq"-type anchors, just like we have
>now. But we can also provide other generators, like the one Timothy
>proposed, or better versions contributed in future if there is
>demand.

Personally, I'd be wholly satisfied with a customization option; the
grail for me would be [pandoc's algorithm], which doesn't look too hard
to reimplement in one's config.

(Although the collision-handling logic could be tricky: since it
revolves around appending a counter to the ID, it needs to keep track of
how many times the same ID has been generated for the whole document.
If the generator API is just {heading ↦ ID}, i.e. there is no extra
"context" argument, one will need to do some bookkeeping "on the
side"…)

[pandoc's algorithm]: https://pandoc.org/MANUAL.html#extension-auto_identifiers

Re: [PATCH v2 00/38] Final call for comments: Merge org-fold feature branch

[Kévin Le Gouguec]

Ihor Radchenko  writes:

> I think I addressed the raised issues.
> Just merged org-fold upstream.



> Kévin Le Gouguec  writes:
>
>> My recipe (based on commit f9dd109bc, Emacs 29.0.50 commit 864c8013fd):
>>
>> $ git switch feature/org-fold-universal-core-tidy
>> $ make autoloads
>> $ emacs -Q -L lisp -eval "(setq org-startup-folded t)" repro.org
>>
>> Restarting Emacs with the above between each step:
>>
>> (1) C-s abc ⇒ no logbook is unfolded,
>> (2) C-s def ⇒ no logbook is unfolded,
>> (3) C-s ghi ⇒ some logbooks are unfolded.
>>
>> Assuming you can reproduce: is it expected that logbooks are expanded in
>> case (3)?  I don't see what's "conceptually" different in situation (3)
>> vs. (1) and (2), so I'm puzzled to get different results.
>
> I had a hard time to reproduce your recipe because it was related to
> font-locking. The only way was increasing the font size, so that line
> widths were larger than the frame width.

Huh!  Interesting; here I could trigger the bug with all lines fitting
comfortably within the frame width; the longest is 70 columns long (line
69).

((frame-width) is 80; M-x describe-char says "DejaVu Sans Mono" 15)

Anyway, thanks for your patience!  font-lock problems are no joke 

> I believe I fixed the issue now.

Seems fixed to me!

Re: [PATCH v2 00/38] Final call for comments: Merge org-fold feature branch

[Kévin Le Gouguec]

Hey Ihor!

Ihor Radchenko  writes:

> This is the final version of the patch. I am going to merge it this
> weekend. If there are any comments, please send them ASAP.

I've thrown a couple of LOGBOOK-heavy Org files at your branch; I'm
observing something that I can't make sense of.  I tried to condense one
of these files into a small reproducer, see attached file; couldn't find
the time to make it smaller, sorry!

My recipe (based on commit f9dd109bc, Emacs 29.0.50 commit 864c8013fd):

$ git switch feature/org-fold-universal-core-tidy
$ make autoloads
$ emacs -Q -L lisp -eval "(setq org-startup-folded t)" repro.org

Restarting Emacs with the above between each step:

(1) C-s abc ⇒ no logbook is unfolded,
(2) C-s def ⇒ no logbook is unfolded,
(3) C-s ghi ⇒ some logbooks are unfolded.

Assuming you can reproduce: is it expected that logbooks are expanded in
case (3)?  I don't see what's "conceptually" different in situation (3)
vs. (1) and (2), so I'm puzzled to get different results.


Also, a bit of idle curiosity:

> (defun org-fold--isearch-reveal ( _)
>   "Reveal text at POS found by isearch."
>   (org-fold-show-set-visibility 'isearch))

org-fold-show-set-visibility calls either
org-fold-show-set-visibility--overlays, or
org-fold-show-set-visibility--text-properties, and AFAICT neither of
these handle 'isearch as an argument…  Is there a (cdr (assq 'isearch
org-fold-show-context-detail)) missing?

(This comes from a very cursory reading of the code; apologies if I've
missed something)


Other than this logbook oddity, I haven't found anything concerning.
Thanks for your efforts!


* xxx 
:LOGBOOK:
CLOCK: [2021-11-02 Tue 17:18]--[2021-11-02 Tue 17:25] =>  0:07
:END:
** xxx xxx x
***  [[:xxx/xx/x]]
:LOGBOOK:
CLOCK: [2021-11-03 Wed 13:51]--[2021-11-03 Wed 14:13] =>  0:22
CLOCK: [2021-11-03 Wed 11:52]--[2021-11-03 Wed 12:00] =>  0:08
CLOCK: [2021-11-03 Wed 11:27]--[2021-11-03 Wed 11:42] =>  0:15
CLOCK: [2021-11-03 Wed 09:06]--[2021-11-03 Wed 10:04] =>  0:58
CLOCK: [2021-11-02 Tue 18:11]--[2021-11-02 Tue 18:45] =>  0:34
CLOCK: [2021-11-02 Tue 17:36]--[2021-11-02 Tue 17:38] =>  0:02
CLOCK: [2021-11-02 Tue 17:25]--[2021-11-02 Tue 17:31] =>  0:06
:END:
- [ ] x://.xx.xxx.xxx/xxx/ ?
  (.xx x xx xxx)
- 
  - "xxx abc xxx  xx"
 [[x://.xxx.xxx/xxx/#/xxx/][xxx xxx  xxx]]
 [[x://.xxx.xxx/xxx/xx/x#x-xxx-xx-xx][x xxx xx xx]]
 [[x://.xxx.xxx/xxx/xx/x#xx-xx][xx xx]]
- def xx:
  - x ()
  - x ()
 [[x://.xxx.xxx/xxx/xx/x#--xxx-xxx-x][xxx xxx]]
xx xx xxx xx xxx xxx? ⇒
xxx-x...@xxx.xxx
 [[:]]
: ~xxx..xxx:/xxx/-~
***  [[x://.xx.xxx//x/xxx-/][xx xxx]]
:LOGBOOK:
CLOCK: [2021-11-03 Wed 15:13]--[2021-11-03 Wed 15:14] =>  0:01
CLOCK: [2021-11-03 Wed 14:55]--[2021-11-03 Wed 14:57] =>  0:02
:END:
***  [[x://.xx.xxx//x//][ xxx xx xx xxx xxx]]
:LOGBOOK:
CLOCK: [2021-11-03 Wed 15:14]--[2021-11-03 Wed 15:17] =>  0:03
:END:
***  [[x://.xx.xxx//x/xx_x/][xx xx xxx xxx]]
:LOGBOOK:
CLOCK: [2021-11-03 Wed 16:00]--[2021-11-03 Wed 16:30] =>  0:30
:END:
***  [[:xxx/]]
:LOGBOOK:
CLOCK: [2021-11-05 Fri 11:02]--[2021-11-05 Fri 11:18] =>  0:16
CLOCK: [2021-11-05 Fri 09:42]--[2021-11-05 Fri 09:55] =>  0:13
CLOCK: [2021-11-04 Thu 18:30]--[2021-11-04 Thu 18:32] =>  0:02
CLOCK: [2021-11-04 Thu 12:07]--[2021-11-04 Thu 12:14] =>  0:07
CLOCK: [2021-11-04 Thu 11:10]--[2021-11-04 Thu 12:00] =>  0:50
CLOCK: [2021-11-03 Wed 18:11]--[2021-11-03 Wed 18:20] =>  0:09
:END:
-  :: xxx + 
- xx ::   xx  xxx
  - xxx xxx "xx":
- x xx xx   x
- xxx xx xxx ∈ >x xxx
-  :: xx xx  xxx + 
  - xx xx x xxx x
  - xx xx  xx
- xxx :: x//xx x
  -   xx
- 
  - [[:xxx//xxx]]
- " xxx : xx xxx x xxx xxx,  
  xx x x "
- "xx xxx  xx x  xx  xxx"
  - [[x://.xx.xxx//x//#xxx=x][xxx x xxx]]
- " xxx xx  xx x xxx  
  x xxx"
  - 

New folding backend & outline (was: [PATCH 22/35] ORG-NEWS: Add list of changes)

[Kévin Le Gouguec]

Ihor Radchenko  writes:

> + =outline-*= functions may no longer work correctly in Org mode
> +
> +The new folding backend breaks some of the =outline-*= functions that
> +rely on the details of visibility state implementation in
> +=outline.el=.  The old Org folding backend was compatible with the
> +=outline.el= folding, but it is not the case anymore with the new
> +backend.  From now on, using =outline-*= functions is strongly
> +discouraged when working with Org files.

>From the perspective of a heavy outline-minor-mode user, who dreams of
Org "backporting" its great outlining features to outline.el, that's a
bit disheartening, since IIUC this will cause Org and outline.el to
drift further apart?

I realize this question might sound outlandish, but I'll ask it anyway:
would it be feasible (and relevant) to add this new folding backend to
outline.el, so that (1) /both/ Org and outline(-minor)-mode benefit from
it, (2) outline.el functions keep working in Org?

(Assuming outline.el could be turned into a :core GNU ELPA package, and
Org would tolerate adding this dependency)

I hope this doesn't come across as negative criticism; the amount of
work that went into this branch is astounding, and as an Org user I'm
indebted to the developers for the energy that goes into maintaining it.

I just wish more of the loving care that goes into Org trickled down to
outline(-minor)-mode; the last couple of months were encouraging because
lots of great improvements have been added to outline.el (TAB-cycling,
buttons, control over default visibility and font-locking), and those
improvements enhance *every* major mode with outline-minor-mode support,
so I was hoping for the trend to continue…

Lest I let this message end on that sour note: great work, and thanks
for the energy you put into Org!

bug#52587: 29.0.50; Wrong block header/footer background in Org

[Kévin Le Gouguec]

Protesilaos Stavrou  writes:

> On 2021-12-26, 22:04 +0100, Rudolf Adamkovič  wrote:
>
>>> - Create a file with demo content, such as ~/test-org-block.org
>>> - Execute 'emacs -Q' on the command-line.
>>> - M-x load-theme RET modus-operandi
>>> - C-x C-f test-org-block.org RET
>
> Gotcha!  I can reproduce that.  It is not a theme issue.  I checked the
> Org code a bit.  There is a function in org-compat.el called
> 'org--set-faces-extend' and in org.el we see it being used in the
> (define-derived-mode org-mode ... part.
>
> In other words, M-x org-mode wants those faces to have ':extend t'.

Only when org-fontify-whole-block-delimiter-line is set (which it is, by
default, I think).

bug#52399: 28.0.60; easier access to ORG-NEWS

[Kévin Le Gouguec]

Kyle Meyer  writes:

>> 3. The entry in NEWS for the Org mode update mentions ORG-NEWS, but it
>> doesn't say how to get to it.  (Yes, I know how to get to ORG-NEWS from
>> NEWS.  Do we expect random users to understand the layout of an
>> installed Emacs at that level of detail?)
>
> And I guess this one is for Emacs's side at the time of the sync.
>
> I was looking through the NEWS* files in the Emacs repo for guidance
> from other packages, but I see mostly the same thing ("GNUS-NEWS",
> "MH-E-NEWS", "ERC-NEWS", ...).  I guess there is one that's a little
> different, though I'm not sure how helpful tacking on "./" is: "See the
> file ./MH-E-NEWS for details".
>
> Do you have any suggestions for what would be more helpful to put there?

28.1 introduces etc-authors-mode; perhaps, in a similar vein, we could
add etc-news-mode, and add have the mode initialization function
buttonize these cross-references?

(Older emacsen would get confused by the "mode: etc-news" clause in
newer NEWS files, though…)

Re: [BUG] org-mode binds C-c C-TAB, which seems illegal [9.5 (9.5-g0a86ad @ /home/il/.config/emacs/elpa/org-9.5/)]

[Kévin Le Gouguec]

Ingo Lohmar  writes:

> It seems the change was introduced in
> 565361eb698b0b39c1d823ad1565f5bd88fa2034 and persists.
>
> Can people actually enter "C-c C-TAB" into their emacs (how?), or has
> everybody has just bound another key in their config?

Mmm, I can't seem to input C-c C-TAB either.  IIUC (but maybe I don't),
this makes sense because

- Emacs translates the function key  into the control character
  TAB=^I when no modifiers are added.  I.e. this can be triggered by
  hitting  or +i:

> (local-set-key (kbd "TAB") (lambda () (interactive) (message "TAB-ish!")))

- But Emacs can't translate + into "C-TAB", because C-TAB
  means "control+control+i", which I guess is not representable at the
  key code level or something?  Hopefully someone can explain this
  better.

(?\C-\t does return something though, and it's consistent with what (kbd
"C-TAB") returns, so I guess there's no reason why Emacs couldn't
translate C- to C-TAB like it does  to TAB? 路)

FWIW, however you decide to fix this, I'd be very grateful if org-cycle
remained bound to TAB, since I'm one of those weirdos who actually hits
+i for TAB instead of …

Re: [PATCH] Re: [BUG] indention of drawer does not work [9.5 (release_9.5-145-gd18beb @ /home/oub/emacs/site-lisp/packages/org/)]

[Kévin Le Gouguec]

Uwe Brauer  writes:

>>>> "KLG" == Kévin Le Gouguec  writes:
>
>> I've applied your fix on top of 281a0e26b; AFAICT it works as expected.
>
> I applied the patch on top of e2fa3c4c4046b6,

(Me too actually; 281a0e26b is the commit ID I got by applying Ihor's
patch; apologies for the confusion)

> the two examples you sent uwe1.org and uwe2.org do not indent the
> drawer with org-indent-mode turned on

Mmm, starting from emacs -Q and running M-x org-indent-mode, I see Org
indent the :PROPERTIES: drawers in both files, both with the tip of
Org's 'main' branch (e2fa3c4) and with Ihor's patch applied.

Re: [PATCH] Re: [BUG] indention of drawer does not work [9.5 (release_9.5-145-gd18beb @ /home/oub/emacs/site-lisp/packages/org/)]

[Kévin Le Gouguec]

Ihor Radchenko  writes:

>The tentative fix is attached, but please
> double check because I am not very familiar with the indentation code.

I've applied your fix on top of 281a0e26b; AFAICT it works as expected.

Thanks for cooking up a test case!  'make check' succeeds over here, so
either this patch is The Right Thing, or we'll have an opportunity to
add more tests soon enough 

Re: [BUG] indention of drawer does not work [9.5 (release_9.5-145-gd18beb @ /home/oub/emacs/site-lisp/packages/org/)]

[Kévin Le Gouguec]

Uwe Brauer  writes:

> In the attached file 

Maybe I missed it, but I don't think your report included an attachment?
I went with the two example files you showed in the previous discussion;
cf. my own attachments.

>  I do:
>
> M-: (setq org-adapt-indentation 'headline-data)
> C-x h
> M-: (indent-for-tab-command)
>
> Result 
>
>  the :PROPERTIES: drawer is not indented.
>
> Emacs  : GNU Emacs 28.0.50 (build 1, x86_64-pc-linux-gnu, X toolkit, cairo 
> version 1.14.6, Xaw3d scroll bars)
>  of 2021-07-31
> Package: Org mode version 9.5 (release_9.5-145-gd18beb @ 
> /home/oub/emacs/site-lisp/packages/org/)

I can reproduce with the latest commit on 'main' (e2fa3c4c4); AFAICT the
release that is included in the emacs-28 branch (52e6f1) works
correctly, which is why I did not see the problem.

I tried to bisect; fc80d052d fails to compile so it's hard to be sure,
but I think it might be when things started to go awry.  I can't
reproduce the bug with the parent commit (6933c1ad7), but I can with the
next one (bc52c4d9a).

fc80d052d was a rather chunky commit so I did not dig into the patch;
Ihor, does it make sense to you that it might have introduced
unfortunate side-effects wrt :PROPERTIES: indentation?


** DONE G1:H1:G1:
:PROPERTIES:
:ID:   G1
:User1:John
:email1:   j...@gmail.com
:Start:<2021-02-16 mar>
:End:  <2021-05-24 lun>
:STATUS:   [ ]
:ST1:  [ ]
:Sheet:H1
:Ex:   E2
:Ex:   E1
:END:
* TODO G1 <2021-10-21 jue 17:10> :H1:G1: Uwe Brauer
:PROPERTIES:
:ID:   
:EMAIL: o...@mat.ucm.es
:Grp1: G1
:Usuario1: Uwe Brauer
:email1: o...@mat.ucm.es
:Speaker: Uwe
:End:

Re: [BUG in org master?]

[Kévin Le Gouguec]

Tim Cross  writes:

> The effect of org-adapt-indentation is somewhat confusing to say the
> least. The effect of that setting is also dependent on the setting for
> electric-indent-mode. Getting the desired result often depends on having
> the right setting for both that variable and electric-indent-mode.

In case that can clear things up:

- org-adapt-indentation dictates what the "canonical" indentation should
  be, i.e. what happens when you select the whole text and M-:
  (indent-for-tab-command):

- nil (default as of 9.5): nothing is indented;
- t (default before 9.5): drawers and text are indented;
- 'headline-data: drawers are indented.

- electric-indent-mode dictates *what key* causes automatic indentation:
- nil: C-j indents, RET does not,
- t: C-j does not indent, RET does.

Re: [BUG in org master?]

[Kévin Le Gouguec]

Uwe Brauer  writes:

>> In Org 9.5, setting org-adapt-indentation to 'headline-data makes
>> :DRAWERS: indented (setting it to t makes everything but headlines
>> indented).
>
> Thanks, but it did not work. I set the variable 
> org-adapt-indentation to 'headline-data makes
>
>
> But when I opened my org file in question I still saw
>
> * TODO G1 <2021-10-21 jue 17:10> :H1:G1: Uwe Brauer
> :PROPERTIES:
> :ID:   
> :EMAIL: o...@mat.ucm.es
> :Grp1: G1
> :Usuario1: Uwe Brauer
> :email1: o...@mat.ucm.es
> :Speaker: Uwe
> :End:
>
> Even restarting emacs did not help.
>
> I am running emacs 28 (master)
> 83a915d3dfafd5f3d737afe1e13b75e4dd3aef96
>
> And org master compiled  two days ago.

Just to make sure I'm not misundertanding: did you try to re-indent the
whole file?  You say "when I opened my org file in question I still
saw…", so I'm not sure if you tried to re-indent, or if you merely set
org-adapt-indentation and expected the file to be automatically
re-indented after opening it again.

If the :PROPERTIES: drawer is not indented after doing the following:

M-: (setq org-adapt-indentation 'headline-data)
C-x h
M-: (indent-for-tab-command)

… then yes, there is a bug.

> Setting the variable to t seems to be the same case using
> org-indent-mode, which is usually try to avoid.

Sure, the reason why it is (now) nil by default is because a fair number
of users on this list have told Org maintainers that they dislike
indented text.

Note that there is a difference between org-adapt-indentation set to t,
and org-indent-mode:

- org-adapt-indentation uses *hard* indentation (Emacs inserts
  whitespace which is written to the file),

- org-indent-mode uses *soft* or "visual" indentation (Emacs adds text
  properties to shift the text, but no whitespace is written to the
  file).

Re: how to indent (or refill) properties

[Kévin Le Gouguec]

Uwe Brauer  writes:

> But I would like them to be indented like this
>
> ** DONE G1:H1:G1:
> :PROPERTIES:

In Org 9.5, setting org-adapt-indentation to 'headline-data makes
:DRAWERS: indented (setting it to t makes everything but headlines
indented).

If your drawers are already written and you want to mass-reindent them
with e.g. C-x h TAB, either make sure point is not on a headline (or TAB
will just cycle visibility) or tweak org-cycle-emulate-tab.  Or use
C-x h M-: (indent-for-tab-command).

Not sure how to tell Org how wide indentation should be though.

bug#51167: 29.0.50; org-indent-line broken

[Kévin Le Gouguec]

Andreas Röhler  writes:

> On 13.10.21 09:34, Kévin Le Gouguec wrote:
>>
>> "Modern" did not factor in; the goal was to have RET and C-j behave
>> consistently in all major modes.
>
> That does not deliver an argument to change the meaning of RET.

If there is a compelling argument that justifies RET and C-j behaving
differently in Org wrt other major modes, I haven't heard it yet.

> BTW the costs of such changes are terribly underestimated in Emacs.

AFAICT, the costs of user-facing changes are regularly discussed on the
Emacs development lists, and different developers have different
opinions on how underestimated they are.

In the specific case of RET and C-j, I'd argue (and Org maintainers seem
to have agreed) that the long-term benefits of Org falling in line with
other modes outweigh the short-term costs of annoying long-time users,
especially since they are offered ways to bring back the previous
behaviour (outlined in ORG-NEWS).

And in the specific case of org-adapt-indentation, again, changing the
default to nil was the result of extensive discussion on emacs-orgmode,
where several users explicitly stated that they did not want text to be
indented (neither with RET, C-j, TAB, nor org-indent-line) and never
realized that org-adapt-indentation was t because Org ignored
electric-indent-mode before 9.4.

>> Since electric-indent-mode is enabled globally in Emacs,
>
> Which IMO was another mistake.
>
> Preferring a clean editor, which does fancy things only if enabled.

There are plenty of things Emacs does by default that I personally find
unhelpful; fortunately I can just disable them.  And as long as release
notes point out changes in default behaviour (and how to revert them),
I'm happy with new releases enabling new features.

YMMV 路

bug#51167: 29.0.50; org-indent-line broken

[Kévin Le Gouguec]

Andreas Röhler  writes:

> Sounds like a chain of confusion.
>
> A command called "indent-line" definitely should indent.

org-indent-line indents just like every indentation function in every
other major mode: if the syntactic convention calls for it, it indents
or de-indents the current line; otherwise it leaves the line untouched.

Or are you saying you would like org-indent-line to also indent "* bla",
because « a command called "indent-line definitely should indent »?

> Seems the original coulprit is that unhappy switch of RET and C-j, in
> order to make Emacs "modern".

"Modern" did not factor in; the goal was to have RET and C-j behave
consistently in all major modes.

> Maybe make RET RET again?

After much discussion on emacs-orgmode, it has been found that a lot of
users expect neither (1) their prose to be indented, nor (2) RET to
indent.

Since electric-indent-mode is enabled globally in Emacs, RET indents
according to the major mode's indentation rules.  Thus it was decided to
change the default value of org-adapt-indentation, to reflect the
expectations of the Org users who took the time to chime in on the
mailing list to describe their workflow and expectations.

If you think the default value should be reverted back to t, I suggest
you make your case on emacs-orgmode.

bug#51167: 29.0.50; org-indent-line broken

[Kévin Le Gouguec]

Andreas Röhler  writes:

> With following stuff in org-mode buffer:
>
> * bla
> asd
>
> M-x org-indent-line RET on second line has no effect.

Org 9.5 changed the default value of org-adapt-indentation from t to
nil, as that seemed to be what a lot of users expect[1], so
org-indent-line should not indent the second line in Emacs 28 onward
unless configured otherwise:

- setting org-adapt-indentation back to t will make Org indent by
  inserting whitespace;

- alternatively, enabling org-indent-mode will make Org "soft-indent"
  with text properties.


[1] Org 9.4 made RET and C-j obey electric-indent-mode like they do in
most other major modes.  Since org-adapt-indentation was t by
default, this led to many dismayed reports on emacs-orgmode that
"RET now messes up indentation", indicating that these users did not
expect their prose to be indented.

Re: Sad tweet

[Kévin Le Gouguec]

(Took the liberty of CC'ing Jonas to make sure he can correct any
mischaracterization, and to show our support, such as it is)

band...@gnu.org writes:

> Ypo writes:
>
>> I've read this:
>>
>> "Contributing to Emacs is so frustrating. It's not worth it for minor
>> things and if I cannot get some experience and confidence with minor
>> things, then I likely won't ever make major contributions."
>> https://twitter.com/magit_emacs/status/1396536686570610697?s=19
>
> Do you know if there is any more context around that?  Did Jonas mention
> any specific pain points around contributing to Emacs and/or concrete
> things that he thinks could be improved?  Last time I'd seen him post on
> emacs-devel it seemed like things were going fairly smoothly with his
> work on adding transient to Emacs(?).

Given the timing, I'd hazard that this stems from bug#48592 (plus a few
more past attempts that Jonas deems similarly fruitless, I assume).

FWIW, to bounce off Amin's reply: Jonas, the patience you demonstrated
in order to get transient in Emacs core was nothing short of saintly,
and I for one am grateful for your perseverance.

I understand how Emacs's development process can feel frustrating,
especially in Jonas's position as maintainer of a popular package like
Magit:

1. on the one hand, each and every attempt at contributing is met with
   varying degrees of skepticism and defiance, on the premise that you
   might e.g. break other people's code, disrupt other people's
   workflow…

2. on the other hand, upstream sometimes adds major features which
   impact your package, and you wake up to lots of disgruntled users
   expecting you to fix something you never saw coming; cf. :extend t,
   the tentative binding for C-x g…

I don't necessarily view 1 nor 2 as inherently problematic: for 1, we're
lucky to have maintainers looking out for breakage, although the line
between "healthy conservatism" and "clinical sclerosis" is blurry; for
2, users of the development branch or the latest release should expect
some measure of breakage in third-party packages.

As a user, watching from the sidelines, the process "works": third-party
additions slowly make their way upstream after some review and a
generous coating of backward-compatibility/accessibility changes; on the
flip side, bleeding-edge users warn third-party maintainers of upcoming
changes which can then be amended before they make it into a release.

Even so, as a third-party maintainer, I assume the combination of 1 and
2 feels like a "power imbalance": one party makes the other's life
consistently harder.

So, once more with feeling: thank you Jonas for your patience and your
perserverance 


Disclaimer: I'm very much just a user, whose free time is mostly gobbled
up catching up with the mailing lists.  This reply is my interpretation
of what I observe and may not be representative of anybody else's
feelings on the subject.

Re: Bug: spurious change in list indent cursor motion [9.4.4 (9.4.4-dist @ /home/powellj/elisp/org-9.4.4/lisp/)]

[Kévin Le Gouguec]

Greg Minshall  writes:

> Kevin,
>
>> FWIW, during the latest poll somebody suggested making org-indent-line
>> cycle through "syntactically valid" indentation levels when hitting TAB
>> repeatedly, like python-indent-line-function; I like this idea.
>
> i think (*) the current "master" branch allows you to type "-
> fu- *bar" (where the " *" bit means "zero or more spaces").
> but, i guess because of folding, it's that sensitive, doesn't cycle when
> any non-blank is typed.

Right, that allows changing the indentation level of a newly inserted
bullet point; my remark was about changing the indentation level of
regular text, e.g. "- list itemunindented paragraph".

C-j works well enough when there are only two valid positions (column 2
and column 0); in this situation though:

- foo
  - bar
  - baz

It would be nice if RET TAB TAB TAB… cycled between columns 2→0→4…, just
like M-RET TAB TAB TAB… cycles between columns 6→2→4…

Re: Bug: spurious change in list indent cursor motion [9.4.4 (9.4.4-dist @ /home/powellj/elisp/org-9.4.4/lisp/)]

[Kévin Le Gouguec]

Nicolas Goaziou  writes:

>> Note also that "- a" goes
>> back to column 0.  (FWIW I think that's a bit unwieldy; going back to
>> column 0 on the /second/  would make more sense to me, as
>> it would correspond to a "paragraph break")
>
> It would make it painful to insert a blank line within a list item.
> OTOH, on the third newline, you are really out of the list. 

Fair enough!  I figured the current behaviour made it easier to write
"multiple paragraphs" in a single list item, but I was unsure if anyone
actually relied on that.  Thanks for confirming this hunch :)

FWIW, during the latest poll somebody suggested making org-indent-line
cycle through "syntactically valid" indentation levels when hitting TAB
repeatedly, like python-indent-line-function; I like this idea.

Re: Bug: spurious change in list indent cursor motion [9.4.4 (9.4.4-dist @ /home/powellj/elisp/org-9.4.4/lisp/)]

[Kévin Le Gouguec]

James Powell  writes:

> I open a new file, /tmp/demo.org, and I start a new list
> by typing "- a".  It appears:
>
> - a
>
> What I expect: my cursor should now be at column 0.  This was the
> behavior in 9.3.6 and it makes perfect sense.
>
> What happens instead: after upgrade to 9.4.4, the cursor rests at
> column 2, under the a.
>
> How do I change the behavior back to the 9.3.6 one?

Short answer: disable electric-indent-local-mode, as suggested in
ORG-NEWS.  This will turn RET into the "dumb newline" key which should
never indent, and set C-j as the "smart newline" key which occasionally
indents.

> Facts about it:
>
> - When I start emacs with 'emacs -Q -nw' the bug does not manifest
>   even in 9.4.4.

That's surprising; on my setups, with Org 9.4.4, emacs -Q -nw stays at
column 2 after hitting RET.

> - I tried setting org-adapt-indentation to 't (the default) as it is
>   an obvious suspect, the bug still manifested.

org-adapt-indentation's determines how TAB and the "smart newline" key
behave:

- t (default): section bodies should be hard-indented,

- nil (to become the default for Org 9.5): nothing should be indented,

- 'headline-data: drawers (e.g. :LOGBOOK: entries and :PROPERTY: blocks)
  should be hard-indented, but not regular section text.

With all settings, "- a" indents at column 2, below "a".
As explained above, depending on electric-indent, the 
key is either RET or C-j.

Note also that "- a" goes
back to column 0.  (FWIW I think that's a bit unwieldy; going back to
column 0 on the /second/  would make more sense to me, as
it would correspond to a "paragraph break")

Re: [POLL] Setting `org-adapt-indentation' to nil by default?

[Kévin Le Gouguec]

Bastien  writes:

>> It is now, as of commit 0a651b746.
>
> ... and I broke some tests.  Sorry for that.  I will fix this next
> week, unless someone does it before me.

Here are two patches that set org-adapt-indentation to t for the tests
which were implicitly relying on that behavior; that lets 'make test'
succeed again for me.

I'm pretty sure the first one is TRT (since I'm the author of the
tests), although Eventually™ we should make a more exhaustive suite
based on the table you referenced earlier.

The second one makes the remaining tests pass again, but I couldn't tell
at a glance whether their expectations still make sense.

>From e136f0d3123173d947bf4c1ce06aaf5f12117ef8 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?K=C3=A9vin=20Le=20Gouguec?= 
Date: Sun, 9 May 2021 18:05:35 +0200
Subject: [PATCH 1/2] Set org-adapt-indentation explicitly in some tests

* testing/lisp/test-org.el (test-org/with-electric-indent,
test-org/without-electric-indent): Make sure org-adapt-indentation is
consistent with expected results.
---
 testing/lisp/test-org.el | 42 +++-
 1 file changed, 24 insertions(+), 18 deletions(-)

diff --git a/testing/lisp/test-org.el b/testing/lisp/test-org.el
index 9f0ede1b9..5ac9173ac 100644
--- a/testing/lisp/test-org.el
+++ b/testing/lisp/test-org.el
@@ -1404,12 +1404,14 @@
 	(electric-indent-local-mode 1)
 	(call-interactively 'org-return)
 	(buffer-string
-  (should
-   (equal "* heading\n  body"
-	  (org-test-with-temp-text "* headingbody"
-	(electric-indent-local-mode 1)
-	(call-interactively 'org-return)
-	(buffer-string
+  ;; TODO: test more values of `org-adapt-indentation'.
+  (let ((org-adapt-indentation t))
+(should
+ (equal "* heading\n  body"
+	(org-test-with-temp-text "* headingbody"
+	  (electric-indent-local-mode 1)
+	  (call-interactively 'org-return)
+	  (buffer-string)
   ;; C-j, like `electric-newline-and-maybe-indent', should not indent.
   (should
(equal "  Para\ngraph"
@@ -1423,12 +1425,14 @@
 	(electric-indent-local-mode 1)
 	(call-interactively 'org-return-and-maybe-indent)
 	(buffer-string
-  (should
-   (equal "* heading\nbody"
-	  (org-test-with-temp-text "* headingbody"
-	(electric-indent-local-mode 1)
-	(call-interactively 'org-return-and-maybe-indent)
-	(buffer-string)
+  ;; TODO: test more values of `org-adapt-indentation'.
+  (let ((org-adapt-indentation t))
+(should
+ (equal "* heading\nbody"
+	(org-test-with-temp-text "* headingbody"
+	  (electric-indent-local-mode 1)
+	  (call-interactively 'org-return-and-maybe-indent)
+	  (buffer-string))
 
 (ert-deftest test-org/without-electric-indent ()
   "Test RET and C-j specifications with `electric-indent-mode' off."
@@ -1467,12 +1471,14 @@
 	(electric-indent-local-mode 0)
 	(call-interactively 'org-return-and-maybe-indent)
 	(buffer-string
-  (should
-   (equal "* heading\n  body"
-	  (org-test-with-temp-text "* headingbody"
-	(electric-indent-local-mode 0)
-	(call-interactively 'org-return-and-maybe-indent)
-	(buffer-string)
+  ;; TODO: test more values of `org-adapt-indentation'.
+  (let ((org-adapt-indentation t))
+(should
+ (equal "* heading\n  body"
+	(org-test-with-temp-text "* headingbody"
+	  (electric-indent-local-mode 0)
+	  (call-interactively 'org-return-and-maybe-indent)
+	  (buffer-string))
 
 (ert-deftest test-org/meta-return ()
   "Test M-RET (`org-meta-return') specifications."
-- 
2.31.1

>From 2a485754a7f04d00ef5e5ebed82924d44f768424 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?K=C3=A9vin=20Le=20Gouguec?= 
Date: Sun, 9 May 2021 18:20:11 +0200
Subject: [PATCH 2/2] Set org-adapt-indentation explicitly in some tests

* testing/lisp/test-org.el (test-org/indent-line,
test-org/indent-region): Make sure org-adapt-indentation is consistent
with expected results.
---
 testing/lisp/test-org.el | 126 ---
 1 file changed, 66 insertions(+), 60 deletions(-)

diff --git a/testing/lisp/test-org.el b/testing/lisp/test-org.el
index 5ac9173ac..95ffb0a80 100644
--- a/testing/lisp/test-org.el
+++ b/testing/lisp/test-org.el
@@ -1007,22 +1007,23 @@
   ;; At the first line of an element, indent like previous element's
   ;; first line, ignoring footnotes definitions and inline tasks, or
   ;; according to parent.
-  (should
-   (= 2
-  (org-test-with-temp-text "A\n\n  B\n\nC"
-	(org-indent-line)
-	(org-get-indentation
-  (should
-   (= 1
-  (org-test-with-temp-text " A\n\n[fn:1] B\n\n\nC"
-	(org-indent-line)
-	(org-get-indentation
-  (should
-   (= 1
-  (org-test-with-temp-text
-	  " #+BEGIN_CENTER\n  Contents\n#+END_CENTER"
-	(org-indent-line)
-	(org-get-indentation
+  (let ((org-adapt-indentation t))
+(should
+ (= 2
+(org-test-with-temp-text "A\n\n  B\n\nC"
+	 (org-indent-line)
+	 

Re: [POLL] Setting `org-adapt-indentation' to nil by default?

[Kévin Le Gouguec]

Bastien  writes:

> Kévin Le Gouguec  writes:
>
>> Great!  One last snag that I can see: when inserting properties or
>> clocking in, the :LOGBOOK:, :PROPERTIES: and :END: lines are indented,
>> but the /first/ :property: or CLOCK: line remains at column 0.
>
> Er.  Can you try this (hopefully last) patch and report any problem?
>
> Thanks for your patience in testing this!

Well, try as I might, I couldn't get this one to misindent  And a good
thing too, I was about to run out of synonyms for wrinkles/nits/snags
and the like 


If anyone is wondering what these exchanges are all about: with
Bastien's latest patch[1], setting org-adapt-indentation to
'headline-data results in the following behaviour, *whether
electric-indent-mode is enabled or not*:

- "* heading RET" starts an unindented line (point goes to column 0).
- The drawers inserted by org-set-property and org-clock-in are
  indented.

So, to bring the discussion back to the poll:

- If nil or 'headline-data becomes the default value, RET will no longer
  indent regular text after a headline (even with electric-indent-mode).
- The difference will be whether :DRAWERS: are indented or not.

As far as I am concerned, my preference has become:

(1) 'headline-data (indented drawers)
(2) nil (nothing indented)
(3) t (the status quo)

although I suppose an argument could be made that 'headline-data might
be a bit "young" and as such, should not be used as default value…

Let's see what the poll yields 


[1] Uncommited, to apply on branch maint:
https://orgmode.org/list/87fsz4ue9e@bzg.fr/2-fix-indent.patch

Re: [POLL] Setting `org-adapt-indentation' to nil by default?

[Kévin Le Gouguec]

Bastien  writes:

> Kévin Le Gouguec  writes:
>
>> A fews more moles to whack, maybe:
>>
>> - RET after an ":END:" starts an indented line,
>> - "* headline RET text TAB" indents "text" (and subsequent RETs are then
>>   indented).
>
> Fixed, thanks.

Great!  One last snag that I can see: when inserting properties or
clocking in, the :LOGBOOK:, :PROPERTIES: and :END: lines are indented,
but the /first/ :property: or CLOCK: line remains at column 0.

I've looked briefly at org-indent-line; IIUC, those lines get caught in
this condition:

(save-excursion
  (beginning-of-line 1)
  (skip-chars-backward "\n")
  (or (org-at-heading-p)
  (org-at-drawer-p)
  (org-at-planning-p)))

I'm guessing we'd need to throw some (org-at-property-p) and
(org-at-clock-log-p) guards *before* skipping backward over "\n", but I
haven't thought this through completely yet.

>> should at least write test cases for them.  Will try to find the time
>> Sometime Later™.
>
> This might help: https://orgmode.org/worg/org-faq.html#indentation

Wonderful!  That will definitely help, thanks.

Re: [POLL] Setting `org-adapt-indentation' to nil by default?

[Kévin Le Gouguec]

Tim Cross  writes:

> Jean Louis  writes:
>
>> If I set `org-adapt-indent' to 'headline-data, I get that same
>> behavior that after pressing ENTER on headline line, position becomes
>> indentend. So it does not make it right.
>>
>> My favour was the behaviour how it was before introduction of
>> indentation change:
>>
>> - after headline, cursors went to beginning of line; I find it
>>   usable, as that is where I write text. 
>>
>> - if I ever wanted to enter properties with C-c C-x p, those were 
>> automatically
>>   indented,
>
> This is  exactly what headline-data does. I suspect what your running
> into is electric-indent-mode and you need to turn it off to get the
> behaviour you want. So set org-adapt-indentation to hedline-data and
> turn off electric-indent-mode and you will get the indentation style you
> are after.

Note that electric-indent-mode means "RET indents according to the major
mode's canonical indentation rules", not "RET always indents".

Org's rules depend on org-adapt-indentation: ideally with
org-adapt-indentation set to 'headline-data, the canonical rules become
"only :DRAWERS: are indented", so RET should *not* indent text; things
should behave as Jean Louis prefers even with electric-indent-mode on.

Bastien has just committed a fix for this earlier today;
cf. <87fsz4gtjs@gmail.com>.

Re: [POLL] Setting `org-adapt-indentation' to nil by default?

[Kévin Le Gouguec]

Kévin Le Gouguec  writes:

> 'headline-data sounds like a reasonable default too, although I think it
> still has some wrinkles[2].
>
> [2] Typing "* headline RET" starts an indented line; further RETs keep
> point indented until I type in something, after which RET finally
> snaps back to column 0.  I'd expect point to always land on column 0
> when hitting RET after a headline, since "headline data"
> (e.g. :PROPERTIES:, :LOGBOOK:) will probably always be entered
> automatically via a command.

And as of 3 hours ago, this complaint is now obsolete[1][2] 

A fews more moles to whack, maybe:

- RET after an ":END:" starts an indented line,
- "* headline RET text TAB" indents "text" (and subsequent RETs are then
  indented).

Sorry for just dumping all these nits at your front door Bastien  I
should at least write test cases for them.  Will try to find the time
Sometime Later™.


[1] 
https://code.orgmode.org/bzg/org-mode/commit/c7331f859d1cc53d5c5f2c6ec58691af15f60b80
[2] 
https://code.orgmode.org/bzg/org-mode/commit/945c019176b2d23c5006377c95ce8d69b4b4de66

Re: [POLL] Setting `org-adapt-indentation' to nil by default?

[Kévin Le Gouguec]

Bastien  writes:

> Indentation is quite sensitive: what do you think of setting a new
> default value of nil for `org-adapt-indentation' in Org 9.5?

I think that makes sense.  If the controversy over the
electric-indent-mode change taught me anything, it's that a lot of
people don't expect their Org files to be hard-indented, which I fully
sympathize with (I personally soft-indent with org-indent-mode).

I don't know whether the folks that protested over the change represent
the majority of Org users, but FWIW:

- That position sounds consistent with the Org files we see in the
  org-mode repo (where .dir-locals.el sets the variable to nil).

- I expect that the resulting behaviour matches what users were used to:
  before Org 9.4, I imagine most folks would hit RET, get unindented
  text, and not think twice about it.  People who preferred indentation
  would have had to go out of their way to get it, by always hitting TAB
  or C-j (which is the "dumb, unindented newline" in other modes which
  honor electric-indent-mode); I'd be surprised if many people worked
  this way[1].

'headline-data sounds like a reasonable default too, although I think it
still has some wrinkles[2].


Thanks for following up on this, and thanks again for your stewardship.


[1] Although I'm glad to learn that the current state of affairs made at
least one user happy 

https://orgmode.org/list/s6mr6r$17cg$1...@ciao.gmane.io/t/#u

[2] Typing "* headline RET" starts an indented line; further RETs keep
point indented until I type in something, after which RET finally
snaps back to column 0.  I'd expect point to always land on column 0
when hitting RET after a headline, since "headline data"
(e.g. :PROPERTIES:, :LOGBOOK:) will probably always be entered
automatically via a command.

Re: Org 9.4.5

[Kévin Le Gouguec]

Detlef Steuer  writes:

> Would it be possible to have the changelog in the mail announcements?
> Either as an attachment or just copied verbatim?

FWIW, if you have the org-mode repo handy, you can hew out a GNU-style
changelog with *breathes in*

$ git log release_9.4.4..release_9.4.5 --format=format:'%as  %an  
<%ae>%n%n%w(0,8,8)%s%n%n%b' | sed 's/^ *$//'

See attached result.

git shortlog release_9.4.4..release_9.4.5 works quite well too,
depending on your taste (attached as well).


2021-03-28  Bastien  

lisp/org.el: Bump version header to 9.4.5


2021-03-19  Sébastien Miquel  

org.el (org-do-latex-and-related): Fix duplicate 'latex faces

* lisp/org.el (org-do-latex-and-related): Do not add a
'org-latex-and-related face beyond the fontification limit.

2021-03-24  Kyle Meyer  

ox-html: Fix org-html-format-drawer-function's docstring

* lisp/ox-html.el (org-html-format-drawer-function): Drop leftover
text from when an example used to be included.

adcebf38f (Fix errors reported by cus-test.el, 2013-11-14) dropped the
example but left the leading part.

Reported-by: Jean-Baptiste Mazon 
Ref: 
https://orgmode.org/list/0e5569e6-15a7-d4c4-0558-8b0ef96a5...@gmail.com

2021-03-24  Kyle Meyer  

manual: Grammar fix

* doc/org-manual.org (Publishing action): Fix conjugation.

2021-03-22  Greg Minshall  

Update example :publishing-function names in manual

* doc/org-manual.org (Publishing action): Fix references to
org-latex-publish-to-pdf and org-org-publish-to-org.

2021-03-21  Kyle Meyer  

ob-export: Give more informative error on unknown call reference

* lisp/ob-exp.el (org-babel-exp-process-buffer): Signal user-error
with an informative message rather than letting
org-babel-exp-do-export call fail due to an invalid INFO argument.
* testing/lisp/test-ob-exp.el (ob-exp/unknown-call-reference): Add
test.

Reported-by: Greg Minshall 
Ref: https://orgmode.org/list/628738.1616259...@apollo2.minshall.org

2021-03-20  Maxim Nikulin  

org.el: Avoid xdg-open silent failure

* lisp/org.el (org-open-file): Use 'pipe :connection-type instead of
'pty to prevent killing of background process on handler exit.
(Bug#44824)

Problem happens only in some desktop environments where configured
through `org-file-apps' or mailcap handlers launches actual viewer
(as defined in .desktop files and obtained from mimeapps.list)
in background.  E.g. xdg-open invokes "gio open" or kde-open5 for Gnome
or KDE accordingly and these handlers launches e.g. eog or okular in
background.  As soon as main process exits, temporary terminal session
created by `start-process-shell-command' is terminated.  As a result
background processes receive SIGHUP.

Previously command were executed with no buffer, so the change
does not affect "needsterminal" and "copiousoutput" mailcap features,
they are not supported as earlier.

If handler main process fails then show a message with exit reason.
Output (including error messages) is ignored as before.
Gtk application tends to report significant amount of failed asserts
hardly informative for majority of users.

TINYCHANGE

2021-03-19  Kyle Meyer  

ob-smiles.el: Fix reference to free variable

* contrib/lisp/ob-smiles.el (molecule-jump): Format string with NAME
argument rather than undefined variable `path'.

2021-03-16  Lein Matsumaru  

ob-smiles.el: Update org babel API

* contrib/lisp/ob-smiles.el (org-link): Fix from org-add-link-type to
org-link-set-parameters

TINYCHANGE

2021-03-15  Kyle Meyer  

manual: Fix org-latex-listings reference in footnote

* doc/org-manual.org (Footnotes): Refer to org-latex-listings instead
of org-export-latex-listings.

The last occurrence of org-export-latex-listings was deleted with the
contrib/oldexp/ removal in Org 8.

2021-03-14  Kyle Meyer  

test-org-clock: Avoid daylight saving time failure

* testing/lisp/test-org-clock.el (test-org-clock/clocktable/match):
Shift times away from the beginning of the day to avoid unexpected
time totals due to DST changes.

test-org-clock/clocktable/match fails today in the US because at 2:00
clocks jumped to 3:00, and the total time check uses the range
2:00-4:00.

2021-03-02  Kyle Meyer  

manual: Add publishing-function to publishing example

* doc/org-manual.org (Example: simple publishing configuration): Add
:publishing-function to org-publish-project-alist.

This appears to have been necessary since 0ccf650b4 (org-e-publish:
Remove default value for publishing function, 2012-10-08).

Re: Turning off all indentation in 9.4.4

[Kévin Le Gouguec]

Raoul Comninos  writes:

> I noticed a change in org since I updated to the latest version when it
> comes to automatic indentation. I prefer to turn this off globally,
> including for lists. I have tinkered with various settings but have had
> no luck so far.

ORG-NEWS provides these hints:

> To get the previous behaviour back, disable ~electric-indent-mode~
> explicitly:
> 
> #+begin_src emacs-lisp
> (add-hook 'org-mode-hook (lambda () (electric-indent-local-mode -1)))
> #+end_src
> 
> Alternatively, if you wish to keep =RET= as the "smart-return" key,
> but dislike Org's default indentation of sections, you may prefer to
> customize ~org-adapt-indentation~ to either =nil= or ='headline-data=.

Normally I would recommend customizing org-adapt-indentation over
disabing electric-indent-local-mode, but if you'd rather move back to
column 0 when hitting RET in a list, the hook is probably the way to go.

Re: org-fontify-done-headline t by default?

[Kévin Le Gouguec]

TRS-80  writes:

> The change was so jarring, and in total contravention of my (otherwise)
> nice looking Nord theme.  Something like that might have put me off from
> Orgmode /completely/ if I did not know how to quickly fix it.
>
> For all of above reasons (and probably some others, too!), I do not
> think this should be the default!

I don't have a strong opinion about org-fontify-done-headline being t by
default (though I disable it), but I agree that the current definition
of org-headline-done is jarring with most of the themes out there.

Maybe it would make sense to make it inherit from one of Emacs's core
faces (e.g. shadow): that would pretty much guarantee that themes would
cover it, so it would stand out less.

Re: [9.4] Fixing logbook visibility during isearch

[Kévin Le Gouguec]

Ihor Radchenko  writes:

> Kévin Le Gouguec  writes:
>
>> 1.2. stumps me: is there an isearch API I can use while in the callback
>> to know where the matches are located?
>
> I do not think that there is direct API for this, but the match should
> be accessible through match-beginning/match-end, as I can see from the
> isearch.el code.

Right, I've seen this too; I wonder if it's a hard guarantee or an
implementation detail.  I might page help-gnu-emacs about this.

Re: [9.4] Fixing logbook visibility during isearch

[Kévin Le Gouguec]

Ihor Radchenko  writes:

>  However, org-cycle-hide-drawers might be called in
> isearch-open-invisible-temporary.

This callback receives two arguments:
- the overlay which contains a match,
- whether we are un-hiding the overlay's span or hiding it back.

To get the same behaviour as Org≤9.3, IIUC we want to do the following:

1. When isearch asks us to un-hide,
1. go over all drawers within the overlay,
2. hide those that do not contain a match, by adding an
   invisible overlay.

2. When isearch asks us to hide back,
1. remove the invisible overlays we have put on these drawers.

1.1. is straightforward: overlay-start and overlay-end tell us where to
look for drawers.

1.2. stumps me: is there an isearch API I can use while in the callback
to know where the matches are located?

For 2.1, I guess we will need to cache the temporary invisible overlays
we add during step 1. in a global list; that way when it's time to
destroy them, we can simply iterate on the list?

(Sorry for being so slow  I never seem to be able to spend more than
10 minutes on this issue before having to switch to something else…)

Re: [9.4] Fixing logbook visibility during isearch

[Kévin Le Gouguec]

Ihor Radchenko  writes:

> My current plan is supporting the overlay-based approach even after
> merging the branch (by default). So, overlays should be around for a
> while and the issue with drawer visibility will be around as well,
> unless you fix it. I will probably work on this in distant future, but
> that's not the priority now.

Mmm; is the current state of your branch representative of your plan?
If I compile it and run

emacs -Q -L $yourbranch/lisp --eval '(setq org-startup-folded t)' 
$someorgfile

Then isearching does not reveal logbook drawers unless matches are found
inside, which as far as I am concerned fixes my issue with 9.4.

>> I wonder if the path of least resistance couldn't be found in
>> org-cycle-hide-drawers: right now this function just skips over drawers
>> which are covered with an invisible overlay, but maybe it should not
>> skip a drawer if the overlay starts before it (i.e. the overlay is not
>> specific to this drawer but covers a whole containing section).
>
> That would defeat the purpose why the number of overlays was reduced in
> Org 9.4. However, org-cycle-hide-drawers might be called in
> isearch-open-invisible-temporary.

Thanks for the tip.

Re: [9.4] Fixing logbook visibility during isearch

[Kévin Le Gouguec]

Ihor Radchenko  writes:

> Kévin Le Gouguec  writes:
>
>> Since the changes in Org 9.4 aimed at improving performance, is there a
>> test case somewhere in the "Mitigating the poor Emacs performance on
>> huge org files" thread that could help ensure that a tentative fix will
>> not degrade performance?
>
> The first message in the thread ;) I believe it was also used to
> benchmark the change in 9.4.

Thanks for the pointer!

I've looked at your branch for inspiration, and my takeaway is that the
isearch-open-invisible-temporary route might be too involved for a
bugfix, especially if it's going to be reverted wholesale when your
branch gets merged.  Then again, maybe I'm not smart enough to devise a
solution.

I wonder if the path of least resistance couldn't be found in
org-cycle-hide-drawers: right now this function just skips over drawers
which are covered with an invisible overlay, but maybe it should not
skip a drawer if the overlay starts before it (i.e. the overlay is not
specific to this drawer but covers a whole containing section).

Re: did behaviour of RET change again?

[Kévin Le Gouguec]

Bastien  writes:

> I see something wrong right now: RET after a headline should only try
> to indent below the beginning of the headline with org-adapt-indentation 
> is t, but not nil or headline-data.
>
> For now, when org-adapt-indentation is headline-data, RET still indents.
>
> I will see how to fix this.
>
> Also, I'm thinking of using headline-data as the new default for the
> org-adapt-indentation option.  WDYT?

I personally agree that headline-data makes more sense as a default
given the feedback we received a few weeks ago; as you noticed though
there might be a few loose ends to tie up before making the switch:

- As you said, RET after a headline indents, but the common case for
  hitting RET after a headline is probably to write text, since IME
  headline drawers are always inserted with dedicated commands; thus RET
  should not indent after a header.

- RET after a headline drawer's :END: also indents.

- RET in a list item does not indent; it's not obvious that it should,
  but FWIW (1) RET indents when org-adapt-indentation is t (2) that
  would be my preference.

Also, Greg Minshall (CC'ed) helpfully laid out how org-adapt-indentation
and electric-indent-mode play together in one neat table:

https://orgmode.org/list/2020-11-13t18-23...@devnull.karl-voit.at/t/#mec37faab85f3de59e25a7c1640e5f50be5494192

I didn't take the time to properly review his findings, but there might
be more inconsistencies lurking in there.

Finally, not a big problem if headline-data becomes the default, but:
the :safe predicate is still booleanp.  Patch attached:

>From fd8dab0c42d7104566437b51526b25979f1056fe Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?K=C3=A9vin=20Le=20Gouguec?= 
Date: Mon, 21 Dec 2020 12:09:56 +0100
Subject: [PATCH] * lisp/org.el (org-adapt-indentation): Mark 'headline-data as
 safe

---
 lisp/org.el | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/lisp/org.el b/lisp/org.el
index 1f7e434ce..f75745aba 100644
--- a/lisp/org.el
+++ b/lisp/org.el
@@ -1610,7 +1610,7 @@ time in Emacs."
 	  (const :tag "Adapt indentation for headline data lines"
 		 'headline-data)
 	  (const :tag "Do not adapt indentation at all" nil))
-  :safe #'booleanp)
+  :safe (lambda (x) (memq x '(t nil headline-data
 
 (defvaralias 'org-special-ctrl-a 'org-special-ctrl-a/e)
 
-- 
2.29.2



Eric S Fraga  writes:

> Just to say that RET seems to be working again.  No idea what happened
> or changed, mind you...  Sorry for the noise.

Glad things fell back in place somehow.  The only explanation I can
conjure for weird/transient/irreproducible behaviour is directory-local
settings, e.g. org-mode.git's .dir-locals.el sets org-adapt-indentation
to nil… but there might have been something else at work in your case 路

Re: [9.4] Fixing logbook visibility during isearch

[Kévin Le Gouguec]

Ihor Radchenko  writes:

> You will probably need to implement this from scratch (or use the
> feature/org-fold branch from github.com/yantar92/org).

Gotcha.  TBH I don't know if I'll have the time to cook up a patch
before 27.2 is released; all the same, I appreciate you taking the time
to explain all this.

Since the changes in Org 9.4 aimed at improving performance, is there a
test case somewhere in the "Mitigating the poor Emacs performance on
huge org files" thread that could help ensure that a tentative fix will
not degrade performance?

Re: [9.4] Fixing logbook visibility during isearch

[Kévin Le Gouguec]

Ihor Radchenko  writes:

> Kévin Le Gouguec  writes:
>
>> The debugger only fires *after* we exit isearch, and by that time it's
>> too late: my issue comes from all those logbooks cluttering the screen
>> while I'm mashing C-s to iterate through matches.
>>
>> I can try to dig deeper into this, but before doing so: would you have
>> any insight as to what's going on here?
>
> org-mode is relying on default isearch behaviour during interactive C-s
> session. By default, isearch simply makes all the overlays at match
> visible and re-hide them once we move to the next match. In case of
> org-mode, this reveals drawers as well, since they are in the same
> overlay with the rest of the folded heading.
>
> The way to change default isearch behaviour *during* isearch session is
> setting undocumented 'isearch-open-invisible-temporary property of the
> overlay (see isearch-open-overlay-temporary).

Thanks for taking the time to explain this.

I can't find any reference to this property in Org <9.4 (e.g. 9.3 as
shipped in 27.1, where the bug does not happen) so do I understand
correctly that the root cause ("since [drawers] are in the same overlay
with the rest of the folded heading") dates from Org 9.4?

(Just trying to understand if I should keep looking at Org 9.3 for
inspiration, or if your proposed solution based on
isearch-open-invisible-temporary should be implemented from scratch)

[9.4] Fixing logbook visibility during isearch

[Kévin Le Gouguec]

Ihor Radchenko  writes:

> However, I can try to suggest a way to fix the issue on master. The way
> isearch handles folded text in org is set from org-flag-region
> (org-macs.el):
>
> (overlay-put o
>  'isearch-open-invisible
>  (lambda ( _) (org-show-context 'isearch)))
>
> It means that isearch calls org-show-context (org.el) to reveal hidden
> text. Then, it calls org-show-set-visibility with argument defined in
> org-show-context-detail (now, it is 'lineage). With current defaults,
> the searched text is revealed using org-flag-heading, which reveals both
> heading body and drawers.
>
> The easiest way to write the fix would be changing org-flag-heading
> directly, but there might be unforeseen consequences on other folding
> commands.
>
> Another way would be changing the way org-show-set-visibility handles
> 'lineage argument. Again, it may affect other things.
>
> Finally, one can add an extra possible argument to
> org-show-set-visibility and alter default value of
> org-show-context-detail accordingly.
>
> The last way will have least risk to break something else.
>
> I guess, patches welcome ;)

Since Org 9.4 has landed in the emacs-27 branch, I have renewed interest
in finding a fix for this before 27.2 is released (… and more selfishly,
before emacs-27 is merged into master ).

I'm a bit confused, because AFAICT org-show-context is called *after*
exiting isearch, so IIUC by the time org-show-set-visibility is called
it's too late to undo the damage.

Recipe using my repro file[1]:

- C-x C-f logbooks.org
- M-x toggle-debug-on-entry org-show-context
- C-s bug

The debugger only fires *after* we exit isearch, and by that time it's
too late: my issue comes from all those logbooks cluttering the screen
while I'm mashing C-s to iterate through matches.

I can try to dig deeper into this, but before doing so: would you have
any insight as to what's going on here?


[1] wget https://orgmode.org/list/87eepuz0bj@gmail.com/2-logbooks.org -O 
tmp/logbooks.org

[PATCH][9.4] Mention org-adapt-indentation as escape hatch in ORG-NEWS

[Kévin Le Gouguec]

If it's not too late for this to make it into 27.2, I think this could
make dealing with the electric-indent-mode kerfuffle easier for
unsuspecting users.

>From 829a98c8ee4ba8b914b9ac81bae510ae6f2820aa Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?K=C3=A9vin=20Le=20Gouguec?= 
Date: Fri, 11 Dec 2020 19:47:37 +0100
Subject: [PATCH] Offer alternative to disabling electric-indent-mode

* etc/ORG-NEWS (=RET= and =C-j= now obey ~electric-indent-mode~):
Mention alternative values for org-adapt-indentation.
---
 etc/ORG-NEWS | 4 
 1 file changed, 4 insertions(+)

diff --git a/etc/ORG-NEWS b/etc/ORG-NEWS
index 0ed626fb7..b8c6e9d3b 100644
--- a/etc/ORG-NEWS
+++ b/etc/ORG-NEWS
@@ -234,6 +234,10 @@ explicitly:
 (add-hook 'org-mode-hook (lambda () (electric-indent-local-mode -1)))
 #+end_src
 
+Alternatively, if you wish to keep =RET= as the "smart-return" key,
+but dislike Org's default indentation of sections, you may prefer to
+customize ~org-adapt-indentation~ to either =nil= or ='headline-data=.
+
 *** =ob-C.el= allows the inclusion of non-system header files
 
 In C and C++ blocks, ~:includes~ arguments that do not start with a
-- 
2.29.2

Org 9.3.8 for Emacs 27.2?

[Kévin Le Gouguec]

Hi Org,

It seems the Emacs maintainers are gearing up for the release of 27.2,
the first bugfix version of Emacs 27[1][2].

27.1 ships with Org 9.3, which has seen its last bugfix release with
9.3.8.  Is there any way we[3] can help bring 9.3.8 into the emacs-27
branch in time for 27.2?

Thanks for your time.


[1] bug-gnu-em...@gnu.org <83k0ubvppt@gnu.org>
https://lists.gnu.org/archive/html/bug-gnu-emacs/2020-11/msg01630.html

[2] bug-gnu-em...@gnu.org 

https://lists.gnu.org/archive/html/bug-gnu-emacs/2020-11/msg01649.html

[3] Read: users with no commit access to any repository, and no
first-hand experience with the synchronization process.

Re: Changed list indentation behavior: how to revert?

[Kévin Le Gouguec]

Tim Cross  writes:

> I am a little confused about the purpose of org-adapt-indentation
> though. According to the org news file, to get back the old behaviour,
> it says to explicity disable electric-indent mode using org-mode-hook.
> There is no mention of org-adapt-indentation.

Yep; as I mentioned in <87k0umjn74@gmail.com>, I didn't know about
org-adapt-indentation when I wrote this NEWS entry.  If I had, I would
have
- mentioned this variable in the NEWS entry, and/or
- campaigned for a change to the default value.

Re: Changed list indentation behavior: how to revert?

[Kévin Le Gouguec]

Jean Louis  writes:

> There is just slight difference, and that is what I learned from
> introduction to Org mode that it is "plain text" kind of mode. I can
> do and write how I wish. My habit comes from being used to indent when
> I want and then to follow indentation in that specific paragraph. That
> is really great.
>
> But I was not used to have it indented by programmer like the
> introduction of this new default feature, which I consider is not
> useful to be default.

Note that even before this change, Org's indentation already behaved
like a "programming mode".  TAB does not allow you to move to "any
column":

- either org-adapt-indentation is t (the default) and TAB moves your
  paragraph to column LEVEL+1,
- or it is nil, and TAB is a no-op.

> Observe this official presentation and you will see how current
> indentation is not consistent to what is shown:
> https://orgmode.org/resources/img/features/folding.gif
>
> Look at this official presentation and you will see that even headings
> are indented for which we say it should not be so:
> https://orgmode.org/resources/img/features/clocking.svg

Yep, AFAICT this has been produced with org-indent-mode, which
soft-indents using overlays.

> The official presentation here:
> https://orgmode.org/
>
> does not show any indentation at all.
>
> And in Info file I find nothing of it.

Yep; what this (along with the way org-adapt-indentation is unset in
Org's own repo) suggests to me is that Org, by default, should not
indent section bodies.

This means *not only* that RET should not indent, but that /TAB/ should
not rigidly indent to column LEVEL+1 (I don't have a strong opinion
about whether it should rigidly indent to column 0, or if it should
behave as in text-mode).

So AFAIU the issue lies not with RET becoming consistent with the rest
of Emacs and doing "insert newline then indent smartly"; rather it lies
with how Org defines "smartly".  From what I gather from this thread,
lots of folks would like Org to keep section bodies at column 0.

> All I say, when default is introduced, should be well documented how
> and why. Before it is introduced it is better to discuss wider with
> people.
>
> Few of people reading these exchanges may find how to turn it off,
> majority will not find it.

Before being applied, this change has been discussed on emacs-devel and
emacs-orgmode; it has then been documented in ORG-NEWS.  Which other
places do you think we should have reached out to?

>> IIUC this can be toggled off by setting org-adapt-indentation to nil;
>> FWIW this is what the .dir-locals.el file at the root of Org's
>> repository doe
>
> With 2000+ directories containing Org file of persons, held on this
> system that would mean turning it on 2000+ times. Because in general I
> do not use that type of indentation I have just set it in main
> ~/.emacs.d/init.el file.
>
> We concluded that configuring is easy and that is great.
>
> What is not concluded is that the default impacts too many people who
> may not find out how to configure it back and that designing user
> interface shall be made with more care.

I admit to not having put as much thought in a "migration plan" as I
could have.  My reasoning was that since Org indents text by default
(/when/ hitting TAB or using the "smart newline" command), users were
probably fine with it.

IIUC I failed to understand that:

- Plenty of Org users do not expect it to behave like programming modes
  wrt indentation (they might not even use programming modes).

- These users were using RET as a "dumb newline" command, unaware
  that by default, Org considers that text should be indented.

- org-adapt-indentation…

- exists (really, I just found out about it today, after wondering
  why on Earth Org does not indent text in doc/org-guide.org, and
  tracing it to the repository's directory-local variables).

- has a default value that does not reflect how Org text is indented
  in official examples, nor in Org's own repository.

Re: Changed list indentation behavior: how to revert?

[Kévin Le Gouguec]

Detlef Steuer  writes:

> I'm constantly bitten by that change, but was too lazy to dig for the
> cause. But now that I know, I want to add 2c.
>
> If one writes prose it looks much more natural to have
>
> * Healine
>
> start editing in column 1 of next row.
> (Personally I would prefer to start at row 3, but independent
>  of the depth of the heading. Probably there is a setting already?)
>
> C-j is fine and nice, but I *feel* the default should be the other
> way round.
>
> I'm in no way emotional about these changes, but as Arne demonstrates
> in his example text, org files become less readable when using the new
> default. Heavy indenting is not what we are used to see if we have
> subheadings in prose. Readability of org on the screen should be very high
> in list of usability target. (Most probably it indeed is for the developers!
> I'm not assuming you would neglect it!)
> Maybe all boils down to a matter of taste, but at least imho Arne's
> example shows the problem quite clearly.
>
> For lists or sequences of mostly empty headings this does not matter
> as much.
>
> Furthermore: If I understand correctly electric-ident mode is thought to
> be a helper for programming major modes. In my opinion org is no (not
> only, much more than a) programming mode, so maybe electric ident is not
> the optimal default. 

Note that indenting section bodies by default predates Org 9.4: in Org
9.3, hitting TAB on the first line of text after a heading indents it to
column LEVEL+1.

IMHO, the default value of org-adapt-indentation might be the issue here
(made more visible by the change in 9.4): I agree that hard-indenting
prose should not be the default behaviour.  FWIW the .dir-locals.el file
at the root of Org's own repository sets this variable to nil; maybe
that suggests that it would be a better default?

(As I said in my reply to Jean Louis: I've only skimmed over this
thread; apologies if I've missed anything.)

Re: Changed list indentation behavior: how to revert?

[Kévin Le Gouguec]

Jean Louis  writes:

> Indentation in fundamental mode:
>
> ** HereRET
> I come here.
> But only if I start indenting
>Like hereRET
>Then I continue here

Hi Jean,

My understanding of electric-indent-mode is that it tries to make "RET"
equivalent to "insert newline; indent according to major mode rules".
E.g. in c-mode, when point is after the brace:

if (condition) {

RET will move point to column 2, while C-j will just insert a newline
and stay at column 0.

Likewise in python-mode, when point is after the colon:

def foobar():

RET will insert a newline and move point to column 4; C-j will stay at
column 0.

Your counter-example in fundamental-mode only shows that there are no
"smart indentation" rules in this mode; hitting TAB more than once keeps
on inserting horizontal space unconditionally instead of snapping to the
"correct" indentation level.


I've used Emacs's programming language modes for years before finally
trying out Org, where I noticed that the keys were swapped: RET was the
"plain dumb newline" key, and C-j was the "smart newline-then-indent"
key.  IIUC this was how the rest of Emacs behaved before
electric-indent-mode became enabled by default.

I personally found the difference infuriating.  Everywhere else in
Emacs,
- C-m and  do smart indentation,
- C-j ≡ ^J ≡ (insert "\N{LINE FEED (LF)}")

The changes in Org 9.4 aimed to make Org consistent with this "new"
convention, so that hitting RET immediately indents paragraphs below a
heading (as if the user hit TAB right after inserting a newline), and a
user wishing to "just insert some vertical space" can just whale on C-j.


FWIW, what I wonder about is /why/ Org hard-indents section bodies by
default (org-indent-mode, which I use, soft-indents instead using
overlays).

IIUC this can be toggled off by setting org-adapt-indentation to nil;
FWIW this is what the .dir-locals.el file at the root of Org's
repository does…


I haven't read this whole thread thoroughly; I've had trouble staying on
top of Emacs's mailing lists this week.

Apologies if I've missed something, and thanks for your patience.

Re: [patch suggestion] Mitigating the poor Emacs performance on huge org files: Do not use overlays for PROPERTY and LOGBOOK drawers

[Kévin Le Gouguec]

Ihor Radchenko  writes:

> Thanks for reporting! I accidentally reintroduced the bug because of
> mistake when converting org-hide-drawers to new folding library.
> (:facepalm:).
>
> Should be fixed in the gist now.

Can confirm, thanks!

I understand from your answer to Bastien's query that this fix is
specific to your branch; would it be hard to backport it to Org's maint
branch?  Otherwise IIUC Org 9.4 will keep this regression, and users
will have to wait until Org 9.5 for a fix.

Also, just in case there's been a misunderstanding:

Bastien  writes:

> Can you share this gist as a patch against Org's current master?

Bastien asked for the /gist/ as a patch against master, whereas your
answer explained why you couldn't share the /fix/ as a patch against
master.  If Bastien did mean the whole gist, here is the corresponding
patch against master:

https://gist.githubusercontent.com/yantar92/6447754415457927293acda43a7fcaef/raw/7e43948e6c21220661534b79770bc1a6784b7893/featuredrawertextprop.patch

Apologies if I'm the one misunderstanding, and thank you for all your
efforts!

Re: [patch suggestion] Mitigating the poor Emacs performance on huge org files: Do not use overlays for PROPERTY and LOGBOOK drawers

[Kévin Le Gouguec]

Ihor Radchenko  writes:

>>> then M-x toggle-debug-on-error and M-: (org-make-manuals), but I can't
>>> get a stacktrace.  I'm guessing this is because this error (which IIUC
>>> originates from org-back-to-heading in org.el) is a user-error; however,
>>> if I change the function to raise a "regular error", then everything
>>> compiles fine… 
>>
>> I suspect that you forgot to run =make clean= (to remove old untracked
>> .elc files).
>
> I was wrong. It was actually a problem with org-back-to-heading. Should
> be fixed now.

Thanks!  The new patch applies cleanly (to aea1109ef), and "make" runs
to completion.

I have seen no obvious breakage so far; I'll make sure to report if
anything funny shows up.


Apologies for maybe changing the subject, but earlier this summer you
mentioned[1] you were working on a patch to the folding system that
would fix an issue I have[2] with LOGBOOKs since 9.4.  AFAICT the patch
you are sharing now does not fix that; is this issue still on your
radar?


At any rate, thank you for your work!


[1] https://orgmode.org/list/87r1ts3s8r.fsf@localhost/
[2] https://orgmode.org/list/87eepuz0bj@gmail.com/

tl;dr even with #+STARTUP: overview, isearching opens all logbooks
near search results, even though there are no matches inside
logbooks themselves.

Re: Default description for abbreviated links

[Kévin Le Gouguec]

Hello Bastien!  Thank you for following up on this.

Bastien  writes:

> Kévin Le Gouguec  writes:
>
>> I like #+LINK keywords because they make documents self-sufficient:
>> anyone opening my document can follow these links or export the buffer;
>> they do not need to run some Elisp to add to org-link-parameters.
>>
>> One thing I don't know how to customize, however, is how these links are
>> exported when they have no description.  
>
> thanks for sharing your need and ideas.
>
> I think we can allow
>
>   #+LINK: bug [[https://debbugs.gnu.org/%s][bug:%s]]
>
> to define an abbreviated link producing the output you want.
>
> Same in org-link-abbrev-alist(-local):
>
>   (("bug" . "[[https://debbugs.gnu.org/%s][bug:%s]];))
>
> What do you think?  I'd rather not add an option or modify the
> structure of org-link-abbrev-alist(-local).

That's an interesting idea!  It sounds fairly more powerful than what I
had in mind (only allowing KEY:TAG or TAG), but I'm sure someone could
find some use for free-form formatting.


I'm not sure how to implement it though.  I just came back from a hike
through ox-html.el, ox.el, org-element.el and ol.el; going backward,
here is how descriptions are given to export backends:

(1) Link-transcoding functions (e.g. org-html-link) are given two
arguments: a link object, and its description.

(2) The description argument is set in org-export-data, from whatever
org-element-contents returns.

(3) This (IIUC) is defined by what org-element--parse-objects `push'es
on the link object, which is determined by a recursive call to
org-element--parse-objects from the link's :contents-begin property
to its :contents-end property.

(4) org-element-link-parser sets these properties to the bounds of
org-link-bracket-re's optional second group, if it exists.

AFAICT steps (2) and (3) are not specific to links; they are generic
steps that are independent of what kind of elements they are processing.
I don't think this is where the "description fallback" feature should be
implemented, since it would add special-casing just for links.

I'm guessing I should keep aiming at step (4), like in my first patch;
my problem is that this step just defines the link's :contents-begin and
:contents-end properties.  My first patch works because I'm only
allowing KEY:TAG and TAG, so I can re-use positions inside
org-link-bracket-re's first group.

If we are to create a completely new format string, how can we pass it
to the backends?


FWIW, I would still favor only allowing KEY:TAG and TAG, since that
covers all use-cases I've thought of so far (the one you quoted, and the
"doc:" links in ORG-NEWS).

I know you said you'd rather not modifying the structure of
org-link-abbrev-alist, but maybe adding an optional third item would be
the least intrusive way to go?  I.e. going from:

("linkkey" . REPLACE)

To:

("linkkey" REPLACE [DESCRIPTION-FALLBACK])

Where DESCRIPTION-FALLBACK could be nil (the default), 'key-tag or 'tag.


Thank you for your time.

Re: [patch suggestion] Mitigating the poor Emacs performance on huge org files: Do not use overlays for PROPERTY and LOGBOOK drawers

[Kévin Le Gouguec]

Hi!

Ihor Radchenko  writes:

> The current version of the patch (against master) is in
> https://gist.github.com/yantar92/6447754415457927293acda43a7fcaef

I'm probably missing something obvious, but when applying your patch on
top of master[1], make fails when generating manuals:

> emacs  -Q -batch --eval '(setq vc-handled-backends nil org-startup-folded 
> nil)' \
>   --eval '(add-to-list '"'"'load-path "../lisp")' \
>   --eval '(load "../mk/org-fixup.el")' \
>   --eval '(org-make-manuals)'
> Loading /home/peniblec/Downloads/sources/emacs-meta/org-mode/mk/org-fixup.el 
> (source)...
> Before first headline at position 760959 in buffer org-manual.org<2>
> make[1]: *** [Makefile:31: org.texi] Error 255

I've tried going to doc/, running 

emacs -Q --eval '(setq vc-handled-backends nil org-startup-folded nil)' \
  --eval '(add-to-list '"'"'load-path "../lisp")' \
  --eval '(load "../mk/org-fixup.el")'

then M-x toggle-debug-on-error and M-: (org-make-manuals), but I can't
get a stacktrace.  I'm guessing this is because this error (which IIUC
originates from org-back-to-heading in org.el) is a user-error; however,
if I change the function to raise a "regular error", then everything
compiles fine… 


[1] git apply --3way, on top of commit b64ba64fe.

I get a conflict in org.el, on the hunk where org-reveal-location
and org-show-context-detail are defined; since your patch just
deletes them, I resolve this with:

git checkout --theirs -- lisp/org.el

Check for master branch from Elisp (was: Release 9.3.8)

[Kévin Le Gouguec]

Kyle Meyer  writes:

> Can't you inspect the return value of org-git-version?

That can work out, though unless I'm missing something, I need to move
to the org-mode repository, ask "git branch --contains", and parse the
output.  Possible, but somewhat involved.

(TIL: git-describe's "{tag}-{nbcommits}-g{hash}" is actually a valid
revision format that other Git commands understand.)


For the sake of completeness, I've tried visiting org.el and evaluating

(package-desc-version (package-buffer-info))

but package-buffer-info ends up calling (version-to-list "9.4-dev"),
which chokes on "-dev".  FWIW, that can be worked around with:

(add-to-list 'version-regexp-alist (cons "^-dev$" -1))

> (Though in my
> view, distinguishing based on the functionality present with things like
> fboundp, which you mention below, is typically a better approach, if
> possible.)

Right, that's my conclusion as well :)

Re: Release 9.3.8

[Kévin Le Gouguec]

Bastien  writes:

>> - Will Emacs's maintenance branch (emacs-27) be updated with Org 9.3.8,
>>   so that Emacs 27.2 includes all bugfixes for 9.3?  (If so, I can open
>>   a new report on Debbugs to track this, as suggested by Stefan K.)
>
> Yes, thanks.

ACK; see bug#43268!

>> - During the development of 9.4, AFAICT, while the "Version:" comment in
>>   org.el sayd "9.4-dev", the org-version variable matched the latest
>>   tag, i.e. 9.3.x.
>>
>>   I therefore couldn't figure out a way to check for 9.4
>>   programmatically.  
>
> ... because 9.4 is not yet released - or am I missing something?

See Emacs's master branch for a counter-example: even though 28.1 is not
out yet, emacs-version says "28.0.50", so one can determine that they're
running on the master branch.

It's clearly not a big deal; cf. below.

> On what commit would I add the "release_x.(y+1)-rc" on master, since
> master is always moving forward?

If a new major release is immediately merged to the maint branch, it
would be enough to have a followup empty commit on master, and tag that.

I'm not suggesting to do that though; I don't find empty commits very
elegant.  IIUC, for the Emacs repository, the source of truth is not the
latest tag, but configure.ac's AC_INIT clause, so it takes a (decidedly
non-empty) bump-commit to increase the version.  See e.g. 64fe67beff.

> I would like to keep things simple here: let's have annotated tags for
> releases and... master.
>
> Let me know if I miss a very obvious use-case for a better setup.

That's fair.  My "use-case" was to conditionally swap RET and C-j for
Org<9.4, to palliate the lack of electric-indent-mode.  It's far from a
critical problem, and there are other ways for me to solve this (rely on
fboundp, run "make ORGVERSION=9.4"…).

Re: Release 9.3.8

[Kévin Le Gouguec]

Bastien  writes:

> Hi all,
>
> I'm releasing Org 9.3.8, a bugfix release.
>
> Enjoy!

Thanks for this release 

> The next release will be 9.4: if you have outstanding important bugs
> you would like to point to or to report, please go ahead.

No bug to report, however I've got a couple of questions wrt. versions:

- Once 9.4 is released, will the maint branch be updated to this
  version?

- Will Emacs's maintenance branch (emacs-27) be updated with Org 9.3.8,
  so that Emacs 27.2 includes all bugfixes for 9.3?  (If so, I can open
  a new report on Debbugs to track this, as suggested by Stefan K.)

- During the development of 9.4, AFAICT, while the "Version:" comment in
  org.el sayd "9.4-dev", the org-version variable matched the latest
  tag, i.e. 9.3.x.

  I therefore couldn't figure out a way to check for 9.4
  programmatically.  Would it make sense to:

- set a "release_x.(y+1)-rc" tag on master once version "x.y" is
  released and goes in the maint branch,

- in targets.mk, when computing ORGVERSION, add "--first-parent" to
  "git describe", so that org-version matches this rc-tag when
  compiling Org's master branch?

At any rate, thanks for all the work.

Re: [b...@gnu.org: Re: bug#42484: 26.1: org-mode should display value of links in mini-buffer]

[Kévin Le Gouguec]

> Boruch Baum  writes:
>
>> In org-mode, when POINT is moved over an org-mode link, wouldn't it be
>> reasonable for the value of that link to appear in the mini-buffer? The
>> advantage of that is the user would know where the link points and what
>> would happen if the link is opened (eg. would an external program open,
>> would the network be queried).

That would be very welcome, IMO.  FWIW, markdown-mode does that (when
markup is hidden) using ElDoc; cf. markdown-eldoc-function.

Re: Bug#42184: org-fontify-whole-*-line in emacs 27

[Kévin Le Gouguec]

Kyle Meyer  writes:

>   Okay, let's go with
>
>   `((t :inherit shadow ,@(and (>= emacs-major-version 27) '(:extend t
>
> as the org-block spec then.

Done; patch attached.

>From 41ab9f8f0c2f02f1951a412265b01a9ac5fa5a96 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?K=C3=A9vin=20Le=20Gouguec?= 
Date: Fri, 7 Aug 2020 11:04:53 +0200
Subject: [PATCH] Fix org-fontify-whole-*-line by setting face extension
 (bug#42184)
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

* lisp/org-faces.el (org-block): Set background extension beyond
end-of-line.
* lisp/org-compat.el (org--set-faces-extend): New function to
temporarily (re)set :extend for Emacs≥27.
* lisp/org.el (org-mode): Call it to set the extend attribute of
relevant faces to the correct value.
---
 lisp/org-compat.el | 8 
 lisp/org-faces.el  | 3 ++-
 lisp/org.el| 6 +-
 3 files changed, 15 insertions(+), 2 deletions(-)

diff --git a/lisp/org-compat.el b/lisp/org-compat.el
index c757355ba..e405df0b5 100644
--- a/lisp/org-compat.el
+++ b/lisp/org-compat.el
@@ -101,6 +101,14 @@ is nil)."
   (defun org-time-convert-to-list (time)
 (seconds-to-time (float-time time
 
+(defun org--set-faces-extend (faces extend-p)
+  "Set the :extend attribute of FACES to EXTEND-P.
+
+This is a no-op for Emacs versions lower than 27, since face
+extension beyond end of line was not controllable."
+  (when (fboundp 'set-face-extend)
+(mapc (lambda (f) (set-face-extend f extend-p)) faces)))
+

 ;;; Emacs < 26.1 compatibility
 
diff --git a/lisp/org-faces.el b/lisp/org-faces.el
index 30eab9bc6..ff2b0e611 100644
--- a/lisp/org-faces.el
+++ b/lisp/org-faces.el
@@ -393,7 +393,8 @@ follows a #+DATE:, #+AUTHOR: or #+EMAIL: keyword."
   "Face for #+TITLE:, #+AUTHOR:, #+EMAIL: and #+DATE: keywords."
   :group 'org-faces)
 
-(defface org-block '((t :inherit shadow))
+(defface org-block `((t :inherit shadow
+			,@(and (>= emacs-major-version 27) '(:extend t
   "Face text in #+begin ... #+end blocks.
 For source-blocks `org-src-block-faces' takes precedence."
   :group 'org-faces
diff --git a/lisp/org.el b/lisp/org.el
index 007dd6e2a..34c0235c1 100644
--- a/lisp/org.el
+++ b/lisp/org.el
@@ -4940,7 +4940,11 @@ The following commands are available:
   ;; Try to set `org-hide' face correctly.
   (let ((foreground (org-find-invisible-foreground)))
 (when foreground
-  (set-face-foreground 'org-hide foreground
+  (set-face-foreground 'org-hide foreground)))
+  ;; Set face extension as requested.
+  (org--set-faces-extend '(org-block-begin-line org-block-end-line)
+ org-fontify-whole-block-delimiter-line)
+  (org--set-faces-extend org-level-faces org-fontify-whole-heading-line))
 
 ;; Update `customize-package-emacs-version-alist'
 (add-to-list 'customize-package-emacs-version-alist
-- 
2.28.0


> Thanks.

Again, thanks for the review!

Re: Bug#42184: org-fontify-whole-*-line in emacs 27

[Kévin Le Gouguec]

Kyle Meyer  writes:

>> +(defmacro org--extended-face (attributes)
>> +  "Make face that extends beyond end of line.
>> +
>> +Up to Emacs 26, all faces extended beyond end of line; getting
>> +the same behaviour starting with Emacs 27 requires :extend t."
>> +  `(nconc ,attributes (when (>= emacs-major-version 27) '(:extend t
>
> Two minor thoughts, neither really important:
>
>   * Style nit-pick: s/when/and/, as the return value is of interest.

OK; I didn't know 'when' had a "for side-effect only" connotation, I was
using it as a shorthand for (if COND FORM nil).

> ... the main thing I wonder is whether this kludge is necessary at all.
> AFAICT unconditionally including :extend in the face spec doesn't seem
> to bother earlier Emacs versions.

Huh.  Based on the discussion for bug#37774[1][2][3][4], I had assumed
this kind of kludge would be necessary, but both Emacs 25.3 and 26.3
seem to evaluate and byte-compile the following snippet with no errors:

#+begin_src elisp
(defface foobar '((t (:extend t)))
  "Test extend in 26.3"
  :group 'org-faces)

(custom-set-faces
 '(foobar ((t (:extend nil))) t))
#+end_src

Obviously I'm all for removing this shim if it's not needed.  From some
light testing it looks like removing it breaks the customization widget,
though?

I'll post a revised patch as soon as someone can confirm or refute my
observations.

[1] https://debbugs.gnu.org/cgi/bugreport.cgi?bug=37774#41
[2] https://debbugs.gnu.org/cgi/bugreport.cgi?bug=37774#53
[3] https://debbugs.gnu.org/cgi/bugreport.cgi?bug=37774#71
[4] https://debbugs.gnu.org/cgi/bugreport.cgi?bug=37774#80

>Even if there is a reason to avoid
> :extend on older versions, it's perhaps an overkill to add a
> compatibility macro for just one spot.

Right, that macro dates from an earlier patch, where I unconditionally
set :extend t on a bunch of faces[5].  I agree that it's overkill for
just org-block.

[5] https://debbugs.gnu.org/cgi/bugreport.cgi?bug=42184#17

> If org--extended-face stays, org-face.el should be adjusted to load
> org-compat.el.  (`make single' flags this.)

(Ugh, I actually got that right in earlier patches.)


Thanks for the review.  As I said, I'll post an updated patch as soon as
someone confirms or refutes my impression that :extend messes with the
Customize widget for Emacs <27.

Re: Bug#42184: org-fontify-whole-*-line in emacs 27

[Kévin Le Gouguec]

Kyle Meyer  writes:

> Kévin Le Gouguec writes:
>
>> Since 27.1-rc1 is out, I'd like to bump this; it'd be a shame if 27.1
>> shipped with this bug, which seems to be getting some attention (I just
>> spotted a Reddit thread[1] about it, in addition to the original report
>> on Debbugs).
>
> In the associated emacs-bug thread, Eli said that ship has sailed.  With
> the possibility of making it into 27.1 out of the picture, I think this
> patch should be made against the Org repo, as usual.
>
> As I said in the emacs-bug thread, the patch (which you also included
> upstream in this thread) looks fine to me.  If you'd prefer to bundle
> your org-block change in the same patch, could you send an updated patch
> here against the Org repo?

I've just noticed that my patch for org-block (bug#42184#44) was missing
a compatibility shim for Emacs<27; here are updated patches against
maint and master:

>From 6ca21259589e212452411acd4ac2de3258ede508 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?K=C3=A9vin=20Le=20Gouguec?= 
Date: Fri, 7 Aug 2020 11:04:53 +0200
Subject: [PATCH] Fix org-fontify-whole-*-line by setting face extension
 (bug#42184)
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

* lisp/org-faces.el (org-block): Set background extension beyond
end-of-line.
* lisp/org-compat.el (org--extended-face): New function to add :extend
attribute to face definition for Emacs≥27.
(org--set-faces-extend): New function to temporarily (re)set :extend
for Emacs≥27.
* lisp/org.el (org-mode): Call it to set the extend attribute of
relevant faces to the correct value.
---
 lisp/org-compat.el | 15 +++
 lisp/org-faces.el  |  2 +-
 lisp/org.el|  6 +-
 3 files changed, 21 insertions(+), 2 deletions(-)

diff --git a/lisp/org-compat.el b/lisp/org-compat.el
index c757355ba..c0f4833a3 100644
--- a/lisp/org-compat.el
+++ b/lisp/org-compat.el
@@ -101,6 +101,21 @@ is nil)."
   (defun org-time-convert-to-list (time)
 (seconds-to-time (float-time time
 
+(defmacro org--extended-face (attributes)
+  "Make face that extends beyond end of line.
+
+Up to Emacs 26, all faces extended beyond end of line; getting
+the same behaviour starting with Emacs 27 requires :extend t."
+  `(nconc ,attributes (when (>= emacs-major-version 27) '(:extend t
+
+(defun org--set-faces-extend (faces extend-p)
+  "Set the :extend attribute of FACES to EXTEND-P.
+
+This is a no-op for Emacs versions lower than 27, since face
+extension beyond end of line was not controllable."
+  (when (fboundp 'set-face-extend)
+(mapc (lambda (f) (set-face-extend f extend-p)) faces)))
+

 ;;; Emacs < 26.1 compatibility
 
diff --git a/lisp/org-faces.el b/lisp/org-faces.el
index 30eab9bc6..4c5a51624 100644
--- a/lisp/org-faces.el
+++ b/lisp/org-faces.el
@@ -393,7 +393,7 @@ follows a #+DATE:, #+AUTHOR: or #+EMAIL: keyword."
   "Face for #+TITLE:, #+AUTHOR:, #+EMAIL: and #+DATE: keywords."
   :group 'org-faces)
 
-(defface org-block '((t :inherit shadow))
+(defface org-block `((t ,(org--extended-face '(:inherit shadow
   "Face text in #+begin ... #+end blocks.
 For source-blocks `org-src-block-faces' takes precedence."
   :group 'org-faces
diff --git a/lisp/org.el b/lisp/org.el
index 007dd6e2a..34c0235c1 100644
--- a/lisp/org.el
+++ b/lisp/org.el
@@ -4940,7 +4940,11 @@ The following commands are available:
   ;; Try to set `org-hide' face correctly.
   (let ((foreground (org-find-invisible-foreground)))
 (when foreground
-  (set-face-foreground 'org-hide foreground
+  (set-face-foreground 'org-hide foreground)))
+  ;; Set face extension as requested.
+  (org--set-faces-extend '(org-block-begin-line org-block-end-line)
+ org-fontify-whole-block-delimiter-line)
+  (org--set-faces-extend org-level-faces org-fontify-whole-heading-line))
 
 ;; Update `customize-package-emacs-version-alist'
 (add-to-list 'customize-package-emacs-version-alist
-- 
2.28.0

>From 715927aa2f1baae32040e52d2cfca4aef2edc14b Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?K=C3=A9vin=20Le=20Gouguec?= 
Date: Fri, 7 Aug 2020 11:04:53 +0200
Subject: [PATCH] Fix org-fontify-whole-*-line by setting face extension
 (bug#42184)
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

* lisp/org-faces.el (org-block): Set background extension beyond
end-of-line.
* lisp/org-compat.el (org--extended-face): New function to add :extend
attribute to face definition for Emacs≥27.
(org--set-faces-extend): New function to temporarily (re)set :extend
for Emacs≥27.
* lisp/org.el (org-mode): Call it to set the extend attribute of
relevant faces to the correct value.
---
 lisp/org-compat.el | 15 +++
 lisp/org-faces.el  |  2 +-
 lisp/org.el|  6 +-
 3 files changed, 21 insertions(+), 2 deletions(-)

diff --git a/lisp/org-compat.el b/lisp/org

Re: Bug#42184: org-fontify-whole-*-line in emacs 27

[Kévin Le Gouguec]

Since 27.1-rc1 is out, I'd like to bump this; it'd be a shame if 27.1
shipped with this bug, which seems to be getting some attention (I just
spotted a Reddit thread[1] about it, in addition to the original report
on Debbugs).

[1]: 
https://old.reddit.com/r/emacs/comments/i26n46/why_does_my_orgmode_look_different_to_leuven/


Kévin Le Gouguec  writes:

> Could Org maintainers weigh in on bug#42184?
>
> To recap my understanding of the issue:
>
> - org-fontify-whole-heading-line controls whether
>   org-set-font-lock-defaults includes the final newline in the headline
>   regexp (resp. org-fontify-whole-block-delimiter-line with begin/end
>   block regexps).
>
> - This assumes that fontifying the final newline is enough to fontify
>   everything beyond this newline.
>
> - This assumption is no longer valid with Emacs 27, where this extension
>   is opt-in, using the :extend face attribute.
>
> With Eli's help, I proposed a patch for org-mode against the emacs-27
> branch that does something similar to what is done for the org-hide
> face: when setting up the major mode, depending on those user options,
> the :extend attribute is (re)set for the relevant faces (using a
> compatibility function defined in org-compat).
>
> I've reattached the patch for convenience.  Does it look sound?
>
> From 07d123c548051eb7f6bbac5c7f5a4e4b8411f976 Mon Sep 17 00:00:00 2001
> From: =?UTF-8?q?K=C3=A9vin=20Le=20Gouguec?= 
> Date: Thu, 9 Jul 2020 16:02:49 +0200
> Subject: [PATCH] Fix org-fontify-whole-*-line by setting face extension
>  (bug#42184)
>
> * lisp/org/org-compat.el (org--set-faces-extend): New function to set
> face extension, for Emacs versions where this attribute exists.
> * lisp/org/org.el (org-mode): Call it to set the extend attribute of
> relevant faces to the correct value.
> ---
>  lisp/org/org-compat.el | 4 
>  lisp/org/org.el| 6 +-
>  2 files changed, 9 insertions(+), 1 deletion(-)
>
> diff --git a/lisp/org/org-compat.el b/lisp/org/org-compat.el
> index c1aaf17ca2..fcc325e475 100644
> --- a/lisp/org/org-compat.el
> +++ b/lisp/org/org-compat.el
> @@ -101,6 +101,10 @@ org-table1-hline-regexp
>(defun org-time-convert-to-list (time)
>  (seconds-to-time (float-time time
>  
> +(defun org--set-faces-extend (faces extend-p)
> +  (when (fboundp 'set-face-extend)
> +(mapc (lambda (f) (set-face-extend f extend-p)) faces)))
> +
>  
>  ;;; Emacs < 26.1 compatibility
>  
> diff --git a/lisp/org/org.el b/lisp/org/org.el
> index 568f5b9b87..fb31336ea4 100644
> --- a/lisp/org/org.el
> +++ b/lisp/org/org.el
> @@ -4944,7 +4944,11 @@ org-mode
>;; Try to set `org-hide' face correctly.
>(let ((foreground (org-find-invisible-foreground)))
>  (when foreground
> -  (set-face-foreground 'org-hide foreground
> +  (set-face-foreground 'org-hide foreground)))
> +  ;; Set face extension as requested.
> +  (org--set-faces-extend '(org-block-begin-line org-block-end-line)
> + org-fontify-whole-block-delimiter-line)
> +  (org--set-faces-extend org-level-faces org-fontify-whole-heading-line))
>  
>  ;; Update `customize-package-emacs-version-alist'
>  (add-to-list 'customize-package-emacs-version-alist

[PATCH] Fix recommendation in ORG-NEWS (was: Binding RET to org-return-and-maybe-indent)

[Kévin Le Gouguec]

Jarmo Hurri  writes:

> Kévin Le Gouguec  writes:
>
>> Can you tell me whether electric-indent-local-mode works better for
>> you?  If it does, I'll followup with a patch to ORG-NEWS.
>
> Seems to be working fine. Thank you very much.

Thanks for the confirmation.

Here is a patch for ORG-NEWS, then:

>From e5ed2be19d7ada3a0b6dd16fc220c4414b2af4e6 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?K=C3=A9vin=20Le=20Gouguec?= 
Date: Mon, 27 Jul 2020 15:00:03 +0200
Subject: [PATCH] Fix recommendation in 9.4 release notes

Cf. <https://orgmode.org/list/87pn8huuq2@iki.fi/t/#m4f86f6baf790e88ab905007757487a1f481cc579>.

Reported-by: Jarmo Hurri 

* etc/ORG-NEWS (=RET= and =C-j= now obey ~electric-indent-mode~):
Recommend disabling electric-indent-local-mode rather than
electric-indent-mode, as the latter impacts all buffers rather than
just the newly-created Org buffer.
---
 etc/ORG-NEWS | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/etc/ORG-NEWS b/etc/ORG-NEWS
index bc93f8e4f..1ac7486a7 100644
--- a/etc/ORG-NEWS
+++ b/etc/ORG-NEWS
@@ -253,7 +253,7 @@ To get the previous behaviour back, disable ~electric-indent-mode~
 explicitly:
 
 #+begin_src emacs-lisp
-(add-hook 'org-mode-hook (lambda () (electric-indent-mode -1)))
+(add-hook 'org-mode-hook (lambda () (electric-indent-local-mode -1)))
 #+end_src
 
 *** New optional numeric argument for ~org-return~
-- 
2.27.0

Re: Binding RET to org-return-and-maybe-indent

[Kévin Le Gouguec]

Jarmo Hurri  writes:

> * Demo of the effect of disabling elint
>   1. Save this org into file =org-elint-disable.org=
>   2. Save the following elisp into =minimal-org.el=, replacing the
>  location of org mode with your path:
>
>  #+begin_src elisp
>(add-to-list 'load-path (expand-file-name "~/src/org-mode/lisp"))
>(add-to-list 'load-path (expand-file-name 
> "~/src/org-mode/contrib/lisp" t))
>(add-hook 'org-mode-hook (lambda () (electric-indent-mode -1)))
>  #+end_src
>
>   3. Toggle the last line
>
>  #+begin_src elisp
>  (add-hook 'org-mode-hook (lambda () (electric-indent-mode -1)))
>  #+end_src
>
>  in =minimal-org.el= to see the following effect:
>  1. Open this file with
>
> #+begin_src sh
> emacs -Q -l minimal-org.el org-elint-disable.org
> #+end_src
>
>  2. Type C-c ' for (org-edit-special) in the source code block below,
>   and follow the instructions on the comment line.
>
>   #+begin_src java :exports none :classname Demo
> class Demo
> {
> // 1st press RET at the end of this line, then type TAB and }
>   #+end_src

OK, here are my observations:

* Emacs 28, Org 9.3
  - RET: indented
  - TAB: nothing
  - }: de-indents
* Emacs 28, Org master, electric-indent-mode on
  - RET: indented
  - TAB: nothing
  - }: de-indents
* Emacs 28, Org master, electric-indent-mode off
  - RET: not indented
  - TAB: indents
  - }: does not indent

I think this is just because disabling electric-indent-mode is the wrong
thing to do: it should be electric-indent-local-mode.  The former
changes the default value of electric-indent-mode for *all buffers*,
whereas the intent is to only disable it in Org buffers; we don't want
to affect Org Src buffers…

If I replace (electric-indent-mode -1) with (electric-indent-local-mode
-1) in org-mode-hook, I get the behaviour we have with "Org 9.3" and
"Org master, electric-indent-mode on".

Can you tell me whether electric-indent-local-mode works better for you?
If it does, I'll followup with a patch to ORG-NEWS.

Re: Binding RET to org-return-and-maybe-indent

[Kévin Le Gouguec]

Jarmo Hurri  writes:

>> #+begin_src emacs-lisp
>> (add-hook 'org-mode-hook (lambda () (electric-indent-mode -1)))
>> #+end_src
>
> Unfortunately this has side effects: it changes at least the way
> parentheses and indentation interact when opening a Babel source code
> block. It might be a good idea to mention this in ORG-NEWS.

Could you give us a precise recipe?  (Starting from emacs -Q and an
empty Org buffer)

I've fiddled a bit with source blocks just now and I'm noticing some
weirdness that I suspect might be due to electric-indent-mode
re-indenting the previous line when hitting RET (or C-j when disabling
electric-indent-mode), but nothing specific to parentheses.

Since I'm a bit pressed for time ATM it would help if you could give a
step-by-step explanation of what goes wrong.

Re: Binding RET to org-return-and-maybe-indent

[Kévin Le Gouguec]

Jarmo Hurri  writes:

> Is there any downside to binding RET to org-return-and-maybe-indent?
>
> I want to remove RET indentation in org mode. For example
>
> # ---
> * Demo of indentation
>   - when I press return at the end of the word THIS
>   - I get indentation
> # ---

RET indentation is something that has been introduced recently on the
master branch (which will become Org 9.4 soon).  In Org 9.3, with your
example, RET does not indent, while C-j does.

As ORG-NEWS notes, if you want RET to stop indenting, you can disable
electric-indent-mode in org-mode-hook:

#+begin_src emacs-lisp
(add-hook 'org-mode-hook (lambda () (electric-indent-mode -1)))
#+end_src

> However, if I call org-return-and-maybe-indent at the same point, I do
> not get indentation.
>
> But would changing the binding of RET cause issues elsewhere?

I can't think of any bad side-effect, but my imagination might be
lacking.  The only downside I can think of is that RET will become
redundant with C-j.

Default description for abbreviated links

[Kévin Le Gouguec]

Hello Org,

I like #+LINK keywords because they make documents self-sufficient:
anyone opening my document can follow these links or export the buffer;
they do not need to run some Elisp to add to org-link-parameters.

One thing I don't know how to customize, however, is how these links are
exported when they have no description.  Let's say I define this
abbreviation:

#+LINK: bug https://debbugs.gnu.org/

If I jot down references to [[bug:12345]] in my document, these will be
exported as:

https://debbugs.gnu.org/12345;>https://debbugs.gnu.org/12345

Whereas I'd prefer:

https://debbugs.gnu.org/12345;>bug:12345

Looking at org-element-link-parser, I see that this is because the
:contents-begin and :contents-end properties are nil, since they
correspond to an unmatched optional group in org-link-bracket-re.


I could probably customize org-link-parameters, but then my document
would not be self-sufficient anymore.  Besides, depending on the
document I might use the same abbreviation for different URLs.

Would it make sense to add a way for abbreviated links with no
description to fallback to LINKKEY:TAG[1] instead of the full URL?  If
so, what would be the best way to go about it?

(1) A single variable (e.g. org-link-abbrev-default-description), default
nil, which a user could set to 'key-tag or just 'tag.

(2) A third entry in org-link-abbrev-alist(-local), default nil, which a
user could set to 'key-tag or just 'tag.

(3) Something else (e.g. a new alist).

I've attached a very crude patch for (1): now if I stick this footer in
my Org files:

#+LINK: bug https://debbugs.gnu.org/
# Local variables:
# org-link-abbrev-default-description: key-tag
# end:

Then [[bug:12345]] is exported as
https://debbugs.gnu.org/12345;>bug:12345.


WDYT?  If the idea is sound, I will clean up the patch, clarify
docstrings, add :safe predicates and unit tests, and re-submit.

Thank you for your time.


diff --git a/lisp/ol.el b/lisp/ol.el
index 82fc69769..fa0ade8b8 100644
--- a/lisp/ol.el
+++ b/lisp/ol.el
@@ -255,6 +255,16 @@ See the manual for examples."
 	(`(,(pred stringp) . ,(pred stringp)) t)
 	(_ nil
 
+(defcustom org-link-abbrev-default-description nil
+  "Fallback behaviour for abbreviated links with no description.
+If this is nil, do not set a description; some export backends
+will use the fully expanded link as fallback.  If 'key-tag, then
+use the abbreviated form KEY:TAG.  If 'tag, then use TAG."
+  :group 'org-link
+  :type '(choice (const :tag "None" nil)
+		 (const :tag "KEY:TAG" key-tag)
+		 (const :tag "TAG" tag)))
+
 (defgroup org-link-follow nil
   "Options concerning following links in Org mode."
   :tag "Org Follow Link"
@@ -993,6 +1003,16 @@ and then used in capture templates."
 	   if store-func
 	   collect store-func))
 
+(defun org-link-abbrev-default-desc (link)
+  (save-match-data
+(when (string-match "^\\([^:]*\\)::?\\(.+\\)$" link)
+  (pcase org-link-abbrev-default-description
+	('nil '(nil nil))
+	('key-tag
+	 (list (match-beginning 1) (match-end 2)))
+	('tag
+	 (list (match-beginning 2) (match-end 2)))
+
 (defun org-link-expand-abbrev (link)
   "Replace link abbreviations in LINK string.
 Abbreviations are defined in `org-link-abbrev-alist'."
diff --git a/lisp/org-element.el b/lisp/org-element.el
index a693cb68d..a82fce52a 100644
--- a/lisp/org-element.el
+++ b/lisp/org-element.el
@@ -3146,11 +3146,19 @@ Assume point is at the beginning of the link."
 	;; (e.g., insert [[shell:ls%20*.org]] instead of
 	;; [[shell:ls *.org]], which defeats Org's focus on
 	;; simplicity.
-	(setq raw-link (org-link-expand-abbrev
-			(org-link-unescape
-			 (replace-regexp-in-string
-			  "[ \t]*\n[ \t]*" " "
-			  (match-string-no-properties 1)
+	(let ((link-spec (match-string-no-properties 1))
+	  (link-beg (match-beginning 1)))
+	  (setq raw-link (org-link-expand-abbrev (org-link-unescape
+		  (replace-regexp-in-string
+		   "[ \t]*\n[ \t]*" " "
+		   link-spec
+	  ;; If there is no description, try to find a fallback.
+	  (unless contents-begin
+	(pcase-let ((`(,default-beg ,default-end)
+			 (org-link-abbrev-default-desc link-spec)))
+	  (when default-beg
+		(setq contents-begin (+ link-beg default-beg)
+		  contents-end (+ link-beg default-end))
 	;; Determine TYPE of link and set PATH accordingly.  According
 	;; to RFC 3986, remove whitespaces from URI in external links.
 	;; In internal ones, treat indentation as a single space.


[1] Or just TAG.  If I look at ORG-NEWS for examples, I see a lot of
[[doc:org-symbol][org-symbol]].

Re: Couple of issues with org block meta lines faces

[Kévin Le Gouguec]

Hi!

Some points I think I can help with:

Sébastien Miquel  writes:

>   1) begin-line applies to both begin and end lines. This might be 
> intended. If you define an org-block-end-line face, it gets applied instead.

Yup, by default org-block-end-line :inherits from org-block-begin-line.

>   2) org-fontify-whole-block-delimiter-line is ignored. I'm aware I can 
> set the :extend t property to the face. If it does nothing, maybe this 
> variable should be removed.

See emacs bug#42184[1].


[1]: https://debbugs.gnu.org/cgi/bugreport.cgi?bug=42184

Re: [9.4] LOGBOOK visibility

[Kévin Le Gouguec]

Thank you for taking the time to make this write-up.

Nicolas Goaziou  writes:

> With overlays, you can't have your cake
> and eat it too.

FWIW, I think Emacs has a branch (feature/noverlay) which has been
reported to improve performance with overlays.  AFAICT it's just hanging
there waiting to be merged (though a naive "git merge" reveals some
conflicts); some quick Googling suggests it has last been discussed two
years ago:

https://lists.gnu.org/archive/html/emacs-devel/2018-03/msg00565.html

Re: [9.4] LOGBOOK visibility

[Kévin Le Gouguec]

Ihor Radchenko  writes:

>   (see the discussion in [1]). 

Ah, that makes sense!  No idea how I managed to miss that thread.

> I am currently working on a patch to rewrite the whole folding system.
> Your issue should disappear once it is applied.

Good to know!

> Meanwhile, you may, for example, advice
> org-fold--isearch-filter-predicate to re-fold drawers during isearch. 

Duly noted, thanks for the advice.

And thank you and Nicolas for all the hard work!

Re: [9.4] LOGBOOK visibility

[Kévin Le Gouguec]

I haven't reached the bottom of this rabbit hole yet.  Since I think
I've spent all the time I had to spend in this issue for the day, here's
where I'm at.

If I open my example file[1] with Org 9.3 and master (86fada6b5), and
compare the overlays covering the hidden part of the first LOGBOOK
drawer, like so:

#+begin_src sh
#!/bin/bash

set -eu

point=67

orgs=(
"${path_to_org_9_3}"
"${path_to_org_master}"
)

for o in "${orgs[@]}"
do
emacs -Q -batch \
  -L "${o}"/lisp \
  -eval "(find-file \"logbooks.org\")" \
  -eval "(message org-version)" \
  -eval "(dolist (o (overlays-at ${point}))
   (message \"from %s to %s\" (overlay-start o) (overlay-end o))
   (message \"%s\" (overlay-properties o)))"
echo
done
#+end_src

Here is what I get:

> 9.3
> from 32 to 2015
> (evaporate t invisible outline isearch-open-invisible #[lambda gibberish 
> involving (org-show-context 'isearch)])
> from 67 to 262
> (isearch-open-invisible delete-overlay invisible org-hide-drawer evaporate t)
> 
> 9.3.7
> from 32 to 2015
> (isearch-open-invisible delete-overlay invisible outline evaporate t)

- The LOGBOOK overlay (from 67 to 262) vanished.

- The isearch-open-invisible property of the "project 1" overlay (from
  32 to 2015) changed from a lambda[2] to just delete-overlay.


Again, not sure this shows there's an issue; for all I know Org 9.4 just
works differently and nothing's wrong there.  I hope someone can chime
in and take it up from here; otherwise I'll resume my investigations
sometime later.


[1] https://orgmode.org/list/87eepuz0bj@gmail.com/2-logbooks.org

[2] Which I guess is the value of
outline-isearch-open-invisible-function, set locally by org-mode in
the major mode function?

Re: [9.4] LOGBOOK visibility

[Kévin Le Gouguec]

Kévin Le Gouguec  writes:

> For example, in the attached file, if I search for "bug#5":
>
> - Org 9.3 keeps the drawers closed,
> - Org 9.4 opens every drawer for bugs 4, 5 and 6.
>
> AFAICT Org 9.3 never opened these drawers unless either
> org-startup-folded was showeverything, the user explicitly opened one
> individually, or their /content/ matched a search.
>
> Note that when cycling with S-TAB, even when "[SHOWING] ALL", Org 9.4
> keeps these drawers closed.

I've tried to bisect this; if I'm not mistaken, the commit that
introduced this behaviour is 1027e0256.  I don't know why or how this
commit caused this change yet, so I'm not sure if it's deliberate or
not.

[9.4] LOGBOOK visibility

[Kévin Le Gouguec]

Hi,

I've noticed that LOGBOOK drawers are now (9.4) open by default when
visiting an Org file, AFAICT as a result of org-startup-folded now
defaulting to showeverything.

That's not much of a problem by itself; what I find more of a hindrance
is that even with "#+STARTUP overview", when one starts isearching for
something, all "nearby" LOGBOOK drawers are opened; this adds a lot of
visual noise that hides the document structure.

For example, in the attached file, if I search for "bug#5":

- Org 9.3 keeps the drawers closed,
- Org 9.4 opens every drawer for bugs 4, 5 and 6.

AFAICT Org 9.3 never opened these drawers unless either
org-startup-folded was showeverything, the user explicitly opened one
individually, or their /content/ matched a search.

Note that when cycling with S-TAB, even when "[SHOWING] ALL", Org 9.4
keeps these drawers closed.

Is there a knob I've missed that would allow me to tell Org to keep
LOGBOOKs closed?


BTW, while reading ORG-NEWS, I came across this bit, added in 074ea1629:

> *** Deprecated ~org-cycle-hide-drawers~ function
> 
> Use the new function ~org-hide-drawer-all~ instead.  Note that there
> is also a new ~org-cycle-hide-property-drawers~ function.

AFAICT org-cycle-hide-property-drawers has been removed in 1aa095ccf?


Thank you for your time, and for your work with Org 9.4.


#+STARTUP: overview
* project 1
** bug#1
*** investigate
:LOGBOOK:
CLOCK: [2020-07-02 Thu 11:43]--[2020-07-02 Thu 11:43] =>  0:00
CLOCK: [2020-07-02 Thu 11:43]--[2020-07-02 Thu 11:43] =>  0:00
CLOCK: [2020-07-02 Thu 11:43]--[2020-07-02 Thu 11:43] =>  0:00
:END:
*** fix
:LOGBOOK:
CLOCK: [2020-07-02 Thu 11:43]--[2020-07-02 Thu 11:43] =>  0:00
CLOCK: [2020-07-02 Thu 11:43]--[2020-07-02 Thu 11:43] =>  0:00
CLOCK: [2020-07-02 Thu 11:43]--[2020-07-02 Thu 11:43] =>  0:00
:END:
*** document
:LOGBOOK:
CLOCK: [2020-07-02 Thu 11:43]--[2020-07-02 Thu 11:43] =>  0:00
CLOCK: [2020-07-02 Thu 11:43]--[2020-07-02 Thu 11:43] =>  0:00
CLOCK: [2020-07-02 Thu 11:43]--[2020-07-02 Thu 11:43] =>  0:00
:END:
** bug#2
*** investigate
:LOGBOOK:
CLOCK: [2020-07-02 Thu 11:43]--[2020-07-02 Thu 11:43] =>  0:00
CLOCK: [2020-07-02 Thu 11:43]--[2020-07-02 Thu 11:43] =>  0:00
CLOCK: [2020-07-02 Thu 11:43]--[2020-07-02 Thu 11:43] =>  0:00
:END:
*** fix
:LOGBOOK:
CLOCK: [2020-07-02 Thu 11:43]--[2020-07-02 Thu 11:43] =>  0:00
CLOCK: [2020-07-02 Thu 11:43]--[2020-07-02 Thu 11:43] =>  0:00
CLOCK: [2020-07-02 Thu 11:43]--[2020-07-02 Thu 11:43] =>  0:00
:END:
*** document
:LOGBOOK:
CLOCK: [2020-07-02 Thu 11:43]--[2020-07-02 Thu 11:43] =>  0:00
CLOCK: [2020-07-02 Thu 11:43]--[2020-07-02 Thu 11:43] =>  0:00
CLOCK: [2020-07-02 Thu 11:43]--[2020-07-02 Thu 11:43] =>  0:00
:END:
** bug#3
*** investigate
:LOGBOOK:
CLOCK: [2020-07-02 Thu 11:43]--[2020-07-02 Thu 11:43] =>  0:00
CLOCK: [2020-07-02 Thu 11:43]--[2020-07-02 Thu 11:43] =>  0:00
CLOCK: [2020-07-02 Thu 11:43]--[2020-07-02 Thu 11:43] =>  0:00
:END:
*** fix
:LOGBOOK:
CLOCK: [2020-07-02 Thu 11:43]--[2020-07-02 Thu 11:43] =>  0:00
CLOCK: [2020-07-02 Thu 11:43]--[2020-07-02 Thu 11:43] =>  0:00
CLOCK: [2020-07-02 Thu 11:43]--[2020-07-02 Thu 11:43] =>  0:00
:END:
*** document
:LOGBOOK:
CLOCK: [2020-07-02 Thu 11:43]--[2020-07-02 Thu 11:43] =>  0:00
CLOCK: [2020-07-02 Thu 11:43]--[2020-07-02 Thu 11:43] =>  0:00
CLOCK: [2020-07-02 Thu 11:43]--[2020-07-02 Thu 11:43] =>  0:00
:END:
* project 2
** bug#4
*** investigate
:LOGBOOK:
CLOCK: [2020-07-02 Thu 11:43]--[2020-07-02 Thu 11:43] =>  0:00
CLOCK: [2020-07-02 Thu 11:43]--[2020-07-02 Thu 11:43] =>  0:00
CLOCK: [2020-07-02 Thu 11:43]--[2020-07-02 Thu 11:43] =>  0:00
:END:
*** fix
:LOGBOOK:
CLOCK: [2020-07-02 Thu 11:43]--[2020-07-02 Thu 11:43] =>  0:00
CLOCK: [2020-07-02 Thu 11:43]--[2020-07-02 Thu 11:43] =>  0:00
CLOCK: [2020-07-02 Thu 11:43]--[2020-07-02 Thu 11:43] =>  0:00
:END:
*** document
:LOGBOOK:
CLOCK: [2020-07-02 Thu 11:43]--[2020-07-02 Thu 11:43] =>  0:00
CLOCK: [2020-07-02 Thu 11:43]--[2020-07-02 Thu 11:43] =>  0:00
CLOCK: [2020-07-02 Thu 11:43]--[2020-07-02 Thu 11:43] =>  0:00
:END:
** bug#5
*** investigate
:LOGBOOK:
CLOCK: [2020-07-02 Thu 11:43]--[2020-07-02 Thu 11:43] =>  0:00
CLOCK: [2020-07-02 Thu 11:43]--[2020-07-02 Thu 11:43] =>  0:00
CLOCK: [2020-07-02 Thu 11:43]--[2020-07-02 Thu 11:43] =>  0:00
:END:
*** fix
:LOGBOOK:
CLOCK: [2020-07-02 Thu 11:43]--[2020-07-02 Thu 11:43] =>  0:00
CLOCK: [2020-07-02 Thu 11:43]--[2020-07-02 Thu 11:43] =>  0:00
CLOCK: [2020-07-02 Thu 11:43]--[2020-07-02 Thu 11:43] =>  0:00
:END:
*** document
:LOGBOOK:
CLOCK: [2020-07-02 Thu 11:43]--[2020-07-02 Thu 11:43] =>  0:00
CLOCK: [2020-07-02 Thu 11:43]--[2020-07-02 Thu 11:43] =>  0:00
CLOCK: [2020-07-02 Thu 11:43]--[2020-07-02 Thu 11:43] =>  0:00
:END:
** bug#6
*** investigate
:LOGBOOK:
CLOCK: [2020-07-02 Thu 11:43]--[2020-07-02 Thu 11:43] =>  0:00
CLOCK: [2020-07-02 Thu 11:43]--[2020-07-02 Thu 11:43] =>  0:00
CLOCK: [2020-07-02 Thu 11:43]--[2020-07-02 Thu 11:43] =>  0:00
:END:
*** fix
:LOGBOOK:
CLOCK: [2020-07-02 Thu 11:43]--[2020-07-02 Thu 11:43] =>  0:00

Re: Breaking line inside list item

[Kévin Le Gouguec]

Sébastien Miquel  writes:

>> There is RET or C-j, depending on your settings.
>
> C-j (~org-return-indent~) does work, thank you.

Just a heads-up: the next version of org-mode (9.4) will obey
electric-indent-mode, which is a global minor mode that is turned on by
default.

When this mode is enabled, RET means "smart return", i.e. "newline &
indent", while C-j means "just newline".  This is how things work in
most modes (except Org) since Emacs 24.4.

When org-mode 9.4 is released, you can either use RET instead of C-j, or
disable electric-indent-mode in org-mode-hook, as will be explained in
ORG-NEWS.

Re: Setting org-todo-keywords through directory-local variables

[Kévin Le Gouguec]

Hello,

I would like to re-submit this idea, now that I am reasonably sure that
my implementation will not impact users who do not use the feature.

(I understand that Org 9.4 is on the way.  I am not asking for this
feature to be included right now; I would like to get some feedback
first, then move on to documenting it.)

* Motivation

To recap: AFAIK it is not possible to define both org-todo-keywords and
org-todo-keyword-faces per-directory.  The former can be set with
#+SETUPFILE, but the latter simply can't be set locally, unless I'm
mistaken.

I'd like to specify, for all Org files in a directory, which keywords to
use and how they must look.  Setting both org-todo-* variables in
.dir-locals.el would be ideal IMO: the definitions for keywords and
their faces would be right next to each other.

This cannot work right now because (1) of a stray call to default-value
(2) Org computes the TODO machinery (regexps and font-lock) when setting
up the major mode, before directory-local variables are in effect.

* Prior art

AUCTeX[1] and markdown-mode[2] both solve this using
hack-local-variables-hook.  This seems to be the expected way for modes
to (re)compute things that may be affected by file or directory-local
variables.

[1] 
http://git.savannah.gnu.org/cgit/auctex.git/tree/font-latex.el?h=release_12_2#n1331
[2] https://github.com/jrblevin/markdown-mode/blob/v2.4/markdown-mode.el#L9403

The idea is to register a function that will check whether the user
overloaded variables we care about; if that's the case, then we
recompute what we need to.

* Patch

The attached patch:

- does not change org-mode's default setup logic,

- adds a function to hack-local-variables-hook that will look for
  org-todo-* variables, and recompute org-set-regexps-and-options and
  org-set-font-lock-defaults if needed,

- adds :safe predicates for these variables,

- adds unit tests.

>From 148c5fa45e1fb8d58ecc86bb266d0fa33b8994a6 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?K=C3=A9vin=20Le=20Gouguec?= 
Date: Wed, 27 May 2020 22:53:56 +0200
Subject: [PATCH] Allow users to configure TODO keywords from dir-locals.el

This uses the same method as AUCTeX and markdown-mode to refresh
fontification based on file-local and directory-local variables:

http://git.savannah.gnu.org/cgit/auctex.git/tree/font-latex.el?h=release_12_2#n1331
https://github.com/jrblevin/markdown-mode/blob/v2.4/markdown-mode.el#L9403

* lisp/org-faces.el (org-todo-keyword-faces): Add safe-local-variable
predicate.

* lisp/org.el (org-todo-keywords): Add safe-local-variable predicate.
(org-set-regexps-and-options): Use buffer-local value of
org-todo-keywords.
(org-mode): Register a function to reset regexps and font-lock once
file-local variables have been applied.
(org--process-local-variables): Recompute regexps and font-lock if the
user set relevant variables.

* testing/examples/dir-locals/.dir-locals.el:
* testing/examples/dir-locals/todo.org: Support files for new tests.

* testing/lisp/test-org.el (test-org/dir-local-todo-keyword-faces):
(test-org/dir-local-todo-cycling): New tests.
---
 lisp/org-faces.el  | 10 ++-
 lisp/org.el| 28 +++
 testing/examples/dir-locals/.dir-locals.el | 11 
 testing/examples/dir-locals/todo.org   |  8 ++
 testing/lisp/test-org.el   | 32 ++
 5 files changed, 82 insertions(+), 7 deletions(-)
 create mode 100644 testing/examples/dir-locals/.dir-locals.el
 create mode 100644 testing/examples/dir-locals/todo.org

diff --git a/lisp/org-faces.el b/lisp/org-faces.el
index d78b606ec..fc834f37d 100644
--- a/lisp/org-faces.el
+++ b/lisp/org-faces.el
@@ -291,7 +291,15 @@ determines if it is a foreground or a background color."
 	   (string :tag "Keyword")
 	   (choice :tag "Face   "
 		   (string :tag "Color")
-		   (sexp :tag "Face")
+		   (sexp :tag "Face"
+  :safe (lambda (x)
+  (cl-every
+   (lambda (pair)
+	 (let ((keyword (car pair))
+		   (face (cdr pair)))
+	   (and (stringp keyword)
+		(or (facep face) (listp face)
+   x)))
 
 (defface org-priority '((t :inherit font-lock-keyword-face))
   "Face used for priority cookies."
diff --git a/lisp/org.el b/lisp/org.el
index 4d46b4173..c0183dbff 100644
--- a/lisp/org.el
+++ b/lisp/org.el
@@ -1945,7 +1945,13 @@ taken from the (otherwise obsolete) variable `org-todo-interpretation'."
 	 org-todo-interpretation-widgets))
 		  widget))
 		   (repeat
-		(string :tag "Keyword"))
+		(string :tag "Keyword")
+  :safe (lambda (x)
+  (cl-every
+   (lambda (seq)
+ (and (memq (car seq) '(sequence type))
+  (cl-every (lambda (i) (stringp i)) (cdr seq
+   x)))
 
 (defvar-local org-todo-keywords-1 nil
   "All TODO and DONE keywords active in a buffer.")
@@ -4358,10 +4364,9 @@ related expressions."
  (cons 'sequence (split-string value)))
    

Re: [RFC] Rewrite org-(forward|backward)-paragraph

[Kévin Le Gouguec]

Hi Nicolas!

I don't know how useful my feedback will be, since I'm not a heavy user
of paragraph-based movement[1], but here goes!

Nicolas Goaziou  writes:

> In any case, the purpose of this rewrite is to mimic more closely
> expected behaviour from `forward-paragraph' and `backward-paragraph'
> functions, as found, e.g., in Text mode. Unlike Text mode, navigation in
> Org mode is usually not linear, but both should feel the same, for
> example, when the document is indeed linear.

I've danced around ORG-NEWS to assess the changes; what I observed does
feel closer to text-mode (point moves to the blank lines between
paragraphs instead of to the paragraph starts), the other changes I
could spot do not strike me as deal-breaking:

- point now jumps over tight lists[2] instead of stopping at each item,

- point stops a few more times within code blocks, acting like
  #+begin_src and #+end_src are paragraphs of their own, instead of
  jumping over the whole block; also, forward and backward movements are
  now symmetric 

Are there other situations where you think your changes could be
controversial?

> WDYT? Also, what should be done with M-{ and M-}?

FWIW, I think that reducing the distance between Org mode and The Rest
of Emacs™ is a commendable goal, so I would vote for binding paragraph
functions to M-{ and M-}, and moving element functions to C- and
C-.  I realize that this might be too big a change for the sake of
conformity though.

(And again: I don't use these functions very often, so my vote probably
shouldn't carry too much weight.)


Thank you for working on this!


[1] Curly brackets are cumbersome with AZERTY, so I never took the habit
of moving by paragraphs outside org-mode.  Likewise with Org's
 bindings: my fingers are too lazy to reach for the arrow
keys for something as often-used as movement.

[2] I.e. lists without newlines between items.

Re: Possible fix for :includes header argument in org-babel C source blocks

[Kévin Le Gouguec]

Bastien  writes:

> Brandon Guttersohn  writes:
>
>> So this patch is sort of a
>> new feature, but a trivial one.
>
> Agreed.  Could you or Kevin propose a sentence to advertise this small
> enhancement in etc/ORG-NEWS?

Here goes nothing.

>From b18f6dc66ea4a05c95a4ee6825723da4beaa1c83 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?K=C3=A9vin=20Le=20Gouguec?= 
Date: Mon, 1 Jun 2020 21:33:01 +0200
Subject: [PATCH] * etc/ORG-NEWS: Announce a recent fix in ob-C.el.

---
 etc/ORG-NEWS | 6 ++
 1 file changed, 6 insertions(+)

diff --git a/etc/ORG-NEWS b/etc/ORG-NEWS
index c0df785d4..d3f2bb1ca 100644
--- a/etc/ORG-NEWS
+++ b/etc/ORG-NEWS
@@ -202,6 +202,12 @@ Org provides a new tool ~org-link-open-as-file~, useful when defining
 new link types similar to "file"-type links.  See docstring for
 details.
 
+*** =ob-C.el= allows you to include non-system header files
+
+In C and C++ blocks, ~:includes~ arguments that do not start with a
+~<~ character will now be formatted as double-quoted ~#include~
+statements.
+
 *** =ob-clojure.el= supports inf-clojure.el and ClojureScript evaluation
 
 You can now set ~(setq org-babel-clojure-backend 'inf-clojure)~ and
-- 
2.26.2


Note that IIUC, for non-system includes to work, either

- the filenames must be absolute, or
- the compiler must be given -I arguments through org-babel-C-compiler.

This variable can be set (e.g. to "gcc -I .") with file or
directory-local variables.  Should we promote this method in NEWS?  A
downside is that the user will be warned about the variable's value
being potentially unsafe, and we can't really avoid that unless we throw
a blanket :safe #'stringp on this defcustom.

Re: Failing tests

[Kévin Le Gouguec]

Kévin Le Gouguec  writes:

> Absolutely.  I've attached a patch to that effect.

I just realized that these let-bindings probably deserved explanatory
comments.  Here is an updated patch:

>From f996ec3a10a845abae2fa463ab0ea7a761af1707 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?K=C3=A9vin=20Le=20Gouguec?= 
Date: Mon, 1 Jun 2020 16:07:44 +0200
Subject: [PATCH] Make tests robust with respect to mailcap entries

When /etc/mailcap specifies a program for text/plain
files (e.g. less(1)), org-open-file will run this program instead of
visiting the file.  This throws off some tests which expect
extension-less temporary files to be visited.

* testing/lisp/test-ob-tangle.el (ob-tangle/jump-to-org):
* testing/lisp/test-org-attach.el (test-org-attach/dir): Rig
org-file-apps so that temporary files are visited inside Emacs.
---
 testing/lisp/test-ob-tangle.el  | 124 +-
 testing/lisp/test-org-attach.el | 148 
 2 files changed, 138 insertions(+), 134 deletions(-)

diff --git a/testing/lisp/test-ob-tangle.el b/testing/lisp/test-ob-tangle.el
index 2283037fc..7b1f617ed 100644
--- a/testing/lisp/test-ob-tangle.el
+++ b/testing/lisp/test-ob-tangle.el
@@ -124,24 +124,26 @@ echo 1
 
 (ert-deftest ob-tangle/jump-to-org ()
   "Test `org-babel-tangle-jump-to-org' specifications."
-  ;; Standard test.
-  (should
-   (equal
-"* H\n#+begin_src emacs-lisp\n1\n#+end_src"
-(org-test-with-temp-text-in-file
-"* H\n#+begin_src emacs-lisp\n1\n#+end_src"
-  (let ((file (buffer-file-name)))
-(org-test-with-temp-text
-(format ";; [[file:%s][H:1]]\n1\n;; H:1 ends here\n"
-(file-name-nondirectory file))
-  (org-babel-tangle-jump-to-org)
-  (buffer-string))
-  ;; Multiple blocks in the same section.
-  (should
-   (equal
-"2"
-(org-test-with-temp-text-in-file
-"* H
+  ;; Make sure temporary files will be visited inside Emacs.
+  (let ((org-file-apps '((t . emacs
+;; Standard test.
+(should
+ (equal
+  "* H\n#+begin_src emacs-lisp\n1\n#+end_src"
+  (org-test-with-temp-text-in-file
+  "* H\n#+begin_src emacs-lisp\n1\n#+end_src"
+	(let ((file (buffer-file-name)))
+  (org-test-with-temp-text
+  (format ";; [[file:%s][H:1]]\n1\n;; H:1 ends here\n"
+  (file-name-nondirectory file))
+(org-babel-tangle-jump-to-org)
+(buffer-string))
+;; Multiple blocks in the same section.
+(should
+ (equal
+  "2"
+  (org-test-with-temp-text-in-file
+  "* H
 
 first block
 
@@ -155,49 +157,49 @@ another block
 2
 #+end_src
 "
-  (let ((file (buffer-file-name)))
-(org-test-with-temp-text
-(format ";; [[file:%s][H:2]]\n2\n;; H:2 ends here\n"
-(file-name-nondirectory file))
-  (org-babel-tangle-jump-to-org)
-  (buffer-substring (line-beginning-position)
-(line-end-position)))
-  ;; Preserve position within the source code.
-  (should
-   (equal
-"1)"
-(org-test-with-temp-text-in-file
-"* H\n#+begin_src emacs-lisp\n(+ 1 1)\n#+end_src"
-  (let ((file (buffer-file-name)))
-(org-test-with-temp-text
-(format ";; [[file:%s][H:1]]\n(+ 1 1)\n;; H:1 ends here\n"
-(file-name-nondirectory file))
-  (org-babel-tangle-jump-to-org)
-  (buffer-substring-no-properties (point) (line-end-position)))
-  ;; Blocks before first heading.
-  (should
-   (equal
-"Buffer start\n#+begin_src emacs-lisp\n1\n#+end_src\n* H"
-(org-test-with-temp-text-in-file
-"Buffer start\n#+begin_src emacs-lisp\n1\n#+end_src\n* H"
-  (let ((file (buffer-file-name)))
-(org-test-with-temp-text
-(format ";; [[file:%s][H:1]]\n1\n;; H:1 ends here\n"
-(file-name-nondirectory file))
-  (org-babel-tangle-jump-to-org)
-  (buffer-string))
-  ;; Special case: buffer starts with a source block.
-  (should
-   (equal
-"#+begin_src emacs-lisp\n1\n#+end_src\n* H"
-(org-test-with-temp-text-in-file
-"#+begin_src emacs-lisp\n1\n#+end_src\n* H"
-  (let ((file (buffer-file-name)))
-(org-test-with-temp-text
-(format ";; [[file:%s][H:1]]\n1\n;; H:1 ends here\n"
-(file-name-nondirectory file))
-  (org-babel-tangle-jump-to-org)
-  (buffer-string)))
+	(let ((file (buffer-file-name)))
+  (org-test-with-temp-text
+  (format ";; [[file:%s][H:2]]\n2\n;; H:2 ends here\n"
+  (file-name-nondirectory file))
+(org-babel-tangle-jump-to-org)
+(buffer-substring (line-b

Re: Failing tests

[Kévin Le Gouguec]

Kyle Meyer  writes:

>> I think I've narrowed this down to org-open-file running "less
>> examples/att1/fileA" instead of visiting this file.
> [...]
>> Let-binding org-file-apps to '(("." . emacs)) makes the tests pass, but
>> I don't know if that's the way we want to solve this.
>
> Thanks for looking into the failures.  Let-binding org-file-apps sounds
> like a good approach to me.  Rather than the catch-all regular
> expression, I believe the value could be ((t . emacs)).

Absolutely.  I've attached a patch to that effect.

I wonder though, shouldn't org-open-file always visit text/plain files?
Why would we ever want to send those to an external viewer?

I think this would need special-casing inside org-open-file, since I
don't see a way to catch all text/plain files with org-file-apps.


>From 05a71740c662fcde3fcfad7c07975052781ec589 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?K=C3=A9vin=20Le=20Gouguec?= 
Date: Mon, 1 Jun 2020 16:07:44 +0200
Subject: [PATCH] Make tests robust with respect to mailcap entries

When /etc/mailcap specifies a program for text/plain
files (e.g. less(1)), org-open-file will run this program instead of
visiting the file.  This throws off some tests which expect
extension-less temporary files to be visited.

* testing/lisp/test-ob-tangle.el (ob-tangle/jump-to-org):
* testing/lisp/test-org-attach.el (test-org-attach/dir): Rig
org-file-apps so that temporary files are visited inside Emacs.
---
 testing/lisp/test-ob-tangle.el  | 121 +-
 testing/lisp/test-org-attach.el | 147 
 2 files changed, 135 insertions(+), 133 deletions(-)

diff --git a/testing/lisp/test-ob-tangle.el b/testing/lisp/test-ob-tangle.el
index 2283037fc..35490f538 100644
--- a/testing/lisp/test-ob-tangle.el
+++ b/testing/lisp/test-ob-tangle.el
@@ -125,23 +125,24 @@ echo 1
 (ert-deftest ob-tangle/jump-to-org ()
   "Test `org-babel-tangle-jump-to-org' specifications."
   ;; Standard test.
-  (should
-   (equal
-"* H\n#+begin_src emacs-lisp\n1\n#+end_src"
-(org-test-with-temp-text-in-file
-"* H\n#+begin_src emacs-lisp\n1\n#+end_src"
-  (let ((file (buffer-file-name)))
-(org-test-with-temp-text
-(format ";; [[file:%s][H:1]]\n1\n;; H:1 ends here\n"
-(file-name-nondirectory file))
-  (org-babel-tangle-jump-to-org)
-  (buffer-string))
-  ;; Multiple blocks in the same section.
-  (should
-   (equal
-"2"
-(org-test-with-temp-text-in-file
-"* H
+  (let ((org-file-apps '((t . emacs
+(should
+ (equal
+  "* H\n#+begin_src emacs-lisp\n1\n#+end_src"
+  (org-test-with-temp-text-in-file
+  "* H\n#+begin_src emacs-lisp\n1\n#+end_src"
+	(let ((file (buffer-file-name)))
+  (org-test-with-temp-text
+  (format ";; [[file:%s][H:1]]\n1\n;; H:1 ends here\n"
+  (file-name-nondirectory file))
+(org-babel-tangle-jump-to-org)
+(buffer-string))
+;; Multiple blocks in the same section.
+(should
+ (equal
+  "2"
+  (org-test-with-temp-text-in-file
+  "* H
 
 first block
 
@@ -155,49 +156,49 @@ another block
 2
 #+end_src
 "
-  (let ((file (buffer-file-name)))
-(org-test-with-temp-text
-(format ";; [[file:%s][H:2]]\n2\n;; H:2 ends here\n"
-(file-name-nondirectory file))
-  (org-babel-tangle-jump-to-org)
-  (buffer-substring (line-beginning-position)
-(line-end-position)))
-  ;; Preserve position within the source code.
-  (should
-   (equal
-"1)"
-(org-test-with-temp-text-in-file
-"* H\n#+begin_src emacs-lisp\n(+ 1 1)\n#+end_src"
-  (let ((file (buffer-file-name)))
-(org-test-with-temp-text
-(format ";; [[file:%s][H:1]]\n(+ 1 1)\n;; H:1 ends here\n"
-(file-name-nondirectory file))
-  (org-babel-tangle-jump-to-org)
-  (buffer-substring-no-properties (point) (line-end-position)))
-  ;; Blocks before first heading.
-  (should
-   (equal
-"Buffer start\n#+begin_src emacs-lisp\n1\n#+end_src\n* H"
-(org-test-with-temp-text-in-file
-"Buffer start\n#+begin_src emacs-lisp\n1\n#+end_src\n* H"
-  (let ((file (buffer-file-name)))
-(org-test-with-temp-text
-(format ";; [[file:%s][H:1]]\n1\n;; H:1 ends here\n"
-(file-name-nondirectory file))
-  (org-babel-tangle-jump-to-org)
-  (buffer-string))
-  ;; Special case: buffer starts with a source block.
-  (should
-   (equal
-"#+begin_src emacs-lisp\n1\n#+end_src\n* H"
-(org-test-with-temp-text-in-file
-"#+begin_src emacs-lisp\n1\n#+end_src\n* H"
-  (let ((file (buffer-file-name)))
-(org-test-with-temp-text
-(format ";; [[file:%s][H:1]]\n1\n;; H:1 ends here\n"
-(file-name-nondirectory file))
-  (org-babel-tangle-jump-to-org)
-  

bug#41584: 26.3; org-indent-mode's line-prefix text property flickers near overlays

[Kévin Le Gouguec]

Eli Zaretskii  writes:

> Fixed.

AFAICT you've scared the moles away, so I'll boldly close this report.
Thanks a lot for the efficient whacking!

(I guess xdisp.c is something we won't to touch with a 10-foot pole on
the release branch, especially for a bug that users have lived with for
so long?)

bug#41584: 26.3; org-indent-mode's line-prefix text property flickers near overlays

[Kévin Le Gouguec]

Eli Zaretskii  writes:

> Should be fixed now on the master branch.

Thanks a lot!

I'm sorry I didn't notice it earlier, but I may have found another mole
for you to whack; starting with the same recipe:

> - emacs -Q
> - C-x C-f repro.org
> - M-x org-indent-mode
> - M-: (insert "* heading\ntext")
> - M-:
> (let ((ov (make-overlay (point-at-bol) (point-at-bol)))
>   (val (propertize " " 'display '((left-fringe right-triangle)
>   (overlay-put ov 'before-string val))

This time, instead of hitting RET, insert a pair of parentheses, and
wait blink-matching-delay seconds until the opening parenthesis stops
being highlighted.  Before and after your fix, the line-prefix
disappears until I enter another command.

Failing tests (was: Possible fix for :includes header argument in org-babel C source blocks)

[Kévin Le Gouguec]

Kyle Meyer  writes:

> The source for that page is in the worg repo:
> https://code.orgmode.org/bzg/worg/src/master/org-contrib/babel/languages/ob-doc-C.org

Thanks for the pointer, and for applying the patches!

>>>   FAILED  ob-tangle/jump-to-org
>>>   FAILED  test-org-attach/dir
>
> :(
>
> After your first patch, all tests now pass on my end.  Would you mind
> posting the details of those failures in a new thread?

Mmm, on further inspection, those tests fail on one of my setups but
pass on another.

I think I've narrowed this down to org-open-file running "less
examples/att1/fileA" instead of visiting this file.

The following snippet returns "less '%s'" on the failing setup, and nil
on the passing one:

#+begin_src elisp
(mailcap-mime-info (mailcap-extension-to-mime ""))
#+end_src

IIUC this comes from /etc/mailcap; the failing setup (Xubuntu 18.04) has
an entry saying "less '%s'" for "text/plain"; the passing setup
(OpenSUSE Tumbleweed) simply has no /etc/mailcap.  mailcap-mime-info
falls back to "text/plain" when mailcap-extension-to-mime returns nil.

Let-binding org-file-apps to '(("." . emacs)) makes the tests pass, but
I don't know if that's the way we want to solve this.

Re: Possible fix for :includes header argument in org-babel C source blocks

[Kévin Le Gouguec]

Brandon Guttersohn  writes:

> Apologies for the regression, and thank you for fixing it. I neglected
> to run the tests before suggesting that fix -- I'll try not to do that
> again..

No biggie, that got me to finally try out Babel ;)


I don't know if it's been mentioned in the "issue tracker?" thread, but
if I could pick just *one* feature off web-based forges, it'd be
automated testing with CI…

I find the "make-test-before-commit" discipline easy enough to adhere to
at $DAYJOB; it's not as straightforward when contributing to free
software, when I'm frequently pressed for time, running on battery on a
low-end laptop…

Running a few unit tests is not a big deal, but it's not trivial to
anticipate which ones to run; test-foo.el is rarely enough to catch
regressions introduced by tweaking foo.el.  

Having something (e.g. emba.gnu.org) pick up patches sent to the mailing
list and report new test failures would be very helpful, for
contributors if not for maintainers.
 

> I can at least confirm that the patch wasn't intended to change how
> C-header-files are specified in the org-babel-block-header. The goal
> was only to change how the headers are formatted in the generated
> C-language file during execution, and only for headers which were not
> wrapped in <>'s.

OK; IIUC, before the patch it was not possible to generate double-quoted
includes short of backslash-escaping the double quotes; that's why I
assumed that the goal of the patch was to make it easier to use
double-quoted includes, which I thought worth advertising in ORG-NEWS.

26.3; org-indent-mode's line-prefix text property flickers near overlays

[Kévin Le Gouguec]

Hello,

The line-prefix text property set by org-indent-mode sometimes vanishes
when typing near a line where an overlay is applied.  To reproduce:

- emacs -Q
- C-x C-f repro.org
- M-x org-indent-mode
- M-: (insert "* heading\ntext")
- M-:
(let ((ov (make-overlay (point-at-bol) (point-at-bol)))
  (val (propertize " " 'display '((left-fringe right-triangle)
  (overlay-put ov 'before-string val))
- RET
- a

When "a" is inserted, org-indent-mode's line-prefix disappears on the
*previous* line ("text").  It remains gone as long as I type
self-inserting characters, until

- I type certain commands, e.g. RET or C-j, or

- I insert a closing delimiter that makes
  blink-paren-post-self-insert-function blink the corresponding opening
  delimiter, e.g. ')' or ']'.

Then the line-prefix shows up again.


This recipe is simplified; I originally found this bug in Org files
under version control with diff-hl-flydiff-mode enabled.  When typing in
new content, the preceding line is shoved to the left until I stop
typing; after diff-hl-flydiff-delay, diff-hl's idle timer kicks in and
updates its overlay, which as a side-effect makes org-indent-mode's
line-prefix come back up.

See [1] for my original bug report to diff-hl and some crude
"analysis".


I'm clearly out of my depth; at least I hope I used the correct terms to
describe the symptoms I've observed.  Let me know if there is anything I
can do to help debug this.

Thank you for your time.


[1] https://github.com/dgutov/diff-hl/issues/142


In GNU Emacs 28.0.50 (build 21, x86_64-pc-linux-gnu, GTK+ Version 3.24.20, 
cairo version 1.16.0)
 of 2020-05-18 built on my-little-tumbleweed
Repository revision: b1fe27d77db8f819641231ca46725f3eed0b4d9b
Repository branch: master
Windowing system distributor 'The X.Org Foundation', version 11.0.12008000
System Description: openSUSE Tumbleweed

Configured using:
 'configure --with-xwidgets --with-cairo'

Configured features:
XPM JPEG TIFF GIF PNG RSVG CAIRO SOUND GPM DBUS GSETTINGS GLIB NOTIFY
INOTIFY ACL LIBSELINUX GNUTLS LIBXML2 FREETYPE HARFBUZZ M17N_FLT LIBOTF
ZLIB TOOLKIT_SCROLL_BARS GTK3 X11 XDBE XIM MODULES THREADS XWIDGETS JSON
PDUMPER LCMS2 GMP

Important settings:
  value of $LC_CTYPE: en_US.UTF-8
  value of $LC_TIME: en_GB.UTF-8
  value of $LANG: en_US.UTF-8
  value of $XMODIFIERS: @im=local
  locale-coding-system: utf-8-unix

Re: Possible fix for :includes header argument in org-babel C source blocks

[Kévin Le Gouguec]

Kévin Le Gouguec  writes:

> That leads me to believe that the coercion was an unintended side-effect
> of (format …).

Never mind, the ORG-NEWS entry for 9.1 shows an example of unquoted
header, so I guess it is intentional.

Here is a patch to fix the regression:

>From b68122821a26578470506938c3a358f52f5d7a46 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?K=C3=A9vin=20Le=20Gouguec?= 
Date: Thu, 28 May 2020 11:09:18 +0200
Subject: [PATCH 1/2] Coerce symbols found in :includes header arguments to
 strings

Fix regression from 2020-05-24T16:23:26Z!bran...@guttersohn.org
(commit 44cb98fdb), which broke test ob-C/string-var.

* lisp/ob-C.el (org-babel-C-expand-C): Make sure items in :includes
arguments are strings before performing string operations on them.
---
 lisp/ob-C.el | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/lisp/ob-C.el b/lisp/ob-C.el
index c3e72c680..42c60c296 100644
--- a/lisp/ob-C.el
+++ b/lisp/ob-C.el
@@ -233,6 +233,9 @@ its header arguments."
 		;; includes
 		(mapconcat
 		 (lambda (inc)
+		   ;; :includes '( ) gives us a list of
+		   ;; symbols; convert those to strings.
+		   (when (symbolp inc) (setq inc (symbol-name inc)))
 		   (if (string-prefix-p "<" inc)
 		   (format "#include %s" inc)
 		 (format "#include \"%s\"" inc)))
-- 
2.17.1


And here is a patch to add a test for the unquoted-single-header case,
since otherwise it's hard to tell whether this behaviour is intentional:

>From cf1bb27215a46a521bb2f50d16b7dbc7441d81ec Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?K=C3=A9vin=20Le=20Gouguec?= 
Date: Thu, 28 May 2020 11:47:25 +0200
Subject: [PATCH 2/2] Add test case for symbol coercion in C Babel blocks

The ORG-NEWS entry for version 9.1 suggests that this coercion was
always intended, though AFAICT there was no test case for it.

* testing/lisp/test-ob-C.el (ob-C/symbol-include): Check explicitly
that :includes  (with no double-quotes around )
will be parsed correctly.
(ob-D/simple-program, ob-C/integer-var, ob-D/integer-var,
ob-C/two-integer-var, ob-D/two-integer-var, ob-C/string-var,
ob-D/string-var, ob-C/preprocessor): Adjust block indices.

* testing/examples/ob-C-test.org (Simple tests): Add input for the new
test.
---
 testing/examples/ob-C-test.org |  6 ++
 testing/lisp/test-ob-C.el  | 23 +++
 2 files changed, 21 insertions(+), 8 deletions(-)

diff --git a/testing/examples/ob-C-test.org b/testing/examples/ob-C-test.org
index 0faf630b9..347607cae 100644
--- a/testing/examples/ob-C-test.org
+++ b/testing/examples/ob-C-test.org
@@ -10,6 +10,12 @@
   return 0;
 #+end_src
 
+#+source: simple
+#+begin_src cpp :includes  :results silent
+  std::cout << 42;
+  return 0;
+#+end_src
+
 #+source: simple
 #+begin_src D :results silent
   writefln ("%s", 42);
diff --git a/testing/lisp/test-ob-C.el b/testing/lisp/test-ob-C.el
index 3e4a63f88..6b6b728a2 100644
--- a/testing/lisp/test-ob-C.el
+++ b/testing/lisp/test-ob-C.el
@@ -32,60 +32,67 @@
 		  (org-babel-next-src-block 1)
 		  (should (= 42 (org-babel-execute-src-block))
 
+(ert-deftest ob-C/symbol-include ()
+  "Hello world program with unquoted :includes."
+  (if (executable-find org-babel-C++-compiler)
+  (org-test-at-id "fa6db330-e960-4ea2-ac67-94bb845b8577"
+		  (org-babel-next-src-block 2)
+		  (should (= 42 (org-babel-execute-src-block))
+
 (ert-deftest ob-D/simple-program ()
   "Hello world program."
   (if (executable-find org-babel-D-compiler)
   (org-test-at-id "fa6db330-e960-4ea2-ac67-94bb845b8577"
-		  (org-babel-next-src-block 2)
+		  (org-babel-next-src-block 3)
 		  (should (= 42 (org-babel-execute-src-block))
 
 (ert-deftest ob-C/integer-var ()
   "Test of an integer variable."
   (if (executable-find org-babel-C++-compiler)
   (org-test-at-id "fa6db330-e960-4ea2-ac67-94bb845b8577"
-		  (org-babel-next-src-block 3)
+		  (org-babel-next-src-block 4)
 		  (should (= 12 (org-babel-execute-src-block))
 
 (ert-deftest ob-D/integer-var ()
   "Test of an integer variable."
   (if (executable-find org-babel-D-compiler)
   (org-test-at-id "fa6db330-e960-4ea2-ac67-94bb845b8577"
-		  (org-babel-next-src-block 4)
+		  (org-babel-next-src-block 5)
 		  (should (= 12 (org-babel-execute-src-block))
 
 (ert-deftest ob-C/two-integer-var ()
   "Test of two input variables"
   (if (executable-find org-babel-C++-compiler)
   (org-test-at-id "fa6db330-e960-4ea2-ac67-94bb845b8577"
-		  (org-babel-next-src-block 5)
+		  (org-babel-next-src-block 6)
 		  (should (= 22 (org-babel-execute-src-block))
 
 (ert-deftest ob-D/two-integer-var ()
   "Test of two input variables"
   (if (executable-find org-babel-D-compiler)
   (org-test-at-id "fa6db330-e960-4ea2-ac67-94bb845b8577"
-		 

Re: Possible fix for :includes header argument in org-babel C source blocks

[Kévin Le Gouguec]

Kyle Meyer  writes:

> I think this discussion was on emacs-devel only, so here are some links
> for others who might go looking for more context:
>
>   https://lists.gnu.org/archive/html/emacs-devel/2020-04/msg01880.html
>   https://lists.gnu.org/archive/html/emacs-devel/2020-05/msg03051.html

(Thanks, I should have thought to add some context before forwarding.)

>> I guess there might be some people out there who will expect things to
>> keep working without double-quotes?  I have never used Babel, so I have
>> no idea…
>
> I don't know either, but the test does make me think so.  Hopefully
> Brandon, Bastien, or someone else will chime in if that's not the case.

My opinion should only carry so much weight since I don't use Babel, but
from a quick reading of the sources, I couldn't find other examples of
this (symbol → string) coercion.  The only other instances of
lists-of-strings I could find in testing/examples were

> :var a='("abc" "def")

That leads me to believe that the coercion was an unintended side-effect
of (format …).

Of course, backward compatibility alone would mandate keeping the
coercion.

> Could you send the first patch with a commit message tacked on?

Will do ASAP.

BTW, does the change from 44cb98fdb deserve an ORG-NEWS entry?

Re: Possible fix for :includes header argument in org-babel C source blocks

[Kévin Le Gouguec]

Hi,

Bastien  writes:

> Brandon Guttersohn  writes:
>
>> Hey all, I think I may have a small fix for executing C source blocks
>> in org-babel. Or, possibly just a bad case of user error.
>>
>> The issue (in emacs 27 with -q) is that it doesn't seem possible to
>> specify non-system header files with the :includes header argument.
>>
>> [...]
>>
>> The attached patch will wrap filenames in quotes if they do not begin
>> with "<", and works for me.
>
> Thanks for reporting this and for suggesting this patch, I think it is
> good enough.  I have applied it to the master branch of Org:
>
> https://code.orgmode.org/bzg/org-mode/commit/44cb98fdb6
>
> Best,

I think this commit might have broken test ob-C/string-var: running
"make test" on master (516c038e5) right now I get:

> Test ob-C/string-var condition:
> (wrong-type-argument sequencep )
>FAILED8/834  ob-C/string-var (0.004651 sec)

The following patch fixes the test:

diff --git a/lisp/ob-C.el b/lisp/ob-C.el
index c3e72c680..ae7b2ed1c 100644
--- a/lisp/ob-C.el
+++ b/lisp/ob-C.el
@@ -233,6 +233,7 @@ its header arguments."
 		;; includes
 		(mapconcat
 		 (lambda (inc)
+		   (when (symbolp inc) (setq inc (symbol-name inc)))
 		   (if (string-prefix-p "<" inc)
 		   (format "#include %s" inc)
 		 (format "#include \"%s\"" inc)))

I don't know if it's the best way forward; another way to make the test
pass is double-quoting "" and "" in ob-C-test.org:

diff --git a/testing/examples/ob-C-test.org b/testing/examples/ob-C-test.org
index 0faf630b9..efae02a19 100644
--- a/testing/examples/ob-C-test.org
+++ b/testing/examples/ob-C-test.org
@@ -38,7 +38,7 @@
 #+end_src
 
 #+source: string_var
-#+begin_src cpp :var q="word" :includes '( ) :results silent
+#+begin_src cpp :var q="word" :includes '("" "") :results silent
   std::cout << q << ' ' << std::strlen(q);
   return 0;
 #+end_src

I guess there might be some people out there who will expect things to
keep working without double-quotes?  I have never used Babel, so I have
no idea…

I hope this has not already been brought up; apologies if so.

Re: Setting org-todo-keywords through directory-local variables

[Kévin Le Gouguec]

Nicolas Goaziou  writes:

> This looks hackish.

Any bit in particular?  AFAICT hack-local-variables-hook is the expected
way to perform plumbing that might be affected by file/directory-local
variables.  It is used by e.g. shell-script-mode, cc-mode, markdown-mode
and AUCTeX, to name a few.  The docstring says:

> Major modes can use this to examine user-specified local variables
> in order to initialize other data structure based on them.

I think the buffer-file-name bit looks dodgy; I mainly did that to get
the unit tests to pass on this POC.

> Also, Org needs the STARTUP part early on, so you
> cannot really delay it.
>
> I /think/ the rest can be delayed, but I admit I'm not too sure either.

Right.  Now that I've looked at other major modes (especially
AUCTeX[1]), it seems a more conventional approach would be to

- keep the calls to org-set-regexps-and-options and
  org-set-font-lock-defaults where they are now,

- in hack-local-variables-hook, *if* file-local-variables-alist contains
  Org variables that affect those functions, and call them again to
  refresh regexps and fontification.

IIUC this would pretty much guarantee that things can only break for
weirdos like me who try to use directory-local variables.

> I think the expected way to do this is to add a SETUPFILE.

Thanks for the pointer!  Unless I'm misreading (info "(org) In-buffer
Settings"), I could move my SEQ_TODO settings there, but that wouldn't
work for org-todo-keyword-faces, right?


In light of your comments, and based on what I've seen in AUCTeX, I'm
attaching what I believe to be a less intrusive patch.  WDYT?


Thank you for taking the time to review this.  I'm not opposed to using
SETUPFILE (if I can handle org-todo-keyword-faces there); I'm just
wondering if this could be one more opportunity to have Org play nice
with other Emacs facilities (directory-local variables).


[1] 
https://git.savannah.gnu.org/cgit/auctex.git/tree/font-latex.el?h=release_12_2#n1435


diff --git a/lisp/org-faces.el b/lisp/org-faces.el
index d78b606ec..fc834f37d 100644
--- a/lisp/org-faces.el
+++ b/lisp/org-faces.el
@@ -291,7 +291,15 @@ determines if it is a foreground or a background color."
 	   (string :tag "Keyword")
 	   (choice :tag "Face   "
 		   (string :tag "Color")
-		   (sexp :tag "Face")
+		   (sexp :tag "Face"
+  :safe (lambda (x)
+  (cl-every
+   (lambda (pair)
+	 (let ((keyword (car pair))
+		   (face (cdr pair)))
+	   (and (stringp keyword)
+		(or (facep face) (listp face)
+   x)))
 
 (defface org-priority '((t :inherit font-lock-keyword-face))
   "Face used for priority cookies."
diff --git a/lisp/org.el b/lisp/org.el
index e577dc661..da38beb45 100644
--- a/lisp/org.el
+++ b/lisp/org.el
@@ -1945,7 +1945,13 @@ taken from the (otherwise obsolete) variable `org-todo-interpretation'."
 	 org-todo-interpretation-widgets))
 		  widget))
 		   (repeat
-		(string :tag "Keyword"))
+		(string :tag "Keyword")
+  :safe (lambda (x)
+  (cl-every
+   (lambda (seq)
+ (and (memq (car seq) '(sequence type))
+  (cl-every (lambda (i) (stringp i)) (cdr seq
+   x)))
 
 (defvar-local org-todo-keywords-1 nil
   "All TODO and DONE keywords active in a buffer.")
@@ -4358,10 +4364,9 @@ related expressions."
  (cons 'sequence (split-string value)))
    (append (cdr (assoc "TODO" alist))
 	   (cdr (assoc "SEQ_TODO" alist)
-		   (let ((d (default-value 'org-todo-keywords)))
-		 (if (not (stringp (car d))) d
-		   ;; XXX: Backward compatibility code.
-		   (list (cons org-todo-interpretation d)))
+		   (if (not (stringp (car org-todo-keywords))) org-todo-keywords
+		 ;; XXX: Backward compatibility code.
+		 (list (cons org-todo-interpretation org-todo-keywords))
 	  (dolist (sequence todo-sequences)
 	(let* ((sequence (or (run-hook-with-args-until-success
   'org-todo-setup-filter-hook sequence)
@@ -4909,7 +4914,18 @@ The following commands are available:
   ;; Try to set `org-hide' face correctly.
   (let ((foreground (org-find-invisible-foreground)))
 (when foreground
-  (set-face-foreground 'org-hide foreground
+  (set-face-foreground 'org-hide foreground)))
+
+  (add-hook 'hack-local-variables-hook #'org--process-local-variables nil t))
+
+(defun org--process-local-variables ()
+  "Refresh settings affected by file-local or directory-local variables."
+  (when
+  (let ((local-vars (mapcar #'car file-local-variables-alist)))
+	(or (memq 'org-todo-keywords local-vars)
+	(memq 'org-todo-keyword-faces local-vars)))
+(org-set-regexps-and-options)
+(org-set-font-lock-defaults)))
 
 ;; Update `customize-package-emacs-version-alist'
 (add-to-list 'customize-package-emacs-version-alist

Re: Setting org-todo-keywords through directory-local variables

[Kévin Le Gouguec]

Kévin Le Gouguec  writes:

> Can anyone confirm that this would (in principle) be the way forward, or
> tell me if I am missing something[3]?

I went ahead and cooked up a proof-of-concept patch, which

(1) adds safe-local-variable properties to org-todo-keywords and
org-todo-keyword-faces,

(2) stops applying default-value to org-todo-keywords,

(3) delays regexps/font-lock setups until after file- and dir-local
variables have been set.

While this patch contains a few things that make me weary[1], it solves
my use-case, and passes the current test suite with Emacs 26.3 and 28.

Does this look sound overall?  Does anyone have any idea what kind of
breakage might be slipping through the test suite?

Thank you for your time.


[1] - It's hard to feel confident that moving org-regexps-and-options
  and org-set-font-lock-defaults like this will not introduce
  regressions.

- Also, these safe-local-variable validation functions could
  probably use some unit tests.


diff --git a/lisp/org-faces.el b/lisp/org-faces.el
index d78b606ec..544563276 100644
--- a/lisp/org-faces.el
+++ b/lisp/org-faces.el
@@ -291,7 +291,15 @@ determines if it is a foreground or a background color."
 	   (string :tag "Keyword")
 	   (choice :tag "Face   "
 		   (string :tag "Color")
-		   (sexp :tag "Face")
+		   (sexp :tag "Face"
+  :safe (lambda (x)
+  (cl-every
+   (lambda (pair)
+ (pcase pair
+   (`(,keyword . ,face)
+(and (stringp keyword)
+ (or (facep face) (listp face))
+   x)))
 
 (defface org-priority '((t :inherit font-lock-keyword-face))
   "Face used for priority cookies."
diff --git a/lisp/org.el b/lisp/org.el
index e577dc661..7f4672058 100644
--- a/lisp/org.el
+++ b/lisp/org.el
@@ -1945,7 +1945,13 @@ taken from the (otherwise obsolete) variable `org-todo-interpretation'."
 	 org-todo-interpretation-widgets))
 		  widget))
 		   (repeat
-		(string :tag "Keyword"))
+		(string :tag "Keyword")
+  :safe (lambda (x)
+  (cl-every
+   (lambda (seq)
+ (and (memq (car seq) '(sequence type))
+  (cl-every (lambda (i) (stringp i)) (cdr seq
+   x)))
 
 (defvar-local org-todo-keywords-1 nil
   "All TODO and DONE keywords active in a buffer.")
@@ -4358,10 +4364,9 @@ related expressions."
  (cons 'sequence (split-string value)))
    (append (cdr (assoc "TODO" alist))
 	   (cdr (assoc "SEQ_TODO" alist)
-		   (let ((d (default-value 'org-todo-keywords)))
-		 (if (not (stringp (car d))) d
-		   ;; XXX: Backward compatibility code.
-		   (list (cons org-todo-interpretation d)))
+		   (if (not (stringp (car org-todo-keywords))) org-todo-keywords
+		 ;; XXX: Backward compatibility code.
+		 (list (cons org-todo-interpretation org-todo-keywords))
 	  (dolist (sequence todo-sequences)
 	(let* ((sequence (or (run-hook-with-args-until-success
   'org-todo-setup-filter-hook sequence)
@@ -4801,8 +4806,6 @@ The following commands are available:
  (vconcat (mapcar (lambda (c) (make-glyph-code c 'org-ellipsis))
 		  org-ellipsis)))
 (setq buffer-display-table org-display-table))
-  (org-set-regexps-and-options)
-  (org-set-font-lock-defaults)
   (when (and org-tag-faces (not org-tags-special-faces-re))
 ;; tag faces set outside customize force initialization.
 (org-set-tag-faces 'org-tag-faces org-tag-faces))
@@ -4909,7 +4912,16 @@ The following commands are available:
   ;; Try to set `org-hide' face correctly.
   (let ((foreground (org-find-invisible-foreground)))
 (when foreground
-  (set-face-foreground 'org-hide foreground
+  (set-face-foreground 'org-hide foreground)))
+
+  ;; For file-visiting buffers, delay some setup until after
+  ;; file-local and directory-local variables have been set.
+  (if (buffer-file-name)
+  (progn
+	(add-hook 'hack-local-variables-hook 'org-set-regexps-and-options 1 t)
+	(add-hook 'hack-local-variables-hook 'org-set-font-lock-defaults 1 t))
+(org-set-regexps-and-options)
+(org-set-font-lock-defaults)))
 
 ;; Update `customize-package-emacs-version-alist'
 (add-to-list 'customize-package-emacs-version-alist

Re: issue tracker?

[Kévin Le Gouguec]

Anthony Carrico  writes:

> On 5/21/20 10:18 AM, Anthony Carrico wrote:
>> which is a big ask for users.
>
> ... given that the community expressed that it would like to interact on
> a mailing list without other user facing tooling.

AFAICT, the only thing users have to do to participate in Debbugs
conversations is keeping bugnum...@debbugs.gnu.org in the CC list;
maintainers handle control commands themselves (e.g. tagging, merging,
closing).

Re: issue tracker?

[Kévin Le Gouguec]

Nicolas Goaziou  writes:

> - As pointed out, Org has a bug tracker : Emacs' Debbugs. See
>   . Org users do not send bugs
>   through it much.

(In the event that they do, should whoever follows bug-gnu-emacs refer
these users to emacs-orgmode?)

> - Considering the previous point, I doubt switching to a bug tracker
>   today would help handling more bug reports. It will induce more work,
>   though. For example, some triage happens currently on the ML: if
>   a so-called bug report is clearly a misunderstanding, someone here
>   often helps the OP without the developers interfering. This never
>   happens in the bug tracker Org has actually.

I wouldn't be so categoric; it is my impression that there are a number
of lurkers on bug-gnu-emacs who skim through reports that touch on
topics they are interested in[1], and will occasionally pop up to help
the OP.

At least I know I try to do so (cf. bug#41364, about org-mode as it
happens).


[1] I'm saying this based on off-list discussions that sometimes sprout
off bug such reports…  and based on the fact that I do that myself.

Re: issue tracker?

[Kévin Le Gouguec]

"James R Miller"  writes:

>> I think issue tracking could happen on a mailing list. If you tag an 
>> issue's subject line with OPEN: or CLOSE:, a bot could mail a summary of 
>> the OPEN: issues to the list periodically (in theory).
>
> Something like that would be nice; the bot could even store such info in an 
> org file that could be exported the html occasionally too. 

I think you've just described, in order:

- Debbugs (the issue tracking software),

- bug-gnu-em...@gnu.org (the mailing list),

- cont...@debbugs.gnu.org and bugnumber-d...@debbugs.gnu.org (the bots
  to contact to tag, close or reopen bugs),

- the debbugs-org function from the debbugs.el package (the current bug
  list formatted in an Org buffer),

- https://debbugs.gnu.org (the current bug list formatted as HTML).

Setting org-todo-keywords through directory-local variables

[Kévin Le Gouguec]

Hello,

I'd like to set org-todo-keywords and org-todo-keyword-faces through
directory-local variables, to get rid of duplicate #+SEQ_TODO lines in
my Org files[1].

Right now I see two obstacles for this to work:

(1) org-set-regexps-and-options, which sets up a bunch of TODO-related
machinery, insists on using (default-value 'org-todo-keywords),

(2) this function is called in the major mode function, which IIUC means
that directory-local values have not been applied yet.

The first obstacle looks like it can be easily removed[2]; the second
obstacle looks more substantial.  It is trivially side-stepped by
sticking (hack-local-variables) at the top of org-mode; to my untrained
eye, it looks like TRT would rather be for Org to add
org-set-regexps-and-options to hack-local-variables-hook.

This sounds like a risky change though: I imagine that a lot of what
happens in the major mode function depends on what
org-set-regexps-and-options sets up, and would therefore need to be
moved to this hook as well.  Figuring which parts should be moved seems
like a non-trivial task that might introduce some regressions…


Can anyone confirm that this would (in principle) be the way forward, or
tell me if I am missing something[3]?


Thank you for your time.


[1] For example:

#+begin_src elisp
((org-mode
  . ((org-todo-keywords
  . ((sequence "REPORT" "REPORTED" "WAITING" "FIXED")
 (sequence "CANCELED")))
 (org-todo-keyword-faces
  . (("REPORT" . org-todo)
 ("REPORTED" . warning)
 ("WAITING" . warning)
 ("FIXED" . org-done)
 ("CANCELED" . shadow))
#+end_src

I'd like that so much that I went through the trouble of writing
safe-local-variable predicates for these variables:

#+begin_src elisp
(put 'org-todo-keywords
 'safe-local-variable
 (lambda (x)
   (cl-every
(lambda (seq)
  (and (memq (car seq) '(sequence type))
   (cl-every (lambda (i) (stringp i)) (cdr seq
x)))

(put 'org-todo-keyword-faces
 'safe-local-variable
 (lambda (x)
   (cl-every
(lambda (pair)
  (pcase pair
(`(,keyword . ,face)
 (and (stringp keyword)
  (or (facep face) (listp face))
x)))
#end_src

[2] I tried to go through org.el's history, but I could not find a
rationale for using default-value.

[3] Alternatively, maybe the answer is as simple as "Org documents
should be self-sufficient; keywords should be explicitly set in
#+SEQ_TODO lines"; that wouldn't sound right though, since
org-todo-keywords is customizable.

Re: [bug] error on RET in read-only buffer with org-return-follows-link set to t

[Kévin Le Gouguec]

Nicolas Goaziou  writes:

> Hello,
>
> Gregor Zattler  writes:
>
>> with `org-return-follows-link` set to `t` in a read-only
>> buffer I now get a `Buffer is read-only: #> notmuch-startpage.org>` error when pressing ENTER/RETURN
>> with point on an org-mode link.
>
> Fixed. Thank you.

Thanks for fixing this Nicolas, and for adding a unit test.

Shouldn't the call to org-return be wrapped in (call-interactively …)
though?  Since the problem was with the interactive spec, org-return
will work (i.e. follow links) in read-only buffers if called
programmatically, so the test will fail to catch a regression if I'm not
mistaken.

IOW: if I revert the fix, the new test still passes.

Re: [bug] error on RET in read-only buffer with org-return-follows-link set to t

[Kévin Le Gouguec]

Gregor Zattler  writes:

> Dear Kévin, org-mode developers, 
>
> with `org-return-follows-link` set to `t` in a read-only
> buffer I now get a `Buffer is read-only: # notmuch-startpage.org>` error when pressing ENTER/RETURN
> with point on an org-mode link.

Oh, right, I added '*' to org-return's interactive spec because I was
mimicking newline's; I had not considered the link-following case.

Should be a simple matter of removing this '*' if I'm not mistaken:

diff --git a/lisp/org.el b/lisp/org.el
index be1d1c701..339418314 100644
--- a/lisp/org.el
+++ b/lisp/org.el
@@ -17702,7 +17702,7 @@ a timestamp or a link, call `org-open-at-point'.  However, it
 will not happen if point is in a table or on a \"dead\"
 object (e.g., within a comment).  In these case, you need to use
 `org-open-at-point' directly."
-  (interactive "*i\nP\np")
+  (interactive "i\nP\np")
   (let ((context (if org-return-follows-link (org-element-context)
 		   (org-element-at-point
 (cond

> I use this dozens of times a day and it would be convenient
> if it was possible to resurrect the old behaviour.

Right, terribly sorry for this blunder.  I'll try to followup with a
unit test to make sure such a regression doesn't creep in again.

Re: [PATCH] Make RET and C-j obey `electric-indent-mode' in org-mode

[Kévin Le Gouguec]

Kévin Le Gouguec  writes:

> (I hope I got that right.)

Except I forgot the explanatory comment, and I left a typo in the
docstring.  Ahem.

diff --git a/lisp/org-compat.el b/lisp/org-compat.el
index 2b35535fa..caaf5ce58 100644
--- a/lisp/org-compat.el
+++ b/lisp/org-compat.el
@@ -102,6 +102,12 @@ is nil)."
   (defun org-time-convert-to-list (time)
 (seconds-to-time (float-time time
 
+;; `newline-and-indent' did not take a numeric argument before 27.1.
+(if (version< emacs-version "27")
+(defsubst org-newline-and-indent ( _arg)
+  (newline-and-indent))
+  (defalias 'org-newline-and-indent #'newline-and-indent))
+
 
 ;;; Emacs < 26.1 compatibility
 
diff --git a/lisp/org.el b/lisp/org.el
index 8ad437a20..142bfb999 100644
--- a/lisp/org.el
+++ b/lisp/org.el
@@ -17649,12 +17649,12 @@ call `open-line' on the very first character."
 
 (defun org--newline (indent arg interactive)
   "Call `newline-and-indent' or just `newline'.
-If INDENT is non-nil, call `newline-and-indent' to indent
-unconditionally; otherwise, call `newline' with ARG and
-INTERACTIVE, which can trigger indentation if
+If INDENT is non-nil, call `newline-and-indent' with ARG (if
+supported) to indent unconditionally; otherwise, call `newline'
+with ARG and INTERACTIVE, which can trigger indentation if
 `electric-indent-mode' is enabled."
   (if indent
-  (newline-and-indent)
+  (org-newline-and-indent arg)
 (newline arg interactive)))
 
 (defun org-return ( indent arg interactive)

Re: [PATCH] Make RET and C-j obey `electric-indent-mode' in org-mode

[Kévin Le Gouguec]

Nicolas Goaziou  writes:

> AFAICT, `newline-and-indent' doesn't accept any argument. Keeping it
> introduces a build warning and test failures. Hence the removal.
>
> Since you were calling it with an argument I assume this may be
> a novelty in Emacs 27.

Wow, you're right.  That caught me off-guard.

>However Org still supports Emacs 24.4. If that's
> the case, we need an additional compatibility layer to support both
> cases. WDYT?

I don't know if we want to jump through these hoops for a feature that
people have done without so far?  FWIW though, the following patch seems
to work ("make test" works with both 26.3 and 28.0 on my end):

diff --git a/lisp/org-compat.el b/lisp/org-compat.el
index 2b35535fa..ed12b9d18 100644
--- a/lisp/org-compat.el
+++ b/lisp/org-compat.el
@@ -102,6 +102,11 @@ is nil)."
   (defun org-time-convert-to-list (time)
 (seconds-to-time (float-time time
 
+(if (version< emacs-version "27")
+(defsubst org-newline-and-indent ( _arg)
+  (newline-and-indent))
+  (defalias 'org-newline-and-indent #'newline-and-indent))
+
 
 ;;; Emacs < 26.1 compatibility
 
diff --git a/lisp/org.el b/lisp/org.el
index 8ad437a20..57e78599f 100644
--- a/lisp/org.el
+++ b/lisp/org.el
@@ -17649,12 +17649,12 @@ call `open-line' on the very first character."
 
 (defun org--newline (indent arg interactive)
   "Call `newline-and-indent' or just `newline'.
-If INDENT is non-nil, call `newline-and-indent' to indent
-unconditionally; otherwise, call `newline' with ARG and
-INTERACTIVE, which can trigger indentation if
+If INDENT is non-nil, call `newline-and-indent' with ARG (if
+supported) )to indent unconditionally; otherwise, call `newline'
+with ARG and INTERACTIVE, which can trigger indentation if
 `electric-indent-mode' is enabled."
   (if indent
-  (newline-and-indent)
+  (org-newline-and-indent arg)
 (newline arg interactive)))
 
 (defun org-return ( indent arg interactive)

(I hope I got that right.)

> Meanwhile, I fixed the docstring. Thanks!

And thanks again.

Re: [PATCH] Make RET and C-j obey `electric-indent-mode' in org-mode

[Kévin Le Gouguec]

Stefan Monnier  writes:

>> +(defmacro org-test-with-minor-mode (mode state  body)
>> +  "Run BODY after setting MODE to STATE.
>> +Restore MODE to its former state afterward."
>> +  (declare (debug (sexp sexp body)) (indent 2))
>> +  `(let ((old-state ,mode))
>> +   (,mode (if ,state 1 0))
>> +   ,@body
>> +   (,mode (if old-state 1 0
>
> This should probably use `unwind-protect` in case `body` exits
> non-locally.
> [ And also, for buffer-local minor modes, we should try and be careful
>   to restore the mode in the same buffer, tho this can be pushed as
>   a responsability of `body`.  ]

Thanks for confirming my nagging feeling that this macro was a bit too
naive :)

Nicolas actually ditched it and turned on/off electric-indent-local-mode
in the temporary buffer where the Org commands are run; that should take
care of these issues IIUC.

Re: [PATCH] Make RET and C-j obey `electric-indent-mode' in org-mode

[Kévin Le Gouguec]

Nicolas Goaziou  writes:

> I fixed a typo and applied your patch.

Thank you for fixing the typo in ORG-NEWS.

I see you've removed ARG from the call to `newline-and-indent'; I don't
have a strong opinion about this (though I don't see a reason not to
keep it), but I guess the docstring of `org--newline' should be changed
to match?

> However, when I have to reproduce a failing test, I don't even want to
> think about the recipe and rather concentrate on the results. Hence,
> I expect `should' macro's body to be self-sufficient. Therefore,
> I skipped this part of the patch.

Fair enough; toggling the local variant of the minor mode is probably
cleaner anyway!


Thanks for the review, and for applying.