[PATCH] lisp/org-agenda.el: Fix filter preset problem for sticky agenda

[Liu Hui]

Hi,

This patch fixes the bug originally reported in
https://list.orgmode.org/59e02fb6.1462370a.fffe8.5...@mx.google.com/
with the following reproducing recipe. The bug still exists in latest
org-mode.

> To reproduce, take the following org file:
>
> ---8<---
> * Foo
>   :PROPERTIES:
>   :CATEGORY: foo
>   :END:
> ** TODO Foo todo one
>DEADLINE: <2017-10-13 Fri>
>- foo 1
>
> * Bar
>   :PROPERTIES:
>   :CATEGORY: bar
>   :END:
> ** TODO Bar todo one
>DEADLINE: <2017-10-13 Fri>
>- bar 1
> --->8---
>
> And the following elisp file to setup a minimal environment:
>
> ---8<---
> (require 'org)
> (setq org-agenda-files (list (expand-file-name "repro.org")))
> (setq org-agenda-sticky t)
> (setq org-agenda-custom-commands
>   '(("f" "foo"
>  ((agenda "" ())
>   (tags-todo "+CATEGORY=\"foo\"" ()))
>  ((org-agenda-category-filter-preset '("+foo"
>
> ("b" "bar"
>  ((agenda "" ())
>   (tags-todo "+CATEGORY=\"bar\"" ()))
>  ((org-agenda-category-filter-preset '("+bar"))
>
> (global-set-key (kbd "C-c a") #'org-agenda)
> --->8---
>
> Then run (something like):
>
> emacs -nw -Q -L ~/git/org-mode/lisp -l repro.el repro.org
>
> Followed by:
>
> C-c a f
> C-c a b
> C-x o
> r
>
> I would expect pressing `r' in the `*Org Agenda(f)*' buffer would
> keep the original `org-agenda-category-filter-preset' and that the
> preset in `*Org Agenda(b)*' should have no bearing on it.  But, as you
> can (probably) see from the mode line, the filter is set to "+bar", and
> refreshing the buffer causes all the TODOs to be lost because it now has
> the wrong filter.

Because the filter presets are stored as symbol properties, they
cannot be used with multiple sticky agenda buffers. The patch fixes
the problem by adding a new variable `org-agenda-filters-preset' for
getting and setting per-buffer filter presets. I have signed the FSF
copyright assignment paper.

Best,
Hui

>From 4fc03d86ab1df761ab26655f043b05b449b68716 Mon Sep 17 00:00:00 2001
From: Liu Hui 
Date: Tue, 4 Oct 2022 11:12:41 +0800
Subject: [PATCH] lisp/org-agenda.el: Fix filter preset problem for sticky
 agenda

* lisp/org-agenda.el (org-agenda-local-vars):
(org-agenda-filters-preset): Add a new variable
`org-agenda-filters-preset' for storing per-buffer filter presets.
(org-agenda-filter-any):
(org-agenda-prepare):
(org-agenda-finalize):
(org-agenda-redo):
(org-agenda-filter-by-tag):
(org-agenda-filter-make-matcher):
(org-agenda-set-mode-name):
(org-agenda-reapply-filters): Use `org-agenda-filters-preset' for
getting and setting per-buffer filter presets, rather than modifying
the global symbol property.
---
 lisp/org-agenda.el | 91 --
 1 file changed, 40 insertions(+), 51 deletions(-)

diff --git a/lisp/org-agenda.el b/lisp/org-agenda.el
index 2b56dd0fb..7ee0a6ad1 100644
--- a/lisp/org-agenda.el
+++ b/lisp/org-agenda.el
@@ -2276,6 +2276,7 @@ When nil, `q' will kill the single agenda buffer."
 org-agenda-top-headline-filter
 org-agenda-regexp-filter
 org-agenda-effort-filter
+org-agenda-filters-preset
 org-agenda-markers
 org-agenda-last-search-view-search-was-boolean
 org-agenda-last-indirect-buffer
@@ -3808,6 +3809,9 @@ the entire agenda view.  In a block agenda, it will not work reliably to
 define a filter for one of the individual blocks.  You need to set it in
 the global options and expect it to be applied to the entire view.")
 
+(defvar org-agenda-filters-preset nil
+  "Preset of filters, which becomes buffer-local in org-agenda buffers.")
+
 (defconst org-agenda-filter-variables
   '((category . org-agenda-category-filter)
 (tag . org-agenda-tag-filter)
@@ -3818,7 +3822,7 @@ the global options and expect it to be applied to the entire view.")
   "Is any filter active?"
   (cl-some (lambda (x)
 	 (or (symbol-value (cdr x))
-		 (get :preset-filter x)))
+ (assoc-default (car x) org-agenda-filters-preset)))
 	   org-agenda-filter-variables))
 
 (defvar org-agenda-category-filter-preset nil
@@ -3927,10 +3931,6 @@ FILTER-ALIST is an alist of filters we need to apply when
 			(cat . ,org-agenda-category-filter))
 (if (org-agenda-use-sticky-p)
 	(progn
-	  (put 'org-agenda-tag-filter :preset-filter nil)
-	  (put 'org-agenda-category-filter :preset-filter nil)
-	  (put 'org-agenda-regexp-filter :preset-filter nil)
-	  (put 'org-agenda-effort-filter :preset-filter nil)
 	  ;; Popup existing buffer
 	  (org-agenda-prepare-window (get-buffer org-agenda-buffer-name)
  filter-alist)
@@ -3938,14 +3938,6 @@ FILTER-ALIST is an alist of filters we need to apply when
 	  (or org-agenda-multi (org-agenda-fit-window-to-buffer))
 	  (throw 'exit "Sticky Agenda buffer, use `r' to refresh"))
   (setq org-todo-keywords-for-agenda nil)
-  (put 'org-agenda-tag-filter :preset-filter
-	   org-agenda-tag-filter-preset)
-  (put 'org-agenda-category-filter :preset-filter
-	   

Re: [PATCH] Unit-test for Please add support for dlangs packagemanager to ob-C.el

[Ihor Radchenko]

Christian Köstlin  writes:

>> If the programming language is using its own package manager, is it any
>> different? Or do you refer to something else?
>>
> Most of the packages that a programming language like rust/crates,
> ruby/rubygems
> or dlang/dub-packages provide will probably never make it into the package
> management system of the operating system. Also the programming language
> package management usually has finer grained dependency management.
> But it's just a theoretical question as no other ob-* integration
> integrates a package
> manager at the momen.

If we do anything about such integration, we should not enable such
testing automatically. What we certainly do not want is unsuspecting
user running make test and getting some packages installed into his/her
computer.

However, the side effects requiring installing certain packages to the
OS might be shielded behind Makefile customization that will be set to
non-nil in CI tests. Of course, such customization should be documented
in testing/README.

>>> Throwing the 'missing-test-dependency has exactly same result, but also
>> >> allows making our test suite indicate such issues simply by altering
>> >> the 'missing-test-dependency catch code. That's why prefer
>> >> 'missing-test-dependency approach.
>> >>
>> > Nice ... I just tested the aforementioned `(ert--skip-unless
>> > (executable-find "dub")`
>> > which also works nicely. I could post another patch that updates the
>> whole
>> > test-ob-C.el with one of those strategies.
>> I am confused. Why do you need to use internal ert function?
>>
> I just tried what was proposed in one of the other emails ...

I do not blame you. Do not take it for granted that every Org
contributor, maintainer, or mailing list dweller is always right.

In this particular case, I disagree with Max and I tried to put
arguments why I disagree. If you think that I am wrong, you can refute
my arguments.

>> Note that we only have skip-unless feature in
>> ob-shell/remote-with-stdin-or-cmdline, which is probably just an
>> incident.
>>
>> Most of the test files use (signal 'missing-test-dependency ...)
>> For example, at the very top of test-ob-C.el:
>>
>> (unless (featurep 'ob-C)
>>   (signal 'missing-test-dependency "Support for C code blocks"))
>>
>> We even have a dedicated function to test if executable exists:
>> `org-test-for-executable':
>>
>> (org-test-for-executable "awk")
>> (unless (featurep 'ob-awk)
>>   (signal 'missing-test-dependency "Support for Awk code blocks"))
>>
> yes .. the difference is, that with org-test-for-executable i get a failing
> test (same for `signal 'missing-test-dependency`)
> with the ert function one gets a skipped test.

I guess you tried to throw 'missing-test-dependency _inside_ the test.
AFAIK, 'missing-test-dependency is caught when test-*.el file is being
loaded, not when the ERT tests are being executed. See `org-test-load'.

I think you can create a separate file like test-ob-D.el and run
(org-test-for-executable ...) at the top-level there.

> There is only one most important policy - if we recommend anything
>> publicly, it must be Free software. The detailed rules are covered by
>> https://www.gnu.org/prep/maintain/maintain.html
>
> yes ... still i am now lawyer enough to e.g. judge if gitlab/github is
> according
> to that.

We generally avoid GitHub because of some of their practices (see
https://www.fsf.org/search?SearchableText=github).

However, FSF policy is of a secondary importance here.

WRT CI tooling, we just prefer something Org maintainers do use.
Bastien, the chief maintainer, prefers Sourcehut.
Since he maintains the Org infrastructure, we should also prefer
Sourcehut, unless there are strong reasons to use other service. (Do not
hesitate to name those reasons, if you know any)

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at .
Support Org development at ,
or support my work at 

Re: [PATCH 2-v4] New: auto display inline images under subtree when `org-cycle'.

[Ihor Radchenko]

"Christopher M. Miles"  writes:

> I think the patch is finished. Please review, Ihor. Thanks for helping.

Thanks for the update!
I have cleaned the manual and ORG-NEWS wordings.

Upon reading the exported version of the manual, I realized that the
part in 2.2.1 Global and local cycling section sounds very awkward and
that we do not mention other cycle hooks like hiding archived
subtrees there. So, I decided to remove it.

I also rewrote the other part in a bit more direct tone and moved the
index entry.

(In general, it is useful to try reading the exported version of the
manual with the additions to check if things are still coherent - manual
is not just a collection of independent statements; we need to make sure
that it is not confusing the new readers).

I also updated the commit message adding the CHANGELOG entries about the
manual and ORG-NEWS.

However, upon testing your patch, tests are still failing on my side.
Can you please investigate?

I am attaching the updated version of the patch with my amendments.

Best,
Ihor

>From ab303eb5c007822d045f5e6ba3f00779c0abf1bc Mon Sep 17 00:00:00 2001
Message-Id: 
From: stardiviner 
Date: Wed, 28 Sep 2022 20:46:52 +0800
Subject: [PATCH] org.el: Support auto display inline images when cycling

* lisp/org.el (org-toggle-inline-images): Support region.
(org-display-inline-images): Fix refresh argument logic.
(org-remove-inline-images): Support region.

* lisp/org-keys.el (org-toggle-inline-images): Update arguments.

* lisp/org-cycle.el (org-cycle-inline-images-display): Add new option to
control whether auto display inline images when cycling.
(org-cycle-display-inline-images): Add new hook function to auto display
inline images when cycling.
(org-cycle-hook): Add `org-cycle-display-inline-images' into cycling
hook by default.

* testing/lisp/test-org-fold.el (test-org-fold/org-fold-display-inline-images):
Add test for inline image displaying when cycling.

* doc/org-manual.org (Exporting):
* etc/ORG-NEWS: Document the new option.
---
 doc/org-manual.org|   6 ++
 etc/ORG-NEWS  |   7 +++
 lisp/org-cycle.el |  38 -
 lisp/org-keys.el  |   2 +-
 lisp/org.el   |  21 ---
 .../images/Org mode logo mono-color.png   | Bin 0 -> 7523 bytes
 testing/lisp/test-org-fold.el |  52 ++
 7 files changed, 115 insertions(+), 11 deletions(-)
 create mode 100755 testing/examples/images/Org mode logo mono-color.png

diff --git a/doc/org-manual.org b/doc/org-manual.org
index a5fec8cf9..ec838df97 100644
--- a/doc/org-manual.org
+++ b/doc/org-manual.org
@@ -11470,6 +11470,12 @@ ** Images
   - When set to nil, try to get the width from an =#+ATTR.*= keyword
 and fall back on the original width if none is found.
 
+
+#+vindex: org-cycle-inline-images-display
+Inline images can also be displayed when cycling the folding state.
+When custom option ~org-cycle-inline-images-display~ is set, the
+visible inline images under subtree will be displayed automatically.
+
 ** Captions
 :PROPERTIES:
 :DESCRIPTION: Describe tables, images...
diff --git a/etc/ORG-NEWS b/etc/ORG-NEWS
index 4728528f8..75a77378a 100644
--- a/etc/ORG-NEWS
+++ b/etc/ORG-NEWS
@@ -307,6 +307,13 @@ to pass the contents of a named code block as a string argument.
 The new property =ORG-IMAGE-ACTUAL-WIDTH= can override the global
 variable ~org-image-actual-width~ value for inline images display width.
 
+*** Outline cycling can now include inline image visibility
+
+New ~org-cycle-hook~ function ~org-cycle-display-inline-images~ for
+auto-displaying inline images in the visible parts of the subtree.
+This behavior is controlled by new custom option
+~org-cycle-inline-images-display~.
+
 *** New ~org-babel-tangle-finished-hook~ hook run at the very end of ~org-babel-tangle~
 
 This provides a proper counterpart to ~org-babel-pre-tangle-hook~, as
diff --git a/lisp/org-cycle.el b/lisp/org-cycle.el
index 656ca83f2..212d25a44 100644
--- a/lisp/org-cycle.el
+++ b/lisp/org-cycle.el
@@ -208,8 +208,9 @@ (defcustom org-cycle-pre-hook nil
   :type 'hook)
 
 (defcustom org-cycle-hook '(org-cycle-hide-archived-subtrees
-		   org-cycle-show-empty-lines
-		   org-cycle-optimize-window-after-visibility-change)
+org-cycle-show-empty-lines
+org-cycle-optimize-window-after-visibility-change
+org-cycle-display-inline-images)
   "Hook that is run after `org-cycle' has changed the buffer visibility.
 The function(s) in this hook must accept a single argument which indicates
 the new state that was set by the most recent `org-cycle' command.  The
@@ -229,6 +230,13 @@ (defcustom org-cycle-open-archived-trees nil
   :group 'org-cycle
   :type 'boolean)
 
+(defcustom org-cycle-inline-images-display nil
+  "Non-nil means auto display inline images under subtree when 

Re: Code block syntax highlighting in async export

[Timothy]

Hi Ihor,

>> Try it 
>
> You are right. With default preset, engrave-faces does not need to query
> Emacs about face colors and attributes and should not depend on the
> `display-graphic-p’.

Yep, it works because it’s basically implementing its own face-resolving layer.
This was originally done to allow exporting in a theme different to the one
currently being used as seamlessly as possible, it working with batch mode was a
bonus.

For the information of anyone else following the thread, you can easily add new
presets by creating an elisp setup file for batch mode and running `C-u M-x
engrave-faces-use-theme' in it — code defining a new preset will then be
generated and inserted at point.

All the best,
Timothy

-- 
Timothy (‘tecosaur’/‘TEC’), Org mode contributor.
Learn more about Org mode at .
Support Org development at ,
or support my work at .

Re: Code block syntax highlighting in async export

[Ihor Radchenko]

Timothy  writes:

> Hi Ihor,
>
>> I would not be so sure. If the issue is major-mode not doing
>> fontification in batch mode, engrave-faces cannot do much.
>
> Try it 

You are right. With default preset, engrave-faces does not need to query
Emacs about face colors and attributes and should not depend on the
`display-graphic-p'.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at .
Support Org development at ,
or support my work at 

Re: Code block syntax highlighting in async export

[Ihor Radchenko]

David Lukeš  writes:

>> Could you please provide an example?
>
>> Certainly more details is necessary, even export format (backend) is not
>> specified.
>
> Sorry for that! I somehow thought that the issue would sound familiar
> to more seasoned Org users, and someone would come back with either
> "yeah, you need to do X" or "yeah, that's a known limitation".
>
> The export backend is HTML. In case anyone wants to try reproducing
> this, I made a minimal working example here:
> https://github.com/dlukes/org-html-a-sync-export-syntax-highlighting

Thanks for the easy reproducer!
I played around with the export there, and I think I have found the
cause.

As expected, the problem originates from batch mode of Emacs
operation.

ox-html relies upon htmlize-region -> htmlize-buffer-1 ->
htmlize-make-face-map -> htmlize-face-to-fstruct ->
htmlize-face-to-fstruct-1.

In synchronous export `htmlize-face-to-fstruct-1' returns:

font-lock-comment-face -> #s(htmlize-fstruct "#b2" nil nil nil nil nil nil 
nil "org-comment")
font-lock-comment-delimiter-face -> #s(htmlize-fstruct "#b2" nil nil nil 
nil nil nil nil "org-comment-delimiter")
font-lock-doc-face -> #s(htmlize-fstruct "#8b2252" nil nil nil nil nil nil nil 
"org-doc")
font-lock-keyword-face -> #s(htmlize-fstruct "#a020f0" nil nil nil nil nil nil 
nil "org-keyword")
font-lock-function-name-face -> #s(htmlize-fstruct "#ff" nil nil nil nil 
nil nil nil "org-function-name")

In asynchronous export:

default -> #s(htmlize-fstruct "#00" "#ff" 1 nil nil nil nil nil 
"org-default")
font-lock-comment-face -> #s(htmlize-fstruct nil nil nil t t nil nil nil 
"org-comment")
font-lock-comment-delimiter-face -> #s(htmlize-fstruct nil nil nil t t nil nil 
nil "org-comment-delimiter")
font-lock-doc-face -> #s(htmlize-fstruct nil nil nil nil t nil nil nil 
"org-doc")
font-lock-keyword-face -> #s(htmlize-fstruct nil nil nil t nil nil nil nil 
"org-keyword")
font-lock-function-name-face -> #s(htmlize-fstruct nil nil nil t nil nil nil 
nil "org-function-name")

-

What can we do on the Org side to fix the issue?

One way could be avoiding batch execution altogether, like in the
attached patch. However, it will break our process sentinel watching for
the export results. Moreover, I am not sure how it will work when export
is done from terminal Emacs. Also, I know no way to _not_ create an
extra Emacs frame during such export - I imagine that a new frame (even
minimized) might be annoying for the users.

Maybe we should request Emacs to add --graphical-batch mode feature that
will preserve graphical features while not interfering with the WM?

>From 8bc0a66275ca6ffdf142e8e2c2b38d79923531d8 Mon Sep 17 00:00:00 2001
Message-Id: <8bc0a66275ca6ffdf142e8e2c2b38d79923531d8.1664855997.git.yanta...@gmail.com>
From: Ihor Radchenko 
Date: Tue, 4 Oct 2022 11:58:45 +0800
Subject: [PATCH] ox: Do not disable graphical Emacs capabilities during async
 export

* lisp/ox.el (org-export-async-start): Run Emacs in possibly iconized
window instead of batch mode.  This way, font colors can be processed
by htmlize during async export.
---
 lisp/ox.el | 8 ++--
 1 file changed, 6 insertions(+), 2 deletions(-)

diff --git a/lisp/ox.el b/lisp/ox.el
index 42204ffde..a31459632 100644
--- a/lisp/ox.el
+++ b/lisp/ox.el
@@ -6432,7 +6432,9 @@ (defun org-export-async-start  (fun body)
 (funcall ',copy-fun)
 (restore-buffer-modified-p nil)
 ;; Sexp to evaluate in the buffer.
-(print ,body)))
+(print ,body)
+;; Kill Emacs at the end.
+(kill-emacs 0)))
  nil temp-file nil 'silent))
   ;; Start external process.
   (let* ((process-connection-type nil)
@@ -6444,7 +6446,9 @@ (defun org-export-async-start  (fun body)
 		(list "org-export-process"
 		  proc-buffer
 		  (expand-file-name invocation-name invocation-directory)
-		  "--batch")
+		  ;; "--batch"
+  "--iconic"
+  )
 		(if org-export-async-init-file
 		(list "-Q" "-l" org-export-async-init-file)
 		  (list "-l" user-init-file))
-- 
2.35.1



-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at .
Support Org development at ,
or support my work at 

Re: Concatenating results

[Felix Dorner]

On Mon, Oct 3, 2022 at 11:28 AM Ihor Radchenko  wrote:

>
> #+begin_src emacs-lisp :noweb yes
>
> Thank you Ihor.
Felix



-- 
Linux. The choice of a GNU generation.

Re: Post-process table without changing result for empty table(/list)

[Ihor Radchenko]

Jonas Bernoulli  writes:

> It used to behave like that before 51a628bc5efc from 2009, which started
> turning all symbols, including nil, into strings, but without giving any
> reason why that should be done.
>
> It has worked like this for a long time now, so reverting that is
> probably not feasible in the short run.  However, I feel it would
> make sense to change now how nil/'() is treated.  Currently it is
> being treated as the symbol nil, but IMO it would make more sense
> to treat it as the empty list.  That could be achieved with
>
> diff --git a/lisp/ob-ref.el b/lisp/ob-ref.el
> index b79e47900..2b4a16aea 100644
> --- a/lisp/ob-ref.el
> +++ b/lisp/ob-ref.el
> @@ -199,7 +199,7 @@ (defun org-babel-ref-resolve (ref)
> (org-babel-execute-src-block nil info 
> params
>  (error "Reference `%s' not found in this buffer" ref
>  (cond
> - ((symbolp result) (format "%S" result))
> + ((and result (symbolp result)) (format "%S" result))
>   ((and index (listp result))
>(org-babel-ref-index-list index result))
>   (t result)

Looks reasonable.
Could you please prepare a patch and possibly also add a test that
covers your use-case to testing/lisp/test-ob.el?
See https://orgmode.org/worg/org-contribute.html

> In the long run you might want to consider not turning any symbols
> into strings, at least not when the "regular" block as well as the
> post-processing block both use elisp.

This may be tricky. Introducing any kind of special case will make the
code fragile. We should better make things as generic as possible.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at .
Support Org development at ,
or support my work at 

Re: Numbered footnotes in the manual interfere with diff

[Ihor Radchenko]

Bastien  writes:

> Hi Ihor,
>
> Ihor Radchenko  writes:
>
>> Should we just inline the one-sentence footnotes?
>
> Yes, this would be a progress already.

Done.
https://git.savannah.gnu.org/cgit/emacs/org-mode.git/commit/?id=0641ece57b9d980b63e3a3bb6dc4d467eff3051b

I also documented this in doc/Documentation_Standards.org

>> Also, there is one footnote that is indexed (does it even work?):
>
> AFACT it does not work and I think it should not work, we can remove
> these index entries.

Instead of removing, I moved the entries to the footnote reference.
https://git.savannah.gnu.org/cgit/emacs/org-mode.git/commit/?id=180966c645547bf8e0f78ea28d63a686a286d2b2

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at .
Support Org development at ,
or support my work at 

Re: Code block syntax highlighting in async export

[David Lukeš]

> I have realized that
> https://orgmode.org/worg/org-contrib/babel/languages/ob-doc-elisp.html
> has colorized source blocks. The file is created using
> https://git.sr.ht/~bzg/worg/blob/master/publish.sh and the script is
> executed inside a container with no available DISPLAY.

Well spotted! I think the trick there is setting
org-html-htmlize-output-type to 'css. See the variable's
documentation:

  Choices are css to export the CSS selectors only,inline-css
  to export the CSS attribute values inline in the HTML or nil to
  export plain text.  We use as default inline-css, in order to
  make the resulting HTML self-containing.

  However, this will fail when using Emacs in batch mode for export, because
  then no rich font definitions are in place.  It will also not be good if
  people with different Emacs setup contribute HTML files to a website,
  because the fonts will represent the individual setups.  In these cases,
  it is much better to let Org/Htmlize assign classes only, and to use
  a style file to define the look of these classes.
  To get a start for your css file, start Emacs session and make sure that
  all the faces you are interested in are defined, for example by loading files
  in all modes you want.  Then, use the command
  M-x org-html-htmlize-generate-css to extract class definitions.

So, still a workaround, technically, but a nice and clean one that
definitely deserves to be mentioned in this context!

Best,

David

Post-process table without changing result for empty table(/list)

[Jonas Bernoulli]

Hello,

If a code-block that is supposed to produce a table returns an empty
list, then nothing at all is inserted, as demonstrated by the second
example below:

#+begin_src emacs-lisp :results table
  ;; Some sql query that returns one or more rows:
  '(("foo" "bar"))
#+end_src
#+RESULTS:
| foo | bar |

#+begin_src emacs-lisp :results table
  ;; Another sql query that returns zero rows:
  '()
#+end_src
#+RESULTS:

I would like to keep it that way, but also use post-processing to add
a header to all non-empty tables (and only those).

Unfortunately post-processing changes how nil aka empty list aka empty
table is handled:

#+name: addheader
#+header: :var rows="" :var header=""
#+begin_src elisp :hlines yes
  (and rows `(("Key" "Value") hline ,@rows))
#+end_src

#+header: :post addheader(*this*,'("Key" "Value"))
#+begin_src emacs-lisp :results table
  ;; Some sql query that returns one or more rows:
  '(("foo" "bar"))
#+end_src
#+RESULTS:
| Key | Value |
|-+---|
| foo | bar   |

#+header: :post addheader(*this*,'("Key" "Value"))
#+begin_src emacs-lisp :results table
  ;; Another sql query that returns zero rows:
  '()
#+end_src
#+RESULTS:
: nil

I would like it to result in just

#+RESULTS:

like when no post-processing is involved.

It used to behave like that before 51a628bc5efc from 2009, which started
turning all symbols, including nil, into strings, but without giving any
reason why that should be done.

It has worked like this for a long time now, so reverting that is
probably not feasible in the short run.  However, I feel it would
make sense to change now how nil/'() is treated.  Currently it is
being treated as the symbol nil, but IMO it would make more sense
to treat it as the empty list.  That could be achieved with

diff --git a/lisp/ob-ref.el b/lisp/ob-ref.el
index b79e47900..2b4a16aea 100644
--- a/lisp/ob-ref.el
+++ b/lisp/ob-ref.el
@@ -199,7 +199,7 @@ (defun org-babel-ref-resolve (ref)
(org-babel-execute-src-block nil info params
 (error "Reference `%s' not found in this buffer" ref
 (cond
- ((symbolp result) (format "%S" result))
+ ((and result (symbolp result)) (format "%S" result))
  ((and index (listp result))
   (org-babel-ref-index-list index result))
  (t result)

In the long run you might want to consider not turning any symbols
into strings, at least not when the "regular" block as well as the
post-processing block both use elisp.

 Thanks for considering my suggestions!
 Jonas

Re: Code block syntax highlighting in async export

[Max Nikulin]

On 03/10/2022 19:08, David Lukeš wrote:

Could you please provide an example?



A problem that has some similar symptoms:
M. Pger. Re: no syntax highlighting for code blocks with org-publish.
Mon, 18 Jul 2022 22:55:15 +.
https://list.orgmode.org/inTtQOKWAhnFydjBE9fQOOdo_oV6kIPG5_E0d8DsfFiYMIJ4RNLRk68Id6dyrzy1qIDK8YJu0Wty7objv22vR31PNrh-qB9vKxMgHPxNAu8=@protonmail.com


This looks indeed very similar, thank you! I'll investigate if I can
leverage any of the workarounds mentioned.


I have realized that 
https://orgmode.org/worg/org-contrib/babel/languages/ob-doc-elisp.html 
has colorized source blocks. The file is created using 
https://git.sr.ht/~bzg/worg/blob/master/publish.sh and the script is 
executed inside a container with no available DISPLAY.

Re: Code block syntax highlighting in async export

[Timothy]

Hi Ihor,

> I would not be so sure. If the issue is major-mode not doing
> fontification in batch mode, engrave-faces cannot do much.

Try it 

All the best,
Timothy

-- 
Timothy (‘tecosaur’/‘TEC’), Org mode contributor.
Learn more about Org mode at .
Support Org development at ,
or support my work at .

Re: Code block syntax highlighting in async export

[Ihor Radchenko]

David Lukeš  writes:

>> I'll investigate if I can leverage any of the workarounds mentioned.
>
> So, for batch processing, avoiding --batch or --script and just using
> --load with a script that either ends by killing Emacs, or killing it
> via an additional --eval, works, i.e. syntax highlighting uses colors.
> This was suggested in [1]. As [2] points out, adding -nw removes the
> colors, and so does adding --daemon (suggested in [3]), so don't do
> either.
>
> Enabling font-lock-mode manually and defining all the fontification
> (see [4]) sounds like it would also work, but honestly, that's too
> involved, so I didn't try it.

To clarify, the latest Org version should not care about font-lock being
enabled. `font-lock-ensure' usually works in batch mode as well even
though font-lock-mode is technically disabled.

Except when it does not. Some major modes do fontification awkwardly,
which can be considered a bug in those major modes.
You can refer to https://yhetil.org/emacs-bugs/874jzdhh5m.fsf@localhost/

> Using an external colorizer (as shown in [5]), seems like a good
> alternative, if non-Emacs dependencies are acceptable.
>
> These two last options can also be made to work with async export (my
> original question), where you don't have control over how the batch
> instance of Emacs is invoked.
>
>> There’s a decent chance that `engrave-faces' (a package of mine that can now 
>> be
>> used for code blocks in LaTeX) will work in batch mode. Integrating it with
>> `ox-html', `ox-ansii', and `ox-odt' are all (distantly) on my todo list.
>
> Sounds great! If it will work in regular batch mode, then it should
> work with async as well :)

I would not be so sure. If the issue is major-mode not doing
fontification in batch mode, engrave-faces cannot do much.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at .
Support Org development at ,
or support my work at 

Re: Code block syntax highlighting in async export

[David Lukeš]

> I'll investigate if I can leverage any of the workarounds mentioned.

So, for batch processing, avoiding --batch or --script and just using
--load with a script that either ends by killing Emacs, or killing it
via an additional --eval, works, i.e. syntax highlighting uses colors.
This was suggested in [1]. As [2] points out, adding -nw removes the
colors, and so does adding --daemon (suggested in [3]), so don't do
either.

Enabling font-lock-mode manually and defining all the fontification
(see [4]) sounds like it would also work, but honestly, that's too
involved, so I didn't try it.

Using an external colorizer (as shown in [5]), seems like a good
alternative, if non-Emacs dependencies are acceptable.

These two last options can also be made to work with async export (my
original question), where you don't have control over how the batch
instance of Emacs is invoked.

> There’s a decent chance that `engrave-faces' (a package of mine that can now 
> be
> used for code blocks in LaTeX) will work in batch mode. Integrating it with
> `ox-html', `ox-ansii', and `ox-odt' are all (distantly) on my todo list.

Sounds great! If it will work in regular batch mode, then it should
work with async as well :)

Best,

David

[1] 
https://list.orgmode.org/orgmode/vZh_aXsUhlwNcRMF3miS6hNUo6xxUCFuzm4dwfCNQHcm4Ib53t8sJVVJ-PmYD4sRfoCvWRjtEG7L5rfN4pRe5nDxLFWxWEG8ZqcZ5YvJyDs=@protonmail.com/
[2] 
https://list.orgmode.org/orgmode/YmBR481ZMgZpA6svOfEMt3nYa8UPEKgvFUY5s8SpRWu2YdJz0pb0gPz8ajcTeKFbF7SD1fD9XibICa8Z-JdTiIqgjOlD1y4yg9uHCB176ZA=@protonmail.com/
[3] https://list.orgmode.org/orgmode/87a6923dbw.fsf@localhost/
[4] https://list.orgmode.org/ae5693f1-f63d-4383-8840-0fd2dbaac...@gmail.com/
[5] https://linevi.ch/en/org-pygments.html

Re: Code block syntax highlighting in async export

[Timothy]

Hi David,

> Sorry for that! I somehow thought that the issue would sound familiar
> to more seasoned Org users, and someone would come back with either
> “yeah, you need to do X” or “yeah, that’s a known limitation”.

Each backend is its own beast, somewhat .

> The export backend is HTML. In case anyone wants to try reproducing
> this, I made a minimal working example here:
> 

> This looks indeed very similar, thank you! I’ll investigate if I can
> leverage any of the workarounds mentioned.

There’s a decent chance that `engrave-faces' (a package of mine that can now be
used for code blocks in LaTeX) will work in batch mode. Integrating it with
`ox-html', `ox-ansii', and `ox-odt' are all (distantly) on my todo list.

All the best,
Timothy

-- 
Timothy (`tecosaur'/`TEC'), Org mode contributor.
Learn more about Org mode at .
Support Org development at ,
or support my work at .

Re: [PATCH] ox-odt: Prevent auto-formatting in export buffers

[David Lukeš]

Noted, will do!

David

Re: Code block syntax highlighting in async export

[David Lukeš]

> Could you please provide an example?

> Certainly more details is necessary, even export format (backend) is not
> specified.

Sorry for that! I somehow thought that the issue would sound familiar
to more seasoned Org users, and someone would come back with either
"yeah, you need to do X" or "yeah, that's a known limitation".

The export backend is HTML. In case anyone wants to try reproducing
this, I made a minimal working example here:
https://github.com/dlukes/org-html-a-sync-export-syntax-highlighting

> A problem that has some similar symptoms:
> M. Pger. Re: no syntax highlighting for code blocks with org-publish.
> Mon, 18 Jul 2022 22:55:15 +.
> https://list.orgmode.org/inTtQOKWAhnFydjBE9fQOOdo_oV6kIPG5_E0d8DsfFiYMIJ4RNLRk68Id6dyrzy1qIDK8YJu0Wty7objv22vR31PNrh-qB9vKxMgHPxNAu8=@protonmail.com

This looks indeed very similar, thank you! I'll investigate if I can
leverage any of the workarounds mentioned.

Best,

David

Re: Concatenating results

[Ihor Radchenko]

Felix Dorner  writes:

> I get an error with this:
> executing Emacs-Lisp code block...
> progn: Symbol’s value as variable is void: >>

That's because noweb references are disabled by default (see
https://orgmode.org/manual/Noweb-Reference-Syntax.html#Noweb-Reference-Syntax)

With default Org settings you need

#+begin_src emacs-lisp :noweb yes
(append
'<>
'<>)
#+end_src

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at .
Support Org development at ,
or support my work at 

Re: Concatenating results

[Felix Dorner]

On Mon, Oct 3, 2022 at 10:44 AM Ihor Radchenko  wrote:

> #+name: block1
> #+begin_src emacs-lisp :results table
> '((1 2) (3 4))
> #+end_src
>
> #+name: block2
> #+begin_src emacs-lisp :results table
> '((5 6) (7 8))
> #+end_src
>
>
> #+begin_src emacs-lisp
> (append
> '<>
> '<>)
> #+end_src
>

I get an error with this:
executing Emacs-Lisp code block...
progn: Symbol’s value as variable is void: >>

Re: `org-fill-paragraph' (`M-q') in Org Mode source blocks

[Ihor Radchenko]

Sébastien Miquel  writes:

>> Try
>> 1. emacs -Q
>> 2. insert
>> ;; A comment
>> (+ 2 2)
>> 3. M-h M-q
>> 
>> Is it emacs-lisp-mode bug? Or is it illegal to fill-region in source
>> code buffers?
>
> I think the original report is about M-q, not M-h M-q. M-q behaves as 
> expected in emacs-lisp-mode.
>
> Currently, org-fill-paragraph will not work properly when called inside 
> src blocks. This can be easily fixed, but probably requires yet another 
> org-fill-paragraph-act-natively variable.

See the attached.
I am not sure if we really need the variable.
AFAIU, acting natively was the default in the past for M-q with no
region selection. Then, I "fixed" some bug report in 05ee1e6ee06db and
introduced the bug herein.

>From 8777839a4fe5da6c9a780eac946e1a8a892d4f22 Mon Sep 17 00:00:00 2001
Message-Id: <8777839a4fe5da6c9a780eac946e1a8a892d4f22.1664788728.git.yanta...@gmail.com>
From: Ihor Radchenko 
Date: Mon, 3 Oct 2022 17:03:15 +0800
Subject: [PATCH] org-fill-element: Respect region selection when filling
 src-block

* lisp/org.el (org-fill-element): When region is not active, run
`fill-paragraph' at point inside src block.  When region is active and
within src block boundaries, run `fill-paragraph' preserving the
region.  When region is active and crosses src block boundaries, fill
the whole src block.

Reported-by: Fabio Natali 
Fixes: https://orgmode.org/list/201b44de-1f97-1b23-1767-970ee00f2...@posteo.eu
---
 lisp/org.el | 17 -
 1 file changed, 12 insertions(+), 5 deletions(-)

diff --git a/lisp/org.el b/lisp/org.el
index 036384a04..23cf4198e 100644
--- a/lisp/org.el
+++ b/lisp/org.el
@@ -19135,11 +19135,18 @@ (defun org-fill-element ( justify)
   ;; the buffer.  In that case, ignore filling.
   (cl-case (org-element-type element)
 	;; Use major mode filling function is source blocks.
-	(src-block (org-babel-do-in-edit-buffer
-(push-mark (point-min))
-(goto-char (point-max))
-(setq mark-active t)
-(funcall-interactively #'fill-paragraph justify 'region)))
+(src-block
+ (let ((regionp (region-active-p)))
+   (org-babel-do-in-edit-buffer
+;; `org-babel-do-in-edit-buffer' will preserve region if it
+;; is within src block contents.  Otherwise, the region
+;; crosses src block boundaries.  We re-fill the whole src
+;; block in such scenario.
+(when (and regionp (not (region-active-p)))
+  (push-mark (point-min))
+  (goto-char (point-max))
+  (setq mark-active t))
+(funcall-interactively #'fill-paragraph justify 'region
 	;; Align Org tables, leave table.el tables as-is.
 	(table-row (org-table-align) t)
 	(table
-- 
2.35.1


-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at .
Support Org development at ,
or support my work at 

Re: Numbered footnotes in the manual interfere with diff

[Bastien]

Hi Ihor,

Ihor Radchenko  writes:

> Should we just inline the one-sentence footnotes?

Yes, this would be a progress already.

> Also, there is one footnote that is indexed (does it even work?):

AFACT it does not work and I think it should not work, we can remove
these index entries.

-- 
 Bastien

Re: Concatenating results

[Ihor Radchenko]

Felix Dorner  writes:

> I have a list of N source blocks, of which each produces a table, and all
> these tables have the same column count. What is the easiest way to now
> make a single table which is the result of calling all these blocks and
> concatenating the result?
>
> Essentially, would be also the same as N CALL together but all append to
> the same result instead of a result for each CALL.

There are can be multiple ways to achieve what you want.

One way could be using noweb references:

#+name: block1
#+begin_src emacs-lisp :results table
'((1 2) (3 4))
#+end_src

#+name: block2
#+begin_src emacs-lisp :results table
'((5 6) (7 8))
#+end_src

 
#+begin_src emacs-lisp
(append
'<>
'<>)
#+end_src


-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at .
Support Org development at ,
or support my work at 

Concatenating results

[Felix Dorner]

I have a list of N source blocks, of which each produces a table, and all
these tables have the same column count. What is the easiest way to now
make a single table which is the result of calling all these blocks and
concatenating the result?

Essentially, would be also the same as N CALL together but all append to
the same result instead of a result for each CALL.

Btw thanks, this Org stuff is all so mind blowing.

-- 
Linux. The choice of a GNU generation.

Re: Numbered footnotes in the manual interfere with diff

[Ihor Radchenko]

Bastien  writes:

> Ihor Radchenko  writes:
>
>> Do you have any suggestion on how to deal with changing footnotes in the
>> manual? When I delete this footnote, all the footnotes must be
>> re-numbered creating a lot of garbage in the diff. Is it ok? Or should
>> we prefer inline footnote definitions in the manual to avoid such
>> situations?
>
> I propose to use inline footnotes for notes of one paragraph and to
> use regular footnotes for notes spanning over more than one paragraph.
> This will enhance both diffs readability and that of the manual's .org
> source.
>
> WDYT?

All the footnotes only span a single paragraph.
However, multiple footnotes span several sentences and thus look
cumbersome when inlined.

Should we just inline the one-sentence footnotes?

Also, there is one footnote that is indexed (does it even work?):

#+findex: org-date
#+findex: org-anniversary
#+findex: org-cyclic
#+findex: org-block

[fn:60] When working with the standard diary expression functions, you

I am not attaching the patch. Instead, you can run M-:
(yant/replace-all-footnotes) inside doc/org-manual.org


(defun yant/replace-footnote-with-def (fn)
  "Replace FN footnote-reference with inline footnote.
Move point to the end of the FN."
  (goto-char (org-element-property :end fn))
  (when (and (org-element-property :label fn)
 (eq (org-element-type fn) 'footnote-reference))
(let ((definition
(save-excursion
  (org-footnote-goto-definition (org-element-property :label fn))
  (let ((def (org-element-context)))
(org-wrap
 (org-trim
  (buffer-substring-no-properties
   (org-element-property :contents-begin def)
   (org-element-property :contents-end def)))
 nil 1)
  (when definition
(replace-region-contents
 (org-element-property :begin fn)
 (org-element-property :end fn)
 (lambda () (format "[fn:: %s]%s" definition (make-string (or 
(org-element-property :post-blank fn) 0) ?\s
(org-fill-paragraph)

(defun yant/replace-all-footnotes ()
  "Replace all footnotes in buffer with their definition."
  (org-with-wide-buffer
   (goto-char (point-min))
   (while (re-search-forward org-footnote-re nil t)
 (backward-char)
 (let ((fn (org-element-context)))
   (yant/replace-footnote-with-def fn)


-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at .
Support Org development at ,
or support my work at 

Re: `org-fill-paragraph' (`M-q') in Org Mode source blocks

[Sébastien Miquel]

Hi,

Ihor Radchenko writes:

I am still getting the described behaviour.
However, it does not happen in Org mode itself.
`fill-paragraph' in emacs-lisp-mode does exactly the observed behaviour.

Try
1. emacs -Q
2. insert
;; A comment
(+ 2 2)
3. M-h M-q

Is it emacs-lisp-mode bug? Or is it illegal to fill-region in source
code buffers?


I think the original report is about M-q, not M-h M-q. M-q behaves as 
expected in emacs-lisp-mode.


Currently, org-fill-paragraph will not work properly when called inside 
src blocks. This can be easily fixed, but probably requires yet another 
org-fill-paragraph-act-natively variable.


--
Sébastien Miquel

Re: Explicit page breaks

[Ihor Radchenko]

Juan Manuel Macías  writes:

> ...
> With all this, I mean: to what extent should Org care about all these
> details, more related to fine-tuning the output format?

Thanks for the detailed explanation!

It is now clear that pagebreak by itself may very much depend on the
specifics of the export backend and the place in the document.

Using page break at the same place for different export backends is
unlikely to be useful.

On the other hand, some backends (odt) are too cumbersome to put page
breaks within @@odt:...@@ export snippets.

What we do want is some way to put a page break just for odt or just for
latex.

May we introduce a new standard macro {{{page-break(backend)}}}
that will expand to an appropriate
@@backend:https://orgmode.org/>.
Support Org development at ,
or support my work at 

Re: `org-fill-paragraph' (`M-q') in Org Mode source blocks

[Ihor Radchenko]

Sébastien Miquel  writes:

> Fabio Natali writes:
>> Thanks for getting back to me and thank you very much for the code
>> snippet, which I think I'm going to integrate in my configuration.
>
> Thank you for the report. With regard to the snippet, It seems the
> advice function needs ~( justify region)~ as arguments rather
> than ().

I am still getting the described behaviour.
However, it does not happen in Org mode itself.
`fill-paragraph' in emacs-lisp-mode does exactly the observed behaviour.

Try
1. emacs -Q
2. insert
;; A comment
(+ 2 2)
3. M-h M-q

Is it emacs-lisp-mode bug? Or is it illegal to fill-region in source
code buffers?

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at .
Support Org development at ,
or support my work at 

Re: Some broken links in pages under https://orgmode.org/worg/org-contrib/babel

[Ihor Radchenko]

"Thomas S. Dye"  writes:

> A bad link blocked HTML compilation on the server.  Fixed now, 
> thanks.

I am thus closing this bug report.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at .
Support Org development at ,
or support my work at 

Re: [BUG] Weird sparse tree folding [9.5 (release_9.5-225-g494c20.dirty @ /Users/carlos/Install/Source/org-mode/lisp/)]

[Ihor Radchenko]

Ihor Radchenko  writes:

> Confirmed on current main.
>
> Though I plan a bunch of commits that will also fix this particular
> issue.

I believe that the issue is no longer present on current main.
I am closing thus bug.

Let me know if you are still experiencing the issue.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at .
Support Org development at ,
or support my work at 

Fixed.