[BUG] org-element-cache error using org-store-link file [9.6.1 (9.6.1-g44e1cb @ c:/users/scott/.emacs.d/straight/build/org/)]

[Scott Otterson]

When I run org-store-link inside a .bib file, I get the cache error below.

If somebody's interested, I can send the log separately in a zip file.
It's too big for my email client to send uncompressed, and I believe I
can't send compressed files to this list.

This is using:

Emacs  : GNU Emacs 29.0.60 (build 1, x86_64-w64-mingw32) of 2023-03-10
Package: Org mode version 9.6.1 (9.6.1-g44e1cb @
c:/users/scott/.emacs.d/straight/build/org/)

Thanks,

Scott


--- Warning


 ■  Warning (org-element-cache): org-element--cache: Org parser error in
energy.bib::761517. Resetting.
 The error was: (error "rx ‘**’ range error")
 Backtrace:
nil
 Please report this to Org mode mailing list (M-x org-submit-bug-report).
 ■  Warning (org-element-cache): org-element--cache: Org parser error in
energy.bib::761517. Resetting.
 The error was: (error "rx ‘**’ range error")
 Backtrace:
nil
 Please report this to Org mode mailing list (M-x org-submit-bug-report).

Re: [RFC] ox-icalendar: Unscheduled tasks & repeating tasks

[Jack Kamm]

Ihor Radchenko  writes:

> So, we should probably override `org-export-coding-system', even when it
> is set. iCalendar demands UTF8 anyway.

Also, ox-icalendar already sets ":ascii-charset utf-8" in the ext-plist
during export.

> We likely want (according to 34.10.1 Basic Concepts of Coding Systems):

I attach a new patch, which takes the approach of converting to
utf-8-dos in `org-icalendar-after-save-hook', instead of converting
newlines in `org-icalendar-fold-string'.

I think this way is simpler, and should be more robust across locales.

Note, this means the string returned by `org-export-as' won't contain
CRLF. Instead, the newlines are converted during post-process.

>From 04761429f82bfd2aee63f4978afec3449abaa37d Mon Sep 17 00:00:00 2001
From: Jack Kamm 
Date: Sat, 1 Apr 2023 16:53:35 -0700
Subject: [PATCH] ox-icalendar: Use consistent CRLF line endings

Fixes issue where the ox-icalendar export uses an inconsistent mix of
dos and unix style line endings.

* lisp/ox-icalendar.el (org-icalendar-fold-string): No longer converts
to CRLF, instead delegating that to `org-icalendar--convert-eol'.
(org-icalendar--convert-eol): New function to convert EOL to CRLF. It
runs early in `org-icalendar-after-save-hook'.
* testing/lisp/test-ox-icalendar.el: New file for unit tests of
ox-icalendar.  Add an initial test for CRLF line endings.

See also:

https://list.orgmode.org/87o7oetneo.fsf@localhost/T/#m3e3eb80f9fc51ba75854b33ebfe9ecdefa2ded24

https://list.orgmode.org/orgmode/87ilgljv6i.fsf@localhost/
---
 etc/ORG-NEWS  | 12 +
 lisp/ox-icalendar.el  | 14 +++---
 testing/lisp/test-ox-icalendar.el | 44 +++
 3 files changed, 67 insertions(+), 3 deletions(-)
 create mode 100644 testing/lisp/test-ox-icalendar.el

diff --git a/etc/ORG-NEWS b/etc/ORG-NEWS
index ac233a986..9f7d01707 100644
--- a/etc/ORG-NEWS
+++ b/etc/ORG-NEWS
@@ -23,6 +23,18 @@ If you still want to use python-mode with ob-python, you might
 consider [[https://gitlab.com/jackkamm/ob-python-mode-mode][ob-python-mode-mode]], where the code to support python-mode
 has been ported to.
 
+*** =ox-icalendar.el= line ending fix may affect downstream packages
+
+iCalendar export now uses dos-style CRLF ("\r\n") line endings
+throughout, as required by the iCalendar specification (RFC 5545).
+Previously, the export used an inconsistent mix of dos and unix line
+endings.
+
+This might cause errors in external packages that parse output from
+ox-icalendar.  In particular, older versions of org-caldav may
+encounter issues, and users are advised to update to the most recent
+version of org-caldav.  See [[https://github.com/dengste/org-caldav/commit/618bf4cdc9be140ca1993901d017b7f18297f1b8][this org-caldav commit]] for more information.
+
 ** New and changed options
 *** New ~org-cite-natbib-export-bibliography~ option defining fallback bibliography style
 
diff --git a/lisp/ox-icalendar.el b/lisp/ox-icalendar.el
index 81a77a770..7f675b5d0 100644
--- a/lisp/ox-icalendar.el
+++ b/lisp/ox-icalendar.el
@@ -540,12 +540,20 @@ (defun org-icalendar-fold-string (s)
 	;; line, real contents must be split at 74 chars.
 	(while (< (setq chunk-end (+ chunk-start 74)) len)
 	  (setq folded-line
-		(concat folded-line "\r\n "
+		(concat folded-line "\n "
 			(substring line chunk-start chunk-end))
 		chunk-start chunk-end))
-	(concat folded-line "\r\n " (substring line chunk-start))
-(org-split-string s "\n") "\r\n")))
+	(concat folded-line "\n " (substring line chunk-start))
+(org-split-string s "\n") "\n")))
 
+(defun org-icalendar--convert-eol (f)
+  "Convert line endings to CRLF as per RFC 5545."
+  (with-temp-buffer
+(insert-file-contents f)
+(let ((coding-system-for-write 'utf-8-dos))
+  (write-region nil nil f
+
+(add-hook 'org-icalendar-after-save-hook #'org-icalendar--convert-eol -90)
 
 
 ;;; Filters
diff --git a/testing/lisp/test-ox-icalendar.el b/testing/lisp/test-ox-icalendar.el
new file mode 100644
index 0..bfc756d51
--- /dev/null
+++ b/testing/lisp/test-ox-icalendar.el
@@ -0,0 +1,44 @@
+;;; test-ox-icalendar.el --- tests for ox-icalendar.el  -*- lexical-binding: t; -*-
+
+;; Copyright (C) 2023  Jack Kamm
+
+;; Author: Jack Kamm 
+
+;; This program is free software; you can redistribute it and/or modify
+;; it under the terms of the GNU General Public License as published by
+;; the Free Software Foundation, either version 3 of the License, or
+;; (at your option) any later version.
+
+;; This program is distributed in the hope that it will be useful,
+;; but WITHOUT ANY WARRANTY; without even the implied warranty of
+;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+;; GNU General Public License for more details.
+
+;; You should have received a copy of the GNU General Public License
+;; along with this program.  If not, see .
+
+;;; Commentary:
+
+;; Tests checking 

Re: [BUG] No font lock in src blocks for shells in org-babel-shell-names (was Re: Font lock for org-babel shell scripts?)

[Matt]

  On Thu, 30 Mar 2023 04:55:32 -0400  Ihor Radchenko  wrote --- 
 > Matt m...@excalamus.com> writes:
 > 
 > > I think this approach will work fine.   I tried examples for each shell 
 > > type and keywords like if/then/else and function names are highlighted.
 > 
 > Even for posh (powershell)?

Yes.  It's not great since sh-mode looks for Korn-based keywords.  It does 
string highlighting and common keywords like 'if', 'exit', and 'param'.

Re: [PATCH] ox-texinfo: Fix invalid syntax in Texinfo version detection code

[Rudolf Adamkovič]

Ihor Radchenko  writes:

> I finally managed to fix the tests.

Thank you!  I have had this on my to-do list since you brought it up,but
I could not find the time to fix it.

Rudy
-- 
"'Contrariwise,' continued Tweedledee, 'if it was so, it might be; and
if it were so, it would be; but as it isn't, it ain't.  That's logic.'"
-- Lewis Carroll, Through the Looking Glass, 1871/1872

Rudolf Adamkovič  [he/him]
Studenohorská 25
84103 Bratislava
Slovakia

Re: org-ref-formatted-citation-formats for books: ISBN

[Uwe Brauer]

>>> "JK" == John Kitchin  writes:

> there isn't enough information to tell what you are trying to do or if it
> is the right thing to do. I don't think that is how org-ref exports to HTML
> anymore, now it uses CSL. There is something like that syntax for
> bibtex-completion-display-formats.

> You should file an issue at https://github.com/jkitchin/org-ref/issues with
> a small example org-file, bibtex example, and configuration that reproduces
> your problem.

I don't know precisely why, but after following Ihor's advice, the (old)
html export worked in the sense that it included the isbn field. 
Not sure what happened.

Regards



-- 
Warning: Content may be disturbing to some audiences
I strongly condemn Putin's war of aggression against the Ukraine.
I support to deliver weapons to Ukraine's military. 
I support the ban of Russia from SWIFT.
I support the EU membership of the Ukraine. 
https://addons.thunderbird.net/en-US/thunderbird/addon/gmail-conversation-view/


smime.p7s
Description: S/MIME cryptographic signature

Re: [PATCH] Autoload `org-assert-version' and remove org-loaddefs.el

[Max Nikulin]

On 29/03/2023 23:52, Ihor Radchenko wrote:

Max Nikulin writes:

Sorry, I have not tried to compile Emacs myself and I am really puzzled
why you was not able to reproduce the issue.


I was able to reproduce on Debian using virtual machine.


To be precise, I am surprised that you are unable to reproduce the issue 
with older Emacs version compiled from source.



https://old.reddit.com/r/orgmode/comments/123qnqq/workaround_for_orgassertversion_problem_not/


I have no clue why your patch should help in this case.

My hypothesis: the user has `use-package' org-roam in their init file. 
Attempt to update org caused compiling new org when old version is 
already loaded due to Emacs-27 does not contain fixes that reload 
package that is installed. Next time load of org failed because it is 
result of mixed version compilation.



I am in doubts if emacs version should be checked or it should be e.g.
(fboundp 'org-assert-version).


It is indeed a cleaner approach.


I am not sure. Perhaps it should be (or (fboundp 'org-assert-version) 
(new-package-management-code)). Since testing for private function is 
not a reliable solution, only version check is available.

Re: [PATCH] Add compat.el support to Org (was: [POLL] Use compat.el in Org? (was: Useful package? Compat.el))

[Max Nikulin]

On 01/04/2023 18:38, Daniel Mendler wrote:

 From my side, the change looks good. The current Compat version 29.1.4.x
is stable with no known issues.


Debian Bookworm and Ubuntu 23.04 (currently frozen testing and beta 
accordingly) have elpa-compat-29.1.3.4 and 29.1.3.2. Are some issues 
expected if a bit earlier version is required?

Re: org-ref-formatted-citation-formats for books: ISBN

[John Kitchin]

there isn't enough information to tell what you are trying to do or if it
is the right thing to do. I don't think that is how org-ref exports to HTML
anymore, now it uses CSL. There is something like that syntax for
bibtex-completion-display-formats.

You should file an issue at https://github.com/jkitchin/org-ref/issues with
a small example org-file, bibtex example, and configuration that reproduces
your problem.

On Thu, Mar 30, 2023 at 8:11 AM Ihor Radchenko  wrote:

> Uwe Brauer  writes:
>
> > The following setting in my custom file
> > ...
> >   ("book" . "${author}, /${title}/ (${year}), ${ISBN}, (pp.
> ${pages}) ${address}: ${publisher}.")
> >
> > However when I want to export a book reference to HTML, the ISBN field
> > is not inserted.
> >
> > Any idea what is wrong?
>
> Maybe it is case-sensitive.
>
> --
> Ihor Radchenko // yantar92,
> Org mode contributor,
> Learn more about Org mode at .
> Support Org development at ,
> or support my work at 
>
>

-- 
John

---
Professor John Kitchin (he/his)
Doherty Hall A207F
Department of Chemical Engineering
Carnegie Mellon University
Pittsburgh, PA 15213
412-268-7803
https://kitchingroup.cheme.cmu.edu
https://pointbreezepubs.gumroad.com/ pycse bookstore

Re: [FR] Allow to #+include files verbatim without any processing (was: Have export treat file: paths in INCLUDED file relative to the INCLUDING file's dir)

[gnuric]

Hi,

I was wondering if anyone had any new ideas on this question (or an explanation 
on what Timothy meant by a :dir parameter).

Right now I'm having to use a search and replace back on the whole generated 
html buffer, which is fragile and inefficient.

Thanks,
Omid

 Original Message 
On Feb 28, 2023, 1:34 PM, wrote:

> Hi Timothy, Thanks for your reply. "Timothy"  writes: > Hi Ihor, > >> This 
> sounds like a reasonable request. >> What we may do here is allowing a new 
> parameter :verbatim > > From a read of the original email, it sounds like a 
> `:dir' > parameter could also > solve this use case, and allow for a bit more 
> flexibility. > Is `:dir' a parameter to pass to `#+INCLUDE:'? I don't see it 
> in the manual: https://orgmode.org/manual/Include-Files.html > All the best, 
> > Timothy Regards, Omid

Re: [PATCH] Add compat.el support to Org (was: [POLL] Use compat.el in Org? (was: Useful package? Compat.el))

[Daniel Mendler]

On 4/1/23 12:31, Ihor Radchenko wrote:
> Ihor Radchenko  writes:
> 
>> I have recently been contacted by the current compat.el maintainer
>> asking if we are willing to adapt compat.el in Org.
> 
> See the attached patch set adding support of compat.el.
> 
> I had to update Org's build system to handle third-party packages.
> Please, give it a close check (first patch).
> 
> The second patch adds the actual Elisp part and replaces some
> compatibility functions with what is provided by compat.el.
> Note that not all the functions are available in compat.el for now.
> Daniel, you might want to take a look.
> 
> The last patch adds the necessary explanation for users who install Org
> from git. Now, they must install compat.el manually to make Org work.

>From my side, the change looks good. The current Compat version 29.1.4.x
is stable with no known issues.

Daniel

[PATCH] Add compat.el support to Org (was: [POLL] Use compat.el in Org? (was: Useful package? Compat.el))

[Ihor Radchenko]

Ihor Radchenko  writes:

> I have recently been contacted by the current compat.el maintainer
> asking if we are willing to adapt compat.el in Org.

See the attached patch set adding support of compat.el.

I had to update Org's build system to handle third-party packages.
Please, give it a close check (first patch).

The second patch adds the actual Elisp part and replaces some
compatibility functions with what is provided by compat.el.
Note that not all the functions are available in compat.el for now.
Daniel, you might want to take a look.

The last patch adds the necessary explanation for users who install Org
from git. Now, they must install compat.el manually to make Org work.

>From f95433f53878e8371bb28a045fdb5d06cf0877b9 Mon Sep 17 00:00:00 2001
Message-Id: 
From: Ihor Radchenko 
Date: Sat, 1 Apr 2023 12:00:48 +0200
Subject: [PATCH 1/3] Upgrade Org build system to handle third-party
 dependencies

* mk/default.mk (pkgdir): New variable holding the location of
third-party packages to be downloaded if necessary during compilation.
(EFLAGS): New variable holding extra flags to be passed to Emacs
executable when running make.
(EPACKAGES): List of packages to be installed (unless already present
in the `load-path') during compilation.
(package-install):
(INSTALL_PACKAGES): New command to download and install missing packages.
(BATCH): Update, setting default package location to pkgdir.
* mk/targets.mk (uppkg): New target to download install missing
packages.
(check test):
(compile compile-dirty): Use the new uppkg target.
(cleanpkg): New target cleaning up the downloaded packages.
(cleanall): Use the new target.
(.PHONY):
(CONF_BASE):
(CONF_DEST):
(CONF_CALL): Update according to the new variables and targets.
* .gitignore: Ignore the downloaded packages.

This commit paves the way towards third-party built-time dependencies
for Org.  In particular, towards including compat.el dependency.

According to EPACKAGES, we can auto-download necessary packages,
unless they are manually specified via -L switches in EFLAGS.

Link: https://orgmode.org/list/87v8ks6rhf.fsf@localhost
---
 .gitignore|  1 +
 mk/default.mk | 24 +++-
 mk/targets.mk | 24 
 3 files changed, 40 insertions(+), 9 deletions(-)

diff --git a/.gitignore b/.gitignore
index 4bb81c359..a58670c90 100644
--- a/.gitignore
+++ b/.gitignore
@@ -71,6 +71,7 @@ t
 auto
 tmp
 TODO
+/pkg-deps
 
 # and collateral damage from Emacs
 
diff --git a/mk/default.mk b/mk/default.mk
index fa46661e8..997b22b66 100644
--- a/mk/default.mk
+++ b/mk/default.mk
@@ -31,6 +31,15 @@ GIT_BRANCH =
 TMPDIR ?= /tmp
 testdir = $(TMPDIR)/tmp-orgtest
 
+# Where to store Org dependencies
+pkgdir = $(shell pwd)/pkg-deps
+
+# Extra flags to be passed to Emacs
+EFLAGS ?=
+
+# Third-party packages to install when running make
+EPACKAGES ?=
+
 # Configuration for testing
 # Verbose ERT summary by default for Emacs-28 and above.
 # To override:
@@ -72,12 +81,22 @@ REPRO_ARGS ?=
 req-ob-lang = --eval '(require '"'"'ob-$(ob-lang))'
 lst-ob-lang = ($(ob-lang) . t)
 req-extra   = --eval '(require '"'"'$(req))'
+package-install = --eval '(unless (require '"'"'$(package) nil t) (message "%s" load-path) (package-install '"'"'$(package)))'
 BTEST_RE   ?= \\(org\\|ob\\|ox\\)
 BTEST_LOAD  = \
 	--eval '(add-to-list '"'"'load-path (concat default-directory "lisp"))' \
 	--eval '(add-to-list '"'"'load-path (concat default-directory "testing"))'
 BTEST_INIT  = $(BTEST_PRE) $(BTEST_LOAD) $(BTEST_POST)
 
+ifeq (,$(EPACKAGES))
+INSTALL_PACKAGES =
+else
+INSTALL_PACKAGES = \
+	$(BATCH) \
+--eval '(package-refresh-contents)' \
+$(foreach package,$(EPACKAGES),$(package-install))
+endif
+
 BTEST = $(BATCH) $(BTEST_INIT) \
 	  -l org-batch-test-init \
 	  --eval '(setq \
@@ -120,7 +139,10 @@ EMACSQ  = $(EMACS)  -Q
 
 # Using emacs in batch mode.
 BATCH	= $(EMACSQ) -batch \
-	  --eval '(setq vc-handled-backends nil org-startup-folded nil org-element-cache-persistent nil)'
+	  $(EFLAGS) \
+	  --eval '(setq vc-handled-backends nil org-startup-folded nil org-element-cache-persistent nil)' \
+  --eval '(make-directory "$(pkgdir)" t)' \
+	  --eval '(setq package-user-dir "$(pkgdir)")' --eval '(package-initialize)'
 
 # Emacs must be started in toplevel directory
 BATCHO	= $(BATCH) \
diff --git a/mk/targets.mk b/mk/targets.mk
index 0bd293d68..ab8b830bb 100644
--- a/mk/targets.mk
+++ b/mk/targets.mk
@@ -27,21 +27,21 @@ ifneq ($(GITSTATUS),)
   GITVERSION := $(GITVERSION:.dirty=).dirty
 endif
 
-.PHONY:	all oldorg update update2 up0 up1 up2 single $(SUBDIRS) \
+.PHONY:	all oldorg update update2 up0 up1 up2 uppkg single $(SUBDIRS) \
 	check test install $(INSTSUB) \
 	info html pdf card refcard doc docs \
 	autoloads cleanall clean $(CLEANDIRS:%=clean%) \
 	clean-install cleanelc cleandirs \
-	cleanlisp cleandoc cleandocs cleantest \
+	cleanlisp cleandoc cleandocs cleantest cleanpkg \
 	compile compile-dirty uncompiled \
 	config config-test 

Re: [RFC] ox-icalendar: Unscheduled tasks & repeating tasks

[Ihor Radchenko]

Jack Kamm  writes:

> For issue 1, what `org-icalendar-fold-string' does when string already
> contains \r\n, you can see that it produces \r\r\n as follows:
>
> emacs -Q -l ox-icalendar
> M-:
> (org-icalendar-fold-string (org-icalendar-fold-string "Line1\nLine2"))
>
> This is why the patch removes the calls to `org-icalendar-fold-string'
> in `org-icalendar--vevent' and `org-icalendar--vtodo' -- otherwise we
> would add \r multiple times to the same string.
>
> To change this behavior of `org-icalendar-fold-string', we could
> modify the patch to do:
>
>  (defun org-icalendar-fold-string (s)
>"Fold string S according to RFC 5545."
>(replace-regexp-in-string
> -   "\n" "\r\n"
> +   "\r*\n" "\r\n"
>
> which would strip out any extra \r at end of line. Another alternative
> would be to use "\r?\n" instead of "\r*\n".

"\r*\n" looks safer.

> For the second issue -- when `org-export-coding-system' is dos (or
> similar), the file created by `org-export-to-file' will contain
> \r\r\n. This was already the pre-existing behavior, but note the patch
> does cause a minor change here: before the patch just the main body
> will have \r\r\n, but after the patch, the preamble will also have it.

I see.
Looking at
https://icalendar.org/iCalendar-RFC-5545/3-1-4-character-set.html:

There is not a property parameter to declare the charset used in a
property value. The default charset for an iCalendar stream is UTF-8 as
defined in [RFC3629].

So, we should probably override `org-export-coding-system', even when it
is set. iCalendar demands UTF8 anyway.

We likely want (according to 34.10.1 Basic Concepts of Coding Systems):

   The coding system ‘utf-8-emacs’ specifies that the data is
represented in the internal Emacs encoding (*note Text
Representations::).  This is like ‘raw-text’ in that no code conversion
happens, but different in that the result is multibyte data.  The name
‘emacs-internal’ is an alias for ‘utf-8-emacs-unix’ (so it forces no
conversion of end-of-line, unlike ‘utf-8-emacs’, which can decode all 3
kinds of end-of-line conventions).

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at .
Support Org development at ,
or support my work at 

Re: Worg: Possible bug about inline-image example using four brackets

[Ihor Radchenko]

Corwin Brust  writes:

>> Thanks, but you cannot put language as example block argument.
>>
>
> When I remove the language argument ("org"), org resumes thinking it
> is a link: it becomes a button and hides the markup we are giving an
> example of.

This is just a visual fontification glitch (known).
On export, the brackets will be normally displayed.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at .
Support Org development at ,
or support my work at