Re: [patch] fix ox-latex async export bug

2021-11-30 Thread Sébastien Miquel

Hi,

Rasmus Pank Roulund writes:

This is most likely due to native compilation which compiles the
unquoted lambda. Once compiled, it (presumably) fails to be passed to
the external emacs process.

Ah, interesting.  Is this a bug or a feature?


I think the bug's with org-mode. As I explain elsewhere in the thread,
this lambda is to be treated as data, and written to a file to be
executed by the external emacs instance. It should always have been
quoted (naming it happens to work also).

Regards,

--
Sébastien Miquel


[PATCH] org.el: Fix the filling of regions containing lists

2021-11-30 Thread Sébastien Miquel

Hi,

The attached patch fixes the following issues with the functions
=fill-region= and =fill-paragraph=, when called with an active region
containing a list.

In the examples, replace 'long line' by long lines to be filled.

 + Calling =fill-region= on a region which contains a list with single
   line items (such as the one below) breaks the list structure.
   - long line
   - long line
 + Calling =fill-region= on a region with a list such as the one below
   doesn't fill the list
   - short line
 short line
   - short line
 short line
 + Calling =fill-paragraph= on a region containing a list such as the
   one below doesn't fill the first item
   - long line
   - long line
   - long line
 + Calling =fill-paragraph= on a region containing a list such as the
   one below doesn't fill the list
   - long line
   - long line
   - short line

Regards,

--
Sébastien Miquel
From 6c60e8e43e39074fa87da2f0ed272a69a6793862 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?S=C3=A9bastien=20Miquel?= 
Date: Sat, 6 Nov 2021 22:41:20 +0100
Subject: [PATCH] org.el: Fix the filling of regions containing lists

* lisp/org.el (org-setup-filling): Set fill-forward-paragraph-function.
(org--single-lines-list-is-paragraph): New internal variable. Whether
a list with single lines items should be considered a single
paragraph.
(org--paragraph-at-point): use org--single-lines-list-is-paragraph.
(org-fill-paragraph): When an active region contains a list ensure
every item get filled.
* testing/lisp/test-org.el (test-org/fill-paragraph):
(test-org/fill-region): Test behaviour of fill-paragraph and
fill-region with an active region containing a list.

When filling paragraphs in a region, do not treat a list with single
lines items as a single paragraph.
---
 lisp/org.el  | 19 +++
 testing/lisp/test-org.el | 41 
 2 files changed, 56 insertions(+), 4 deletions(-)

diff --git a/lisp/org.el b/lisp/org.el
index 025513e7a..17046f391 100644
--- a/lisp/org.el
+++ b/lisp/org.el
@@ -19629,6 +19629,8 @@ assumed to be significant there."
 ;; `org-setup-filling' installs filling and auto-filling related
 ;; variables during `org-mode' initialization.
 
+(defvar org--single-lines-list-is-paragraph) ; defined later
+
 (defun org-setup-filling ()
   (require 'org-element)
   ;; Prevent auto-fill from inserting unwanted new items.
@@ -19644,6 +19646,10 @@ assumed to be significant there."
 (setq-local paragraph-start paragraph-ending)
 (setq-local paragraph-separate paragraph-ending))
   (setq-local fill-paragraph-function 'org-fill-paragraph)
+  (setq-local fill-forward-paragraph-function
+  (lambda ( arg)
+(let ((org--single-lines-list-is-paragraph nil))
+  (org-forward-paragraph arg
   (setq-local auto-fill-inhibit-regexp nil)
   (setq-local adaptive-fill-function 'org-adaptive-fill-function)
   (setq-local normal-auto-fill-function 'org-auto-fill-function)
@@ -19873,9 +19879,11 @@ filling the current element."
 	(progn
 	  (goto-char (region-end))
 	  (skip-chars-backward " \t\n")
-	  (while (> (point) start)
-		(org-fill-element justify)
-		(org-backward-paragraph)))
+	  (let ((org--single-lines-list-is-paragraph nil))
+(while (> (point) start)
+		  (org-fill-element justify)
+		  (org-backward-paragraph)
+  (skip-chars-backward " \t\n"
 	  (goto-char origin)
 	  (set-marker origin nil
  (t
@@ -21218,6 +21226,9 @@ It also provides the following special moves for convenience:
 ;; Return moves left.
 arg))
 
+(defvar org--single-lines-list-is-paragraph t
+  "Treat plain lists with an item every line as a whole paragraph")
+
 (defun org--paragraph-at-point ()
   "Return paragraph, or equivalent, element at point.
 
@@ -21279,7 +21290,7 @@ Function may return a real element, or a pseudo-element with type
 	  (while (memq (org-element-type (org-element-property :parent l))
 			   '(item plain-list))
 		(setq l (org-element-property :parent l)))
-	  (and l
+	  (and l org--single-lines-list-is-paragraph
 		   (org-with-point-at (org-element-property :post-affiliated l)
 		 (forward-line (length (org-element-property :structure l)))
 		 (= (point) (org-element-property :contents-end l)))
diff --git a/testing/lisp/test-org.el b/testing/lisp/test-org.el
index 056ea7d87..4375456da 100644
--- a/testing/lisp/test-org.el
+++ b/testing/lisp/test-org.el
@@ -765,8 +765,49 @@
 	  (push-mark (point) t t)
 	  (goto-char (point-max))
 	  (call-interactively #'org-fill-paragraph)
+	  (buffer-string)
+  ;; Fill every list item in a region
+  (should
+   (equal "\n- 2345678\n  9\n- 2345678\n  9"
+	  (org-test-with-temp-text "\n- 2345678 9\n- 2345678 9"
+	(let ((fill-column 10))
+	  (transient-mark-mode 1)
+	  (push-mark (point-min) t t)
+	  

Re: [PATCH] Fontification for inline src blocks

2021-11-30 Thread Sébastien Miquel

Hi,

Timothy writes:

Pushed .


Sorry for the late reply, but isn't there a =message= call leftover from 
debugging ?


Regards,

--
Sébastien Miquel




Re: [patch] fix ox-latex async export bug

2021-11-30 Thread Sébastien Miquel

Hi,

Nicolas Goaziou writes:

This is not really the same fix.


Indeed, but I tested it and it does work.


You're quoting a lambda, which should
not be required if the problem disappeared. IOW, the true fix probably
belong in the `org-export-async-start' function.


What happens is as follows
 1. This lambda is passed as the =post-process= variable of
    =org-export-to-file=.
 2. Which converts it to code using =`(funcall ',post-process)=
 3. Which =org-export-async-start= writes to a file, to be executed by
    the external emacs process.

I think native compilation compiles the lamdba in
=org-latex-export-to-pdf= and that there is no way to get back this
original lambda (the code) from within =org-export-to-file= or
=org-export-async-start=. Quoting the lambda prevents this
compilation.

My understanding of elisp and the native compilation business is quite
superficial, so this may be wrong.

Regards,

--
Sébastien Miquel




Re: [patch] fix ox-latex async export bug

2021-11-29 Thread Sébastien Miquel

Hi,

Nicolas Goaziou writes:

I don’t really understand why this bug happens to be honest.

The patch is already an improvement, but the beast is still lurking,
indeed.

This is most likely due to native compilation which compiles the
unquoted lambda. Once compiled, it (presumably) fails to be passed to
the external emacs process.

Attached is a patch that applies the same fix where affected.

Regards,

--
Sébastien Miquel
From 35ae093113d9a04a99b55f0747848b373a7463f3 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?S=C3=A9bastien=20Miquel?= 
Date: Mon, 29 Nov 2021 09:54:33 +0100
Subject: [PATCH] ox: Fix async export with native compilation

* lisp/ox-beamer.el (org-beamer-export-to-pdf):
* lisp/ox-icalendar.el (org-icalendar-export-to-ics):
* lisp/ox-koma-letter.el (org-koma-letter-export-to-pdf):
* lisp/ox-man.el (org-man-export-to-pdf):
* lisp/ox-texinfo.el (org-texinfo-export-to-info): Quote lambda.

Quote or name lambdas to prevent their compilation into anonymous
functions which cannot be passed to the external async emacs process.
---
 lisp/ox-beamer.el  | 2 +-
 lisp/ox-icalendar.el   | 4 ++--
 lisp/ox-koma-letter.el | 2 +-
 lisp/ox-man.el | 2 +-
 lisp/ox-texinfo.el | 2 +-
 5 files changed, 6 insertions(+), 6 deletions(-)

diff --git a/lisp/ox-beamer.el b/lisp/ox-beamer.el
index c9a67f891..3bfcd01d4 100644
--- a/lisp/ox-beamer.el
+++ b/lisp/ox-beamer.el
@@ -1059,7 +1059,7 @@ Return PDF file's name."
   (let ((file (org-export-output-file-name ".tex" subtreep)))
 (org-export-to-file 'beamer file
   async subtreep visible-only body-only ext-plist
-  (lambda (file) (org-latex-compile file)
+  #'org-latex-compile)))
 
 ;;;###autoload
 (defun org-beamer-select-environment ()
diff --git a/lisp/ox-icalendar.el b/lisp/ox-icalendar.el
index 0a682c7c8..68c5679ea 100644
--- a/lisp/ox-icalendar.el
+++ b/lisp/ox-icalendar.el
@@ -888,8 +888,8 @@ Return ICS file name."
 (org-export-to-file 'icalendar outfile
   async subtreep visible-only body-only
   '(:ascii-charset utf-8 :ascii-links-to-notes nil)
-  (lambda (file)
-	(run-hook-with-args 'org-icalendar-after-save-hook file) nil
+  '(lambda (file)
+	 (run-hook-with-args 'org-icalendar-after-save-hook file) nil
 
 ;;;###autoload
 (defun org-icalendar-export-agenda-files ( async)
diff --git a/lisp/ox-koma-letter.el b/lisp/ox-koma-letter.el
index 6a895a6a2..978e4e41f 100644
--- a/lisp/ox-koma-letter.el
+++ b/lisp/ox-koma-letter.el
@@ -982,7 +982,7 @@ Return PDF file's name."
 (org-koma-letter-special-contents))
 (org-export-to-file 'koma-letter file
   async subtreep visible-only body-only ext-plist
-  (lambda (file) (org-latex-compile file)
+  #'org-latex-compile)))
 
 
 (provide 'ox-koma-letter)
diff --git a/lisp/ox-man.el b/lisp/ox-man.el
index 6d3476cda..9a1f00f35 100644
--- a/lisp/ox-man.el
+++ b/lisp/ox-man.el
@@ -1117,7 +1117,7 @@ Return PDF file's name."
   (let ((outfile (org-export-output-file-name ".man" subtreep)))
 (org-export-to-file 'man outfile
   async subtreep visible-only body-only ext-plist
-  (lambda (file) (org-latex-compile file)
+  #'org-latex-compile)))
 
 (defun org-man-compile (file)
   "Compile a Groff file.
diff --git a/lisp/ox-texinfo.el b/lisp/ox-texinfo.el
index b0125894a..734c8a4f3 100644
--- a/lisp/ox-texinfo.el
+++ b/lisp/ox-texinfo.el
@@ -1701,7 +1701,7 @@ Return INFO file's name."
 	(org-export-coding-system org-texinfo-coding-system))
 (org-export-to-file 'texinfo outfile
   async subtreep visible-only body-only ext-plist
-  (lambda (file) (org-texinfo-compile file)
+  #'org-texinfo-compile)))
 
 ;;;###autoload
 (defun org-texinfo-publish-to-texinfo (plist filename pub-dir)
-- 
2.34.1



[BUG] org-element-at-point returns wrong element

2021-11-03 Thread Sébastien Miquel

Hi,

With point at the bol of the empty line after the keyword and before
the heading at the end of this mail, =org-element-at-point= returns
the headline element. It used to (a month ago, before all the caching)
return the keyword.

This breaks =org-element-context= which errors out when called from
the same point.

Regards,

#+LATEX_CLASS: my-class

* Head1

* Head2

--
Sébastien Miquel




[PATCH] org.el (org-display-inline-image--width): Small fix

2021-10-23 Thread Sébastien Miquel

Hi,

The variable display-line-numbers-width is nil by default.

Regards,

--
Sébastien Miquel
From 2e306750caff0474aa9a6443c7d7eb68e3aca83b Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?S=C3=A9bastien=20Miquel?= 
Date: Sat, 23 Oct 2021 13:28:59 +0200
Subject: [PATCH] org.el (org-display-inline-image--width): Small fix

---
 lisp/org.el | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/lisp/org.el b/lisp/org.el
index 474171b55..3140c4d1f 100644
--- a/lisp/org.el
+++ b/lisp/org.el
@@ -16867,7 +16867,7 @@ buffer boundaries with possible narrowing."
 (/ (or (and (bound-and-true-p visual-fill-column-mode)
 (or visual-fill-column-width auto-fill-function))
(when auto-fill-function fill-column)
-   (- (window-text-width) display-line-numbers-width))
+   (- (window-text-width) (or display-line-numbers-width 0)))
(float (window-total-width)
 width)))
((numberp org-image-actual-width)
-- 
2.33.1


Re: [PATCH] Re: Bug: org-edit-special indents inline latex [9.5 (nil @ /home/david/.emacs.d/.local/straight/build-27.2/org-mode/)]

2021-09-30 Thread Sébastien Miquel

Hi Bastien,

Bastien writes:


Sébastien, have you been able to test this patch heavily and is it
still needed, or has it been made somehow irrelevant?  (I see it does
apply well on the bugfix branch, but not on main.)

Thanks for any feedback,

I've rebased the patch on main. It is still relevant (Ihor and I have 
provided

reproducers earlier in the thread).

I had done some testing, and some more recently. The patch only affects
latex-fragments and org entities to be edited which do not need to start 
at a
beginning of line (latex-fragments, inline src blocks, footnote 
definitions).


Regards,

--
Sébastien Miquel
From b80124aa6edbd3b6992817dd8c37253705c82ae3 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?S=C3=A9bastien=20Miquel?= 
Date: Mon, 30 Aug 2021 23:18:41 +0200
Subject: [PATCH] org-src.el: Fix special editing of LaTeX fragments

* lisp/org-macs.el (org-do-remove-indentation): Add optional argument
to skip the first line.
* lisp/org-src.el (org-src--coordinates): Fix coordinates for inline
LaTeX fragments.
(org-src--contents-for-write-back): Do not indent first line for LaTeX
fragments.
(org-src--edit-element): Compute block-indentation according to parent
for LaTeX fragments.  Skip first line when removing common indentation
for LaTeX fragments.
---
 lisp/org-macs.el |  9 ++---
 lisp/org-src.el  | 18 ++
 2 files changed, 20 insertions(+), 7 deletions(-)

diff --git a/lisp/org-macs.el b/lisp/org-macs.el
index bf1340b0a..1ef89a04d 100644
--- a/lisp/org-macs.el
+++ b/lisp/org-macs.el
@@ -326,17 +326,19 @@ it for output."
 
 ;;; Indentation

-(defun org-do-remove-indentation ( n)
+(defun org-do-remove-indentation ( n skip-fl)
   "Remove the maximum common indentation from the buffer.
 When optional argument N is a positive integer, remove exactly
-that much characters from indentation, if possible.  Return nil
-if it fails."
+that much characters from indentation, if possible.  When
+optional argument SKIP-FL is non-nil, skip the first
+line.  Return nil if it fails."
   (catch :exit
 (goto-char (point-min))
 ;; Find maximum common indentation, if not specified.
 (let ((n (or n
 		 (let ((min-ind (point-max)))
 		   (save-excursion
+ (when skip-fl (forward-line))
 		 (while (re-search-forward "^[ \t]*\\S-" nil t)
 		   (let ((ind (current-indentation)))
 			 (if (zerop ind) (throw :exit nil)
@@ -344,6 +346,7 @@ if it fails."
 		   min-ind
   (if (zerop n) (throw :exit nil)
 	;; Remove exactly N indentation, but give up if not possible.
+(when skip-fl (forward-line))
 	(while (not (eobp))
 	  (let ((ind (progn (skip-chars-forward " \t") (current-column
 	(cond ((eolp) (delete-region (line-beginning-position) (point)))
diff --git a/lisp/org-src.el b/lisp/org-src.el
index 3b25fad60..d78f46186 100644
--- a/lisp/org-src.el
+++ b/lisp/org-src.el
@@ -327,7 +327,8 @@ a cons cell (LINE . COLUMN) or symbol `end'.  See also
   (if (>= pos end) 'end
 (org-with-wide-buffer
  (goto-char (max beg pos))
- (cons (count-lines beg (line-beginning-position))
+ (cons (count-lines (save-excursion (goto-char beg) (line-beginning-position))
+(line-beginning-position))
 	   ;; Column is relative to the end of line to avoid problems of
 	   ;; comma escaping or colons appended in front of the line.
 	   (- (point) (min end (line-end-position)))
@@ -445,6 +446,7 @@ Assume point is in the corresponding edit buffer."
 		  org-src--content-indentation
 		0
 	(use-tabs? (and (> org-src--tab-width 0) t))
+(preserve-fl (eq org-src--source-type 'latex-fragment))
 	(source-tab-width org-src--tab-width)
 	(contents (org-with-wide-buffer
(let ((eol (line-end-position)))
@@ -466,7 +468,8 @@ Assume point is in the corresponding edit buffer."
   ;; Add INDENTATION-OFFSET to every line in buffer,
   ;; unless indentation is meant to be preserved.
   (when (> indentation-offset 0)
-	(while (not (eobp))
+	(when preserve-fl (forward-line))
+(while (not (eobp))
 	  (skip-chars-forward " \t")
   (when (or (not (eolp))   ; not a blank line
 (and (eq (point) (marker-position marker)) ; current line
@@ -518,7 +521,13 @@ Leave point in edit buffer."
 	 (source-tab-width (if indent-tabs-mode tab-width 0))
 	 (type (org-element-type datum))
 	 (block-ind (org-with-point-at (org-element-property :begin datum)
-			  (current-indentation)))
+  (cond
+   ((save-excursion (skip-chars-backward " \t") (bolp))
+			(current-indentation))
+   ((org-element-property :parent datum)
+(org--get-expected-indentation
+ (org-element-property :parent datum) nil))
+   (t (current-indenta

Re: [kisara.moe] Re: [kisara.moe] Re: [kisara.moe] Re: [kisara.moe] Re: [kisara.moe] Re: Bug: async latex export fails due to post-process lambda [9.4.4 (release_9.4.4-188-ga8df76 @ /home/mohkale/.con

2021-09-20 Thread Sébastien Miquel

Hi,

Mohsin Kaleem writes:

Hi, just following up. This is still an issue.


I can confirm this. I've run into the same issue and fix.

Are you using native compilation ? I think this must be the cause, 
though no one

else could reproduce. If I byte compile the function instead, things work.

Regards,

--
Sébastien Miquel




[PATCH] Re: Bug: org-edit-special indents inline latex [9.5 (nil @ /home/david/.emacs.d/.local/straight/build-27.2/org-mode/)]

2021-08-31 Thread Sébastien Miquel

Hi,

Ihor Radchenko writes:

Dávid Jakab  writes:

When using org-edit-special to edit inline latex, i.e., equations
between \( and \), in an org-mode buffer, a number of
spaces may get inserted before \( after the latex editing minibuffer is
closed.

The attached patch fixes this as well as a couple other issues with special
editing of latex fragments :
 + the coordinates computation was wrong, so point position inside fragment
   isn't preserved when calling ~org-edit-special~ from an inline fragment.
 + common leading indentation wasn't getting trimmed when editing a 
multiline

   fragment inside an org list such as
   $$abc
   def$$

Thanks for reporting and confirming.

Regards,

--
Sébastien Miquel

>From 5c3254d42e3d359021d41dae9a0549244e6fddff Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?S=C3=A9bastien=20Miquel?= 
Date: Mon, 30 Aug 2021 23:18:41 +0200
Subject: [PATCH] org-src.el: Fix special editing of LaTeX fragments

* lisp/org-macs.el (org-do-remove-indentation): Add optional argument
to skip the first line.
* lisp/org-src.el (org-src--coordinates): Fix coordinates for inline
LaTeX fragments.
(org-src--contents-for-write-back): Do not indent first line for LaTeX
fragments.
(org-src--edit-element): Compute block-indentation according to parent
for LaTeX fragments.  Skip first line when removing common indentation
for LaTeX fragments.
---
 lisp/org-macs.el |  9 ++---
 lisp/org-src.el  | 18 ++
 2 files changed, 20 insertions(+), 7 deletions(-)

diff --git a/lisp/org-macs.el b/lisp/org-macs.el
index 77458db96..5c90c92f6 100644
--- a/lisp/org-macs.el
+++ b/lisp/org-macs.el
@@ -326,17 +326,19 @@ it for output."
 
 ;;; Indentation
 
-(defun org-do-remove-indentation ( n)
+(defun org-do-remove-indentation ( n skip-fl)
   "Remove the maximum common indentation from the buffer.
 When optional argument N is a positive integer, remove exactly
-that much characters from indentation, if possible.  Return nil
-if it fails."
+that much characters from indentation, if possible.  When
+optional argument SKIP-FL is non-nil, skip the first
+line.  Return nil if it fails."
   (catch :exit
 (goto-char (point-min))
 ;; Find maximum common indentation, if not specified.
 (let ((n (or n
 		 (let ((min-ind (point-max)))
 		   (save-excursion
+ (when skip-fl (forward-line))
 		 (while (re-search-forward "^[ \t]*\\S-" nil t)
 		   (let ((ind (current-indentation)))
 			 (if (zerop ind) (throw :exit nil)
@@ -344,6 +346,7 @@ if it fails."
 		   min-ind
   (if (zerop n) (throw :exit nil)
 	;; Remove exactly N indentation, but give up if not possible.
+(when skip-fl (forward-line))
 	(while (not (eobp))
 	  (let ((ind (progn (skip-chars-forward " \t") (current-column
 	(cond ((eolp) (delete-region (line-beginning-position) (point)))
diff --git a/lisp/org-src.el b/lisp/org-src.el
index 4698c6dd2..2e72b1755 100644
--- a/lisp/org-src.el
+++ b/lisp/org-src.el
@@ -324,7 +324,8 @@ a cons cell (LINE . COLUMN) or symbol `end'.  See also
   (if (>= pos end) 'end
 (org-with-wide-buffer
  (goto-char (max beg pos))
- (cons (count-lines beg (line-beginning-position))
+ (cons (count-lines (save-excursion (goto-char beg) (line-beginning-position))
+(line-beginning-position))
 	   ;; Column is relative to the end of line to avoid problems of
 	   ;; comma escaping or colons appended in front of the line.
 	   (- (point) (min end (line-end-position)))
@@ -442,6 +443,7 @@ Assume point is in the corresponding edit buffer."
 		  org-src--content-indentation
 		0
 	(use-tabs? (and (> org-src--tab-width 0) t))
+(preserve-fl (eq org-src--source-type 'latex-fragment))
 	(source-tab-width org-src--tab-width)
 	(contents (org-with-wide-buffer (buffer-string)))
 	(write-back org-src--allow-write-back))
@@ -456,7 +458,8 @@ Assume point is in the corresponding edit buffer."
   ;; Add INDENTATION-OFFSET to every line in buffer,
   ;; unless indentation is meant to be preserved.
   (when (> indentation-offset 0)
-	(while (not (eobp))
+	(when preserve-fl (forward-line))
+(while (not (eobp))
 	  (skip-chars-forward " \t")
 	  (let ((i (current-column)))
 	(delete-region (line-beginning-position) (point))
@@ -504,7 +507,13 @@ Leave point in edit buffer."
 	 (source-tab-width (if indent-tabs-mode tab-width 0))
 	 (type (org-element-type datum))
 	 (block-ind (org-with-point-at (org-element-property :begin datum)
-			  (current-indentation)))
+  (cond
+   ((save-excursion (skip-chars-backward " \t") (bolp))
+			(current-indentation))
+   ((org-element-property :parent datum)
+(org--get-expected-indentation
+ (org-element-property :parent datum) nil))
+

Re: Bug: [PATCH] Can't set background color of latex fragment

2021-08-29 Thread Sébastien Miquel

Sébastien Miquel writes:

Here's a patch that fixes this bug by calling `dvipng' with the `-bg
Transparent' argument only when no background color is set.


Setting X-Woof-Bug and X-Woof-Patch, as this doesn't show on 
https://updates.orgmode.org/


--
Sébastien Miquel




Re: [BUG] Async pdf export broken with native-comp

2021-07-16 Thread Sébastien Miquel

Hi,

Timothy writes:
> FWIW I use native-comp (with Doom + my config) and async PDF export
> works.

Would you mind checking that ~org-latex-export-to-pdf~ is native 
compiled, using

=describe-function= ?

I've updated to latest emacs master and I'm still hitting this issue,
although the steps to reproduce in my original mail are wrong :

 + with =emacs -q= async export fails, but for a different reason : I get a
   =wrong argument stringp, nil= error because ~user-init-file~ is nil 
inside

   ~org-export-async-start~.
 + The issue I described occurs when you use an empty init.el file and 
start

   =emacs=. Perhaps you need to make sure that ~org-latex-export-to-pdf~ is
   native compiled.

Regards,

--
Sébastien Miquel




[BUG] Async pdf export broken with native-comp

2021-07-15 Thread Sébastien Miquel

Hi,

The async pdf export functionality appears to be broken with latest org 
and a

recent emacs version compiled with native-comp enabled (I have not tested
without native-comp).

To reproduce:

 - use `emacs -q` and an empty init.el file (your init file gets picked 
up by

   the async emacs instance)
 - (setq org-export-async-debug t)
 - find any org file and hit =C-c C-e C-a C-l C-p= to export as pdf file.

The async process will exit abnormaly.

The issue stems from this line in `org-export-to-file`
 :   (ignore-errors (funcall ',post-process ,file))

In `org-latex-export-to-pdf`, this `post-process` is set to
 :   (lambda (file) (org-latex-compile file))

I think native-comp compiles this lambda, which messes things up.

As a fix, you can quote the lambda in `org-latex-export-to-pdf`
 :   '(lambda (file) (org-latex-compile file))

The same applies to other backends but I don't know if it's the right 
thing to do.


Regards,

--
Sébastien Miquel




Re: [PATCH] Do not throw error when parameter of :tangle is not a string

2021-07-08 Thread Sébastien Miquel

Hi,

Jacopo De Simoi writes:

  in the current master branch, if the parameter :tangle of a src block is not
a string, tangling fails by throwing  an error when calling `file-name-
directory'  This patch checks if the parameter is a string before calling
`file-name-directory'.

This makes construct such as :tangle (when condition-applies "filename") work
again (as they did a few versions ago…)

Thanks for the patch. It looks good to me (had to run `dos2unix' to apply).

To clarify, it fixes `:tangle (when condition-applies "filename")' when the
condition doesn't apply, such as `(when nil "filename")'.

Regards,

--
Sébastien Miquel




Re: Large source block causes org-mode to be unusable

2021-06-28 Thread Sébastien Miquel

Hi,

Léo Ackermann writes:

@EricSFrada, would you mind sharing your code for your proof sections ?

This functionality is now built-in: headings with an `ignore' tag do not get
exported (their contents do). For very large proof, this seems like the 
right

thing to do.

In small to moderate sized blocks, the delay can still be noticeable and 
ought

to be fixed. Attached is a patch that seems to resolve this issue. I haven't
noticed any drawbacks so far.

Regards,

--
Sébastien Miquel

>From d843bdc5887a6e50a57e349128ebbe032086dc17 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?S=C3=A9bastien=20Miquel?= 
Date: Sun, 27 Jun 2021 16:24:22 +0200
Subject: [PATCH] WIP : do not refontify special blocks

---
 lisp/org.el | 99 ++---
 1 file changed, 64 insertions(+), 35 deletions(-)

diff --git a/lisp/org.el b/lisp/org.el
index 1bd9e02eb..9fd3f8514 100644
--- a/lisp/org.el
+++ b/lisp/org.el
@@ -5265,22 +5265,32 @@ by a #."
   (org-fontify-meta-lines-and-blocks-1 limit)
 (error (message "Org mode fontification error in %S at %d"
 		(current-buffer)
-		(line-number-at-pos)
+		(line-number-at-pos
+  nil)
 
 (defun org-fontify-meta-lines-and-blocks-1 (limit)
   "Fontify #+ lines and blocks."
-  (let ((case-fold-search t))
-(when (re-search-forward
-	   (rx bol (group (zero-or-more (any " \t")) "#"
-			  (group (group (or (seq "+" (one-or-more (any "a-zA-Z")) (optional ":"))
-	(any " \t")
-	eol))
- (optional (group "_" (group (one-or-more (any "a-zA-Z"))
-			  (zero-or-more (any " \t"))
-			  (group (group (zero-or-more (not (any " \t\n"
- (zero-or-more (any " \t"))
- (group (zero-or-more any)
-	   limit t)
+  (let* ((case-fold-search t)
+ (fl-beg (point))
+ (fl-end limit)
+ (fbeg (when (and (> fl-beg (point-min))
+  (get-text-property (1- fl-beg) 'font-lock-multiline-block))
+ (or (previous-single-property-change
+  fl-beg 'font-lock-multiline-block)
+ (point-min)
+(when fbeg (goto-char fbeg))
+(while (and (< (point) limit)
+(re-search-forward
+	 (rx bol (group (zero-or-more (any " \t")) "#"
+			(group (group (or (seq "+" (one-or-more (any "a-zA-Z")) (optional ":"))
+	  (any " \t")
+	  eol))
+   (optional (group "_" (group (one-or-more (any "a-zA-Z"))
+			(zero-or-more (any " \t"))
+			(group (group (zero-or-more (not (any " \t\n"
+   (zero-or-more (any " \t"))
+   (group (zero-or-more any)
+	 limit t))
   (let ((beg (match-beginning 0))
 	(end-of-beginline (match-end 0))
 	;; Including \n at end of #+begin line will include \n
@@ -5318,7 +5328,7 @@ by a #."
 	  (remove-text-properties beg end-of-endline
   '(display t invisible t intangible t)))
 	(add-text-properties
-	 beg end-of-endline '(font-lock-fontified t font-lock-multiline t))
+	 beg end-of-endline '(font-lock-fontified t font-lock-multiline-block t))
 	(org-remove-flyspell-overlays-in beg bol-after-beginline)
 	(org-remove-flyspell-overlays-in nl-before-endline end-of-endline)
 	(cond
@@ -5327,7 +5337,8 @@ by a #."
 	  (add-text-properties bol-after-beginline block-end '(src-block t)))
 	 (quoting
 	  (add-text-properties
-	   bol-after-beginline beg-of-endline
+	   (max bol-after-beginline (or fl-beg bol-after-beginline))
+   (min beg-of-endline (or fl-end beg-of-endline))
 	   (list 'face
 		 (list :inherit
 			   (let ((face-name
@@ -5426,26 +5437,41 @@ by a #."
 	(add-text-properties closing-start end '(invisible t)))
 	  t)
 
-(defun org-fontify-extend-region (beg end _old-len)
-  (let ((begin-re "\\(\\[\\|\\(#\\+begin_\\|begin{\\)\\S-+\\)")
-	(end-re "\\(\\]\\|\\(#\\+end_\\|end{\\)\\S-+\\)")
-	(extend
- (lambda (r1 r2 dir)
-	   (let ((re (replace-regexp-in-string
-  "\\(begin\\|end\\)" r1
-		  (replace-regexp-in-string
-   "[][]" r2
-		   (match-string-no-properties 0)
-	 (re-search-forward (regexp-quote re) nil t dir)
-(save-match-data
-  (save-excursion
-	(goto-char beg)
-	(back-to-indentation)
-	(cond ((looking-at end-re)
-	   (cons (or (funcall extend "begin" "[" -1) beg) end))
-	  ((looking-at begin-re)
-	   (cons beg (or (funcall extend "end" "]" 1) end)))
-	  (t (cons beg end)))
+(defun org-fontify-extend-region (bego endo _old-len)
+  (let* ((beg bego) 

Re: [PATCH] extra space at the end of lines in source

2021-06-26 Thread Sébastien Miquel

Greg Minshall writes:

thanks.  my trivial test shows this works*except*  in the particular
case where, when closing the Org Src buffer, `point` is on an empty
line.  in this case, that one empty line is given extra spaces.
Yes, I was aware of this, but didn't think we could do better in this 
case. I've

thought about it some more, and came up with a solution.

Here's a new patch that's smarter about indenting the current line, and 
resolves

this remaining issue.

Regards,

--
Sébastien Miquel

>From 8c31e6b732a45c0ddbbf0a0db7a2cb4ef3af0414 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?S=C3=A9bastien=20Miquel?= 
Date: Wed, 23 Jun 2021 15:25:58 +0200
Subject: [PATCH] org-src.el: Do not indent blank lines, except current one

* lisp/org-src.el (org-src--contents-for-write-back): Do not indent blank lines, except for the
current line maybe.
(org-src--preserve-blank-line): New variable, whether to preserve
indentation of the current blank line.
(org-src--edit-element): Set `org-src--preserve-blank-line'.
* lisp/org.el (org-indent-line): When tab acts natively, do some
preindentation, which signals `org-src--edit-element' to
preserve the indentation of current blank line.

Removing all the whitespace was the original behaviour for all blank lines, before `857ae366b3`.
---
 lisp/org-src.el | 34 +++---
 lisp/org.el | 15 +++
 2 files changed, 42 insertions(+), 7 deletions(-)

diff --git a/lisp/org-src.el b/lisp/org-src.el
index 79f002e56..c6311adbb 100644
--- a/lisp/org-src.el
+++ b/lisp/org-src.el
@@ -299,6 +299,9 @@ is 0.")
   "File name associated to Org source buffer, or nil.")
 (put 'org-src-source-file-name 'permanent-local t)
 
+(defvar-local org-src--preserve-blank-line nil)
+(put 'org-src--preserve-blank-line 'permanent-local t)
+
 (defun org-src--construct-edit-buffer-name (org-buffer-name lang)
   "Construct the buffer name for a source editing buffer."
   (concat "*Org Src " org-buffer-name "[ " lang " ]*"))
@@ -443,14 +446,21 @@ Assume point is in the corresponding edit buffer."
 		0
 	(use-tabs? (and (> org-src--tab-width 0) t))
 	(source-tab-width org-src--tab-width)
-	(contents (org-with-wide-buffer (buffer-string)))
-	(write-back org-src--allow-write-back))
+	(contents (org-with-wide-buffer
+   (let ((eol (line-end-position)))
+ (list (buffer-substring (point-min) eol)
+   (buffer-substring eol (point-max))
+	(write-back org-src--allow-write-back)
+(preserve-blank-line org-src--preserve-blank-line)
+marker)
 (with-current-buffer write-back-buf
   ;; Reproduce indentation parameters from source buffer.
   (setq indent-tabs-mode use-tabs?)
   (when (> source-tab-width 0) (setq tab-width source-tab-width))
   ;; Apply WRITE-BACK function on edit buffer contents.
-  (insert (org-no-properties contents))
+  (insert (org-no-properties (car contents)))
+  (setq marker (point-marker))
+  (insert (org-no-properties (car (cdr contents
   (goto-char (point-min))
   (when (functionp write-back) (save-excursion (funcall write-back)))
   ;; Add INDENTATION-OFFSET to every line in buffer,
@@ -458,10 +468,14 @@ Assume point is in the corresponding edit buffer."
   (when (> indentation-offset 0)
 	(while (not (eobp))
 	  (skip-chars-forward " \t")
-	  (let ((i (current-column)))
-	(delete-region (line-beginning-position) (point))
-	(indent-to (+ i indentation-offset)))
-	  (forward-line))
+  (when (or (not (eolp))   ; not a blank line
+(and (eq (point) (marker-position marker)) ; current line
+ preserve-blank-line))
+	(let ((i (current-column)))
+	  (delete-region (line-beginning-position) (point))
+	  (indent-to (+ i indentation-offset
+	  (forward-line)))
+  (set-marker marker nil
 
 (defun org-src--edit-element
 (datum name  initialize write-back contents remote)
@@ -506,6 +520,11 @@ Leave point in edit buffer."
 	 (block-ind (org-with-point-at (org-element-property :begin datum)
 			  (current-indentation)))
 	 (content-ind org-edit-src-content-indentation)
+ (blank-line (save-excursion (beginning-of-line)
+ (looking-at-p "^[[:space:]]*$")))
+ (empty-line (and blank-line (looking-at-p "^$")))
+ (preserve-blank-line (or (and blank-line (not empty-line))
+  (and empty-line (= (+ block-ind content-ind) 0
 	 (preserve-ind
 	  (and (memq type '(example-block src-block))
 		   (or (org-element-property :preserve-indent datum)
@@ -554,6 +573,7 @@ Leave point in edit buffer."
 	(setq org-src--overlay overlay)
 	(setq org-src--allow-write-back write-back)
 	(setq org-src-sour

Re: [PATCH] extra space at the end of lines in source

2021-06-23 Thread Sébastien Miquel

Greg Minshall writes:

- the next time i open the Org Src buffer, whatever lint-like process is
   running for that language may complain about extra spaces at the end
   of a line.  (does that mean my experience is different from yours, or
   at least from your expectation?)
If I try your original examples with `emacs -q' I do not get extra 
whitespace in
the org src buffer. Those two spaces in the original org buffer -- that 
are due
to `org-edit-src-content-indentation' -- are removed in the org src 
buffer. If

you do not find it to be the case, then I'm missing something.

Anyway, here's a patch that cleans up blank lines, except the current 
one. It

preserves the fix for the original issue.

Can you try it out ?

Regards,

--
Sébastien Miquel

>From 405c2be7487c564e72a9f01a940f96dc19ff16ad Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?S=C3=A9bastien=20Miquel?= 
Date: Wed, 23 Jun 2021 15:25:58 +0200
Subject: [PATCH] org-src.el (org-src--contents-for-write-back): Do not indent
 blank lines

* lisp/org-src.el (org-src--contents-for-write-back): Do not indent
blank lines, except for the current line.

This was the original behaviour for all blank lines, before `857ae366b3`.
---
 lisp/org-src.el | 23 ---
 1 file changed, 16 insertions(+), 7 deletions(-)

diff --git a/lisp/org-src.el b/lisp/org-src.el
index 79f002e56..faacb53e3 100644
--- a/lisp/org-src.el
+++ b/lisp/org-src.el
@@ -443,14 +443,20 @@ Assume point is in the corresponding edit buffer."
 		0
 	(use-tabs? (and (> org-src--tab-width 0) t))
 	(source-tab-width org-src--tab-width)
-	(contents (org-with-wide-buffer (buffer-string)))
-	(write-back org-src--allow-write-back))
+	(contents (org-with-wide-buffer
+   (let ((eol (progn (end-of-line) (point
+ (list (buffer-substring (point-min) eol)
+   (buffer-substring eol (point-max))
+	(write-back org-src--allow-write-back)
+marker)
 (with-current-buffer write-back-buf
   ;; Reproduce indentation parameters from source buffer.
   (setq indent-tabs-mode use-tabs?)
   (when (> source-tab-width 0) (setq tab-width source-tab-width))
   ;; Apply WRITE-BACK function on edit buffer contents.
-  (insert (org-no-properties contents))
+  (insert (org-no-properties (car contents)))
+  (setq marker (point-marker))
+  (insert (org-no-properties (car (cdr contents
   (goto-char (point-min))
   (when (functionp write-back) (save-excursion (funcall write-back)))
   ;; Add INDENTATION-OFFSET to every line in buffer,
@@ -458,10 +464,13 @@ Assume point is in the corresponding edit buffer."
   (when (> indentation-offset 0)
 	(while (not (eobp))
 	  (skip-chars-forward " \t")
-	  (let ((i (current-column)))
-	(delete-region (line-beginning-position) (point))
-	(indent-to (+ i indentation-offset)))
-	  (forward-line))
+  (when (or (not (eolp)) ; ignore blank lines
+(eq (point) (marker-position marker)))
+	(let ((i (current-column)))
+	  (delete-region (line-beginning-position) (point))
+	  (indent-to (+ i indentation-offset
+	  (forward-line)))
+  (set-marker marker nil
 
 (defun org-src--edit-element
 (datum name  initialize write-back contents remote)
-- 
2.32.0



Re: extra space at the end of lines in source

2021-06-22 Thread Sébastien Miquel
For the record, the original issue is that with 
`org-src-tab-acts-natively' set
to t (which is the new default) you couldn't use tab to indent an empty 
line, and

electric-indent-mode wouldn't work properly.

Greg Minshall writes:

i don't know what
would be involved to keep the fix for the original problem (which i was
also seeing), without adding these extra spaces.  but, i suspect it
would be worth it, if feasible.


I think the only way would be to change 
`org-src--contents-for-write-back' to

keep track of where the point was in the edit buffer, and clean up spaces in
other lines. Doesn't seem difficult ; maybe you could replace the 
`buffer-string`

call by two calls, to get the part before point and the one after, then work
with that.


it's long-term emacs behavior to eliminate spaces
at the end of lines, at least in programming modes.
As for the `org-src--content-indentation' spaces, they are quite 
peculiar. Note
that they are removed when you call =C-c '= and when you tangle (they 
should,
but I haven't tested it), so strictly speaking, they are removed in 
programming
modes. What if your org block itself is indented, do you also expect 
blank lines

to be entirely empty ?

As for additional indentation in a blank line, it will indeed never get 
cleaned

up by org, which isn't ideal. But then, spaces at the end of non blank lines
don't get cleaned up either (I think) and never were. It's up to the user to
remove them, in the appropriate language mode.

Despite these arguments, I have no opinion on the matter.

Regards,

--
Sébastien Miquel




Re: extra space at the end of lines in source

2021-06-22 Thread Sébastien Miquel

Hi Greg,

Greg Minshall writes:

hi.  i can't date it exactly, but in the last week or so, editing a
source buffer (with =C-c '=) adds spaces (to the "tab location") of
previously blank lines.


See https://lists.gnu.org/archive/html/emacs-orgmode/2021-05/msg00867.html

The goal of the change was to fix some (electric) indentation issues when
editing a src block directly.

As I write in the linked thread, setting `org-src--preserve-indentation' 
should

revert this behaviour.

Regards,

--
Sébastien Miquel




Re: org-edit-src-exit randomizes / mixes up code in source-buffer on exit

2021-06-22 Thread Sébastien Miquel

Hi,

This has been reported before.

There's a patch that fixes this here : 
https://lists.gnu.org/archive/html/emacs-orgmode/2021-06/msg7.html


To fix this bug, you can can either apply this patch, downgrade org, or 
update emacs to 27.


Could anyone with commit access have a look and apply this patch to 
master ?


Regards,

--
Sébastien Miquel




Re: Large source block causes org-mode to be unusable

2021-06-21 Thread Sébastien Miquel

Hi Léo,

Léo Ackermann writes:
I am working in an org-file of reasonable size (<2000 lines): my first 
paper written in org-mode. Everything fine (and fast) until I started 
to add `#+BEGIN_proof / #+END_proof` within my .org to make my .pdf 
export prettier. This caused the editing of the proofs to be very 
slow: navigation within the proof is fast but adding/removing any char 
takes around 4s per char.
It seems that the fontify function is responsible for that (see 
screenshot). As far as I understand, this function tries to fontify 
the whole block as soon as a single char is modified. In my case, it 
then tries to fontify a whole proof (~4 pages in my .pdf, with many 
LaTeX formulas) several times per second...

You can try setting org-highlight-latex-and-related to '(latex) instead of
'(native).

Even with this setting, latex fontification in special blocks is slow. The
reason being that the whole block has the `font-lock-multiline' 
property, hence

every char insertion triggers refontification of the whole block.

In the function `org-do-latex-and-related', I commented out the second 
condition

of the `cond' form, which makes calls to `face-at-point'. This yields a
significant speedup, and was enough to make things bearable in my cases. 
You can

also try to simplify the latex regexp.

If you try these, I'd be interested to hear how much of an improvement 
theymake.


Regards,

--
Sébastien Miquel




Re: Default entry in org-src-block-faces

2021-06-19 Thread Sébastien Miquel

Hi,

Augusto Stoffel writes:

Would it make sense to allow a default entry (t FACE) in
`org-src-block-faces', to be applied to all src blocks without a more
specific entry in that alist?  There seems to be no other way to specify
a face to be applied to all src blocks, and only src blocks.


Makes sense to me.

I also wonder if there's any reason for the `org-block' face not to 
apply to "greater" blocks.


--
Sébastien Miquel




Re: An org-latex face

2021-06-01 Thread Sébastien Miquel

Sébastien Miquel writes:

There's already an `org-latex-and-relatex` regexp


I meant the `org-latex-and-related` face.

--
Sébastien Miquel




Re: [PATCH] source blocks mangled when edited

2021-06-01 Thread Sébastien Miquel

Michael Gauland writes:

This is all the*trace-output*  buffer shows:

==
1 -> (replace-buffer-contents #)
1 <- replace-buffer-contents: nil

Indeed, the `replace-buffer-contents` call is failing.

I've been able to reproduce with earlier versions of emacs 26.1. With
later versions of emacs 26, the problem goes away.

It seems earlier versions of `replace-buffer-contents` are not quite
reliable. It was patched prior to 27.1 and the new documentation
string makes some guarantees of correctness, so let's just change the
minimal version to 27.1.

Thank you for the report.

Regards,

--
Sébastien Miquel

X-Woof-Bug: confirmed

>From 8ebdbc5eca92de4429d3994a3663d8aa3b9877fe Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?S=C3=A9bastien=20Miquel?= 
Date: Tue, 1 Jun 2021 08:56:48 +0200
Subject: [PATCH] org-src.el: Use `replace-buffer-contents' only for emacs >=
 27

* lisp/org-src.el: Use `replace-buffer-contents' only for emacs >= 27.

It was introduced in emacs 26.1, but earlier versions made no
guarantees of correctness.
---
 lisp/org-src.el | 8 
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/lisp/org-src.el b/lisp/org-src.el
index 79f002e56..4698c6dd2 100644
--- a/lisp/org-src.el
+++ b/lisp/org-src.el
@@ -1199,12 +1199,12 @@ Throw an error if there is no such buffer."
   ;; insert new contents.
   (delete-overlay overlay)
   (let ((expecting-bol (bolp)))
-	(if (version< emacs-version "26.1")
+	(if (version< emacs-version "27.1")
 	(progn (delete-region beg end)
 		   (insert (with-current-buffer write-back-buf (buffer-string
 	(save-restriction
 	  (narrow-to-region beg end)
-	  (replace-buffer-contents write-back-buf)
+	  (replace-buffer-contents write-back-buf 0.1 nil)
 	  (goto-char (point-max
 	(when (and expecting-bol (not (bolp))) (insert "\n")))
   (kill-buffer write-back-buf)
@@ -1246,13 +1246,13 @@ Throw an error if there is no such buffer."
(undo-boundary)
(goto-char beg)
(let ((expecting-bol (bolp)))
-	 (if (version< emacs-version "26.1")
+	 (if (version< emacs-version "27.1")
 	 (progn (delete-region beg end)
 		(insert (with-current-buffer write-back-buf
   (buffer-string
 	 (save-restriction
 	   (narrow-to-region beg end)
-	   (replace-buffer-contents write-back-buf)
+	   (replace-buffer-contents write-back-buf 0.1 nil)
 	   (goto-char (point-max
 	 (when (and expecting-bol (not (bolp))) (insert "\n")
 (when write-back-buf (kill-buffer write-back-buf))
-- 
2.31.1



Re: An org-latex face

2021-06-01 Thread Sébastien Miquel

Hi Léo,

Léo Ackermann writes:

When it comes to preview inline LaTeX fragments within org-mode, the 
org package applies the `org-block` face. It would be nice to *treat 
inline latex block specially*, to make integration of the those 
preview-block easier when surrounded by plaintext. This feature would 
allow to have a theme that highlight blocks (source code, examples) 
without the background artifact you can see in attachment when 
previewing latex fragments.

There's already an `org-latex-and-relatex` regexp, that will be used
if you set `org-highlight-latex-and-related` to `'(latex)` instead of
`'(native)`. The org-block face won't be applied anymore.

Regards,

--
Sébastien Miquel




Re: source blocks mangled when edited

2021-05-31 Thread Sébastien Miquel

Michael Gauland writes:

I didn't instrument the functions, but found that there are two places
that test '(if (version< emacs-version "26.1"...'. If I change that to
use "version<=", the problem goes away (I'm still running 26.1). I don't
know whether this is the right fix (the underlying problem may be a
quirk in my system, and this is just masking it).  I'm hesitant to
submit a patch unless someone else can replicate the problem.

The commit `d02ad1f207e1579aff8f36f740a065d71472c182` introduced the
use of `replace-buffer-contents` when available. This function was
introduced in emacs 26.1, hence the version tests.

There must be an issue with the `replace-buffer-contents` calls. Can
you call `trace-function` on `replace-buffer-contents` and trigger the
behaviour again to see what it returns ?

You said it also happens with `emacs -q`, right ? I'll try to see if I
can reproduce with emacs 26.1 tomorrow.

--
Sébastien Miquel




Re: source blocks mangled when edited

2021-05-31 Thread Sébastien Miquel

Hi Michael,

Michael Gauland writes:

The file has two identical source blocks. The first generally behaves
fine, though some lines get extra indentation.

The second suffers more serious distortions. For example, the first line
changes from "digraph G {" to "aph G {".


I'm unable to reproduce with a recent emacs version.


I'm not even sure how to start tracking this down. Any help would be
greatly appreciated!

The relevant functions are `org-edit-src-exit` and perhaps
`org-src--contents-for-write-back`.

Can you instrument these functions to see what's happening ? Is
`org-src--contents-for-write-back` populating the buffer correctly ?
Does the `replace-buffer-contents` call in `org-edit-src-exit` return
`t` ?

Regards,

--
Sébastien Miquel




Re: [PATCH] Bug: No highlighting after [9.4.6 (release_9.4.6-544-gc5573b @ /home/user/.emacs.d/straight/build/org/)]

2021-05-28 Thread Sébastien Miquel

Hi,

Axel Svensson writes:

Steps to reproduce:
1) In an empty org-mode buffer, type a * SPC a C-b C-b C-b .
2) The buffer now has two lines, the first one "a" and the second "* a".

Expected:
3) The second line is formatted as a first-level heading.

Actual:
3) The second line is formatted as normal text.


Thanks for the report, I can confirm this bug with latest org.

Here's a patch that fixes this.

Regards,

--
Sébastien Miquel

>From c598f705bf1d8003514751fffc07fd64620a7e42 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?S=C3=A9bastien=20Miquel?= 
Date: Fri, 28 May 2021 21:14:22 +0200
Subject: [PATCH] org.el (org-fontify-extend-region): Fix headline
 fontification in edge case

* lisp/org.el (org-fontify-extend-region): Fix fontification of
headline or meta line created by inserting a newline.

Unrelated to the fix: `org-fontify-extend-region' is added to
`font-lock-extend-after-change-region-function' and doesn't need to
use `save-excursion'.
---
 lisp/org.el | 19 ++-
 1 file changed, 10 insertions(+), 9 deletions(-)

diff --git a/lisp/org.el b/lisp/org.el
index 1bd9e02eb..b7b1416bd 100644
--- a/lisp/org.el
+++ b/lisp/org.el
@@ -5427,7 +5427,9 @@ by a #."
 	  t)

 (defun org-fontify-extend-region (beg end _old-len)
-  (let ((begin-re "\\(\\[\\|\\(#\\+begin_\\|begin{\\)\\S-+\\)")
+  (let ((end (if (progn (goto-char end) (looking-at-p "^[*#]"))
+ (1+ end) end))
+(begin-re "\\(\\[\\|\\(#\\+begin_\\|begin{\\)\\S-+\\)")
 	(end-re "\\(\\]\\|\\(#\\+end_\\|end{\\)\\S-+\\)")
 	(extend
  (lambda (r1 r2 dir)
@@ -5437,15 +5439,14 @@ by a #."
"[][]" r2
 		   (match-string-no-properties 0)
 	 (re-search-forward (regexp-quote re) nil t dir)
+(goto-char beg)
+(back-to-indentation)
 (save-match-data
-  (save-excursion
-	(goto-char beg)
-	(back-to-indentation)
-	(cond ((looking-at end-re)
-	   (cons (or (funcall extend "begin" "[" -1) beg) end))
-	  ((looking-at begin-re)
-	   (cons beg (or (funcall extend "end" "]" 1) end)))
-	  (t (cons beg end)))
+  (cond ((looking-at end-re)
+	 (cons (or (funcall extend "begin" "[" -1) beg) end))
+	((looking-at begin-re)
+	 (cons beg (or (funcall extend "end" "]" 1) end)))
+	(t (cons beg end))

 (defun org-activate-footnote-links (limit)
   "Add text properties for footnotes."
-- 
2.31.1


Re: Question Regarding Yasnippet With Org Mode (Emacs 27.2)

2021-05-23 Thread Sébastien Miquel

Samuel Banya writes:
Do you think that maybe changing the setting you had mentioned before, 
'org-src-tab-acts-natively' to false (aka nil or '0' (zero) value) via 
a change in my configuration would make this error not happen within 
Org-Mode in that case?


Yes, that should work. I have not tested it though.

Regards,

--
Sébastien Miquel




Re: Question Regarding Yasnippet With Org Mode (Emacs 27.2)

2021-05-23 Thread Sébastien Miquel

Hi Samuel,

I'm guessing its some kind of Org Mode vs Yasnippet issue where 
Org-mode is expanding it too fast, when it should wait for user input 
hence the "$1" section.


I guess yasnippet tries to indent the inside of the block (see
=yas-indent-line=) before the lang part of the src block is specified.

In Emacs 27.2, the default value of =org-src-tab-acts-natively= was
changed to `t`. With this setting, trying to indent a src block with
no language results in this error.

--
Sébastien Miquel




Re: Bug: [PATCH] Can't set background color of latex fragment

2021-05-23 Thread Sébastien Miquel

Hi,

Here's a patch that fixes this bug by calling `dvipng' with the `-bg
Transparent' argument only when no background color is set.

Regards,

--
Sébastien Miquel

>From 5872fc3143162fbda11cf2aa5a3798567664be99 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?S=C3=A9bastien=20Miquel?= 
Date: Sun, 23 May 2021 22:07:25 +0200
Subject: [PATCH] org.el (org-create-formula-image): Fix ignored background
 color

* lisp/org.el (org-preview-latex-process-alist): add a `:transparent-image-converter'
property for `dvipng'.
(org-create-formula-image): If available, use
`:transparent-image-converter' when no background color is set.
---
 lisp/org.el | 14 +++---
 1 file changed, 11 insertions(+), 3 deletions(-)

diff --git a/lisp/org.el b/lisp/org.el
index 1bd9e02eb..d544e62fb 100644
--- a/lisp/org.el
+++ b/lisp/org.el
@@ -3319,7 +3319,9 @@ All available processes and theirs documents can be found in
  :image-output-type "png"
  :image-size-adjust (1.0 . 1.0)
  :latex-compiler ("latex -interaction nonstopmode -output-directory %o %f")
- :image-converter ("dvipng -D %D -T tight -bg Transparent -o %O %f"))
+ :image-converter ("dvipng -D %D -T tight -o %O %f")
+ :transparent-image-converter
+ ("dvipng -D %D -T tight -bg Transparent -o %O %f"))
 (dvisvgm
  :programs ("latex" "dvisvgm")
  :description "dvi > svg"
@@ -3374,6 +3376,9 @@ PROPERTIES accepts the following attributes:
   given to the shell and supports any of the following
   place-holders defined below.

+If set, :transparent-image-converter is used instead of :image-converter to
+convert an image when the background color is nil or \"Transparent\".
+
 Place-holders used by `:image-converter' and `:latex-compiler':

   %finput file name
@@ -16288,7 +16293,6 @@ a HTML file."
 	   org-format-latex-header
 	   'snippet)))
 	 (latex-compiler (plist-get processing-info :latex-compiler))
-	 (image-converter (plist-get processing-info :image-converter))
 	 (tmpdir temporary-file-directory)
 	 (texfilebase (make-temp-name
 		   (expand-file-name "orgtex" tmpdir)))
@@ -16302,7 +16306,11 @@ a HTML file."
 		 "Black"))
 	 (bg (or (plist-get options (if buffer :background :html-background))
 		 "Transparent"))
-	 (log-buf (get-buffer-create "*Org Preview LaTeX Output*"))
+	 (image-converter
+  (or (and (string= bg "Transparent")
+   (plist-get processing-info :transparent-image-converter))
+  (plist-get processing-info :image-converter)))
+ (log-buf (get-buffer-create "*Org Preview LaTeX Output*"))
 	 (resize-mini-windows nil)) ;Fix Emacs flicker when creating image.
 (dolist (program programs)
   (org-check-external-command program error-message))
-- 
2.31.1


Re: Bug: Can't set background color of latex fragment

2021-05-19 Thread Sébastien Miquel

Hi Roshan,

Roshan Shariff writes:

I can confirm this bug with dvipng --- with the "-bg Transparent"
option, dvipng ignores the background color from the input tex file,
whereas without that option it always produces an opaque background.
There's currently no way to dynamically change command line
parameters, so I'm not sure how to solve this problem while supporting
both use cases.


I asked about this behaviour on the dvipng mailing list, and the
maintainer doesn't consider it a bug, see
https://lists.nongnu.org/archive/html/dvipng/2021-05/msg2.html.

Can you explain the use for a `Transparent` background ? Transparent
images are poorly supported in emacs, all it does is render it with
the background color of the default face -- which may not be the
expected result.

Regards,

--
Sébastien Miquel



On Wed, Apr 7, 2021 at 1:38 PM Sébastien Miquel
  wrote:

To reproduce with `emacs -Q` :
   - Open an org buffer
   - Call ~(setq org-format-latex-options '(:foreground default
 :background "Black" :matchers ("$")))~
   - Call =C-c C-x C-l= (org-latex-preview) on a latex fragment such as
 $abc$

This bug was introduced by the commit 2f9e1569f which adds the option
`-bg Transparent` to the arguments of `dvipng`. According to its
manual, this option should be ignored if a background is already set,
but it doesn't seem to be. Perhaps org should set it differently.

--
Sébastien Miquel





Re: [PATCH] Fontification for inline src blocks

2021-05-18 Thread Sébastien Miquel

Timothy writes:


In src blocks, you have the org-block-begin-line face applied. This (in
any sensible theme) has the same background as org-block.


I might be confused by my own config, but that doesn't seem to be the
case. Unless customized, the =org-block-begin-line= inherits from
org-meta-line, and the org-block documentation does specify that it
applies *inside* blocks.

I personaly dislike any inline change of background color. It makes
some sense for the python code, since it isn't org anymore (indeed,
the fontification is done in another buffer), but the src_lang, and
the result part are just org syntax.

Here's an example of a reasonable -- I hope -- use of those faces.





The ~org-src-font-lock-fontify-block~ function could be modified to
take an optional =inline= argument. When =t=, it should not set the
=multiline= font property. Although this is very minor, it would allow
one to easily advice this function to behave differently in inline src
blocks. For example, to not use the =org-block= face in this case.

I don't see where the multiline property is currently set, would you mind
pointing it out to me?


Right at the end of ~org-src-font-lock-fontify-block~. The property
is =font-lock-multiline=.


I'm going to be using the original symbols in my configuration anyway
because I think they're nicer, but clearly this is contentious. I'd want
to hear from more people on this.

If results prettification were disabled by default, There would be much
less contention.


Since ~prettify-symbols~ seems to be raising some usability concerns,
perhaps ~org-inline-src-prettify-results~ should default to ~nil~.
It'd be unlike org to hide things from the user in the default
configuration.

This seems somewhat sensible to me, but I must say that {{{results()}}}
is /ugly/ and I suspect that many users would like the effect, but a
minority will be aware of this option. Perhaps this is worth doing
anyway.

I agree. But org-mode is ugly by default, so that is consistent.


So are you suggesting I do or don't create new faces for this?

You should create new faces yes. I do not know whether one or two faces is
best.

Regards,

--
Sébastien Miquel



Re: begin_src Indentation in org 9.4.4, 9.4.5

2021-05-18 Thread Sébastien Miquel

Hi Bastien,

Bastien writes:

Before I revert the commit and try your suggestion, can you share a
patch that add both changes (the revert and your fix) manually so I
can test it?  If this fixes the original issue while preserving
electric indentation, I'm okay with it.

Here's such a patch.


Also, do you want to become the maintainer for org-src.el?  We need
more people taking charge of specific areas in Org's code.

I do intend to keep monitoring this list and help around for the
foreseeable future, and I would certainly agree to whatever sort of
maintainer position eventually, but I hold no particular interest (or
deep understanding) in this specific file.

Regards,

--
Sébastien Miquel

>From 1be7fa790e68d1fc2d198eee81c0d3bb72156d08 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?S=C3=A9bastien=20Miquel?= 
Date: Tue, 18 May 2021 14:39:33 +0200
Subject: [PATCH] org.el (org-src--contents-for-write-back): Indent blank lines

* lisp/org.el (org-src--contents-for-write-back): Indent blank lines.
* lisp/org-src.el (org-return): Revert part of commit bfda3cc7df.
---
 lisp/org-src.el | 9 -
 lisp/org.el | 6 +-
 2 files changed, 5 insertions(+), 10 deletions(-)

diff --git a/lisp/org-src.el b/lisp/org-src.el
index 5604e6568..79f002e56 100644
--- a/lisp/org-src.el
+++ b/lisp/org-src.el
@@ -453,15 +453,14 @@ Assume point is in the corresponding edit buffer."
   (insert (org-no-properties contents))
   (goto-char (point-min))
   (when (functionp write-back) (save-excursion (funcall write-back)))
-  ;; Add INDENTATION-OFFSET to every non-empty line in buffer,
+  ;; Add INDENTATION-OFFSET to every line in buffer,
   ;; unless indentation is meant to be preserved.
   (when (> indentation-offset 0)
 	(while (not (eobp))
 	  (skip-chars-forward " \t")
-	  (unless (eolp)		;ignore blank lines
-	(let ((i (current-column)))
-	  (delete-region (line-beginning-position) (point))
-	  (indent-to (+ i indentation-offset
+	  (let ((i (current-column)))
+	(delete-region (line-beginning-position) (point))
+	(indent-to (+ i indentation-offset)))
 	  (forward-line))
 
 (defun org-src--edit-element
diff --git a/lisp/org.el b/lisp/org.el
index ae09f3e99..0add9bc2e 100644
--- a/lisp/org.el
+++ b/lisp/org.el
@@ -18018,10 +18018,6 @@ object (e.g., within a comment).  In these case, you need to use
 	 (delete-and-extract-region (point) (line-end-position
 	(org--newline indent arg interactive)
 	(save-excursion (insert trailing-data
- ;; FIXME: In a source block, don't try to indent as it may result
- ;; in weird results due to `electric-indent-mode' being `t'.
- ((eq element-type 'src-block)
-  (org--newline nil nil nil))
  (t
   ;; Do not auto-fill when point is in an Org property drawer.
   (let ((auto-fill-function (and (not (org-at-property-p))
@@ -19167,7 +19163,7 @@ Also align node properties according to `org-property-format'."
 		   (line-beginning-position 2
 	 nil)
 	((and (eq type 'src-block)
-  org-src-tab-acts-natively
+		  org-src-tab-acts-natively
 		  (> (line-beginning-position)
 		 (org-element-property :post-affiliated element))
 		  (< (line-beginning-position)
-- 
2.31.1



Re: [PATCH] Fontification for inline src blocks

2021-05-18 Thread Sébastien Miquel

Hi Timothy,

Thanks for your work. I hope this can be merged.

Here are a few comments.

Doesn't this line in ~org-toggle-inline-results-display~ throw the
configured delimiters away when called twice ?
: (setq org-inline-src-prettify-results (not 
org-inline-src-prettify-results))


I think the =org-block= face should only be applied to the actual
code, note the =src_lang= part, nor the result. For normal src blocks,
it is only used inside the block.

The ~org-src-font-lock-fontify-block~ function could be modified to
take an optional =inline= argument. When =t=, it should not set the
=multiline= font property. Although this is very minor, it would allow
one to easily advice this function to behave differently in inline src
blocks. For example, to not use the =org-block= face in this case.

I think the default parenthesis pair around results are bad. I much
preferred your original brackets. Yes, as Tom said, they look alien,
but alien is appropriate for use of ~prettify-symbols~.

Since ~prettify-symbols~ seems to be raising some usability concerns,
perhaps ~org-inline-src-prettify-results~ should default to ~nil~.
It'd be unlike org to hide things from the user in the default
configuration.

As Tom points out, the two faces used (for the =src_= and bracket and
the language part) should be customizable. The default value you chose
are fine IMO. Perhaps the language one could also be used to highlight
the language of normal src blocks, though It might be easier to use a
single face.

Timothy writes:

P.S. Nitpick: You do not need to run fontification in while loops. Just
fontifying next match before limit should be enough. Font-lock will call
the function again if needed.

I'm guessing for this to work I'd need to return the final char
fortified? Or is the moving of point enough?

Maybe related - I've noticed this doesn't seem to work with multiple
src_ blocks per line, might you have any insight here?


You need only return =t= if some fontification has been done (and set
point after the fontified part). If your function returns =t=, it will
be called again.

A case can be made for keeping the loop though. It works fine and is
clearer since the aforementioned fontlock behaviour is poorly
documented. Really, the only downside is the loss of consistency, since
the function ~org-fontify-meta-lines-and-blocks-1~ doesn't loop.

Regards,

--
Sébastien Miquel



Re: begin_src Indentation in org 9.4.4, 9.4.5

2021-05-17 Thread Sébastien Miquel

Hi Bastien,

The commit `bfda3cc7df31fa79222efb4c190618c3c85a3d04` breaks automatic
(electric) indentation  in src blocks for all configurations.

Here's what happens with the original issue.

When you press `RET` after the first ~hello hi~, the result is that
the first line is indented by two spaces, and the second (where the
point is) isn't.
 - since ~electric-indent-mode~ is on, ~org-return~ is called with
   `indent` set to `t`.
 - Since ~org-src-tab-acts-natively~ is `t`, indentation is done by
   calling `tab` in a src edit buffer, which by itself, does nothing.
 - The observed indentation comes from ~org-src--contents-for-write-back~.
   Since ~org-src--content-indentation~ is 2 and 
~org-src--preserve-indentation~

   is ~nil~, this functions further indents each *non blank* line by 2.

At this point, the first line is indented, cursor is at bol. Note
that you cannot indent your current empty line with `tab`. You can
either indent it manually, or call ~org-edit-special~.

When you write your second line, then press `RET`,
~org-src--contents-for-write-back~ will add an additional two spaces,
producing the reported result.

I think a reasonable fix is to have ~org-src--contents-for-write-back~
also indent blank lines. To do this, you need only remove following
line from its definition.

: (unless (eolp)        ;ignore blank lines

With this done and `bfda3cc7df31fa79222efb4c190618c3c85a3d04`
reverted, the original issue is fixed, and the behaviour is better:
when you press `RET` to enter a newline in a src block, it is
automatically indented.

The downside is that, unless ~org-src--preserve-indentation~ is `t`,
when editing a src block, every empty line will be indented with
spaces (according to ~org-edit-src-content-indentation~ + the
indentation of the #+begin_src line). I think this is reasonable, but
perhaps some might disagree.

Regards,

--
Sébastien Miquel




Re: [Patch] Bug: org-indent-region doesn't restore cursor position when org-src-tab-acts-natively is t [9.1.9 (release_9.1.9-65-g5e4542 @ /usr/share/emacs/26.3/lisp/org/)]

2021-05-16 Thread Sébastien Miquel

Hi Bastien,

Bastien writes:

Sorry it took so long to apply this patch, this is now done in maint.

No problem, thank you for getting back on this.

I think something went wrong though. I've attached as a new patch a
part of the previous one that wasn't applied. Without it, some
write-back buffers are never killed.

Regards,

--
Sébastien Miquel

>From f293a9d5808c413ce785646ebf5f480df3a00a2f Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?S=C3=A9bastien=20Miquel?= 
Date: Sun, 16 May 2021 19:13:53 +0200
Subject: [PATCH] org-src.el (org-edit-src-exit): Fix write-back-buf not
 getting killed

* lisp/org.el (org-edit-src-exit): Fix write-back-buf not getting
killed
---
 lisp/org-src.el | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/lisp/org-src.el b/lisp/org-src.el
index df3c76e13..5604e6568 100644
--- a/lisp/org-src.el
+++ b/lisp/org-src.el
@@ -1255,8 +1255,8 @@ Throw an error if there is no such buffer."
 	   (narrow-to-region beg end)
 	   (replace-buffer-contents write-back-buf)
 	   (goto-char (point-max
-	 (when (and expecting-bol (not (bolp))) (insert "\n")))
-   (when write-back-buf (kill-buffer write-back-buf
+	 (when (and expecting-bol (not (bolp))) (insert "\n")
+(when write-back-buf (kill-buffer write-back-buf))
 ;; If we are to return to source buffer, put point at an
 ;; appropriate location.  In particular, if block is hidden, move
 ;; to the beginning of the block opening line.
-- 
2.31.1



Re: [PATCH] tangling seems to have broken today

2021-05-06 Thread Sébastien Miquel

Hi Bastien,

Bastien writes:

Also, nitpick: please don't add a full-stop at the end of the first
line of the commit message (or the subject of your patch).

Unless I misunderstand what you mean, the patch seems to follow this
style.


I think this should be applied to the maint branch, right?  In this
case, the patch does not apply.  Can you check and repropose a patch
against maint?  Or just let me know if I'm mistaken.

It fixes an issue introduced by a commit in master, so it should be
applied to master.


So we're all set here?

This should be it!


Regards,

--
Sébastien Miquel




Re: [PATCH] tangling seems to have broken today

2021-05-05 Thread Sébastien Miquel

Hi Bastien,

The issue is that currently tangling a single block by calling
`org-babel-tangle` with a prefix argument fails. This is unrelated to
the earlier thread today, but was introduced by my original commit
a2cb9b853d.

Unfortunately, fixing it requires some refactorisation to avoid code
duplication. To keep the patch as simple as possible, I have
introduced a new helper function, I hope this is okay.

I have tested the attached patch with the uses cases shown so far, as
well as my own and the org test suite.

I apologise again for the troubles.

--
Sébastien Miquel

>From 69be5864d9c936396317d81b6c39ddb166ac45d6 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?S=C3=A9bastien=20Miquel?= 
Date: Wed, 5 May 2021 17:45:09 +0200
Subject: [PATCH] ob-tangle.el: Fix single block tangle

lisp/ob-tangle.el (org-babel-tangle-single-block): Fix the result when
`only-this-block' is `t' to match what is expected by
`org-babel-tangle'.
(org-babel-effective-tangled-filename): Extract the
computation of filename of tangled src block.
(org-babel-tangle-collect-blocks): Use `org-babel-effective-tangled-filename'.
---
 lisp/ob-tangle.el | 34 ++
 1 file changed, 22 insertions(+), 12 deletions(-)

diff --git a/lisp/ob-tangle.el b/lisp/ob-tangle.el
index 8af03b11a..562776ae8 100644
--- a/lisp/ob-tangle.el
+++ b/lisp/ob-tangle.el
@@ -350,6 +350,22 @@ that the appropriate major-mode is set.  SPEC has the form:
 	   (org-fill-template
 		org-babel-tangle-comment-format-end link-data)

+(defun org-babel-effective-tangled-filename (buffer-fn src-lang src-tfile)
+  "Return effective tangled filename of a source-code block.
+BUFFER-FN is the name of the buffer, SRC-LANG the language of the
+block and SRC-TFILE is the value of the :tangle header argument,
+as computed by `org-babel-tangle-single-block'."
+  (let ((base-name (cond
+((string= "yes" src-tfile)
+ ;; Use the buffer name
+ (file-name-sans-extension buffer-fn))
+((> (length src-tfile) 0) src-tfile)))
+(ext (or (cdr (assoc src-lang org-babel-tangle-lang-exts)) src-lang)))
+(when base-name
+  ;; decide if we want to add ext to base-name
+  (if (and ext (string= "yes" src-tfile))
+  (concat base-name "." ext) base-name
+
 (defun org-babel-tangle-collect-blocks ( lang-re tangle-file)
   "Collect source blocks in the current Org file.
 Return an association list of language and source-code block
@@ -378,17 +394,8 @@ be used to limit the collected code blocks by target file."
 	;; file name.
 	(let* ((block (org-babel-tangle-single-block counter))
(src-tfile (cdr (assq :tangle (nth 4 block
-		   (base-name (cond
-			   ((string= "yes" src-tfile)
-;; buffer name
-(file-name-sans-extension
- (nth 1 block)))
-			   ((> (length src-tfile) 0) src-tfile)))
-		   (ext (or (cdr (assoc src-lang org-babel-tangle-lang-exts)) src-lang))
-		   (file-name (when base-name
-;; decide if we want to add ext to base-name
-(if (and ext (string= "yes" src-tfile))
-(concat base-name "." ext) base-name)))
+		   (file-name (org-babel-effective-tangled-filename
+   (nth 1 block) src-lang src-tfile))
 		   (by-fn (assoc file-name blocks)))
 	  (if by-fn (setcdr by-fn (cons (cons src-lang block) (cdr by-fn)))
 		(push (cons file-name (list (cons src-lang block))) blocks)))
@@ -482,7 +489,10 @@ non-nil, return the full association list to be used by
 		  (org-trim (org-remove-indentation body)))
 		comment)))
 (if only-this-block
-	(list (cons src-lang (list result)))
+(let* ((src-tfile (cdr (assq :tangle (nth 4 result
+   (file-name (org-babel-effective-tangled-filename
+   (nth 1 result) src-lang src-tfile)))
+  (list (cons file-name (list (cons src-lang result)
   result)))

 (defun org-babel-tangle-comment-links ( info)
-- 
2.31.1


Re: tangling seems to have broken today

2021-05-05 Thread Sébastien Miquel

Eric S Fraga writes:

I was using org-babel-tangle.  Although, in this case, with C-u before
to tangle just the particular block.

Indeed, I have broken this. I'll provide a patch, shortly.

--
Sébastien Miquel




Re: tangling seems to have broken today

2021-05-05 Thread Sébastien Miquel

Hi Eric, Timothy,

Eric S Fraga writes:

I can no longer tangle with a given file name.  The error I get
indicates that the tangling procedure is looking at the first line of
the actual code block.

What command are you using ?

Running `org-babel-tangle` works for me with latest master using your
example.


--
Sébastien Miquel




Re: Bug: [PATCH] org-babel-tangle: persmission denied when tangling [9.4.5 (9.4.5-gbc2659 @ /home/n/.emacs.d/straight/build/org/)]

2021-05-05 Thread Sébastien Miquel

No Wayman writes:

Another related bug to the changes:

I have the :tangle header-arg set to evaluate some elisp to return the 
file name:


org-babel no longer interprets this elisp. It is being used literally 
as the file name:

e.g.

Wrote /home/n/.emacs.d/(concat (file-name-sans-extension 
(buffer-file-name)) ".el") 

Here's another patch, to be applied on top of the previous one, that
fixes this.

The specific case you mention can also be achieved by setting the
:tangle argument to `yes`: in this case, the tangled file name is
computed using the buffer file name and changing the extension
according to the src block language.


Thank you again for the report, and sorry for breaking everything.

--
Sébastien Miquel

>From b7c5103fdd05c3d30805ebcc5084ef82c44cd8ff Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?S=C3=A9bastien=20Miquel?= 
Date: Wed, 5 May 2021 08:31:43 +0200
Subject: [PATCH] ob-tangle.el: (org-babel-tangle-collect-blocks): Use correct
 tangle name

* lisp/ob-tangle.el: (org-babel-tangle-collect-blocks): Use correct
tangle name.

The :tangle header argument might be some elisp, to be evaluated.
---
Range-diff:
-:  - > 1:  b7c5103fd ob-tangle.el: (org-babel-tangle-collect-blocks): Use correct tangle name

 lisp/ob-tangle.el | 1 +
 1 file changed, 1 insertion(+)

diff --git a/lisp/ob-tangle.el b/lisp/ob-tangle.el
index 96a4ef049..8af03b11a 100644
--- a/lisp/ob-tangle.el
+++ b/lisp/ob-tangle.el
@@ -377,6 +377,7 @@ be used to limit the collected code blocks by target file."
 	;; Add the spec for this block to blocks under its tangled
 	;; file name.
 	(let* ((block (org-babel-tangle-single-block counter))
+   (src-tfile (cdr (assq :tangle (nth 4 block
 		   (base-name (cond
 			   ((string= "yes" src-tfile)
 ;; buffer name
-- 
2.31.1


Re: Bug: org-babel-tangle: persmission denied when tangling [9.4.5 (9.4.5-gbc2659 @ /home/n/.emacs.d/straight/build/org/)]

2021-05-04 Thread Sébastien Miquel

Hi Bastien,

Bastien writes:

I tried to apply this (transitory?) patch against maint and it did not
apply.  It applies okay on master.  For bug fixes, please make patches
againt the maint branch.

This fixes a bug introduced by a commit in master. I've attached the same
patch here, properly formated. I think it should be applied to master.

It reverts a part of a2cb9b853: permissions are no longer set before writing
to the tangled file.

I've CC'd Tom, which made the original suggestion. I guess we could set the
write and execute permissions before writing, and set the read permissions
afterwards.

--
Sébastien Miquel

>From e56a05f4f5a3cce9cfdeb71854475e29aac1a6e8 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?S=C3=A9bastien=20Miquel?= 
Date: Tue, 4 May 2021 22:59:36 +0200
Subject: [PATCH] ob-tangle.el (org-babel-tangle): Fix readonly tangle

* lisp/ob-tangle.el (org-babel-tangle): Fix readonly tangle.
---
 lisp/ob-tangle.el | 8 
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/lisp/ob-tangle.el b/lisp/ob-tangle.el
index 36144d6ae..96a4ef049 100644
--- a/lisp/ob-tangle.el
+++ b/lisp/ob-tangle.el
@@ -268,11 +268,11 @@ matching a regular expression."
 		lspecs)
 		   (when make-dir
 		 (make-directory fnd 'parents))
-   ;; erase previous file and set permissions on empty
-   ;; file before writing
-   (write-region "" nil file-name nil 0)
-		   (mapc (lambda (mode) (set-file-modes file-name mode)) modes)
+   ;; erase previous file
+   (when (file-exists-p file-name)
+ (delete-file file-name))
 		   (write-region nil nil file-name)
+		   (mapc (lambda (mode) (set-file-modes file-name mode)) modes)
(push file-name path-collector))
 	 (if (equal arg '(4))
 	 (org-babel-tangle-single-block 1 t)
-- 
2.31.1



Re: Bug: org-babel-tangle: persmission denied when tangling [9.4.5 (9.4.5-gbc2659 @ /home/n/.emacs.d/straight/build/org/)]

2021-05-04 Thread Sébastien Miquel

No Wayman writes:

Subsequent tangles did not fail for me.

Ah yes, I understand, it is possible to delete a file without write
permission.

I'll see if I can fix this bug and keep the security improvements. In
the meantime, you can apply the attached patch that should fix
your issue.

Thank you for the report.

--
Sébastien Miquel

diff --git a/lisp/ob-tangle.el b/lisp/ob-tangle.el
index 36144d6ae..c041ff4b3 100644
--- a/lisp/ob-tangle.el
+++ b/lisp/ob-tangle.el
@@ -270,9 +270,10 @@ matching a regular expression."
 		 (make-directory fnd 'parents))
;; erase previous file and set permissions on empty
;; file before writing
-   (write-region "" nil file-name nil 0)
-		   (mapc (lambda (mode) (set-file-modes file-name mode)) modes)
+   (when (file-exists-p file-name)
+ (delete-file file-name))
 		   (write-region nil nil file-name)
+		   (mapc (lambda (mode) (set-file-modes file-name mode)) modes)
(push file-name path-collector))
 	 (if (equal arg '(4))
 	 (org-babel-tangle-single-block 1 t)


Re: Bug: org-babel-tangle: persmission denied when tangling [9.4.5 (9.4.5-gbc2659 @ /home/n/.emacs.d/straight/build/org/)]

2021-05-04 Thread Sébastien Miquel

Hi,

No Wayman writes:
I'm tangling my early-init/init.el with the :tangle-mode header arg 
set to (identity (#o444)).

This should be `(identity #o444)` I believe ?

The idea behind this was to prevent myself from accidentally editing 
the tangled source files
instead of the Org files. 
Unfortunately, since a3cb9b853 there seems to be a behavior change for 
org-babel-tangle which prevents this.

File permissions are now set before writing to the file, for security
reasons. In this case, you remove write permission so emacs fails to
write to the file. Perhaps we should try to support this use case.

However, even with the previous version, it seems that subsequent
tangles should have failed (emacs should fail to delete the previous
tangled file). Can you confirm this and explain how you dealt with it ?

As a workaround, you could use a file-local
`org-babel-post-tangle-hook` to set file permission. Although
subsequent tangles will still fail.

--
Sébastien Miquel




Re: [Feature request] String escaped noweb expansion

2021-05-02 Thread Sébastien Miquel



Timothy writes:

Just quickly, this works:

#+begin_src emacs-lisp :noweb yes
(setq my-latex-code "\
<>
")
#+end_src

I don't understand the purpose of your backslash, is it a typo ? With
or without it, this doesn't tangle to working lisp with my example.

The point of my proposal is to allow one to write unescaped latex in
the latex src block. (Mostly to avoid doubling every backslash). It is
quite the shame to have to write non latex code in a latex src block.

Regards,

--
Sébastien Miquel




[Feature request] String escaped noweb expansion

2021-05-02 Thread Sébastien Miquel

Hi,

Would there be some interest in extending the noweb syntax to allow
for string escaped expansion ?

I've modified my setup so that if the noweb delimiter `<<` is preceded
by a quote `"` then the expansion is string escaped (and the prefix
isn't duplicated for each line). This allows the following.

#+BEGIN_SRC latex :noweb-ref latex-ref
\some \multiline
\unescaped \latex \code
#+END_SRC

#+BEGIN_SRC emacs-lisp :noweb yes
(setq my-latex-code "<>\n")
#+END_SRC

I don't think there's any way to achieve such functionality currently.

--
Sébastien Miquel




Re: <> and ?font-lock? fly-check, ...

2021-05-02 Thread Sébastien Miquel

Hi Greg,

Greg Minshall writes:

is there an obvious thing to do to either get whatever syntax checker is
running to ignore the <> reference, or some such?

From the org side of things, you could customize the variables
`org-babel-noweb-wrap-start` and `org-babel-noweb-wrap-end` to
something that doesn't interfere with bash syntax.

Regards,

--
Sébastien Miquel




Re: [PATCH] ob-tangle.el: Speed up tangling

2021-05-01 Thread Sébastien Miquel

Hi Bastien,

Bastien writes:

The compiler is complaining with

   In toplevel form:
   ob-tangle.el:196:1: Warning: Variable ‘modes’ left uninitialized

Also, it breaks these two tests for me:

2 unexpected results:
FAILED  ob-tangle/block-order
FAILED  ob-tangle/continued-code-blocks-w-noweb-ref


Indeed, I hadn't thought to run the tests, sorry. I've fixed my code
and modified the `block-order` test in order for it to pass.

The patch does modify the order of the tangled blocks. When several
blocks with different languages are tangled to the same file, they
used to be grouped according to language, and are now tangled in the
order in which they appear. I assumed this was an oversight in the
previous code, but since this test exists, maybe it was intended ?

Nicolas Goaziou wrote this test, perhaps he could comment on this.

Regards,

--
Sébastien Miquel

>From 2aa09e8d2f4e8703190e9035d711508c11b3a8eb Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?S=C3=A9bastien=20Miquel?= 
Date: Sat, 1 May 2021 21:18:44 +0200
Subject: [PATCH] ob-tangle.el: Improve tangling

* lisp/ob-tangle.el (org-babel-tangle-collect-blocks): Group
collected blocks by tangled file name.
(org-babel-tangle): Avoid quadratic behavior in number of blocks and
set modes before writing to file.
* testing/lisp/test-ob-tangle.el (ob-tangle/block-order): Update test.
---
 lisp/ob-tangle.el  | 151 -
 testing/lisp/test-ob-tangle.el |   2 +-
 2 files changed, 74 insertions(+), 79 deletions(-)

diff --git a/lisp/ob-tangle.el b/lisp/ob-tangle.el
index 4c0c3132d..36144d6ae 100644
--- a/lisp/ob-tangle.el
+++ b/lisp/ob-tangle.el
@@ -225,67 +225,55 @@ matching a regular expression."
 	   (or (cdr (assq :tangle (nth 2 (org-babel-get-src-block-info 'light
 		   (user-error "Point is not in a source code block"
 	path-collector)
-	(mapc ;; map over all languages
-	 (lambda (by-lang)
-	   (let* ((lang (car by-lang))
-		  (specs (cdr by-lang))
-		  (ext (or (cdr (assoc lang org-babel-tangle-lang-exts)) lang))
-		  (lang-f (org-src-get-lang-mode lang))
-		  she-banged)
-	 (mapc
-	  (lambda (spec)
-		(let ((get-spec (lambda (name) (cdr (assoc name (nth 4 spec))
-		  (let* ((tangle (funcall get-spec :tangle))
-			 (she-bang (let ((sheb (funcall get-spec :shebang)))
- (when (> (length sheb) 0) sheb)))
-			 (tangle-mode (funcall get-spec :tangle-mode))
-			 (base-name (cond
- ((string= "yes" tangle)
-  (file-name-sans-extension
-   (nth 1 spec)))
- ((string= "no" tangle) nil)
- ((> (length tangle) 0) tangle)))
-			 (file-name (when base-name
-  ;; decide if we want to add ext to base-name
-  (if (and ext (string= "yes" tangle))
-	  (concat base-name "." ext) base-name
-		(when file-name
-		  ;; Possibly create the parent directories for file.
-		  (let ((m (funcall get-spec :mkdirp))
-			(fnd (file-name-directory file-name)))
-			(and m fnd (not (string= m "no"))
-			 (make-directory fnd 'parents)))
-		  ;; delete any old versions of file
-		  (and (file-exists-p file-name)
-			   (not (member file-name (mapcar #'car path-collector)))
-			   (delete-file file-name))
-		  ;; drop source-block to file
-		  (with-temp-buffer
-			(when (fboundp lang-f) (ignore-errors (funcall lang-f)))
-			(when (and she-bang (not (member file-name she-banged)))
+	(mapc ;; map over file-names
+	 (lambda (by-fn)
+	   (let ((file-name (car by-fn)))
+	 (when file-name
+   (let ((lspecs (cdr by-fn))
+		 (fnd (file-name-directory file-name))
+		 modes make-dir she-banged lang)
+	 ;; drop source-blocks to file
+	 ;; We avoid append-to-file as it does not work with tramp.
+	 (with-temp-buffer
+		   (mapc
+		(lambda (lspec)
+		  (let* ((block-lang (car lspec))
+			 (spec (cdr lspec))
+			 (get-spec (lambda (name) (cdr (assq name (nth 4 spec)
+			 (she-bang (let ((sheb (funcall get-spec :shebang)))
+ (when (> (length sheb) 0) sheb)))
+			 (tangle-mode (funcall get-spec :tangle-mode)))
+		(unless (string-equal block-lang lang)
+			  (setq lang block-lang)
+			  (let ((lang-f (org-src-get-lang-mode lang)))
+			(when (fboundp lang-f) (ignore-errors (funcall lang-f)
+		;; if file contains she-bangs, then make it executable
+		(when she-bang
+			  (unless tangle-mode (setq tangle-mode #o755)))
+		(when tangle-mode
+			  (add-to-list 'modes tangle-mode))
+		;; Possibly create the parent directories for file.
+		(let ((m (funcall get-spec :mkdirp)))
+			  (and m fnd (not (string= m "no"))
+			   (setq make-dir t)))
+		;; Handle :padlines unless first line in file
+		(unless (or (string= "no" (funcall get-spec :padline))
+			

Re: [PATCH] ob-tangle.el: Speed up tangling

2021-04-21 Thread Sébastien Miquel

Hi Tom,

Thank you again for your comments.

Tom Gillespie writes:

I think that the location of condition-case is ok, but I wonder what
would happen if something were to fail before entering that? I think
that only a subset of the files would be tangled, but they would all
have their correct modes, so I think that that is ok.

On second thought, I'm uneasy about my approach. If tangling fails,
the user might miss the error message since it is quickly replaced by
the tangling info. Ideally we should backup all the tangled files and
restore them all if a single one fails to ensure we're back to a
consistent state.

I'm unsure what would be best practices here. In case of a remote
tangled files, I don't know if temporary files should be remote or
not, and what guarantees do emacs primitives such as ~rename-file~
offer.

Although a robust tangling system that deals with errors and
guarantees that the state ends up consistent would be nice to have,
I'll take the failure considerations off this patch to keep it simple.
It'll make a better starting point for future work at least.

As is currently the case, if tangling fails, an error with be thrown,
the user will certainly notice and should assume that everything is
broken until another tangling succeeds.

I've kept the modes improvements.

Regards,

--
Sébastien Miquel

>From 6b123c956ac7abe0210cf7b1145ebe0a68f04713 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?S=C3=A9bastien=20Miquel?= 
Date: Sat, 17 Apr 2021 21:48:30 +0200
Subject: [PATCH] ob-tangle.el: Improve tangling

,* lisp/ob-tangle.el (org-babel-tangle-collect-blocks): Group
collected blocks by tangled file name.
(org-babel-tangle): Avoid quadratic behavior in number of blocks and
set modes before writing to file.
---
 lisp/ob-tangle.el | 151 ++
 1 file changed, 73 insertions(+), 78 deletions(-)

diff --git a/lisp/ob-tangle.el b/lisp/ob-tangle.el
index 4c0c3132d..8ca6b66fe 100644
--- a/lisp/ob-tangle.el
+++ b/lisp/ob-tangle.el
@@ -225,67 +225,55 @@ matching a regular expression."
 	   (or (cdr (assq :tangle (nth 2 (org-babel-get-src-block-info 'light
 		   (user-error "Point is not in a source code block"
 	path-collector)
-	(mapc ;; map over all languages
-	 (lambda (by-lang)
-	   (let* ((lang (car by-lang))
-		  (specs (cdr by-lang))
-		  (ext (or (cdr (assoc lang org-babel-tangle-lang-exts)) lang))
-		  (lang-f (org-src-get-lang-mode lang))
-		  she-banged)
-	 (mapc
-	  (lambda (spec)
-		(let ((get-spec (lambda (name) (cdr (assoc name (nth 4 spec))
-		  (let* ((tangle (funcall get-spec :tangle))
-			 (she-bang (let ((sheb (funcall get-spec :shebang)))
- (when (> (length sheb) 0) sheb)))
-			 (tangle-mode (funcall get-spec :tangle-mode))
-			 (base-name (cond
- ((string= "yes" tangle)
-  (file-name-sans-extension
-   (nth 1 spec)))
- ((string= "no" tangle) nil)
- ((> (length tangle) 0) tangle)))
-			 (file-name (when base-name
-  ;; decide if we want to add ext to base-name
-  (if (and ext (string= "yes" tangle))
-	  (concat base-name "." ext) base-name
-		(when file-name
-		  ;; Possibly create the parent directories for file.
-		  (let ((m (funcall get-spec :mkdirp))
-			(fnd (file-name-directory file-name)))
-			(and m fnd (not (string= m "no"))
-			 (make-directory fnd 'parents)))
-		  ;; delete any old versions of file
-		  (and (file-exists-p file-name)
-			   (not (member file-name (mapcar #'car path-collector)))
-			   (delete-file file-name))
-		  ;; drop source-block to file
-		  (with-temp-buffer
-			(when (fboundp lang-f) (ignore-errors (funcall lang-f)))
-			(when (and she-bang (not (member file-name she-banged)))
+	(mapc ;; map over file-names
+	 (lambda (by-fn)
+	   (let ((file-name (car by-fn)))
+	 (when file-name
+   (let ((lspecs (cdr by-fn))
+		 (fnd (file-name-directory file-name))
+		 modes make-dir she-banged lang)
+	 ;; drop source-blocks to file
+	 ;; We avoid append-to-file as it does not work with tramp.
+	 (with-temp-buffer
+		   (mapc
+		(lambda (lspec)
+		  (let* ((block-lang (car lspec))
+			 (spec (cdr lspec))
+			 (get-spec (lambda (name) (cdr (assq name (nth 4 spec)
+			 (she-bang (let ((sheb (funcall get-spec :shebang)))
+ (when (> (length sheb) 0) sheb)))
+			 (tangle-mode (funcall get-spec :tangle-mode)))
+		(unless (string-equal block-lang lang)
+			  (setq lang block-lang)
+			  (let ((lang-f (org-src-get-lang-mode lang)))
+			(when (fboundp lang-f) (ignore-errors (funcall lang-f)
+		;; if file contains she-bangs, then make it executable
+		(when she-bang
+			  (unless tangle-mode (setq tangle-mode #o755)))
+		(when tangle-mode
+			  (add-to-list mode

Re: [PATCH] ob-tangle.el: Speed up tangling

2021-04-19 Thread Sébastien Miquel

Hi Tom,

Thank you for the comments.

Tom Gillespie writes:
> All of the issues that I'm aware of are related to what
> happens if tangling fails part way through the process.

That's not something I had considered. I wrote a new version of the
patch (attached) which addresses the insecure behaviour and the
possibility of failure. Please tell me what you think.

I've also
 + silenced the ~write-region~ messages, since I'm now writing to
   temporary files.
 + added the list of tangled files to the message to the user at the
   end of the tangling process.
 + replaced the use of ~when-let~.

Regards,

--
Sébastien Miquel
>From 82e4c1beade71194c90d377cdff7ef23532f4aa2 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?S=C3=A9bastien=20Miquel?= 
Date: Sat, 17 Apr 2021 21:48:30 +0200
Subject: [PATCH] ob-tangle.el: Improve tangling

,* lisp/ob-tangle.el (org-babel-tangle-collect-blocks): Group
collected blocks by tangled file name.
(org-babel-tangle): Avoid quadratic behavior in number of blocks.
Preserve original file in case of failure. Display the list of tangled
files at the end.
---
 lisp/ob-tangle.el | 167 --
 1 file changed, 87 insertions(+), 80 deletions(-)

diff --git a/lisp/ob-tangle.el b/lisp/ob-tangle.el
index 4c0c3132d..efafef5b8 100644
--- a/lisp/ob-tangle.el
+++ b/lisp/ob-tangle.el
@@ -225,87 +225,83 @@ matching a regular expression."
 	   (or (cdr (assq :tangle (nth 2 (org-babel-get-src-block-info 'light
 		   (user-error "Point is not in a source code block"
 	path-collector)
-	(mapc ;; map over all languages
-	 (lambda (by-lang)
-	   (let* ((lang (car by-lang))
-		  (specs (cdr by-lang))
-		  (ext (or (cdr (assoc lang org-babel-tangle-lang-exts)) lang))
-		  (lang-f (org-src-get-lang-mode lang))
-		  she-banged)
-	 (mapc
-	  (lambda (spec)
-		(let ((get-spec (lambda (name) (cdr (assoc name (nth 4 spec))
-		  (let* ((tangle (funcall get-spec :tangle))
-			 (she-bang (let ((sheb (funcall get-spec :shebang)))
- (when (> (length sheb) 0) sheb)))
-			 (tangle-mode (funcall get-spec :tangle-mode))
-			 (base-name (cond
- ((string= "yes" tangle)
-  (file-name-sans-extension
-   (nth 1 spec)))
- ((string= "no" tangle) nil)
- ((> (length tangle) 0) tangle)))
-			 (file-name (when base-name
-  ;; decide if we want to add ext to base-name
-  (if (and ext (string= "yes" tangle))
-	  (concat base-name "." ext) base-name
-		(when file-name
-		  ;; Possibly create the parent directories for file.
-		  (let ((m (funcall get-spec :mkdirp))
-			(fnd (file-name-directory file-name)))
-			(and m fnd (not (string= m "no"))
-			 (make-directory fnd 'parents)))
-		  ;; delete any old versions of file
-		  (and (file-exists-p file-name)
-			   (not (member file-name (mapcar #'car path-collector)))
-			   (delete-file file-name))
-		  ;; drop source-block to file
-		  (with-temp-buffer
-			(when (fboundp lang-f) (ignore-errors (funcall lang-f)))
-			(when (and she-bang (not (member file-name she-banged)))
+	(mapc ;; map over file-names
+	 (lambda (by-fn)
+	   (let ((file-name (car by-fn)))
+	 (when file-name
+   (let ((lspecs (cdr by-fn))
+		 (fnd (file-name-directory file-name))
+		 modes make-dir she-banged lang)
+	 ;; drop source-blocks to file
+	 ;; We avoid append-to-file as it does not work with tramp.
+	 (with-temp-buffer
+		   (mapc
+		(lambda (lspec)
+		  (let* ((block-lang (car lspec))
+			 (spec (cdr lspec))
+			 (get-spec (lambda (name) (cdr (assq name (nth 4 spec)
+			 (she-bang (let ((sheb (funcall get-spec :shebang)))
+ (when (> (length sheb) 0) sheb)))
+			 (tangle-mode (funcall get-spec :tangle-mode)))
+		(unless (string-equal block-lang lang)
+			  (setq lang block-lang)
+			  (let ((lang-f (org-src-get-lang-mode lang)))
+			(when (fboundp lang-f) (ignore-errors (funcall lang-f)
+		;; if file contains she-bangs, then make it executable
+		(when she-bang
+			  (unless tangle-mode (setq tangle-mode #o755)))
+		(when tangle-mode
+			  (add-to-list modes tangle-mode))
+		;; Possibly create the parent directories for file.
+		(let ((m (funcall get-spec :mkdirp)))
+			  (and m fnd (not (string= m "no"))
+			   (setq make-dir t)))
+		;; Handle :padlines unless first line in file
+		(unless (or (string= "no" (funcall get-spec :padline))
+(= (point) (point-min)))
+			  (insert "\n"))
+		(when (and she-bang (not she-banged))
 			  (insert (concat she-bang "\n"))
-			  (setq she-banged (cons file-name she-banged)))
-			(org-babel-spec-to-string spec)
-			;; We avoid append-to-file as it does not work with tramp.

[PATCH] ob-tangle.el: Speed up tangling

2021-04-18 Thread Sébastien Miquel

Hi,

The attached patch modifies the ~org-babel-tangle~ function to avoid a
quadratic behavior in the number of blocks tangled to a single file.

Tangling an org buffer with 200 blocks to 5 different files yields a
25 % speedup.


* lisp/ob-tangle.el (org-babel-tangle-collect-blocks): Group
collected blocks by tangled file name.
(org-babel-tangle): Avoid quadratic behavior in number of blocks.

--
Sébastien Miquel
>From 939fedb0fa94f044eda6966f55f460aa292e345f Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?S=C3=A9bastien=20Miquel?= 
Date: Sat, 17 Apr 2021 21:48:30 +0200
Subject: [PATCH] ob-tangle.el: Speed up tangling

,* lisp/ob-tangle.el (org-babel-tangle-collect-blocks): Group
collected blocks by tangled file name.
(org-babel-tangle): Avoid quadratic behavior in number of blocks.
---
 lisp/ob-tangle.el | 148 ++
 1 file changed, 71 insertions(+), 77 deletions(-)

diff --git a/lisp/ob-tangle.el b/lisp/ob-tangle.el
index 4c0c3132d..eef300c3d 100644
--- a/lisp/ob-tangle.el
+++ b/lisp/ob-tangle.el
@@ -225,67 +225,54 @@ matching a regular expression."
 	   (or (cdr (assq :tangle (nth 2 (org-babel-get-src-block-info 'light
 		   (user-error "Point is not in a source code block"
 	path-collector)
-	(mapc ;; map over all languages
-	 (lambda (by-lang)
-	   (let* ((lang (car by-lang))
-		  (specs (cdr by-lang))
-		  (ext (or (cdr (assoc lang org-babel-tangle-lang-exts)) lang))
-		  (lang-f (org-src-get-lang-mode lang))
-		  she-banged)
-	 (mapc
-	  (lambda (spec)
-		(let ((get-spec (lambda (name) (cdr (assoc name (nth 4 spec))
-		  (let* ((tangle (funcall get-spec :tangle))
-			 (she-bang (let ((sheb (funcall get-spec :shebang)))
- (when (> (length sheb) 0) sheb)))
-			 (tangle-mode (funcall get-spec :tangle-mode))
-			 (base-name (cond
- ((string= "yes" tangle)
-  (file-name-sans-extension
-   (nth 1 spec)))
- ((string= "no" tangle) nil)
- ((> (length tangle) 0) tangle)))
-			 (file-name (when base-name
-  ;; decide if we want to add ext to base-name
-  (if (and ext (string= "yes" tangle))
-	  (concat base-name "." ext) base-name
-		(when file-name
-		  ;; Possibly create the parent directories for file.
-		  (let ((m (funcall get-spec :mkdirp))
-			(fnd (file-name-directory file-name)))
-			(and m fnd (not (string= m "no"))
-			 (make-directory fnd 'parents)))
-		  ;; delete any old versions of file
-		  (and (file-exists-p file-name)
-			   (not (member file-name (mapcar #'car path-collector)))
-			   (delete-file file-name))
-		  ;; drop source-block to file
-		  (with-temp-buffer
-			(when (fboundp lang-f) (ignore-errors (funcall lang-f)))
-			(when (and she-bang (not (member file-name she-banged)))
-			  (insert (concat she-bang "\n"))
-			  (setq she-banged (cons file-name she-banged)))
-			(org-babel-spec-to-string spec)
-			;; We avoid append-to-file as it does not work with tramp.
-			(let ((content (buffer-string)))
-			  (with-temp-buffer
-			(when (file-exists-p file-name)
-			  (insert-file-contents file-name))
-			(goto-char (point-max))
-			;; Handle :padlines unless first line in file
-			(unless (or (string= "no" (cdr (assq :padline (nth 4 spec
-	(= (point) (point-min)))
-			  (insert "\n"))
-			(insert content)
-			(write-region nil nil file-name
-		  ;; if files contain she-bangs, then make the executable
+	(mapc ;; map over file-names
+	 (lambda (by-fn)
+	   (when-let ((file-name (car by-fn)))
+	 (let ((lspecs (cdr by-fn))
+		   (fnd (file-name-directory file-name))
+		   modes make-dir she-banged lang)
+	   ;; delete any old version of file
+	   (when (file-exists-p file-name) (delete-file file-name))
+	   ;; drop source-blocks to file
+	   ;; We avoid append-to-file as it does not work with tramp.
+	   (with-temp-buffer
+		 (mapc
+		  (lambda (lspec)
+		(let* ((block-lang (car lspec))
+			   (spec (cdr lspec))
+			   (get-spec (lambda (name) (cdr (assq name (nth 4 spec)
+			   (she-bang (let ((sheb (funcall get-spec :shebang)))
+   (when (> (length sheb) 0) sheb)))
+			   (tangle-mode (funcall get-spec :tangle-mode)))
+		  (unless (string-equal block-lang lang)
+			(setq lang block-lang)
+			(let ((lang-f (org-src-get-lang-mode lang)))
+			  (when (fboundp lang-f) (ignore-errors (funcall lang-f)
+		  ;; if files contain she-bangs, then make them executable
 		  (when she-bang
 			(unless tangle-mode (setq tangle-mode #o755)))
-		  ;; update counter
-		  (setq block-counter (+ 1 block-counter))
-		  (unless (assoc file-name path-collector)
-			(push (cons file-name tangle-mode) path-collector))
-	  specs)))
+		  (when tangle-mode
+			(push tangle-mode modes

Bug: Can't set background color of latex fragment

2021-04-07 Thread Sébastien Miquel



To reproduce with `emacs -Q` :
 - Open an org buffer
 - Call ~(setq org-format-latex-options '(:foreground default
   :background "Black" :matchers ("$")))~
 - Call =C-c C-x C-l= (org-latex-preview) on a latex fragment such as
   $abc$

This bug was introduced by the commit 2f9e1569f which adds the option
`-bg Transparent` to the arguments of `dvipng`. According to its
manual, this option should be ignored if a background is already set,
but it doesn't seem to be. Perhaps org should set it differently.

--
Sébastien Miquel



Re: Using backticks for the inline code delimeter?

2021-03-31 Thread Sébastien Miquel

George Mauer writes:

is there a straightforward way to extend the org parser to do this?


I don't think so. It seems the emphasis markers are hard-coded
in various places.

From a quick look at the code, you'd have to customize
`org-emphasis-alist` and redefine `org-set-emph-re`  and
`org-do-emphasis-faces`. Maybe that'd be enough.


For the cosmetic part, there's this  piece of code from
https://archive.casouri.cat/note/2020/better-looking-verbatim-markup-in-org-mode/index.html
that displays org's `=` and `~` markers as ```.

--
Sébastien Miquel




Re: Why is there no inline-src syntax highlighting?

2021-03-29 Thread Sébastien Miquel

Hi,

I don't use inline src blocks, but your work makes me want to start !

Timothy writes:

there's just one thing I
haven't been able to work out --- how to have font-lock apply to
*multiple*  inline src blocks on one line.


With multiple inline src blocks that each have a result part,
fontification appears to work.

I think there are these two issues with the code :
 - When an inline src blocks has no matching result part,
   `org-fontify-inline-src-blocks-1` returns nil. It should return t
   instead, then it will be called again, to fontify the next block.
 - If an inline src block without results is followed by a src block
   with results, your function skips over the second src block.
   Perhaps you should assume that only whitespace separates a src
   block and its results (this is already assumed for subsequent
   evaluations of the block  to replace the results).

Regards,

--
Sébastien Miquel




Re: [PATCH] org.el (org-do-latex-and-related): Fix duplicate 'latex-and-related faces

2021-03-25 Thread Sébastien Miquel

Kyle Meyer writes:

> Please add a period after the changelog entry.

Done.

> Any reason not to pass limit as re-search-forward's BOUND instead?

I've looked at the history of this code, and some earlier comment
indicated that limit was ignored on purpose.

The only case where it'd make a difference with my patch is when limit
is in the middle of a latex fragment (since re-search-forward's BOUND
bounds the end of the match).

Now, AFAIU, this should almost never happen, since the fontified
region is extended
 + according to the font-lock-multiline text property, that latex
   fragment do have
 + and to contain matching begin/end delimiters.

I can think of a few edge cases where it may make a difference :
 - If you write a multiline fragment, and add $ delimiters
   afterwards, starting with the closing one. Then when you add the
   opening one, you wouldn't get fontification, I think
 - If you add an empty line in a multiline $ fragment by mistake, which
   breaks fontification, then remove it

--
Sébastien Miquel

>From 7fb4e2d408791742281bf220d227ccdcfd5baa71 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?S=C3=A9bastien=20Miquel?= 
Date: Fri, 19 Mar 2021 16:55:27 +0100
Subject: [PATCH] org.el (org-do-latex-and-related): Fix duplicate 'latex faces

* lisp/org.el (org-do-latex-and-related): Do not add a
'org-latex-and-related face beyond the fontification limit.
---
 lisp/org.el | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/lisp/org.el b/lisp/org.el
index 4db2dbe04..a0c703630 100644
--- a/lisp/org.el
+++ b/lisp/org.el
@@ -5502,6 +5502,8 @@ highlighting was done, nil otherwise."
 	(while (and (< (point) limit)
 		(re-search-forward org-latex-and-related-regexp nil t))
 	  (cond
+   ((>= (match-beginning 0) limit)
+	(throw 'found nil))
 	   ((cl-some (lambda (f)
 		   (memq f '(org-code org-verbatim underline
 	  org-special-keyword)))
-- 
2.31.0



[PATCH] org.el (org-do-latex-and-related): Fix duplicate 'latex-and-related faces

2021-03-19 Thread Sébastien Miquel

Hi,

With the current org-do-latex-and-related function, fontification of
a region before a LaTeX fragment repeatedly adds the
'org-latex-and-related face to the fragment.

You can reproduce with an org buffer with the following content.

#+BEGIN_SRC org
Foo
 $a^2 + b^2 = c^2$
#+END_SRC

 - Use (setq org-highlight-latex-and-related '(latex))
 - Write some text at the end of the first line, triggering
   refonfication of the line
 - For each character inserted, the org-latex-and-related face is
   added (as a duplicate) to the fragment, as you can check using
   describe-text-properties.

The attached patch fixes this.

--
Sébastien Miquel

>From ccce24e08b903e0e955a6b22f6c7321851ec5750 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?S=C3=A9bastien=20Miquel?= 
Date: Fri, 19 Mar 2021 16:55:27 +0100
Subject: [PATCH] org.el (org-do-latex-and-related): Fix duplicate 'latex faces

* lisp/org.el (org-do-latex-and-related): Do not add a
'org-latex-and-related face beyond the fontification limit
---
 lisp/org.el | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/lisp/org.el b/lisp/org.el
index 4db2dbe04..a0c703630 100644
--- a/lisp/org.el
+++ b/lisp/org.el
@@ -5502,6 +5502,8 @@ highlighting was done, nil otherwise."
 	(while (and (< (point) limit)
 		(re-search-forward org-latex-and-related-regexp nil t))
 	  (cond
+   ((>= (match-beginning 0) limit)
+	(throw 'found nil))
 	   ((cl-some (lambda (f)
 		   (memq f '(org-code org-verbatim underline
 	  org-special-keyword)))
-- 
2.30.2



Re: [PATCH] org-compat.el (org-mode-flyspell-verify): Do not spell check code in headline

2021-03-15 Thread Sébastien Miquel

Kyle Meyer writes:

Applied to master (7c4d057cd), adding a TINYCHANGE cookie to the commit
message.  You're bumping up against the cumulative number of changes
allowed without assigning copyright, so please consider completing the
copyright paperwork (or, if you've already done so, please let me know
I'll update your status listed at
<https://orgmode.org/worg/org-contribute.html>).


Thanks. I have completed the paperwork.

--
Sébastien Miquel




[PATCH] org-compat.el (org-mode-flyspell-verify): Do not spell check code in headline

2021-03-07 Thread Sébastien Miquel

Hi,

Currently code and verbatim snippets, and LaTeX fragments are spell
checked by `flyspell` if they're in a headline.

The attached patch fixes that.

--
Sébastien Miquel
>From b4291ce0ea455af499e75d3c9313183a0e8f46ec Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?S=C3=A9bastien=20Miquel?= 
Date: Sun, 7 Mar 2021 17:06:34 +0100
Subject: [PATCH] org-compat.el (org-mode-flyspell-verify): Do not check code
 in headline

* lisp/org-compat.el (org-mode-flyspell-verify): Do not spell check
code, verbatim and LaTeX fragments in headline title.
---
 lisp/org-compat.el | 7 ---
 1 file changed, 4 insertions(+), 3 deletions(-)

diff --git a/lisp/org-compat.el b/lisp/org-compat.el
index 8cbf33137..3d45bed7f 100644
--- a/lisp/org-compat.el
+++ b/lisp/org-compat.el
@@ -1025,8 +1025,7 @@ ELEMENT is the element at point."
 (defun org-mode-flyspell-verify ()
   "Function used for `flyspell-generic-check-word-predicate'."
   (if (org-at-heading-p)
-  ;; At a headline or an inlinetask, check title only.  This is
-  ;; faster than relying on `org-element-at-point'.
+  ;; At a headline or an inlinetask, check title only.
   (and (save-excursion (beginning-of-line)
 			   (and (let ((case-fold-search t))
   (not (looking-at-p "\\*+ END[ \t]*$")))
@@ -1035,7 +1034,9 @@ ELEMENT is the element at point."
 	   (match-beginning 4)
 	   (>= (point) (match-beginning 4))
 	   (or (not (match-beginning 5))
-	   (< (point) (match-beginning 5
+	   (< (point) (match-beginning 5)))
+   ;; Ignore checks in code, verbatim and others.
+   (org--flyspell-object-check-p (org-element-at-point)))
 (let* ((element (org-element-at-point))
 	   (post-affiliated (org-element-property :post-affiliated element)))
   (cond
-- 
2.30.1



[PATCH] ~org-font-lock-add-priority-faces~: ensure priority cookies are in a headline

2021-02-26 Thread Sébastien Miquel

Priority cookies are always in a headline.

The attached patch speeds up fontification of a 1k lines buffer by 0.1
second.

Note that the variable org-priority-regexp can't be modified since
it is used in the agenda and in org-get-priority.

Regards,

--
Sébastien Miquel
>From a348a3834b79608a20bfd4e28815ae3995c7eb5a Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?S=C3=A9bastien=20Miquel?= 
Date: Fri, 26 Feb 2021 18:02:32 +0100
Subject: [PATCH] org.el (org-font-lock-add-priority-faces): Speed up regexp

* org.el (org-font-lock-add-priority-faces): Speed up regexp.

Only fontify priority cookies in headlines.

TINYCHANGE
---
 lisp/org.el | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/lisp/org.el b/lisp/org.el
index 00596564f..8c976213d 100644
--- a/lisp/org.el
+++ b/lisp/org.el
@@ -5855,7 +5855,7 @@ If TAG is a number, get the corresponding match group."
 
 (defun org-font-lock-add-priority-faces (limit)
   "Add the special priority faces."
-  (while (re-search-forward org-priority-regexp limit t)
+  (while (re-search-forward (concat "^\\*+" org-priority-regexp) limit t)
 (let ((beg (match-beginning 1))
 	  (end (1+ (match-end 2
   (add-face-text-property
-- 
2.30.1



Re: Typing latency

2021-02-15 Thread Sébastien Miquel

Timothy writes:

#+begin_src diff
org-priority-regexp
- (defvar org-priority-regexp ".*?\\(\\[#\\([A-Z0-9]+\\)\\] ?\\)"
+ (defvar org-priority-regexp "^\\*+.*\\(\\[#\\([A-Z0-9]+\\)\\] ?\\)"
#+end_src

Since this thread seems to suggest it's (1) faster (2) more correct.

If anyone's considering writing a patch, note that this breaks
~org-get-priority~ and ~org-agenda-fontify-priorities~ (used in the
agenda). Best to keep both regexps around.

In a 1k lines org file, it reduces the time to fontify the whole
buffer by 0.1 second. I do not know how much of a change this makes
wrt to typing latency.

--
Sébastien Miquel




Re: local variables and export processing in hooks

2021-02-11 Thread Sébastien Miquel



M. ‘quintus’ Gülker writes:

Now I wonder whether #+BIND is more elegant. But my macro expansion
function modifies a buffer-local variable. Does #+BIND allow for that,
so that the changed value is available in the original org buffer?


The purpose of #+BIND is to set some variables in the copied buffer.

Also, these variables are set after macro expansion (and 
org-export-before-parsing-hook).


I'm curious, could you explain why/how you use macros to set variables 
in the original buffer ?


--
Sébastien Miquel




Re: Typing latency

2021-02-09 Thread Sébastien Miquel

Ihor Radchenko writes:

I have the following in my config to speed things up:

(setq org-priority-regexp "^\\*+.*\\(\\[#\\([A-Z0-9]+\\)\\] ?\\)")

For the latex fontification, I never had issues, but you might play with
org-highlight-latex-and-related.

Hope it helps.
Thanks, I've set it so aswell. It does speed up the initial 
fontification of the buffer.


I've looked into org-do-latex-and-related, but even replacing it with a 
dud doesn't fix

the delay, which must be caused by the already fontified parts.
I guess the issue must be wih emacs' internals.

--
Sébastien Miquel




Re: Typing latency

2021-02-09 Thread Sébastien Miquel

Ihor Radchenko writes:

You can try font-lock-profiler package
(https://github.com/Lindydancer/font-lock-profiler)

This is useful, thank you.

It reports that most of the time is spent in org-do-latex-and-related, 
and some 20% in something related to the priority faces (despite the 
lack of priority cookies).


I doubt that those two account for the whole of the slowdown, but I'll 
be checking it out.


--
Sébastien Miquel




Re: Typing latency

2021-02-09 Thread Sébastien Miquel



Russell Adams writes:

My only suggestion is have you looked at disabling flyspell?

Yes, thank you. Flyspell can cause some delay on its own aswell,
but in this situation/configuration it doesn't change the results.

--
Sébastien Miquel




Re: Typing latency

2021-02-09 Thread Sébastien Miquel

Eric S Fraga writes:

What Emacs version are you using?

This is using the latest ative-comp git branch.

--
Sébastien Miquel




Re: local variables and export processing in hooks

2021-02-09 Thread Sébastien Miquel



Eric S Fraga writes:

I tried but this doesn't seem to be propagated to the export as the
export works on a copy of the buffer, not the buffer itself.  That's
what #+BIND is for, supposedly...


I think this buffer copy preserves local variables. I know I use a such 
a local variable and its value can be read during macro expansion.


It seems the org-export-before-parsing-hook functions are run before the 
#+BIND values are collected (see org-export-as in ox.el).



--
Sébastien Miquel




Re: Bug: Unexpected movement of cursor when commenting code blocks

2021-01-15 Thread Sébastien Miquel

Hi,

yoctocell writes:

When I am inside a code block like this:

and I hit "C-x C-;" to comment the line, the cursor automatically jumps
to the second line of the code block after the comment is made.  I would
expect the cursor to move forward one line like it usually does and this
only happens in Org-mode, not when I edit the code block in a separate
buffer with "C-c C-'".


I have previously proposed a patch that I think fixes this issue, see

https://orgmode.org/list/3072e244-4615-aaad-4019-621bb3d1d...@posteo.eu/#t

Regards,

--
Sébastien Miquel




Re: [Patch] Bug: org-indent-region doesn't restore cursor position when org-src-tab-acts-natively is t [9.1.9 (release_9.1.9-65-g5e4542 @ /usr/share/emacs/26.3/lisp/org/)]

2020-10-23 Thread Sébastien Miquel
I've fixed an issue in my previous patch with the write-back buffer not 
getting killed.


Quick recap of the issue for anyone who might be interested:

When ~org-src-tab-acts-natively~ is t (which is now the default) 
functions such as ~indent-region~ and ~comment-region~ acting on a src 
block will call  ~org-edit-src~ and indent/comment from there.


The ~org-edit-src~ machinery replaces the original buffer with the 
src-edit buffer contents, and in doing so screws up the ~save-excursion~ 
calls from ~indent-region~ and ~comment-region~.


Because of this, calling ~indent-region~ or ~comment-region~ inside a 
src-block  sends the cursor to the beginning of the block.



The attached patch uses the ~replace-buffer-contents~ function when 
available in emacs >= 26.1 to fix this issue.



There's also a remaining issue with the ~undo-boundary~ call in 
~org-edit-src-exit~. Calling ~undo~ after ~indent-region~ or 
~comment-region~ will send the cursor to the beginning of the src block




Hi,

The attached patch fixes this issue for emacs >= 26.1.

It also fixes a similar issue where comment-line moves the cursor to 
the beginning of the block aswell.


Can this patch be applied ?


Sébastien

>From 8788ec81130130b538ca1d2f91853c9640040506 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?S=C3=A9bastien=20Miquel?= 
Date: Sun, 4 Oct 2020 21:33:30 +0200
Subject: [PATCH] org-edit-src: Use replace-buffer-contents

* lisp/org-src.el (org-src--contents-for-write-back): Use a write back
buffer

* lisp/org-src.el (org-edit-src-exit): Use replace-buffer-contents

* lisp/org-src.el (org-edit-src-save): Use replace-buffer-contents
---
 lisp/org-src.el | 42 +-
 1 file changed, 29 insertions(+), 13 deletions(-)

diff --git a/lisp/org-src.el b/lisp/org-src.el
index 28733d011..3509e148f 100644
--- a/lisp/org-src.el
+++ b/lisp/org-src.el
@@ -432,8 +432,8 @@ spaces after it as being outside."
 		(line-end-position)
 	  (point))
 
-(defun org-src--contents-for-write-back ()
-  "Return buffer contents in a format appropriate for write back.
+(defun org-src--contents-for-write-back (write-back-buf)
+  "Populate write-back-buf with contents in a format appropriate for write back.
 Assume point is in the corresponding edit buffer."
   (let ((indentation-offset
 	 (if org-src--preserve-indentation 0
@@ -445,7 +445,7 @@ Assume point is in the corresponding edit buffer."
 	(source-tab-width org-src--tab-width)
 	(contents (org-with-wide-buffer (buffer-string)))
 	(write-back org-src--allow-write-back))
-(with-temp-buffer
+(with-current-buffer write-back-buf
   ;; Reproduce indentation parameters from source buffer.
   (setq indent-tabs-mode use-tabs?)
   (when (> source-tab-width 0) (setq tab-width source-tab-width))
@@ -462,8 +462,7 @@ Assume point is in the corresponding edit buffer."
 	(let ((i (current-column)))
 	  (delete-region (line-beginning-position) (point))
 	  (indent-to (+ i indentation-offset
-	  (forward-line)))
-  (buffer-string
+	  (forward-line))
 
 (defun org-src--edit-element
 (datum name  initialize write-back contents remote)
@@ -1189,20 +1188,27 @@ Throw an error if there is no such buffer."
   (interactive)
   (unless (org-src-edit-buffer-p) (user-error "Not in a sub-editing buffer"))
   (set-buffer-modified-p nil)
-  (let ((edited-code (org-src--contents-for-write-back))
+  (let ((write-back-buf (generate-new-buffer "*org-src-write-back*"))
 	(beg org-src--beg-marker)
 	(end org-src--end-marker)
 	(overlay org-src--overlay))
+(org-src--contents-for-write-back write-back-buf)
 (with-current-buffer (org-src-source-buffer)
   (undo-boundary)
   (goto-char beg)
   ;; Temporarily disable read-only features of OVERLAY in order to
   ;; insert new contents.
   (delete-overlay overlay)
-  (delete-region beg end)
   (let ((expecting-bol (bolp)))
-	(insert edited-code)
+	(if (version< emacs-version "26.1")
+	(progn (delete-region beg end)
+		   (insert (with-current-buffer write-back-buf (buffer-string
+	(save-restriction
+	  (narrow-to-region beg end)
+	  (replace-buffer-contents write-back-buf)
+	  (goto-char (point-max
 	(when (and expecting-bol (not (bolp))) (insert "\n")))
+  (kill-buffer write-back-buf)
   (save-buffer)
   (move-overlay overlay beg (point
   ;; `write-contents-functions' requires the function to return
@@ -1219,23 +1225,33 @@ Throw an error if there is no such buffer."
 	 (remote org-src--remote)
 	 (coordinates (and (not remote)
 			   (org-src--coordinates (point) 1 (point-max
-	 (code (and write-back (org-src--contents-for-write-back
+	 (write-back-buf (and write-back (generate-new-buffer "*org-src-write-back*"
+(when write-back (org-src--contents-for-write-back write-back-buf))
 (set-buffer-modified-p nil)
 ;; Switch to source buffer.  Kill sub-editing buffer.
 (let ((edit-buffer 

Re: [Patch] Bug: org-indent-region doesn't restore cursor position when org-src-tab-acts-natively is t [9.1.9 (release_9.1.9-65-g5e4542 @ /usr/share/emacs/26.3/lisp/org/)]

2020-10-04 Thread Sébastien Miquel

Hi,

The attached patch fixes this issue for emacs >= 26.1.

It also fixes a similar issue where comment-line moves the cursor to the 
beginning of the block aswell.


Can this patch be applied ?

Regards,
Sébastien Miquel


>From 36f01b8e9d8dc335564f8dc932caf21c43c374eb Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?S=C3=A9bastien=20Miquel?= 
Date: Sun, 4 Oct 2020 21:33:30 +0200
Subject: [PATCH] org-edit-src: Use replace-buffer-contents

* lisp/org-src.el (org-src--contents-for-write-back): Use a write back
buffer

* lisp/org-src.el (org-edit-src-exit): Use replace-buffer-contents

* lisp/org-src.el (org-edit-src-save): Use replace-buffer-contents
---
 lisp/org-src.el | 40 +++-
 1 file changed, 27 insertions(+), 13 deletions(-)

diff --git a/lisp/org-src.el b/lisp/org-src.el
index 28733d011..4f0044394 100644
--- a/lisp/org-src.el
+++ b/lisp/org-src.el
@@ -432,8 +432,8 @@ spaces after it as being outside."
 		(line-end-position)
 	  (point))
 
-(defun org-src--contents-for-write-back ()
-  "Return buffer contents in a format appropriate for write back.
+(defun org-src--contents-for-write-back (write-back-buf)
+  "Populate write-back-buf with contents in a format appropriate for write back.
 Assume point is in the corresponding edit buffer."
   (let ((indentation-offset
 	 (if org-src--preserve-indentation 0
@@ -445,7 +445,7 @@ Assume point is in the corresponding edit buffer."
 	(source-tab-width org-src--tab-width)
 	(contents (org-with-wide-buffer (buffer-string)))
 	(write-back org-src--allow-write-back))
-(with-temp-buffer
+(with-current-buffer write-back-buf
   ;; Reproduce indentation parameters from source buffer.
   (setq indent-tabs-mode use-tabs?)
   (when (> source-tab-width 0) (setq tab-width source-tab-width))
@@ -462,8 +462,7 @@ Assume point is in the corresponding edit buffer."
 	(let ((i (current-column)))
 	  (delete-region (line-beginning-position) (point))
 	  (indent-to (+ i indentation-offset
-	  (forward-line)))
-  (buffer-string
+	  (forward-line))
 
 (defun org-src--edit-element
 (datum name  initialize write-back contents remote)
@@ -1189,20 +1188,27 @@ Throw an error if there is no such buffer."
   (interactive)
   (unless (org-src-edit-buffer-p) (user-error "Not in a sub-editing buffer"))
   (set-buffer-modified-p nil)
-  (let ((edited-code (org-src--contents-for-write-back))
+  (let ((write-back-buf (generate-new-buffer "*org-src-write-back*"))
 	(beg org-src--beg-marker)
 	(end org-src--end-marker)
 	(overlay org-src--overlay))
+(org-src--contents-for-write-back write-back-buf)
 (with-current-buffer (org-src-source-buffer)
   (undo-boundary)
   (goto-char beg)
   ;; Temporarily disable read-only features of OVERLAY in order to
   ;; insert new contents.
   (delete-overlay overlay)
-  (delete-region beg end)
   (let ((expecting-bol (bolp)))
-	(insert edited-code)
+	(if (version< emacs-version "26.1")
+	(progn (delete-region beg end)
+		   (insert (with-current-buffer write-back-buf (buffer-string
+	(save-restriction
+	  (narrow-to-region beg end)
+	  (replace-buffer-contents write-back-buf)
+	  (goto-char (point-max
 	(when (and expecting-bol (not (bolp))) (insert "\n")))
+  (kill-buffer write-back-buf)
   (save-buffer)
   (move-overlay overlay beg (point
   ;; `write-contents-functions' requires the function to return
@@ -1219,7 +1225,8 @@ Throw an error if there is no such buffer."
 	 (remote org-src--remote)
 	 (coordinates (and (not remote)
 			   (org-src--coordinates (point) 1 (point-max
-	 (code (and write-back (org-src--contents-for-write-back
+	 (write-back-buf (and write-back (generate-new-buffer "*org-src-write-back*"
+(when write-back (org-src--contents-for-write-back write-back-buf))
 (set-buffer-modified-p nil)
 ;; Switch to source buffer.  Kill sub-editing buffer.
 (let ((edit-buffer (current-buffer))
@@ -1229,13 +1236,20 @@ Throw an error if there is no such buffer."
   (kill-buffer edit-buffer))
 ;; Insert modified code.  Ensure it ends with a newline character.
 (org-with-wide-buffer
- (when (and write-back (not (equal (buffer-substring beg end) code)))
+ (when (and write-back (not (equal (buffer-substring beg end)
+   (with-current-buffer write-back-buf (buffer-string)
(undo-boundary)
(goto-char beg)
-   (delete-region beg end)
(let ((expecting-bol (bolp)))
-	 (insert code)
-	 (when (and expecting-bol (not (bolp))) (insert "\n")
+	 (if (version< emacs-version "26.1")
+	 (progn (delete-region beg end)
+		(insert (with-current-buffer write-back-buf (buffer-string
+	 (save-restriction
+	   (narrow-to-region beg end)
+	   (replace-buffer-contents write

Re: [PATCH] org-preview-latex-fragment dvipng fix for packages in org-latex-packages-alist breaking 'latex' command

2020-09-20 Thread Sébastien Miquel

Hi,

If I've understood your problem correctly, simply using ("" "fontspec" 
nil) in org-latex-packages-alist should fix your issue.


Per the documentation, this third argument stops the package from being 
used when compiling preview fragments.


Regards,

Sébastien



Gabriel S. X. Smith writes:

This is where my fix starts to come in. When org mode does the small

export of the latex fragment it would include the contents of
org-latex-packages-alist in the LaTex headers. Some packages (namely
'fontspec,' used for xelatex) would lead to a failure to create the
dvi file when the .tex file was compiled with the necessary
'latex' command.

To alleviate this problem I designed this wrapper function which
sets org-latex-packages-alist to nil just before executing the
org-latex-preview function.



Re: [PATCH] Bug: Fontification: Heading following a comment

2020-09-16 Thread Sébastien Miquel

Hi,

Thanks for taking a look.
Afaict, the only difference between blank and [ \t] are some weird 
unicode characters.
It isn't clear to me which one should be preferable and in what 
situation. I couldn't find any reference to this in any documentation of 
Org syntax.


I've replaced all instances of blank with (any " \t") in this function 
(that's the only instances in org.el).

The attached patch fixes the original issue.

Regards,
Sébastien Miquel

Nicolas Goaziou wrote:


Hello,

Sebastien Miquel  writes:


   (rx bol (group (zero-or-more blank) "#"
  (group (group (or (seq "+" (one-or-more (any "a-zA-Z")) 
(optional ":"))
-   space
+   blank

This looks wrong, but so does the current regexp. It should not be
`space' nor `blank', but [ \t] per Org syntax.

Regards,

>From 2bb847473f2199e1dfd03ddcafdd3563ed46ab78 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?S=C3=A9bastien=20Miquel?= 
Date: Wed, 16 Sep 2020 07:49:34 +0200
Subject: [PATCH] org.el (org-fontify-meta-lines-and-blocks-1): Fix meta lines
 regexp

* lisp/org.el (org-fontify-meta-lines-and-blocks-1): Fix meta lines
regexp to work correctly for lines with only a #.

Replace blank in regexp by (any " \t").

TINYCHANGE
---
 lisp/org.el | 14 +++---
 1 file changed, 7 insertions(+), 7 deletions(-)

diff --git a/lisp/org.el b/lisp/org.el
index d09d6c8d2..053635c85 100644
--- a/lisp/org.el
+++ b/lisp/org.el
@@ -5219,14 +5219,14 @@ by a #."
   "Fontify #+ lines and blocks."
   (let ((case-fold-search t))
 (when (re-search-forward
-	   (rx bol (group (zero-or-more blank) "#"
+	   (rx bol (group (zero-or-more (any " \t")) "#"
 			  (group (group (or (seq "+" (one-or-more (any "a-zA-Z")) (optional ":"))
-	space
+	(any " \t")
 	eol))
  (optional (group "_" (group (one-or-more (any "a-zA-Z"))
-			  (zero-or-more blank)
+			  (zero-or-more (any " \t"))
 			  (group (group (zero-or-more (not (any " \t\n"
- (zero-or-more blank)
+ (zero-or-more (any " \t"))
  (group (zero-or-more any)
 	   limit t)
   (let ((beg (match-beginning 0))
@@ -5249,7 +5249,7 @@ by a #."
 		quoting (member block-type org-protecting-blocks))
 	  (when (re-search-forward
 		 (rx-to-string `(group bol (or (seq (one-or-more "*") space)
-	   (seq (zero-or-more blank)
+	   (seq (zero-or-more (any " \t"))
 		"#+end"
 		,(match-string 4)
 		word-end
@@ -5323,11 +5323,11 @@ by a #."
 	  ;; Handle short captions
 	  (save-excursion
 	(beginning-of-line)
-	(looking-at (rx (group (zero-or-more blank)
+	(looking-at (rx (group (zero-or-more (any " \t"))
    "#+caption"
    (optional "[" (zero-or-more any) "]")
    ":")
-			(zero-or-more blank
+			(zero-or-more (any " \t")
 	  (add-text-properties (line-beginning-position) (match-end 1)
 			   '(font-lock-fontified t face org-meta-line))
 	  (add-text-properties (match-end 0) (line-end-position)
-- 
2.28.0


Re: Bug: org-block face isn't applied to special blocks

2020-09-14 Thread Sébastien Miquel
As far as src blocks being treated differently, there's already the 
~org-src-block-faces~ variable.


How about the following :

 + Make the ~org-block~ face apply to every block
 + either
   - add an ~org-src-block~ face
   - modify ~org-src-block-faces~ to allow a catch-all case
   It'd be nice for those to apply to unrecognized languages
 + keep ~org-fontify-quote-and-verse-blocks~

I think the only users that would notice the difference are the few (?) 
who use special blocks and expect them not to be fontified.
All they'd have to do to restore the previous behavior is set the 
~org-src-block~ and unset the ~org-block~ faces.





It isn't applied
  - to src blocks, when the language isn't recognized
  - to special blocks (that is, blocks with arbitrary names)

I make heavy use of special blocks and I'd like this face to apply to them.

I also make heavy use of special blocks but, for me, these are often
more like verse or quote blocks (i.e. the contents of the block are
prose) than src blocks so I would not want them to be treated as src
blocks are.  I think we'll need to have (yet) another option to control
this?





Re: Bug: org-block face isn't applied to special blocks

2020-09-10 Thread Sébastien Miquel

Hi Bastien,

As far as I can tell, with 8a083514a7 from master, the situation is now 
as follows.


The org-block face is applied
 - to src blocks, when the language is recognized
 - to example, export blocks
 - to verse and quote blocks, if ~org-fontify-quote-and-verse-blocks~ 
is ~t~.


It isn't applied
 - to src blocks, when the language isn't recognized
 - to special blocks (that is, blocks with arbitrary names)


I make heavy use of special blocks and I'd like this face to apply to them.
I think it would make more sense than the current behavior, and is less 
surprising. (It is also more in line with the previous docstring).


(I also think it should apply to unrecognized src blocks)

Regards,
Sebastien




Hi Sébastien

Sébastien Miquel  writes:


Hi Bastien,

With latest org-mode master, and emacs -q,
run (defface org-block '((t (:background "#494949" :extend t))) "")
before loading org-mode,
then visit an org buffer containing the three following blocks.

When I do so, the org-block face only gets applied to the src block, and
not to the quote block, nor the special block.

Fixed in maint, as 7769518f3.  Thanks for the report again,





Re: Bug: org-block face isn't applied to special blocks

2020-09-04 Thread Sébastien Miquel

Hi Bastien,

With latest org-mode master, and emacs -q,
run (defface org-block '((t (:background "#494949" :extend t))) "") 
before loading org-mode,

then visit an org buffer containing the three following blocks.

When I do so, the org-block face only gets applied to the src block, and 
not to the quote block, nor the special block.



#+BEGIN_theorem

#+END_theorem


#+BEGIN_SRC sh

#+END_SRC

#+BEGIN_QUOTE

#+END_QUOTE

Regards,




Hi Sébastien,

Sébastien Miquel  writes:


Afaict, the org-block face isn't applied to special blocks. Its
documentation implies it applies to any block.

The relevant function is org-fontify-meta-lines-and-blocks-1.

It may be a simple matter of changing the logic a bit and adding
(add-face-text-property bol-after-beginline beg-of-endline 'org-block
t).

Is it still the case with latest master?  If so, can you provide
a minimal example?

Thanks,





Couple of issues with org block meta lines faces

2020-07-10 Thread Sébastien Miquel

Hi,

If you start emacs as follows, making sure emacs picks up the latest org 
from org-plus-contrib, and with file.org being the content of this mail,


#+BEGIN_SRC sh :async
emacs -q \
  --eval "(defface org-block-begin-line '((t (:background \"blue\" 
:height 0.8))) \"\")" \

  --eval "(setq org-fontify-whole-block-delimiter-line t)" \
  file.org
#+END_SRC

You may observe the following issues:

 1) begin-line applies to both begin and end lines. This might be 
intended. If you define an org-block-end-line face, it gets applied instead.
 2) org-fontify-whole-block-delimiter-line is ignored. I'm aware I can 
set the :extend t property to the face. If it does nothing, maybe this 
variable should be removed.

 3) The following block has no face applied.
    - This only happens when the line with # at the top is empty
    - The :height part of the face seems to be responsible
    - It also works fine with default org version
 4) If you go to the end of the fontified end_src line (first src 
block), then press enter a couple of times the buffer position has the 
org-block-begin-line face applied (move cursor to see it).


#
#+BEGIN_SRC elisp
auie
auriaest
#+END_SRC

Regards,




Re: Breaking line inside list item

2020-06-29 Thread Sébastien Miquel




There is RET or C-j, depending on your settings.


C-j (~org-return-indent~) does work, thank you.


I'm not sure what you mean here.

Regards,


I meant to go from

 - Cogito,
   - ergo
 sum

to

 - Cogito,
   - ergo
   sum

But I don't really care about that use case anyway.



Hello,

Sébastien Miquel  writes:


Is there a binding or functionality to go from this, (where | is the
cursor position):

  - Cogito, |ergo sum.

To this:

  - Cogito,
    |ergo sum.

There is RET or C-j, depending on your settings.


In the case of nested list, a way to cycle the indentation to that of
the main or inside list would also be nice.

I'm not sure what you mean here.

Regards,




Breaking line inside list item

2020-06-29 Thread Sébastien Miquel

Hi,

Is there a binding or functionality to go from this, (where | is the 
cursor position):


 - Cogito, |ergo sum.

To this:

 - Cogito,
   |ergo sum.

Pressing enter + tab (calls ~indent-for-tab-command~) doesn't work : the 
line is indented to the dash instead. Although ~indent-for-tab-command~ 
does indent correctly when called in an empty line after a list item.


In the case of nested list, a way to cycle the indentation to that of 
the main or inside list would also be nice.





org-babel block with :exports code that gets evaluated on export

2020-04-01 Thread Sébastien Miquel

Hi,

Is there a way to have an org-babel block which only exports its code 
but still gets evaluated when exporting ?






Re: Bug: org-indent-region doesn't restore cursor position when org-src-tab-acts-natively is t [9.1.9 (release_9.1.9-65-g5e4542 @ /usr/share/emacs/26.3/lisp/org/)]

2020-02-29 Thread Sébastien Miquel

Hi Bastien,

Thank you again for looking into this. I guess you couldn't find an easy 
fix.


This bug is quite the pain, since it gets triggered by the comment-line 
function: every time I try to comment something, the cursor goes to the 
beginning of the block.


With org-src-tab-acts-natively being set to t by default, it may impact 
a lot of people.


Sorry to be nagging you, my elisp-fu is barely above kill/yank level, 
but I'm still curious why save-excursion seemingly isn't doing its job here.




Hi Sébastien,

yes, the initial fix was wrong.

The current behavior when the region is active in a src block is to
indent the whole source block, which is not what the user expect.  The
problem with the cursor going back to the beginning of the block comes
on top of this (bigger?) problem.

I could not find a proper fix yet, I will try later this week.

Thanks,





Bug: org-block face isn't applied to special blocks

2020-02-29 Thread Sébastien Miquel
Afaict, the org-block face isn't applied to special blocks. Its 
documentation implies it applies to any block.


The relevant function is org-fontify-meta-lines-and-blocks-1.

It may be a simple matter of changing the logic a bit and adding 
(add-face-text-property bol-after-beginline beg-of-endline 'org-block t).





Re: Bug: org-indent-region doesn't restore cursor position when org-src-tab-acts-natively is t [9.1.9 (release_9.1.9-65-g5e4542 @ /usr/share/emacs/26.3/lisp/org/)]

2020-02-09 Thread Sébastien Miquel

Thank you for looking into this.

As far as I can tell, this is still broken in 9.3.3, which has your fix 
included.


I've updated org to 9.3.3, describe-function brings me to the org.el 
file with your save-window-excursion modification, and I've deleted the 
.elc file just in case.




Hi Sébastien,

Sébastien Miquel  writes:


Thank you for your reply.

I can still reproduce with latest Org

Mh, yes, I can reproduce this now, and this is fixed in maint.

Thanks!





Re: Bug: org-indent-region doesn't restore cursor position when org-src-tab-acts-natively is t [9.1.9 (release_9.1.9-65-g5e4542 @ /usr/share/emacs/26.3/lisp/org/)]

2020-02-03 Thread Sébastien Miquel

Hi,

Thank you for your reply.

I can still reproduce with latest Org: what I did was start emacs with 
emacs -q --load empty_init.el, with empty_init.el setting the org 
package-archive to http://orgmode.org/elpa/, update org to 9.3.2 then 
proceed with the previous steps. The result is the same.


I'd be happy to provide any additional info.



Hi Sébastien,

I cannot reproduce this bug with latest Org version.

If you can, please test with the latest Org version and
report any problem.

Thanks,





Bug: org-indent-region doesn't restore cursor position when org-src-tab-acts-natively is t [9.1.9 (release_9.1.9-65-g5e4542 @ /usr/share/emacs/26.3/lisp/org/)]

2020-02-02 Thread Sébastien Miquel

Given the following src block in org-mode:

#+BEGIN_SRC emacs-lisp
(defun my-function ()
  (instruction1)
  (instruction2)
;;  (instruction3)
  )
#+END_SRC

Steps to reproduce (with emacs -Q):
 1) set org-src-tab-acts-natively to t
 2) set the mark at the beginning of the function definition inside the 
src block

 3) move point to the last line of the src block
 4) call org-indent-region

Effect: the code is indented, but the cursor is moved to first line of
the block.

Expected: the code is indented, and the cursor doesn't move.

Emacs  : GNU Emacs 26.3 (build 1, x86_64-pc-linux-gnu, GTK+ Version 3.24.10)
 of 2019-08-29
Package: Org mode version 9.1.9 (release_9.1.9-65-g5e4542 @ 
/usr/share/emacs/26.3/lisp/org/)


current state:
==
(setq
 org-src-mode-hook '(org-src-babel-configure-edit-buffer
         org-src-mode-configure-edit-buffer)
 org-after-todo-state-change-hook '(org-clock-out-if-current)
 org-metadown-hook '(org-babel-pop-to-session-maybe)
 org-clock-out-hook '(org-clock-remove-empty-clock-drawer)
 org-mode-hook '(#[0 "\300\301\302\303\304$\207"
           [add-hook change-major-mode-hook org-show-block-all append
            local]
           5]
         #[0 "\300\301\302\303\304$\207"
           [add-hook change-major-mode-hook org-babel-show-result-all
            append local]
           5]
         org-babel-result-hide-spec org-babel-hide-all-hashes)
 org-archive-hook '(org-attach-archive-delete-maybe)
 org-confirm-elisp-link-function 'yes-or-no-p
 org-agenda-before-write-hook '(org-agenda-add-entry-text)
 org-metaup-hook '(org-babel-load-in-session-maybe)
 org-bibtex-headline-format-function #[257 "\300\236A\207" [:title] 3 
"\n\n(fn ENTRY)"]

 org-babel-pre-tangle-hook '(save-buffer)
 org-tab-first-hook '(org-babel-hide-result-toggle-maybe
          org-babel-header-arg-expand)
 org-occur-hook '(org-first-headline-recenter)
 org-cycle-hook '(org-cycle-hide-archived-subtrees org-cycle-hide-drawers
          org-cycle-show-empty-lines
          org-optimize-window-after-visibility-change)
 org-speed-command-hook '(org-speed-command-activate
              org-babel-speed-command-activate)
 org-confirm-shell-link-function 'yes-or-no-p
 org-link-parameters '(("id" :follow org-id-open)
           ("rmail" :follow org-rmail-open :store
            org-rmail-store-link)
           ("mhe" :follow org-mhe-open :store org-mhe-store-link)
           ("irc" :follow org-irc-visit :store org-irc-store-link)
           ("info" :follow org-info-open :export org-info-export
            :store org-info-store-link)
           ("gnus" :follow org-gnus-open :store
            org-gnus-store-link)
           ("docview" :follow org-docview-open :export
            org-docview-export :store org-docview-store-link)
           ("bibtex" :follow org-bibtex-open :store
            org-bibtex-store-link)
           ("bbdb" :follow org-bbdb-open :export org-bbdb-export
            :complete org-bbdb-complete-link :store
            org-bbdb-store-link)
           ("w3m" :store org-w3m-store-link) ("file+sys")
           ("file+emacs") ("doi" :follow org--open-doi-link)
           ("elisp" :follow org--open-elisp-link)
           ("file" :complete org-file-complete-link)
           ("ftp" :follow
            (lambda (path) (browse-url (concat "ftp:" path
           ("help" :follow org--open-help-link)
           ("http" :follow
            (lambda (path) (browse-url (concat "http:" path
           ("https" :follow
            (lambda (path) (browse-url (concat "https:" path
           ("mailto" :follow
            (lambda (path) (browse-url (concat "mailto:; path
           ("news" :follow
            (lambda (path) (browse-url (concat "news:; path
           ("shell" :follow org--open-shell-link))
 )