Re: [PATCH] New LaTeX code export option: engraved

[Sébastien Miquel]

Hi,

Timothy writes:

  2. `minted` supports a `mathescape` option to render math content
     inside code. `fvextra` supports the same option, but maths
     characters are escaped by engrave-faces-latex-face-mapper.

I’l also take a look at this:)

Implemented in engrave-faces-latex, but not worked into ox-html yet. If you
could check it out on the engrave-faces-latex side and check it’s behaving
sanely, that would be helpful.


I've tried it, and mathescape works, thanks !

--
Sébastien Miquel

Re: [PATCH] New LaTeX code export option: engraved

[Timothy]

Hi Sebastien,

>>  1. engrave-faces-generate-preset generates emacs colors such as
>>     `:foreground “grey70”` which are not supported by
>>     engrave-faces-latex-gen-preamble-line and co.
>
> I’ll see if I can incorporate this into engrave-faces v0.3.1.

Done.

>>  2. `minted` supports a `mathescape` option to render math content
>>     inside code. `fvextra` supports the same option, but maths
>>     characters are escaped by engrave-faces-latex-face-mapper.
>
> I’l also take a look at this :)

Implemented in engrave-faces-latex, but not worked into ox-html yet. If you
could check it out on the engrave-faces-latex side and check it’s behaving
sanely, that would be helpful.

All the best,
Timothy

Re: [PATCH] New LaTeX code export option: engraved

[Timothy]

Hi Sebastien,

> I’m quite exited and impressed to see this alternative to minted,
> thank you.

That’s great to hear!

> I haven’t been able to test your patches to org, but I have tried
> engrave-faces.el. Here’s a couple issues I’ve run into.

Still helpful, thanks.

>  1. engrave-faces-generate-preset generates emacs colors such as
>     `:foreground “grey70”` which are not supported by
>     engrave-faces-latex-gen-preamble-line and co.

Ah. I’ve just found this snippet from `list-colors-print'
┌
│ (apply 'format "#%02x%02x%02x"
│(mapcar (lambda (c) (ash c -8))
│(color-values (car color
└

I’ll see if I can incorporate this into engrave-faces v0.3.1.

>  2. `minted` supports a `mathescape` option to render math content
>     inside code. `fvextra` supports the same option, but maths
>     characters are escaped by engrave-faces-latex-face-mapper.

I’l also take a look at this :)

Thanks for the comments Sebastien.

All the best,
Timothy

Re: [PATCH] New LaTeX code export option: engraved

[Sébastien Miquel]

Hi Timothy,

I'm quite exited and impressed to see this alternative to minted,
thank you.

I haven't been able to test your patches to org, but I have tried
engrave-faces.el. Here's a couple issues I've run into.

 1. engrave-faces-generate-preset generates emacs colors such as
    `:foreground "grey70"` which are not supported by
    engrave-faces-latex-gen-preamble-line and co.
 2. `minted` supports a `mathescape` option to render math content
    inside code. `fvextra` supports the same option, but maths
    characters are escaped by engrave-faces-latex-face-mapper.

Regards,

--
Sébastien Miquel

Re: [PATCH] New LaTeX code export option: engraved

[Daniel Fleischer]

Hi Timothy,

Very nice set of patches, it looks good. Listings and minted are still
workable options and your new engrave library is available. I must admit
it's not clear how to customize faces since engrave is a bridge between
major modes choices and latex but maybe show one or two examples (say
elisp and python) of how to change/override some face.

I think next we should apply these patches and be open to feedback
about source code latex export using these new changes. I also wish
there was a bit more documentation, but we can add more with time.

Maybe some feedback from Nicolas G.?

Best,

-- 

Daniel Fleischer

Re: [PATCH] New LaTeX code export option: engraved

[Timothy]

Ooops, a paren error managed to slip in there.

Here’s a new set, with a little extra refactoring slipped in.

–
Timothy
>From d231437e2c9f96bf70520d9ddda810a95667fcdd Mon Sep 17 00:00:00 2001
From: TEC 
Date: Sun, 21 Nov 2021 14:35:34 +0800
Subject: [PATCH 1/7] ox-latex: Refactor `org-latex-src-block'

* lisp/ox-latex.el (org-latex-src-block): Extract the per-format logic
from `org-latex-src-block' into new dedicated functions:
+ `org-latex-src-block--verbatim'
+ `org-latex-src-block--custom'
+ `org-latex-src-block--minted'
+ `org-latex-src-block--listings'
This makes `org-latex-src-block' much less monolithic, taking it from
175 lines to 30, and I find also makes it easier to understand.
---
 lisp/ox-latex.el | 339 ++-
 1 file changed, 185 insertions(+), 154 deletions(-)

diff --git a/lisp/ox-latex.el b/lisp/ox-latex.el
index 841ad48bc..c2f728a1c 100644
--- a/lisp/ox-latex.el
+++ b/lisp/ox-latex.el
@@ -2997,164 +2997,195 @@ (defun org-latex-src-block (src-block _contents info)
 	   (float (plist-get attributes :float))
 	   (listings (plist-get info :latex-listings)))
   (cond
-   ;; Case 1.  No source fontification.
((or (not lang) (not listings))
-	(let ((caption-str (org-latex--caption/label-string src-block info))
-  (verbatim (format "\\begin{verbatim}\n%s\\end{verbatim}"
-(org-export-format-code-default src-block info
-  (cond ((string= "multicolumn" float)
- (format "\\begin{figure*}[%s]\n%s%s\n%s\\end{figure*}"
- (plist-get info :latex-default-figure-position)
- (if caption-above-p caption-str "")
- verbatim
- (if caption-above-p "" caption-str)))
-(caption (concat
-  (if caption-above-p caption-str "")
-  verbatim
-  (if caption-above-p "" (concat "\n" caption-str
-(t verbatim
-   ;; Case 2.  Custom environment.
+(org-latex-src-block--verbatim src-block info lang caption caption-above-p label
+   num-start retain-labels attributes float))
(custom-env
-	(let ((caption-str (org-latex--caption/label-string src-block info))
-  (formatted-src (org-export-format-code-default src-block info)))
-  (if (string-match-p "\\`[a-zA-Z0-9]+\\'" custom-env)
-	  (format "\\begin{%s}\n%s\\end{%s}\n"
-		  custom-env
-		  (concat (and caption-above-p caption-str)
-			  formatted-src
-			  (and (not caption-above-p) caption-str))
-		  custom-env)
-	(format-spec custom-env
-			 `((?s . ,formatted-src)
-			   (?c . ,caption)
-			   (?f . ,float)
-			   (?l . ,(org-latex--label src-block info))
-			   (?o . ,(or (plist-get attributes :options) "")))
-   ;; Case 3.  Use minted package.
+(org-latex-src-block--custom src-block info lang caption caption-above-p label
+ num-start retain-labels attributes float custom-env))
((eq listings 'minted)
-	(let* ((caption-str (org-latex--caption/label-string src-block info))
-	   (placement (or (org-unbracket-string "[" "]" (plist-get attributes :placement))
-			  (plist-get info :latex-default-figure-position)))
-	   (float-env
-		(cond
-		 ((string= "multicolumn" float)
-		  (format "\\begin{listing*}[%s]\n%s%%s\n%s\\end{listing*}"
-			  placement
-			  (if caption-above-p caption-str "")
-			  (if caption-above-p "" caption-str)))
-		 (caption
-		  (format "\\begin{listing}[%s]\n%s%%s\n%s\\end{listing}"
-			  placement
-			  (if caption-above-p caption-str "")
-			  (if caption-above-p "" caption-str)))
-		 ((string= "t" float)
-		  (concat (format "\\begin{listing}[%s]\n"
-  placement)
-			  "%s\n\\end{listing}"))
-		 (t "%s")))
-	   (options (plist-get info :latex-minted-options))
-	   (body
-		(format
-		 "\\begin{minted}[%s]{%s}\n%s\\end{minted}"
-		 ;; Options.
-		 (concat
-		  (org-latex--make-option-string
-		   (if (or (not num-start) (assoc "linenos" options))
-		   options
-		 (append
-		  `(("linenos")
-			("firstnumber" ,(number-to-string (1+ num-start
-		  options)))
-		  (let ((local-options (plist-get attributes :options)))
-		(and local-options (concat "," local-options
-		 ;; Language.
-		 (or (cadr (assq (intern lang)
- (plist-get info :latex-minted-langs)))
-		 (downcase lang))
-		 ;; Source code.
-		 (let* ((code-info (org-export-unravel-code src-block))
-			(max-width
-			 (apply 'max
-(mapcar 'length
-	(org-split-string (car code-info)
-			  "\n")
-		   (org-export-format-code
-		(car code-info)
-		(lambda (loc _num ref)
-		  (concat
-		   loc
-		   (when ref
-			 ;; Ensure references are flushed to the right,
-			 ;; separated with 6 spaces from the widest line
-			 

Re: [PATCH] New LaTeX code export option: engraved

[Timothy]

Hi Ihor,

Find attached an updated patchset, and comments below :)

Ihor Radchenko  writes:

> Maybe “The other two options”? Also, using first/second here is a bit
> confusing because: (1) they are actually 3/4 in the above list; (2)
> engraved is listed last.

Docstring changed.

>> +The second more comprehensive option can be used with,
>
> *can be set with

Changed.

> I feel slightly confused about using the word “package” here. Which one
> refers to LaTeX package and which one to Emacs? I would state “Emacs
> package” explicitly to highlight contrast with “LaTeX package”.

Clarifications added to the docstring.

>> +\\{\\\FancyVerbLine}
>
> I’d like to see a comment on what it does.

Added.

> Same request to provide a comment. Also, what it that % TODO doing
> there? It is confusing.

A comment has been added, and the TODO removed.

> Also, it is unclear what the [breakable,xparse] options to tcolorbox are
> doing there and what will happen if the user removes them.

“breakable” allows the box to be broken across pages, and “xparse” provides
`\DeclareTColorBox'.

> Further, I am wondering what is going to happen if the user happens to
> have tcolorbox without options loaded via org-latex-packages-alist.

They could see: ERROR: LaTeX Error: Option clash for package tcolorbox.

They would need to tweak such a tcolorbox entry to include the options used
here.

>> +There is quite a lot of flexibility in what this preamble can be, as long 
>> as it:
>> +- Loads the fvextra package.
>> +- Loads the package xcolor (if it is not already loader elsewhere).
>> +- Defines a \“Code\” environment (note the capital C), which can be
>> +  later used to wrap \“Verbatim\” environments (provided by fvextra).
>
> The last point is not very clear. What kind of environment?

Anything that the user wants to do to modify the display of the generated
`Verbatim' environment.

>> +(defun org-latex-generate-engraved-preamble (info syntax-colours-p)
>> +  “Generate the preamble to setup engraved code.
>> +The result is constructed from `org-latex-engraved-preamble’ and
>> +`org-latex-engraved-options’.”
>
> This is relying on
>
> (:latex-engraved-options nil nil org-latex-engraved-options)
> (:latex-engraved-preamble nil nil org-latex-engraved-preamble)
>
> If it changes any time in future (e.g. to allow per-file settings), the
> docstring may be overlooked.

Docstring tweaked.

> I’d use FIRST-MATCH argument for org-element-map. It will be slightly
> faster on large buffers.

Ah nice, I’ll make use of that.

>> -   (downcase org-lang)))
>> +   (downcase lang)))
>
> I am not sure if this belongs to this patch. Please double check.

Ooops, moved to the correct patch.

>> +(mapcar ’length
>> +(org-split-string (car code-info)
>> +  “”)
>
> I am not sure how well it will work with e.g. Chinese characters in comments.

I’ve added a patch replacing `length' with `string-width'

> Maybe the functions could be rewritten using cl-defun with keys and
>  and then called via apply on a let-bound arg plist?

Done.

All the best,
Timothy
>From d231437e2c9f96bf70520d9ddda810a95667fcdd Mon Sep 17 00:00:00 2001
From: TEC 
Date: Sun, 21 Nov 2021 14:35:34 +0800
Subject: [PATCH 1/7] ox-latex: Refactor `org-latex-src-block'

* lisp/ox-latex.el (org-latex-src-block): Extract the per-format logic
from `org-latex-src-block' into new dedicated functions:
+ `org-latex-src-block--verbatim'
+ `org-latex-src-block--custom'
+ `org-latex-src-block--minted'
+ `org-latex-src-block--listings'
This makes `org-latex-src-block' much less monolithic, taking it from
175 lines to 30, and I find also makes it easier to understand.
---
 lisp/ox-latex.el | 339 ++-
 1 file changed, 185 insertions(+), 154 deletions(-)

diff --git a/lisp/ox-latex.el b/lisp/ox-latex.el
index 841ad48bc..c2f728a1c 100644
--- a/lisp/ox-latex.el
+++ b/lisp/ox-latex.el
@@ -2997,164 +2997,195 @@ (defun org-latex-src-block (src-block _contents info)
 	   (float (plist-get attributes :float))
 	   (listings (plist-get info :latex-listings)))
   (cond
-   ;; Case 1.  No source fontification.
((or (not lang) (not listings))
-	(let ((caption-str (org-latex--caption/label-string src-block info))
-  (verbatim (format "\\begin{verbatim}\n%s\\end{verbatim}"
-(org-export-format-code-default src-block info
-  (cond ((string= "multicolumn" float)
- (format "\\begin{figure*}[%s]\n%s%s\n%s\\end{figure*}"
- (plist-get info :latex-default-figure-position)
- (if caption-above-p caption-str "")
- verbatim
- (if caption-above-p "" caption-str)))
-(caption (concat
-  (if caption-above-p 

Re: [PATCH] New LaTeX code export option: engraved

[Ihor Radchenko]

Timothy  writes:

Thanks for the updated patch! See some more comments below.

> +The first two options provide basic syntax
> +highlighting (listings), or none at all (verbatim).
> ...
> -Alternatively,
> +There are two options for more comprehensive fontification. The
> +first can be set with,

Maybe "The other two options"? Also, using first/second here is a bit
confusing because: (1) they are actually 3/4 in the above list; (2)
engraved is listed last.

> +The second more comprehensive option can be used with,

*can be set with

> -causes source code to be exported using the minted package as
> -opposed to listings.  If you want to use minted, you need to add
> -the minted package to `org-latex-packages-alist', for example
> +which causes source code to be exported using the minted package
> +as opposed to listings.  If you want to use minted, you need to
> +add the minted package to `org-latex-packages-alist', for example
>  using customize, or with

I feel slightly confused about using the word "package" here. Which one
refers to LaTeX package and which one to Emacs? I would state "Emacs
package" explicitly to highlight contrast with "LaTeX package".
  
> +(defcustom org-latex-engraved-preamble
> +  "\\usepackage{fvextra}
> +
> +[FVEXTRA-SETUP]
> +
> +\\renewcommand\\theFancyVerbLine{\\footnotesize\\color{black!40!white}\\arabic{FancyVerbLine}}

I'd like to see a comment on what it does.

> +\\usepackage{xcolor}
> +
> +\\providecolor{codebackground}{HTML}{f7f7f7}
> +\\providecolor{codeborder}{HTML}{f0f0f0}
> +\\providecolor{EFD}{HTML}{28292e}
> +
> +% TODO have code boxes keep line vertical alignment
> +\\usepackage[breakable,xparse]{tcolorbox}
> +\\DeclareTColorBox[]{Code}{o}%

Same request to provide a comment. Also, what it that % TODO doing
there? It is confusing.

Also, it is unclear what the [breakable,xparse] options to tcolorbox are
doing there and what will happen if the user removes them.

Further, I am wondering what is going to happen if the user happens to
have tcolorbox without options loaded via org-latex-packages-alist.

> +There is quite a lot of flexibility in what this preamble can be, as long as 
> it:
> +- Loads the fvextra package.
> +- Loads the package xcolor (if it is not already loader elsewhere).
> +- Defines a \"Code\" environment (note the capital C), which can be
> +  later used to wrap \"Verbatim\" environments (provided by fvextra).

The last point is not very clear. What kind of environment?

> +(defun org-latex-generate-engraved-preamble (info syntax-colours-p)
> +  "Generate the preamble to setup engraved code.
> +The result is constructed from `org-latex-engraved-preamble' and
> +`org-latex-engraved-options'."

This is relying on

(:latex-engraved-options nil nil org-latex-engraved-options)
(:latex-engraved-preamble nil nil org-latex-engraved-preamble)

If it changes any time in future (e.g. to allow per-file settings), the
docstring may be overlooked. 

> +   (let ((src-p (org-element-map (plist-get info :parse-tree)
> +'(src-block inline-src-block) #'identity))
> + (fixedw-p
> +  (org-element-map (plist-get info :parse-tree)
> +  '(example-block fixed-width) #'identity)))

I'd use FIRST-MATCH argument for org-element-map. It will be slightly
faster on large buffers.

> -   (downcase org-lang)))
> +   (downcase lang)))

I am not sure if this belongs to this patch. Please double check.

> +(mapcar 'length
> +(org-split-string (car code-info)
> +  "\n")

I am not sure how well it will work with e.g. Chinese characters in comments.

Best,
Ihor

Re: [PATCH] New LaTeX code export option: engraved

[Ihor Radchenko]

Max Nikulin  writes:

>> +(defun org-latex-src-block--verbatim
>> +(src-block info _lang caption caption-above-p _label
>> +   _num-start _retain-labels _attributes float)
>
> On the one hand I have no a better suggestion, but on the other hand 10 
> arguments is too much for straightforward code from my point of view. 
> Unsure that getting parameters in each function or passing a property 
> list would not be worse however.

I agree that having that many unused arguments is awkward, especially
when the arguments are repeated in all the calls.
Maybe the functions could be rewritten using cl-defun with keys and
 and then called via apply on a let-bound arg plist?

Best,
Ihor

Re: [PATCH] New LaTeX code export option: engraved

[Ihor Radchenko]

Timothy  writes:

>> Let me learn a bit about the different code highlighting options in
>> order to understand what you offer.
>
> Sure. If it’s any help, here’s a comparison example I whipped up:
> 

Timothy, could you please not use 0x0 to share not-so-large images? 0x0
deletes uploaded files within ~1year time and this link will no longer
be valid if someone tries to read this discussion years later.

Also, this illustration makes me wonder if engrave-faces can provide a
color scheme that is mimicking minted. Generally, easy customisation of
highlight schemes could be useful. On per-src-block basis or even on
per-language basis if document contains course blocks with different
programming languages.

Best,
Ihor

Re: [PATCH] New LaTeX code export option: engraved

[Max Nikulin]

On 05/05/2022 22:17, Timothy wrote:

Subject: [PATCH 1/4] ox-latex: Refactor `org-latex-src-block'


When I was trying to fix an issue with caption, I was not brave enough 
for massive changes. I do not like the original code of the function.


There are some unit tests for exporting of src blocks. Maybe the new 
option should be tested as well.



+(defun org-latex-src-block--verbatim
+(src-block info _lang caption caption-above-p _label
+   _num-start _retain-labels _attributes float)


On the one hand I have no a better suggestion, but on the other hand 10 
arguments is too much for straightforward code from my point of view. 
Unsure that getting parameters in each function or passing a property 
list would not be worse however.


It reminds me a case when a colleague failed in despair trying to figure 
out what was wrong with his 15 argument fortran function. I do not 
remember exactly whether it was a missed argument or an extra one. The 
last one was assumed to be a parameter (constant) and the identifier was 
not available in debugger since compiler just substituted the number 
without creation of a symbol and it added even more confusion.


My other notes are rather loosely related to your patches. The issues 
may be postponed till someone will start a new thread for them. My hope 
is that some of them might be easily addressed so that they will not 
need separate testing.


A problem with percent sign in captions is tracked on 
updates.orgmode.org. Some environments has not fixed yet:

https://list.orgmode.org/yt2pr01mb45101e27dc6251d8f8b7b366f6...@yt2pr01mb4510.canprd01.prod.outlook.com/

That time I was puzzled why the option is named "multicolumn" while only 
regular "figure" is added:



+(cond ((string= "multicolumn" float)
+   (format "\\begin{figure*}[%s]\n%s%s\n%s\\end{figure*}"


If language is added to "#+begin_example:" than the block content is 
properly fontified in the Emacs buffer. It would be great to treat such 
example similar to source block during export. When some code is not 
supposed to be executed, it is natural to use example block. I am 
curious if new functions may be reused.

Re: [PATCH] New LaTeX code export option: engraved

[Timothy]

> [5. text/x-patch; 0004-ox-latex-Introduce-engraved-code-highlighting.patch]…

Ooops, I had some ucommited changes. The correct version is attached.

Timothy
>From b66c291b1f0d1419742449bcde42bf0c4d620c23 Mon Sep 17 00:00:00 2001
From: TEC 
Date: Sun, 21 Nov 2021 20:04:12 +0800
Subject: [PATCH] ox-latex: Introduce "engraved" code highlighting

* lisp/ox-latex.el (org-latex-src-block, org-latex-src-block--engraved,
org-latex-inline-src-block, org-latex-inline-src-block--engraved,
org-latex-template, org-latex-listings): Make use of the engraved-faces
package (available on ELPA) to provide an alternative LaTeX code
highlighting backend which functions similarly to htmlize.el for HTML
exports.

(org-latex-engraved-preamble, org-latex-engraved-options): Introduce
variables to construct the preamble for engraved code blocks.

* lisp/ox-beamer.el (org-beamer-template): Modify to add engrave-faces
preamble when applicable.
---
 lisp/ox-beamer.el |   9 ++
 lisp/ox-latex.el  | 273 --
 2 files changed, 272 insertions(+), 10 deletions(-)

diff --git a/lisp/ox-beamer.el b/lisp/ox-beamer.el
index 6be73c91e..73bd95539 100644
--- a/lisp/ox-beamer.el
+++ b/lisp/ox-beamer.el
@@ -857,6 +857,15 @@ (defun org-beamer-template (contents info)
  (let ((template (plist-get info :latex-hyperref-template)))
(and (stringp template)
 	(format-spec template (org-latex--format-spec info
+ ;; engrave-faces-latex preamble
+ (when (eq org-latex-listings 'engraved)
+   (let ((src-p (org-element-map (plist-get info :parse-tree)
+'(src-block inline-src-block) #'identity))
+ (fixedw-p
+  (org-element-map (plist-get info :parse-tree)
+  '(example-block fixed-width) #'identity)))
+ (when (or src-p fixedw-p)
+   (org-latex-generate-engraved-preamble info src-p
  ;; Document start.
  "\\begin{document}\n\n"
  ;; Title command.
diff --git a/lisp/ox-latex.el b/lisp/ox-latex.el
index 4181db175..83bb6f078 100644
--- a/lisp/ox-latex.el
+++ b/lisp/ox-latex.el
@@ -37,6 +37,8 @@ (defvar org-latex-default-packages-alist)
 (defvar org-latex-packages-alist)
 (defvar orgtbl-exp-regexp)
 
+(declare-function engrave-faces-latex-gen-preamble "ext:engrave-faces-latex")
+(declare-function engrave-faces-latex-buffer "ext:engrave-faces-latex")
 
 
 ;;; Define Back-End
@@ -125,6 +127,8 @@ (org-export-define-backend 'latex
 (:latex-default-quote-environment nil nil org-latex-default-quote-environment)
 (:latex-default-table-mode nil nil org-latex-default-table-mode)
 (:latex-diary-timestamp-format nil nil org-latex-diary-timestamp-format)
+(:latex-engraved-options nil nil org-latex-engraved-options)
+(:latex-engraved-preamble nil nil org-latex-engraved-preamble)
 (:latex-footnote-defined-format nil nil org-latex-footnote-defined-format)
 (:latex-footnote-separator nil nil org-latex-footnote-separator)
 (:latex-format-drawer-function nil nil org-latex-format-drawer-function)
@@ -937,22 +941,48 @@ (defcustom org-latex-listings nil
   "Non-nil means export source code using the listings package.
 
 This package will fontify source code, possibly even with color.
-If you want to use this, you also need to make LaTeX use the
-listings package, and if you want to have color, the color
-package.  Just add these to `org-latex-packages-alist', for
-example using customize, or with something like:
+There are four implementations of this functionality you may
+choose from (ordered from least to most capable):
+1. Verbatim (nil)
+2. Listings (t)
+3. Minted (minted)
+4. Engraved (engraved)
+
+The first two options provide basic syntax
+highlighting (listings), or none at all (verbatim).
+
+When using listings, you also need to make use of the LaTeX
+\"listings\" package. The \"color\" package is also needed if you
+would like color too.  These can simply be added to
+`org-latex-packages-alist', using customise or something like:
 
   (require \\='ox-latex)
   (add-to-list \\='org-latex-packages-alist \\='(\"\" \"listings\"))
   (add-to-list \\='org-latex-packages-alist \\='(\"\" \"color\"))
 
-Alternatively,
+There are two options for more comprehensive fontification. The
+first can be set with,
+
+  (setq org-latex-listings \\='engraved)
+
+which causes source code to be run through
+`engrave-faces-latex-buffer', which generates colorings using
+Emacs' font-lock information.  This requires the engrave-faces
+package (availible from ELPA), and the fvextra LaTeX package be
+installed.
+
+The styling of the engraved result can customised with
+`org-latex-engraved-preamble' and `org-latex-engraved-options'.
+The default preamble also uses the tcolorbox LaTeX package in
+addition to fvextra.
+
+The second more comprehensive option can be used with,
 
   (setq org-latex-listings \\='minted)
 
-causes source code to be exported using the minted package as
-opposed to listings.  If 

Re: [PATCH] New LaTeX code export option: engraved

[Timothy]

Hi Daniel,

> Hi Timothy, thank you very much for your work!

Thanks for the kind words 

> Let me learn a bit about the different code highlighting options in
> order to understand what you offer.

Sure. If it’s any help, here’s a comparison example I whipped up:


All the best,
Timothy

Re: [PATCH] New LaTeX code export option: engraved

[Timothy]

Hi Ihor,

> Thanks!
> Implementing fontification using Emacs capabilities is certainly a step
> in the right direction. LaTeX support for fontification has always been
> tricky.

I’m glad to hear you’re of a similar mind to me with this.

> - I tried to test your patch, and it only works partially. There is some
>   stray text caused by LaTeX errors:
>
> [2. application/vnd.lotus-organizer; test.org]…
> [3. application/pdf; test.pdf]…

Ah. I thought that hyperref loaded xcolor, but it seems my assumption was
incorrect. I’ve added `\usepackage{xcolor}' to the default
`org-latex-engraved-preamble', but maybe I should ask people to modify
`org-latex-packages-alist'. I’m not sure.

> - You did not add a NEWS entry and did not update the manual in your patch.

I’m waiting till the functional content of these packages is settled/accepted,
and then I’ll write NEWS and manual entries.

> - There are many compiler warnings emitted when compiling Org with your patch

Oops, I keep on forgetting to check byte compilation. These should all be fixed 
now.

> The docstrings are missing in the above.

Docstrings have been added.

> I am not sure why, but the word fancy feels slightly annoying here.

Docsting rewritten.

> Since engraved is not entirely relying on LaTeX options, a lot of
> customisation is not mentioned in this docstring. AFAIU, color
> customisation is only possible by changing defcustoms from engrave-faces
> package.
>
> Another related note is what is going to happen in beamer export with
> dark background. The default face mapping in engrave-faces is using some
> kind of light theme, which may lose all the contrast on not-light
> background.

Modifying the style of engraved-faces-latex’s output is indeed done by
customising a engraved-faces variable. I don’t think we should attempt to do
anything further with this within Org.

To elaborate a bit, the generated LaTeX uses the styling information given in
`engrave-faces-preset-styles'. Changing this to use the current Emacs theme is 
as
simple as `(setq engrave-faces-preset-styles (engrave-faces-generate-preset))'.

> It feels that codebackground, codeborder, and EFD should be customizable
> by org-latex-engraved-options.

Hmm. I don’t think so. They are entirely self-contained within the preamble
variable. Should there be a nice existing `org-latex-user-colors' variable or
such, it would make a lot of sense to shove this there, but since no such
variable exists I’m not sure we can really do much better than just asking users
to modify `org-latex-engraved-preamble'.

> Docstring?

Added.

>> +(message “Cannot engrave inline src block, `engrave-faces-latex’ is 
>> unavailible.”)
>
> Why message instead of error?

User errors are now thrown.

Thanks for the feedback!
Timothy
>From d231437e2c9f96bf70520d9ddda810a95667fcdd Mon Sep 17 00:00:00 2001
From: TEC 
Date: Sun, 21 Nov 2021 14:35:34 +0800
Subject: [PATCH 1/4] ox-latex: Refactor `org-latex-src-block'

* lisp/ox-latex.el (org-latex-src-block): Extract the per-format logic
from `org-latex-src-block' into new dedicated functions:
+ `org-latex-src-block--verbatim'
+ `org-latex-src-block--custom'
+ `org-latex-src-block--minted'
+ `org-latex-src-block--listings'
This makes `org-latex-src-block' much less monolithic, taking it from
175 lines to 30, and I find also makes it easier to understand.
---
 lisp/ox-latex.el | 339 ++-
 1 file changed, 185 insertions(+), 154 deletions(-)

diff --git a/lisp/ox-latex.el b/lisp/ox-latex.el
index 841ad48bc..c2f728a1c 100644
--- a/lisp/ox-latex.el
+++ b/lisp/ox-latex.el
@@ -2997,164 +2997,195 @@ (defun org-latex-src-block (src-block _contents info)
 	   (float (plist-get attributes :float))
 	   (listings (plist-get info :latex-listings)))
   (cond
-   ;; Case 1.  No source fontification.
((or (not lang) (not listings))
-	(let ((caption-str (org-latex--caption/label-string src-block info))
-  (verbatim (format "\\begin{verbatim}\n%s\\end{verbatim}"
-(org-export-format-code-default src-block info
-  (cond ((string= "multicolumn" float)
- (format "\\begin{figure*}[%s]\n%s%s\n%s\\end{figure*}"
- (plist-get info :latex-default-figure-position)
- (if caption-above-p caption-str "")
- verbatim
- (if caption-above-p "" caption-str)))
-(caption (concat
-  (if caption-above-p caption-str "")
-  verbatim
-  (if caption-above-p "" (concat "\n" caption-str
-(t verbatim
-   ;; Case 2.  Custom environment.
+(org-latex-src-block--verbatim src-block info lang caption caption-above-p label
+   num-start retain-labels attributes float))
(custom-env
-	(let ((caption-str 

Re: [PATCH] New LaTeX code export option: engraved

[Ihor Radchenko]

Timothy  writes:

> This patchset accomplishes two things:
> 1. It refactors the overly large `org-latex-src-block' function, and makes a 
> few
>other improvements to pre-existing code
> 2. It adds a new option for exporting code, named (you guessed it!) “engraved”
>
> What is this new option, and why do we want it?
>
> About a year ago I started work on a package that generalises the 
> functionality
> of `htmlize.el', termed `engrave-faces'
> (). It provides the ability 
> to
> extract font-lock information and export it to a number of formats: html, 
> ansi,
> and (crucially) LaTeX! Since the LaTeX export is built on the `fvextra' 
> (LaTeX)
> package (like pygments), the vast majority of the Minted options you’re used 
> to
> just carry over.
>
> This allows for a result that is, I think, straight up better than all the
> pre-existing options. For starters, you can now apply syntax highlighting to 
> any
> language you have a major mode for.

Thanks!
Implementing fontification using Emacs capabilities is certainly a step
in the right direction. LaTeX support for fontification has always been
tricky.

Some comments:

- I tried to test your patch, and it only works partially. There is some
  stray text caused by LaTeX errors:


test.org
Description: Lotus Organizer


test.pdf
Description: Adobe PDF document

- You did not add a NEWS entry and did not update the manual in your patch.
- There are many compiler warnings emitted when compiling Org with your patch

> +(defun org-latex-inline-src-block--minted (info code lang)
> +  (let ((mint-lang (or (cadr (assq (intern lang)
> +
> +(defun org-latex-inline-src-block--listings (info code lang)
> +  (let* ((lst-lang (or (cadr (assq (intern lang)

The docstrings are missing in the above.

> -Alternatively,
> +There are two fancier options for fontification.
> +
> +The first fancy alternative,

I am not sure why, but the word fancy feels slightly annoying here.

> +
> +The styling of the engraved result can customised with
> +`org-latex-engraved-preamble' and `org-latex-engraved-options'.
> +The default preamble also uses the tcolorbox LaTeX package in
> +addition to fvextra.

Since engraved is not entirely relying on LaTeX options, a lot of
customisation is not mentioned in this docstring. AFAIU, color
customisation is only possible by changing defcustoms from engrave-faces
package.

Another related note is what is going to happen in beamer export with
dark background. The default face mapping in engrave-faces is using some
kind of light theme, which may lose all the contrast on not-light
background.

> +\\renewcommand\\theFancyVerbLine{\\footnotesize\\color{black!40!white}\\arabic{FancyVerbLine}}
> +
> +\\providecolor{codebackground}{HTML}{f7f7f7}
> +\\providecolor{codeborder}{HTML}{f0f0f0}
> +\\providecolor{EFD}{HTML}{28292e}

> +(defcustom org-latex-engraved-options
> +  '(("commandchars" . "\\{\\}")
> +("highlightcolor" . "white!95!black!80!blue")
> +("breaklines" . "true")
> +("breaksymbol" . 
> "\\color{white!60!black}\\tiny\\ensuremath{\\hookrightarrow}"))
> +  "Association list of options for the latex fvextra package when engraving 
> code.

It feels that codebackground, codeborder, and EFD should be customizable
by org-latex-engraved-options.

> +(defun org-latex-generate-engraved-preamble (info syntax-colours-p)

Docstring?

> +(defun org-latex-inline-src-block--engraved (info code lang)
> +  (if (require 'engrave-faces-latex nil t)
> ...
> +(message "Cannot engrave inline src block, `engrave-faces-latex' is 
> unavailible.")
> +(insert (org-latex--text-markup code 'code info

Why message instead of error?

Best,
Ihor

Re: [PATCH] New LaTeX code export option: engraved

[Daniel Fleischer]

Timothy [2022-05-04 Wed 23:59] wrote:

> I’ve been fairly busy as of late (hence my recent silence on this ML), 
> however I have a patchset that’s been in the
> works for a while that I’ve finally polished up. 

Hi Timothy, thank you very much for your work!

Let me learn a bit about the different code highlighting options in
order to understand what you offer.

Best,

-- 

Daniel Fleischer

Re: [PATCH] New LaTeX code export option: engraved

[Daniel Fleischer]

Timothy [2022-05-04 Wed 23:59] wrote:

> I’ve been fairly busy as of late (hence my recent silence on this ML), 
> however I have a patchset that’s been in the
> works for a while that I’ve finally polished up. 

Hi Timothy, thank you very much for your work!

Let me learn a bit about the different code highlighting options in
order to understand what you offer.

Best,

-- 

Daniel Fleischer

[PATCH] New LaTeX code export option: engraved

[Timothy]

Hi All,

I’ve been fairly busy as of late (hence my recent silence on this ML), however I
have a patchset that’s been in the works for a while that I’ve finally polished
up.

Short version: It adds a new (superior) option for exporting code blocks with
syntax highlighting to LaTeX.

The long version follows.

Currently there are four options for exporting code to LaTeX (ordered by
complexity/quality).
1. Verbatim, which simply includes the code unstyled
2. Custom, which puts the code in an enviroment of your choice
3. Listings, which uses the LaTeX package by this name. This is quick, but
   exceedingly basic.
4. Minted, which uses the LaTeX package by this name. This is slow, but produces
   better results than Listings

This patchset accomplishes two things:
1. It refactors the overly large `org-latex-src-block' function, and makes a few
   other improvements to pre-existing code
2. It adds a new option for exporting code, named (you guessed it!) “engraved”

What is this new option, and why do we want it?

About a year ago I started work on a package that generalises the functionality
of `htmlize.el', termed `engrave-faces'
(). It provides the ability to
extract font-lock information and export it to a number of formats: html, ansi,
and (crucially) LaTeX! Since the LaTeX export is built on the `fvextra' (LaTeX)
package (like pygments), the vast majority of the Minted options you’re used to
just carry over.

This allows for a result that is, I think, straight up better than all the
pre-existing options. For starters, you can now apply syntax highlighting to any
language you have a major mode for.

There are a number of optimisations unattempted, but it already significantly
outperforms Minted. Here are some timings from my `config.org':


 LaTeX code backend  Compile time  Overhead  Overhead ratio 

 verbatim12 s  00.0 
 lstlistings 15 s  3 s  0.2 
 Engrave 34 s  22 s 1.8 
 Pygments (Minted)   184 s 172 s   14.3 


Compared to Minted, we also no longer have to install `pygments' or pass the
`--shelll-escape' flag to LaTeX to generate passable code blocks.

Having an all-emacs system also allows for some nifty things, like having entire
documents/presentations based on your Emacs theme (see
). Demo images: 
,
.

Here are some more screenshots to see what the result can look like in practice:
• 
• 
• 

So, please take a look at the patches, give this a whirl, and let me know what
you think! 

All the best,
Timothy
>From 87872cc8f2fb1da1a03dc4aadfbd4af6541d8c13 Mon Sep 17 00:00:00 2001
From: TEC 
Date: Sun, 21 Nov 2021 14:35:34 +0800
Subject: [PATCH 1/4] ox-latex: Refactor `org-latex-src-block'

* lisp/ox-latex.el (org-latex-src-block): Extract the per-format logic
from `org-latex-src-block' into new dedicated functions:
+ `org-latex-src-block--verbatim'
+ `org-latex-src-block--custom'
+ `org-latex-src-block--minted'
+ `org-latex-src-block--listings'
This makes `org-latex-src-block' much less monolithic, taking it from
175 lines to 30, and I find also makes it easier to understand.
---
 lisp/ox-latex.el | 339 ++-
 1 file changed, 185 insertions(+), 154 deletions(-)

diff --git a/lisp/ox-latex.el b/lisp/ox-latex.el
index 841ad48bc..63855d2f6 100644
--- a/lisp/ox-latex.el
+++ b/lisp/ox-latex.el
@@ -2997,164 +2997,195 @@ (defun org-latex-src-block (src-block _contents info)
 	   (float (plist-get attributes :float))
 	   (listings (plist-get info :latex-listings)))
   (cond
-   ;; Case 1.  No source fontification.
((or (not lang) (not listings))
-	(let ((caption-str (org-latex--caption/label-string src-block info))
-  (verbatim (format "\\begin{verbatim}\n%s\\end{verbatim}"
-(org-export-format-code-default src-block info
-  (cond ((string= "multicolumn" float)
- (format "\\begin{figure*}[%s]\n%s%s\n%s\\end{figure*}"
- (plist-get info :latex-default-figure-position)
- (if caption-above-p caption-str "")
- verbatim
- (if caption-above-p "" caption-str)))
-(caption (concat
-  (if caption-above-p caption-str "")
-  verbatim
-  (if caption-above-p "" (concat "\n" caption-str
-(t verbatim
-   ;; Case 2.  Custom environment.
+(org-latex-src-block--verbatim