patch: add custom latex->html conversion command

2020-02-15 Thread Matt Huszagh
Hi all,

The patch below allow's you to provide your own shell command the
generates HTML from a latex fragment and places the output directly in
the exported HTML file.

I've added this feature to be able to use latexml for processing latex
fragments to HTML. Unfortunately, mathjax can't process macros from the
siunitx package, but latexml can. Here's how I use it

  (setq org-html-with-latex 'html)
  (setq org-latex-to-html-convert-command "latexmlc 'literal:%i' --profile=math 
--preload=siunitx.sty 2>/dev/null")

Any/all feedback appreciated!

>From 056d23d9e5caa6fc22907014e0128519fcc84b6e Mon Sep 17 00:00:00 2001
From: Matt Huszagh 
Date: Sat, 15 Feb 2020 18:42:11 -0800
Subject: [PATCH] add custom command option when converting latex fragments to
 html

This allows you to set a custom command
`org-latex-to-html-convert-command' that will take as input a latex
fragment and use it to generate html for export. This is very
open-ended in the sense that you can use any shell-command you want. I
envisioned this for use with latexml, but there's nothing preventing
you from using something else.
---
 lisp/org.el | 30 ++
 lisp/ox-html.el |  9 +++--
 2 files changed, 37 insertions(+), 2 deletions(-)

diff --git a/lisp/org.el b/lisp/org.el
index 97ce7ec43..94557bf86 100644
--- a/lisp/org.el
+++ b/lisp/org.el
@@ -3203,6 +3203,22 @@ When using LaTeXML set this option to
 	  (const :tag "None" nil)
 	  (string :tag "\nShell command")))
 
+(defcustom org-latex-to-html-convert-command nil
+  "Command to convert LaTeX fragments to HTML.
+This command is very open-ended: the output of the command will
+directly replace the latex fragment in the resulting HTML.
+Replace format-specifiers in the command as noted below and use
+`shell-command' to convert LaTeX to HTML.
+%i: The latex fragment to be converted.
+
+For example, this could be used with LaTeXML as
+\"latexmlc 'literal:%i' --profile=math --preload=siunitx.sty 2>/dev/null\"."
+  :group 'org-latex
+  :version "26.1"
+  :type '(choice
+	  (const :tag "None" nil)
+	  (string :tag "\nShell command")))
+
 (defcustom org-preview-latex-default-process 'dvipng
   "The default process to convert LaTeX fragments to image files.
 All available processes and theirs documents can be found in
@@ -15613,6 +15629,10 @@ Some of the options can be changed using the variable
 		(if (string= (match-string 0 value) "$$")
 			(insert "\\[" (substring value 2 -2) "\\]")
 		  (insert "\\(" (substring value 1 -1) "\\)"
+		 ((eq processing-type 'html)
+		  (goto-char beg)
+		  (delete-region beg end)
+		  (insert (org-format-latex-as-html value)))
 		 ((assq processing-type org-preview-latex-process-alist)
 		  ;; Process to an image.
 		  (cl-incf cnt)
@@ -15778,6 +15798,16 @@ inspection."
   ;; Failed conversion.  Return the LaTeX fragment verbatim
   latex-frag)))
 
+(defun org-format-latex-as-html (latex-frag)
+  "Convert latex to html with a custom conversion command.
+`LATEX-FRAG' is the latex fragment
+Set the custom command with `org-latex-to-html-convert-command'."
+  (let ((cmd (format-spec
+	  org-latex-to-html-convert-command
+	  `((?i . ,latex-frag)
+(message "Running %s" cmd)
+(setq shell-command-output (shell-command-to-string cmd
+
 (defun org--get-display-dpi ()
   "Get the DPI of the display.
 The function assumes that the display has the same pixel width in
diff --git a/lisp/ox-html.el b/lisp/ox-html.el
index ea6aa63c3..b5cbf4cfc 100644
--- a/lisp/ox-html.el
+++ b/lisp/ox-html.el
@@ -776,6 +776,8 @@ e.g. \"tex:mathjax\".  Allowed values are:
   `verbatim'Keep everything in verbatim
   `mathjax', t  Do MathJax preprocessing and arrange for MathJax.js to
 be loaded.
+  `html'Use `org-latex-to-html-convert-command' to convert
+LaTeX fragments to HTML.
   SYMBOLAny symbol defined in `org-preview-latex-process-alist',
 e.g., `dvipng'."
   :group 'org-export-html
@@ -2776,12 +2778,13 @@ CONTENTS is nil.  INFO is a plist holding contextual information."
 (defun org-html-format-latex (latex-frag processing-type info)
   "Format a LaTeX fragment LATEX-FRAG into HTML.
 PROCESSING-TYPE designates the tool used for conversion.  It can
-be `mathjax', `verbatim', nil, t or symbols in
+be `mathjax', `verbatim', `html', nil, t or symbols in
 `org-preview-latex-process-alist', e.g., `dvipng', `dvisvgm' or
 `imagemagick'.  See `org-html-with-latex' for more information.
 INFO is a plist containing export properties."
   (let ((cache-relpath "") (cache-dir ""))
-(unless (eq processing-type 'mathjax)
+(unless (or (eq processing-type 'mathjax)
+(eq processing-type 'html))
   (let ((bfn (or (buffer-file-name)
 		 (make-temp-name
 		  (expand-file-name "latex" temporary-file-directory
@@ -2895,6 +2898,8 @@ CONTENTS is nil.  INFO is a plist holding contextual information."
 (cond
  ((memq 

Re: Step by step tutorial on Worg on how to create a new export backend

2020-02-15 Thread Jean-Christophe Helary



> On Feb 16, 2020, at 2:55, Marcin Borkowski  wrote:
> 
> 
> On 2020-02-14, at 21:48, Bastien  wrote:
> 
>> We have a good reference documentation for creating export backends:
>> https://orgmode.org/worg/dev/org-export-reference.html
>> 
>> But we *badly* need a step by step tutorial on Worg.
>> 
>> Anyone would like to volunteer for writing such a tutorial?
> 
> I might try to at least start it, though I'll need some time.  When is
> that needed?  (I assume that the sooner, the better, so if there is
> anyone who would beat me to it, go on.  I might do some proofreading
> then.)

Marcin,

Aren't you supposed to write a book about Emacs already ? ;)


Jean-Christophe Helary
---
http://mac4translators.blogspot.com @brandelune





Re: attachment: link type export to HTML invalid attach dir

2020-02-15 Thread Kyle Meyer
Nicolas Goaziou  writes:

> In a nutshell:
>
> `org-link-parameters' accepts a new property, :open, which is
> like :follow, but function installed there is called with a universal
> prefix argument. It could replace :follow altogether, but for
> compatibility reasons, they both live side by side for the moment.
>
> Also, :export function is called with a fourth argument, the export info
> channel. I updated the export back-ends to handle the new signature, and
> provided a compatibility layers for link libraries in the wild, which
> may still use three arguments.
>
> There are two new tools: `org-link-open-as-file' and
> `org-export-link-as-file'. They can be used as helper functions for,
> respectively, :open and :export functions.
>
> WDYT?

It looks like a good direction to go.

> Subject: [PATCH 1/2] Extend export tooling in link parameters
[...]
>  (defcustom org-link-parameters nil
> -  "An alist of properties that defines all the links in Org mode.
> +  "Alist of properties that defines all the links in Org mode.
> +
[...]
> +`:export'
> +
> +  Function that accepts three mandatory arguments :

nit: extra space before colon

> +  Optionally, it may accept the export communication channel as
> +  a fourth argument, in case you need using Org Export tooling,

s/need using/need to use/ ?

> Subject: [PATCH 2/2] ol: Extend open tooling in link parameters
[...]
> -  Function that takes the link path (a string) as an argument and
> -  \"opens\" the link.
> +  Function used to follow the link, when `org-open-at-point' is
> +  command runs on it.

Delete "is"?  (Also, assuming the "is" is dropped, to my ears it'd read
better with a "the" in front of org-open-at-point.)

> [ @@ -990,37 +998,14 @@ (defun org-link-open (link  arg) ]
> -  (if (string-match "[*?{]" (file-name-nondirectory path))
> -   (dired path)
> - ;; Look into `org-link-parameters' in order to find
> - ;; a DEDICATED-FUNCTION to open file.  The function will be
> - ;; applied on raw link instead of parsed link due to the
> - ;; limitation in `org-add-link-type' ("open" function called
> - ;; with a single argument).  If no such function is found,
> - ;; fallback to `org-open-file'.
> - (let* ((option (org-element-property :search-option link))
> -(app (org-element-property :application link))
> -(dedicated-function
> - (org-link-get-parameter (if app (concat type "+" app) type)
> - :follow)))

As far as I can see (by looking at the diff and quickly testing), the
file+{sys,emacs} functionality is being removed.  That's probably
okay/intended, given that it was marked as deprecated in 9.0.  But
perhaps it's worth mentioning in ORG-NEWS that this functionality has
now been removed entirely.

> +(defun org-link-open-as-file (path arg)
> +  "Pretend PATH is a file name and open it.
> +
> +According to \"file\"-link syntax, PATH may include addition

s/addition/additional/

> +search options, separated from the file name with \"::\".
> +
> +This function is meant to be used as a possible tool for `:open'
> +property in `org-link-parameters.'"

nit: I think the period should go after the quote (regardless of the
strange preference in American English to put commas and periods within
quotes).



Re: Step by step tutorial on Worg on how to create a new export backend

2020-02-15 Thread Robert Klein
On Fri, 14 Feb 2020 21:48:02 +0100
Bastien  wrote:

> We have a good reference documentation for creating export backends:
> https://orgmode.org/worg/dev/org-export-reference.html
> 
> But we *badly* need a step by step tutorial on Worg.
> 
> Anyone would like to volunteer for writing such a tutorial?
> 

I actually began writing something about “deriving a export backend”
back in 2013.

Deriving a backend might be an ideal starting point of creating an
export backend; IIRC the main difference is, for a new backend you'll
have to provide /all/ the options whereas in a derived exporter you
could simply provide e.g. another src-block interpretation as
alternative to the html exporter.

Back then for some reason I didn't like punchagan's “org2blog” (which
is not an exporter) and wrote my own ox-blog (derived from ox-html) for
exporting into wordpress (i don't use wordpress anymore, so it's
rotting away on github...).

Best regards
Robert



Re: attachment: link type export to HTML invalid attach dir

2020-02-15 Thread Nicolas Goaziou
Hello,

Bastien  writes:

>>> Now, I'm not sure who wants to try implementing Nicolas suggestion to
>>> let ox.el expand attachments into file links: since you both know the
>>> issue better than I do, I suggest one of you can try?
>>
>> I will try to propose a patch by tomorrow evening.
>
> Thanks!
>
> If things are okay, I'll then have some time on sunday to prepare for
> the release on monday.

This is the first part of the suggested changes. These do not touch
attachment modifications, but rather improve the tooling in "ol.el".

In a nutshell:

`org-link-parameters' accepts a new property, :open, which is
like :follow, but function installed there is called with a universal
prefix argument. It could replace :follow altogether, but for
compatibility reasons, they both live side by side for the moment.

Also, :export function is called with a fourth argument, the export info
channel. I updated the export back-ends to handle the new signature, and
provided a compatibility layers for link libraries in the wild, which
may still use three arguments.

There are two new tools: `org-link-open-as-file' and
`org-export-link-as-file'. They can be used as helper functions for,
respectively, :open and :export functions.

WDYT?

Regards,

-- 
Nicolas Goaziou
>From 59172eb6c5edd5e74a6fd248e975065437bf5072 Mon Sep 17 00:00:00 2001
From: Nicolas Goaziou 
Date: Fri, 14 Feb 2020 10:00:15 +0100
Subject: [PATCH 1/2] Extend export tooling in link parameters

* lisp/ol.el (org-link-parameters): Allow a fourth "info" argument for
`:export' property.  Expound docstring.
* lisp/ox.el (org-export-custom-protocol-maybe): Accept a fourth
optional argument.
* lisp/ox-ascii.el (org-ascii--describe-links):
(org-ascii-link):
* lisp/ox-beamer.el (org-beamer-link):
* lisp/ox-html.el (org-html-link):
* lisp/ox-latex.el (org-latex-link):
* lisp/ox-man.el (org-man-link):
* lisp/ox-md.el (org-md-link):
* lisp/ox-odt.el (org-odt-link):
* lisp/ox-org.el (org-org-link):
* lisp/ox-texinfo.el (org-texinfo-link):
* contrib/lisp/ox-groff.el (org-groff-link): Provide expected fourth
argument.
* lisp/ox.el (org-export-link-as-file): New function.
* lisp/ol.el (org-link-parameters): Add reference to new function in docstring.
* testing/lisp/test-ox.el (test-org-export/link-as-file): Add tests.
(test-org-export/custom-protocol-maybe): Update tests.
---
 contrib/lisp/ox-groff.el |  2 +-
 lisp/ol.el   | 99 ++--
 lisp/ox-ascii.el |  4 +-
 lisp/ox-beamer.el|  2 +-
 lisp/ox-html.el  |  2 +-
 lisp/ox-latex.el |  2 +-
 lisp/ox-man.el   |  4 +-
 lisp/ox-md.el|  2 +-
 lisp/ox-odt.el   |  2 +-
 lisp/ox-org.el   |  4 +-
 lisp/ox-texinfo.el   |  2 +-
 lisp/ox.el   | 37 +++
 testing/lisp/test-ox.el  | 32 +++--
 13 files changed, 146 insertions(+), 48 deletions(-)

diff --git a/contrib/lisp/ox-groff.el b/contrib/lisp/ox-groff.el
index 9f7d3f11f..9f0a32432 100644
--- a/contrib/lisp/ox-groff.el
+++ b/contrib/lisp/ox-groff.el
@@ -1248,7 +1248,7 @@ INFO is a plist holding contextual information.  See
 ((string= type "file") (org-export-file-uri raw-path))
 (t raw-path
 (cond
- ((org-export-custom-protocol-maybe link desc 'groff))
+ ((org-export-custom-protocol-maybe link desc 'groff info))
  ;; Image file.
  (imagep (org-groff-link--inline-image link info))
  ;; import groff files
diff --git a/lisp/ol.el b/lisp/ol.el
index ce53b3e69..f850a5313 100644
--- a/lisp/ol.el
+++ b/lisp/ol.el
@@ -86,42 +86,93 @@
   :group 'org)
 
 (defcustom org-link-parameters nil
-  "An alist of properties that defines all the links in Org mode.
+  "Alist of properties that defines all the links in Org mode.
+
 The key in each association is a string of the link type.
-Subsequent optional elements make up a plist of link properties.
+Subsequent optional elements make up a property list for that
+type.
+
+All properties ar optional.  However, the most important ones
+are, in this order, `:follow', `:export', and `:store', described
+below.
+
+`:follow'
+
+  Function that takes the link path (a string) as an argument and
+  \"opens\" the link.
+
+`:export'
+
+  Function that accepts three mandatory arguments :
+  - the path, as a string,
+  - the description as a string, or nil,
+  - the export back-end.
+
+  Optionally, it may accept the export communication channel as
+  a fourth argument, in case you need using Org Export tooling,
+  e.g., `org-export-link-as-file'.
+
+  When nil, export for that type of link is delegated to the
+  back-end.
+
+`:store'
+
+  Function responsible for storing the link.  See the function
+  `org-store-link-functions' for a description of the expected
+  arguments.
+
+Additional properties provide more specific control over the
+link.
+
+`:activate-func'
+
+  Function to run at the end of Font Lock activation.  It must
+  accept four arguments:
+  - the 

Re: Step by step tutorial on Worg on how to create a new export backend

2020-02-15 Thread Marcin Borkowski


On 2020-02-14, at 21:48, Bastien  wrote:

> We have a good reference documentation for creating export backends:
> https://orgmode.org/worg/dev/org-export-reference.html
>
> But we *badly* need a step by step tutorial on Worg.
>
> Anyone would like to volunteer for writing such a tutorial?

I might try to at least start it, though I'll need some time.  When is
that needed?  (I assume that the sooner, the better, so if there is
anyone who would beat me to it, go on.  I might do some proofreading
then.)

Best,

--
Marcin Borkowski
http://mbork.pl



Re: ox-html: Bug or feature for export of title and meta information?

2020-02-15 Thread Nicolas Goaziou
Hello,

Bastien  writes:

> I see org-export-options-alist is not mentioned in the manual - would
> it be worth mentioning it in the "Adding Export Back-ends" section?

I don't think so. It is a constant, which may only matter when you write
an export-backend. Its value is not even particularly important since
any back-end can ignore it. Only its syntax really matters, as pointed
out in, e.g., `org-export-define-derived-backend' docstring. To sum it
up, I see it is an implementation detail.

The manual might, however, be more rigorous about what keywords each
back-end really interprets. I don't know how, and where, it could fit,
tho.

Regards,

-- 
Nicolas Goaziou



Re: ox-html: Bug or feature for export of title and meta information?

2020-02-15 Thread Nicolas Goaziou
Hello,

Jens Lechtenboerger  writes:

> the W3C Markup Validator [1] disagrees with this output from
> ox-html (which I reduced to the essential parts):

OK. I was talking about the Org syntax. I misread your initial report.

I won't comment about the generated HTML. If needed, the HTML exporter
can prevent parsing TITLE keyword.

Regards,

-- 
Nicolas Goaziou