Karthik Chikmagalur <[email protected]> writes:
>>> 1. The image-converter generates one image per dvi/pdf page,
>>> corresponding to one tex fragment. This means it should generate many
>>> images for a typical run over the Org buffer/section.
>>
>> I think that typical path is one image per tex fragment per process.
>
> This is the current path on main, but this is also one of the big
> problems with the current design. One of the main reasons the new LaTeX
> previews are fast (and Emacs remains responsive) is because we use a
> single process for all tex fragments.
I do not question that. My main issue is that we are reducing the
flexibility of the previews. If some people need (for whatever reason) a
very custom preview scheme, I do not want to break this possibility.
Another question is whether alternative routes can be made fast, but
requiring speed and also flexibility is a too big ask given the
existing complexity of the patch. So, I just want to make sure that we
preserve flexibility (even if slow).
> 1. tex-fragment/image mismatches and the other issues I
> listed are only potential problems for custom image importers and not
> for the "golden path" dvisvgm/dvipng processing route.
>
> 2. If you preview one fragment at a time there are no issues.
I still feel that I am missing something.
Is there a simple example of a custom entry in
`org-latex-preview-process-alist' that will cause problems?
>>> If condition 1 is not met, previews will simply not be generated for
>>> fragments 2-# in a run.
>>
>> (Please see my above comment, but I feel that I do not clearly
>> understand condition 1)
>
> Suppose you preview three tex fragments, 1-3, and that no image is
> generated for fragment 2. Then under the current design of
> --generic-callback,
>
> 1. Image 1 is previewed over fragment 1
> 2. Image 3 is previewed over fragment 2
> 3. No preview for fragment 3
>
> So one preview is wrong, and one preview is missing.
Isn't is easy to fix this? If we require the image names to contain
their number, can't you know from the image name which fragment any
given image belongs to? Then,
(cl-loop
for fragment-info in (plist-get extended-info :fragments)
for image-file in images
can be rewritten to match image-file with appropriate fragment without
running into the problem you described.
>> Do I understand correctly that you need the converter to process
>> multiple fragments in a single invocation? If yes, can that be
>> approximated by something like
>> (org-async-chain
>> "convert fragment1.tex fragment1.png"
>> "echo fragment1.png"
>> "convert fragment2.tex fragment2.png"
>> "echo fragment2.png"
>> ...)
>> ?
>
> Yes, but org-latex-preview is not set up to do this. This is also a
> very slow way to do it, as slow as the current system on main (which is
> borderline unusable for more than a few fragments at once).
It is ok for it to be slow. What I want to avoid is existing custom
setups to be irreversibly broken.
>>> I thought we disabled the aliasing of org-preview-latex-process-alist
>>> to org-latex-preview-process-alist for this reason. Users have to
>>> manually edit the latter to break things now.
>>
>> That's true. But look at the docstring of
>> org-latex-preview-process-alist.
>> If the customizationas are much more limited, we will need to adjust
>> that docstring accordingly.
>> We may even have to completely re-design it, as the old design seemingly
>> does not reflect the new preview system.
>> But the decision about this depends on the discussion about whether we
>> can handle arbitrary processors or not.
>
> I can edit the docstring, but it's not clear what the problem is.
> Everything in it is still correct.
>
> The requirement that the image converter should produce one image/PDF
> page (lexicographically ordered) is missing, but then many other
> pre-existing assumptions are not documented either.
>
> The logic of the process chain used to generate previews in encoded only
> in the source code. It is not clear at all to a user reading the
> documentation of both org-preview-latex-process-alist and
> org-latex-preview-process-alist (and without reading the source code
> where these are used) how to modify these options to change the preview
> image generation process. This is one of the reasons I wanted to turn
> org-latex-preview-process-alist from a defcustom into a defvar.
>From the previous discussion, I got an impression that some entries in
`org-latex-preview-process-alist' that fit the docstring may no longer
work. If that's indeed that case, it should either be fixed or the
limitations should be documented. What am I missing?
>>>> I think the way org-compile-file works now is that output of all the
>>>> command in the chain are dumped into the same buffer. It is handy when
>>>> processes throw warnings/errors - one can see problems from all commands
>>>> in the chain.
>>>
>>> This will break all our process filters, whose logic depend on text
>>> parsing of the incomplete process stdout.
>>
>> Maybe, but keep in mind that org-compile-file is a part of org-macs and
>> is not intended to be used only by the latex preview.
>> If you need split output for latex preview, we should make
>> org-compile-file optionally do it, but only optionally.
>
> We are not talking about org-compile-file, though. org-async-call is
> new and free to implement the behavior that makes the most sense for it,
> doesn't it?
Right. I think my second reply is irrelevant. I somehow thought that
we are indeed talking about org-compile-file.
> If you think it should be an option to dump all the process output in
> the same buffer when using (org-async-chain proc1 ...), I can implement
> it but it's not clear how to specify this in the function call. Maybe
>
> (org-async-call (org-async-chain proc1 proc2 ...) ...)
>
> to capture all the process output in the provided :buffer, and
>
> (org-async-call (org-async-chain-1 proc1 proc2 ...) ...)
>
> to capture only the output of the last process?
Not mandatory.
>> In the future, if we implement inlining for all the images, not just
>> for latex fragments, we will need a new global option for that, and
>> the change herein will create compatibility headaches.
>
> IIUC,
>
> 1. In the future, there will be a variable for inlining data in HTML
> exports. Let's call it 'org-html-inline`.
>
> 2. It will be possible to specify specific image types that should be
> inlined, including LaTeX preview images, via this variable. We would
> like something like
>
> (push '("file" . (".jpg" ".png")) org-html-inline) ;inline jpg/png files
> (push 'dvipng org-html-inline) ;inline dvipng LaTeX
> images
>
> 3. When this is implemented, Org will also have to respect the :inline
> field in the proposed org-latex-html-image-options, which will cause
> compatibility issues.
Yes, that's exactly my concern.
> I can see three ways around this:
>
> 1. Include the :inline field in org-latex-html-image-options and handle
> the compatibility issues when they arise.
>
> 2. Design the org-html-inline feature right now, supporting only
> inlining LaTeX preview images, with support for inlining images from
> other sources to be implemented later.
>
> 3. Don't include :inline support for LaTeX preview images with this
> patchset.
>
> I don't want to do 3, because inlining/embedding LaTeX fragment images
> is already fully implemented and it is extremely useful for generating
> portable HTML pages with lots of maths in a JS-free way (no mathjax).
>
> If you have ideas for the structure of 2, I can implement an early
> version of that and integrate it with the LaTeX preview image inlining
> feature. Then I can remove :inline from org-latex-html-image-options.
>
> Or we can do 1 and worry about compatibility later. It's going to be a
> while until Org 10.0, so we could switch to the new inlining option
> system without compatibility issues before then.
>
> Let me know which one you would prefer. Again, I don't want to leave it
> out of the patchset, it is one of the marquee features.
I think that (2) should be the best. Maybe you can factor out the
relevant logic from `org-html-latex-image' into
`org-html--format-image'.
>> Also, :inline clashes with `org-html-inline-image-rules' where "inline"
>> has entirely different meaning.
>
> This sense of "inlining" should have been called something else, like
> org-html-link-images (meaning not-inlined), but I guess it's too late
> now.
The concept of inline images goes beyond html export. It is general
concept that is used across all the exporters, and even in the image
preview.
I think that "embed" should be a better word to describe when image data
is embedded into the exported document directly.
> org-latex-html-image-options already distinguishes between inlining
> (base64-encoded image data directly in the HTML file) and embedding
> (svg-only, svg contents added as part of DOM). If you have a better
> suggestion for these actions, let me know.
Please check out previous discussion about this feature in
https://list.orgmode.org/orgmode/[email protected]/
There are possible pitfalls.
> Remaining issues:
>
> 1. "%O" handling to minimize breakage.
> 2. process output handling when executing an org-async-chain. (Retain
> all process stdout in one buffer?)
> 3. caption support/fuzzy links to captions for LaTeX fragments in
> ox-html
> 4. ox-odt feedback
> 5. Final cleanup tasks before merging
> 6. Global image inlining in ox-html (non-actionable?)
> 7. `make repro' produces warnings
> 8. Insertion tracking is over-eager, leading to spurious LaTeX fragment
> generation in programmatic editing (multiple insert calls in one
> function)
> 9. Possible breakage re: org-html-format-latex mentioned in
> https://list.orgmode.org/87qzqwbtkw.fsf@localhost
--
Ihor Radchenko // yantar92,
Org mode maintainer,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>
