Hi Ikumi,
However, this proposal leads to false positive for a dollar sign
following 3 (or more) consecutive backslashes; M-x texmathp says
"math-mode is on" at the end the following line:
\\\$
Could you further tune the regexp?
In addition, we have to take another perspetive into consideration;
`texmathp` is used in font-lock machinery in AUCTeX, thus every
non-trivial edition in the buffer calls `texmathp` to compute correct
syntax hilighting. For this reason, `texmathp` must be lightweight
enough not to interrupt the user editing process. If the regular
expression is too complex and it slows down `texmathp` siginificantly,
it wouldn't be a good idea to implement it into `texmathp-compile`.
(However, I'm optimistic about this aspect. I suppose we can find regexp
simple enough.)
Ikumi Keita
before going into the technical details: I should mention that I don't
actually use AUCTeX as a whole. I use texmathp.el on its own, as a
standalone library, to discriminate math from text during automated
find-and-replace operations -- I work in copy editing / journal
typesetting, and I need a reliable way to tell whether a given buffer
position is inside math mode before applying certain substitutions.
That's the context this bug report and this follow-up come from.
Thank you for catching that counterexample. I looked into it further,
and it turns out the problem is not a tuning issue but a structural one:
any regex that tries to encode backslash parity via "anchor + pairs of
backslashes" alternation is vulnerable to a leftmost-match slippage. I
verified this systematically: with the previously proposed regex, every
odd N >= 3 (3, 5, 7, ...) produces a false positive, not just N = 3. The
engine can always find a way to start the match one backslash later
inside the run, silently discarding the parity of the skipped backslash.
A regex without lookbehind cannot reliably encode "preceded by an even
number of backslashes" through alternation alone. The robust fix is to
capture the whole backslash run with a single, non-alternating
quantifier, and check its parity in Lisp:
In texmathp-compile:
texmathp-toggle-regexp
(concat "\\(\\\\*\\)"
(regexp-opt togglers t))
In texmathp, the toggle-scanning loop needs a matching change (group 1
no longer means "real toggle", it now holds the backslash run, so parity
must be checked before deciding to toggle):
(while (re-search-forward texmathp-toggle-regexp pos t)
(when (zerop (mod (length (match-string-no-properties 1)) 2))
(if (setq math-on (not math-on))
(setq sw-match (cons (match-string-no-properties 2)
(match-beginning 2)))
(setq sw-match nil))))
So this does end up touching texmathp itself, not just texmathp-compile
-- a slightly larger patch than my original proposal, but still small
and contained to that one loop. Group 2 (the toggler string itself, "$"
or "$$") and its position are completely unaffected, so texmathp-why
keeps exactly the same contents as before.
Why I'm confident this is correct rather than just "tuned better": "\\*"
is a single quantifier with no alternation, so the engine always finds
the true start of the backslash run as the leftmost match position --
there is no ambiguity to backtrack into, unlike the paired-backslash
approach. I tested it against backslash runs from N=0 up to N=101 (even
and odd), the original tabular/TikZ case, your \\\$ counterexample, $$
display math combined with backslash runs, and multi-cell tabular rows
-- all correct. I also confirmed the compiled regexp still lets
regexp-opt factor "$" and "$$" exactly as before
("\\(\\\\*\\)\\($\\$?\\)"), so display math is unaffected.
On your performance concern: I don't think this adds meaningful
overhead. "\\*" is linear with no backtracking, arguably simpler for the
regex engine to evaluate than the original "[^\\$]" character-class
alternative, and the added Lisp-side check is just a mod on the length
of a typically 0-2 character string per match. For what it's worth,
AUCTeX already uses exactly this parity-based idiom elsewhere for the
same class of problem -- TeX-escaped-p in tex.el determines whether a
character is escaped by checking whether skip-chars-backward over
backslashes returns an odd number. So this isn't a new pattern for the
codebase, just the same one applied here.
I also went through the whole AUCTeX tree (not just texmathp.el) to
check for anything else that might depend on the internals I'm changing.
texmathp-toggle-regexp itself is only ever referenced inside
texmathp.el. The only external consumers of the result are the
electric-dollar matching in tex.el (TeX-insert-dollar-action,
TeX--blink-matching-dollar) and the font-lock-beg optimization in
font-latex.el, and both only read (car texmathp-why) / (cdr
texmathp-why), i.e. group 2 and its position -- neither is touched by
this change.
For full transparency, as with my initial report: I used an AI assistant
(Claude) throughout this follow-up as well -- to analyze why the
paired-backslash regex fails structurally, to design this alternative,
and to build and run a test harness (an actual Emacs instance,
byte-compiling texmathp.el and exercising the real texmathp function
against dozens of cases, including the ones above) rather than reasoning
about the regex in the abstract. I independently reviewed the test cases
and the reasoning, and I'm satisfied the fix and the testing are sound,
but of course I'd still welcome your review, especially given your
closer familiarity with texmathp's history and edge cases.
I'd be happy to prepare a proper patch (against texmathp-compile and
texmathp) with a short changelog entry if this formulation looks
reasonable to you. Let me know if you'd like it adjusted, or if you see
a case I haven't considered.
Best,
Gabriele Nicolardi
--
SISSA Medialab
* Publishing *
*Gabriele Nicolardi*
*Production specialist*
*[email protected]*
------------------------------------------------------------------------
*Via Bonomea, 265 - 34136 Trieste, Italy - medialab.sissa.it
<http://medialab.sissa.it/en>*<http://medialab.sissa.it/en>
_______________________________________________
bug-auctex mailing list
[email protected]
https://lists.gnu.org/mailman/listinfo/bug-auctex
