Re: Unescaped content in HTML attributes in EPUB output of texi2any

[Patrice Dumas]

On Sun, Sep 22, 2024 at 05:27:34PM -0400, Benjamin Kalish wrote:
> It looks like the problem occurs only with the use of raw HTML, directly
> (as in the minimal example here), or indirectly through a macro (as I first
> encountered it):

I can reproduce with your example (in inline_in_chap.texi file), with
USE_NODES set to 0 (as is done for epub):

$srcdir/texi2any.pl --html -c 'USE_NODES 0' inline_in_chap.texi

I am not sure that this is a bug, though, looks like a feature to me, as
@inlineraw turns off escaping of HTML characters.  The situation is not
ideal, because there is no way to specify something different for
attributes (called 'string' context in texi2any) such as 
name="description" content attribute, and in the main output, for
instance in the chapter heading .  When Texinfo code is used, the
formatting to HTML can be different in 'string' context and in normal
context, but I can't see how this could be specified for @inlineraw raw
HTML.

Maybe we could say something in the documentation, for instance that
HTML elements should not be used in @inlineraw in @node or sectioning
commands?

> \input texinfo
> 
> @node Top
> @top
> 
> @node Cap 1
> @chapter @inlineraw{html,}One@inlineraw{html,}
> 
> @bye
> 
> Benjamin Kalish
> 
> 
> On Sun, Sep 22, 2024 at 4:47 PM Gavin Smith 
> wrote:
> 
> > On Sun, Sep 22, 2024 at 02:20:36PM -0400, Benjamin Kalish wrote:
> > > EPUB output contains unescaped content in a number of HTML attributes.
> > I'm
> > > seeing this with:
> > >
> > > - The content attribute for  with name="description"
> > > - The content attribute for  name="keywords"
> > > - The title attribute of the  elements with rel="next" and
> > rel="prev"
> > >
> > > HTML output also has these same tags and attributes, but the content
> > seems
> > > fine in my case. This may not actually be due to better escaping, as it
> > > looks like entirely different content is being used for the attribute
> > > values when generating HTML, and the content is, in this case at least,
> > > safe without escaping.
> > >
> > > Changing the values to be the same as those used when generating HTML
> > would
> > > solve the problem in my case, but it is probably best to make sure that
> > > attribute values are always escaped.
> > >
> > > What should be escaped? Quotation marks must be. Ambiguous ampersands
> > must
> > > be. But it is probably prudent to escape all ampersands and all
> > > occurrences of < or >.
> > >
> > > I'm sorry I can't suggest a fix in the code—I'm not familiar with the
> > > Texinfo codebase and it's been decades since I've coded in Perl or C.
> > >
> > > I'm using texi2any 7.1.1
> >
> > I tried testing this on the master development branch and it looked
> > ok:
> >
> > $ cat test.texi
> > \input texinfo
> >
> > @node Top
> > @top
> >
> > @node Cap 1
> > @chapter One "<>
> >
> > @bye
> >
> > After running "texi2any --epub3 test.texi" and extracting the
> > resulting "test.epub" file, the output file in the ZIP archive had, in
> > "test/EPUB/xhtml/Cap-1.xhtml", the " < and > escaped (see output below).
> > Can you please explain how to reproduce the problem?
> >
> >
> > 
> > 
> > http://www.w3.org/1999/xhtml";>
> > 
> > 
> > 
> > 1 One "<> (Untitled Document)
> >
> > 
> > 
> > 
> > 
> > 
> > 
> >
> > 
> > 
> > 
> > 
> >
> >
> > 
> >
> > 
> > 
> >
> > 1 One "<>
> >
> >
> >
> > 
> >
> >
> >
> > 
> > 
> >

Re: Rounded cartouche corners look bad in HTML output

[Patrice Dumas]

On Sun, Sep 22, 2024 at 01:25:11PM +0100, Gavin Smith wrote:
> It would be better not to have rounded corners at all, as with the
> old output "cartouche-borders.png".

The new relative border indeed looks wrong but the previous output was
not very good either.

> Alternatively, using an absolute value, such as "border-radius: 10px"
> produces a better result: see "rounded-pixels.png".  "border-radius: 0.5em"
> may be better as this should depend on the font size: see "rounded-em.png".

border-radius: 0.5em
seems to be the best to me, I'll do the change.

-- 
Pat

Re: multiple failures for info tests on cygwin

[Patrice Dumas]

On Sat, Sep 21, 2024 at 05:54:15PM -0400, Ken Brown wrote:
> On 9/21/2024 8:07 AM, Gavin Smith wrote:
> > On Sat, Sep 21, 2024 at 12:10:33PM +0200, Patrice Dumas wrote:
> > > This platform does not test the native Windows Info reader, it is the
> > > cygwin info reader.  So these tests fail in a 'pure' Cygwin setup, there
> > > is no mingw here, the comparison with mingw was in order to explain why
> > > the tests are skipped with mingw (on MSYS or on Cygwin) and not on
> > > 'pure' Cygwin.
> > 
> > It's hard for me to know what the problem is without building on Cygwin
> > myself, which I am not able or willing to do at the current time.  The
> > log files you sent appear to show that the pseudotty program is built
> > and used in the tests, but all the tests time out.
> I'm Cygwin's Texinfo maintainer, and I have two comments about this thread.
> 
> 1. According to the OP, these failures occurred on 32-bit Cygwin.  This
> platform has been unsupported by the Cygwin project for about 3 years. In
> particular, Cygwin will never provide recent Texinfo packages for the 32-bit
> case, so I don't think the Texinfo maintainers should waste time worrying
> about this.

Ok, thanks for the information.

> 2. Cygwin provides its own build service (based on GitHub actions) for use
> by package maintainers.  When I've used this with recent versions of
> Texinfo, I have found that many tests fail; but they all pass when I run
> them locally.  I've never been able to figure out what's different about the
> GitHub actions setup that causes these failures, but I haven't worried about
> it since all tests pass on my system.

Good to hear.  I'll probably try to skip the info tests on Cygwin 64 and
see if the texi2any testsuites pass.

-- 
Pat

Re: do formatting_cpp_lines tests fail with native mingw?

[Patrice Dumas]

On Sat, Sep 21, 2024 at 02:29:13PM +0300, Eli Zaretskii wrote:
> > Date: Sat, 21 Sep 2024 12:44:38 +0200
> > From: Patrice Dumas 
> > 
> I didn't have time to build anything after Texinfo 7.1 (neither 7.1.1
> nor the current release/7.1 branch).  In 7.1, the
> split_nocopying_split_dev_null test did not fail for me, so if it
> haven't changed since then, and if Texinfo didn't change anything that
> is related to what the test tests, it should not fail now.

You are right, there is no reason for this file to have changed, the
release/7.1 branch changes should be bugfixes only.

Thanks for your answer.


The MinGW on Cygwin with native Perl platform shows more test failures
for the master branch.  Given that the failures are different from MinGW
on MSYS, and given all the issues of the MinGW on Cygwin with native
Perl platform my current stance is not to try to setup specific checks
to skip tests for that platform and let the test fail.  Although overall
texi2any on that platform should give correct results for most real
world manuals, at least for now I do not think that it is worth doing
something specific for tests.

-- 
Pat

FreeBSD failures with env for in document translations for master

[Patrice Dumas]

Hello,

The CI for the master branch for FreeBSD shows errors related to in
document translations in the XS/C code.  These errors seem to trigger
some kind of memory corruption, with an errno code that is not supposed
to be issued by setenv 'Bad address' and those errors seem to propagate
across processes:

perl: environment corrupt; missing value for
sh: environment corrupt; missing value for 

...
perl: environment corrupt; missing value for
translate_string: setenv `ja' error for string `{name} of {class}': Bad address

It is unclear to me whether the issue here is in Texinfo code (in that
case probably in tp/Texinfo/XS/main/translations.c translate_string) or
in FreeBSD.

I attach the full tp/t*.t log.

-- 
Pat

   GNU Texinfo 7.1.90-20240921: tp/test-suite.log


# TOTAL: 22866
# PASS:  22834
# SKIP:  25
# XFAIL: 0
# FAIL:  6
# XPASS: 0
# ERROR: 1

.. contents:: :depth: 2

PASS: t/automatic_nodes
===

1..27
ok 1
PASS: t/automatic_nodes.t 1
ok 2 # skip test perl not XS
SKIP: t/automatic_nodes.t 2 # SKIP test perl not XS
ok 3 # skip test perl not XS
SKIP: t/automatic_nodes.t 3 # SKIP test perl not XS
ok 4 # skip test perl not XS
SKIP: t/automatic_nodes.t 4 # SKIP test perl not XS
ok 5 # skip test perl not XS
SKIP: t/automatic_nodes.t 5 # SKIP test perl not XS
ok 6 # skip test perl not XS
SKIP: t/automatic_nodes.t 6 # SKIP test perl not XS
ok 7 # skip test perl not XS
SKIP: t/automatic_nodes.t 7 # SKIP test perl not XS
ok 8 # skip test perl not XS
SKIP: t/automatic_nodes.t 8 # SKIP test perl not XS
ok 9 # skip test perl not XS
SKIP: t/automatic_nodes.t 9 # SKIP test perl not XS
ok 10 # skip test perl not XS
SKIP: t/automatic_nodes.t 10 # SKIP test perl not XS
ok 11 # skip test perl not XS
SKIP: t/automatic_nodes.t 11 # SKIP test perl not XS
ok 12 # skip test perl not XS
SKIP: t/automatic_nodes.t 12 # SKIP test perl not XS
ok 13 # skip test perl not XS
SKIP: t/automatic_nodes.t 13 # SKIP test perl not XS
ok 14 # skip test perl not XS
SKIP: t/automatic_nodes.t 14 # SKIP test perl not XS
ok 15 # skip test perl not XS
SKIP: t/automatic_nodes.t 15 # SKIP test perl not XS
ok 16 # skip test perl not XS
SKIP: t/automatic_nodes.t 16 # SKIP test perl not XS
ok 17 # skip test perl not XS
SKIP: t/automatic_nodes.t 17 # SKIP test perl not XS
ok 18 # skip test perl not XS
SKIP: t/automatic_nodes.t 18 # SKIP test perl not XS
ok 19 # skip test perl not XS
SKIP: t/automatic_nodes.t 19 # SKIP test perl not XS
ok 20 # skip test perl not XS
SKIP: t/automatic_nodes.t 20 # SKIP test perl not XS
ok 21 # skip test perl not XS
SKIP: t/automatic_nodes.t 21 # SKIP test perl not XS
ok 22 # skip test perl not XS
SKIP: t/automatic_nodes.t 22 # SKIP test perl not XS
ok 23 # skip test perl not XS
SKIP: t/automatic_nodes.t 23 # SKIP test perl not XS
ok 24 - add nodes
PASS: t/automatic_nodes.t 24 - add nodes
ok 25 - new node has a menu
PASS: t/automatic_nodes.t 25 - new node has a menu
ok 26 - reassociated menu is correct
PASS: t/automatic_nodes.t 26 - reassociated menu is correct
ok 27 - index entry reassociated
PASS: t/automatic_nodes.t 27 - index entry reassociated

ERROR: t/languages
==

ok 1
PASS: t/languages.t 1
ok 2 - simple_documentlanguage tree
PASS: t/languages.t 2 - simple_documentlanguage tree
ok 3 - simple_documentlanguage sectioning
PASS: t/languages.t 3 - simple_documentlanguage sectioning
ok 4 - simple_documentlanguage nodes
PASS: t/languages.t 4 - simple_documentlanguage nodes
ok 5 - simple_documentlanguage menus
PASS: t/languages.t 5 - simple_documentlanguage menus
ok 6 - simple_documentlanguage floats
PASS: t/languages.t 6 - simple_documentlanguage floats
ok 7 - simple_documentlanguage errors
PASS: t/languages.t 7 - simple_documentlanguage errors
ok 8 - simple_documentlanguage indices
PASS: t/languages.t 8 - simple_documentlanguage indices
ok 9 - simple_documentlanguage indices sort
PASS: t/languages.t 9 - simple_documentlanguage indices sort
ok 10 - simple_documentlanguage texi
PASS: t/languages.t 10 - simple_documentlanguage texi
ok 11 - simple_documentlanguage text
PASS: t/languages.t 11 - simple_documentlanguage text
ok 12 - simple_documentlanguage converted plaintext
PASS: t/languages.t 12 - simple_documentlanguage converted plaintext
ok 13 - simple_documentlanguage errors plaintext
PASS: t/languages.t 13 - simple_documentlanguage errors plaintext
ok 14 # skip no_documentlanguage_before_copying: incorrect result
SKIP: t/languages.t 14 # SKIP no_documentlanguage_before_copying: incorrect 
result
ok 15 - multiple_documentlanguage tree
PASS: t/languages.t 15 - multiple_documentlanguage tree
ok 16 - multiple_documentlanguage sectioning
PASS: t/languages.t 16 - multiple_documentlanguage sectioning
ok 17 - multiple_documentlanguage nodes
PASS: t/languages.t 17 - multiple_documentlanguage nodes
ok 18 - multiple_documentlanguage menus
PASS: t/languages.t 18 - multiple_documentlanguage 

Re: do formatting_cpp_lines tests fail with native mingw?

[Patrice Dumas]

Hello,

The test with /dev/null failure is explained by MinGW on MSYS mapping to
NUL (as explained by Eli in another thread).

Could you please, Eli, check whether the other test,
split_nocopying_split_dev_null in formatting fails or not with mingw on
MSYS, if possible with the release/7.1 branch?

On Fri, Sep 20, 2024 at 06:56:03PM +0200, Patrice Dumas wrote:
> Hello,
> 
> I am looking at mingw (on cygwin) GNU Texinfo 7.1 tests failures using
> Bruno CI
> https://github.com/gnu-texinfo/ci-check
> 
> formatting_cpp_lines
> with a diff in results
> -accentêd:7: warning: lãng is not a valid language code
> +accentêd:7: warning: lãng is not a valid language code
> 
> For formatting_cpp_lines my guess is that the decoding of accentêd as
> latin1 instead of UTF-8 is because the detected codepage locale is used
> for accentêd, while the document encoding (UTF-8) is used for the
> content of the document if I understood well.
> 
> I can skip the tests if the platform is Windows or with some other
> condition, but before I do that, it would be nice if people being able
> to test native Windows builds could check if they get the same failures,
> as I do not want to skip tests that actually work on some platforms
> variations.
> 
> This should be achieved by running, in tp/tests:
> 
> ./test_scripts/formatting_cpp_lines.sh
> 
> Could people having access to native Windows mingw build try to run
> those tests and report if they get the same errors, please?
> 
> -- 
> Pat

Re: finding prove with mingw on cygwin?

[Patrice Dumas]

On Fri, Sep 20, 2024 at 10:05:57PM +0300, Eli Zaretskii wrote:
> > The shell is not MSYS, it is:
> > shell: C:\cygwin\bin\bash.exe -eo pipefail -o igncr '{0}'
> 
> Using the Cygwin tools to build MinGW ports is an uncharted territory
> for me: I never did that.  MSYS was created (as a fork of Cygwin)
> precisely for this purpose: allow building MinGW ports using native
> Windows development tools (MinGW GCC and Binutils, native Windows
> Perl, etc.), and it includes several special tricks up its sleeve to
> that end.  Examples include transparent conversion of /d/foo/bar file
> names to the Windows D:\foo\bar form, conversion of foo:bar:baz in
> PATH to Windows-style foo;bar;baz, EOL conversion where possible,
> conversion of /dev/null to NUL etc.

This last part about /dev/null probably explains why the test with
/dev/null does not fail in your setup, but fails in mingw on Cygwin.

> By contrast, Cygwin has a very different purpose.
> 
> I think the correct way of building MinGW ports using Cygwin Bash and
> development tools is to use Cygwin-to-MinGW cross-compiler (I think
> the Cygwin project provides such cross-tools).  Using native Windows
> executables (including MinGW GCC and Binutils, and other MinGW ports)
> is bound to cause problems such as this one.

That is what this CI platfrom does.  But there is no Perl provided that
way.

> And I have no idea how to invoke Strawberry Perl from Cygwin in a way
> that will avoid these problems.  I guess you could try removing CR
> characters, but that could bite you in some other cases, where a lone
> CR character is produced and is supposed to be kept.  If you are
> lucky, this will never happen in Texinfo, but only if you are lucky.

Most of the tests that call texi2any directly pass, except one with
/dev/null and another with an issue with a mix of encodings, so it is
not that bad.  I will continue to try to have the XS modules built, I am
curious to see if the tests still pass in that case.

-- 
Pat

Re: finding prove with mingw on cygwin?

[Patrice Dumas]

On Sat, Sep 21, 2024 at 10:57:33AM +0300, Eli Zaretskii wrote:
> > Date: Sat, 21 Sep 2024 00:32:43 +0200
> > From: Patrice Dumas 
> > 
> > On Fri, Sep 20, 2024 at 08:14:33PM +0200, Patrice Dumas wrote:
> > > On Fri, Sep 20, 2024 at 08:42:18PM +0300, Eli Zaretskii wrote:
> > > >  And if not, does the
> > > > Strawberry Perl's bin/ subdirectory appear on your PATH?
> > > 
> > > I think so, there is:
> > > Path:   C:\cygwin\bin
> > >   ..
> > > C:\Strawberry\c\bin
> > > C:\Strawberry\perl\site\bin
> > > C:\Strawberry\perl\bin
> > >   ..
> > > 
> > > I will check what a ls on those directories gives, to check if prove is
> > > actually there.
> > 
> > I have checked, there is indeed prove.bat in
> > /cygdrive/c/Strawberry/perl/bin
> > 
> > Yet it is not found.  Could be because of the .bat, and it could make
> > sense, as it is not clear to me that cygwin bash.exe can interpret .bat
> > files.
> 
> I'm now confused as to what port of Texinfo are you building and
> testing. 

In this thread, the port is mingw on Cygwin. Cygwin is used as a pseudo host.
In
https://www.gnu.org/software/libtool/manual/html_node/Cygwin-to-MinGW-Cross.html
it is referred to as 'fake' Cygwin to MinGW cross compile case:
In the fake Cygwin to MinGW cross compile case, you actually use a
native MinGW compiler, but you do so from within a Cygwin environment.

In this Cygwin to MinGW cross compile case, we use Strawberry Perl
as a native Perl, which should mix well with MinGW compiled binaries
at the binary level.  (Though we have all sort of issues because we call
native builds in Cygwin as you noted in another mail).

> If it's a Cygwin build of Texinfo, then Strawberry Perl
> should not be used; you should use a Cygwin port of Perl instead.  And
> 'prove' should not be a batch file in a Cygwin Perl, it should be a
> Perl script named just "prove", as it is on any Unix system.

On the one hand it should be, to be called from Cygwin, on the other
hand since we are compiling for mingw, we use the native perl...

> Indeed, I don't believe Cygwin supports batch file execution, except
> via the likes of "cmd /c foo.bat".

At this point, I think that it is not worth it trying to have the tp/*.t
testsuite work for that platform, some perl modules will be needed and I
do not know how to install them.  For this platform, we probably should
stick to what works easily as it is 'wobbly'.

-- 
Pat

Re: multiple failures for info tests on cygwin

[Patrice Dumas]

On Sat, Sep 21, 2024 at 10:32:07AM +0300, Eli Zaretskii wrote:
> > Date: Fri, 20 Sep 2024 23:43:16 +0200
> > From: Patrice Dumas 
> > Cc: bug-texinfo@gnu.org
> > 
> > On Fri, Sep 20, 2024 at 10:21:01PM +0300, Eli Zaretskii wrote:
> > > > Date: Fri, 20 Sep 2024 20:43:45 +0200
> > > > From: Patrice Dumas 
> > > > 
> > > > Many info tests on cygwin-32 fail (release 7.1 branch).  I attach the
> > > > info tests log, I can provide more information on configure output, and
> > > > some other logs if needed.
> > > > 
> > > > =
> > > >GNU Texinfo 7.1.1-20240920: info/test-suite.log
> > > > =
> > > > 
> > > > # TOTAL: 87
> > > > # PASS:  51
> > > > # SKIP:  0
> > > > # XFAIL: 0
> > > > # FAIL:  36
> > > > # XPASS: 0
> > > > # ERROR: 0
> > > 
> > > That's most probably the side effect of using Cygwin tools again: the
> > > native port of info cannot run many of the tests because the test rig
> > > uses features not supported by native Windows programs (emulation of
> > > terminals and other such stuff).  When running the tests with MSYS,
> > > the test suite detects that and skips those tests, but since you run
> > > them with Cygwin, I'm guessing that the way the test suite detects
> > > Windows ports fails for some reason.
> > > 
> > > My records from running the test suite in Texinfo-7.1 indicate that 56
> > > of the info tests were skipped, whereas above you say that none were
> > > skipped.  So I'm quite sure this is the reason.
> > 
> > Actually, my feeling is that it is the absence of posix_openpt that
> > triggers have_ptys to be false, which in turn causes pseudotty not to
> > be built.  In the CI tests, cygwin have posix_openpt, while mingw (in
> > cygwin) does not.
> 
> It's that, and the fact that the tests use fifos, which native Windows
> programs cannot support, either.  And maybe more.
> 
> Whatever PTY Cygwin can produce, the native Windows Info reader is
> extremely unlikely to be able to use.

This platform does not test the native Windows Info reader, it is the
cygwin info reader.  So these tests fail in a 'pure' Cygwin setup, there
is no mingw here, the comparison with mingw was in order to explain why
the tests are skipped with mingw (on MSYS or on Cygwin) and not on
'pure' Cygwin.

-- 
Pat

Re: multiple failures for info tests on cygwin

[Patrice Dumas]

On Sat, Sep 21, 2024 at 10:53:55AM +0300, Eli Zaretskii wrote:
> > Date: Sat, 21 Sep 2024 00:16:45 +0200
> > From: Patrice Dumas 
> > 
> > On Fri, Sep 20, 2024 at 11:00:22PM +0100, Gavin Smith wrote:
> > > On Fri, Sep 20, 2024 at 11:43:16PM +0200, Patrice Dumas wrote:
> > > > On Fri, Sep 20, 2024 at 10:21:01PM +0300, Eli Zaretskii wrote:
> > > > > > Date: Fri, 20 Sep 2024 20:43:45 +0200
> > > > > > From: Patrice Dumas 
> > > > > > 
> > > > > > Many info tests on cygwin-32 fail (release 7.1 branch).  I attach 
> > > > > > the
> > > > > > info tests log, I can provide more information on configure output, 
> > > > > > and
> > > > > > some other logs if needed.
> > > > > > 
> > > > > > =
> > > > > >GNU Texinfo 7.1.1-20240920: info/test-suite.log
> > > > > > =
> > > > > > 
> > > > > > # TOTAL: 87
> > > > > > # PASS:  51
> > > > > > # SKIP:  0
> > > > > > # XFAIL: 0
> > > > > > # FAIL:  36
> > > > > > # XPASS: 0
> > > > > > # ERROR: 0
> > > > > 
> > > > > That's most probably the side effect of using Cygwin tools again: the
> > > > > native port of info cannot run many of the tests because the test rig
> > > > > uses features not supported by native Windows programs (emulation of
> > > > > terminals and other such stuff).  When running the tests with MSYS,
> > > > > the test suite detects that and skips those tests, but since you run
> > > > > them with Cygwin, I'm guessing that the way the test suite detects
> > > > > Windows ports fails for some reason.
> > > > > 
> > > > > My records from running the test suite in Texinfo-7.1 indicate that 56
> > > > > of the info tests were skipped, whereas above you say that none were
> > > > > skipped.  So I'm quite sure this is the reason.
> > > > 
> > > > Actually, my feeling is that it is the absence of posix_openpt that
> > > > triggers have_ptys to be false, which in turn causes pseudotty not to
> > > > be built.  In the CI tests, cygwin have posix_openpt, while mingw (in
> > > > cygwin) does not.
> > > > 
> > > > Gavin, maybe you could have a look?
> > > 
> > > My first question is what exactly "mingw in cygwin" means.  If this is
> > > some mixture of mingw (i.e. a native MS-Windows environment) and cygwin
> > > then it may not be a setup worth supporting, due to subtle 
> > > incompatibilities
> > > between the two, as has been recently discussed.
> > > 
> > > If it is Texinfo built on cygwin to run in cygwin, as a cygwin program
> > > (not a native MS-Windows program), that is different and may be more worth
> > > supporting.
> > 
> > This case is indeed Texinfo built on cygwin to run in cygwin, as a cygwin
> > program.
> 
> ??? Then why are you using Strawberry Perl?  That's not a Cygwin build
> of Perl.  The Cygwin Perl should not output CR-LF end-of-line format,
> it should output the Unix newline-only EOLs.

This is in another thread, here this is the Cygwin build of Perl, and it
does not output CR-LF end-of-lines, this is another issue.

> Cygwin builds should normally need no patching and fixing, since
> Cygwin's purpose is to provide a faithful emulation of a Posix system.
> 
> Your original message about this build of Texinfo said:
> 
> > I would like to have more tests for mingw (on cygwin) using Bruno CI
> > https://github.com/gnu-texinfo/ci-check
> > 
> > However, 'prove' is not found by configure, by
> > AC_CHECK_PROGS([PROVE], [prove], [])
> > while perl is found with
> > AC_PATH_PROG([PERL], [perl])
> > 
> > I checked the binary zip provided for strawberry perl, which seems to be
> > the Perl used in that case, and there is a prove.bat file provided.
> > 
> > Do testers with mingw and strawberry Perl have prove found by configure?
> > Any idea on what should be done?
> 
> This seems to say you are building and testing the MinGW port of
> Texinfo, not a Cygwin port.  If it's a Cygwin port, why is MinGW
> mentioned and why are Strawberry Perl and the Windows C:\foo\bar
> format of file names relevant? none of that are supported by Cygwin,
> AFAIK.

Again this is another thread and another CI platform...

-- 
Pat

Re: XDG dirs for texi2any

[Patrice Dumas]

On Fri, Sep 20, 2024 at 09:45:09PM +0100, Gavin Smith wrote:
> On Thu, Aug 29, 2024 at 09:11:20AM +0200, Patrice Dumas wrote:
> > I do not think that this is the right way to look at this.  To me the
> > logic is that we search both in "config" locations, as it is where the
> > system/user may place an htmlxref.cnf files, and we search there first,
> > and in "data", which is where the default from texi2any is installed.
> 
> It may be too complicated and confusing to look in both.  Do any other
> programs look for the same file in both "config" and "data" locations?

I think that I have seen that in discussions, but more importantly, it
makes sense and it is our practice.

> The current draft of the documentation lists 13 different locations
> that are checked which is a bit overwhelming.

Indeed, I agree that it is too much.  I propose to drop the hardcoded
paths, leading to (after the locations in *HOME):

XDG_CONFIG_DIRS/texinfo
$sysconfdir/texinfo/ or $sysconfdir/xdg/texinfo/
XDG_DATA_DIRS/texinfo
$datadir/texinfo/

We won't follow exactly the XDG base directory standard, but it is
better, in my opinion to be consistent with installation paths.

If this is ok, there remains to decide
$sysconfdir/texinfo/ or $sysconfdir/xdg/texinfo/

I have no real preference.

-- 
Pat

Re: finding prove with mingw on cygwin?

[Patrice Dumas]

On Fri, Sep 20, 2024 at 08:14:33PM +0200, Patrice Dumas wrote:
> On Fri, Sep 20, 2024 at 08:42:18PM +0300, Eli Zaretskii wrote:
> >  And if not, does the
> > Strawberry Perl's bin/ subdirectory appear on your PATH?
> 
> I think so, there is:
> Path:   C:\cygwin\bin
>   ..
> C:\Strawberry\c\bin
> C:\Strawberry\perl\site\bin
> C:\Strawberry\perl\bin
>   ..
> 
> I will check what a ls on those directories gives, to check if prove is
> actually there.

I have checked, there is indeed prove.bat in
/cygdrive/c/Strawberry/perl/bin

Yet it is not found.  Could be because of the .bat, and it could make
sense, as it is not clear to me that cygwin bash.exe can interpret .bat
files.

-- 
Pat

Re: multiple failures for info tests on cygwin

[Patrice Dumas]

On Fri, Sep 20, 2024 at 11:00:22PM +0100, Gavin Smith wrote:
> On Fri, Sep 20, 2024 at 11:43:16PM +0200, Patrice Dumas wrote:
> > On Fri, Sep 20, 2024 at 10:21:01PM +0300, Eli Zaretskii wrote:
> > > > Date: Fri, 20 Sep 2024 20:43:45 +0200
> > > > From: Patrice Dumas 
> > > > 
> > > > Many info tests on cygwin-32 fail (release 7.1 branch).  I attach the
> > > > info tests log, I can provide more information on configure output, and
> > > > some other logs if needed.
> > > > 
> > > > =
> > > >GNU Texinfo 7.1.1-20240920: info/test-suite.log
> > > > =
> > > > 
> > > > # TOTAL: 87
> > > > # PASS:  51
> > > > # SKIP:  0
> > > > # XFAIL: 0
> > > > # FAIL:  36
> > > > # XPASS: 0
> > > > # ERROR: 0
> > > 
> > > That's most probably the side effect of using Cygwin tools again: the
> > > native port of info cannot run many of the tests because the test rig
> > > uses features not supported by native Windows programs (emulation of
> > > terminals and other such stuff).  When running the tests with MSYS,
> > > the test suite detects that and skips those tests, but since you run
> > > them with Cygwin, I'm guessing that the way the test suite detects
> > > Windows ports fails for some reason.
> > > 
> > > My records from running the test suite in Texinfo-7.1 indicate that 56
> > > of the info tests were skipped, whereas above you say that none were
> > > skipped.  So I'm quite sure this is the reason.
> > 
> > Actually, my feeling is that it is the absence of posix_openpt that
> > triggers have_ptys to be false, which in turn causes pseudotty not to
> > be built.  In the CI tests, cygwin have posix_openpt, while mingw (in
> > cygwin) does not.
> > 
> > Gavin, maybe you could have a look?
> 
> My first question is what exactly "mingw in cygwin" means.  If this is
> some mixture of mingw (i.e. a native MS-Windows environment) and cygwin
> then it may not be a setup worth supporting, due to subtle incompatibilities
> between the two, as has been recently discussed.
> 
> If it is Texinfo built on cygwin to run in cygwin, as a cygwin program
> (not a native MS-Windows program), that is different and may be more worth
> supporting.

This case is indeed Texinfo built on cygwin to run in cygwin, as a cygwin
program.

-- 
Pat

Re: multiple failures for info tests on cygwin

[Patrice Dumas]

On Fri, Sep 20, 2024 at 10:21:01PM +0300, Eli Zaretskii wrote:
> > Date: Fri, 20 Sep 2024 20:43:45 +0200
> > From: Patrice Dumas 
> > 
> > Many info tests on cygwin-32 fail (release 7.1 branch).  I attach the
> > info tests log, I can provide more information on configure output, and
> > some other logs if needed.
> > 
> > =
> >GNU Texinfo 7.1.1-20240920: info/test-suite.log
> > =
> > 
> > # TOTAL: 87
> > # PASS:  51
> > # SKIP:  0
> > # XFAIL: 0
> > # FAIL:  36
> > # XPASS: 0
> > # ERROR: 0
> 
> That's most probably the side effect of using Cygwin tools again: the
> native port of info cannot run many of the tests because the test rig
> uses features not supported by native Windows programs (emulation of
> terminals and other such stuff).  When running the tests with MSYS,
> the test suite detects that and skips those tests, but since you run
> them with Cygwin, I'm guessing that the way the test suite detects
> Windows ports fails for some reason.
> 
> My records from running the test suite in Texinfo-7.1 indicate that 56
> of the info tests were skipped, whereas above you say that none were
> skipped.  So I'm quite sure this is the reason.

Actually, my feeling is that it is the absence of posix_openpt that
triggers have_ptys to be false, which in turn causes pseudotty not to
be built.  In the CI tests, cygwin have posix_openpt, while mingw (in
cygwin) does not.

Gavin, maybe you could have a look?

-- 
Pat

Re: SOURCE_DATE_EPOCH for dcterms:modified?

[Patrice Dumas]

On Fri, Sep 20, 2024 at 10:53:12PM +0200, Patrice Dumas wrote:
> If we use the current date, the dcterms:modified, in theory, should
> never be set in the past of the current date for the same publication
> identifier.  The current specification
> https://www.w3.org/TR/epub-33/#sec-metadata-last-modified
> refers to the previous specification that explains the difference between
> the Unique Identifier, which does not change with each minor revision
> and the Release Identifier
> https://www.w3.org/publishing/epub32/epub-packages.html#sec-metadata-elem-identifiers-pid
> It is said, in particular "The sequential, chronological order inherent
> in the format of the timestamp also places EPUB Publications in order
> without requiring knowledge of the exact identifier that came before."
> The Release Identifier became the dcterms:modified metadata.

That being said, after rereading the specification, I realize that I may
have mistaken the last modification date dcterms:modified with the
dc:date element, which is the publication date, and is optional.  Having
the last modification date that is too recent may not be such an issue.
If it was the publication date, it would have been more problematic.

-- 
Pat

Re: SOURCE_DATE_EPOCH for dcterms:modified?

[Patrice Dumas]

On Fri, Sep 20, 2024 at 09:09:33PM +0100, Gavin Smith wrote:
> 
> SOURCE_DATE_EPOCH would give the user a chance to override the value
> and is a standard variable that users might be familiar with, unlike
> any other variable or option that we might invent.

To me SOURCE_DATE_EPOCH is for reproducible builds and should not be
used for publication dates, this is very different.

> I really don't see what is wrong with using the current date for
> dcterms:modified by default.  Hardly anyone is going to notice or
> care that it does not correspond to the last date that the manual
> sources were modified.

If we use the current date, the dcterms:modified, in theory, should
never be set in the past of the current date for the same publication
identifier.  The current specification
https://www.w3.org/TR/epub-33/#sec-metadata-last-modified
refers to the previous specification that explains the difference between
the Unique Identifier, which does not change with each minor revision
and the Release Identifier
https://www.w3.org/publishing/epub32/epub-packages.html#sec-metadata-elem-identifiers-pid
It is said, in particular "The sequential, chronological order inherent
in the format of the timestamp also places EPUB Publications in order
without requiring knowledge of the exact identifier that came before."
The Release Identifier became the dcterms:modified metadata.

I wanted to avoid such an irreversible setting automatically.  That's
why I preferred the automatic setting to be conditional to having the
EPUB_STRICT customization variable set, until dcterms:modified/the
publication date can be set explicitely.

> However, people will notice and have noticed EPUB standard-noncompliance
> and epubcheck failures.

With EPUB_STRICT, dcterms:modified is automatically set, it should allow
to have compliance without forcing to set a dcterms:modified value
automatically.

-- 
Pat

Re: multiple failures for info tests on cygwin

[Patrice Dumas]

On Fri, Sep 20, 2024 at 10:21:01PM +0300, Eli Zaretskii wrote:
> > Date: Fri, 20 Sep 2024 20:43:45 +0200
> > From: Patrice Dumas 
> > 
> That's most probably the side effect of using Cygwin tools again: the
> native port of info cannot run many of the tests because the test rig
> uses features not supported by native Windows programs (emulation of
> terminals and other such stuff).  When running the tests with MSYS,
> the test suite detects that and skips those tests, but since you run
> them with Cygwin, I'm guessing that the way the test suite detects
> Windows ports fails for some reason.
> 
> My records from running the test suite in Texinfo-7.1 indicate that 56
> of the info tests were skipped, whereas above you say that none were
> skipped.  So I'm quite sure this is the reason.

Ok, thanks.  I'll try to find a way to have the info tests similarily
skipped for cygwin and MSYS/mingw.  I am pretty sure that it is already
done that way in some places in the code where both mingw and cygwin are
treated the same.

-- 
Pat

multiple failures for info tests on cygwin

[Patrice Dumas]

Hello,

Many info tests on cygwin-32 fail (release 7.1 branch).  I attach the
info tests log, I can provide more information on configure output, and
some other logs if needed.

-- 
Pat
=
   GNU Texinfo 7.1.1-20240920: info/test-suite.log
=

# TOTAL: 87
# PASS:  51
# SKIP:  0
# XFAIL: 0
# FAIL:  36
# XPASS: 0
# ERROR: 0

.. contents:: :depth: 2

FAIL: t/dir-entry-to-subdir.sh
==

++ ginfo='./ginfo --init-file ../../info/t/Infokey-config'
++ INFOPATH=../../info/t/infodir
++ export INFOPATH
++ t=../../info/t
++ export t
+++ basename ../../info/t/dir-entry-to-subdir.sh.out
++ ginfo_output=t/dir-entry-to-subdir.sh.out
+++ basename ../../info/t/dir-entry-to-subdir.sh.pipein
++ pipein=t/dir-entry-to-subdir.sh.pipein
+++ basename ../../info/t/dir-entry-to-subdir.sh.pipeout
++ pty_type=t/dir-entry-to-subdir.sh.pipeout
++ rm -f t/dir-entry-to-subdir.sh.out
++ INFO_PRINT_COMMAND='>t/dir-entry-to-subdir.sh.out'
++ export INFO_PRINT_COMMAND
++ pty_pid=0
++ LC_ALL=C
++ export LC_ALL
++ TERM=dumb
++ export TERM
++ trap cleanup 2
++ path_sep=:
++ initialization_done=0
++ true 10
+ run_ginfo subdir
+ init_interactive_test
+ test 0 -eq 1
+ initialization_done=1
+ test -x pseudotty
+ TERM=vt100
+ export TERM
+ rm -f t/dir-entry-to-subdir.sh.pipein t/dir-entry-to-subdir.sh.pipeout
+ findprog mkfifo
+ saveIFS=' 
'
+ IFS=:
+ for dir in $PATH
+ IFS=' 
'
+ test -f /usr/bin/mkfifo
+ test -x /usr/bin/mkfifo
+ return 0
+ mkfifo t/dir-entry-to-subdir.sh.pipein
+ mkfifo t/dir-entry-to-subdir.sh.pipeout
+ test '!' -r t/dir-entry-to-subdir.sh.pipein
+ test '!' -r t/dir-entry-to-subdir.sh.pipeout
+ pty_pid=11034
+ read pts_device
+ exec ./pseudotty t/dir-entry-to-subdir.sh.pipeout
./pseudotty: getting pty master fd
./pseudotty: unlocking slave device
./pseudotty: getting file name of slave device...
./pseudotty: /dev/pty0
./pseudotty: opening slave device
./pseudotty: attempting to set window size
./pseudotty: ...succeeded
./pseudotty: opening control channel
+ exec
./pseudotty: entering main loop
+ LIBC_FATAL_STDERR_=1
+ export LIBC_FATAL_STDERR
+ ginfo_pid=11035
+ exec
+ exec ./ginfo --init-file ../../info/t/Infokey-config subdir
+ killer_pid=11036
+ printf Dq
+ sleep 10
+ timeout_test
./pseudotty: trying to read
+ wait 11035
./pseudotty: read byte 0x44
./pseudotty: trying to read
./pseudotty: read byte 0x71
info: "../../info/t/Infokey-config", line 2: unknown action `xxx-not-recognized'
info: "../../info/t/Infokey-config", line 3: cannot bind key sequence to 
menu-digit
+ status=0
+ test 0 -lt 128
+ kill 11036
+ retval=0
+ test 0 '!=' 0
+ ginfo_pid=0
+ cat t/dir-entry-to-subdir.sh.out
cat: t/dir-entry-to-subdir.sh.out: No such file or directory
../../info/t/dir-entry-to-subdir.sh: line 25: 11036 Terminated  ( 
sleep $TEXINFO_INFO_TESTS_TIMEOUT; kill $ginfo_pid )
+ grep '^File: file-in-subdir.info' t/dir-entry-to-subdir.sh.out
grep: t/dir-entry-to-subdir.sh.out: No such file or directory
+ retval=2
+ cleanup
+ test 11034 -ne 0
+ kill 11034
+ rm -f t/dir-entry-to-subdir.sh.out
../../info/t/Init-test.inc: line 58: 11034 Terminated  ( exec 
./pseudotty "$pty_type" > $pipein )
+ rm -f t/dir-entry-to-subdir.sh.pipein t/dir-entry-to-subdir.sh.pipeout
+ exit 2
FAIL t/dir-entry-to-subdir.sh (exit status: 2)

FAIL: t/all-only.sh
===

++ ginfo='./ginfo --init-file ../../info/t/Infokey-config'
++ INFOPATH=../../info/t/infodir
++ export INFOPATH
++ t=../../info/t
++ export t
+++ basename ../../info/t/all-only.sh.out
++ ginfo_output=t/all-only.sh.out
+++ basename ../../info/t/all-only.sh.pipein
++ pipein=t/all-only.sh.pipein
+++ basename ../../info/t/all-only.sh.pipeout
++ pty_type=t/all-only.sh.pipeout
++ rm -f t/all-only.sh.out
++ INFO_PRINT_COMMAND='>t/all-only.sh.out'
++ export INFO_PRINT_COMMAND
++ pty_pid=0
++ LC_ALL=C
++ export LC_ALL
++ TERM=dumb
++ export TERM
++ trap cleanup 2
++ path_sep=:
++ initialization_done=0
++ true 10
+ MANPATH=.
+ export MANPATH
+ run_ginfo --all intera
+ init_interactive_test
+ test 0 -eq 1
+ initialization_done=1
+ test -x pseudotty
+ TERM=vt100
+ export TERM
+ rm -f t/all-only.sh.pipein t/all-only.sh.pipeout
+ findprog mkfifo
+ saveIFS=' 
'
+ IFS=:
+ for dir in $PATH
+ IFS=' 
'
+ test -f /usr/bin/mkfifo
+ test -x /usr/bin/mkfifo
+ return 0
+ mkfifo t/all-only.sh.pipein
+ mkfifo t/all-only.sh.pipeout
+ test '!' -r t/all-only.sh.pipein
+ test '!' -r t/all-only.sh.pipeout
+ pty_pid=5
+ read pts_device
+ exec ./pseudotty t/all-only.sh.pipeout
./pseudotty: getting pty master fd
./pseudotty: unlocking slave device
./pseudotty: getting file name of slave device...
./pseudotty: /dev/pty0
./pseudotty: opening slave device
./pseudotty: attempting to set window size
./pseudotty: ...succeeded
./pseudotty: opening control channel
+ exec
./pseudotty: entering main loop
+ LIBC_FATAL_STDERR_=1
+ export LIBC_FATAL_STDERR
+ ginfo_

Re: safe to remove carriage returns in configure for mingw?

[Patrice Dumas]

On Fri, Sep 20, 2024 at 08:54:09PM +0300, Eli Zaretskii wrote:
> > Date: Fri, 20 Sep 2024 19:29:14 +0200
> > From: Patrice Dumas 
> > 
> > Is this issue also happening with native builds?  Would it be safe to
> > always remove carriage returns?
> 
> What do you mean by "native builds"?  A MinGW build is a native
> Windows build, so I'm confused about the above dichotomy.

What the CI uses is mingw in Cygwin build.  If I understood well, this
is what is called a "fake" Cygwin to MinGW cross compile case in
https://www.gnu.org/software/libtool/manual/html_node/Cygwin-to-MinGW-Cross.html

So it is, in a way less native that mingw on MSYS, although it is also
native in a way...

> FWIW, I don't see any such problems in Texinfo 7.0.94: there are no ^M
> characters in tp/Texinfo/XS/config.log.  I don't know why.  Maybe
> `...` removes the ^M characters in my version of Bash?  Hmm...
> 
> Yes, looks that way:
> 
>   $ /d/usr/Perl/bin/perl -V:cc > foo
> 
> produces 'foo' with CRLF end-of-line, but
> 
>   $ echo `/d/usr/Perl/bin/perl -V:cc` > foo
> 
> produces 'foo' with Unix newlines.
> 
> Doesn't this happen with the Bash you are using?

That could be the difference.  The shell is
C:\cygwin\bin\bash.exe -eo pipefail -o igncr '{0}'

I would have thought that -o igncr would have meant that CR are removed,
though.

In any case, if the CR is removed for you too, removing it explicitely
with sed should make things right, so this is something I can do anyway,
thanks.  I will add a comment explaining in which case it seems to be
useful.

-- 
Pat

Re: finding prove with mingw on cygwin?

[Patrice Dumas]

On Fri, Sep 20, 2024 at 08:42:18PM +0300, Eli Zaretskii wrote:
> > Date: Fri, 20 Sep 2024 19:14:46 +0200
> > From: Patrice Dumas 
> > 
> > Hello,
> > 
> > I would like to have more tests for mingw (on cygwin) using Bruno CI
> > https://github.com/gnu-texinfo/ci-check
> > 
> > However, 'prove' is not found by configure, by
> > AC_CHECK_PROGS([PROVE], [prove], [])
> > while perl is found with
> > AC_PATH_PROG([PERL], [perl])
> > 
> > I checked the binary zip provided for strawberry perl, which seems to be
> > the Perl used in that case, and there is a prove.bat file provided.
> > 
> > Do testers with mingw and strawberry Perl have prove found by configure?
> 
> On my system, I get
> 
>   $ which prove
>   /bin/prove
> 
> in the MSYS Bash.  So it does find 'prove'.
> 
> > Any idea on what should be done?
> 
> Do you have the file in your MSYS installation?

The shell is not MSYS, it is:
shell: C:\cygwin\bin\bash.exe -eo pipefail -o igncr '{0}'

and the Cygwin Perl is not installed on purpose such that Strawberry
Perl is used.

>  And if not, does the
> Strawberry Perl's bin/ subdirectory appear on your PATH?

I think so, there is:
Path:   C:\cygwin\bin
  ..
C:\Strawberry\c\bin
C:\Strawberry\perl\site\bin
C:\Strawberry\perl\bin
  ..

Perl is found there by configure:

checking for perl... /cygdrive/c/Strawberry/perl/bin/perl


I will check what a ls on those directories gives, to check if prove is
actually there.

-- 
Pat

safe to remove carriage returns in configure for mingw?

[Patrice Dumas]

Hello,

An issue with XS modules configure for mingw (on cygwin) using Bruno CI
https://github.com/gnu-texinfo/ci-check

In tp/Texinfo/XS/configure.ac, we use perl -V:var to get information on
Perl configuration for the var.  The perl output in that case has ^M
carriage returns at end of lines.  This prevents the configure script to
work correctly.  We parse perl -V:var output using a sed script.  It
would be easy to add
  s/\r//
to remove carriage returns.  However, I fear that it could break some
native builds that expect carriage returns at end of lines.

The symptom of this issue are line like that in tp/Texinfo/XS/configure
run output, while previous lines do not have a ^M:
checking Perl configuration value cc... gcc';^M

Is this issue also happening with native builds?  Would it be safe to
always remove carriage returns?

-- 
Pat

finding prove with mingw on cygwin?

[Patrice Dumas]

Hello,

I would like to have more tests for mingw (on cygwin) using Bruno CI
https://github.com/gnu-texinfo/ci-check

However, 'prove' is not found by configure, by
AC_CHECK_PROGS([PROVE], [prove], [])
while perl is found with
AC_PATH_PROG([PERL], [perl])

I checked the binary zip provided for strawberry perl, which seems to be
the Perl used in that case, and there is a prove.bat file provided.

Do testers with mingw and strawberry Perl have prove found by configure?
Any idea on what should be done?

-- 
Pat

do formatting_split_nocopying_split_dev_null and formatting_cpp_lines tests fail with native mingw?

[Patrice Dumas]

Hello,

I am looking at mingw (on cygwin) GNU Texinfo 7.1 tests failures using
Bruno CI
https://github.com/gnu-texinfo/ci-check

There are two tests that fail in tp/tests:

split_nocopying_split_dev_null
with 
texi2any: rename /dev/null failed: No such file or directory

and
formatting_cpp_lines
with a diff in results
-accentêd:7: warning: lãng is not a valid language code
+accentêd:7: warning: lãng is not a valid language code

The reasons for those failures can be guessed, for
split_nocopying_split_dev_null, my guess would be that there is no
/dev/null with mingw, but NUL.

For formatting_cpp_lines my guess is that the decoding of accentêd as
latin1 instead of UTF-8 is because the detected codepage locale is used
for accentêd, while the document encoding (UTF-8) is used for the
content of the document if I understood well.

I can skip the tests if the platform is Windows or with some other
condition, but before I do that, it would be nice if people being able
to test native Windows builds could check if they get the same failures,
as I do not want to skip tests that actually work on some platforms
variations.

This should be achieved by running, in tp/tests:

./test_scripts/formatting_split_nocopying_split_dev_null.sh

and

./test_scripts/formatting_cpp_lines.sh

Could people having access to native Windows mingw build try to run
those tests and report if they get the same errors, please?

-- 
Pat

Re: Texinfo 7.1.0.91 pretest results

[Patrice Dumas]

On Wed, Sep 18, 2024 at 11:46:47PM +0200, Patrice Dumas wrote:
> # Use the programs built in our distribution.  Use absolute directory
> # names so we can run from a different working directory.
> MAKEINFO_ENVIRONMENT = TEXINFO_DEV_SOURCE=1 \
>top_srcdir="$(abs_top_srcdir)" \
>top_builddir="$(abs_top_builddir)"
> 
> Then, I think that abs_top_builddir comes from configure substitution in
> Makefile.in, maybe from pwd.
> 
> I do not really understand why abs_top_srcdir is used here and not
> top_srcdir, I did not really understand what the comment is about.
> Maybe using top_srcdir instead of abs_top_srcdir and top_builddir
> instead of abs_top_builddir could fix that issue?

In the end I added a new option to pod2texi to be able to specify
independently the destination of the included files and the @include in
the main Texinfo file.

-- 
Pat

Re: Texinfo 7.1.0.91 pretest results

[Patrice Dumas]

On Wed, Sep 18, 2024 at 11:46:47PM +0200, Patrice Dumas wrote:
> 
> I do not really understand why abs_top_srcdir is used here and not
> top_srcdir, I did not really understand what the comment is about.
> Maybe using top_srcdir instead of abs_top_srcdir and top_builddir
> instead of abs_top_builddir could fix that issue?

I had a look at ChangeLog and mails, Gavin used the abs directories for
a good reason, as there is a cd $srcdir before calling texi2any.  This
issue cannot be solved that way.

-- 
Pat

Re: Texinfo 7.1.0.91 pretest results

[Patrice Dumas]

On Mon, Sep 02, 2024 at 08:02:26PM +0200, Bruno Haible wrote:
> Hi Gavin,
> 
> > Could it be an issue with backslashes being used as directory separators
> > in \cygdrive\d\a\ci-check\ci-check ... ?  It is strange that the
> > path is passed with forward slashes on the command line in the environment
> > variables top_builddir but is given with backslashes in the error.  Is
> > there any automatic conversion of backslashes on this platform that
> > could be responsible?
> 
> This looks like it is the problem, yes.
> 
> Texinfo/ModulePath.pm is meant to reside in build/tp/.
> 
> The line texi2any.pl:84 is meant to add this directory to @INC.

I had a look at the build log and there is
if TEXINFO_DEV_SOURCE=1 
top_srcdir="/cygdrive/d/a/ci-check/ci-check/texinfo-7.1.1-20240916/build/.." 
top_builddir="/cygdrive/d/a/ci-check/ci-check/texinfo-7.1.1-20240916/build" 
/cygdrive/c/Strawberry/perl/bin/perl ../../../tp/texi2any.pl -c 
INFO_SPECIAL_CHARS_WARNING=0 -c TREE_TRANSFORMATIONS=complete_tree_nodes_menus  
-I ../../../doc/tp_api \
 -o texi2any_internals.info `test -f 'texi2any_internals.texi' || echo 
'../../../doc/tp_api/'`texi2any_internals.texi; \

Therefore the top_builddir
is /cygdrive/d/a/ci-check/ci-check/texinfo-7.1.1-20240916/build
and is used l 81 of texi2any.pl:
  unshift @INC, File::Spec->catdir($ENV{'top_builddir'}, 'tp');

> We can see that it the value that it adds is
>   \cygdrive\d\a\ci-check\ci-check\texinfo-7.1.0.91-20240901\build\tp
> but since this is a native Windows perl, it should really be
>   D:\a\ci-check\ci-check\texinfo-7.1.0.91-20240901\build\tp
> (with slashes or backslahes, doesn't matter).

I think that it comes from doc/tp_api/Makefile.am

# Use the programs built in our distribution.  Use absolute directory
# names so we can run from a different working directory.
MAKEINFO_ENVIRONMENT = TEXINFO_DEV_SOURCE=1 \
   top_srcdir="$(abs_top_srcdir)" \
   top_builddir="$(abs_top_builddir)"

Then, I think that abs_top_builddir comes from configure substitution in
Makefile.in, maybe from pwd.

I do not really understand why abs_top_srcdir is used here and not
top_srcdir, I did not really understand what the comment is about.
Maybe using top_srcdir instead of abs_top_srcdir and top_builddir
instead of abs_top_builddir could fix that issue?

-- 
Pat

HTML @multitable table cell width as attribute or inline style?

[Patrice Dumas]

Hello,

Right now the @multitable HTML output table td or th elements width is
set with the width= attribute.  This attribute is deprecated in favor of
using CSS, but to me this is highly debatable, as the width may be part
of the structure of the table as opposed to styling, more similar to the
colspan attribute.  I hoped that this deprecation would be reverted as
has been the case for ol type (for which both CSS and attribute can be
used), but this has not happened for now.

We could change to use inline CSS style for width.  We have avoided
using inline CSS style in texi2any for now, all the styling is set
through classes, as should be.

I can imagine that text browsers that try to render tables could use the
width attribute but ignore inline CSS, but I have no evidence.

I think that we have 4 options
1) stay as we are now with width= attribute
2) change to inline style width
3) keep width= attribute but add a customization variable to switch
   to inline style width
4) change the default to inline style width and add a customization
   variable to switch to attribute width=

Do you have an idea on what would be best?

-- 
Pat

Re: tightening mathjax tex options

[Patrice Dumas]

On Fri, Sep 06, 2024 at 12:28:48PM +0200, Patrice Dumas wrote:
> Hello,
> 
> I propose to set the following in mathjax defaults, to avoid possible
> interactions with text (although it may not be possible anyway because
> of ignoreHtmlClass set in body), and perhaps simplifying the task of mathjax:
> 
>   tex: {
> processEscapes: false,  // use \$ to produce a literal dollar sign
> processEnvironments: false, // process \begin{xxx}...\end{xxx} outside 
> math mode
> processRefs: false, // process \ref{...} outside of math mode
> displayMath: [ // start/end delimiter pairs for display math
>   ['\\[', '\\]']
> ],
>   }

It is commited.  If it is problematic in some situation, do not hesitate
to report.

-- 
Pat

Re: EPUB 3.3 spec conformity issues

[Patrice Dumas]

On Sun, Sep 08, 2024 at 09:14:15PM +0100, Gavin Smith wrote:
> > It is not set in the default case... If we do not document it, I can't
> > see how it would be used (outside of
> > maintain/check_epubcheck_texi_t.sh)...
> 
> Oh, I thought it would be set in epub3.pm for EPUB output.

Here is the commit with the change in dicumentation, please check and
revise...

https://git.savannah.gnu.org/cgit/texinfo.git/commit/?id=b38857002b173c69bab1baea007b59f80997eae6

-- 
Pat

Re: EPUB 3.3 spec conformity issues

[Patrice Dumas]

On Sun, Sep 08, 2024 at 09:14:15PM +0100, Gavin Smith wrote:
> On Sun, Sep 08, 2024 at 09:38:13PM +0200, Patrice Dumas wrote:
> > On Sun, Sep 08, 2024 at 08:07:41PM +0100, Gavin Smith wrote:
> > > On Sun, Sep 08, 2024 at 06:46:39PM +0200, Patrice Dumas wrote:
> > > > Hello,
> > > > 
> > > > I have added the EPUB_STRICT customization variable, it does 2 things
> > > > 1) sets dcterms:modified
> > > > 2) set an internal customization variable such that td width is not
> > > >set with an attribute but with inline CSS
> > > > 
> > > > Should I document it?
> > > 
> > > THanks for doing this.  I don't think it needs to be documented but would
> > > needed added as an exception in doc/refcard/txivarcheck.
> > 
> > It is not set in the default case... If we do not document it, I can't
> > see how it would be used (outside of
> > maintain/check_epubcheck_texi_t.sh)...
> 
> Oh, I thought it would be set in epub3.pm for EPUB output.

I think that it would be wrong for the dcterms:modified setting.  To me,
we should only set dcterms:modified automatically if the user has not
already set the value.  As long as we do not provide a way to set
dcterms:modified, I think that we should not set dcterms:modified by
default and automatically.  Setting dcterms:modified automatically when
EPUB_STRICT is set is fine, though.

The other effect, on td width could be done automatically, but then we
would need make the 'internal' customization variable not internal and
document it such that it may be modified. 

-- 
Pat

Re: EPUB 3.3 spec conformity issues

[Patrice Dumas]

On Sun, Sep 08, 2024 at 08:07:41PM +0100, Gavin Smith wrote:
> On Sun, Sep 08, 2024 at 06:46:39PM +0200, Patrice Dumas wrote:
> > Hello,
> > 
> > I have added the EPUB_STRICT customization variable, it does 2 things
> > 1) sets dcterms:modified
> > 2) set an internal customization variable such that td width is not
> >set with an attribute but with inline CSS
> > 
> > Should I document it?
> 
> THanks for doing this.  I don't think it needs to be documented but would
> needed added as an exception in doc/refcard/txivarcheck.

It is not set in the default case... If we do not document it, I can't
see how it would be used (outside of
maintain/check_epubcheck_texi_t.sh)...

> >   -  in  for nested @-commands formatted with 
> 
> This is certainly a bug in texi2any's output but one that may not be
> easy to fix and only occurs for rare input that the user can change
> themselves.

I do not think that it is a bug in texi2any's output.  It is simply that
Texinfo allow for nested references in some cases, while HTML never
allows that.  Since the browser render nested  as we would expect to
match Texinfo semantics, I see no point in doing something different in
texi2any.  We could document that nested Texinfo @-commands output as
 will lead to invalid HTML, to help users change their input, but to
me that's it.

-- 
Pat

Re: EPUB 3.3 spec conformity issues

[Patrice Dumas]

Hello,

I have added the EPUB_STRICT customization variable, it does 2 things
1) sets dcterms:modified
2) set an internal customization variable such that td width is not
   set with an attribute but with inline CSS

Should I document it?


I set testing of tp/t/*.t Texinfo code, in tp:
./maintain/check_epubcheck_texi_t.sh

Even though when generating EPUB, NO_TOP_NODE_OUTPUT is set, it is unset
in check_epubcheck_texi_t.sh because it leads to many false positives.
Because of that I could have missed some true errors.  If anybody feels
like removing the option and checking all the errors, that would be nice,
but I do not feel motivated to do it.

I have fixed some bugs, and main epubcheck errors and warnings.  To me
the current state is good.

There were few warnings to begin with, the warnings that remain (there
is also one because of an empty @section that we need not care about)
are about spaces in file names, to me this is up to the user to avoid
spaces in filenames.

Here are the main errors that remain:
* false positives (for instance when we have incorrect HTML generated
  for the test, wrong @U, missing image files...)

* errors we can not do much about
  - reference to a manual without htmlxref.cnf entry
  - missing nav for document without sectioning commands
  -  in  for nested @-commands formatted with 

* errors for unusual/wrong Texinfo constructs
  - multiple @printindex for a given index
  - multiple @insertcopying with anchors/footnotes
  - empty sections

* some errors that I think should be handled by the user who cares
  -  formatting error for @table and similar when there is content
before the first @item or missing content for 
  - Form feeds remaining in output:
Character reference "" is an invalid XML character

Tell me if something else should be done.

-- 
Pat

Re: removing border=0 for tables as it is the default

[Patrice Dumas]

On Fri, Sep 06, 2024 at 04:16:04PM +0100, Gavin Smith wrote:
> ( which we were disucssing has the same effect as
> .)

Actually, it is  that epubcheck does not like, it only
accepts  or .  So, this solves the
issue of EPUB conformance.  We may want to use CSS to replace the uses
of  in the future, though.  It requires a bit more
code, if I understood well, as  is replaced by CSS for
the table, th and td.

-- 
Pat

Re: tightening mathjax tex options

[Patrice Dumas]

On Sat, Sep 07, 2024 at 07:58:47AM +0200, Patrice Dumas wrote:
> > Oh, you’re right. I think I know why. Someone wants to use an really ancient
> > version of texinfo (5.x) even though the official build says it should be at
> > least 6.8. I tried to make it a requirement, but that didn’t work.

I had a look at the maxima source code, and indeed, you can either use
texi2any mathjax support or do it yourself, with m4 substitutions.  For
some reason, in configure.ac, 6.8 is included in the platforms where you
set MathJax by yourself:

   dnl If the makeinfo version is older than or equal to 6.8, we
   dnl need to include MathJax ourselves when building the HTML
   dnl files.
   need_mathjax=false
   if test ${makeinfo_version} -le 608; then
   need_mathjax=true
   fi

So starting with 7.0, your build system is set such that to use the
regular tex2any MathJax support.  In that case, the m4 code generates
@math and @displaymath.

I actually checked a build with texinfo 7.1 and it looks fine and the
'regular' MathJax support is used.  I also tested adding the tex options
block I propose and saw no visible difference.

So, for your use, the new options should work.

-- 
Pat

Re: tightening mathjax tex options

[Patrice Dumas]

On Fri, Sep 06, 2024 at 07:31:27PM -0700, Raymond Toy wrote:
> On 9/6/24 2:36 PM, Patrice Dumas wrote:
> 
> > On Fri, Sep 06, 2024 at 02:16:56PM -0700, Raymond Toy wrote:
> > > On 9/6/24 9:29 AM, Patrice Dumas wrote:
> > > > The change in settings may only affect LaTeX rendered by MathJax outside
> > > > of @math and @displaymath, using different structures as what is done in
> > > > texi2any.  This would be possible if you directly ouput HTML like
> > > Ok, Maxima’s manual doesn’t do that. All math should be done my @math or
> > > @displaymath. At least that’s how it’s supposed to be in the manual.
> > But this means that (according to my reading of the HTML) you change the
> > output somehow to be directly
> > $
> > ...
> > $
> Oh, you’re right. I think I know why. Someone wants to use an really ancient
> version of texinfo (5.x) even though the official build says it should be at
> least 6.8. I tried to make it a requirement, but that didn’t work.

If you use 5.x, indeed, you have to add MathJax support by yourself.

> > for displaymath, without the expected , and use $ instead of
> > \[.  Similarly, @math does not have .  The MathJax block in the
> > header is also missing, as well as the tex2jax_ignore in .
> Does it matter that it was built with texinfo 6.8? (I think.) I haven’t
> updated the site to use a newer version of texinfo, but I’d like to.

No, the MathJax support was added in 6.8 and has not changed much until
now.

-- 
Pat

Re: tightening mathjax tex options

[Patrice Dumas]

On Fri, Sep 06, 2024 at 02:16:56PM -0700, Raymond Toy wrote:
> On 9/6/24 9:29 AM, Patrice Dumas wrote:
> > The change in settings may only affect LaTeX rendered by MathJax outside
> > of @math and @displaymath, using different structures as what is done in
> > texi2any.  This would be possible if you directly ouput HTML like
> Ok, Maxima’s manual doesn’t do that. All math should be done my @math or
> @displaymath. At least that’s how it’s supposed to be in the manual.

But this means that (according to my reading of the HTML) you change the
output somehow to be directly
$$
...
$$
for displaymath, without the expected , and use $$ instead of
\[.  Similarly, @math does not have .  The MathJax block in the
header is also missing, as well as the tex2jax_ignore in .

> > Do you have a page which shows some math that are rendered by MathJax?
> Sure. See this section on Bessel functions 
> <https://maxima.common-lisp.dev/docs/Special-Functions.html#Bessel-Functions>.
> There are lots of examples in the Special Functions section.

Thanks.

-- 
Pat

Re: texi2any export to html with static LaTeX

[Patrice Dumas]

On Fri, Sep 06, 2024 at 10:12:08PM +0100, Jeremy Bryant wrote:
> Patrice Dumas  writes:
> 
> > On Thu, Sep 05, 2024 at 09:49:02PM +0100, Jeremy Bryant wrote:
> >> Hello,
> >> 
> >> Version: texi2any (GNU texinfo) 7.1
> >> 
> >> This appears to depend on latex2html which appears not to be maintained.
> >> Is there a fix?
> >
> > You could try this patch and apply to $datadir/texinfo/ext/latex2html.pm
> 
> Thanks for the patch, it works locally on my machine.
> 
> Perhaps you could submit it upstream to latex2html?

There is no need for that, as it is not a change in latex2html, but a
change in how we call it from texi2any, we are upstream.  Maybe it could
be possible to report that calling latex2html with -dir . fails,
however.

-- 
Pat

Re: Build report for texinfo-7.1.0.91

[Patrice Dumas]

On Fri, Sep 06, 2024 at 12:47:46PM -0600, Nelson H. F. Beebe wrote:
> The one complete failure was for MidnightBSD 3.0.0, released and
> installed in March 2023.  On that system, the libintl package is not
> available, and compilation fails with
> 
>   parsetexi/api.c:33:10: fatal error: 'libintl.h' file not found
>   #include 
>^~~
>   1 error generated.
> 
> Apart from that system, which has only a small user base, I consider
> the texinfo-7.1.0.91 release a substantial success.  Thanks for
> keeping it working close to everywhere!

There is indeed an

  #include 

not protected by

  #ifdef ENABLE_NLS.

in this file in texinfo-7.1.0.90. (I haven't checked texinfo-7.1.0.91,
but I guess that it is the same).

The master branch do not have this issue, there is no #include 
anymore in parsetexi/api.c, and I checked that all the #include 
occurences are correctly protected.

I do not know if it is worth fixing, I'll let Gavin decide.

Thanks for the report anyway!

-- 
Pat

Re: tightening mathjax tex options

[Patrice Dumas]

On Fri, Sep 06, 2024 at 08:47:05AM -0700, Raymond Toy wrote:
> What are the current values for these? I’m most worried that if these are
> different, you’ll break lots of existing uses. Maxima’s user manual
>  uses Mathjax a lot to get
> nice HTML display of math.

I had a look of what you do in the maxima manual, but your case is very
different from other manuals.  Indeed, you do not use MathJax as
provided by texi2any for @math and @displaymath, but you have math all
over your document, not in any specific HTML element.  The settings I
propose would not work for you, but you cannot use texi2any mathjax
support anyway, as this adds a tex2jax_ignore class to the body, such
that MathJax would not render any of your inline math.  If you do not
have @math nor @displaymath, it would not do anything, though, but the
change in options would not affect you anyway.

So in any case, I think that your case is quite specific, and would
probably require changes to the texi2any code to avoid
class="tex2jax_ignore" being added to the .  The new options I
want to add could make things worse, but right now, what you do probably
do not integrate well with texi2any generated MathJax.  Do you use
@math/@displaymath and texi2any MathJax?

-- 
Pat

Re: tightening mathjax tex options

[Patrice Dumas]

On Fri, Sep 06, 2024 at 08:47:05AM -0700, Raymond Toy wrote:
> On 9/6/24 3:28 AM, Patrice Dumas wrote:
> 
> > Hello,
> > 
> > I propose to set the following in mathjax defaults, to avoid possible
> > interactions with text (although it may not be possible anyway because
> > of ignoreHtmlClass set in body), and perhaps simplifying the task of 
> > mathjax:
> > 
> >tex: {
> >  processEscapes: false,  // use \$ to produce a literal dollar sign
> >  processEnvironments: false, // process \begin{xxx}...\end{xxx} outside 
> > math mode
> >  processRefs: false, // process \ref{...} outside of math mode
> >  displayMath: [ // start/end delimiter pairs for display 
> > math
> >['\\[', '\\]']
> >  ],
> >}
> > 
> > Any comment?
> What are the current values for these? I’m most worried that if these are
> different, you’ll break lots of existing uses. Maxima’s user manual
> <https://maxima.common-lisp.dev/docs/index.html> uses Mathjax a lot to get
> nice HTML display of math.

The change in settings may only affect LaTeX rendered by MathJax outside
of @math and @displaymath, using different structures as what is done in
texi2any.  This would be possible if you directly ouput HTML like

$$ \frac{a}{b} $$


some text
\begin{equation}
 f(h) = a^{b^c}
\end{equation}




Do you have a page which shows some math that are rendered by MathJax?



The defaults are:

displayMath: [ // start/end delimiter pairs for display math
  ['$$', '$$'],
  ['\\[', '\\]']
],
processEscapes: true,  // use \$ to produce a literal dollar sign
processEnvironments: true, // process \begin{xxx}...\end{xxx} outside math 
mode
processRefs: true, // process \ref{...} outside of math mode


see
https://docs.mathjax.org/en/latest/options/input/tex.html#tex-options

tightening mathjax tex options

[Patrice Dumas]

Hello,

I propose to set the following in mathjax defaults, to avoid possible
interactions with text (although it may not be possible anyway because
of ignoreHtmlClass set in body), and perhaps simplifying the task of mathjax:

  tex: {
processEscapes: false,  // use \$ to produce a literal dollar sign
processEnvironments: false, // process \begin{xxx}...\end{xxx} outside math 
mode
processRefs: false, // process \ref{...} outside of math mode
displayMath: [ // start/end delimiter pairs for display math
  ['\\[', '\\]']
],
  }

Any comment?

-- 
Pat

removing border=0 for tables as it is the default

[Patrice Dumas]

Hello,

It seems that border=0 is the default for tables formatting in HTML, it
is not specified anywhere, but it has been like that for a long time,
therefore I propose to simply remove border=0 everywhere.

Any opposition?

-- 
Pat

Re: EPUB 3.3 spec conformity issues

[Patrice Dumas]

On Wed, Sep 04, 2024 at 12:29:55AM +0200, Patrice Dumas wrote:
> I have not checked the thread, but I recall that the output we obtained
> with  only was not good, at least for some browser.  I do not think
> that we need to change to  for epubcheck compliance, though.

I only had partial recall, actually Per objected the index letter being
in the table and not simply before the entries.  That is probably
something that can be changed.  However using div for subentries and two
column table is something to be left for later, I think.

-- 
Pat

Re: texi2any export to html with static LaTeX

[Patrice Dumas]

On Thu, Sep 05, 2024 at 09:49:02PM +0100, Jeremy Bryant wrote:
> Hello,
> 
> Version: texi2any (GNU texinfo) 7.1
> 
> This appears to depend on latex2html which appears not to be maintained.
> Is there a fix?

You could try this patch and apply to $datadir/texinfo/ext/latex2html.pm

-- 
Pat
diff --git a/tp/ext/latex2html.pm b/tp/ext/latex2html.pm
index 4d539cf6f4..44408211d6 100644
--- a/tp/ext/latex2html.pm
+++ b/tp/ext/latex2html.pm
@@ -436,7 +436,8 @@ sub l2h_to_html($$$)
   # set output dir
   my $encoded_destination_dir_option = ' -no_subdir';
   my $destination_dir_option = $encoded_destination_dir_option;
-  if ($destination_directory ne '') {
+  my $curdir = File::Spec->curdir();
+  if ($destination_directory ne '' and $destination_directory ne $curdir) {
 $encoded_destination_dir_option = " -dir ".$destination_directory_string;
 $destination_dir_option = " -dir ".$destination_directory;
   }

Re: texi2any export to html with static LaTeX

[Patrice Dumas]

Hello,

On Thu, Sep 05, 2024 at 09:49:02PM +0100, Jeremy Bryant wrote:
> Hello,
> 
> Version: texi2any (GNU texinfo) 7.1
> 
> Goal:
> The manual recommends a command like the following to export a .texi
> file containing LaTeX to HTML:
> 
> texi2any --html file1.texi -c HTML_MATH=l2h --split=none
> 

Note that --split=none will not do what you want.  It triggers a
warning:
  texi2any: warning: none is not a valid split possibility
it should be --no-split.


The following works for me on a debian testing, with 'texi2any (GNU texinfo)
7.1' and latex2html 'This is LaTeX2HTML Version 2024 (Released January
1, 2024)'

texi2any --html toto.texi  -c HTML_MATH=l2h --split=none

with toto.texi:

@node Top
@top top

@math{\frac{h}{\atan j}}


Beware that texi2any caches latex2html output, in some cases to force an
actual call of latex2html you would need to remove the result directory.


However, the following fails with similar errors:

texi2any --html --no-split -c HTML_MATH=l2h toto.texi

The most relevant error is probably:

Error (Make_directory_absolute): chdir "" failed: No such file or directory at 
/usr/bin/latex2html line 770.

> Error:
> But this fails with:
> texexpand V2024 (Released January 1, 2024)

> texexpand: Error: Cannot open style file '/TMP/styles': No such file or 
> directory
> 

> This appears to depend on latex2html which appears not to be maintained.
> Is there a fix?

It seems to me that latex2html is maintained nowadays.  For some years
in the 201X it seemed to be unmaintained, but this does not seems to be
the case now.


On the failure, it seems that latex2html -dir . does not work.  I will
propose a patch to set -dir only if the output directory is not the
current directory.


-- 
Pat

Re: EPUB 3.3 spec conformity issues

[Patrice Dumas]

On Mon, Sep 02, 2024 at 06:23:10PM +0100, Gavin Smith wrote:
> > Been the compliance done with CSS, or maybe not using the Texinfo HTML
> > generator, and creating a dedicated EPUB's XML generator. In order for
> > the index table border to be in compliance.
> 
> I believe it should be fairly easy to override the border attribute
> for EPUB output only, for the 7.2 release.

It won't be the only issue, there are other removed attributes that we
use.  All should probably be fixable, but it will involve some
relatively important changes (more or less every  use), although
it will be conditional on a customization variable being set.

>  After that we could revise
> Texinfo indices not to use  at all (as said in previous email
> by Per Bothner) which would ameliorate the issue.

I have not checked the thread, but I recall that the output we obtained
with  only was not good, at least for some browser.  I do not think
that we need to change to  for epubcheck compliance, though.

>  In terms of implementation
> all we would have to do is set something when the init file epub3.pm
> runs that would be checked by the code in HTML.pm - some kind of internal
> variable.

It seems to me that a customization variable would be better, as those
change are in general needed to have HTML compliant with recent
standards, so having that possibility should be interesting outside of
EPUB.  It would be set in epub3.pm only by default.

> > Regardind the dcterms:modified, you may want to increase complexity,
> > trying to make Texinfo builds reproducible For me that is a lost
> > cause since the pdftex does not, and won't, make PDF builds reproducible
> > either (and Texinfo depends on (La)TeX).
> 
> I am not an expert on reproducible builds and you make a valid point
> about PDF builds not being reproducible.
> 
> It appears that pdftex obeys the SOURCE_DATE_EPOCH variable and we
> could do the same for texi2any for EPUB (as mentioned by Werner Lemberg
> on this list).

I do not think that it would be needed, we only generate EPUB for tests,
and we need to have reproducible tests even more than we need
reproducible builds.  We should use TEST customization variable for
that, and would not need SOURCE_DATE_EPOCH.

To me the issue with dcterms:modified is not about reproducible builds
or reproducible output, the issue is that it is not right to set
dcterms:modified automatically without a possibility for the user to
override the value.

> > I would like to discuss your opinions in regard of the EPUB compliance,
> > to see where you stand, and to summarize which ideas do you think is the
> > best approach, for the next release.
> 
> I am in favour of automatic compliance with no further action needed on
> the part of the user of texi2any as it seems that this should be easy
> to achieve, although I would like to take more time to understand Patrice's
> view on this matter.

I agree that there is no need for user action for compliance.  Although,
as I said above, there is a need to hold up setting dcterms:modified
until we have added the possibility for the user to set it.  We could
add a customization for that if this cannot wait for after the release,
but it would only be needed for one release.

-- 
Pat

Re: distribution of texi2any_internals.info

[Patrice Dumas]

On Tue, Sep 03, 2024 at 06:49:57PM +0100, Gavin Smith wrote:
> 
> I don't understand why we are building the texi2any_internals.info file
> in the first place.  I would have expected the file to be distributed
> with Texinfo, like other .info files.

I think that I made that change when I did a review of all the
distributed and cleaned files.  I realized that we did not distribute
the info files except for info files in tp_api, so I changed tp_api
Makefile.am such that these info files are also not distributed.

In doc/Makefile.am there is

# Do not create info files for distribution.
dist-info:

Looking at git blame, seems to be old:
Karl Berry  2002-08-25

> I couldn't find any emails from around that time discussing why the change
> was made.  Patrice, do you remember?

I do not think that I started a discussion on it, because to me it
seemed to be an error of myself, committed when I did doc/tp_api
Makefile.am and did not consistently do as in doc/.

Also it seemed somewhat logical to me, in GNU Texinfo there is no reason
why we could not regenerate info files, which is also why I did not
start a discussion on that but did the change.

> Running texi2any as part of the build from the tarball is likely problematic
> for cross-compiling.  (In fact I don't know if Texinfo has been successully
> cross-compiled in recent years as I don't remember anyone talking about it.)

I doubt it since it seems like makeinfo/texi2any is used to generate the
info files since 2002.  It seems to me that this is what the TOOLS_ONLY
and $cross_compiling part in configure.ac and Makefile.am are about,
there is this comment:

# We need to run some of our own binaries, most notably makedoc, but as
# long as we have this process, we also use our own makeinfo and
# install-info.

However, it is possible that it may be broken now because of some
changes -- though probably not removing the tp_api info files.  For
instance maybe the configure call in if test "$cross_compiling" != no
should include PERL_EXT_CFLAGS.  Currently it is:

 env -i CC="$BUILD_CC" AR="$BUILD_AR" RANLIB="$BUILD_RANLIB" \
  PATH="$PATH" \
  tools_only=1 \
${confdir}/configure --build=${build} --host=${build} \
 --disable-rpath --disable-nls

I guess that it worked at least to some extent in 2019 as there are some
discussions on it:
https://github.com/ElvishJerricco/cross-nixos-aarch64/issues/1
https://lists.libreplanet.org/archive/html/guix-commits/2019-08/msg01053.html
https://github.com/NixOS/nixpkgs/pull/76598/commits/31f12acea46d3ad3787bb1ac332dbd2a40739622

-- 
Pat

Re: Texinfo 7.1.0.91 pretest available

[Patrice Dumas]

On Sun, Sep 01, 2024 at 06:49:59PM +0100, Gavin Smith wrote:
> 
> I resisted making this change for a while but it seemed to be the
> right thing.  There may have been other ways to fix the "make distclean"
> issue that people were complaining about.  If it causes a problem on
> some platforms we could revisit this.

In addition to make disctclean issues, it seems to me that using the
xsubpp consistent with the installed Perl interface is a good thing.  I
think that there are checks of interfaces compatibility in xsubpp/Perl,
but still, my feeling is that xsubpp is somewhat part of the Perl
interface.

-- 
Pat

Re: XDG Base Directory Specification ignores installation directories

[Patrice Dumas]

I already have done an implementation, but I can easily modify it:

https://git.savannah.gnu.org/cgit/texinfo.git/commit/?id=7ba51006395793fe6bdfdd2fc195cba27c6f758b

On Wed, Aug 28, 2024 at 09:26:55PM +0100, Gavin Smith wrote:
> On Wed, Aug 28, 2024 at 01:34:23PM +0200, Patrice Dumas wrote:
> > Hello,
> > 
> > Using the XDG Base Directory Specification is not as straightforward as
> > what I thought.  I have changed my mind compared to my previous
> > proposition, because interaction between defaults if $XDG_DATA_DIRS is
> > not set and $datadir installation directory are not really considered by
> > the specification (I'd say rightly as it is out of scope for the
> > specification) but we need to make decisions about all that.
> 
> I'm a bit confused as I thought it was configuration files which we
> were discussing, which would be XDG_CONFIG_DIRS, not XDG_DATA_DIRS.
> 
> I see that htmlxref.cnf is installed under $datadir, e.g. as
> "/usr/local/share/texinfo/htmlxref.cnf".  I suppose there is a question
> of whether this file counts as "data" or "configuration".  We should
> decide which one each file we are talking about is, otherwise this issue
> is more confusing.

To me data vs configuration is, in general, not about the nature of the
information, but about the intent on how a resource is modifiable and by
whom.  If the resource is considered to be a non-changeable asset, it is
data, but the exact same file could be configuration in another place
where it is editable.  It is the established practice, in sysconfdir the
installed or searched files are supposed to be edited, while the files
installed in datadir are not supposed to be edited, and could be mounted
read-only.  The XDG Base Directory Specification allows this
interpretation by not defining precisely what data and configuration is.

> > Here is my current proposal:
> > 
> > - replace ~/.texinfo by the XDG specification, with $XDG_CONFIG_HOME.
> >   In my opinion, it is an improvement compared to using ~/.texinfo directly
> >   in HOME. The specification is unclear on what should be in $XDG_DATA_HOME
> >   or $XDG_CONFIG_HOME but my feeling is that $XDG_CONFIG_HOME is better. 
> >   Still find files in ~/.texinfo/ after specification locations for some
> >   time, with a deprecation message if found in that directory.
> 
> I added similar logic for the "info" program, in infomap.c.  XDG_CONFIG_HOME
> is checked first (both as an environment variable and in the default
> "$HOME/.config" location).  If the file is found there, then the old
> location ("$HOME/infokey") is not checked.
> 
> The case of htmlxref.cnf (I assume that's the file we're talking about)

It is about both htmlxref.cnf and init files lookup.

> is more complicated as more than one file can be loaded.  I would say
> to check under XDG_CONFIG_HOME first, and if found, do *not* check under
> the old location ("$HOME/.texinfo/").  This would allow setting
> XDG_CONFIG_HOME to override all configuration in the user's home
> directory.

It is about what I did.  If XDG_CONFIG_HOME is set, do not check under
the old location nor .config/texinfo.  If XDG_CONFIG_HOME is not set,
check under .config/texinfo and then the old location, and issue a
deprecation message if a file is found in the old location.

> Check under XDG_CONFIG_DIRS in any case for htmlxref.cnf files (with
> any files there having a lower priority).
> 
> There is no rush to add a deprecation message.

I did it.  It can easily be removed.

> > - to replace the former search in $datadir/texinfo:
> >   if $XDG_DATA_DIRS is set, search in those directories.  Then search
> >   in $datadir (if not already in $XDG_DATA_DIRS).  Last search in
> >   /usr/local/share and /usr/share (if not already done), which are
> >   the default for $XDG_DATA_DIRS that have to be used if $XDG_DATA_DIRS
> >   is not set.  In what I propose, they are used even if $XDG_DATA_DIRS
> >   is set.
> 
> If we consider htmlxref.cnf a "config" file rather than a "data" file,
> then the copy of htmlxref.cnf under $datadir (e.g. /usr/local/share/texinfo)
> would not be covered as a "config" file by the XDG specification.
> Therefore setting XDG_DATA_DIRS should have no effect on whether this
> file is read or not.

I do not think that this is the right way to look at this.  To me the
logic is that we search both in "config" locations, as it is where the
system/user may place an htmlxref.cnf files, and we search there first,
and in "data", which is where the default from texi2any is installed.

I proposed and implemented an extension to that logic, by proposing
a third type of

use all files in some directories for htmlxref

[Patrice Dumas]

Hello,

I think that it would be better to read all the files in some
directories (only files ending with .conf) to find the htmlxref entries.

I think that it is better to read only one file htmlxref.cnf (or
HTMLXREF_FILE In the current directory, in the manual directory and in
the user HOME.

But in $datadir/texinfo, $sysconfdir/texinfo, $XDG_DATA_DIRS,
$XDG_CONFIG_DIRS, /usr/share/texinfo and /usr/local/share/texinfo, it
would be better, in my opinion, to have a directory named htmlxref/
(or htmlxref.d) and merge all the files ending with .conf.  We could
also read htmlxref.cnf (or HTMLXREF_FILE) directly in $dir/texinfo/ for
backward compatibility.

The idea is to make it simpler
* to have diverse groups add their list of htmlxref entries.  This could be
  a step towards what Eli proposed.
* to have GNU/Linux distributions add htmlxref entries.
* for other software packages to add their lists (texi2html if it was still
  maintained, or other software that generate online manuals that can be
  targets of Texinfo cross-references).

We could split the htmlxref.cnf we maintain in GNU/nonGNU, over GNU we
would be authoritative, but not for other entries.

The only drawback I see is that it becomes easier to put a wrong entry.
Maybe we could add a customization variable to report double entries to
mitigate that issue.

What do you think?

-- 
Pat

XDG Base Directory Specification ignores installation directories

[Patrice Dumas]

Hello,

Using the XDG Base Directory Specification is not as straightforward as
what I thought.  I have changed my mind compared to my previous
proposition, because interaction between defaults if $XDG_DATA_DIRS is
not set and $datadir installation directory are not really considered by
the specification (I'd say rightly as it is out of scope for the
specification) but we need to make decisions about all that.

Here is my current proposal:

- replace ~/.texinfo by the XDG specification, with $XDG_CONFIG_HOME.
  In my opinion, it is an improvement compared to using ~/.texinfo directly
  in HOME. The specification is unclear on what should be in $XDG_DATA_HOME
  or $XDG_CONFIG_HOME but my feeling is that $XDG_CONFIG_HOME is better. 
  Still find files in ~/.texinfo/ after specification locations for some
  time, with a deprecation message if found in that directory.

- to replace the former search in $datadir/texinfo:
  if $XDG_DATA_DIRS is set, search in those directories.  Then search
  in $datadir (if not already in $XDG_DATA_DIRS).  Last search in
  /usr/local/share and /usr/share (if not already done), which are
  the default for $XDG_DATA_DIRS that have to be used if $XDG_DATA_DIRS
  is not set.  In what I propose, they are used even if $XDG_DATA_DIRS
  is set.

  I think that $XDG_DATA_DIRS should be used first as it corresponds
  to the users/system preferences.  Then $datadir is where the files
  have been installed (by the user).  The default XDG directories last
  can be thought as a way to always have implementation and user preference
  independent locations (as a last choice), that could be shared
  by other implementations too (texi2html for example).  It is not
  dedicated to that use, though, as /usr/local/share and /usr/share
  could also be in $XDG_DATA_DIRS or correspond to $datadir.

- do the same for $sysconfdir/texinfo as for $datadir/texinfo.

What do you think?

-- 
Pat

Re: Texinfo 7.0 changed the name of HTML output directory

[Patrice Dumas]

On Mon, Aug 26, 2024 at 09:10:44PM +0300, Eli Zaretskii wrote:
> > I have not seen anything platform specific in the specification, as far
> > as I can tell (except for the path separator), no more than what we
> > currently do.
> 
> The directories that are the default values of the various XDG
> locations do not necessarily exist (or even make sense) on other
> systems. 

Maybe, but so do the directories we currently use, they are basically
the same base directories, the differences are the subdirectories only.

>  And system utilities and services do not recognize them, so,
> for example, the cache directory is not cleaned up from time to time.

I see.  For now for our use this is not relevant as we only need to find
config/data files.

> > But that means that you support forever two locations, the traditional
> > places, and the XDG-mandated locations if variables are not set.  I
> > wanted to support only one, the XDG-mandated locations if variables are
> > not set and deprecate the traditional locations (and use the XDG
> > environment variables if set, but there is no trade-off with that case).
> 
> I understand the desire, but I don't think it's practical to make such
> abrupt changes in user-facing facilities.

It is not so abrupt, we'll have a message saying which directory should
be used for a long period of time (I proposed 5 years).

> But yes, most of the burden related to this complication is on the
> Emacs developers, not on users.

Being that those directories should not be used a lot to begin with,
having a transitory period instead of both systems forever seems
preferable to me.  We can always change our mind if we get users
complaints.

-- 
Pat

Re: Texinfo 7.0 changed the name of HTML output directory

[Patrice Dumas]

On Mon, Aug 26, 2024 at 02:29:10PM +0300, Eli Zaretskii wrote:
> > Date: Mon, 26 Aug 2024 09:02:37 +0200
> > From: Patrice Dumas 
> > 
> > On Sun, Aug 25, 2024 at 06:57:42PM +0100, Gavin Smith wrote:
> > > On Sun, Aug 25, 2024 at 02:15:17PM +0200, Patrice Dumas wrote:
> > > > On Sun, Aug 25, 2024 at 11:32:01AM +0100, Gavin Smith wrote:
> > > > > We should at least support the "XDG Base Directory Specification".
> > 
> > I see three possibilities for the locations that do not already follow
> > the XDG Specification, locations like /etc/texinfo or /etc/texi2any, and
> > locations like $HOME/.texinfo or $HOME/.texi2any
> > 
> > 1) remove them as possibilities completly and mark in the documentation
> >   that the new directory have to be used
> > 2) keep them as possibilities but mark them deprecated in the
> >   documentation, add warnings in the code and keep them about 5 years
> > 3) keep them and maintain them
> > 
> > I would lean towards 2), otherwise there would be too many directories
> > searched for.
> > 
> > Opinions?
> 
> XDG is supported only on some systems.  For example, MS-Windows
> doesn't support it, and I'm not sure about macOS.

I have not seen anything platform specific in the specification, as far
as I can tell (except for the path separator), no more than what we
currently do.

> So my suggestion is look for the files in their traditional places,
> and failing that to fall back on XDG-mandated locations (with suitable
> defaults as defined by XDG, if the relevant environment variables are
> not set).

But that means that you support forever two locations, the traditional
places, and the XDG-mandated locations if variables are not set.  I
wanted to support only one, the XDG-mandated locations if variables are
not set and deprecate the traditional locations (and use the XDG
environment variables if set, but there is no trade-off with that case).

> This is what Emacs does, and we have yet to hear a single
> complaint.  We found this to be the best alternative in terms of
> backward compatibility: it allows users smooth migration to
> XDG-compliant configuration if and when they so desire.

It's no wonder that no one complain, all the added complexity is on you. 
I wanted to avoid that.

> (Btw, AFAIK XDG directories can be ephemeral, and the environment
> variables can be removed when the user logs off.  This should be kept
> in mind when designing the support for XDG.)

Ok.

-- 
Pat

Re: Texinfo 7.0 changed the name of HTML output directory

[Patrice Dumas]

On Sun, Aug 25, 2024 at 06:57:42PM +0100, Gavin Smith wrote:
> On Sun, Aug 25, 2024 at 02:15:17PM +0200, Patrice Dumas wrote:
> > On Sun, Aug 25, 2024 at 11:32:01AM +0100, Gavin Smith wrote:
> > > We should at least support the "XDG Base Directory Specification".

I see three possibilities for the locations that do not already follow
the XDG Specification, locations like /etc/texinfo or /etc/texi2any, and
locations like $HOME/.texinfo or $HOME/.texi2any

1) remove them as possibilities completly and mark in the documentation
  that the new directory have to be used
2) keep them as possibilities but mark them deprecated in the
  documentation, add warnings in the code and keep them about 5 years
3) keep them and maintain them

I would lean towards 2), otherwise there would be too many directories
searched for.

Opinions?

-- 
Pat

Re: Texinfo 7.0 changed the name of HTML output directory

[Patrice Dumas]

On Sun, Aug 25, 2024 at 08:42:39PM +0300, Eli Zaretskii wrote:
> > Date: Sun, 25 Aug 2024 19:04:39 +0200
> > From: Patrice Dumas 
> > 
> > On Sun, Aug 25, 2024 at 11:32:01AM +0100, Gavin Smith wrote:
> > > It's actually worse.  It's not just a matter of placing the htmlxref.cnf
> > > file in the same directory as the Texinfo source.  It has to be present
> > > in the working directory (.) where texi2any runs.  (I just tested
> > > this with current git 'master'.)  
> > 
> > Now an htmlxref.cnf file in the same directory as the Texinfo source
> > should be used.
> 
> Only there or _also_ there (in addition to looking in the directory
> where texi2any runs)?

Also there.  First in the directory where texi2any runs and then in the
same directory as the Texinfo source before the other directories.

-- 
Pat

Re: Texinfo 7.0 changed the name of HTML output directory

[Patrice Dumas]

Hello,

Here is an updated version of the documentation patch, with Eli text.

On Sat, Aug 24, 2024 at 08:26:13AM +0300, Eli Zaretskii wrote:
> > +all HTML manuals generated by @command{texi2any} use this location for
> > +cross-references in the default case.  You can get the latest version from
> >  @url{http://ftpmirror.gnu.org/@/texinfo/@/htmlxref.cnf}.
> 
> I suggest to tell here explicitly that having a local htmlxref.cnf
> file with the correct information is the solution for outdated or
> missing data in the file as distributed by Texinfo.

I did that, but before in the node.

-- 
Pat
diff --git a/doc/texinfo.texi b/doc/texinfo.texi
index cc772d93ad..d88155397b 100644
--- a/doc/texinfo.texi
+++ b/doc/texinfo.texi
@@ -17515,7 +17515,19 @@ @node HTML Xref Link Basics
 (@pxref{Cross References}), and from @dfn{external information}
 (@pxref{HTML Xref Configuration}).
 
-We now consider each part in turn.
+@c Eli comment on the next paragraph:
+@c This part is unclear, I could not understand what it means and how
+@c that affects me as the user of Texinfo.
+The defaults for the @var{host} and @var{dir} correspond with cross-references
+links for a collection of manuals located side by side on a single host.  This
+is different from the usual case of HTML manuals published on the World Wide
+Web served from different hosts in directories chosen by the publisher, for
+which the @var{host} and @var{dir} must be specially defined to have working
+cross-references.  However this requires external information and is detailed
+later on (@pxref{HTML Xref Configuration}).
+
+We start with the case of manuals all located on the same
+host and consider each part in turn.
 
 The @var{host} is hardwired to be the local host.  This could either
 be the literal string @samp{localhost}, or, according to the rules for
@@ -17830,6 +17842,14 @@ @node HTML Xref Configuration
 @cindex Cross-reference configuration, for HTML
 @cindex Configuration, for HTML cross-manual references
 
+For other manuals published on the World Wide Web to be able to have
+correct cross-references to your manual, the @var{host} and
+@var{dir} parts of the cross-reference need to be known by Texinfo
+converters when they produce HTML for those other manuals.
+Similarly, for your manual in HTML format to have correct
+cross-references to other manuals, @command{texi2any} needs to know
+how to construct the URLs for other manuals.
+
 @command{texi2any} reads a file named @file{htmlxref.cnf} to gather
 information for cross-references to other manuals in HTML output.  It
 is looked for in the following directories:
@@ -17862,6 +17882,11 @@ @node HTML Xref Configuration
 GNU manuals; it is installed in the last of the above directories,
 i.e., @file{@var{datadir}/texinfo/htmlxref.cnf}.
 
+A local @file{htmlxref.cnf} should be used to complete or correct
+the default file from the Texinfo distribution.  This could be useful to
+correct outdated data or make sure that cross-references to local files
+are generated for some manuals.
+
 The @code{HTMLXREF_MODE} customization variable can be set to modify how the
 files are found.  For instance, if set to @samp{none}, no external information
 is used.  @code{HTMLXREF_FILE} sets the file name to something else than
@@ -17939,9 +17964,17 @@ @node HTML Xref Configuration
 only when the target manual was created using @option{--node-files};
 this is the default for split output.
 
+Right now, the @file{htmlxref.cnf} file distributed with GNU Texinfo serves
+as the main resource to locate Texinfo HTML manuals in the World Wide Web.
+Since it is installed in a location used by @command{texi2any}, HTML manuals
+information found in this file will be used for cross-references by default.
+
 If you have additions or corrections to the @file{htmlxref.cnf}
 distributed with Texinfo, please email @email{bug-texinfo@@gnu.org} as
-usual.  You can get the latest version from
+usual.  If you publish a Texinfo HTML manual on the World Wide Web,
+having an up-to-date location listed in @file{htmlxref.cnf} should ensure that
+all HTML manuals generated by @command{texi2any} use this location for
+cross-references in the default case.  You can get the latest version from
 @url{http://ftpmirror.gnu.org/@/texinfo/@/htmlxref.cnf}.
 
 

Re: Texinfo 7.0 changed the name of HTML output directory

[Patrice Dumas]

On Sun, Aug 25, 2024 at 11:32:01AM +0100, Gavin Smith wrote:
> It's actually worse.  It's not just a matter of placing the htmlxref.cnf
> file in the same directory as the Texinfo source.  It has to be present
> in the working directory (.) where texi2any runs.  (I just tested
> this with current git 'master'.)  

Now an htmlxref.cnf file in the same directory as the Texinfo source
should be used.

-- 
Pat

Re: Texinfo 7.0 changed the name of HTML output directory

[Patrice Dumas]

On Sun, Aug 25, 2024 at 02:13:07PM +0200, Patrice Dumas wrote:
> On Sun, Aug 25, 2024 at 12:58:21PM +0100, Gavin Smith wrote:
> > I think it makes sense to treat relative paths in a similar way to
> > absolute paths.  The only question is whehther the path should be
> > relative to current working directory or the source file directory
> > (or possibly both).  The current behaviour is to be relative to cwd
> > and so that should be kept.
> 
> Agreed.  I'll do that change before the release.

This should be done.

> > > We could make this list of directories a customization option, like we
> > > currently do with INCLUDE_DIRECTORIES, which is influenced by -P and -I,
> > > for example named TEXINFO_LANGUAGE_DIRECTORIES, and add a command line
> > > option to prepend to this list?
> 
> I think that having a customization option is a good thing in any case,

I have implemented that, and also changed the Texinfo::Config code such
that it is possible to prepend directories.  Motivated users could
easily do an init file that places a directory after the current
directory and the input file path directory but before the other
directories.  This cannot be general, though.

-- 
Pat

Re: Texinfo 7.0 changed the name of HTML output directory

[Patrice Dumas]

On Sun, Aug 25, 2024 at 06:54:03PM +0300, Eli Zaretskii wrote:
> > I think that, for now, we should have all the manuals in htmlxref.cnf
> > not only GNU manuals.
> 
> This might be too much work for you, for no good reason.  There are
> gazillions of manuals out there, and having all of them in that file,
> let alone keeping it up to date, is not necessarily the business of
> the Texinfo project.  By contrast, leaving it to individual projects
> to take care of references only they need makes a lot of sense to me.

Independently of what we have in the Texinfo htmlxref.cnf, we should
make what you propose possible, and we could indeed decide that it is
enough.  But my feeling is that the inter Texinfo manual
cross-references web is not easily split and that a main database has
merit.

> But you are the Texinfo maintainers, so it's your call.

The idea is not to maintain it by ourselves, but rather accept
patches...

> Is doi.org run by 2 volunteers, or does it maybe have a slightly
> larger staff?

;-)

> > and have the non GNU manuals be delagated somehow, but for now, as long
> > as we do not have such an structure, I think that it is better if we add
> > all the known Texinfo HTML manuals.
> 
> Why do you assume the referent manual must necessarily be a Texinfo
> manual?  That is completely not guaranteed.  It could be produced by
> Sphinx, for example, or some other HTML-authoring tool in use out
> there.

I do not really understand what you refer to, because I do not know much
outside of Texinfo, but I agree on the principle that, in a perfect
world, we should coordinate with all the software/publishing processes
that lead to resources that may be used as Texinfo manual names in
cross-references (@xref and similar, menus, nodes), even if they are not
obtained through conversion of a Texinfo manual.  This does not concern
the @url/@uref URLs, though.

-- 
Pat

Re: Texinfo 7.0 changed the name of HTML output directory

[Patrice Dumas]

On Sun, Aug 25, 2024 at 05:27:19PM +0300, Eli Zaretskii wrote:
> As you discuss the various changes and improvements to this, please
> consider one additional aspect: it is quite reasonable for a project
> to have cross-references to other manuals, which are not necessarily
> of interest for the GNU Project, at least not enough to have that
> referent manual in htmlxref.cnf.

I think that, for now, we should have all the manuals in htmlxref.cnf
not only GNU manuals.  I view the htmlxref.cnf file as a way to
translate any manual identifier (the manual name right now) as a
ressource located in the World Wide Web (that can be used in a
cross-reference).  A bit like what doi.org does but restricted to the
Texinfo HTML manuals world -- for that objective I cannot see how a
distributed database could do the job.  I guess that doi.org delegates
to organizations, we should do that too, do the GNU manuals ourselves,
and have the non GNU manuals be delagated somehow, but for now, as long
as we do not have such an structure, I think that it is better if we add
all the known Texinfo HTML manuals.  We also should do that in
collaboration with other software that convert Texinfo to HTML, as they
should have the same World Wide Web manuals htmlxref.cnf we have here
and, if possible, help adding entries for manuals that are in general
converted by those softwares.  I have no idea if there exist other
softwares that convert Texinfo to HTML, actually.

> So maybe we need two such files: one maintained by Texinfo, where only
> GNU manuals are mentioned, the other local to a project, where the
> project could augment the Texinfo-maintained file with additions that
> are of interest only to that project.

We should have that, I think, even if we try to have all the manuals in
Texinfo htmlxref.cnf.

> Yes, I know that this should already be possible, but at least the
> Texinfo manual should mention this.  E.g., currently the Texinfo
> manual asks for every update to the file to be reported to the Texinfo
> developers, but that makes little sense for such "project-specific"
> external references.  Maybe this project-local file should also be
> named differently, I don't know.

It is not really possible as we discussed previously, to have a per
project htmlxref.cnf.  It is possible to get the information from
diverse directories, but none of those directories correspond to what
would be a per project directory.

I just did a commit that adds TEXINFO_LANGUAGE_DIRECTORIES, so it is now
somewhat possible to prepend a directory there, but only from an init
file, and only prepending for now.

-- 
Pat

Re: Texinfo 7.0 changed the name of HTML output directory

[Patrice Dumas]

On Sun, Aug 25, 2024 at 11:32:01AM +0100, Gavin Smith wrote:
> 
> This is a hack but may be the best way at the moment.  I think we
> should have a better way of supporting this if there isn't one already.
> We should at least support the "XDG Base Directory Specification".

I can work on that.  Should it be done before or after the release?

I think that adding the manual source directory could be done before the
release as it is not a big change.

-- 
Pat

Re: Texinfo 7.0 changed the name of HTML output directory

[Patrice Dumas]

On Sun, Aug 25, 2024 at 12:58:21PM +0100, Gavin Smith wrote:
> I think it makes sense to treat relative paths in a similar way to
> absolute paths.  The only question is whehther the path should be
> relative to current working directory or the source file directory
> (or possibly both).  The current behaviour is to be relative to cwd
> and so that should be kept.

Agreed.  I'll do that change before the release.

> > This is already possible, it is CHECK_HTMLXREF (which is set to 1 in the
> > default case for EPUB as relative paths in EPUB are not possible).
> 
> Good point.  I think we should make CHECK_HTMLXREF=1 the default for
> HTML as otherwise various web manuals will have broken links that nobody
> knows about (we already found a broken link for "eglot", and the link
> for "smtpmail" would have been broken if a newer version of Texinfo had
> been used).

I think that I agree on the principle, but I think that we should
announce it before we make that change, such that people may voice their
concern, and maybe we should do it after the release and not now?

> > We should probably implement the two changes proposed above, to me they
> > are improvements.  But I do not think that they will solve this use case
> > either, if I understood well.
> > 
> > To me, to solve this use case, a natural solution would be the
> > possibility to prepend a directory to the list of directories searched
> > for htmlxref.cnf, such that all the manuals in a project can use the
> > same htmlxref.cnf to override or add to the default htmlxref.cnf.
> > However, this is not possible for now.  For other directory lists, we
> > have this possibility, with -I (and we even have -P) for @include and
> > images and for init files there is --conf-dir.
> 
> If we implemented the XDG specification then the user could use or
> append to XDG_CONFIG_DIRS, which is a colon-separated list of
> directories to search.  However, I am not sure if this is an appropriate
> use of this variable as this is for user- or system-specific configuration,
> but not necessarily for project-specific configuration.
> 
> https://specifications.freedesktop.org/basedir-spec/latest/index.html
> 
> So I suppose we need to devise some other mechanism for this.

I think so too.

> > htmlxref.cnf is searched for in a list of directories that I called
> > @language_config_dirs in texi2any, with the idea that this list of
> > directory could be (at least partly) implementation independent.  For
> > example, if texi2html was still developped, it could use the same search
> > path for htmlxref.cnf files.  For now, this @language_config_dirs list
> > is used for two purposes:
> > 1) to set some paths in the default directories list used for init files,
> >   with init subdirectory prepended.  This corresponds to the path 6 to 9
> >   in 
> > https://www.gnu.org/software/texinfo/manual/texi2any_api/html_node/Loading-Init-Files.html
> > 2) for htmlxref files
> > 
> > We could make this list of directories a customization option, like we
> > currently do with INCLUDE_DIRECTORIES, which is influenced by -P and -I,
> > for example named TEXINFO_LANGUAGE_DIRECTORIES, and add a command line
> > option to prepend to this list?

I think that having a customization option is a good thing in any case,
but I am not sure about a command line option.

> One possiblility is to use --conf-dir to look for htmlxref.cnf files as
> well, although I haven't thought about this deeply.

I am not sure either.  It avoids adding a new option, but this is not
exactly the same type of directory.

-- 
Pat

Re: Texinfo 7.0 changed the name of HTML output directory

[Patrice Dumas]

On Sun, Aug 25, 2024 at 11:32:01AM +0100, Gavin Smith wrote:
> It's actually worse.  It's not just a matter of placing the htmlxref.cnf
> file in the same directory as the Texinfo source.  It has to be present
> in the working directory (.) where texi2any runs.  (I just tested
> this with current git 'master'.)  
> 
> So I can see a few ways to use the same htmlxref.cnf file:
> 
> * In the build process for the web manuals, copy the project-specific
> htmlxref.cnf file to the current working directory.  How this is done
> would depend on the project.
> 
> * texi2any checks in ~/.texinfo/ where ~ is the "current user's home
> directory" according to the manual.  This is the value of the HOME
> environment variable.  Hence, you could override HOME.  If I have the
> files "subdir/.texinfo/htmlxref.cnf" and "manual/manual.texi", then
> I can run:
> 
> HOME=subdir texi2any manual/manual.texi --html --no-split
> 
> This is a hack but may be the best way at the moment.  I think we
> should have a better way of supporting this if there isn't one already.
> We should at least support the "XDG Base Directory Specification".

Agreed.

> * Use HTMLXREF_FILE.  Below I explain why this variable is probably not
> what you want to use.
> 
> (HTMLXREF_FILE should possibly be used along with HTMLXREF_MODE=file although
> this seemed not to be necessary when I tested it.)

If HTMLXREF_MODE is not file, then HTMLXREF_FILE is used instead of
htmxref.cnf as the base file name.

> For example, if both "manual.texi" and "htmxref.cnf" are in "..", then
> 
>   texi2any ../manual.texi --html --no-split

This could be changed, we could add the manual source directory to the
htmlxref.cnf search path (actually the @language_config_dirs, see below),
for example, although only after the input file name directory has been
determined, so right before parsing (as we do for include files). 

> doesn't read htmlxref.cnf, whereas
> 
>   texi2any ../manual.texi --html --no-split -c HTMLXREF_FILE=../htmlxref.cnf
> 
> does.

This is not how HTMLXREF_FILE is supposed to be used/to work with
default HTMLXREF_MODE.  Normally (and according to my reading of the
code), HTMLXREF_FILE will be prepended to the directories, so the
files tried should be:

../htmlxref.cnf
./.texinfo/../htmlxref.cnf
$HOME/../htmlxref.cnf
SYSCONFDIR/texinfo/../htmlxref.cnf
DATADIR/texinfo/../htmlxref.cnf

> Unfortunately, the file specified by HTMLXREF_FILE is the *only* htmlxref.cnf
> file that is read, so this variable should probably not be used.

I do not think that it is true, however the directories searched are not
the expected ones, so I think that it is wrong to have HTMLXREF_FILE
specified with a relative path to lead to that list of files to be
looked at.  I think that this should be changed.  I see two
possibilities, either disallow completly relative paths, or if there is
a relative path only try it and not any other path, which is already
what is done with an absolute path, which actually has my preference
(and we already do something like that in another setting).  In any case,
this is not the way to go for this use case for the reasons you say, it
only works to find one file.

> I suggest that we should add a warning to texi2any any time an htmlxref.cnf
> entry is missing for a manual and we revert to the default "../MANUAL_html/"
> link.  That way manual authors would be aware of such problems.  We
> could add an option to turn the warning off for special uses.

This is already possible, it is CHECK_HTMLXREF (which is set to 1 in the
default case for EPUB as relative paths in EPUB are not possible).

> > There is the HTMLXREF_FILE variable we could use to point to a single
> > htmlxref.cnf file, but was this variable supported starting from the
> > same Texinfo version which introduced htmlxref.cnf?  It doesn't sound
> > that way: htmlxref.cnf is mentioned in NEWS under Texinfo 5.0, whereas
> > HTMLXREF_FILE is mentioned only under Texinfo 7.0.  So using this
> > variable would mean we'd need to restrict HTML generation to Texinfo
> > 7.0 and newer, right?
> 
> Yes.

Also, I think that there is no situation in which it would solve this use
case.

We should probably implement the two changes proposed above, to me they
are improvements.  But I do not think that they will solve this use case
either, if I understood well.

To me, to solve this use case, a natural solution would be the
possibility to prepend a directory to the list of directories searched
for htmlxref.cnf, such that all the manuals in a project can use the
same htmlxref.cnf to override or add to the default htmlxref.cnf.
However, this is not possible for now.  For other directory lists, we
have this possibility, with -I (and we even have -P) for @include and
images and for init files there is --conf-dir.

htmlxref.cnf is searched for in a list of directories that I called
@language_config_dirs in texi2any, with the idea that this list of
directory could be (at least partly) implementation independ

Re: Texinfo 7.0 changed the name of HTML output directory

[Patrice Dumas]

On Sun, Aug 25, 2024 at 11:33:45AM +0100, Gavin Smith wrote:
> 
> I think it is most likely a mistake.

Another possibility would be that the manual name got renamed and the
old name is used here.  But even if it was the case, the new name should
be there too, and there should be a comment explaining the situation.

-- 
Pat

Re: removing the HTML Xref Mismatch node

[Patrice Dumas]

On Fri, Aug 23, 2024 at 11:01:25PM +0200, Patrice Dumas wrote:
> Hello,
> 
> While we are on the subject of HTML Xref, I think that the "HTML Xref
> Mismatch" node should be removed
> 
>  
> https://www.gnu.org/software/texinfo/manual/texinfo/html_node/HTML-Xref-Mismatch.html

Here is what I propose.  Could you please check the change in the
last paragraphs of HTML Xref Link Basics?  Here is that text obtained
after applying the patch:

 Whether the present manual is split or mono is determined by user
 option; @command{texi2any} defaults to split, with the
 @option{--no-split} option overriding this.
 Whether the referent manual is split or mono, however, is another bit
 of the external information (@pxref{HTML Xref Configuration}).

 By default, @command{texi2any} uses the same form of the referent manual as the
 present manual.  Thus, there can be a mismatch between the format of the
 referent manual that the generating software assumes, and the format it's
 actually present in.  If there is no external information, both mono
 and split form should be generated to be sure that the cross-references
 work irrespective of the split option.  This is something the software
 which generated the @emph{referent} manual has to do in advance, it's
 not something the software generating the cross-reference in the present
 manual can control.

-- 
Pat
diff --git a/doc/texinfo.texi b/doc/texinfo.texi
index 71a34c32ae..f03777aa0f 100644
--- a/doc/texinfo.texi
+++ b/doc/texinfo.texi
@@ -17495,7 +17495,6 @@ @node HTML Xref
 * Node Expansion:HTML Xref Node Name Expansion.
 * Command Expansion: HTML Xref Command Expansion.
 * 8-bit Expansion:   HTML Xref 8-bit Character Expansion.
-* Mismatch:  HTML Xref Mismatch.
 * Configuration: HTML Xref Configuration.
 @end menu
 
@@ -17573,15 +17572,18 @@ @node HTML Xref Link Basics
 Whether the present manual is split or mono is determined by user
 option; @command{texi2any} defaults to split, with the
 @option{--no-split} option overriding this.
-
 Whether the referent manual is split or mono, however, is another bit
-of the external information (@pxref{HTML Xref Configuration}).  By
-default, @command{texi2any} uses the same form of the referent manual
-as the present manual.
+of the external information (@pxref{HTML Xref Configuration}).
 
-Thus, there can be a mismatch between the format of the referent
-manual that the generating software assumes, and the format it's
-actually present in.  @xref{HTML Xref Mismatch}.
+By default, @command{texi2any} uses the same form of the referent manual as the
+present manual.  Thus, there can be a mismatch between the format of the
+referent manual that the generating software assumes, and the format it's
+actually present in.  If there is no external information, both mono
+and split form should be generated to be sure that the cross-references
+work irrespective of the split option.  This is something the software
+which generated the @emph{referent} manual has to do in advance, it's
+not something the software generating the cross-reference in the present
+manual can control.
 
 
 @node HTML Xref Node Name Expansion
@@ -17816,57 +17818,14 @@ @node HTML Xref 8-bit Character Expansion
 web.
 
 
-@node HTML Xref Mismatch
-@subsection HTML Cross-reference Mismatch
-@cindex HTML cross-references @subentry mismatch
-@cindex Mismatched HTML cross-reference source and target
-
-As mentioned earlier (@pxref{HTML Xref Link Basics}), the generating
-software may need to guess whether a given manual being cross-referenced is available in split or monolithic form---and, inevitably,
-it might guess wrong.  However, when the @emph{referent} manual is
-generated, it is possible to handle at least some mismatches.
-
-In the case where we assume the referent is split, but it is actually
-available in mono, the only recourse would be to generate a
-@file{manual_html/} subdirectory full of HTML files which redirect back to
-the monolithic @file{manual.html}.  Since this is essentially the same
-as a split manual in the first place, it's not very appealing.
-
-On the other hand, in the case where we assume the referent is mono,
-but it is actually available in split, it is possible to use
-JavaScript to redirect from the putatively monolithic
-@file{manual.html} to the different @file{manual_html/node.html} files.
-Here's an example:
-
-@example
-function redirect() @{
-  switch (location.hash) @{
-case "#Node1":
-  location.replace("manual_html/Node1.html#Node1"); break;
-case "#Node2" :
-  location.replace("manual_html/Node2.html#Node2"); break;
-@dots{}
-default:;
-  @}
-@}
-@end example
-
-Then, in the @code{} tag of @file{manual.html}:
-
-@example
-
-@end example
-
-Once again, this is something the software which generated the
-@emph{referent} manual has to do in advance, it's not something the

Re: Texinfo 7.0 changed the name of HTML output directory

[Patrice Dumas]

On Sat, Aug 24, 2024 at 08:26:13AM +0300, Eli Zaretskii wrote:
> > Date: Fri, 23 Aug 2024 22:17:33 +0200
> > From: Patrice Dumas 
> > 
> > +The defaults for the @var{host} and @var{dir} correspond with 
> > cross-references
> > +links for a collection of manuals located side by side on a single host.  
> > This
> > +is different from the usual case of HTML manuals published on the World 
> > Wide
> > +Web served from different hosts in directories chosen by the publisher, for
> > +which the @var{host} and @var{dir} must be specially defined to have 
> > working
> > +cross-references.  However this requires external information and is 
> > detailed
> > +later on (@pxref{HTML Xref Configuration}).
> 
> This part is unclear, I could not understand what it means and how
> that affects me as the user of Texinfo.

Maybe I'll let Gavin propose something...

> > +If you want to generate cross-references to HTML manuals published on
> > +the World Wide Web, the @var{host} and @var{dir} parts of
> > +the cross-reference need to be known by Texinfo converters to be able
> > +to generate cross-references.  More generally you may want to specify
> > +the location of other manuals for cross-references.
> 
> This is written from the wrong POV, IMO.  The correct POV is that of a
> manual author/maintainer who wants cross-references from _other_
> manual to his/her one to be correct.  So I suggest the following
> rewording.:
> 
>   For other manuals published on the World Wide Web to be able to have
>   correct cross-references to your manual, the @var{host} and
>   @var{dir} parts of the cross-reference need to be known by Texinfo
>   converters when they produce HTML for those other manuals.
>   Similarly, for your manual in HTML format to have correct
>   cross-references to other manuals, @command{texi2any} needs to know
>   how to construct the URLs for other manuals.  This information is
>   provided by @file{htmlxref.cnf}.

I think that the two points of view need to be addressed.  I'll try to
propose something.

> > +Right now, the @file{htmlxref.cnf} file distributed with GNU Texinfo serves
> > +as the main resource to locate Texinfo HTML manuals in the World Wide Web.
> > +Since it is installed in a location used by @command{texi2any}, HTML 
> > manuals
> > +information found in this file will be used for cross-references by 
> > default.
> > +
> >  If you have additions or corrections to the @file{htmlxref.cnf}
> >  distributed with Texinfo, please email @email{bug-texinfo@@gnu.org} as
> > -usual.  You can get the latest version from
> > +usual.  If you publish a Texinfo HTML manual on the World Wide Web,
> > +having an up-to-date location listed in @file{htmlxref.cnf} should ensure 
> > that
> > +all HTML manuals generated by @command{texi2any} use this location for
> > +cross-references in the default case.  You can get the latest version from
> >  @url{http://ftpmirror.gnu.org/@/texinfo/@/htmlxref.cnf}.
> 
> I suggest to tell here explicitly that having a local htmlxref.cnf
> file with the correct information is the solution for outdated or
> missing data in the file as distributed by Texinfo.

Good idea.

-- 
Pat

Re: removing the HTML Xref Mismatch node

[Patrice Dumas]

On Sat, Aug 24, 2024 at 08:43:49AM +0300, Eli Zaretskii wrote:
> > Date: Fri, 23 Aug 2024 23:01:25 +0200
> > From: Patrice Dumas 
> > 
> > While we are on the subject of HTML Xref, I think that the "HTML Xref
> > Mismatch" node should be removed
> > 
> >  
> > https://www.gnu.org/software/texinfo/manual/texinfo/html_node/HTML-Xref-Mismatch.html
> > 
> > The information is still true, but I do not think that the solution
> > proposed is useful in practice as:
> > 1) in general both split and non-split manuals are generated
> > 2) with htmlxref.cnf the nature of the target manual is known
> > 
> > I suggest removing the node from the manual, and put the code example
> > somewhere else (maybe in tp/TODO).
> > 
> > Opinions?
> 
> I think the situation that node tries to "solve" is basically
> insoluble.  I think when producing a manual that references other
> manuals, the person who runs texi2any must know the form (mono, node,
> etc.) of the referent manual(s).

The above mentionned trick could have allowed for the creator of a split
manual to redirect mono references to split files.  That way, when
another manual links to that manual with links assuming a mono
manual they would still find the split manual.  This would lift the
necessity to know the form of the referent manual, but this is not
relevant in practice as, as you noted below, in general, the form is
known in htmlxref.cnf.

> My problem is that even if I know that, I'm not sure I understand how
> to use that information at HTML-generation time, unless all the
> referent manuals are split the same as the manual I'm producing (this
> is the only situation clearly described in then Texinfo manual, and
> the default operation of texi2any).  But what if the referent manuals
> use different splitting, let alone if they use splitting different
> from one another (e.g., the manual being produced is node-split,
> whereas some referent manuals use chapter-splitting, some others
> section-splitting, and some mono)?  How can I tell this to texi2any to
> have it generate correct references?  It sounds like the only way is
> to have a local htmlxref.cnf file that mentions _only_ the actual
> splitting of each referent manual?  If this is the way to go, the
> Texinfo manual should say that explicitly.  And if even this will not
> solve the problem, then IMO a solution should be designed and
> implemented.

For mono versus non-mono, having a local htmlxref.cnf file that mentions
_only_ the actual splitting of each referent manual is indeed the only
way to get correct cross-references.  The different split possibilities
(node, section, chapter), however, should be equivalent.  Indeed, if a
split manual is created using ‘--node-files’, targets files for each
node redirecting to the real file are generated for each node even if
the manual is split by section or by chapter.

> In this connection, this text of "HTML Xref Configuration" is
> completely unclear to me:
> 
>   However, if a manual is not available in that form, anything that is
>   available can be used.  Here is the search order for each style:
> 
>node⇒ node,section, chapter, mono
>section ⇒ section, chapter, node,mono
>chapter ⇒ chapter, section, node,mono
>mono⇒ mono,chapter, section, node
> 
>  These section- and chapter-level cross-manual references can succeed
>   only when the target manual was created using ‘--node-files’; this is
>   the default for split output.
> 
> When is this search performed and by whom?  Is it even relevant to the
> issue of cross-manual references, and if so, how?  And what does the
> last sentence about --node-files want to say?

It is used when generating the manual that links to the target manual,
and the sentence about --node-files is about what I explained above.
I'll try to propose changes to clarify this part.

-- 
Pat

Re: References in section titles (invalid HTML generated)

[Patrice Dumas]

On Thu, Aug 22, 2024 at 04:18:24PM +0100, Gavin Smith wrote:
> On Thu, Aug 22, 2024 at 03:42:20PM +0200, Rudolf Adamkovič wrote:
> > Alternatively, if references in titles cannot possibly be supported,
> > then Texinfo should warn about them, instead of generating invalid HTML.
> 
> References inside section headings was discussed before and it seemed
> that we were going to allow this nesting:
> 
> https://lists.gnu.org/archive/html/bug-texinfo/2023-02/msg6.html
> 
> The incorrect HTML output is definitely a bug though.

Some context seems to be missing, but some Texinfo constructs
necessarily lead to invalid HTML, as HTML forbids nested , while it
may be the natural conversion.  My check with firefox showed that this
construct was correctly rendered.

There is, for example @url in @xref, but also, if I recall well, @*ref
in @section and similar.

-- 
Pat

removing the HTML Xref Mismatch node

[Patrice Dumas]

Hello,

While we are on the subject of HTML Xref, I think that the "HTML Xref
Mismatch" node should be removed

 
https://www.gnu.org/software/texinfo/manual/texinfo/html_node/HTML-Xref-Mismatch.html

The information is still true, but I do not think that the solution
proposed is useful in practice as:
1) in general both split and non-split manuals are generated
2) with htmlxref.cnf the nature of the target manual is known

I suggest removing the node from the manual, and put the code example
somewhere else (maybe in tp/TODO).

Opinions?

-- 
Pat 

Re: Texinfo 7.0 changed the name of HTML output directory

[Patrice Dumas]

On Fri, Aug 23, 2024 at 07:58:04PM +0200, Patrice Dumas wrote:
> I'll try to come up with some proposal.

Here it is.  I may have been too explicit and redundant.

-- 
Pat
diff --git a/doc/texinfo.texi b/doc/texinfo.texi
index 71a34c32ae..2556115ef8 100644
--- a/doc/texinfo.texi
+++ b/doc/texinfo.texi
@@ -17517,7 +17517,16 @@ @node HTML Xref Link Basics
 (@pxref{Cross References}), and from @dfn{external information}
 (@pxref{HTML Xref Configuration}).
 
-We now consider each part in turn.
+The defaults for the @var{host} and @var{dir} correspond with cross-references
+links for a collection of manuals located side by side on a single host.  This
+is different from the usual case of HTML manuals published on the World Wide
+Web served from different hosts in directories chosen by the publisher, for
+which the @var{host} and @var{dir} must be specially defined to have working
+cross-references.  However this requires external information and is detailed
+later on (@pxref{HTML Xref Configuration}).
+
+We start with the case of manuals all located on the same
+host and consider each part in turn.
 
 The @var{host} is hardwired to be the local host.  This could either
 be the literal string @samp{localhost}, or, according to the rules for
@@ -17872,6 +17881,12 @@ @node HTML Xref Configuration
 @cindex Cross-reference configuration, for HTML
 @cindex Configuration, for HTML cross-manual references
 
+If you want to generate cross-references to HTML manuals published on
+the World Wide Web, the @var{host} and @var{dir} parts of
+the cross-reference need to be known by Texinfo converters to be able
+to generate cross-references.  More generally you may want to specify
+the location of other manuals for cross-references.
+
 @command{texi2any} reads a file named @file{htmlxref.cnf} to gather
 information for cross-references to other manuals in HTML output.  It
 is looked for in the following directories:
@@ -17976,9 +17991,17 @@ @node HTML Xref Configuration
 only when the target manual was created using @option{--node-files};
 this is the default for split output.
 
+Right now, the @file{htmlxref.cnf} file distributed with GNU Texinfo serves
+as the main resource to locate Texinfo HTML manuals in the World Wide Web.
+Since it is installed in a location used by @command{texi2any}, HTML manuals
+information found in this file will be used for cross-references by default.
+
 If you have additions or corrections to the @file{htmlxref.cnf}
 distributed with Texinfo, please email @email{bug-texinfo@@gnu.org} as
-usual.  You can get the latest version from
+usual.  If you publish a Texinfo HTML manual on the World Wide Web,
+having an up-to-date location listed in @file{htmlxref.cnf} should ensure that
+all HTML manuals generated by @command{texi2any} use this location for
+cross-references in the default case.  You can get the latest version from
 @url{http://ftpmirror.gnu.org/@/texinfo/@/htmlxref.cnf}.
 
 

Re: Texinfo 7.0 changed the name of HTML output directory

[Patrice Dumas]

On Fri, Aug 23, 2024 at 06:07:10PM +0100, Gavin Smith wrote:
> On Fri, Aug 23, 2024 at 06:29:04PM +0200, Patrice Dumas wrote:
> > Another possibility could be to leave the HTML Xref specification with
> > manual_html/ but document that the directory portion of the "HTML Xref
> > specification" is not important except for local collections of manuals
> > and that in practice the directory name should be chosen based on the
> > desired location and htmlxref.cnf should be used instead of the "HTML
> > Xref specification" to resolve the directory part of the specification
> > in general, be it only because otherwise the manual cannot be a target
> > for cross-references of manuals from other hosts.
> 
> Yes, that makes sense.
> 
> Another thing in the "HTML Xref Link Basics" node that looks wrong is
> the following text:
> 
>http://HOST/DIR/FILE.html#TARGET
> 
>   ...
>   
> The HOST is hardwired to be the local host.  This could either be the
>   literal string ‘localhost’, or, according to the rules for HTML links,
>   the ‘http://localhost/’ could be omitted entirely.

It is not wrong, but it is for the case of a collection of manuals in a
computer.

> This is not an accurate description of most intermanual HTML links
> where HOST is not "the local host".

Agreed.

I'll try to come up with some proposal.

-- 
Pat

Re: Texinfo 7.0 changed the name of HTML output directory

[Patrice Dumas]

On Fri, Aug 23, 2024 at 05:18:25PM +0100, Gavin Smith wrote:
> > If the common case is to rename "eglot_html" to "eglot", then why is
> > the default "eglot_html"?  In my book, defaults should be identical to
> > what happens in common cases, otherwise we force everyone and their
> > dog to customize texi2any.  No?
> 
> I assume that the "_html" suffix marks the output directory as containing
> HTML so is useful to have when the HTML output appears among other files
> in a directory on someone's computer.

Also it leaves out the eglot/ directory that can be used for something
else than a specific format output.

> >  IOW, I'd expect the version of
> > htmlxref.cnf distributed by Texinfo to be mostly empty, with the
> > possible exception of G and GS variables, and possibly other such
> > variables that document the site of each manual, like BINUTILS, EMACS,
> > etc. -- so that cross-manual references that point to another project
> > could be correct.  But the "manual information" lines should not be
> > present in this file for most if not all manuals, AFAUI, if the
> > defaults are reasonable.
> 
> It is only two links, "mono" and "node" for most manuals and we have
> to say which webpage the manuals are found on.  I doubt there is
> much benefit to trying to abbreviate the htmlxref.cnf format any, if
> I understand what you said correctly.

If I understand correctly, it would also require defining collections of
manuals somewhere so the gain would not be that important.

> Changing the way texi2any outputs such links seems easy to implement.
> In Texinfo 6.8, in HTML.pm:_external_node_href, the following lines
> gave the default output:

...

> We might need to go back to the way it was done before and not append
> this suffix.  I don't know if it is worth adding a customization variable
> for this.

I do not think that this is the way to go.  If we go back on something,
I think that it would better be the HTML Xref specification, and not the
output directory.  I think that using consistently the output format
name in the output directory is much better than what was done before.

-- 
Pat

Re: Texinfo 7.0 changed the name of HTML output directory

[Patrice Dumas]

On Fri, Aug 23, 2024 at 04:40:37PM +0100, Gavin Smith wrote:
> I said at the time that it makes little sense to have "_html" in
> URLs:
> 
> https://lists.gnu.org/archive/html/bug-texinfo/2022-02/msg4.html
> 
> The part of the "HTML Xref specification" mentioned here dates only to
> 2022 and actually doesn't matter that much as usually htmlxref.cnf will
> be used instead.  I don't think we should be encouraging people to put
> "_html" in their URLs.  (It's obvious that it's an HTML page if people
> are accessing it with a web browser.)  It is not the important part of
> this specification as far as I know - it's what is done with node names
> that is the important part.
> 
> I hope that people didn't feel encouraged to change the web URL
> of manuals based on what was written in the "HTML Xref specification".

I did not really take your point into account in the end, I thought that
the counter argument that consistency between default output directory
and HTML Xref specification was more important, given that the "HTML
Xref specification" directory is mainly relevant when the host is
localhost, htmlxref.cnf should be used otherwise.  But this could be
changed back (maybe for 8.0, given what I said before ;-), together
with some explanation of why there is a discrepancy between the "HTML
Xref specification" and the default split output directory.

Another possibility could be to leave the HTML Xref specification with
manual_html/ but document that the directory portion of the "HTML Xref
specification" is not important except for local collections of manuals
and that in practice the directory name should be chosen based on the
desired location and htmlxref.cnf should be used instead of the "HTML
Xref specification" to resolve the directory part of the specification
in general, be it only because otherwise the manual cannot be a target
for cross-references of manuals from other hosts.

Any preference?

-- 
Pat

Re: Texinfo 7.0 changed the name of HTML output directory

[Patrice Dumas]

On Fri, Aug 23, 2024 at 03:53:35PM +0300, Eli Zaretskii wrote:
> > For the reference, it was discussed here:
> > https://lists.gnu.org/archive/html/bug-texinfo/2022-02/msg3.html
> 
> That discussion failed to examine the existing practices, and failed
> to draw the attention of the relevant package maintainers to the fact
> that their practices were different.
> 
> I'm quite sure that you and Gavin are aware of the FSF infrastructure
> that GNU projects use to upload their manuals to the GNU site and the
> fact it uses CVS as its storage, and therefore I'd expect you to look
> there and compare the actual trees with what this change was about to
> do.

We did not do that, however, for the packages that use gendoc.sh or use
the "html" target in Makefile generated by automake, the change of the
default output directory, by itself does not change the trees, because
in both cases -o/--output is used, such that the default directory is
not used to determine the split HTML output directory.

> > We noted at that time that manuals generated by gendoc.sh and automake
> > generated html targets were not affected directly as they used --output,
> > in a way that was already incompatible with the HTML Xref specification.
> > Many GNU projects use gendoc.sh, but not all, in particular big projects
> > tend to use their own systems.  I have not checked, but I guess that non
> > GNU project often use the texi2any defaults.
> 
> I don't think I follow.  A typical GNU project has an "html" target in
> its Makefile, which invokes makeinfo/texi2any, not gendoc.sh.

As said above, if this html target comes from automake, it uses
-o/--output in makeinfo/texi2any invokation, such that the change in
default output directory do not have any effect.

Regarding gendoc.sh, I do not know what is done by package maintainers,
but the fact is that many GNU packages have their HTML manuals obtained
through https://www.gnu.org/manual/manual.html generated by gendoc.sh,
which leads to a typical gendoc generated page like:
 https://www.gnu.org/software/cpio/manual/

It is less the case for 'big' projects (emacs, lilypond, gcc, gnome,
grub...).  And also probably non GNU manuals do not use much gendoc.

>  And how
> does the --output flag affect the cross-references to other manuals?

It does not directly.  But if the directory given to --output does not
follow the HTML Xref specification, which is the case both for gendoc.sh
generated HTML and the automake generated html target, you need to use
htmlxref.cnf to specify the actual split manuals base URLs, unless you
move HTML files afterwards to the HTML Xref specified location.

> > All the changes in the HTML Xref specification directory lead to that
> > kind of unfortunate incompatible change.
> 
> Then why make such changes at all?  What was so bad about the previous
> defaults?

The rational was exposed in the thread aforementionned that I also
summarised below.

> > Here, you can update all the
> > manuals at once, however, even if it is a hassle.  But for links to
> > truely external manuals it is worse, as there is no possibility of
> > changing already generated manuals.  htmlxref.cnf can be used for
> > external manuals, but it needs to be changed two time, before and after
> > the target manual is regenerated with the new HTML Xref specification.
> 
> Exactly.  Which once again tells me this stuff should not be changed,
> not unless there's a very significant problem with the existing
> defaults.

There was.

> > Here it depends.  If all the manuals are generated using the same HTML
> > Xref specification, it should be ok -- though is still may require
> > change in the build/VCS/... as you report.
> 
> You cannot rely on everybody using the latest version of Texinfo.
> This is impractical and unreasonable.

This is not needed, what is needed is much less stringent.  First it is
only for pairs of manuals for which the target manual do not use
htmlxref.cnf.  Then for such a pair of manual, what matters is the HTML
Xref specification of the Texinfo release, not the Texinfo release.  The
HTML Xref specification changes much more slowly than the Texinfo
release.  In practice, the specification has changed in the 4.7 release,
when it was designed, and in 7.0 release (it changed in 7.1 too, but for
@sc and @point only).  That is not a dramatic requirement.  One could
even consider that this is the type of change to expect for first digit
change release.

>  For example, I'm using one of
> the GNU servers to produce Emacs releases and Emacs manuals to go with
> them, but I'm not the sysadmin of that server, so I cannot always make
> sure I have the latest Texinfo (nor do I always want to have it: for
> example, a bugfix release had better used the same Texinfo version as
> the original one, to avoid introducing new issues).

Of course, this can happen, it is not always possible nor practical to
coordinate on a HTML Xref specification to use for all the manuals with
cross-references t

Re: EPUB 3.3 spec conformity issues

[Patrice Dumas]

On Thu, Aug 22, 2024 at 12:31:47PM +0100, Daniel Cerqueira wrote:
> Patrice Dumas  writes:
> 
> This seems like a progress to me.
> 
> Regarding having other issues, the dcterms:modified and the border of
> the index table element, were the only issues that I encountered when
> producing an EPUB file from my book's Texi file.

I remember other issues, corresponding with some constructs that are not
valid in the new HTML specification and that epubcheck requires that
will have to be addressed too, I think.

What I will do when the strict epubcheck conformance is implemented is
to setup a script that converts all the manuals used in the Texinfo test
suite to epub and run epubcheck on the result to get all the conformity
issues (there is something similar for generated Texinfo XML conformance
check).  I actually thought that I had already done that when
developping the EPUB output, but I can't find any trace of it.

-- 
Pat

Re: EPUB 3.3 spec conformity issues

[Patrice Dumas]

On Thu, Aug 22, 2024 at 10:32:05AM +0200, Patrice Dumas wrote:
> Hello,
> 
> Here is my proposal:
> 
> Does it look ok?

Here is a commit that 'implements' that proposal:

https://git.savannah.gnu.org/cgit/texinfo.git/commit/?id=e142a9015c7b403b10207f77acacad72a88d01ef

-- 
Pat

Re: EPUB 3.3 spec conformity issues

[Patrice Dumas]

On Thu, Aug 22, 2024 at 01:13:06AM +0200, Patrice Dumas wrote:
> 
> My current thinking is that the best would be a way to specify the last
> modification time in the manual 'manually', and if users want to modify it
> using an external information, such as the EPUB file generation time or
> the Texinfo file modification time, they should set it up in their build
> process, and we should make it as easy as possible by designing how it
> is specified in Texinfo accordingly and/or adding customization
> variables to support the most wanted use-cases.

After more thinking, I think that the best would be to have two dates in
metadata, one for publication (this one would be used in EPUB for
example) and a date of modification of the Texinfo manual (what is
currectly set with automake and accessible as @value{UPDATED}).

For the publication date, what the user would put in the (still to
designed and implemented) metadata would be used first, and if there is
none what you proposed in your patch, ie using the date of manual
generation seems to be the best.

For the manual change date, again the user-specified value would take
precedence, and then the modification timestamp of the main texinfo file
(or the newest for the main and all @include'd files) would be the
default.

-- 
Pat

Re: Texinfo 7.0 changed the name of HTML output directory

[Patrice Dumas]

On Thu, Aug 22, 2024 at 10:32:41PM +0300, Eli Zaretskii wrote:
> Texinfo 7.0 changed the name of the directory where texi2any outputs
> split-HTML files for a manual:
> 
>   7.0 (7 November 2022)
>   * texi2any
>   [...]
>. HTML output:
>. use manual_name_html as output directory for split HTML instead of
>manual_name or manual_name.html

For the reference, it was discussed here:
https://lists.gnu.org/archive/html/bug-texinfo/2022-02/msg3.html

We noted at that time that manuals generated by gendoc.sh and automake
generated html targets were not affected directly as they used --output,
in a way that was already incompatible with the HTML Xref specification.
Many GNU projects use gendoc.sh, but not all, in particular big projects
tend to use their own systems.  I have not checked, but I guess that non
GNU project often use the texi2any defaults.

> and references to other manuals now fail because the directory
> structure of Emacs manuals under manual/html_node/ has subdirectories
> named by the old convention: "manual_name", not "manual_name_html".
> 
> Worse, the directory structure of the Emacs manuals is actually a CVS
> tree, because all the Emacs manuals that are regenerated for each new
> release of Emacs are checked into CVS in the
> cvs.savannah.gnu.org:/web/emacs repository, and from there are posted
> on the GNU documentation site by the FSF infrastructure.  So now we
> need either to rename all the subdirectories in CVS (and risk losing
> the VCS history), or write a script that modifies the cross-manual
> references to omit the "_html" suffix.

All the changes in the HTML Xref specification directory lead to that
kind of unfortunate incompatible change.  Here, you can update all the
manuals at once, however, even if it is a hassle.  But for links to
truely external manuals it is worse, as there is no possibility of
changing already generated manuals.  htmlxref.cnf can be used for
external manuals, but it needs to be changed two time, before and after
the target manual is regenerated with the new HTML Xref specification.

> This is exactly the kind of backward-incompatible change that should
> have never been done, because it breaks on-line manuals that are
> regenerated for new releases,

Here it depends.  If all the manuals are generated using the same HTML
Xref specification, it should be ok -- though is still may require
change in the build/VCS/... as you report.

If --output is used to specify the directory to follow the HTML Xref
specification, it may not be corrected that way, but I think that using
--output is always more consistent with also using htmlxref.cnf to
specify the target location.

>  and the problem is a nuisance to fix if
> the manuals are hosted in VCS repositories, because a typical VCS has
> at best very weak support for renaming files, and at worst simply
> loses all history before the rename.

> I guess it's too late to lobby for reverting that change?

Indeed.  It could have been reverted before the published change in HTML
Xref, but now it is too late.  Also, the previous way lead either to a
weird result (with .html) or made HTML output special for no reason.

Overall, we try to avoid making changes in the HTML Xref specification
and it does not change that often, but from time to time it still needs
to change.

-- 
Pat

Re: EPUB 3.3 spec conformity issues

[Patrice Dumas]

Hello,

On Tue, Aug 20, 2024 at 09:23:57AM +0100, Daniel Cerqueira wrote:
> The EPUB file is produced without any errors, and when I check to see if
> the file is valid (using the `epubcheck` program) I get this 2 errors:

My current view on EPUB conformance is that seems to be important to
have full conformance in some cases (as Daniel reported) and even if I
think that having such a level of conformance is pointless or even
wrong, I also think that it is important to provide a possibility of
full epubcheck conformity, even if it is not the default.  For the
release, however, I think that it is too late.

Here is my proposal:

1) for the release, we do not make any change to the code, but we
document that the EPUB we produce is no conformant.  We ship Daniel
patches in contrib/ and we document that they exist too.

2) after the release we fix conformance issues as we can/want, for
instance use CSS for border, add meta data for a publication date. I am
pretty sure there were other issues.  If there are still conformance
issues and we do not want to change the default HTML output, we add a
customization variable for epubcheck strict conformity and add the
corresponding code.

Does it look ok?

-- 
Pat

Re: EPUB 3.3 spec conformity issues

[Patrice Dumas]

On Wed, Aug 21, 2024 at 10:27:54PM +0100, Daniel Cerqueira wrote:
> One common scenario:
> 
> I just finished my book. My texi file is finished and I won't modified
> it. I want to only change the first image of my EPUB, which is the
> interior cover image.
> 
> If you are generating the dcterms:modified based on the .texi file
> timestamp, the Renderer won't know the difference between your EPUB
> with the old cover and the EPUB with the new cover. This might have
> effects on the cover thumbnail of the EPUB file, making the Renderer
> not know that there is a new cover.
> 
> A solution to this is to always generate a new dcterms:modified upon
> EPUB compilation.

I agree that it is a potential use case that makes using the Texinfo
file modification time to set the time incorrect.  Another similar case
is if a user wants to set the modification date because of a change in
texi2any EPUB conversion in the published work.

However, I still do not think that it justifies changing the
dcterms:modified upon EPUB compilation.  This could lead to a change in
the modification date even if nothing of importance changed.

I still think that only a user can know when to change the last modified
date.  Changing it when the Texinfo file timestamp changes could be a
rule relevant for some users.  Your use-case, changing the last modified
date at each EPUB generation is also a valid use-case.  But I also think
that changing only sparingly when a user thinks that a change is needed
is also a possible use-case.

My current thinking is that the best would be a way to specify the last
modification time in the manual 'manually', and if users want to modify it
using an external information, such as the EPUB file generation time or
the Texinfo file modification time, they should set it up in their build
process, and we should make it as easy as possible by designing how it
is specified in Texinfo accordingly and/or adding customization
variables to support the most wanted use-cases.

> Also:
> 
> Texinfo PDF file generation changes every time a new build is made. Are
> you concerned about this? Don't seem like it.

I know next to nothing about PDF, in particular I have no idea what the
timestamp in it means semantically.

> Which brings us to your argument about reproducible builds... More on
> this latter.

On that subject, having non reproducible PDF files is a hurdle.

> If you don't follow the EPUB standard (for the sake of it) you should
> not be calling this generated file an EPUB. Create your own filetype,
> give it a name, and make Texinfo produce whatever that filetype is.
> And make Texinfo stop calling it an EPUB file.

I do not think that non-conformance is a reason enough to stop calling
the generated file an EPUB file.  My feeling is that it is more useful
to call it an EPUB file, as it indeed is an EPUB file, and it is
readable at least with some epub readers (I did check).  We could and
maybe should document better the conformity, though, as I am pretty sure
that even with your patch, epubcheck shows some non conformity for some
Texinfo manuals EPUB output.

> When there is some good faith person trying to contribute to make
> Texinfo project better, there you have an issue with reproducible
> builds. I am sensing poor Free Software project leadership.

This is not a theoretical issue, even before reproducible builds, we
need to have reproducible tests which requires some care in generating
output files.  Nothing insurmountable, but we need to make sure that
with setting some variable (typically the TEST customization variable)
we get a reproducible output.

Having the possibility to have reproducible output is also a relevant
goal that is clearly not, by itself, a mark of poor project leadership.

> My patches are simple, and it should take you very little time, less
> than 3 minutes, to figure out what they are doing.

Your patches are clean and simple, which is good, and indeed, it takes
little time to figure out what they are doing, however the issue here is
not only understanding your patches, but also being sure of, or at least
convinced that they fix the issues in the right way and that they do not
have unintended effects, for instance on other output formats or on
other aspects of the EPUB output.  (And again, there is the timing to
consider.)

-- 
Pat

Re: EPUB 3.3 spec conformity issues

[Patrice Dumas]

On Wed, Aug 21, 2024 at 09:57:28PM +0100, Daniel Cerqueira wrote:
> use CSS inside EPUB (don't know if it is possible and
> compliant),

It is:
https://www.w3.org/TR/epub-33/#sec-overview-relations-css

> Are you guys even using epubcheck on your generated EPUB files?

I think I already answered that, I did use epubcheck a lot when
developping the EPUB output.  I had a look at my mails and actually
we had discussions/tests with Kurt Hornik off-list on epubcheck validity
errors.  (As a side note, I am somewhat surprised that you do not have
epubcheck errors with width= in table too.)

> > the 'border' attribute is deprecated, but we would have to decide
> > whether
> > we wanted to use CSS instead for this style.
> 
> If you have not yet decided if you want to use CSS instead (and it has
> to work for EPUB files too, since this is the thread topic), why are
> you then discarding my patch?

We are not in such a hurry that we have to decide immediately and
implement immediately.  The change of table border requires thinking,
maybe discussions, implementation and testing.  For those in a hurry,
your patches are already there and can be used.

As a side note, I do think that using CSS for border is the way to go,
but that it is much better to do it after a release, not just before.

> And if you decide you don't want CSS for EPUB, will this patch remain
> in the trash? Probably.

I do not exactly understand who is "you" in your sentence.  GNU Texinfo
is a collaborative project, and although there are not that many
contributors, there are lots of users whose needs and wants, feedbacks
and tests are important for the development decisions and the
development process.  It is therefore difficult to predict design
decisions, and the future of a patch even more so.  There is also always
the possibility to fork (though in the case of Texinfo, sadly, this is
somewhat unrealistic, to be honest).

> Texinfo is producing non-compliant EPUB files!
> 
> (This should bother you enough to make something about it).

It does, and we are willing to improve the situation, but not a all
costs nor without considering the impact of changes on the overall
Texinfo project.

-- 
Pat

Re: EPUB 3.3 spec conformity issues

[Patrice Dumas]

On Wed, Aug 21, 2024 at 06:34:34PM +0100, Daniel Cerqueira wrote:
> 
> I contributed, or was trying to,

Even if your patches do not make it to the Texinfo release, they still
are a valuable contribution, as other users who want conformance can
retrieve this exchange and apply your patches, especially since your
patches are clean and small.  They could also be applied in
distributions (such as Fedora, Debien...) if the packages maintainers
find that conformance is needed in their distributions.

We also have a contrib directory in the Texinfo sources where we could
put your patches and document somewhere that they improve conformance of
generated epub.

> These online stores, need my EPUB file to be in conformance (and they
> rightly use EPUBCheck to test for this conformity). If the EPUB file is
> not in conformance, it won't be published.

That's clearly something to consider that we were not aware of.


As Gavin said, even if we favor conformance, we need to weight
against other effects on the project as a whole, in particular it should
fit as best as possible with the overall design of the Texinfo language
and the conversions codes, and the impact on other formats must also be
considered.

There is also the issue of the timing, we are very near a release, so we
should avoid changes that require some testing and having ourselves or
users looking at the generated HTML for diverse constructs and on
diverse browsers.

-- 
Pat

Re: EPUB 3.3 spec conformity issues

[Patrice Dumas]

On Wed, Aug 21, 2024 at 05:21:04PM +0100, Gavin Smith wrote:
> 
> It seems that EPUB standards compliance is more important (as far as
> EPUB itself is important) than HTML standards compliance.  The EPUB
> format appears to be fairly concisely and precisely defined, as far
> I can tell, when compared with HTML, and may be more stable.

For some parts of the EPUB standard, this is true, but for the content,
it is the mostly same as HTML, as it defers to the ever changing HTML
standard.

-- 
Pat

Re: EPUB 3.3 spec conformity issues

[Patrice Dumas]

On Tue, Aug 20, 2024 at 11:13:59PM +0100, Daniel Cerqueira wrote:
> Thinking more deeply, I have changed the HTML.pm patch.
> 
> The border property of the table element, if set to "1" (instead of ""),
> probably is best for HTML generation, and still produces a valid EPUB
> 3.3.

If we change the border to be 1 for printindex tables, it should be
because the output is better also in web browsers.  I am not sure that
it is the case.

Regarding the change of "0" to "", it looks wrong to me, no border
attribute has been specified with a "0" for a long time and correctly
rendered.  The border attribute is not in HTML5 anymore, but it is not a
reason to remove 0 as a possible value -- nor is it a reason to stop
generating it.

That being said, currently the border of tables output when formatting
@multitable, @printindex or @cartouche cannot be customized in HTML
texi2any output except by overriding the conversion functions.  I think
that it would be better to define classes and use CSS for the tables
borders.

-- 
Pat

Re: EPUB 3.3 spec conformity issues

[Patrice Dumas]

On Tue, Aug 20, 2024 at 09:23:57AM +0100, Daniel Cerqueira wrote:
>
> The dcterms:modified meta element is MANDATORY in order to create a
> valid EPUB 3.3 file. It is missing in the current Texinfo EPUB file
> creation. Here is an code example on how to implement this inside the
> EPUB file:
> 
> According to: https://www.w3.org/TR/epub-33/#sec-metadata-last-modified

I agree on the issue, but I do not think that the last modified date
should be the date of the epub file generation, it should rather be the
date of the publication contents or style modification.  None of the
Dublin core nor the EPUB format specification are clear on what the
modification date should correspond to in term of changes, but my
reading of the EPUB specification is that it is about the published
material, not about the epub file creation date.  I also think that it
is much better to have this date correspond to the change in contents
not to the generation.

To me this should be specified by the manual authors.  One possibility
could be to use the last modification time of the input Texinfo file,
possibly through automake setting the UPDATED flag, but this should be a
user's choice, not something automatic.

Adding metadata such as a last modified date to Texinfo is on the TODO
list since some time now, but I think that we are too near from the
release to change the Texinfo language now, this cannot change before
the release.

> I consider this to be a bug, because the EPUB file created by `texi2any`
> is not according to the specs.

We do not try to follow standards for the sake of it, but if there are
actual issues.  Here it would be better to be conforming, as the
modification date is used in readers, but it is not such a big issue
either, as it does not prevent the readers to render the epub files.

> A suggestion when programming this, is to always use the `epubcheck`
> program to validate the creation of EPUB (by texi2any), from now
> onwards.

There were already some reports on epub files validity when the epub
output was implemented and I do use epubcheck to find non conformance
(though not recently). But it does not means that the issue can be nor
should be fixed.

I will answer the border issue in another mail.

-- 
Pat

Re: Adding HTML navigation links [was Re: Suggestions for texi2any parsing]

[Patrice Dumas]

On Wed, Aug 14, 2024 at 06:40:10PM +0100, Gavin Smith wrote:
> It is easy enough, I expect, to achieve such links by post-processing
> the HTML output, but adding built-in support in texi2any to add external
> navigation links might be worth considering, in order to make it more
> convenient.

Not sure that it is convenient enough, but using a function for a button
is how it has been done for the Singular manual for ages.  An init file
based on the Singular manual has been kept for tests, it is in
tp/t/init/t2h_singular.init.  There, we can see the use of a function
for a button to add a link to an external url in the navigation buttons:

sub singular_banner
{
  my $result = '';
  $result .= <
EOT
  return ($result, 0);
}

my @SECTION_BUTTONS =
  (
   \&singular_banner,
   'Back', 'Forward',   'FastBack', 'FastForward',
   'Up', 'Top', 'Contents', 'Index', 'About'
  );




As a side note, it could be preferable to add to the existing buttons,
rather than set @SECTION_BUTTONS, as done in
t/init/special_element_customization.pm:

# avoid doing twice if there are more than one manual processed
my $button_added;
sub _texi2any_tests_special_unit_add_button
{
  my ($self, $document, $stage) = @_;

  if (!$button_added) {
my @section_buttons = @{$self->get_conf('SECTION_BUTTONS')};
push @section_buttons, (' ', 'Overview');
$self->set_conf('SECTION_BUTTONS', \@section_buttons);
$button_added = 1;
  }
}

texinfo_register_handler('setup', \&_texi2any_tests_special_unit_add_button);


-- 
Pat

Re: BUG: format_line_message: error_location_info undef

[Patrice Dumas]

On Fri, Aug 16, 2024 at 12:25:45AM +0100, Gavin Smith wrote:
> What makes the difference is the lack of the "-c INFO_SPECIAL_CHARS_WARNING=0"
> option.  This produces thousands of lines of errors.  Here is just the
> first few lines:
> 
> Texinfo-Convert-Plaintext.texi:1: warning: node next for 
> `Texinfo@asis{::}Convert@asis{::}Plaintext' is `Index' in sectioning but not 
> in menu
> texi2any_internals.texi:374: warning: node prev for `Index' is 
> `Texinfo@asis{::}Convert@asis{::}Plaintext' in sectioning but not in menu
> texi2any_internals.texi:374: warning: node up for `Index' is `Top' in 
> sectioning but not in menu
> texi2any_internals.texi:15: warning: node `Top' lacks menu item for `Index' 
> despite being its Up target
> BUG: format_line_message: error_location_info undef at 
> ../../tp/../tp/Texinfo/Report.pm line 86.
> Texinfo::Report::format_line_message("warning", "menu entry node name 
> should not contain `:'", undef, undef, 0) called at 
> ../../tp/../tp/Texinfo/Convert/Converter.pm line 489

This should be fixed by
 
https://git.savannah.gnu.org/cgit/texinfo.git/commit/?id=4174fe2155d679f935103d07d2e75c382d9d0e30

It would be somewhat strange if it fixes Dario issue, but who knows.
In any case it was a good thing to fix.

-- 
Pat

Re: BUG: format_line_message: error_location_info undef

[Patrice Dumas]

On Fri, Aug 16, 2024 at 12:25:45AM +0100, Gavin Smith wrote:
> BUG: format_line_message: error_location_info undef at 
> ../../tp/../tp/Texinfo/Report.pm line 86.
> Texinfo::Report::format_line_message("warning", "menu entry node name 
> should not contain `:'", undef, undef, 0) called at 
> ../../tp/../tp/Texinfo/Convert/Converter.pm line 489

Indeed, this needs to be fixed.  I started investigating it, as far as I
can tell, this bug corresponds to automatically added menu entries not
having source info.  I'll see how I can fix it.

-- 
Pat

Re: Fix an error during "./configure; make dist"

[Patrice Dumas]

On Sat, Aug 10, 2024 at 04:19:52PM +0100, Gavin Smith wrote:
> If the .pot files are deleted as part of this check it makes sense that
> the newly generated .pot files would have different time stamps.

Indeed.  So I think that we are good except for the autogen.sh issue I
reported in another thread.

I attach a list of recipes mainly based on Bruno, maybe this should be
added to the list of the things to check before a release?

-- 
Pat
 test distcheck after make
./autogen.sh
./configure CPPFLAGS="-Wall"
make
make check
make distcheck

= test distributed files not generated in builddir
./configure
make maintainer-clean
rm -rf bb
mkdir bb
cd bb
../configure
make
make distclean
find . -type f

== test distcheck after maintainer-clean
./configure
make maintainer-clean
./configure
make distcheck

== test distcheck in out of source build after maintainer-clean
./autogen.sh
./configure
make maintainer-clean
rm -rf bb
mkdir bb
cd bb
../configure
make distcheck


== differences between dist in and out source after maintainer-clean
rm -f texinfo-7.*.*.tar.gz
./autogen.sh
./configure
make maintainer-clean
./autogen.sh
./configure
make dist
rm -rf in_source_dist_contents
mkdir in_source_dist_contents
(cd in_source_dist_contents && tar xvf ../texinfo-7.*.*.tar.gz)
make maintainer-clean
rm -rf bb
mkdir bb
cd bb
../configure
make dist
tar xvf texinfo-7.*.*.tar.gz
diff -u -r ../in_source_dist_contents/texinfo-7.*.*/ texinfo-7.*.*/ > 
../build_in_out_source_differences.diff

Re: fix an error message during configure

[Patrice Dumas]

Hello,

The './autogen.sh' step in opencsw fails:
https://buildfarm.opencsw.org/buildbot/waterfall?category=texinfo

It fails since commit
https://git.savannah.gnu.org/cgit/texinfo.git/commit/?id=d695e0fbef88a285884ccd1f1661d13b75013a39

I think that it is because options like --install are missing.  I think
that doing what Bruno proposes below would fix this issue and is easier
to maintain:

On Thu, Aug 08, 2024 at 12:24:18AM +0200, Bruno Haible wrote:
> Oh, this problem again. The common workaround is to tell autoreconf
> not to run 'autopoint' [1]:
> 
> diff --git a/autogen.sh b/autogen.sh
> index 718100fa77..f1d3449aac 100755
> --- a/autogen.sh
> +++ b/autogen.sh
> @@ -48,7 +48,7 @@ $chicken eval $cmd "$@" || exit 1
>  
>  : ${LIBTOOLIZE=libtoolize}
>  cmd="(cd tp/Texinfo/XS && ${LIBTOOLIZE} \
> - && autoreconf --force --verbose --install)"
> + && AUTOPOINT=true autoreconf --force --verbose --install)"
>  echo "  $cmd"
>  $chicken eval $cmd || exit 1
>  
> 
> Bruno
> 
> [1] 
> https://www.gnu.org/software/gnulib/manual/html_node/gettextize-and-autopoint.html
> 
> 
> 
> 

Re: Fix an error during "./configure; make dist"

[Patrice Dumas]

On Sat, Aug 10, 2024 at 04:19:52PM +0100, Gavin Smith wrote:
> When I ran the commands you listed in an earlier email, there were several
> other differences reported in "build_in_out_source_differences.diff":
> 
> 
> diff -u -r ../in_source_dist_contents/texinfo-7.1.90/tp/Texinfo/Commands.pm 
> texinfo-7.1.90/tp/Texinfo/Commands.pm
> --- ../in_source_dist_contents/texinfo-7.1.90/tp/Texinfo/Commands.pm
> 2024-08-10 15:48:05.0 +0100
> +++ texinfo-7.1.90/tp/Texinfo/Commands.pm   2024-08-10 15:50:09.0 
> +0100
> @@ -1,4 +1,4 @@
> -# Automatically generated from ./maintain/regenerate_commands_perl_info.pl
> +# Automatically generated from 
> ../../tp/maintain/regenerate_commands_perl_info.pl
>  
>  package Texinfo::Commands;
>  
> I think this difference is fine.
> 
> I don't think there are any problems here to be fixed.

This should be fixed by commit
https://git.savannah.gnu.org/cgit/texinfo.git/commit/?id=584a597ebba3d542dc3ca54c50f79f8fde995d7e

-- 
Pat

Re: Fix an error during "./configure; make dist"

[Patrice Dumas]

On Sat, Aug 10, 2024 at 12:52:43PM +0200, Bruno Haible wrote:
> For the gnulib part, this is now fixed. Thanks for the report.

Thanks!

With Gavin change and some more I did on other generated files, we are
down to a difference in pdf files but also a difference in *.pot files
in POT-Creation-Date, which seems to be updated at each make dist.  I
attach the diff.

I had a look at internet on the POT-Creation-Date subject but found
nothing clear.  I don't know if it is the reason, but I think that some
files with translated strings are regenerated.

Any idea on what to do on that subject?  We could consider that it is
not an issue, for example.

-- 
Pat
Binary files ../in_source_dist_contents/texinfo-7.1.90/doc/refcard/txirefcard-a4.pdf and texinfo-7.1.90/doc/refcard/txirefcard-a4.pdf differ
Binary files ../in_source_dist_contents/texinfo-7.1.90/doc/refcard/txirefcard.pdf and texinfo-7.1.90/doc/refcard/txirefcard.pdf differ
diff -u -r ../in_source_dist_contents/texinfo-7.1.90/po/texinfo.pot texinfo-7.1.90/po/texinfo.pot
--- ../in_source_dist_contents/texinfo-7.1.90/po/texinfo.pot	2024-08-10 15:57:02.0 +0200
+++ texinfo-7.1.90/po/texinfo.pot	2024-08-10 15:58:14.0 +0200
@@ -8,7 +8,7 @@
 msgstr ""
 "Project-Id-Version: GNU texinfo 7.1.90\n"
 "Report-Msgid-Bugs-To: bug-texinfo@gnu.org\n"
-"POT-Creation-Date: 2024-08-10 15:57+0200\n"
+"POT-Creation-Date: 2024-08-10 15:58+0200\n"
 "PO-Revision-Date: YEAR-MO-DA HO:MI+ZONE\n"
 "Last-Translator: FULL NAME \n"
 "Language-Team: LANGUAGE \n"
diff -u -r ../in_source_dist_contents/texinfo-7.1.90/po_document/texinfo_document.pot texinfo-7.1.90/po_document/texinfo_document.pot
--- ../in_source_dist_contents/texinfo-7.1.90/po_document/texinfo_document.pot	2024-08-10 15:57:05.0 +0200
+++ texinfo-7.1.90/po_document/texinfo_document.pot	2024-08-10 15:58:18.0 +0200
@@ -8,7 +8,7 @@
 msgstr ""
 "Project-Id-Version: GNU texinfo 7.1.90\n"
 "Report-Msgid-Bugs-To: bug-texinfo@gnu.org\n"
-"POT-Creation-Date: 2024-08-10 15:57+0200\n"
+"POT-Creation-Date: 2024-08-10 15:58+0200\n"
 "PO-Revision-Date: YEAR-MO-DA HO:MI+ZONE\n"
 "Last-Translator: FULL NAME \n"
 "Language-Team: LANGUAGE \n"

Re: Fix an error during "./configure; make dist"

[Patrice Dumas]

On Fri, Aug 09, 2024 at 11:08:02PM +0100, Gavin Smith wrote:
> With the attached patch, it cd's to srcdir before running pod2texi.
> This way pod2texi with --subdir works correctly.

Looks like it should work as intended, indeed.

> In order for the change of working directory to work correctly, I had
> to make a few changes.  texi2any_internals.texi is listed as a Makefile
> target without the "$(srcdir)/" prefix.  This still works correctly
> due to the VPATH mechanism.  This allows correct use of $@ in the Makefile
> rule in the new working directory.
> 
> The files in $(texi2any_internals_dependencies) are now listed without the
> "$(top_srcdir)/" prefix; rather, they are given a "../../" prefix.  Again,
> the VPATH mechanism should allow these files to be found in the srcdir.
> The "../../" prefix is still correct for these files after "cd $(srcdir)",
> allowing the list of files to be passed to pod2texi correctly.

I recall that on some platform, a BSD I think, Make only used the VPATH
for the prerequiste (if I recall well), and that's why we systematically
added $srcdir for targets even though it would not have been needed
with VPATH.  But I think that your approach can still work, by not using $@
and having a different path in the target and rule:
$(srcdir)/texi2any_internals.texi: 
. > texi2any_internals.texi 

> Finally, I had to use absolute directory names (abs_top_srcdir and
> abs_top_builddir) in MAKEINFO_ENVIRONMENT so that they would still
> be still correct after the change of directory; otherwise, ModulePath.pm
> wouldn't be found.

Indeed.

> It's possible that the behaviour of pod2texi is wrong here

Actually I don't think so.

> but I don't
> have any suggestions as to how it should be changed.

> > Here is a recipe to get differences between in source and out of source
> > distributions.  There are some changes for generated files, which seem
> > innocuous, but may be an issue for reproductibility, as well as the
> > difference for texi2any_internals.texi.
> 
> Thanks, I'll try it.

If we really want to support make dist in out of source builds, I think
that we sould make sure that there are no differences at all between in
source and out of source distributions.  This should not be difficult
for some file we generate, but there are also some gperf generated files
in gnulib that have some srcdir prepended, which may be more challenging
to get right.  Maybe we should report that to gnulib, or Bruno, you
could have a look?

-- 
Pat

Re: Fix an error during "./configure; make dist"

[Patrice Dumas]

Hello,

The other cases raised by Bruno have been fixed, but this one still
fails:

On Tue, Aug 06, 2024 at 06:59:28AM +0200, Bruno Haible wrote:
> Here's the recipe that currently still fails:
> 
> $ ./autogen.sh
> $ ./configure
> $ make maintainer-clean
> $ git diff -- doc/html_default_commands_args.texi doc/pod2texi.texi 
> man/pod2texi.1 po_document/texinfo_document.pot 
> tp/Texinfo/XS/gnulib/lib/unicase/special-casing-table.h 
> tp/Texinfo/XS/gnulib/lib/uninorm/composition-table.h 
> tp/Texinfo/XS/main/command_ids.h tp/Texinfo/XS/main/element_types.c 
> tp/Texinfo/XS/main/element_types.h | patch -p1 -R
> $ mkdir bb
> $ cd bb
> $ ../configure
> $ make distcheck

My analysis is the following.  doc/tp_api/texi2any_internals.texi is
generated and distributed, but it is not under version control.  It is
removed by make maintainer-clean, rightly I think, but it is not
reinstated by the git diff reverse patch.

Therefore, when make distcheck in bb calls make dist,
texi2any_internals.texi is generated in the main doc/tp_api/ directory.
The --subdir option passed to pod2texi.pl is
--subdir=../../../doc/tp_api/api_includes, which allows to put the
generated files in $(srcdir)/api_includes, and looks right.

However, the --subdir option value is also used in pod2texi.pl for the
generated @include in texi2any_internals.texi.  Therefore, At this
point, we have a distributed file with includes like

  @include ../../../doc/tp_api/api_includes/Texinfo-Convert-Plaintext.texi

which is different from what would have been generated from an in source
build, which would be along

  @include api_includes/Texinfo-Convert-Plaintext.texi

Having a distributed file with differences is incorrect.  And it leads
to the error in make distcheck.


As a side note, if doc/tp_api/texi2any_internals.texi had been under
version control, then it would have been more visible that the files
changed in a problematic way (the po files change too, but not in a
meaningful way).


At that point, I am not sure where the problem is.  I could be that
the testing procedure needs to be modified to complete the git reverse
patch with a copy of the files removed by maintainer clean but
distributed.  It could also be that it is not a good idea to remove
distributed generated files with make maintainer-clean.  It could be
that the generated includes by pod2texi.pl should be done differently.
It could be that make dist should only be done from the source directory
if one want reproducible distributions.

My reading of
https://www.gnu.org/prep/standards/html_node/Standard-Targets.html
maintainer-clean is that doc/tp_api/texi2any_internals.texi should be
removed by maintainer-clean.


Here is a recipe to get differences between in source and out of source
distributions.  There are some changes for generated files, which seem
innocuous, but may be an issue for reproductibility, as well as the
difference for texi2any_internals.texi.




rm -f texinfo-7.*.*.tar.gz
./autogen.sh
./configure
make maintainer-clean
./autogen.sh
./configure
make dist
rm -rf in_source_dist_contents
mkdir in_source_dist_contents
(cd in_source_dist_contents && tar xvf ../texinfo-7.*.*.tar.gz)
make maintainer-clean
rm -rf bb
mkdir bb
cd bb
../configure
make dist
tar xvf texinfo-7.*.*.tar.gz
diff -u -r ../in_source_dist_contents/texinfo-7.*.*/ texinfo-7.*.*/ > 
../build_in_out_source_differences.diff


-- 
Pat

Re: "make maintainer-clean" inconsistency

[Patrice Dumas]

On Thu, Aug 08, 2024 at 08:46:37PM +0100, Gavin Smith wrote:
> When I run "make maintainer-clean", there is a difference depending
> on whether it is an out-of-source build.

My feeling is that maintainer-clean in an out of source build does not
make much sense, and that we should not support it.  But that's a wild
guess, I have never seen anything on that in the documentations and
that's a subject I do not know much about.

-- 
Pat

Re: fun with c23

[Patrice Dumas]

On Fri, Aug 09, 2024 at 10:23:53AM +0100, Gavin Smith wrote:
> On Thu, Aug 08, 2024 at 11:36:18PM -0600, Jeffrey Cliff wrote:
> > tl;dr c23 changes break compilation. 2 small patches
> 
> > 2) because there's a *whole bunch* of functions declared as VFunctions
> > with various kinds of arguments (for example cmd in echo-area.c and
> > m-x.c 's command-func)
> 
> There has already been some reduction in the use of VFunction on the
> "master" branch.  Now it is only used in terminal.c.  It should be
> straightforward to declare the types of the function parameters for
> the remaining use.

I already proposed a patch for VFunction use in terminal.c:
https://lists.gnu.org/archive/html/bug-texinfo/2024-06/msg00050.html

-- 
Pat

Re: Fix an error during "./configure; make dist"

[Patrice Dumas]

On Mon, Aug 05, 2024 at 07:03:08PM +0200, Bruno Haible wrote:
> Patrice Dumas wrote:
> 
> This is not complete.
> 
> 1) With this patch, the main texi2any_internals.texi still is generated in the
> build dir — which it shouldn't, since it is distributed.
> 
> Here is my procedure:
> $ ./configure
> $ make maintainer-clean
> $ git diff -- doc/html_default_commands_args.texi doc/pod2texi.texi 
> man/pod2texi.1 po_document/texinfo_document.pot 
> tp/Texinfo/XS/gnulib/lib/unicase/special-casing-table.h 
> tp/Texinfo/XS/gnulib/lib/uninorm/composition-table.h 
> tp/Texinfo/XS/main/command_ids.h tp/Texinfo/XS/main/element_types.c 
> tp/Texinfo/XS/main/element_types.h | patch -p1 -R
> $ mkdir bb
> $ cd bb
> $ ../configure
> $ make
> $ make distclean
> $ find . -type f
> ./doc/tp_api/texi2any_internals.texi

I attach a patch that fixes this issue by putting all the generated
Texinfo files in srcdir.

> 2) Texinfo/ModulePath.pm is no longer found in a particular situation
> (not sure whether this situation is supported).
> 
> How to reproduce:
> 
> $ ./configure
> $ make maintainer-clean
> $ ./configure
> $ make distcheck

This passes too.  I think that it should pass -- if all the maintainer
tools are present, possibly with a manual reset of
texi2any_internals.texi.

-- 
Pat
diff --git a/doc/tp_api/Makefile.am b/doc/tp_api/Makefile.am
index a2d3a888be..3dc8e521c3 100644
--- a/doc/tp_api/Makefile.am
+++ b/doc/tp_api/Makefile.am
@@ -78,8 +78,8 @@ texi2any_internals_dependencies = \
   $(top_srcdir)/tp/Texinfo/Convert/TexinfoXML.pm \
   $(top_srcdir)/tp/Texinfo/Convert/Plaintext.pm
 
-texi2any_internals.texi: $(texi2any_internals_dependencies) texi2any_internals_indices.texi texi2any_internals_preamble.texi
-	if $(MAKEINFO_ENVIRONMENT) $(PERL) -I $(top_srcdir)/Pod-Simple-Texinfo/lib/ $(top_srcdir)/Pod-Simple-Texinfo/pod2texi.pl --base-level=section --preamble - --subdir=api_includes $(texi2any_internals_dependencies) < $(srcdir)/texi2any_internals_preamble.texi >$@.tmp ; then \
+$(srcdir)/texi2any_internals.texi: $(texi2any_internals_dependencies) $(srcdir)/texi2any_internals_indices.texi $(srcdir)/texi2any_internals_preamble.texi
+	if $(MAKEINFO_ENVIRONMENT) $(PERL) -I $(top_srcdir)/Pod-Simple-Texinfo/lib/ $(top_srcdir)/Pod-Simple-Texinfo/pod2texi.pl --base-level=section --preamble - --subdir=$(srcdir)/api_includes $(texi2any_internals_dependencies) < $(srcdir)/texi2any_internals_preamble.texi >$@.tmp ; then \
 	  cat $@.tmp  | sed -e 's/^@bye/@__bye/' | sed -e '/^@__bye/r $(srcdir)/texi2any_internals_indices.texi' | sed -e 's/^@__bye//' > $@ ; \
 	fi ; rm -f $@.tmp
 

Re: Fix an error during "./configure; make dist"

[Patrice Dumas]

On Wed, Aug 07, 2024 at 02:16:30AM +0100, Gavin Smith wrote:
> However, I hit another problem immediately:
> 
>  (cd tp_api && make  top_distdir=../../texinfo-7.1.90 
> distdir=../../texinfo-7.1.90/doc/tp_api \
>  am__remove_distdir=: am__skip_length_check=: am__skip_mode_fix=: distdir)
> make[5]: Entering directory '/home/g/src/texinfo/GIT/doc/tp_api'
>  cd ../.. && /bin/bash /home/g/src/texinfo/GIT/build-aux/missing 
> automake-1.16 --gnu doc/tp_api/Makefile
> automake-1.16: error: cannot open < doc/tp_api/texi2any_internals.texi: No 
> such file or directory
> make[5]: *** [Makefile:1861: Makefile.in] Error 1
> make[5]: Leaving directory '/home/g/src/texinfo/GIT/doc/tp_api'
> 
> I found it slightly hard to recover from this error and had to resort
> to reverting all my changes and running ./autogen.sh again.
> 
> There is a comment in autogen.sh:
> 
> # missing texi2any_internals.texi stops automake.  Set up a fake
> # texi2any_internals.texi with a file timestamp set in the past,
> # if texi2any_internals.texi is not present
> 
> But texi2any_internals.texi is in MAINTAINERCLEANFILES in
> doc/tp_api/Makefile.am, so will be deleted by "make maintainer-clean".
> I presume that there was some bootstrapping issue with this file that
> necessitated this in autogen.sh, so it is probably likely that many
> "make" targets, include "distcheck", will fail after running
> "make maintainer-clean".

That issue seems to me to be unavoidable.  It seems to me that
texi2any_internals.texi in MAINTAINERCLEANFILES is logical as it is a
generated file.  Then there is this bootstraping issue that requires
running ./autogen.sh or recreating texi2any_internals.texi otherwise.

> texi2any_internals.texi is not tracked in git so its absence is
> not evident from the output of "git status".

I agree that it is better to have those files in git, actually I already
have added texi2any_internals.texi and api_includes/*.texi to git in my
private postrelease wip branch.

-- 
Pat

Re: Fix an error during "./configure; make dist"

[Patrice Dumas]

On Mon, Aug 05, 2024 at 01:41:37PM +0200, Bruno Haible wrote:
> Hi Gavin,
> 
> 2) "make distcheck" fails. (Reported by the weekly CI. It was fine a week 
> ago.)

Thanks for the report.

Regarding the patch, the change related to ModulePath.pm is incorrect,
ModulePath.pm should be in builddir and we do not distribute it.

For the main change, your approach is correct, but it can also be done
more simply without having to list and copy all the result files by
changing the --subdir such that the files are only generated in srcdir.
I attach a patch.

> From 09c76d3172ed0246e5a5f5bc2b1a214c65f48298 Mon Sep 17 00:00:00 2001
> From: Bruno Haible 
> Date: Mon, 5 Aug 2024 13:39:31 +0200
> Subject: [PATCH] build: Fix a "make distcheck" failure.
> 
> * doc/tp_api/Makefile.am (texi2any_internals_byproducts): New variable.
> (texi2any_internals.texi): Depend on ModulePath.pm in the source dir,
> not in thebuild dir. After generating texi2any_internals.texi in the
> build dir, move it and its byproducts to the source dir.
> ---
>  doc/tp_api/Makefile.am | 45 ++
>  1 file changed, 41 insertions(+), 4 deletions(-)
> 
> diff --git a/doc/tp_api/Makefile.am b/doc/tp_api/Makefile.am
> index c6124d4623..90f1919f45 100644
> --- a/doc/tp_api/Makefile.am
> +++ b/doc/tp_api/Makefile.am
> @@ -78,14 +78,51 @@ texi2any_internals_dependencies = \
>$(top_srcdir)/tp/Texinfo/Convert/TexinfoXML.pm \
>$(top_srcdir)/tp/Texinfo/Convert/Plaintext.pm
>  
> +# All the .texi files generated as a byproduct of generating
> +# texi2any_internals.texi, in the order in which they get generated.
> +texi2any_internals_byproducts = \
> +  api_includes/Texinfo-Commands.texi \
> +  api_includes/Texinfo-Common.texi \
> +  api_includes/Texinfo-Parser.texi \
> +  api_includes/Texinfo-Document.texi \
> +  api_includes/Texinfo-ManipulateTree.texi \
> +  api_includes/Texinfo-Structuring.texi \
> +  api_includes/Texinfo-Report.texi \
> +  api_includes/Texinfo-Translations.texi \
> +  api_includes/Texinfo-Transformations.texi \
> +  api_includes/Texinfo-Indices.texi \
> +  api_includes/Texinfo-OutputUnits.texi \
> +  api_includes/Texinfo-Convert-Texinfo.texi \
> +  api_includes/Texinfo-Convert-Utils.texi \
> +  api_includes/Texinfo-Convert-Unicode.texi \
> +  api_includes/Texinfo-Convert-NodeNameNormalization.texi \
> +  api_includes/Texinfo-Convert-Text.texi \
> +  api_includes/Texinfo-Convert-Converter.texi \
> +  api_includes/Texinfo-Convert-Info.texi \
> +  api_includes/Texinfo-Convert-HTML.texi \
> +  api_includes/Texinfo-Convert-DocBook.texi \
> +  api_includes/Texinfo-Convert-TexinfoMarkup.texi \
> +  api_includes/Texinfo-Convert-TexinfoXML.texi \
> +  api_includes/Texinfo-Convert-Plaintext.texi
> +
>  # Needed if make has not run in the "tp" directory yet
> -$(top_builddir)/tp/Texinfo/ModulePath.pm:
> +$(top_srcdir)/tp/Texinfo/ModulePath.pm:
>   cd $(top_builddir)/tp && $(MAKE) $(AM_MAKEFLAGS) Texinfo/ModulePath.pm
>  
> -texi2any_internals.texi: $(texi2any_internals_dependencies) 
> texi2any_internals_indices.texi texi2any_internals_preamble.texi 
> $(top_builddir)/tp/Texinfo/ModulePath.pm
> +texi2any_internals.texi: $(texi2any_internals_dependencies) 
> texi2any_internals_indices.texi texi2any_internals_preamble.texi 
> $(top_srcdir)/tp/Texinfo/ModulePath.pm
>   if $(MAKEINFO_ENVIRONMENT) $(PERL) -I 
> $(top_srcdir)/Pod-Simple-Texinfo/lib/ 
> $(top_srcdir)/Pod-Simple-Texinfo/pod2texi.pl --base-level=section --preamble 
> - --subdir=api_includes $(texi2any_internals_dependencies) < 
> $(srcdir)/texi2any_internals_preamble.texi >$@.tmp ; then \
> -   cat $@.tmp  | sed -e 's/^@bye/@__bye/' | sed -e '/^@__bye/r 
> $(srcdir)/texi2any_internals_indices.texi' | sed -e 's/^@__bye//' > $@ ; \
> - fi ; rm -f $@.tmp
> +   cat $@.tmp | sed -e 's/^@bye/@__bye/' | sed -e '/^@__bye/r 
> $(srcdir)/texi2any_internals_indices.texi' | sed -e 's/^@__bye//' > $@ ; \
> +   if test '$(srcdir)' != '.' ; then \
> + for file in $(texi2any_internals_byproducts) 
> texi2any_internals.texi ; do \
> +   if cmp $$file $(srcdir)/$$file >/dev/null 2>&1 ; then \
> + rm -f $$file ; \
> +   else \
> + mv $$file $(srcdir)/$$file ; \
> +   fi ; \
> + done ; \
> +   fi ; \
> + fi ; \
> + rm -f $@.tmp
>  
>  endif
>  
> -- 
> 2.34.1
> 

diff --git a/doc/tp_api/Makefile.am b/doc/tp_api/Makefile.am
index c6124d4623..fb3a48531a 100644
--- a/doc/tp_api/Makefile.am
+++ b/doc/tp_api/Makefile.am
@@ -83,7 +83,7 @@ $(top_builddir)/tp/Texinfo/ModulePath.pm:
 	cd $(top_builddir)/tp && $(MAKE) $(AM_MAKEFLAGS) Texinfo/ModulePath.pm
 
 texi2any_internals.texi: $(texi2any_internals_dependencies) texi2any_internals_indices.texi texi2any_internals_preamble.texi $(top_builddir)/tp/Texinfo/ModulePath.pm
-	if $(MAKEINFO_ENVIRONMENT) $(PERL) -I $(top_srcdir)/Pod-Simple-Texinfo/lib/ $(top_srcdir)/Pod-Simple-Texinfo/pod2texi.pl --base-level=section --preamble - --subdi

Re: (feature request) footnote formatting in makeinfo --html

[Patrice Dumas]

On Sat, Jul 27, 2024 at 11:36:19AM +0200, Patrice Dumas wrote:
> Still this is interesting, as this calls for including the call
> of convert_tree_new_formatting_context in format_single_footnote. I'll
> make that change.

Here it is:
https://git.savannah.gnu.org/cgit/texinfo.git/commit/?id=056d518faac6d799ca3f461d8849045da927abfb

I also added a test and an init file where
register_pending_formatted_inline_content is used, also based on your
code:
https://git.savannah.gnu.org/cgit/texinfo.git/commit/?id=15f90529ee13a5d2b8bf5364078c9a4902af1058

The result, visible in
tp/t/results/init_files_tests/test_format_single_footnote_in_inline_content/res_html/chap.html
shows that the output is expected output for paragraphs, could be
improved when there are block commands at the beginning of the footnote.
This is probably doable by looking at the contents of the footnote in
the Texinfo tree and find the first formattable content, and if it is
not a paragraph, prepend simply the mark.  Let's wait for actual user
requests.

-- 
Pat