Re: texi2dvi changes

[Gavin Smith]

On Wed, Jun 21, 2017 at 08:28:12PM +0100, Gavin Smith wrote:
> On Fri, Sep 25, 2015 at 09:42:51AM +0100, Gavin Smith wrote:
> > Now only the files that are directly produced are compared, not the 
> > sorted index files.
> 
> This has the unfortunate effect that if the sorted index files are 
> missing but the unsorted index files aren't, then texi2dvi won't
> call texindex.  For example:
> 
> # this creates index files file.??
> $ pdfetex file.texi
> 
> # this sees that file.?? hasn't changed and doesn't create file.??s
> $ texi2pdf file.texi
> 
> This situation won't occur very often as usually people will always use
> texi2pdf or texi2dvi instead of pdfetex or etex, but it's confusing 
> enough when it happens that I think it is worth fixing. I'm looking at
> texi2dvi now to try to see how to fix this.

I've made it so that it requires files with the names of sorted index 
files to exist.  I've tested it with a few Texinfo files as well as a 
LaTeX input file that used bibtex (where .bbl and .blg files are 
required to exist).

Previously, texi2dvi dealt with sorted index files by always 
regenerating them, but this is a bit slower, especially with newer 
texindex implemented in awk.  It should not be necessary to regenerate 
them, anyway.

Re: texi2dvi changes

[Gavin Smith]

(replying to old message)

On Fri, Sep 25, 2015 at 09:42:51AM +0100, Gavin Smith wrote:
> Now only the files that are directly produced are compared, not the 
> sorted index files.

This has the unfortunate effect that if the sorted index files are 
missing but the unsorted index files aren't, then texi2dvi won't
call texindex.  For example:

# this creates index files file.??
$ pdfetex file.texi

# this sees that file.?? hasn't changed and doesn't create file.??s
$ texi2pdf file.texi

This situation won't occur very often as usually people will always use
texi2pdf or texi2dvi instead of pdfetex or etex, but it's confusing 
enough when it happens that I think it is worth fixing. I'm looking at
texi2dvi now to try to see how to fix this.

Re: texi2dvi changes

[Vincent Belaïche]

[...]


>
> They you can't know what the next step if (bibtex, makeglossaries,
> biber, makeindex, texindex, ...)...
>

[...]

I meant "The*n* you can't know what the next step i*s*"

Sorry for the typoes,
   Vincent.

---
L'absence de virus dans ce courrier électronique a été vérifiée par le logiciel 
antivirus Avast.
https://www.avast.com/antivirus

Re: texi2dvi changes

[Gavin Smith]

On 24 September 2015 at 21:54, Vincent Belaïche
 wrote:
>> I pointed out that the log files were being grep'd a lot to get the
>> list of generated files. This won't happen as much now.
>>
>
> I have seen that what you have done goes far beyond reverting the
> TEXINPUTS order to what it was originally and removing the
> .flz. Well... maybe it would have been more careful to sort out what we
> already have on the table, rather than open another item with reducing
> the amount of log grepping. Anyway, on the other hand that was a good
> occasion to do it, as everybody has now warmed up on this texi2dvi.

IIRC this change was part of avoiding an infinite loop happening due
to the fls file being created as a recorder file, saved, and then
overwritten as a sorted index file, and then differing from that which
had been saved, indicating that another TeX run was required. Now only
the files that are directly produced are compared, not the sorted
index files. I wasn't sure if this was right for BibTeX (in case, for
example, TeX produces a file A, which is then processed to create file
B, which then in turn is processed to create a file C: then even if
file A hasn't changed, if file B has we still need another run of the
tools), but it appears to work. It's possible I could have done it in
a simpler way, but I don't know.

It occurred to me why we don't simply check that the dvi or pdf file
hasn't changed, and forget about the other files. It's probably
because those files are considered to be too big to back up and
compare, and only comparing the auxiliary files is more efficient.

> As a preliminary comment I would say that all the xxx_p named function
> have now misleading names as the `_p' suffix stands for `predicate', ie
> involves that the function returns a boolean condition to be used in a
> boolean expression (like within an `if ... ;'. Now these functions do
> some border effect (ie `echo ...'), so the names should be changed (ie
> dropping the `_p' tail) accordingly.

Yes. I want also to rename the variables referring to "xref" files,
because only one of those files is actually used for cross-references.

>> This was a big change to a complex script with many ways of running
>> it, so it's quite possible that something's broken. (I've deliberately
>> refrained from uploading texi2dvi to the FTP site for the last few
>> revisions for that reason.)
>>
>> Can anybody try the script with LaTeX source with BibTeX, or else send
>> me a test case? I want to make sure the BibTeX files are checked
>> properly.
>
> I have done that. See attached test file and the session (essai.session)
> where I try a compilation both in --tidy mode, and then w/o --tidy for
> an essai.tex file using a bib. I also made a trial on a Texinfo file.

Thanks for testing, good to hear it appears to be working.

Re: texi2dvi changes

[Vincent Belaïche]

Le 25/09/2015 10:42, Gavin Smith a écrit :
> On 24 September 2015 at 21:54, Vincent Belaïche
>  wrote:
>>> I pointed out that the log files were being grep'd a lot to get the
>>> list of generated files. This won't happen as much now.
>>>
>>

[...]

>
> IIRC this change was part of avoiding an infinite loop happening due
> to the fls file being created as a recorder file, saved, and then
> overwritten as a sorted index file, and then differing from that which
> had been saved, indicating that another TeX run was required. Now only
> the files that are directly produced are compared, not the sorted
> index files. I wasn't sure if this was right for BibTeX (in case, for
> example, TeX produces a file A, which is then processed to create file
> B, which then in turn is processed to create a file C: then even if
> file A hasn't changed, if file B has we still need another run of the
> tools), but it appears to work. It's possible I could have done it in
> a simpler way, but I don't know.

For BibTeX you need both the .aux file and the .bib file(s), for
instance if you have essai.tex that needs essai-bib.bib, then you will
run ``bibtex "essai"'', that will look into essai.aux, get there the
\bibdata{essai-bib}, then look into essai-bib.bib.

Note that this is for the original bibtex, I don't know whether this is
the same with biblatex & biber which more & more people are using.

>
> It occurred to me why we don't simply check that the dvi or pdf file
> hasn't changed, and forget about the other files.

They you can't know what the next step if (bibtex, makeglossaries,
biber, makeindex, texindex, ...)...

> It's probably because those files are considered to be too big to back
> up and compare, and only comparing the auxiliary files is more
> efficient.
>

Well that may be true, latexmk makes an md5sum of the files to check for
change, that is probably faster if you account for this that the
bottleneck is disk access, not computation.


[...]

>
> Thanks for testing, good to hear it appears to be working.

Thanks to you Gavin,

   Vincent.

---
L'absence de virus dans ce courrier électronique a été vérifiée par le logiciel 
antivirus Avast.
http://www.avast.com

Re: texi2dvi changes

[Gavin Smith]

On 22 September 2015 at 23:34, Karl Berry  wrote:
> gs> Can anybody try the script with LaTeX source with BibTeX, or
> else send me a test case? I want to make sure the BibTeX files are
> checked properly.
>
> 
> % bibsimple.tex
> \documentclass{article}
> \begin{document}
> \nocite{*}
> \bibliographystyle{plain}
> \bibliography{xampl}
> \end{document}
> 
>
> Standard process:
> latex bibsimple
> bibtex !$
> latex !$
> latex !$
>

Okay thanks, this worked without a hitch, both with the manual
instructions above, and with texi2dvi.

--clean and --tidy didn't work though with -recorder, but I think I fixed it.

Re: texi2dvi changes

[Karl Berry]

+  if test $generated_files_get_method = generated_files_get_from_fls; then
+if test -f "$in_noext.fl"; then
+  report 'WARNING!! You may typeset garbage!' # goes to stderr
+fi
+  fi

(Aside: I usually find test -r more useful than test -f, because if by
some weird mischance something other than a regular file exists by that
name, it's still desirable to catch it.)

Seems fine, except I'd explicitly mention that the .fl[s] index is the
problem.  Otherwise the user is left with a complete mystery.  As in:

report 'WARNING!!  The fl index may typeset as garbage!' # goes to stderr

It might even be worth adding:
report 'Use env TEXI2DVI_USE_RECORDER=no as a workaround.'
since anyone who gets this message is likely to just want to get on with
life, not delve into all the maddening details.

thanks,
karl

Re: texi2dvi changes

[Gavin Smith]

On 22 September 2015 at 23:55, Karl Berry  wrote:
> Hi Gavin,
>
> gs> If someone uses an "fl" index with the recorder, I expect either
> the index to be blank, or to be the recorder file interpreted as TeX
> source (a great ugly mess).
>
> There's a comment in texi2dvi that says
>   #  The default behaviour is `nomaybe'.
> But the actual default for TEXI2DVI_USE_RECORDER is yesmaybe (line 1530).
>
> Excuse my opinion, but this seems like the worst possible outcome.  This
> way, an existing document that uses an fl index (as discussed, they
> exist) will silently typeset garbage with a new texi2dvi.  Thus, with
> the yesmaybe default, I strongly believe that texi2dvi should give a
> definite error in the case of both the recorder being used and the fl
> index being used.

I changed this to nomaybe in my last commit (and also noticed the
comment was wrong).

> On another front: evidently some grep for openout is not being redirected.
> (Looks like that's the case in check_openout_in_log_support.)
> With attached, stdout from  texi2dvi indexfl.texi  starts with
> \openout0 = `dum.dum'.

Also fixed in last commit.

Re: texi2dvi changes

[Karl Berry]

Hi Gavin,

gs> If someone uses an "fl" index with the recorder, I expect either
the index to be blank, or to be the recorder file interpreted as TeX
source (a great ugly mess).

There's a comment in texi2dvi that says
  #  The default behaviour is `nomaybe'.
But the actual default for TEXI2DVI_USE_RECORDER is yesmaybe (line 1530).

Excuse my opinion, but this seems like the worst possible outcome.  This
way, an existing document that uses an fl index (as discussed, they
exist) will silently typeset garbage with a new texi2dvi.  Thus, with
the yesmaybe default, I strongly believe that texi2dvi should give a
definite error in the case of both the recorder being used and the fl
index being used.

Also, since the only workaround for this case would be to set
TEXI2DVI_USE_RECORDER, I am back to suggesting that it be in the help
message.  Or change the default to nomaybe, I guess, but as already
discussed, the recorder file seems more reliable in general.

Another approach, though suboptimal IMHO, would be for texinfo.tex and
makeinfo to reject up front an attempt to "@defindex fl".  That would be
better than "successfully" typesetting junk.  (I did my tests with the
texinfo.tex currently in the repo, FWIW.)


On another front: evidently some grep for openout is not being redirected.
(Looks like that's the case in check_openout_in_log_support.)
With attached, stdout from  texi2dvi indexfl.texi  starts with
\openout0 = `dum.dum'.

Best,
Karl

P.S. Triviality: I know you are English, but I think it's better to use
American spelling consistently, when the original is in Am.Eng., to avoid
unnecessary inconsistencies for future readers.  Just as I would use
"behaviour" with an original that was in UK.Eng.  Whatever ...



indexfl.texi
Description: Binary data

Re: texi2dvi changes

[Gavin Smith]

On 22 September 2015 at 23:55, Karl Berry  wrote:
> Excuse my opinion, but this seems like the worst possible outcome.  This
> way, an existing document that uses an fl index (as discussed, they
> exist) will silently typeset garbage with a new texi2dvi.  Thus, with
> the yesmaybe default, I strongly believe that texi2dvi should give a
> definite error in the case of both the recorder being used and the fl
> index being used.

Maybe something like

Index: ../texi2dvi
===
--- ../texi2dvi (revision 6637)
+++ ../texi2dvi (working copy)
@@ -709,6 +709,11 @@
 generated_files_get ()
 {
   $generated_files_get_method "$in_noext"
+  if test $generated_files_get_method = generated_files_get_from_fls; then
+if test -f "$in_noext.fl"; then
+  report 'WARNING!! You may typeset garbage!' # goes to stderr
+fi
+  fi
 }

More another time.

> Another approach, though suboptimal IMHO, would be for texinfo.tex and
> makeinfo to reject up front an attempt to "@defindex fl".  That would be
> better than "successfully" typesetting junk.  (I did my tests with the
> texinfo.tex currently in the repo, FWIW.)

It can be fixed in new texinfo.tex, but not in old texinfo.tex (that
would require time travel). So issuing a warning is better.

Re: texi2dvi changes

[Karl Berry]

gs> Can anybody try the script with LaTeX source with BibTeX, or
else send me a test case? I want to make sure the BibTeX files are
checked properly.


% bibsimple.tex
\documentclass{article}
\begin{document}
\nocite{*}
\bibliographystyle{plain}
\bibliography{xampl}
\end{document}


Standard process:
latex bibsimple
bibtex !$
latex !$
latex !$

(Or pdflatex.)

xampl.bib should be in your tex distribution, it's the example bib that
comes with bibtex, but just in case, it's also attached.

k



xampl.bib
Description: Binary data

Re: texi2dvi changes the behavior of ~

[Karl Berry]

Hi Akim,

How is this code expected to disable the magic tricks it plays with ~?

It should only be enabled when the source is texinfo, not latex.
Can you make that change?

The way things have turned out, I rather think latex2dvi should have
been a separate script.  Most of what's in texi2dvi has nothing to do
with texinfo any more.  Oh well.

karl