[NTG-context] Fwd: lua-widow-control module error in LMTX

2022-04-27 Thread Eduardo Bohoyo via ntg-context

Hello, Max:

As I promised you this morning, I had time this evening to read more 
carefully all your remarks in the last mail. And now I can answer you 
with more basis.


In my particular case, it is not necessary to consider any of the seven 
possible problems you describe from highest to lowest probability. The 
question you asked me in the first paragraph about "interesting" 
features in my ConTeXt code is the key.


When I uncomment grid=yes in my \setuplayout, lwc makes its real 
appearance on the scene. I suppose that in other documents than mine, 
i.e. less complex, this performance would be a success. But in my file 
it is, considering my current aesthetic requirements, a failure. This is 
because for me grid=yes is a non-negotiable part of my code.


As I told you this morning when I sent you my pdf without the module 
activated, but with the layout my way, I understand that lwc can have 
problems with a text whose chapters have capital letters and small caps 
on the first page with less text than on the following pages; with long 
quotations that involve paragraphs of different layout and separated 
from the main text; with more than one footnote in a row, with a 
bibliography at the end with French indentation; with vertical spaces 
separating two lines within the same chapter when in the dummy text 
there is a supposed change of scene within the narrative; with etc. 
etc., etc., etc...


Anyway, after uncommenting grid=yes again, I will send you the log file, 
as you asked me. By the way, I do get some "Widow/Orphan NOT removed on 
page..." in spite of the rest of "successes" that move the final lines 
of my pages away from the result I would like, and that you have seen in 
the pdf I sent you this morning.


Do you want the new pdf with lwc actually acting on my file? Then you 
can get a better idea of what I mean by the end of some pages when I 
comment grid=yes (apart from the mischief that happens with the 
horizontal spaces in some paragraphs of the bibliographic section).


Or, if you prefer, and you need more feedback, I can send you my current 
code to use it as a test bench for " daring " texts.


Greetings,

edu


El 27/4/22 a las 9:14, Max Chernoff escribió:

Quick question before I begin: are you using any especially "interesting"
ConTeXt features? By "interesting" I mean things like grid typesetting,
pagecolumns, bidirectional text, etc. I haven't tested lwc with every
possible ConTeXt feature, so there may be some adverse interaction. If
you are using something like this, try disabling it and see if that solves
anything (then let me know so that I can fix it!)

On 2022-04-26 3:45 a.m., Eduardo Bohoyo wrote:


I can see "modules > 'lua-widow-control' is loaded".

But, luckily, I can also see this:

open source > level 2, order 4, name 
'/opt/luametatex/texmf-modules/tex/context/third/lua-widow-control/t-lua-widow-control.mkxl' 

resolvers   > lua > loading file 
'/opt/luametatex/texmf-modules/tex/luatex/lua-widow-control/lua-widow-control.lua' 
succeeded
close source    > level 2, order 4, name 
'/opt/luametatex/texmf-modules/tex/context/third/lua-widow-control/t-lua-widow-control.mkxl' 


module  > lua-widow-control > Already enabled


Ok, so this is good; lwc is for sure being loaded successfully.


No line such as "Widow/orphan detected. Attempting to delete".

I see interleaved new groups with the same line always repeating a 
warning message throughout the whole file. In short, there are 613 
new lines with the message "luatex warning > tex: left parfill skip 
is gone".
But I didn't give it any importance, because I interpreted that they 
could be inherent to the module.


Well this at least narrows the issue down quite a bit. Lwc runs in pretty
much two stages: when a paragraph has finished being broken by TeX, lwc
saves the paragraph. The second stage is ran just before each time the
output routine is triggered so that lwc can remove the widows and orphans.

Due to an lwc bug, the first stage results in the "left parfill skip"
warning being printed twice for each paragraph. Normally this is quite
annoying, but here it is good -- we know for sure that the first stage
is running just fine.

It is the second stage where you should get the "Widow/orphan detected"
message, but this isn't happening. The code here is at lwc.lua:362-388.

Here is a list of all possible reasons, in order of likelihood, why
"Widow/orphan detected" wouldn't be printed when there is actually
a widow or orphan:

(Just listing all of these to make sure that *I* don't forget 
anything. I'd

 say that 1 and 2 are the only ones that are actually likely -- you can
 probably ignore all of the others)

 1. "\clubpenalty" and/or "\widowpenalty" are either zero or infinite

    In these cases, lwc can't distinguish a widow or orphan from other
    penalties, so it doesn't do anything. You can check for this with

    \showthe\widowpenalty
    \showthe\clubpenalty

    some time 

Re: [NTG-context] lua-widow-control module error in LMTX

2022-04-27 Thread Max Chernoff via ntg-context

Quick question before I begin: are you using any especially "interesting"
ConTeXt features? By "interesting" I mean things like grid typesetting,
pagecolumns, bidirectional text, etc. I haven't tested lwc with every
possible ConTeXt feature, so there may be some adverse interaction. If
you are using something like this, try disabling it and see if that solves
anything (then let me know so that I can fix it!)

On 2022-04-26 3:45 a.m., Eduardo Bohoyo wrote:


I can see "modules > 'lua-widow-control' is loaded".

But, luckily, I can also see this:

open source > level 2, order 4, name 
'/opt/luametatex/texmf-modules/tex/context/third/lua-widow-control/t-lua-widow-control.mkxl' 

resolvers   > lua > loading file 
'/opt/luametatex/texmf-modules/tex/luatex/lua-widow-control/lua-widow-control.lua' 
succeeded
close source    > level 2, order 4, name 
'/opt/luametatex/texmf-modules/tex/context/third/lua-widow-control/t-lua-widow-control.mkxl' 


module  > lua-widow-control > Already enabled


Ok, so this is good; lwc is for sure being loaded successfully.


No line such as "Widow/orphan detected. Attempting to delete".

I see interleaved new groups with the same line always repeating a 
warning message throughout the whole file. In short, there are 613 new 
lines with the message "luatex warning > tex: left parfill skip is gone".
But I didn't give it any importance, because I interpreted that they 
could be inherent to the module.


Well this at least narrows the issue down quite a bit. Lwc runs in pretty
much two stages: when a paragraph has finished being broken by TeX, lwc
saves the paragraph. The second stage is ran just before each time the
output routine is triggered so that lwc can remove the widows and orphans.

Due to an lwc bug, the first stage results in the "left parfill skip"
warning being printed twice for each paragraph. Normally this is quite
annoying, but here it is good -- we know for sure that the first stage
is running just fine.

It is the second stage where you should get the "Widow/orphan detected"
message, but this isn't happening. The code here is at lwc.lua:362-388.

Here is a list of all possible reasons, in order of likelihood, why
"Widow/orphan detected" wouldn't be printed when there is actually
a widow or orphan:

(Just listing all of these to make sure that *I* don't forget anything. I'd
 say that 1 and 2 are the only ones that are actually likely -- you can
 probably ignore all of the others)

 1. "\clubpenalty" and/or "\widowpenalty" are either zero or infinite

In these cases, lwc can't distinguish a widow or orphan from other
penalties, so it doesn't do anything. You can check for this with

\showthe\widowpenalty
\showthe\clubpenalty

some time after "\starttext". This will print the values to the log
file. The values should probably be "1", but things like "150" and
"5000" are fine too. If you see negative values, "0", or values
greater than ten thousand, then something is wrong here.

 2. Something is "hiding" the widow/orphan penalties from lwc.

Something like "\vadjust{\penalty X}" or the e-TeX "\-penalties"
commands could potentially do this.

One potential solution for (a subset of) this problem would be to
modify lwc.mkxl to insert "\directsetup{*reset}" at line 52, to
get:

\startsetups[*default]
\directsetup{*reset}
\clubpenalty=\lwcparameter{orphanpenalty}
\widowpenalty=\lwcparameter{widowpenalty}
\displaywidowpenalty=\lwcparameter{widowpenalty}
\brokenpenalty=\lwcparameter{brokenpenalty}
\stopsetups

If that doesn't do anything, keep the above modifications but also insert

\setups[*default]

after "\starttext" in your test document.

 3. Lwc hasn't saved any paragraphs since the last output routine.

Since we know that the first stage is running, this shouldn't be
possible unless there is a weird bug somewhere else.

 4. A weird engine/format bug.

The LMTX test file was successful on Debian x86_64 three days ago


https://github.com/gucci-on-fleek/lua-widow-control/runs/6144354147?check_suite_focus=true

so something quite weird would have to be going on here.

 5. Something is overwriting the "pre_output_filter" callback.

Now this is an interesting one. ConTeXt has frozen most of the core
engine callbacks, but "pre_output_filter" is unfrozen and unassigned.
There is a ConTeXt interface for most of the other callbacks, but here
I need to fallback to the low-level interface. With this interface,
you can only register a single callback, so it is possible that some
other package is overwriting the callback. To test this, the very
beginning of your document should look like:

\usemodule[lua-widow-control]

\startluacode
lwc.callbacks.disable_box_warnings.enable = function () end
lwc.callbacks.disable_