Bug#987587: libpango1.0-udeb: hangs the installer in various situations

[Steve McIntyre]

On Wed, May 19, 2021 at 05:50:49PM +0200, Cyril Brulebois wrote:
>Simon McVittie  (2021-05-19):

...

>> As for Pango, I'm afraid figuring out whether it is doing something
>> wrong here is beyond my expertise. If I can characterize the maybe-bug
>> in a clear enough way I can try asking upstream - but as I said
>> before, upstream will be very reluctant to touch this as soon as I
>> mention GTK 2, which has been on life-support for a decade and is now
>> officially dead.
>
>As mentioned before, GTK 2 has been working just fine for us… until it
>no longer did. It's *very*hugely*appreciated* that you've helped us that
>much this time around. We'll do the work next time, and I don't think
>it's important enough to try and dive into Pango some more at this
>stage, unless someone experiences similar issues with a hrm more recent
>and supported toolkit version…
>
>Thanks again, you're a lifesaver!

Nod, absolutely. Thanks very much for your efforts here Simon!

-- 
Steve McIntyre, Cambridge, UK.st...@einval.com
“Why do people find DNS so difficult? It’s just cache invalidation and
 naming things.”
   -– Jeff Waugh (https://twitter.com/jdub)

Bug#987587: libpango1.0-udeb: hangs the installer in various situations

[Simon McVittie]

Control: clone 987587 -2 -3
Control: reassign -2 libgtk2.0-0-udeb 2.24.33-1
Control: retitle -2 libgtk2.0-0-udeb: should avoid GtkTextView relayout loop if 
width from Pango oscillates
Control: tags -2 + patch pending
Control: reassign -3 cdebconf-gtk-udeb 0.257
Control: retitle -3 cdebconf-gtk-udeb: should give GTK a chance to do layout 
before adding lots of text
Control: tags -3 + patch
Control: retitle 987587 libpango1.0-udeb: asking for less width can result in 
more width being requested
Control: severity 987587 normal
Control: tags 987587 + help

On Wed, 19 May 2021 at 16:39:21 +0100, Simon McVittie wrote:
> Following the rule of thumb that bad interactions between two components
> should often be fixed on *both* sides, I'd be tempted to clone this bug,
> reassign to both gtk+2.0 and cdebconf, and apply both changes.

As discussed with kibi on the merge requests, let's do this.

Clone -2 is for GTK, and is resolved by
. I'll
handle this.

Clone -3 is for cdebconf, and is resolved by
.
Please don't merge until the new bug number is available.

> As for Pango, I'm afraid figuring out whether it is doing something wrong
> here is beyond my expertise.

I'll leave #987587 assigned to Pango, but I'm de-escalating it to non-RC
since it isn't at all obvious to me whether it's a bug, and resolving the
clones -2 and -3 will fix the release-critical issue.

smcv

Bug#987587: libpango1.0-udeb: hangs the installer in various situations

[Cyril Brulebois]

Simon McVittie  (2021-05-19):
> It wasn't obvious to me where we'd keep track of this, or how correct
> it would be to do that - cache invalidation is reputed to be one of the
> hardest problems in computer science, and this would be one more thing
> that needs to be invalidated on at least those occasions when the layout
> has legitimately changed (but without invalidating it too often because
> that would destroy the workaround).
> 
> Having reproduced the English/rescue issue and got further with the
> Sinhala/install issue with the GTK MR referenced below, I think it can
> also happen that the layout flaps by an amount greater than 1 pixel
> (I think the most I've seen is 4px), so a special case for n/n+1 isn't
> going to be enough.
> 
> One of the reasons this took me a while is that I got distracted by the
> difference I was seeing being exactly 1px, which I thought might be to
> do with GTK adding 1px of extra width to make sure there's space for a
> cursor - but after tracing through it, it seemed like GTK is always
> adding/removing that width correctly.

Looking at the traces I saved when installing in Swedish with just the
GTK patch, I'm seeing a bunch of 1px to 4px differences, but it can even
reach 8px!

[…] we asked Pango to wrap text for width 186px but it now wants 194px. 
Clamping result to 186px!

> > > My next thing to get a try when I get a chance will be to make GTK
> > > refuse to obey Pango when GTK asks Pango to stick to a width limit
> > > and Pango goes outside that limit, with a g_warning().
> 
> This works: https://salsa.debian.org/gnome-team/gtk2/-/merge_requests/2
> 
> The other thing I wanted to try was to make cdebconf create the
> GtkTextView in an empty state, and then populate it with text a little
> later (perhaps after layout but before drawing, or perhaps drawing one
> frame without text and then adding the text in the next frame, I'm not
> actually sure). And that also works:
> https://salsa.debian.org/installer-team/cdebconf/-/merge_requests/6
> 
> Following the rule of thumb that bad interactions between two
> components should often be fixed on *both* sides, I'd be tempted to
> clone this bug, reassign to both gtk+2.0 and cdebconf, and apply both
> changes.

That looks good to me, I'll do so in a moment, and deal with (re)trying
to understand everything going on with your patch before merging and
uploading.

> As for Pango, I'm afraid figuring out whether it is doing something
> wrong here is beyond my expertise. If I can characterize the maybe-bug
> in a clear enough way I can try asking upstream - but as I said
> before, upstream will be very reluctant to touch this as soon as I
> mention GTK 2, which has been on life-support for a decade and is now
> officially dead.

As mentioned before, GTK 2 has been working just fine for us… until it
no longer did. It's *very*hugely*appreciated* that you've helped us that
much this time around. We'll do the work next time, and I don't think
it's important enough to try and dive into Pango some more at this
stage, unless someone experiences similar issues with a hrm more recent
and supported toolkit version…

Thanks again, you're a lifesaver!


Cheers,
-- 
Cyril Brulebois (k...@debian.org)
D-I release manager -- Release team member -- Freelance Consultant


signature.asc
Description: PGP signature

Bug#987587: libpango1.0-udeb: hangs the installer in various situations

[Simon McVittie]

On Mon, 17 May 2021 at 23:54:01 +0200, Cyril Brulebois wrote:
> Simon McVittie  (2021-05-17):
> > It looks as though the problem is that the size GTK chooses for a
> > GtkTextView (a debconf "note" or similar) is flapping between two
> > values.
> 
> Without looking into the code, one might try and keep track of results
> that have been seen, and if N/N+1 is detected, maybe just assume this
> should be N+1 and move on with other computations? But anyway, further
> down your mail you seemed to have ideas already.

It wasn't obvious to me where we'd keep track of this, or how correct
it would be to do that - cache invalidation is reputed to be one of the
hardest problems in computer science, and this would be one more thing
that needs to be invalidated on at least those occasions when the layout
has legitimately changed (but without invalidating it too often because
that would destroy the workaround).

Having reproduced the English/rescue issue and got further with the
Sinhala/install issue with the GTK MR referenced below, I think it can
also happen that the layout flaps by an amount greater than 1 pixel
(I think the most I've seen is 4px), so a special case for n/n+1 isn't
going to be enough.

One of the reasons this took me a while is that I got distracted by the
difference I was seeing being exactly 1px, which I thought might be to
do with GTK adding 1px of extra width to make sure there's space for a
cursor - but after tracing through it, it seemed like GTK is always
adding/removing that width correctly.

> > My next thing to get a try when I get a chance will be to make GTK refuse
> > to obey Pango when GTK asks Pango to stick to a width limit and Pango goes
> > outside that limit, with a g_warning().

This works: https://salsa.debian.org/gnome-team/gtk2/-/merge_requests/2

The other thing I wanted to try was to make cdebconf create the GtkTextView
in an empty state, and then populate it with text a little later (perhaps
after layout but before drawing, or perhaps drawing one frame without text
and then adding the text in the next frame, I'm not actually sure). And
that also works:
https://salsa.debian.org/installer-team/cdebconf/-/merge_requests/6

Following the rule of thumb that bad interactions between two components
should often be fixed on *both* sides, I'd be tempted to clone this bug,
reassign to both gtk+2.0 and cdebconf, and apply both changes.

As for Pango, I'm afraid figuring out whether it is doing something wrong
here is beyond my expertise. If I can characterize the maybe-bug in a clear
enough way I can try asking upstream - but as I said before, upstream will
be very reluctant to touch this as soon as I mention GTK 2, which has been
on life-support for a decade and is now officially dead.

smcv

Bug#987587: libpango1.0-udeb: hangs the installer in various situations

[Cyril Brulebois]

Hi Simon!

Simon McVittie  (2021-05-17):
> My instinct is that it's far, far too late to be moving to GTK 3 this
> cycle, and I'd prefer to get a suitable hack into GTK 2 if we can
> find one. We can make it #ifdef DEBIAN_INSTALLER to avoid disrupting
> the installed system.

If you're happy with further assisting us with getting away with GTK 2
one more time, that's very fine with me! Huge thanks for all your work
and help so far!

> I think d-i will need to move to GTK 3 early in the Debian 12 cycle,
> but hard freeze is not the time to be fast-forwarding through 10 years
> of GUI library development.

From my little “feasibility survey” over the last few days, there seem
to be a number of obvious issues to solve indeed!

> Getting
> https://salsa.debian.org/installer-team/cdebconf/-/merge_requests/4
> into cdebconf would make this easier, although we can revert it for
> the sake of a smaller diff once we think we have a solution if the
> release team are grumpy about it.

I'm very happy to see this patch merged, and possibly released into
bullseye. The most obvious side effect I can anticipate is possibly
bigger /var/log/installer/syslog files in the installed systems, but
users are expected to compress them anyway before attaching them in
installation reports (BTS and/or lists size limits).

Would you like that to be released into unstable right away? I'm happy
to deal with it if that helps testing stuff.

> It looks as though the problem is that the size GTK chooses for a
> GtkTextView (a debconf "note" or similar) is flapping between two
> values. GTK asks Pango "if you wrap lines at width x, how much space
> will you need?", then uses the result to re-run the layout algorithm,
> which changes the width available, which causes the layout algorithm to
> be re-run and so on. Under normal circumstances, this runs for a few
> iterations and then stabilizes at one size, terminating the loop, but
> when I select Sinhala from the language menu, the width flaps between
> 71 and 72 pixels, with each re-layout resulting in the other width.

Without looking into the code, one might try and keep track of results
that have been seen, and if N/N+1 is detected, maybe just assume this
should be N+1 and move on with other computations? But anyway, further
down your mail you seemed to have ideas already.

> I think this might be related to the fact that when the layout is
> calculated at the narrower width, Pango decides that there isn't space
> to put the "." at the end of a paragraph on the same line as the last
> word, and so wraps it to the next line, which is fine; except that
> you'd expect the space required for the last word to get a bit smaller
> when the "." is not included, but actually Pango tells GTK that it will
> need 1 pixel *more* for "අසම්පූර්ණයි" than for "අසම්පූර්ණයි.", resulting in it
> flapping between the narrower and wider width. I don't know why that
> happens - perhaps the Indic shaper is drawing a character at the end of
> a line more elaborately, or something?

Maybe the Indic shaper makes the problem more obvious but I definitely
saw a similar hang (even if I hadn't patched the installer to generate
log lines to make extra sure) with plain English when starting in rescue
mode.

> However, either 71 or 72 pixels seems a ridiculous width to be trying to
> squish multiple sentences into, so arguably it's a bug in GTK 2's layout
> algorithm that it is even considering that as a size. The GTK 2 layout
> algorithm does not necessarily make much sense, and the fact that it
> can feed back into itself is probably a GTK 2 design flaw. During the
> GTK 3 cycle it was redone to be in terms of "height for width" (if I
> give you this width, how much height will you need? the mode cdebconf
> will end up using) and "width for height" (the opposite, rarely used) -
> so hopefully GTK 3 avoids this failure mode. However, it also seems wrong
> that telling Pango that less width is available can result in it saying
> it needs *more* minimum width, and maybe preventing this from happening
> would get GTK 2 to do the right thing.
> 
> My next thing to get a try when I get a chance will be to make GTK refuse
> to obey Pango when GTK asks Pango to stick to a width limit and Pango goes
> outside that limit, with a g_warning(). This might result in the text
> being clipped at the right margin (or left margin in Hebrew/Arabic), or
> even having its "ink rectangle" overlap with the next widget to the right
> (or left); but for d-i, which always uses the full screen width and in
> practice has a generous amount of space for its text, we might get away
> with it? In a very brief test that seemed to resolve the Sinhala thing,
> but I haven't tried the other paths that lead to infinite loops. Please
> could someone who has tried the other scenarios provide a walkthrough?

What I can think of that's not (entirely) full-width:
 - buttons at the bottom;
 - checkboxes to toggle password/passphrase visibility (but they 

Bug#987587: libpango1.0-udeb: hangs the installer in various situations

[Simon McVittie]

On Mon, 17 May 2021 at 18:12:35 +0200, Cyril Brulebois wrote:
> And for those not following #debian-boot, I'm finding myself between a
> rock and a hard place, as both options (trying to work around the
> rendering-related hangs versus switching to GTK 3 at the last moment)
> are very far from ideal… I'm not sure what we'll end up doing, and I'm
> happy to get some “votes” pro or against any of those options, and to
> hear about other options entirely.

My instinct is that it's far, far too late to be moving to GTK 3 this
cycle, and I'd prefer to get a suitable hack into GTK 2 if we can
find one. We can make it #ifdef DEBIAN_INSTALLER to avoid disrupting
the installed system.

I think d-i will need to move to GTK 3 early in the Debian 12 cycle,
but hard freeze is not the time to be fast-forwarding through 10 years
of GUI library development.

I've continued to look into GTK2/Pango with some rather extensive printf
debugging. I have other responsibilities and I'm trying to learn how
GTK/Pango text layout works from first principles (I'm in the GNOME
team but not really a GUI developer), so it's slow going, but I have
some ideas for things that might be able to break the loop. It's not
clear to me yet whether this is a GTK 2 bug, or a Pango bug, or what.
GNOME team members who actually know what they're doing are welcome
to step in any time.

Upstream are going to be incredibly reluctant to help us with GTK 2,
given that it has been deprecated in favour of GTK 3 for a decade, and
is officially EOL now that GTK 4 has stable releases. Pango does seem
like it's doing the wrong thing here, but perhaps GTK 2 or cdebconf is
holding it wrong.

Getting
https://salsa.debian.org/installer-team/cdebconf/-/merge_requests/4
into cdebconf would make this easier, although we can revert it for the
sake of a smaller diff once we think we have a solution if the release
team are grumpy about it.

Philip Hands wanted to get this running under automated testing, but my
current GTK 2 and Pango patches are spamming the syslog at 3 million lines
a minute when they get into the loop, so we're going to have logistical
difficulties in saving logs.

It looks as though the problem is that the size GTK chooses for a
GtkTextView (a debconf "note" or similar) is flapping between two
values. GTK asks Pango "if you wrap lines at width x, how much space
will you need?", then uses the result to re-run the layout algorithm,
which changes the width available, which causes the layout algorithm to
be re-run and so on. Under normal circumstances, this runs for a few
iterations and then stabilizes at one size, terminating the loop, but
when I select Sinhala from the language menu, the width flaps between
71 and 72 pixels, with each re-layout resulting in the other width.

I think this might be related to the fact that when the layout is
calculated at the narrower width, Pango decides that there isn't space
to put the "." at the end of a paragraph on the same line as the last
word, and so wraps it to the next line, which is fine; except that
you'd expect the space required for the last word to get a bit smaller
when the "." is not included, but actually Pango tells GTK that it will
need 1 pixel *more* for "අසම්පූර්ණයි" than for "අසම්පූර්ණයි.", resulting in it
flapping between the narrower and wider width. I don't know why that
happens - perhaps the Indic shaper is drawing a character at the end of
a line more elaborately, or something?

However, either 71 or 72 pixels seems a ridiculous width to be trying to
squish multiple sentences into, so arguably it's a bug in GTK 2's layout
algorithm that it is even considering that as a size. The GTK 2 layout
algorithm does not necessarily make much sense, and the fact that it
can feed back into itself is probably a GTK 2 design flaw. During the
GTK 3 cycle it was redone to be in terms of "height for width" (if I
give you this width, how much height will you need? the mode cdebconf
will end up using) and "width for height" (the opposite, rarely used) -
so hopefully GTK 3 avoids this failure mode. However, it also seems wrong
that telling Pango that less width is available can result in it saying
it needs *more* minimum width, and maybe preventing this from happening
would get GTK 2 to do the right thing.

My next thing to get a try when I get a chance will be to make GTK refuse
to obey Pango when GTK asks Pango to stick to a width limit and Pango goes
outside that limit, with a g_warning(). This might result in the text
being clipped at the right margin (or left margin in Hebrew/Arabic), or
even having its "ink rectangle" overlap with the next widget to the right
(or left); but for d-i, which always uses the full screen width and in
practice has a generous amount of space for its text, we might get away
with it? In a very brief test that seemed to resolve the Sinhala thing,
but I haven't tried the other paths that lead to infinite loops. Please
could someone who has tried the other scenarios provide a 

Bug#987587: libpango1.0-udeb: hangs the installer in various situations

[Samuel Thibault]

Cyril Brulebois, le lun. 17 mai 2021 18:12:35 +0200, a ecrit:
> Additionally, even with all 3 cdebconf-gtk-* packages converted, we
> still get libgtk2.0-0-udeb pulled into a netboot-gtk build, because this
> package pulls it:
> 
> build/pkg-lists/gtk-common:libgail18-udeb
> 
> Adding debian-accessibility@ to the loop for awareness, and for input
> since it's been a while since I last looked into accessibility details…
> Would we need some prospective libgail-3-0-udeb to replace it in a GTK 3
> world?

gail is only needed for gtk2, gtk3 includes the accessibility stack
by itself.

Samuel

Bug#987587: libpango1.0-udeb: hangs the installer in various situations

[Cyril Brulebois]

Cyril Brulebois  (2021-05-17):
> > The steps to use GTK 3 in d-i would be:
> > 
> > - convert cdebconf-gtk-udeb from GTK 2 to GTK 3
> > - convert cdebconf-gtk-entropy from GTK 2 to GTK 3
> > - convert cdebconf-gtk-terminal from GTK 2 to GTK 3 and, simultaneously,
> >   from libvte9-udeb to libvte-2.91-0-udeb
> > 
> > The big blocker for libvte-2.91-0-udeb used to be that it's written in
> > C++, but I've switched it to be built with -static-libstdc++ so that we
> > don't need a libstdc++6-udeb (its public API/ABI is GLib-flavoured C,
> > only the internals are C++). The result is that libvte-2.91-0-udeb isn't
> > much larger than libvte9-udeb.
> 
> Based on the (perceived) unlikeliness that we might find a fix for GTK 2
> (yeah, inertia… but it had been working so neatly for so long that
> moving away from it just hadn't happened…), I've checked what would
> happen with GTK 3 in cdebconf and cdebconf-gtk-terminal (I had forgotten
> about cdebconf-gtk-entropy until writing this reply).
> 
> The installer seems to be working somewhat. I'm seeing strange things
> regarding layout, regarding widget expansion (basically we have some
> wasted vertical space). I'm also seeing a different behaviour regarding
> the expose (GTK 2) vs. draw (GTK 3) event handling, meaning the banner
> doesn't get repainted automatically; trying to do that on my own results
> on it being painted over and over again (that happens with a
> pango_cairo_show_layout), meaning “Mode de récupération” (fr) gets
> rendered on top of “Rescue mode” (en) after selecting French. I wouldn't
> call any of those two issues a blocker for 11.0 *if we were to go the
> “Switch to GTK 3” route*.
> 
> I'm also seeing a warning when spawning a shell, that comes from
> vte2.91; again, not a blocker. But exiting the shell leads to a crash,
> and the installer gets restarted. The syslog has this:
> 
> May 17 15:28:46 debconf: cdebconf_gtk (process:257): GLib - DEBUG: 
> setenv()/putenv() are not thread-safe and should not be used after threads 
> are created
> May 17 15:28:46 debconf: cdebconf_gtk (process:257): VTE - WARNING: 
> (../src/vtepty.cc:670):bool _vte_pty_spawn_sync(VtePty*, const char*, const 
> char* const*, const char* const*, GSpawnFlags, GSpawnChildSetupFunc, 
> gpointer, GDestroyNotify, GPid*, int, GCancellable*, GError**): runtime check 
> failed: ((spawn_flags & forbidden_spawn_flags()) == 0)
> May 17 15:31:14 kernel: [  218.924148] event_listener[263]: segfault at 0 
> ip 7f2ecb006500 sp 7f2ecaf12a98 error 4 in 
> plugin-terminal.so[7f2ecb006000+2000]
> 
> I'm assuming the DEBUG/WARNING parts aren't too scary, and that the
> segfault upon exit might be unrelated. I would call that a blocker for a
> new release candidate of the installer since that would leave /target
> mounted, possibly with other filesystems, and one couldn't re-enter the
> shell.
> 
> I'll check what needs to happen with cdebconf-gtk-entropy, and whether
> that might have a link with the segfault…

(No change.)

Additionally, even with all 3 cdebconf-gtk-* packages converted, we
still get libgtk2.0-0-udeb pulled into a netboot-gtk build, because this
package pulls it:

build/pkg-lists/gtk-common:libgail18-udeb

Adding debian-accessibility@ to the loop for awareness, and for input
since it's been a while since I last looked into accessibility details…
Would we need some prospective libgail-3-0-udeb to replace it in a GTK 3
world?


And for those not following #debian-boot, I'm finding myself between a
rock and a hard place, as both options (trying to work around the
rendering-related hangs versus switching to GTK 3 at the last moment)
are very far from ideal… I'm not sure what we'll end up doing, and I'm
happy to get some “votes” pro or against any of those options, and to
hear about other options entirely.


Cheers,
-- 
Cyril Brulebois (k...@debian.org)
D-I release manager -- Release team member -- Freelance Consultant


signature.asc
Description: PGP signature

Bug#987587: libpango1.0-udeb: hangs the installer in various situations

[Cyril Brulebois]

Hi Simon,

Simon McVittie  (2021-05-13):
> Adding a udeb-specific hack in one of those packages seems reasonable,
> *if* we can find a suitable hack.
> 
> pango1.0 just has one build shared between the udeb and the full .deb,
> so if the hack goes there, a prerequisite would be to build it twice
> (fairly straightforward, just annoying). gtk+2.0 and harfbuzz already
> build twice, so it would be easy to add -DDEBIAN_INSTALLER to the udeb
> build, or similar.

Yeah, that's what I had in mind.

> The problem is finding the right hack. I don't think bisecting Pango is
> necessarily going to get you very far: if you arrive at the version/commit
> where Pango switched from doing text layout internally to using Harfbuzz
> for everything, then that'll be a dead end. Conditionally reverting the
> switch from internal layout engines to Harfbuzz doesn't seem likely to
> yield a reasonable patch - the internal layout engines were deleted about
> 2 years ago and the rest of the pango codebase has moved on since then.

I fear this might be the case, yes…

> I've added some printf debugging to GTK2 (attached) and it looks as
> though the problem is that the layout code flaps between two different
> pixel sizes for the same "line" of text (GTK calls it a "line" but
> it's really more like a paragraph - it will be line-wrapped to fit the
> available width). With the test-case of going back to language selection
> and choosing Sinhala, which I've been using because it's an easy one
> to reproduce:
> 
> May 13 10:24:18 debconf: cdebconf_gtk (process:4590): Gtk - DEBUG: 
> IA__gtk_text_layout_validate_yrange: GtkTextLayout 0x560b7b142110: validating 
> near anchor 0x7f28cd080e20 from 0px to 576px
> May 13 10:24:18 debconf: cdebconf_gtk (process:4590): Gtk - DEBUG: 
> IA__gtk_text_layout_validate_yrange: anchor 0x7f28cd080e20 is in line 0
> May 13 10:24:18 debconf: cdebconf_gtk (process:4590): Gtk - DEBUG: 
> IA__gtk_text_layout_validate_yrange: validating a subsequent line 0
> May 13 10:24:18 debconf: cdebconf_gtk (process:4590): Gtk - DEBUG: 
> IA__gtk_text_layout_validate_yrange: line content: තේරූ භාෂාව සඳහා ස්ථාපකයේ 
> පරිවර්තනය අසම්පූර්ණයි.
> May 13 10:24:18 debconf: cdebconf_gtk (process:4590): Gtk - DEBUG: 
> IA__gtk_text_layout_validate_yrange: was 72x102px
> May 13 10:24:18 debconf: cdebconf_gtk (process:4590): Gtk - DEBUG: 
> IA__gtk_text_layout_validate_yrange: now 73x119px
> May 13 10:24:18 debconf: cdebconf_gtk (process:4590): Gtk - DEBUG: 
> IA__gtk_text_layout_validate_yrange: validating a subsequent line 1
> May 13 10:24:18 debconf: cdebconf_gtk (process:4590): Gtk - DEBUG: 
> IA__gtk_text_layout_validate_yrange: line content:
> May 13 10:24:18 debconf: cdebconf_gtk (process:4590): Gtk - DEBUG: 
> IA__gtk_text_layout_validate_yrange: was 0x16px
> May 13 10:24:18 debconf: cdebconf_gtk (process:4590): Gtk - DEBUG: 
> IA__gtk_text_layout_validate_yrange: now 0x16px
> May 13 10:24:18 debconf: cdebconf_gtk (process:4590): Gtk - DEBUG: 
> IA__gtk_text_layout_validate_yrange: validating a subsequent line 2
> May 13 10:24:18 debconf: cdebconf_gtk (process:4590): Gtk - DEBUG: 
> IA__gtk_text_layout_validate_yrange: line content: මෙහි තේරුම සමහර සංවාද 
> ඉංග්<200d>රීසියෙන් පෙනීමේ සැලකිය යුතු ඉඩක් ඇති බවයි.
> May 13 10:24:18 debconf: cdebconf_gtk (process:4590): Gtk - DEBUG: 
> IA__gtk_text_layout_validate_yrange: was 66x153px
> May 13 10:24:18 debconf: cdebconf_gtk (process:4590): Gtk - DEBUG: 
> IA__gtk_text_layout_validate_yrange: now 66x153px
> May 13 10:24:18 debconf: cdebconf_gtk (process:4590): Gtk - DEBUG: 
> IA__gtk_text_layout_validate_yrange: validating a subsequent line 3
> May 13 10:24:18 debconf: cdebconf_gtk (process:4590): Gtk - DEBUG: 
> IA__gtk_text_layout_validate_yrange: line content:
> May 13 10:24:18 debconf: cdebconf_gtk (process:4590): Gtk - DEBUG: 
> IA__gtk_text_layout_validate_yrange: was 0x16px
> May 13 10:24:18 debconf: cdebconf_gtk (process:4590): Gtk - DEBUG: 
> IA__gtk_text_layout_validate_yrange: now 0x16px
> May 13 10:24:18 debconf: cdebconf_gtk (process:4590): Gtk - DEBUG: 
> IA__gtk_text_layout_validate_yrange: validating a subsequent line 4
> May 13 10:24:18 debconf: cdebconf_gtk (process:4590): Gtk - DEBUG: 
> IA__gtk_text_layout_validate_yrange: line content: ඔබට විකල්ප භාෂාව පිළිබඳ 
> හොඳ අවබෝධයක් නොමැති නම්, එක්කෝ වෙනස් භාෂාවක් තෝරා ගැනීම හෝ ස්ථාපනය නැවැත්වීමට 
> නිර්දේශ කරයි.
> May 13 10:24:18 debconf: cdebconf_gtk (process:4590): Gtk - DEBUG: 
> IA__gtk_text_layout_validate_yrange: was 72x289px
> May 13 10:24:18 debconf: cdebconf_gtk (process:4590): Gtk - DEBUG: 
> IA__gtk_text_layout_validate_yrange: now 70x323px
> May 13 10:24:18 debconf: cdebconf_gtk (process:4590): Gtk - DEBUG: 
> IA__gtk_text_layout_validate_yrange: Revalidated invalid lines from 0 to 4
> May 13 10:24:18 debconf: cdebconf_gtk (process:4590): Gtk - DEBUG: 
> IA__gtk_text_layout_validate_yrange: Top of first line y=0
> May 13 10:24:18 debconf: cdebconf_gtk (process:4590): Gtk - DEBUG: 
> 

Bug#987587: libpango1.0-udeb: hangs the installer in various situations

[Simon McVittie]

On Tue, 04 May 2021 at 15:47:03 +0200, Cyril Brulebois wrote:
> would it seem reasonable to add a hack in whatever would make sense
> (pango1.0, harfbuzz, and/or gtk+2.0), but only in the udeb build, so
> that we dodge the issue for Bullseye without impacting installed
> systems/deb packages? If git bisecting yields a prospective (set of)
> patch(es) of course…

Adding a udeb-specific hack in one of those packages seems reasonable,
*if* we can find a suitable hack.

pango1.0 just has one build shared between the udeb and the full .deb,
so if the hack goes there, a prerequisite would be to build it twice
(fairly straightforward, just annoying). gtk+2.0 and harfbuzz already
build twice, so it would be easy to add -DDEBIAN_INSTALLER to the udeb
build, or similar.

The problem is finding the right hack. I don't think bisecting Pango is
necessarily going to get you very far: if you arrive at the version/commit
where Pango switched from doing text layout internally to using Harfbuzz
for everything, then that'll be a dead end. Conditionally reverting the
switch from internal layout engines to Harfbuzz doesn't seem likely to
yield a reasonable patch - the internal layout engines were deleted about
2 years ago and the rest of the pango codebase has moved on since then.

I've added some printf debugging to GTK2 (attached) and it looks as
though the problem is that the layout code flaps between two different
pixel sizes for the same "line" of text (GTK calls it a "line" but
it's really more like a paragraph - it will be line-wrapped to fit the
available width). With the test-case of going back to language selection
and choosing Sinhala, which I've been using because it's an easy one
to reproduce:

May 13 10:24:18 debconf: cdebconf_gtk (process:4590): Gtk - DEBUG: 
IA__gtk_text_layout_validate_yrange: GtkTextLayout 0x560b7b142110: validating 
near anchor 0x7f28cd080e20 from 0px to 576px
May 13 10:24:18 debconf: cdebconf_gtk (process:4590): Gtk - DEBUG: 
IA__gtk_text_layout_validate_yrange: anchor 0x7f28cd080e20 is in line 0
May 13 10:24:18 debconf: cdebconf_gtk (process:4590): Gtk - DEBUG: 
IA__gtk_text_layout_validate_yrange: validating a subsequent line 0
May 13 10:24:18 debconf: cdebconf_gtk (process:4590): Gtk - DEBUG: 
IA__gtk_text_layout_validate_yrange: line content: තේරූ භාෂාව සඳහා ස්ථාපකයේ 
පරිවර්තනය අසම්පූර්ණයි.
May 13 10:24:18 debconf: cdebconf_gtk (process:4590): Gtk - DEBUG: 
IA__gtk_text_layout_validate_yrange: was 72x102px
May 13 10:24:18 debconf: cdebconf_gtk (process:4590): Gtk - DEBUG: 
IA__gtk_text_layout_validate_yrange: now 73x119px
May 13 10:24:18 debconf: cdebconf_gtk (process:4590): Gtk - DEBUG: 
IA__gtk_text_layout_validate_yrange: validating a subsequent line 1
May 13 10:24:18 debconf: cdebconf_gtk (process:4590): Gtk - DEBUG: 
IA__gtk_text_layout_validate_yrange: line content:
May 13 10:24:18 debconf: cdebconf_gtk (process:4590): Gtk - DEBUG: 
IA__gtk_text_layout_validate_yrange: was 0x16px
May 13 10:24:18 debconf: cdebconf_gtk (process:4590): Gtk - DEBUG: 
IA__gtk_text_layout_validate_yrange: now 0x16px
May 13 10:24:18 debconf: cdebconf_gtk (process:4590): Gtk - DEBUG: 
IA__gtk_text_layout_validate_yrange: validating a subsequent line 2
May 13 10:24:18 debconf: cdebconf_gtk (process:4590): Gtk - DEBUG: 
IA__gtk_text_layout_validate_yrange: line content: මෙහි තේරුම සමහර සංවාද 
ඉංග්<200d>රීසියෙන් පෙනීමේ සැලකිය යුතු ඉඩක් ඇති බවයි.
May 13 10:24:18 debconf: cdebconf_gtk (process:4590): Gtk - DEBUG: 
IA__gtk_text_layout_validate_yrange: was 66x153px
May 13 10:24:18 debconf: cdebconf_gtk (process:4590): Gtk - DEBUG: 
IA__gtk_text_layout_validate_yrange: now 66x153px
May 13 10:24:18 debconf: cdebconf_gtk (process:4590): Gtk - DEBUG: 
IA__gtk_text_layout_validate_yrange: validating a subsequent line 3
May 13 10:24:18 debconf: cdebconf_gtk (process:4590): Gtk - DEBUG: 
IA__gtk_text_layout_validate_yrange: line content:
May 13 10:24:18 debconf: cdebconf_gtk (process:4590): Gtk - DEBUG: 
IA__gtk_text_layout_validate_yrange: was 0x16px
May 13 10:24:18 debconf: cdebconf_gtk (process:4590): Gtk - DEBUG: 
IA__gtk_text_layout_validate_yrange: now 0x16px
May 13 10:24:18 debconf: cdebconf_gtk (process:4590): Gtk - DEBUG: 
IA__gtk_text_layout_validate_yrange: validating a subsequent line 4
May 13 10:24:18 debconf: cdebconf_gtk (process:4590): Gtk - DEBUG: 
IA__gtk_text_layout_validate_yrange: line content: ඔබට විකල්ප භාෂාව පිළිබඳ හොඳ 
අවබෝධයක් නොමැති නම්, එක්කෝ වෙනස් භාෂාවක් තෝරා ගැනීම හෝ ස්ථාපනය නැවැත්වීමට 
නිර්දේශ කරයි.
May 13 10:24:18 debconf: cdebconf_gtk (process:4590): Gtk - DEBUG: 
IA__gtk_text_layout_validate_yrange: was 72x289px
May 13 10:24:18 debconf: cdebconf_gtk (process:4590): Gtk - DEBUG: 
IA__gtk_text_layout_validate_yrange: now 70x323px
May 13 10:24:18 debconf: cdebconf_gtk (process:4590): Gtk - DEBUG: 
IA__gtk_text_layout_validate_yrange: Revalidated invalid lines from 0 to 4
May 13 10:24:18 debconf: cdebconf_gtk (process:4590): Gtk - DEBUG: 
IA__gtk_text_layout_validate_yrange: Top of first 

Bug#987587: libpango1.0-udeb: hangs the installer in various situations

[Cyril Brulebois]

Hi Simon,

And thanks a lot for debugging/digging/sharing!

Simon McVittie  (2021-05-03):
> On Mon, 03 May 2021 at 17:17:21 +0100, Simon McVittie wrote:
> > I've also been able to attach a debugger to debconf. My preliminary finding
> > is: we enter gtk_container_idle_sizer() in GTK 2 and never exit, because
> > every time we go into gtk_container_check_resize(), we end up in
> > gtk_text_layout_emit_changed() which emits a signal that eventually calls
> > _gtk_container_queue_resize(), so gtk_container_idle_sizer() has more work
> > to do and we loop indefinitely.
> ...
> > My next step is going to be to try to hack Harfbuzz to disable the Indic
> > shaper (which is what seems to be in use here) and see whether that stops
> > the infinite loop. That's unlikely to be an acceptable solution, but it'll
> > at least tell us whether the Indic shaper is what's triggering this.
> 
> Yes, this worked! If I disable the Indic shaper with the attached hack,
> that seems to be enough to make installation proceed.

Yes and no: the Indic shaper triggers it for Sinhala indeed (confirmed
locally by deploying a hack harfbuzz udeb with your patch on top of
exactly the components used for D-I Bullseye RC 1), but that doesn't
explain the failure for Persian.

Of course, one can also apply the same kind of hack to other scripts
that share the same dedicated Arabic shaper, via:

-   return &_hb_ot_complex_shaper_arabic;
+   return &_hb_ot_complex_shaper_default;

While the problem was obvious for certain languages (first screen after
language selection, #987449), and could exist for any languages using a
script that comes with a dedicated shaper, that still doesn't explain
the regression with English in rescue mode (#987377). I fear this issue
could actually show up with any languages (e.g. all those sharing the
Latin script), depending on $some_conditions.

I think I'll add a GTK-level patch to easily spot whether that last
issue is similar (infinite loop with signals all around the place), and
maybe try to pinpoint when the problem started to happen in the pango1.0
Git repository.

There's also a problem with Swedish during regular install (not rescue),
that I haven't tried to reproduce yet:
  https://lists.debian.org/debian-boot/2021/05/msg00018.html

This would seem to confirm one doesn't need a “complex” script to get
the issue, Latin is enough…

> Again, this is probably not an acceptable solution: Harfbuzz has
> shapers for complex scripts for a reason, and I suspect someone who
> can speak the relevant language would find the text either ugly or
> unreadable when using the default shaper. However, I hope this does at
> least point someone who knows more about the mechanics of text
> rendering and/or GTK relayout in the right direction...

Despite all the above, once we have a better grasp of what's happening,
would it seem reasonable to add a hack in whatever would make sense
(pango1.0, harfbuzz, and/or gtk+2.0), but only in the udeb build, so
that we dodge the issue for Bullseye without impacting installed
systems/deb packages? If git bisecting yields a prospective (set of)
patch(es) of course…

ISTR you proposed then implemented changes to alleviate one of the
blockers for GTK3 in d-i (vte vs. libstdc++ if memory serves), but I
haven't managed to look at it during the bullseye cycle; we definitely
should look into gtk3-ifying (perhaps gtk4-ifying, time flies) d-i at
some point, but that's very likely bookworm material at this stage…

> I think the reason this regressed between Pango 1.43.0 and 1.44.0
> might just be that Pango 1.44.0 uses Harfbuzz to implement more of its
> own functionality.

Looking at the “user-level” changes in the upstream documentation, that
is indeed what seems very plausible to my inexperienced eyes.


Cheers,
-- 
Cyril Brulebois (k...@debian.org)
D-I release manager -- Release team member -- Freelance Consultant


signature.asc
Description: PGP signature

Bug#987587: libpango1.0-udeb: hangs the installer in various situations

[Simon McVittie]

On Mon, 03 May 2021 at 17:17:21 +0100, Simon McVittie wrote:
> I've also been able to attach a debugger to debconf. My preliminary finding
> is: we enter gtk_container_idle_sizer() in GTK 2 and never exit, because
> every time we go into gtk_container_check_resize(), we end up in
> gtk_text_layout_emit_changed() which emits a signal that eventually calls
> _gtk_container_queue_resize(), so gtk_container_idle_sizer() has more work
> to do and we loop indefinitely.
...
> My next step is going to be to try to hack Harfbuzz to disable the Indic
> shaper (which is what seems to be in use here) and see whether that stops
> the infinite loop. That's unlikely to be an acceptable solution, but it'll
> at least tell us whether the Indic shaper is what's triggering this.

Yes, this worked! If I disable the Indic shaper with the attached hack,
that seems to be enough to make installation proceed.

Again, this is probably not an acceptable solution: Harfbuzz has shapers
for complex scripts for a reason, and I suspect someone who can speak
the relevant language would find the text either ugly or unreadable
when using the default shaper. However, I hope this does at least point
someone who knows more about the mechanics of text rendering and/or GTK
relayout in the right direction...

I think the reason this regressed between Pango 1.43.0 and 1.44.0 might
just be that Pango 1.44.0 uses Harfbuzz to implement more of its own
functionality.

smcv
From: Simon McVittie 
Date: Mon, 3 May 2021 17:02:50 +0100
Subject: HACK: Disable Indic shaper

This appears to trigger a relayout loop in GTK 2 in the d-i environment.

Bug-Debian: https://bugs.debian.org/987587
---
 src/hb-ot-shape-complex.hh | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/src/hb-ot-shape-complex.hh b/src/hb-ot-shape-complex.hh
index a1a7a6a..6c82b87 100644
--- a/src/hb-ot-shape-complex.hh
+++ b/src/hb-ot-shape-complex.hh
@@ -263,7 +263,7 @@ hb_ot_shape_complex_categorize (const hb_ot_shape_planner_t *planner)
   else if ((planner->map.chosen_script[0] & 0x00FF) == '3')
 	return &_hb_ot_complex_shaper_use;
   else
-	return &_hb_ot_complex_shaper_indic;
+	return &_hb_ot_complex_shaper_default;
 
 case HB_SCRIPT_KHMER:
 	return &_hb_ot_complex_shaper_khmer;

Bug#987587: libpango1.0-udeb: hangs the installer in various situations

[Simon McVittie]

On Thu, 29 Apr 2021 at 18:54:56 +0200, Cyril Brulebois wrote:
> Version 1.44.4 is the first one I was able to build, using the packaging
> from debian/1.44.6-1 (first 1.44.x version that was packaged and that's
> also known to be buggy).

I've been able to hack together packaging for 1.43.0 (works) and 1.44.0
(hangs when selecting Sinhala), but so much changed between 1.43.0 and
1.44.0 that bisecting between them might not be a very good route.

I've also been able to attach a debugger to debconf. My preliminary finding
is: we enter gtk_container_idle_sizer() in GTK 2 and never exit, because
every time we go into gtk_container_check_resize(), we end up in
gtk_text_layout_emit_changed() which emits a signal that eventually calls
_gtk_container_queue_resize(), so gtk_container_idle_sizer() has more work
to do and we loop indefinitely.

I assume this isn't meant to happen? But please don't mistake me for an
expert on GTK 2, or Pango, or Harfbuzz.

Steps go something like this (is there a better way?):

Host preparation:
- Build interesting libraries (pango1.0, harfbuzz, glib2.0, gtk+2.0) with
  DEB_BUILD_OPTIONS="noopt nostrip nocheck"; we'll need these later

VM preparation:
- Create a new VM booting from virtio disk /dev/vda
- Have an up-to-date bullseye installation on /dev/vda
  (I copied an existing bullseye GNOME installation and upgraded it)
- Make sure a user can log in via ssh and can sudo to root
- Power off the VM
- Attach a CD-ROM device with firmware-bullseye-DI-rc1-amd64-netinst.iso
- Make it boot from CD-ROM
- Boot it
- Start graphical installer
- Proceed with installation, in English, until we get to "Configure the
  network"
- Hit Cancel and Go Back until we get to the menu of installation steps
- Send Ctrl+Alt+F2
- modprobe ext4
- mount /dev/vda1 /mnt
- mount --rbind /dev /mnt/dev
- mount --rbind /proc /mnt/proc
- mount --rbind /sys /mnt/sys
- mkdir /mnt/d-i
- mount --bind / /mnt/d-i
- chroot /mnt bash
  - /etc/init.d/ssh start
  - exit
- ssh in to the chroot'd ssh server; this is our debugging environment
- On the graphical console, send Ctrl+Alt+F5 to go to X11

Debugging loop:
- Use rsync/ssh/cp to get the unstripped libraries into /d-i in the ssh
  session (the root directory of the d-i environment)
- Ctrl+Alt+F2, use udpkg -i to install them in the d-i environment
- In the ssh session, use pstree -p to look at the process hierarchy,
  then kill debconf and everything below it
- Hit Cancel and Go Back until we get to the menu of installation steps
- "Choose language"
- Select Sinhala
- In the ssh session:
sudo gdb -ex 'set pagination off' \
-ex 'set sysroot /d-i' \
-ex 'set logging file /log' \
-ex 'set logging overwrite off' \
-ex 'set logging on' \
/d-i/usr/bin/debconf `pgrep debconf`
- Debug interactively until you get a better idea

Some sample backtraces are attached.

My next step is going to be to try to hack Harfbuzz to disable the Indic
shaper (which is what seems to be in use here) and see whether that stops
the infinite loop. That's unlikely to be an acceptable solution, but it'll
at least tell us whether the Indic shaper is what's triggering this.

smcv

Thread 2 (Thread 0x7fb793d9b700 (LWP 4430) "event_listener"):
#0  BEInt::operator unsigned short (this=0x7fb792d24fa4) at 
../../src/hb.hh:559
#1  0x7fb7943fe084 in OT::IntType::operator unsigned 
int (this=0x7fb792d24fa4) at ../../src/hb-open-type.hh:63
#2  0x7fb7943f9c4b in OT::CoverageFormat2::get_coverage 
(this=0x7fb792d24fa0, glyph_id=49) at ../../src/hb-ot-layout-common.hh:1300
#3  0x7fb7943fa001 in OT::Coverage::get_coverage (this=0x7fb792d24fa0, 
glyph_id=49) at ../../src/hb-ot-layout-common.hh:1470
#4  0x7fb7944c736a in OT::CursivePosFormat1::apply (this=0x7fb792d249fe, 
c=0x7fb793d95eb0) at ../../src/hb-ot-layout-gpos-table.hh:1603
#5  0x7fb79450cae3 in 
OT::hb_get_subtables_context_t::apply_to 
(obj=0x7fb792d249fe, c=0x7fb793d95eb0) at ../../src/hb-ot-layout-gsubgpos.hh:692
#6  0x7fb7944bf2fc in 
OT::hb_get_subtables_context_t::hb_applicable_t::apply (this=0x7fb78c1db610, 
c=0x7fb793d95eb0) at ../../src/hb-ot-layout-gsubgpos.hh:710
#7  0x7fb7944c4074 in OT::hb_ot_layout_lookup_accelerator_t::apply 
(this=0x7fb78c1db2c0, c=0x7fb793d95eb0) at 
../../src/hb-ot-layout-gsubgpos.hh:3189
#8  0x7fb7944a5a5d in apply_forward (c=0x7fb793d95eb0, accel=...) at 
../../src/hb-ot-layout.cc:1766
#9  0x7fb7944e13df in apply_string (c=0x7fb793d95eb0, 
lookup=..., accel=...) at ../../src/hb-ot-layout.cc:1819
#10 0x7fb7944d82d8 in hb_ot_map_t::apply (this=0x7fb78c1dbbd8, 
proxy=..., plan=0x7fb78c1dbbb0, font=0x7fb78c3ae090, buffer=0x556bd8cdacd0) at 
../../src/hb-ot-layout.cc:1865
#11 0x7fb7944a5cc5 in hb_ot_map_t::position (this=0x7fb78c1dbbd8, 
plan=0x7fb78c1dbbb0, font=0x7fb78c3ae090, buffer=0x556bd8cdacd0) at 
../../src/hb-ot-layout.cc:1891
#12 0x7fb794559cd4 in hb_ot_shape_plan_t::position (this=0x7fb78c1dbbb0, 
font=0x7fb78c3ae090, 

Bug#987587: libpango1.0-udeb: hangs the installer in various situations

[Chris Hofstaedtler]

* Cyril Brulebois  [210430 09:47]:
> Anyway, looking at the diffstat between 1.43.0 (last known good) and
> 1.44.4 (first known bad), we have this:
>  179 files changed, 19157 insertions(+), 7045 deletions(-)
> 
> Between 1.43.0 and 1.44 (if it were an earlier known bad):
>  165 files changed, 11006 insertions(+), 6766 deletions(-)
> 
> which is a huge diff still.

Indeed :-(

> From your earlier report on #987449[1], something around
> pango_default_break seemed to be happening. (In passing, I'd love to
> learn how you generated it so that I can try my luck with #987377.)

Quick steps:
- I created a Debian bullseye install with a kernel version matching
  the installer, and also installed linux-perf(-5.10)
- From the running, already stuck installer, I mounted the existing
  install, did the whole mount --bind stuff for sys, proc, dev.
  I cannot remember if one needs to actually chroot.
- And then its mostly `perf top` and/or `perf record` + `perf
  report`. The former gives you an immediate view, while the latter
  writes into a file which you can look at later.

Having more debug symbols would have been useful, but I did not
manage to set that up.

Chris

Bug#987587: libpango1.0-udeb: hangs the installer in various situations

[Cyril Brulebois]

Chris Hofstaedtler  (2021-04-29):
> > Alright, I think I'm hitting my limits here.
> > 
> > Versions 1.44, 1.44.2, and 1.44.3 all fail in a similar way:
>  ../pango/pangofc-font.c:371:44: error: ‘HB_OT_METRICS_UNDERLINE_SIZE’ 
> undeclared (first use in this function); did you mean 
> ‘HB_OT_METRICS_TAG_UNDERLINE_SIZE’?
> [..]
> 
> Just passing by, but I can build 1.44.0 with Debians 1.44.6 debian/
> directory, when adding this patch from upstream:
> https://gitlab.gnome.org/GNOME/pango/-/commit/d835004502c801a8a16cc436a38900e548ecde52.patch

Yes, the earlier stable releases from the 1.44.x series looked like
build-side stabilization and I suppose I could end up cherry-picking one
or more patches to get a bit further.

Anyway, looking at the diffstat between 1.43.0 (last known good) and
1.44.4 (first known bad), we have this:
 179 files changed, 19157 insertions(+), 7045 deletions(-)

Between 1.43.0 and 1.44 (if it were an earlier known bad):
 165 files changed, 11006 insertions(+), 6766 deletions(-)

which is a huge diff still.

From your earlier report on #987449[1], something around
pango_default_break seemed to be happening. (In passing, I'd love to
learn how you generated it so that I can try my luck with #987377.)

 1. https://bugs.debian.org/987449#15

and that helps a lot dig into that huge diff, which touched the file
holding that function, adding e.g. this hunk:

+   /* Dependent Vowels for Indic language */
+   if (_pango_is_Virama (prev_wc) ||
+   _pango_is_Vowel_Dependent (prev_wc))
+ attrs[i].backspace_deletes_character = TRUE;

Not knowing anything about it:
  https://en.wikipedia.org/wiki/Indic_languages

redirects to:
  https://en.wikipedia.org/wiki/Indo-Aryan_languages

which lists Sinhala, Marathi, but neither Persian nor Kannada.

There are a lot of other changes in that file anyway…


In the end, I might try and bisect deeper, if I can get stuff to build
without too much hassle…


Cheers,
-- 
Cyril Brulebois (k...@debian.org)
D-I release manager -- Release team member -- Freelance Consultant


signature.asc
Description: PGP signature

Bug#987587: libpango1.0-udeb: hangs the installer in various situations

[Samuel Thibault]

Cyril Brulebois, le jeu. 29 avril 2021 18:54:56 +0200, a ecrit:
> Versions 1.44, 1.44.2, and 1.44.3 all fail in a similar way:
> 
> ../pango/pangofc-font.c: In function ‘get_face_metrics’:
> ../pango/pangofc-font.c:371:44: error: ‘HB_OT_METRICS_UNDERLINE_SIZE’ 
> undeclared (first use in this function); did you mean 
> ‘HB_OT_METRICS_TAG_UNDERLINE_SIZE’?

It seems it was just a renaming at some hb point, pango later got

commit d835004502c801a8a16cc436a38900e548ecde52
Author: Ebrahim Byagowi 
Date:   Sat Aug 10 14:05:40 2019 +0430

Use latest version of metrics naming in pangofc-font

diff --git a/pango/pangofc-font.c b/pango/pangofc-font.c
index 98e77288..21644b57 100644
--- a/pango/pangofc-font.c
+++ b/pango/pangofc-font.c
@@ -365,16 +365,16 @@ get_face_metrics (PangoFcFont  *fcfont,
 #if HB_VERSION_ATLEAST(2,5,4)
   hb_position_t position;

-  if (hb_ot_metrics_get_position (hb_font, HB_OT_METRICS_UNDERLINE_SIZE, 
))
+  if (hb_ot_metrics_get_position (hb_font, HB_OT_METRICS_TAG_UNDERLINE_SIZE, 
))
 metrics->underline_thickness = position;

-  if (hb_ot_metrics_get_position (hb_font, HB_OT_METRICS_UNDERLINE_OFFSET, 
))
+  if (hb_ot_metrics_get_position (hb_font, HB_OT_METRICS_TAG_UNDERLINE_OFFSET, 
))
 metrics->underline_position = position;
 
-  if (hb_ot_metrics_get_position (hb_font, HB_OT_METRICS_STRIKEOUT_SIZE, 
))
+  if (hb_ot_metrics_get_position (hb_font, HB_OT_METRICS_TAG_STRIKEOUT_SIZE, 
))
 metrics->strikethrough_thickness = position;
 
-  if (hb_ot_metrics_get_position (hb_font, HB_OT_METRICS_STRIKEOUT_OFFSET, 
))
+  if (hb_ot_metrics_get_position (hb_font, HB_OT_METRICS_TAG_STRIKEOUT_OFFSET, 
))
 metrics->strikethrough_position = position;
 #endif
 }

which probably can thus be used as it is.

Samuel

Bug#987587: libpango1.0-udeb: hangs the installer in various situations

[Chris Hofstaedtler]

* Cyril Brulebois  [210429 18:03]:
> Cyril Brulebois  (2021-04-29):
> > This time around, testing 1.43.0 with debian/ carried over from
> > debian/1.42.4-8, adjusted for docs (some files are missing) and symbols
> > (one new symbol), the problem cannot be triggered in an obvious manner.
> > 
> > Moving to 1.44* tags now.
> 
> Alright, I think I'm hitting my limits here.
> 
> Versions 1.44, 1.44.2, and 1.44.3 all fail in a similar way:
 ../pango/pangofc-font.c:371:44: error: ‘HB_OT_METRICS_UNDERLINE_SIZE’ 
undeclared (first use in this function); did you mean 
‘HB_OT_METRICS_TAG_UNDERLINE_SIZE’?
[..]

Just passing by, but I can build 1.44.0 with Debians 1.44.6 debian/
directory, when adding this patch from upstream:
https://gitlab.gnome.org/GNOME/pango/-/commit/d835004502c801a8a16cc436a38900e548ecde52.patch

HTH,
Chris

Bug#987587: libpango1.0-udeb: hangs the installer in various situations

[Cyril Brulebois]

Cyril Brulebois  (2021-04-29):
> This time around, testing 1.43.0 with debian/ carried over from
> debian/1.42.4-8, adjusted for docs (some files are missing) and symbols
> (one new symbol), the problem cannot be triggered in an obvious manner.
> 
> Moving to 1.44* tags now.

Alright, I think I'm hitting my limits here.

Versions 1.44, 1.44.2, and 1.44.3 all fail in a similar way:

[39/144] cc -Ipango/libpangoft2-1.0.so.0.4400.0.p -Ipango -I../pango -I. 
-I.. -I/usr/include/glib-2.0 -I/usr/lib/x86_64-linux-gnu/glib-2.0/include 
-I/usr/include/fribidi -I/usr/include/harfbuzz -I/usr/include/freetype2 
-I/usr/include/libpng16 -I/usr/include/uuid -I/usr/include/cairo 
-I/usr/include/pixman-1 -fdiagnostics-color=always -pipe -D_FILE_OFFSET_BITS=64 
-Wall -Winvalid-pch -std=gnu99 -D_POSIX_C_SOURCE=200809L 
-D_POSIX_THREAD_SAFE_FUNCTIONS -D_GNU_SOURCE -g -O2 
-ffile-prefix-map=/home/kibi/hack/pango1.0.git=. -fstack-protector-strong 
-Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC 
-Wimplicit-function-declaration -Wstrict-prototypes -Wmissing-prototypes 
-Wnested-externs -Wold-style-definition -Wno-int-conversion 
-Wno-discarded-qualifiers -fno-strict-aliasing -Wpointer-arith 
-Wmissing-declarations -Wformat=2 -Wformat-nonliteral -Wformat-security 
-Wunused -Wcast-align -Wmissing-noreturn -Wmissing-format-attribute 
-Wmissing-include-dirs -Wlogical-op -Wno-uninitialized -Wno-shadow 
-Werror=implicit -Werror=nonnull -Werror=init-self -Werror=main 
-Werror=missing-braces -Werror=sequence-point -Werror=return-type 
-Werror=trigraphs -Werror=array-bounds -Werror=write-strings -Werror=address 
-Werror=int-to-pointer-cast -Werror=pointer-to-int-cast -Werror=empty-body 
-Werror=write-strings -Wundef -Werror=redundant-decls -fvisibility=hidden 
'-DG_LOG_DOMAIN="Pango"' -DG_LOG_USE_STRUCTURED=1 -DPANGO_COMPILATION 
'-DSYSCONFDIR="/etc"' '-DLIBDIR="/usr/lib/x86_64-linux-gnu"' 
-DPANGO_DISABLE_DEPRECATION_WARNINGS -MD -MQ 
pango/libpangoft2-1.0.so.0.4400.0.p/pangofc-font.c.o -MF 
pango/libpangoft2-1.0.so.0.4400.0.p/pangofc-font.c.o.d -o 
pango/libpangoft2-1.0.so.0.4400.0.p/pangofc-font.c.o -c ../pango/pangofc-font.c
FAILED: pango/libpangoft2-1.0.so.0.4400.0.p/pangofc-font.c.o 
cc -Ipango/libpangoft2-1.0.so.0.4400.0.p -Ipango -I../pango -I. -I.. 
-I/usr/include/glib-2.0 -I/usr/lib/x86_64-linux-gnu/glib-2.0/include 
-I/usr/include/fribidi -I/usr/include/harfbuzz -I/usr/include/freetype2 
-I/usr/include/libpng16 -I/usr/include/uuid -I/usr/include/cairo 
-I/usr/include/pixman-1 -fdiagnostics-color=always -pipe -D_FILE_OFFSET_BITS=64 
-Wall -Winvalid-pch -std=gnu99 -D_POSIX_C_SOURCE=200809L 
-D_POSIX_THREAD_SAFE_FUNCTIONS -D_GNU_SOURCE -g -O2 
-ffile-prefix-map=/home/kibi/hack/pango1.0.git=. -fstack-protector-strong 
-Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC 
-Wimplicit-function-declaration -Wstrict-prototypes -Wmissing-prototypes 
-Wnested-externs -Wold-style-definition -Wno-int-conversion 
-Wno-discarded-qualifiers -fno-strict-aliasing -Wpointer-arith 
-Wmissing-declarations -Wformat=2 -Wformat-nonliteral -Wformat-security 
-Wunused -Wcast-align -Wmissing-noreturn -Wmissing-format-attribute 
-Wmissing-include-dirs -Wlogical-op -Wno-uninitialized -Wno-shadow 
-Werror=implicit -Werror=nonnull -Werror=init-self -Werror=main 
-Werror=missing-braces -Werror=sequence-point -Werror=return-type 
-Werror=trigraphs -Werror=array-bounds -Werror=write-strings -Werror=address 
-Werror=int-to-pointer-cast -Werror=pointer-to-int-cast -Werror=empty-body 
-Werror=write-strings -Wundef -Werror=redundant-decls -fvisibility=hidden 
'-DG_LOG_DOMAIN="Pango"' -DG_LOG_USE_STRUCTURED=1 -DPANGO_COMPILATION 
'-DSYSCONFDIR="/etc"' '-DLIBDIR="/usr/lib/x86_64-linux-gnu"' 
-DPANGO_DISABLE_DEPRECATION_WARNINGS -MD -MQ 
pango/libpangoft2-1.0.so.0.4400.0.p/pangofc-font.c.o -MF 
pango/libpangoft2-1.0.so.0.4400.0.p/pangofc-font.c.o.d -o 
pango/libpangoft2-1.0.so.0.4400.0.p/pangofc-font.c.o -c ../pango/pangofc-font.c
../pango/pangofc-font.c: In function ‘get_face_metrics’:
../pango/pangofc-font.c:371:44: error: ‘HB_OT_METRICS_UNDERLINE_SIZE’ 
undeclared (first use in this function); did you mean 
‘HB_OT_METRICS_TAG_UNDERLINE_SIZE’?
  371 |   if (hb_ot_metrics_get_position (hb_font, 
HB_OT_METRICS_UNDERLINE_SIZE, ))
  |
^~~~
  |
HB_OT_METRICS_TAG_UNDERLINE_SIZE
../pango/pangofc-font.c:371:44: note: each undeclared identifier is 
reported only once for each function it appears in
../pango/pangofc-font.c:374:44: error: ‘HB_OT_METRICS_UNDERLINE_OFFSET’ 
undeclared (first use in this function); did you mean 
‘HB_OT_METRICS_TAG_UNDERLINE_OFFSET’?
  374 |   if (hb_ot_metrics_get_position (hb_font, 
HB_OT_METRICS_UNDERLINE_OFFSET, ))
  |
^~
  |  

Bug#987587: libpango1.0-udeb: hangs the installer in various situations

[Cyril Brulebois]

Cyril Brulebois  (2021-04-26):
>  - I'm *not yet* certain this also triggers direct failures for Netinst
>images for several languages, tracked at:
>  https://bugs.debian.org/987449
…
>I'll try to confirm this in the next few days.

It's been confirmed, so I've added a “block” between both reports.

I've also confirmed an earlier version, uploaded to experimental
(1.44.6-1) also triggers the issue, hence the recent “found”.

I'm now down to investigating upstream releases that weren't packaged
into Debian. For that, I'm using #987449 with Sinhala (since that's
almost instantaneous, as opposed to #987377 which needs going through
various menus).

This time around, testing 1.43.0 with debian/ carried over from
debian/1.42.4-8, adjusted for docs (some files are missing) and symbols
(one new symbol), the problem cannot be triggered in an obvious manner.

Moving to 1.44* tags now.


Cheers,
-- 
Cyril Brulebois (k...@debian.org)
D-I release manager -- Release team member -- Freelance Consultant


signature.asc
Description: PGP signature

Bug#987587: libpango1.0-udeb: hangs the installer in various situations

[Cyril Brulebois]

Package: libpango1.0-udeb
Version: 1.44.7-3
Severity: grave
Tags: d-i
Justification: renders package unusable
X-Debbugs-Cc: debian-b...@lists.debian.org

Hi,

In the “better late than never” department… I've completed a debugging session
that shows that upgrading libpango1.0-udeb from 1.42.4-8 to 1.44.7-3 hangs the
installer in various situations.

Interestingly, it doesn't *always* do that.

 - I know for sure this upgrade is responsible for the rescue mode
   breakage *under some locales* tracked at:
 https://bugs.debian.org/987377

   e.g. getting a shell into the system to be rescued works fine in
   French or in German, but not in English (which hasn't any crazy fonts
   or glyphs compared to the two other ones).


 - It seems to a kind user performing many tests that this could be
   linked to the actual hardware configuration:
 https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=987377#54

   (See the “For short […]” conclusion, it's priceless.)

   I'll be trying to get more results from Étienne.


 - I'm *not yet* certain this also triggers direct failures for Netinst
   images for several languages, tracked at:
 https://bugs.debian.org/987449

   but I have a rather strong suspicion, as mentioned in:
 https://bugs.debian.org/987377#59

   I'll try to confirm this in the next few days.


Chris Hofstaedtler provided some perf data, which might help figure out
what's happening for someone who's knowledgeable about pango internals:
  https://bugs.debian.org/987449#15

Please keep debian-b...@lists.debian.org in the loop when replying, and
thanks for your help!


Cheers,
-- 
Cyril Brulebois (k...@debian.org)
D-I release manager -- Release team member -- Freelance Consultant