Bug#894159: transition: icu

[Emilio Pozuelo Monfort]

On 13/05/18 22:52, László Böszörményi (GCS) wrote:
> Last but not least I've a different understanding (again) of package
> transitions with Adrian Bunk. He reassigned #898465 [2] to ICU, when both
> related bugreports of ncmpcpp [3] and viking due to mapnik [4] are due to
> partial upgrades (shown in the package versions 0.8.1-1 and 3.0.20+ds-1
> respectively that these are _not_ the binNMUed ones). Checking the build
> logs show mapnik and ncmpcpp rebuilt normally and I've even tested those
> correctly. The bug reporters need to do a full packages upgrade, that's all.
> Then the migration script knows about dependencies and will not let icu
> enter testing by itself, only with the transitioned dependencies.

icu should migrate to testing even before all the rdeps migrate. That could
cause mapnik or ncmpcpp, for example, to stay at older versions, with boost1.62
and icu at the new ones, breaking those packages. Besides, we support partial
upgrades.

The solution I see here is to add Breaks in boost1.62 against mapnik and ncmpcpp
(and other affected packages, if any). That should solve this bug for now, and
while not ideal, we should have a boost transition eventually so I don't mind
using Breaks in this situation.

Cheers,
Emilio

Bug#894159: transition: icu

[GCS]

On Thu, May 3, 2018 at 8:36 AM Emilio Pozuelo Monfort 
wrote:
> Control: tags -1 confirmed

> On 01/05/18 19:29, László Böszörményi (GCS) wrote:
> > OK, the fresh transition testing is the following.

> Alright, let's do this.
  I was off the grid on the weekend, but a quick summary how it goes.
ICU -> icu-le-hb -> ICU with Paragraph Layout compiled successfully on all
of our primary architectures. But harfbuzz had to be binNMUed as well. On
sparc64 harfbuzz FTBFS for a while (since last October) and this
architecture won't make the transition. :(
Then openttd was binNMUed too quick, please reschedule that strictly with
icu >= 60.2-6 package version.
I don't see if texlive-bin was binNMUed, you might have missed that. Please
start that.
Unfortunately chromium browser FTBFS on armhf since this January and will
not be able to migrate to testing due to this.
haskell-pandoc-citeproc is an other package which will be unable to migrate
as one of its build dependency, panadoc FTBFS on armel and armhf.
frog FTBFS as it needs a transitioned ucto which is done now. Please
reschedule its binNMU. Note that it will fail on hurd-i386 as while ucto is
built on this architecture, it's not uploaded even to the archives.

qt4-x11 doesn't show up in the automatic ICU transition list and it FTBFS
with it. The bug reporter also supplied a working fix[1]. Maintainer may
upload the fixed package version tomorrow.

Last but not least I've a different understanding (again) of package
transitions with Adrian Bunk. He reassigned #898465 [2] to ICU, when both
related bugreports of ncmpcpp [3] and viking due to mapnik [4] are due to
partial upgrades (shown in the package versions 0.8.1-1 and 3.0.20+ds-1
respectively that these are _not_ the binNMUed ones). Checking the build
logs show mapnik and ncmpcpp rebuilt normally and I've even tested those
correctly. The bug reporters need to do a full packages upgrade, that's all.
Then the migration script knows about dependencies and will not let icu
enter testing by itself, only with the transitioned dependencies. Please
tell me if I'm might be wrong, otherwise I plan to close these bug reports.

Cheers,
Laszlo/GCS
[1] https://bugs.debian.org/898542#27
[2] https://bugs.debian.org/898465#22
[3] https://bugs.debian.org/898369#5
[4] https://bugs.debian.org/898465#5

Processed: Re: Bug#894159: transition: icu

[Debian Bug Tracking System]

Processing control commands:

> tags -1 confirmed
Bug #894159 [release.debian.org] transition: icu
Added tag(s) confirmed.

-- 
894159: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=894159
Debian Bug Tracking System
Contact ow...@bugs.debian.org with problems

Bug#894159: transition: icu

[Emilio Pozuelo Monfort]

Control: tags -1 confirmed

On 01/05/18 19:29, László Böszörményi (GCS) wrote:
> On Mon, Apr 23, 2018 at 11:31 PM Emilio Pozuelo Monfort 
> wrote:
>> On 23/04/18 21:40, László Böszörményi (GCS) wrote:
>>>  I do agree and open to do an other compilation test of the dependent
>>> packages with icu-config being available.
> 
>> Thanks. Let me know how that goes, and I'll ack this transition as soon
> as possible.
> OK, the fresh transition testing is the following.

Alright, let's do this.

Cheers,
Emilio

Bug#894159: transition: icu

[Rene Engelhard]

On Wed, May 02, 2018 at 08:06:57PM +0200, Rene Engelhard wrote:
> Yes, that's why various LibreOffice/Document Liberation libraries
> (and LO also patched firebird) - parts of this reverted since
> c++11-using projects apparently don't need it) did add a
> -DUCHAR_TYPE=uint16_t to their defines. Explicitely backported in
> various debian/rules of those libs

Sorry, sent to early. From LOs configure.ac:

if test "$ICU_MAJOR" -ge "59"; then
# As of ICU 59 it defaults to typedef char16_t UChar; which is available
# with -std=c++11 but not all external libraries can be built with that,
# for those use a bit-compatible typedef uint16_t UChar; see
# icu/source/common/unicode/umachine.h
ICU_UCHAR_TYPE="-DUCHAR_TYPE=uint16_t"
else
ICU_UCHAR_TYPE=""
fi

Regards,
 
Rene

Bug#894159: transition: icu

[Rene Engelhard]

Hi,

On Tue, May 01, 2018 at 05:29:33PM +, László Böszörményi wrote:
>   5.10.1+dfsg-2) were uploaded to experimental. Now it FTBFS due to GCC
> being picky. Recent ICU (59.1+) named variable types to be consistent for
> different architectures. That is, UChar defaults to char16_t but can be
> uint16_t if the architecture / compiler requires that.
> Probably newer GCC versions no longer do the automatic type casts. I had to
> add them explicitly. No code change was needed. With my patch, this package
> builds correctly again.

Yes, that's why various LibreOffice/Document Liberation libraries
(and LO also patched firebird) - parts of this reverted since
c++11-using projects apparently don't need it) did add a
-DUCHAR_TYPE=uint16_t to their defines. Explicitely backported in
various debian/rules of those libs

Regards,

Rene

Bug#894159: transition: icu

[Bernd Zeimetz]

Hi,

> pyicu FTBFS, seems unmaintained. Last upload was in 2016 and since then
> there were seven upstream releases. Version 1.9.8+ builds fine with the new
> ICU version. Maintainer also Cc-d, may I overtake it?

unmaintained is the wrong word, I try to have it uptodate before the
next release - and I'm almost sure that I've uploaded the current
version before the last freeze. I'd be happy if you could maintain it -
I'm missing the time for it.


Thanks,

Bernd


-- 
 Bernd ZeimetzDebian GNU/Linux Developer
 http://bzed.dehttp://www.debian.org
 GPG Fingerprint: ECA1 E3F2 8E11 2432 D485  DD95 EB36 171A 6FF9 435F

Bug#894159: transition: icu

[GCS]

On Mon, Apr 23, 2018 at 11:31 PM Emilio Pozuelo Monfort 
wrote:
> On 23/04/18 21:40, László Böszörményi (GCS) wrote:
> >  I do agree and open to do an other compilation test of the dependent
> > packages with icu-config being available.

> Thanks. Let me know how that goes, and I'll ack this transition as soon
as possible.
OK, the fresh transition testing is the following.
Level 1:
I don't know how haskell-yi-rope directly related, but it builds with the
rest of the packages.

Level 1.5:
I choose this intermediate step to build both Boost1.62 and Boost1.63 as
well correctly.

Level 2:
dee FTBFS for an unrealated issue, #876594 [1].
libsimpleini fails only due to it needs library symbols update. Maintainer
is Cc-d.
pyicu FTBFS, seems unmaintained. Last upload was in 2016 and since then
there were seven upstream releases. Version 1.9.8+ builds fine with the new
ICU version. Maintainer also Cc-d, may I overtake it?

Level 3:
node-stringprep FTBFS for an unrelated issue, #895029 [2].
webkitgtk FTBFS, requested to be removed[3] due to unmaintained and many
security bugs. That's why I ignored it. Probably variable casting problems
is the FTBFS reason, see later.

Level 4:
cyrus-imapd FTBFS for an unrelated issue, #883951 [4].
haskell-blogliterately FTBFS for an unrelated issue, #897172 [5].

Level 5:
kbibtex FTBFS for an unrelated issue, #893538 [6].
qtwebengine-opensource-src is strange. Previously it built for me, even
built for the maintainers when previous package versions (5.10.1+dfsg-1 and
  5.10.1+dfsg-2) were uploaded to experimental. Now it FTBFS due to GCC
being picky. Recent ICU (59.1+) named variable types to be consistent for
different architectures. That is, UChar defaults to char16_t but can be
uint16_t if the architecture / compiler requires that.
Probably newer GCC versions no longer do the automatic type casts. I had to
add them explicitly. No code change was needed. With my patch, this package
builds correctly again.

Cheers,
Laszlo/GCS
[1] https://bugs.debian.org/876594
[2] https://bugs.debian.org/895029
[3] https://bugs.debian.org/893863
[4] https://bugs.debian.org/883951
[5] https://bugs.debian.org/897172
[6] https://bugs.debian.org/893538

Bug#894159: transition: icu

[GCS]

On Tue, Apr 24, 2018 at 6:46 PM, Rene Engelhard  wrote:
> On Mon, Apr 23, 2018 at 09:19:06PM +0200, László Böszörményi wrote:
>> > I would appreciate a number of failing rdeps and how many are due to ICU 
>> > API
>> > changes and how many are due to icu-config removal.
>> [...] I do not
>> recall any API change. In short, there are fifteen packages FTBFS and
>
> Interesting. I do.
 Let me rephrase my sentence. I do not remember any FTBFS reason which
happens due to an ICU API change. I could build all dependent packages
without any code change in those. Sorry for the bad wording.
I'm in a run, but double checked my patches and all about ICU
detection without icu-config installed - expect one. That is OpenTTD
which need an additional build dependency on libicu-le-hb-dev.
To be extra sure about the possible changes needed in the packages,
I'll start the rebuild tests soon.

Kind regards,
Laszlo/GCS

Bug#894159: transition: icu

[Rene Engelhard]

On Mon, Apr 23, 2018 at 09:19:06PM +0200, László Böszörményi wrote:
> > I would appreciate a number of failing rdeps and how many are due to ICU API
> > changes and how many are due to icu-config removal.
> [...] I do not
> recall any API change. In short, there are fifteen packages FTBFS and

Interesting. I do.

https://cgit.freedesktop.org/libreoffice/core/diff/i18npool/source/breakiterator/breakiterator_unicode.cxx?id=3e42714c76b1347babfdea0564009d8d82a83af4

Regards,

Rene

Bug#894159: transition: icu

[Emilio Pozuelo Monfort]

On 23/04/18 21:40, László Böszörményi (GCS) wrote:
> On Mon, Apr 23, 2018 at 9:27 PM, Emilio Pozuelo Monfort
>  wrote:
>> On 23/04/18 21:19, László Böszörményi (GCS) wrote:
>>>  As I remember, 99% of the FTBFS reasons were the icu-config removal,
>>> others are the dependent package bugs I've already mentioned. I do not
>>> recall any API change. In short, there are fifteen packages FTBFS and
>>> all is due to the icu-config removal. This is true for the date of my
>>> previous mail. There were several dependent packages upload since
>>> then, but I think the situation remained the same.
>>
>> In that case I'm ok with removing icu-config, but please don't entangle that
>> with the SONAME bump. That is, reintroduce icu-config so we can have an easy
>> transition, and once the transition is finished, then you are free to drop
>> icu-config. Sounds good?
>  I do agree and open to do an other compilation test of the dependent
> packages with icu-config being available.

Thanks. Let me know how that goes, and I'll ack this transition as soon as 
possible.

Cheers,
Emilio

Bug#894159: transition: icu

[GCS]

On Mon, Apr 23, 2018 at 9:27 PM, Emilio Pozuelo Monfort
 wrote:
> On 23/04/18 21:19, László Böszörményi (GCS) wrote:
>>  As I remember, 99% of the FTBFS reasons were the icu-config removal,
>> others are the dependent package bugs I've already mentioned. I do not
>> recall any API change. In short, there are fifteen packages FTBFS and
>> all is due to the icu-config removal. This is true for the date of my
>> previous mail. There were several dependent packages upload since
>> then, but I think the situation remained the same.
>
> In that case I'm ok with removing icu-config, but please don't entangle that
> with the SONAME bump. That is, reintroduce icu-config so we can have an easy
> transition, and once the transition is finished, then you are free to drop
> icu-config. Sounds good?
 I do agree and open to do an other compilation test of the dependent
packages with icu-config being available.

Cheers,
Laszlo/GCS

Bug#894159: transition: icu

[Emilio Pozuelo Monfort]

On 23/04/18 21:19, László Böszörményi (GCS) wrote:
> On Mon, Apr 23, 2018 at 7:57 PM, Emilio Pozuelo Monfort
>  wrote:
>> First of all, sorry for the delay. I saw there were several issues here and
>> decided to postpone this a bit.
>  Emilio! I already owe you a couple of beers. Hope we can meet again
> in Hsinchu and have a chat about this.
> 
>> On 26/03/18 22:29, László Böszörményi (GCS) wrote:
>>> It will need a quick bootstrap. It needs to build without the Layout
>>> Engine API, then the support library (icu-le-hb). Only then ICU can be
>>> built with the Layout Engine API as the two libraries tied together.
>>
>> Can you explain that? Why can't you enable Layout Engine from the start?
>  The Layout Engine was always buggy and was abandoned for a while. It
> was removed in ICU 58.1 [1]. Actually it was deprecated with ICU 54.1,
> released in October, 2014 (three and a half years ago).
> Someone started to re-implement the API using an external library,
> HarfBuzz[2]. But it is also stalled, the last real code commit is from
> 6th of May, 2016 [3]. Only one project still using it via the
> Paragraph Layout and it's the OpenTTD game[4].
> To answer your question, as you see Layout Engine is an external
> project by now. It needs to build with the actual ICU ABI, which
> changes with major releases (hence the need of a transition). When the
> Layout Engine is available for the current ICU ABI, then the
> additional Paragraph Layout API / libraries of ICU can be built. This
> is the cause of the ICU build -> icu-le-hb -> ICU build again steps.
> I might bundle the two sources together into one source package, but
> then every ICU build would do this three step compilation instead of
> one. It's enough to do once IMHO for every ICU major releases.
> In short, I should speak with the OpenTTD folks how / why they use
> Paragraph Layout API and if they can migrate to an other solution.

Ok, I didn't realise that icu-le-hb was a separate source package.

>>> Some patches are simply adding '--with-icu=/usr' to the configure
>>> invocation in debian/rules. Others are for ICU detection in the
>>> configuration phase. No code change is needed in the packages.
>>> If it matters, Ubuntu already transitioned with a bit different method.
>>
>> Are those changes and the large number of FTBFS because of the removal of
>> icu-config? Can we keep it for now and file bugs at severity:important for 
>> the
>> rdeps asking to fix the build when icu-config is removed? That should ease 
>> this
>> transition.
>  Agree, keeping icu-config would help a lot with the transition.
> Ubuntu did the transition this way. On the other hand, icu-config is a
> simple script and don't know much about MultiArch and/or
> cross-compilation. Upstream has pkg-config files for a while, but not
> all dependent packages migrated to it yet. :-/
> OK, riscv64 already over the initial bootstrap and rebuild steps.
> 
>> I would appreciate a number of failing rdeps and how many are due to ICU API
>> changes and how many are due to icu-config removal.
>  As I remember, 99% of the FTBFS reasons were the icu-config removal,
> others are the dependent package bugs I've already mentioned. I do not
> recall any API change. In short, there are fifteen packages FTBFS and
> all is due to the icu-config removal. This is true for the date of my
> previous mail. There were several dependent packages upload since
> then, but I think the situation remained the same.

In that case I'm ok with removing icu-config, but please don't entangle that
with the SONAME bump. That is, reintroduce icu-config so we can have an easy
transition, and once the transition is finished, then you are free to drop
icu-config. Sounds good?

Cheers,
Emilio

Bug#894159: transition: icu

[GCS]

On Mon, Apr 23, 2018 at 7:57 PM, Emilio Pozuelo Monfort
 wrote:
> First of all, sorry for the delay. I saw there were several issues here and
> decided to postpone this a bit.
 Emilio! I already owe you a couple of beers. Hope we can meet again
in Hsinchu and have a chat about this.

> On 26/03/18 22:29, László Böszörményi (GCS) wrote:
>> It will need a quick bootstrap. It needs to build without the Layout
>> Engine API, then the support library (icu-le-hb). Only then ICU can be
>> built with the Layout Engine API as the two libraries tied together.
>
> Can you explain that? Why can't you enable Layout Engine from the start?
 The Layout Engine was always buggy and was abandoned for a while. It
was removed in ICU 58.1 [1]. Actually it was deprecated with ICU 54.1,
released in October, 2014 (three and a half years ago).
Someone started to re-implement the API using an external library,
HarfBuzz[2]. But it is also stalled, the last real code commit is from
6th of May, 2016 [3]. Only one project still using it via the
Paragraph Layout and it's the OpenTTD game[4].
To answer your question, as you see Layout Engine is an external
project by now. It needs to build with the actual ICU ABI, which
changes with major releases (hence the need of a transition). When the
Layout Engine is available for the current ICU ABI, then the
additional Paragraph Layout API / libraries of ICU can be built. This
is the cause of the ICU build -> icu-le-hb -> ICU build again steps.
I might bundle the two sources together into one source package, but
then every ICU build would do this three step compilation instead of
one. It's enough to do once IMHO for every ICU major releases.
In short, I should speak with the OpenTTD folks how / why they use
Paragraph Layout API and if they can migrate to an other solution.

>> Some patches are simply adding '--with-icu=/usr' to the configure
>> invocation in debian/rules. Others are for ICU detection in the
>> configuration phase. No code change is needed in the packages.
>> If it matters, Ubuntu already transitioned with a bit different method.
>
> Are those changes and the large number of FTBFS because of the removal of
> icu-config? Can we keep it for now and file bugs at severity:important for the
> rdeps asking to fix the build when icu-config is removed? That should ease 
> this
> transition.
 Agree, keeping icu-config would help a lot with the transition.
Ubuntu did the transition this way. On the other hand, icu-config is a
simple script and don't know much about MultiArch and/or
cross-compilation. Upstream has pkg-config files for a while, but not
all dependent packages migrated to it yet. :-/
OK, riscv64 already over the initial bootstrap and rebuild steps.

> I would appreciate a number of failing rdeps and how many are due to ICU API
> changes and how many are due to icu-config removal.
 As I remember, 99% of the FTBFS reasons were the icu-config removal,
others are the dependent package bugs I've already mentioned. I do not
recall any API change. In short, there are fifteen packages FTBFS and
all is due to the icu-config removal. This is true for the date of my
previous mail. There were several dependent packages upload since
then, but I think the situation remained the same.

Cheers,
Laszlo/GCS

Bug#894159: transition: icu

[Emilio Pozuelo Monfort]

Hi László,

First of all, sorry for the delay. I saw there were several issues here and
decided to postpone this a bit. Please see some questions below.

On 26/03/18 22:29, László Böszörményi (GCS) wrote:
> It will need a quick bootstrap. It needs to build without the Layout
> Engine API, then the support library (icu-le-hb). Only then ICU can be
> built with the Layout Engine API as the two libraries tied together.

Can you explain that? Why can't you enable Layout Engine from the start?

> Some patches are simply adding '--with-icu=/usr' to the configure
> invocation in debian/rules. Others are for ICU detection in the
> configuration phase. No code change is needed in the packages.
> If it matters, Ubuntu already transitioned with a bit different method.

Are those changes and the large number of FTBFS because of the removal of
icu-config? Can we keep it for now and file bugs at severity:important for the
rdeps asking to fix the build when icu-config is removed? That should ease this
transition.

I would appreciate a number of failing rdeps and how many are due to ICU API
changes and how many are due to icu-config removal.

Cheers,
Emilio

Bug#894159: transition: icu

[GCS]

Package: release.debian.org
Severity: normal
User: release.debian@packages.debian.org
Usertags: transition

Hi RMs,

 While I've packaged new ICU releases, those were uploaded only to
experimental. It's really time to change this, especially that some
packages already need ICU 60.1+.
It will need a quick bootstrap. It needs to build without the Layout
Engine API, then the support library (icu-le-hb). Only then ICU can be
built with the Layout Engine API as the two libraries tied together.

Local rebuild of dependent packages[1] on amd64 revealed the good, the
bad and the ugly. Level 1 rebuilt OK and as level 1.5 I've rebuilt
boost1.62 and boost1.63 normally.

For level 2 I could rebuilt the following packages with a small patch:
389-adminutil
389-ds-base
an
dwdiff
ibus-qt
open-vm-tools
pyicu
yaz

Package aegisub FTBFS for a know issue, #873327 [2], fixed upstream
but the maintainer doesn't respond for six months now. :(
I've a patch for dee, but it FTBFS for a know problem, #876594 [3].
While I've a patch for grcompiler too and it builds, its self tests
are failing. It's a highly outdated package. :( The first Debian
revision of the current package version (v4.2) is uploaded on 2012,
26th of June. But its upstream is more or less still active[4] and has
a much newer version (v5.0.2).
The package libsimpleini compiles fine, but its symbols change with
the new ICU version.
Two packages, pyicu and yaz are highly outdated by the way, I may
overtake the former.

For level 3 two packages need a patch: 389-dsgw and libfolia. Others
build correctly.

Level 4 is a bit more complicated. I've a patch for the following packages:
ucto
php7.0
php7.1
php7.2

Two packages FTBFS for other reasons:
cyrus-imapd for #883951 [5] which is fixed upstream, but maintainer
doesn't respond for four months.
yi is an other example, the problem is reported as #868637 [6] without
any action for nine months. :( Upstream maybe solved the problem, at
least there are more upstream releases out there.

About level 5, only frog needs a patch. There's a failing package,
kbibtex and it's reason is filed as #893538 [7].

Some patches are simply adding '--with-icu=/usr' to the configure
invocation in debian/rules. Others are for ICU detection in the
configuration phase. No code change is needed in the packages.
If it matters, Ubuntu already transitioned with a bit different method.

I hope this transition can be authorized and/or please tell me if any
more information is needed.

Thanks in advance,
Laszlo/GCS
[1] https://release.debian.org/transitions/html/auto-icu.html
[2] https://bugs.debian.org/873327
[3] https://bugs.debian.org/876594
[4] https://github.com/silnrsi/grcompiler/commits/master
[5] https://bugs.debian.org/883951
[6] https://bugs.debian.org/868637
[7] https://bugs.debian.org/893538