Bug#752271: [freetype] Modify the control file for bootstrapping without the demos

[Johannes Schauer]

Hi Steve,

firstly, thanks a lot for your valuable feedback and the time you took to
investigate this.

Quoting Steve Langasek (2015-09-12 04:12:56)
> In bug #752270, Peter refers to the "Feedback arc set" section of
> http://bootstrap.debian.net/amd64/ .  Reviewing the information at
> , I find no references to freetype at
> present.

that means, that Debian can (in theory) be bootstrapped without freetype
dropping its build dependencies on the x11 libraries.

The interesting bit is the "in theory" part. Botch is creating the dependency
graph for Debian Sid and then tries making that graph acyclic by using some
manually collected information about which source packages can drop which build
dependencies in practice. See here for the lists it uses:

https://gitlab.mister-muffin.de/debian-bootstrap/botch/tree/master/droppable

But even with that information, it often cannot make the whole graph acyclic.
Right now there are about 10 source packages where botch will remove build
dependencies even though it cannot know whether these dependencies are actually
droppable so that it can present a feedback arc set.

But even if the files mentioned above contained enough heuristics to make the
whole graph acyclic, they are still missing the information about which binary
packages those source packages would *not* build or *additionally build*
without certain build dependencies present. So this heuristic is a very trivial
method and the only way to get a more reliable result is to use actual build
profiles.

The feedback arc set is *only* a heuristic. That's why, as pointed out in the
talk, I don't think it's a good idea to directly push it to package
maintainers.

In fact, the only information on the whole page that is not a heuristic are the
tables for "Self-Cycles". That's why they are at the top. This information
should show up in the tracker at some point.

> I do find freetype included in the full listing for SCC #1 under "Amount of
> Cycles through Edges", but this only shows that freetype has
> build-dependencies on docbook-to-man and libx11-dev (which I know), it
> doesn't show me how these dependencies are involved in a cycle.

It does not show you the full cycle, yes. It only shows you how many cycles
would at minimum be broken if that build dependency was dropped. The value is
"7" for both docbook-to-man and libx11-dev which is a very low value. You can
see that other edges further up break more than 50 cycles. This is part of the
reason why freetype was not chosen to be part of the feedback arc set.

To get to the list of cycles for freetype from the table "Amount of Cycles
through Edges" you can click on the source package name.

> The docbook-to-man cycle, I'm able to work out for myself manually: freetype
> build-depends docbook-to-man depends opensp build-depends poppler-utils
> build-depends freetype.  This seems to be a false cycle because
> docbook-to-man depends on sp | opensp, and sp does not build-depend on
> freetype.
> 
> The libx11-dev cycle, I do not see where the cycle is when I eyeball the
> dependencies.  So I was hoping to easily be able to find this information in
> the linked bootstrap reports.

From the table "Amount of Cycles through Edges" you can quickly go to the
cycles the source package you are interested in is involved in, by clicking on
the source package name. It is highlighted in blue on the page and I thought
that would indicate a hyperlink:

http://bootstrap.debian.net/source/freetype_2.5.2-4.html

Can you point me at the exact cycle that you think is false?

> Peter then links to .  This
> page doesn't provide much information at all - just a table of versions and
> architectures (is an x good or bad or does it just mean information is
> available?) and a link for version 2.5.2-4.

Thanks, fixed in git.

> If I follow that link, I see it takes me to
> .
> This is obviously incorrect.

Thanks, fixed in git.

> If I manually correct the URL to
> , I get a page that
> includes a list of "cycles", but only shows 10 of these by default (out of
> 236).  And the first 10 cycles it lists are all about opensp again.

Are you suggesting a different way of presenting the information? If yes, then
I'd like to hear about it.

> Finally, after asking for 100 rows at a time, I start to see some relevant
> information which starts to make sense:
> 
>   src:freetype ⇢ libx11-dev → src:libxdmcp ⇢ w3m → src:w3m ⇢ libimlib2-dev
>   src:freetype ⇢ libx11-dev → src:libx11 ⇢ w3m → src:gpm ⇢ texlive-base
>   src:freetype ⇢ libx11-dev → src:libx11 ⇢ w3m → src:w3m ⇢ libimlib2-dev
>   src:freetype ⇢ libx11-dev → src:libxcb ⇢ python-xcbgen → src:python2.7 ⇢ 
> blt-dev
>   src:freetype ⇢ libx11-dev → src:libx11 ⇢ w3m → src:w3m ⇢ libimlib2-dev
>   

Bug#752271: [freetype] Modify the control file for bootstrapping without the demos

[Johannes Schauer]

Hi again,

Quoting Johannes Schauer (2015-09-12 10:26:50)
>  2. missing metadata
> 
>  The way we find source packages to modify with build profiles changed
>  since this bug was filed more than a year ago, mainly due to Helmuts
>  (CC-ed) work on rebootstrap.  The heuristics we currently have point out
>  that the src:freetype->libx11-dev build dependency does affect something
>  but also not that much. I don't know which analysis Peter did prior to
>  filing this bug but right now rebootstrap is AFAIK the most reliable
>  source for build profile bugs.

looking at the rebootstrap source, it seems freetype is part of the source
packages that have to be crossed:

https://anonscm.debian.org/cgit/users/helmutg/rebootstrap.git/tree/bootstrap.sh#n3257

Furthermore, it seems there are no patches (build profiles or otherwise) are
needed to cross build it.

This suggests that Peter's patch to add a build profile is not needed.

Thanks!

cheers, josch


signature.asc
Description: signature

Bug#752271: [freetype] Modify the control file for bootstrapping without the demos

[Helmut Grohne]

On Sat, Sep 12, 2015 at 11:06:28AM +0200, Johannes Schauer wrote:
> looking at the rebootstrap source, it seems freetype is part of the source
> packages that have to be crossed:
> 
> https://anonscm.debian.org/cgit/users/helmutg/rebootstrap.git/tree/bootstrap.sh#n3257
> 
> Furthermore, it seems there are no patches (build profiles or otherwise) are
> needed to cross build it.
> 
> This suggests that Peter's patch to add a build profile is not needed.

We need to clarify what bootstrapping means here. As I learned from
Daniel Schepler, there is more than one way to do it (which I consider a
feature).

The approach taken by rebootstrap is to cross build a big lump of
packages to finally arrive at something that makes build-essential
installable (in theory). With this approach, it seems very likely that
freetype needs to be cross built. Even if there is no reason it needs to
be crossed, crossing can still be used to break native cycles and
freetype works well for crossing as Johannes pointed out.

Daniel Schepler's approach (as far as I understand) consists of building
a cross-LFS system, running it and then natively building Debian
packages from such a system. With this approach, all of what Peter
Pentchev reported may be relevant. I don't know.

So if one assumes that cross building can be used to break cycles, then
no profiles are needed for freetype at all. Whether that assumption is a
good one, is up to others to decide, but I certainly rely on it in
rebootstrap. So you want to discuss this bug with Daniel Schepler
instead of me. CCed.

Helmut

Bug#752271: [freetype] Modify the control file for bootstrapping without the demos

[Steve Langasek]

Control: tags -1 pending

Hi Peter, Johannes,

I have been preparing a new upload of freetype to Debian unstable, and was
about to include these patches.  I am very much a fan of bootstrappability
of the archive, and would like to help move this forward.

However, I have to say that it's been very difficult for me to independently
verify that these are correct changes that will facilitate cycle-breaking
and bootstrapping.

In bug #752270, Peter refers to the "Feedback arc set" section of
http://bootstrap.debian.net/amd64/ .  Reviewing the information at
, I find no references to freetype
at present.

I do find freetype included in the full listing for SCC #1 under "Amount of
Cycles through Edges", but this only shows that freetype has
build-dependencies on docbook-to-man and libx11-dev (which I know), it
doesn't show me how these dependencies are involved in a cycle.

The docbook-to-man cycle, I'm able to work out for myself manually: freetype
build-depends docbook-to-man depends opensp build-depends poppler-utils
build-depends freetype.  This seems to be a false cycle because
docbook-to-man depends on sp | opensp, and sp does not build-depend on
freetype.

The libx11-dev cycle, I do not see where the cycle is when I eyeball the
dependencies.  So I was hoping to easily be able to find this information in
the linked bootstrap reports.

Peter then links to . 
This page doesn't provide much information at all - just a table of versions
and architectures (is an x good or bad or does it just mean information is
available?) and a link for version 2.5.2-4.

If I follow that link, I see it takes me to
.
 
This is obviously incorrect.

If I manually correct the URL to
, I get a page
that includes a list of "cycles", but only shows 10 of these by default (out
of 236).  And the first 10 cycles it lists are all about opensp again.

Finally, after asking for 100 rows at a time, I start to see some relevant
information which starts to make sense:

  src:freetype ⇢ libx11-dev → src:libxdmcp ⇢ w3m → src:w3m ⇢ libimlib2-dev
  src:freetype ⇢ libx11-dev → src:libx11 ⇢ w3m → src:gpm ⇢ texlive-base
  src:freetype ⇢ libx11-dev → src:libx11 ⇢ w3m → src:w3m ⇢ libimlib2-dev
  src:freetype ⇢ libx11-dev → src:libxcb ⇢ python-xcbgen → src:python2.7 ⇢ 
blt-dev
  src:freetype ⇢ libx11-dev → src:libx11 ⇢ w3m → src:w3m ⇢ libimlib2-dev
  src:freetype ⇢ libx11-dev → src:libxcb ⇢ python-xcbgen → src:python2.7 ⇢ 
tk-dev
  src:freetype ⇢ libx11-dev → src:libxcb ⇢ python → src:python2.7 ⇢ tk-dev
  src:freetype ⇢ libx11-dev → src:libx11 ⇢ w3m → src:gpm ⇢ texlive-base
  src:freetype ⇢ libx11-dev → src:libxcb ⇢ python → src:python2.7 ⇢ xvfb

The report then starts to show some repetition, with cycles that have been
listed previously shown again but listed separately for different
architectures for no visible reason.

Some of these cycles make sense.  libimlib2-dev, I recognize as a
reverse-dependency of libfreetype6-dev.  But many of the others do not.  It
appears that even here, the information about the cycle has been *truncated*
to only show the first 6 packages in the cycle.  texlive-base, blt-dev,
tk-dev, xvfb do not depend on freetype.  So it's very difficult to interpret
the provided information to understand where the cycles are, to understand
why freetype is the right place to make the change.

In the end, I have satisfied myself (by manually testing with apt) that
enough of these package cycles are real that yes, freetype should have a
build profile, and I will therefore be applying these patches in my upload.
However, as someone who does care about bootstrappability, I wanted to give
you this feedback about the difficulty I had finding the pertinent
information.

As I commented from the audience during the bootstrapping talk at this
year's DebConf, I think success in making Debian bootstrappable will
heavily depend on being able to surface the required work in a way that is
parallelizable across all Debian maintainers.  I think this experience shows
just how far we are from that currently, if it takes me this much effort to
confirm the cycles when someone else has /already/ done the work of
identifying it and supplying a patch.

Please let me know if there is some other forum where you would like me to
raise these concerns.  I would very much like to see this cycle information
exposed in a way that I can dive into it.


BTW, along the way I made a few changes to the provided patch to
debian/rules.

- Your patch excludes the ft2demos from the unpack and patch targets based
  on the build profile.  I disagree with this; I realize that it makes the
  build faster and this might be useful during the critical path of a
  bootstrap, but I think the unpack+patch stages of the build should always
  

Bug#752271: [freetype] Modify the control file for bootstrapping without the demos

[Johannes Schauer]

Control: block -1 by 763766

Hi,

On Sun, 22 Jun 2014 02:30:15 +0300 Peter Pentchev r...@ringlet.net wrote:
 As promised, here's the second part of the patch; it adds build profile
 annotations to the control file and simplifies the changes to the rules
 file a bit.  There's a little catch: since the build profile-aware
 versions of the build tools (dpkg-dev, debhelper, sbuild) have not hit
 a stable release yet, making this change might have to wait until Jessie
 is released, since right now the Debian archive build infrastructure
 cannot handle these annotations.

the syntax for the Build-Profiles field was changed during the bootstrap
sprint in paris [1,2]. Attached patch is updated to reflect those changes.
It only changes debian/control.in so please regenerate debian/control.

As the patch requires debhelper to understand the new syntax, this bug is
blocked by that bug in debhelper.

cheers, josch

[1] https://wiki.debian.org/Sprints/2014/BootstrapSprint
[2] https://wiki.debian.org/BuildProfileSpec
diff -u freetype-2.5.2/debian/changelog freetype-2.5.2/debian/changelog
--- freetype-2.5.2/debian/changelog
+++ freetype-2.5.2/debian/changelog
@@ -1,3 +1,12 @@
+freetype (2.5.2-2.1) UNRELEASED; urgency=medium
+
+  * Non-maintainer upload.
+  * If building in the stage1 build profile, do not depend on the X
+libraries and do not build the demo programs.  This breaks a circular
+build dependency of src:freetype with src:libx11 and src:x11proto-core
+
+ -- Johannes Schauer j.scha...@email.de  Tue, 07 Oct 2014 11:06:45 +0200
+
 freetype (2.5.2-2) unstable; urgency=medium
 
   * Acknowledge security NMU; thanks to Michael Gilbert.
diff -u freetype-2.5.2/debian/control freetype-2.5.2/debian/control
--- freetype-2.5.2/debian/control
+++ freetype-2.5.2/debian/control
@@ -3,7 +3,7 @@
 Priority: optional
 Maintainer: Steve Langasek vor...@debian.org
 Uploaders: Anthony Fok f...@debian.org, Keith Packard kei...@keithp.com
-Build-Depends: bzip2, debhelper (= 9), docbook-to-man, gettext (= 0.10.36-2), libx11-dev, x11proto-core-dev, libz-dev, quilt, libpng-dev, autoconf, automake, libtool
+Build-Depends: bzip2, debhelper (= 9.FIXME), dpkg-dev (= 1.17.14), docbook-to-man, gettext (= 0.10.36-2), libx11-dev !stage1, x11proto-core-dev !stage1, libz-dev, quilt, libpng-dev, autoconf, automake, libtool
 Standards-Version: 3.9.6
 Homepage: http://www.freetype.org
 
@@ -63,6 +63,7 @@
 Package: freetype2-demos
 Architecture: any
 Section: utils
+Build-Profiles: !stage1
 Depends: ${shlibs:Depends}, ${misc:Depends}
 Replaces: freetype-tools
 Description: FreeType 2 demonstration programs
diff -u freetype-2.5.2/debian/rules freetype-2.5.2/debian/rules
--- freetype-2.5.2/debian/rules
+++ freetype-2.5.2/debian/rules
@@ -46,13 +46,24 @@
 
 libdoc = FTL.TXT CHANGES TODO ft2faq.html
 
+ifeq (,$(filter stage1,$(DEB_BUILD_PROFILES)))
+build_demos = 1
+else
+build_demos = 0
+endif
+
 %:
 	dh $@
 
+tarballs = $(freetype_u) $(ftdocs_u)
+ifeq ($(build_demos),1)
+tarballs += $(ft2demos_u)
+endif
+
 unpack:: unpack-stamp
 unpack-stamp::
 	# Unpack upstream tarballs
-	@for i in $(freetype_u) $(ftdocs_u) $(ft2demos_u); do \
+	@for i in $(tarballs); do \
 	if [ -f $$i.tar.bz2 ]; then \
 		echo Unpacking $$i.tar.bz2 ...; \
 		tar -x --bzip2 -f $$i.tar.bz2; \
@@ -72,8 +83,10 @@
 	cd $(freetype_u) \
 	 QUILT_PATCHES=../debian/patches-freetype quilt --quiltrc /dev/null push -a
 	cd $(freetype_u)  ./autogen.sh
+ifeq ($(build_demos),1)
 	cd $(ft2demos_u) \
 	 QUILT_PATCHES=../debian/patches-ft2demos quilt --quiltrc /dev/null push -a
+endif
 	touch patch-stamp
 
 override_dh_auto_configure: patch
@@ -81,8 +94,10 @@
 
 override_dh_auto_build:
 	dh_auto_build -D $(freetype_u)
+ifeq ($(build_demos),1)
 	dh_auto_build -D $(ft2demos_u) -- TOP_DIR=../$(freetype_u) \
 		OBJ_DIR=../$(freetype_u)/objs
+endif
 	docbook-to-man $(freetype_config_sgml)  $(freetype_config_man)
 
 override_dh_auto_clean:
@@ -92,9 +107,11 @@
 
 override_dh_auto_install:
 	dh_auto_install -D $(freetype_u) --destdir=$(CURDIR)/debian/tmp
+ifeq ($(build_demos),1)
 	$(freetype_u)/builds/unix/libtool --mode=install \
 	cp -av `find $(ft2demos_u)/bin -type f -perm -u=x -maxdepth 1` \
 		$(CURDIR)/debian/$(demospkg)/usr/bin/
+endif
 	sed -i -e'/dependency_libs/s/'.*'//' debian/tmp/usr/lib/$(DEB_HOST_MULTIARCH)/libfreetype.la
 
 override_dh_install:

Bug#752271: [freetype] Modify the control file for bootstrapping without the demos

[Johannes Schauer]

Quoting Johannes Schauer (2014-10-07 11:21:50)
 the syntax for the Build-Profiles field was changed during the bootstrap
 sprint in paris [1,2]. Attached patch is updated to reflect those changes.
 It only changes debian/control.in so please regenerate debian/control.

please ignore the list sentence of above paragraph :)

cheers, josch


--
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org

Bug#752271: [freetype] Modify the control file for bootstrapping without the demos

[Peter Pentchev]

Source: freetype
Version: 2.5.2-1
Severity: wishlist
Tags: patch

Hi,

As promised, here's the second part of the patch; it adds build profile
annotations to the control file and simplifies the changes to the rules
file a bit.  There's a little catch: since the build profile-aware
versions of the build tools (dpkg-dev, debhelper, sbuild) have not hit
a stable release yet, making this change might have to wait until Jessie
is released, since right now the Debian archive build infrastructure
cannot handle these annotations.

If anything should go wrong with the patch, it is also available at
https://gitorious.org/roam-debian-bootstrap/freetype-debian/commits/roam-stage1-control

Thanks again for your work on Debian!

G'luck,
Peter

-- System Information:
Debian Release: jessie/sid
  APT prefers testing
  APT policy: (990, 'testing')
Architecture: amd64 (x86_64)
Foreign Architectures: i386

Kernel: Linux 3.14-1-amd64 (SMP w/4 CPU cores)
Locale: LANG=bg_BG.UTF-8, LC_CTYPE=bg_BG.UTF-8 (charmap=UTF-8)
Shell: /bin/sh linked to /bin/dash


signature.asc
Description: Digital signature