Bug#853285: ghc: Patch to add support for cross-compilation of GHC

2017-01-31 Thread John Paul Adrian Glaubitz 
On 01/31/2017 08:17 PM, Helmut Grohne wrote:
> On Tue, Jan 31, 2017 at 09:20:46AM +0100, John Paul Adrian Glaubitz wrote:
>>   Setting "--target" allows enabling cross-compiling, adding
>>   --enable-unregisterised is important for architectures not
>>   having native code generation support. Disabling the documentation
>>   is required by GHC itself. Trying to build the documentation
>>   when cross-compiling causes an error message.
> 
> I'm not entirely sold on the idea that we cannot cross build haddock.
> What about using the haddock from the ghc:native package to build the
> documentation? But then maybe I understand too little of the
> requirements here.

Oh, I'm absolutely with you on this. The problem is just that GHC upstream
currently doesn't support building the documentation when cross-building
because of these lines in the main ghc.mk makefile:

ifneq "$(CrossCompiling) $(Stage1Only)" "NO NO"
$(error Can not build haddock docs when CrossCompiling or Stage1Only. \
  Set HADDOCK_DOCS=NO in your mk/build.mk file. \
  See Note [No stage2 packages when CrossCompiling or Stage1Only])
endif
endif

> In any case, disabling haddock should not be a thing that just happens
> for cross compilation. We want cross builds to be reproducible (when
> compared against native builds). So disabling haddock should be a
> concious choice (even if cross building without disabling haddock
> doesn't work). Also building ghc without haddock should produce a
> package that doesn't list haddock under Provides. This calls for a build
> profile. So I went ahead and split out this part of your patch into a
> patch that adds a pkg.ghc.nohaddock build profile.

I actually want the cross-built GHC package to match the natively built
package as closely as possible because I prefer it as an alternative to
building GHC on slower architectures. Building GHC on m68k can take
over 20 days since m68k also doesn't have a native code generator.

> I hope that this makes sense and that the resulting profile is useful
> for cross compiling ghc. Do you agree?

As long as the cross-built GHC package can be fully used on the target
system, I'm all for these changes. But, as I explained in my initial
report, the are still issues with ghc-pkg when trying to build haskell-text
that I haven't resolved yet.

Adrian

-- 
 .''`.  John Paul Adrian Glaubitz
: :' :  Debian Developer - glaub...@debian.org
`. `'   Freie Universitaet Berlin - glaub...@physik.fu-berlin.de
  `-GPG: 62FF 8A75 84E0 2956 9546  0006 7426 3B37 F5B5 F913

___
Pkg-haskell-maintainers mailing list
Pkg-haskell-maintainers@lists.alioth.debian.org
http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/pkg-haskell-maintainers

Processed: Re: Bug#853285: ghc: Patch to add support for cross-compilation of GHC

2017-01-31 Thread Debian Bug Tracking System 
Processing control commands:

> clone -1 -2
Bug #853285 [src:ghc] ghc: Patch to add support for cross-compilation of GHC
Bug 853285 cloned as bug 853773
853773 was blocked by: 853743
853773 was not blocking any bugs.
Ignoring request to alter blocking bugs of bug #853773 to the same blocks 
previously set
> retitle -2 support building ghc without building haddock
Bug #853773 [src:ghc] ghc: Patch to add support for cross-compilation of GHC
Changed Bug title to 'support building ghc without building haddock' from 'ghc: 
Patch to add support for cross-compilation of GHC'.
> block -1 by -2
Bug #853285 [src:ghc] ghc: Patch to add support for cross-compilation of GHC
853285 was blocked by: 853743
853285 was not blocking any bugs.
Added blocking bug(s) of 853285: 853773

-- 
853285: http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=853285
853773: http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=853773
Debian Bug Tracking System
Contact ow...@bugs.debian.org with problems

___
Pkg-haskell-maintainers mailing list
Pkg-haskell-maintainers@lists.alioth.debian.org
http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/pkg-haskell-maintainers

Bug#853285: ghc: Patch to add support for cross-compilation of GHC

2017-01-31 Thread Helmut Grohne 
Control: clone -1 -2
Control: retitle -2 support building ghc without building haddock
Control: block -1 by -2

On Tue, Jan 31, 2017 at 09:20:46AM +0100, John Paul Adrian Glaubitz wrote:
>   Setting "--target" allows enabling cross-compiling, adding
>   --enable-unregisterised is important for architectures not
>   having native code generation support. Disabling the documentation
>   is required by GHC itself. Trying to build the documentation
>   when cross-compiling causes an error message.

I'm not entirely sold on the idea that we cannot cross build haddock.
What about using the haddock from the ghc:native package to build the
documentation? But then maybe I understand too little of the
requirements here.

In any case, disabling haddock should not be a thing that just happens
for cross compilation. We want cross builds to be reproducible (when
compared against native builds). So disabling haddock should be a
concious choice (even if cross building without disabling haddock
doesn't work). Also building ghc without haddock should produce a
package that doesn't list haddock under Provides. This calls for a build
profile. So I went ahead and split out this part of your patch into a
patch that adds a pkg.ghc.nohaddock build profile.

It's not clear to me whether some Build-Depends should become droppable
under the profile.

I hope that this makes sense and that the resulting profile is useful
for cross compiling ghc. Do you agree?

Helmut
diff --minimal -Nru ghc-8.0.1/debian/changelog ghc-8.0.1/debian/changelog
--- ghc-8.0.1/debian/changelog  2016-12-17 03:44:45.0 +0100
+++ ghc-8.0.1/debian/changelog  2017-01-31 15:39:43.0 +0100
@@ -1,3 +1,10 @@
+ghc (8.0.1-17.1) UNRELEASED; urgency=medium
+
+  * Non-maintainer upload.
+  * Add pkg.ghc.nohaddock build profile for disabling haddock (Closes: #-1).
+
+ -- Helmut Grohne   Tue, 31 Jan 2017 15:39:43 +0100
+
 ghc (8.0.1-17) unstable; urgency=medium
 
   * Add upstream patch "Add relocation type R_X86_64_REX_GOTPCRELX".
diff --minimal -Nru ghc-8.0.1/debian/control ghc-8.0.1/debian/control
--- ghc-8.0.1/debian/control2016-10-28 07:42:05.0 +0200
+++ ghc-8.0.1/debian/control2017-01-31 15:39:43.0 +0100
@@ -49,8 +49,8 @@
   ${ghci},
   ${provided-devs},
   ${haskell:Provides},
-  haddock,
-  ghc-haddock,
+  haddock ,
+  ghc-haddock ,
   ${haddock:Provides}
 Replaces: ghc6 (<< 7), ghc-dynamic (<< 7.8), ghc-haddock (<< 7.10)
 Conflicts: ghc6 (<< 7), ghc-dynamic (<< 7.8), ${conflicting-devs}
@@ -94,6 +94,7 @@
 Breaks: libghc-terminfo-doc (<< 0.3.2.5-4), libghc-transformers-doc (<< 
0.3.0.0-6), libghc-xhtml-doc (<< 3000.2.1-5)
 Depends: ${haddock:Depends}, ${misc:Depends}, perl, libjs-mathjax
 Pre-Depends: dpkg (>= 1.16.1)
+Build-Profiles: 
 Description: Documentation for the Glasgow Haskell Compilation system
  The Glorious Glasgow Haskell Compilation system (GHC) is a compiler for
  Haskell.
diff --minimal -Nru ghc-8.0.1/debian/rules ghc-8.0.1/debian/rules
--- ghc-8.0.1/debian/rules  2016-11-14 17:07:27.0 +0100
+++ ghc-8.0.1/debian/rules  2017-01-31 15:39:43.0 +0100
@@ -35,7 +35,6 @@
 GHC=$(firstword $(shell bash -c "type -p ghc"))
 EXTRA_CONFIGURE_FLAGS=--with-ghc="$(GHC)"
 
-BUILD_HADDOCK_DOCS=YES
 DEB_HOOGLE_TXT_DIR = /usr/lib/ghc-doc/hoogle/
 
 ifeq (x32,$(DEB_HOST_ARCH))
@@ -48,9 +47,13 @@
 override_dh_auto_configure:
dh_autoreconf perl -- boot
echo "SRC_HC_OPTS += -lffi -optl-pthread" >> mk/build.mk
+ifeq ($(filter pkg.ghc.nohaddock,$(DEB_BUILD_PROFILES)),)
echo "HADDOCK_DOCS := YES" >> mk/build.mk
echo "EXTRA_HADDOCK_OPTS += 
--mathjax=file:///usr/share/javascript/mathjax/MathJax.js" >> mk/build.mk
echo "XSLTPROC_OPTS += --nonet" >> mk/build.mk
+else
+   echo "HADDOCK_DOCS := NO" >> mk/build.mk
+endif
 ifneq (,$(findstring $(DEB_HOST_ARCH), arm64 armel armhf))
echo "SRC_HC_OPTS += -optl-B/usr/bin/ld.gold" >> mk/build.mk
 endif
@@ -161,11 +164,11 @@
if test -e debian/tmp/usr/bin/ghci-$(ProjectVersion); then \
echo 'ghci=ghc-ghci' >> debian/ghc.substvars ; fi
 
+ifeq ($(filter pkg.ghc.nohaddock,$(DEB_BUILD_PROFILES)),)
# Add haddock substvars
echo 
"haddock:Depends=haddock-interface-$$(debian/tmp/usr/lib/ghc/bin/haddock 
--interface-version)" >> debian/ghc-doc.substvars
echo 
"haddock:Provides=haddock-interface-$$(debian/tmp/usr/lib/ghc/bin/haddock 
--interface-version)" >> debian/ghc.substvars
 
-ifeq (YES,$(BUILD_HADDOCK_DOCS))
mkdir -p debian/tmp/usr/lib/ghc-doc
cp debian/gen_contents_index debian/tmp/usr/lib/ghc-doc/
chmod +x debian/tmp/usr/lib/ghc-doc/gen_contents_index
@@ -191,8 +194,10 @@
echo ".so man1/ghc-pkg.1" > debian/ghc-pkg-$(ProjectVersion).1
 
echo debian/*.1 > debian/ghc.manpages
+ifeq ($(filter pkg.ghc.nohaddock,$(DEB_BUILD_PROFILES)),)
cp debian/haddock.man debian/haddock.1
echo debian/haddock.1 >>

Bug#853285: ghc: Patch to add support for cross-compilation of GHC

2017-01-31 Thread Gianfranco Costamagna 
Hello:



>* ghc does have a test suite, but the Debian package doesn't run it.


ghc-testsuite is the source package "running" it.

(please don't ask me the reasons for the split :) )

G.

___
Pkg-haskell-maintainers mailing list
Pkg-haskell-maintainers@lists.alioth.debian.org
http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/pkg-haskell-maintainers

Bug#853285: ghc: Patch to add support for cross-compilation of GHC

2017-01-31 Thread Helmut Grohne 
Control: clone -1 -2
Control: retitle -2 ghc: honour DEB_BUILD_OPTIONS=nocheck
Control: block -1 by -2

On Tue, Jan 31, 2017 at 09:20:46AM +0100, John Paul Adrian Glaubitz wrote:
> Since building GHC natively on slower machines like m68k or sh4 can be quite
> painful, I have started working on making the GHC package cross-buildable.

Thank you. I was always reluctant to try it due to the size of the task,
but having your proof of concept shows that it really is feasible.

I think splitting the discussion into a few smaller subtopics could make
the whole thing easier. Thus I'll reply to various aspects in multiple
replies. This one will only cover running tests.

>   Additionally, I'm also guarding out the section which tries
>   to run the compiler for cross-builds, e.g. ("# Do some very
>   simple tests that the compiler actually works"). But Helmut
>   Grohne already pointed out that the proper way would be to
>   guard this by checking for "nocheck" in DEB_BUILD_OPTIONS
>   as both sbuild and pbuilder set "nocheck" by default when
>   cross-compiling. However, in some cases, like cross-compiling
>   from amd64 to i386, running the tests is still desired.

I refactored this part of the patch into checking DEB_BUILD_OPTIONS for
the presence of nocheck. I think that this is a direct improvement to
the ghc package and tested it with two native amd64 builds (one with
tests and the other without, unfortunately ghc isn't reproducible). I
can commit it to the experimental branch of DHG_packages if some other
DHG member agrees.

Let me also observe two things I found weird:
 * ghc does have a test suite, but the Debian package doesn't run it.
 * debian/rules uses findstring a lot where it should be using filter.

Helmut
diff --minimal -Nru ghc-8.0.1/debian/changelog ghc-8.0.1/debian/changelog
--- ghc-8.0.1/debian/changelog  2016-12-17 03:44:45.0 +0100
+++ ghc-8.0.1/debian/changelog  2017-01-31 13:33:36.0 +0100
@@ -1,3 +1,10 @@
+ghc (8.0.1-17.1) UNRELEASED; urgency=medium
+
+  * Non-maintainer upload.
+  * Do not run tests under DEB_BUILD_OPTIONS=nocheck (Closes: #-1).
+
+ -- Helmut Grohne   Tue, 31 Jan 2017 13:33:36 +0100
+
 ghc (8.0.1-17) unstable; urgency=medium
 
   * Add upstream patch "Add relocation type R_X86_64_REX_GOTPCRELX".
diff --minimal -Nru ghc-8.0.1/debian/rules ghc-8.0.1/debian/rules
--- ghc-8.0.1/debian/rules  2016-11-14 17:07:27.0 +0100
+++ ghc-8.0.1/debian/rules  2017-01-31 13:32:58.0 +0100
@@ -108,6 +108,9 @@
 
 override_dh_auto_build:
dh_auto_build
+
+override_dh_auto_test:
+ifeq ($(filter nocheck,$(DEB_BUILD_OPTIONS)),)
# Do some very simple tests that the compiler actually works
rm -rf debian/testghc
mkdir debian/testghc
@@ -122,6 +125,7 @@
@printf "BEGIN GHC INFO OUTPUT\n"
@inplace/bin/ghc-stage2 --info
@printf "END GHC INFO OUTPUT\n"
+endif
 
 FILES = \( -type f -o -type l \)
 PROF_FILE = \( -name "*.p_*" -o -name "lib*_p.a" \)
___
Pkg-haskell-maintainers mailing list
Pkg-haskell-maintainers@lists.alioth.debian.org
http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/pkg-haskell-maintainers

Processed: Re: Bug#853285: ghc: Patch to add support for cross-compilation of GHC

2017-01-31 Thread Debian Bug Tracking System 
Processing control commands:

> clone -1 -2
Bug #853285 [src:ghc] ghc: Patch to add support for cross-compilation of GHC
Bug 853285 cloned as bug 853743
> retitle -2 ghc: honour DEB_BUILD_OPTIONS=nocheck
Bug #853743 [src:ghc] ghc: Patch to add support for cross-compilation of GHC
Changed Bug title to 'ghc: honour DEB_BUILD_OPTIONS=nocheck' from 'ghc: Patch 
to add support for cross-compilation of GHC'.
> block -1 by -2
Bug #853285 [src:ghc] ghc: Patch to add support for cross-compilation of GHC
853285 was not blocked by any bugs.
853285 was not blocking any bugs.
Added blocking bug(s) of 853285: 853743

-- 
853285: http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=853285
853743: http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=853743
Debian Bug Tracking System
Contact ow...@bugs.debian.org with problems

___
Pkg-haskell-maintainers mailing list
Pkg-haskell-maintainers@lists.alioth.debian.org
http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/pkg-haskell-maintainers

Bug#853285: ghc: Patch to add support for cross-compilation of GHC

2017-01-31 Thread John Paul Adrian Glaubitz 
Source: ghc
Version: 8.0.1-17
Severity: normal
Tags: patch
User: helm...@debian.org
Usertags: rebootstrap

Hi!

Since building GHC natively on slower machines like m68k or sh4 can be quite
painful, I have started working on making the GHC package cross-buildable.
Naturally, being able to cross-build the GHC package also makes it much easier
to bootstrap the compiler for new architecture once basic support has been
added like here [1].

Quick summary what my current patch does:

- use-stage1-binaries-for-install.patch:

  This patches the main ghc.mk to use the stage1 binaries of ghc
  and ghc-pkg to perform some installation jobs during 'make
  install'. This change was suggested by upstream [2]

- build-unlit-and-hp2ps-twice.patch:

  Build the unlit and hp2ps binaries twice. Without the patch,
  the binaries created are for the build architecture, e.g.
  stage1 and not for the host architecture (stage2). See [3].
  This change was suggested by upstream, I extended it for
  hp2ps. Unfortunately, it was never merged. See also this
  upstream ticket [4].

- debian/control:

  Replace "ghc (>= 7.8)" with "ghc:native (>= 7.8)" to tell debhelper
  to use the GHC compiler of the build system. This is what the
  gcc package does as well.

  Move haskell-devscripts-minimal and python-sphinx to
  Build-Depends-Indep. Please note, this is a hack. The
  proper fix would be to make both python-sphinx and
  haskell-devscripts-minimal (and probably haskell-devscripts, too)
  both arch:any. See this bug report [5].

- debian/rules:

  When DEB_BUILD_GNU_TYPE doesn't match DEB_HOST_GNU_TYPE, set:

   confflags += --target $(DEB_HOST_GNU_TYPE) --enable-unregisterised
   BUILD_HADDOCK_DOCS=NO

  Setting "--target" allows enabling cross-compiling, adding
  --enable-unregisterised is important for architectures not
  having native code generation support. Disabling the documentation
  is required by GHC itself. Trying to build the documentation
  when cross-compiling causes an error message.

  Throughout the rest of the debian/rules changes, I basically
  just made the necessary changes to cope with the fact that
  the documentation is not being built. This includes skipping
  the installation of all documentation and manpages.

  Additionally, I'm also guarding out the section which tries
  to run the compiler for cross-builds, e.g. ("# Do some very
  simple tests that the compiler actually works"). But Helmut
  Grohne already pointed out that the proper way would be to
  guard this by checking for "nocheck" in DEB_BUILD_OPTIONS
  as both sbuild and pbuilder set "nocheck" by default when
  cross-compiling. However, in some cases, like cross-compiling
  from amd64 to i386, running the tests is still desired.

With the patch applied, I can successfully cross-build GHC provided
that I have pre-installed "haskell-devscripts-minimal" in the
chroot by simply running "sbuild --host=$HOST --no-arch-all -d sid"
provided that the package cross-build-essential-$HOST exists which
is unfortunately not the case for all architectures in Debian [6].

Adding additional architectures to cross-build-essential was
very easy though. I just edited debian/cross-targets, uncommented
the architectures I needed, ran ./debian/rules debian/control,
rebuilt the package and pre-installed it in the build chroot. So,
we should probably file a bug report against src:build-essential
to add more architectures as needed. I don't see why most of
the architectures are currently missing.

Anyway, without the haskell-devscripts-minimal package pre-installed
in the chroot, the cross-build will fail with:

dh_haskell_provides
make[1]: dh_haskell_provides: Command not found
debian/rules:268: recipe for target 'override_dh_installdeb' failed
make[1]: *** [override_dh_installdeb] Error 127
make[1]: Leaving directory '/<>'
debian/rules:46: recipe for target 'binary-arch' failed
make: *** [binary-arch] Error 2
dpkg-buildpackage: error: fakeroot debian/rules binary-arch gave error exit 
status 2

Additionally, the generated GHC package does not have the proper
symlinks set:

glaubitz@ikarus:/srv/chroots/sid-m68k-sbuild$ dpkg-deb -c ghc_8.0.1-17_m68k.deb 
|grep "/usr/bin"
drwxr-xr-x root/root 0 2016-12-17 03:44 ./usr/bin/
-rwxr-xr-x root/root55 2016-12-17 03:44 ./usr/bin/ghci-8.0.1
-rwxr-xr-x root/root   226 2016-12-17 03:44 
./usr/bin/m68k-unknown-linux-gnu-ghc-8.0.1
-rwxr-xr-x root/root   258 2016-12-17 03:44 
./usr/bin/m68k-unknown-linux-gnu-ghc-pkg-8.0.1
-rwxr-xr-x root/root   188 2016-12-17 03:44 
./usr/bin/m68k-unknown-linux-gnu-hpc
-rwxr-xr-x root/root   967 2016-12-17 03:44 
./usr/bin/m68k-unknown-linux-gnu-hsc2hs
-rwxr-xr-x root/root   234 2016-12-17 03:44 
./usr/bin/m68k-unknown-linux-gnu-runghc-8.0.1
lrwxrwxrwx root/root 0 2016-12-17 03:44 ./usr/bin/ghci -> ghci-8.0.1
lrwxrwxrwx root/root 0 2016-12-17 03:44 
./usr/bin/m68k-unknown-linux-gnu-ghc -> m68k-unknown-linux-gnu-ghc-8.0.1
lrwxrwxrwx root/root