[pkg-go] Bug#842380: gitlab-ci-multi-runner: mk-prebuilt-images.sh fails at install time

2016-10-30 Thread Vincent Danjean
Package: gitlab-ci-multi-runner
Version: 1.7.1+dfsg-1
Followup-For: Bug #842380

  Hi,

  As I discovered while submitting #842579 (against cdeboostrap), a
simple workaround for this bug is to remove the ' --exclude="..."'
line in /usr/lib/gitlab-runner/mk-prebuilt-images.sh when
invoking cdebootstrap.
  That said, removing this line will allow to create a working
gitlab-runner-prebuilt.tar.xz file, but I still think that errors
during its construction must be better handled.

  Regards,
Vincent

-- System Information:
Debian Release: stretch/sid
  APT prefers stable-updates
  APT policy: (500, 'stable-updates'), (500, 'unstable'), (500, 'testing'), 
(200, 'stable'), (1, 'experimental')
Architecture: amd64 (x86_64)
Foreign Architectures: i386, armel, mipsel

Kernel: Linux 4.7.0-1-amd64 (SMP w/8 CPU cores)
Locale: LANG=fr_FR.utf8, LC_CTYPE=fr_FR.utf8 (charmap=UTF-8)
Shell: /bin/sh linked to /bin/dash
Init: systemd (via /run/systemd/system)

Versions of packages gitlab-ci-multi-runner depends on:
ii  adduser  3.115
ii  ca-certificates  20160104
ii  git  1:2.10.1-1
ii  init-system-helpers  1.45
ii  libc62.24-5
ii  lsb-base 9.20161016

Versions of packages gitlab-ci-multi-runner recommends:
ii  cdebootstrap  0.7.5

Versions of packages gitlab-ci-multi-runner suggests:
ii  docker.io  1.11.2~ds1-6

-- Configuration Files:
/etc/gitlab-runner/config.toml [Errno 13] Permission non accordée: 
u'/etc/gitlab-runner/config.toml'

-- no debconf information

___
Pkg-go-maintainers mailing list
Pkg-go-maintainers@lists.alioth.debian.org
http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/pkg-go-maintainers

[pkg-go] Bug#842380: gitlab-ci-multi-runner: mk-prebuilt-images.sh fails at install time

2016-10-30 Thread Vincent Danjean
  Hi,

Le 30/10/2016 à 10:22, Dmitry Smirnov a écrit :
> Hi Vincent,
> 
> Sorry if my response reads a bit grumpy but I do not recognize your report as 
> bug.

So, I will try to argue a little bit more ;-)

> On Friday, 28 October 2016 6:00:49 PM AEDT Vincent Danjean wrote:
>> There is two problems:
>> 1) the cdebootstrap command is failing. I do not know why
> 
> It may be a problem in cdebootstrap, or in your APT proxy or in local mirror.
> We can't guarantee successful execution of cdebootstrap as gitlab-runner is 
> merely a user of cdebootstrap and the latter may fail due to number of 
> reasons.

ok

>>   but the current result (can be easily reproduced) is :
> 
> Not so easy apparently. I've tested gitlab-runner on x86_32 and x86_64 before 
> upload. Works for me.

Do you try the cdebootstrap command itself?
Here is, using a plain up-to-date sid chroot where I install
gitlab-ci-multi-runner to get all its dependencies:

vdanjean@atsina:~$ schroot -b -n test -c sid-amd64
vdanjean@atsina:~$ schroot -u root  -r -c test
atsina:/home/vdanjean# apt update
[...]
atsina:/home/vdanjean# apt full-upgrade
Reading package lists... Done
Building dependency tree
Reading state information... Done
Calculating upgrade... Done
0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
atsina:/home/vdanjean# apt install gitlab-ci-multi-runner
[...]
0 upgraded, 22 newly installed, 0 to remove and 0 not upgraded.
[...]
Setting up gitlab-ci-multi-runner (1.7.1+dfsg-1) ...
W: Docker is not available.
W: please run 'sudo /usr/lib/gitlab-runner/mk-prebuilt-images.sh' to generate 
Docker image.
invoke-rc.d: could not determine current runlevel
Setting up cdebootstrap (0.7.5) ...
[...]
atsina:/home/vdanjean# mkdir /tmp/test
atsina:/home/vdanjean# cd /tmp/test
atsina:/tmp/test# cdebootstrap \
--flavour=minimal \
--exclude="dmsetup,e2fsprogs,init,systemd-sysv,systemd,udev" \
--include="bash,ca-certificates,git,netcat-traditional" \
stable ./debian-minbase |& tee /tmp/log
P: Retrieving Release
P: Retrieving Release.gpg
[...]
P: Configuring helper cdebootstrap-helper-apt
E: Internal error: install
atsina:/tmp/test#

Do the cdebootstrap command really works on your systems?

=> that said, I think there is a bug in cdebootstrap here.
Nevermind, coming back to gitlab-ci-multi-runner.

>>   The mk-prebuilt-images.sh script continues but produce a image that does
>> not have the required tools (git in particular)
>>
>>   => docker is unusable.
> 
> Are you sure this is what happens? More likely image fails to build and 
> that's why docker executor can't work. You may still have older version of 
> gitlab-runner image which new version will not use.

I just installed gitlab-ci-multi-runner as my institution just provided me
a working gitlab server. So, I did not have older/previous version of
the /var/lib/gitlab-runner/gitlab-runner-prebuilt.tar.xz.
  I also tried to remove this file. mk-prebuilt-images.sh recreates
a (not working) one.


>> 2) mk-prebuilt-images.sh does not report this problem and the postinst
>>   does not fail (hence the 'important' severity)
> 
> This is intentional! And for a good reason. Not everyone use Docker executor, 
> you know. Other executors are also useful and they do not require docker.
> 
> Not so long ago mk-prebuilt-images.sh used to print log to the console 
> instead of saving it and failure to configure package is not as useful as you 
> may think...

  In this case, please remove the bad gitlab-runner-prebuilt.tar.xz
Or ask the user to re-create it with a --force parameter to
mk-prebuilt-images.sh if he wants to keep a probably problematic
image.

>>   It tooks me lot of time to discover that

In fact, the only info I get back was from the gitlab server.
If I look at the details of the failed run, I see:
==
Running with gitlab-ci-multi-runner dev (1.7.1)
Using Docker executor with image vdanjean/jessie-git:v1 ...
Pulling docker image vdanjean/jessie-git:v1 ...
Running on runner-61fe6005-project-69-concurrent-0 via eyak...
/bin/bash: line 60: git: command not found
/bin/bash: line 41: cd: /builds/danjean/ocl-icd: No such file or directory
ERROR: Build failed: exit code 1
==

You can see the reported error message from gitlab is minimal.
It took me a long time (and a certain level of expertise) to understand
that the problem does not come from my 'debian' docker image, but
from a special docker image badly created at install time.

If the gitlab-runner-prebuilt.tar.xz file is not present, gitlab gives
this feedback:
==
Running with gitlab-ci-multi-runner dev (1.7.1)
Using Docker executor with image vdanjean/jessie-git:v1 ...
ERROR: Preparation failed: open 
/var/lib/gitlab-runner/gitlab-runner-prebuilt.tar.xz: no such file or directory
Will be retried in 3s ...
Using Docker executor with image vdanjean/

[pkg-go] Bug#842380: gitlab-ci-multi-runner: mk-prebuilt-images.sh fails at install time

2016-10-28 Thread Vincent Danjean
Package: gitlab-ci-multi-runner
Version: 1.7.1+dfsg-1
Severity: important

  Hi,

  During the installation of the Debian package, mk-prebuilt-images.sh.
There is two problems:
1) the cdebootstrap command is failing. I do not know why
  but the current result (can be easily reproduced) is :
  # cdebootstrap --flavour=minimal 
--exclude="dmsetup,e2fsprogs,init,systemd-sysv,systemd,udev" 
--include="bash,ca-certificates,git,netcat-traditional" stable 
./debian-minbase
  [...]
  P: Configuring package apt
  P: Configuring helper cdebootstrap-helper-apt
  E: Internal error: install
  #

  The mk-prebuilt-images.sh script continues but produce a image that does not
  have the required tools (git in particular)

  => docker is unusable.

2) mk-prebuilt-images.sh does not report this problem and the postinst
  does not fail (hence the 'important' severity)
  It tooks me lot of time to discover that
  1) a special docker image is used to clone the repo
  2) this special image is corrupted (the 'E: Internal error: install' can
be seen in the log file whose name is displayed during the package
configuration). But, as the package installs fine, I do not suspect
a problem here until I really llok in depth.

Note that this is a Debian specific bug, as I eventually found:
https://gitlab.com/gitlab-org/gitlab-ci-multi-runner/issues/1669

  Perhaps, this bug can be duplicated and the first part reassigned
to cdeboostrap (the second part is really about this package)
if the cdeboostrap invocation is correct (I do not know cdeboostrap).


  Regards,
Vincent

-- System Information:
Debian Release: stretch/sid
  APT prefers stable-updates
  APT policy: (500, 'stable-updates'), (500, 'unstable'), (500, 'testing'), 
(200, 'stable'), (1, 'experimental')
Architecture: amd64 (x86_64)
Foreign Architectures: i386, armel, mipsel

Kernel: Linux 4.7.0-1-amd64 (SMP w/8 CPU cores)
Locale: LANG=fr_FR.utf8, LC_CTYPE=fr_FR.utf8 (charmap=UTF-8)
Shell: /bin/sh linked to /bin/dash
Init: systemd (via /run/systemd/system)

Versions of packages gitlab-ci-multi-runner depends on:
ii  adduser  3.115
ii  ca-certificates  20160104
ii  git  1:2.10.1-1
ii  init-system-helpers  1.45
ii  libc62.24-5
ii  lsb-base 9.20161016

Versions of packages gitlab-ci-multi-runner recommends:
ii  cdebootstrap  0.7.5

Versions of packages gitlab-ci-multi-runner suggests:
ii  docker.io  1.11.2~ds1-6

-- Configuration Files:
/etc/gitlab-runner/config.toml [Errno 13] Permission non accordée: 
u'/etc/gitlab-runner/config.toml'

-- no debconf information

___
Pkg-go-maintainers mailing list
Pkg-go-maintainers@lists.alioth.debian.org
http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/pkg-go-maintainers