Re: [pkg-go] RFC: autopkgtest

2017-06-01 Thread Martín Ferrari
On 01/06/17 13:30, Antonio Terceiro wrote:

> I have just installed the whitelist, and the packages in it started to
> be tested.

Excellent, thanks a lot!


-- 
Martín Ferrari (Tincho)

___
Pkg-go-maintainers mailing list
Pkg-go-maintainers@lists.alioth.debian.org
http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/pkg-go-maintainers


Re: [pkg-go] RFC: autopkgtest

2017-05-29 Thread Martín Ferrari
Update: autodep8 and dh-golang have been uploaded, and so we are ready
to start using autopkgtest.

Antonio will whitelist most of our packages in the CI infrastructure to
help bootstrapping, but from now on, we should start adding this field
to debian/control to explicitly enable the automatic tests:

Testsuite: autopkgtest-pkg-go

If you want a package to run other tests instead of the default tests,
you can place a debian/tests/control file. If you want to run extra
tests in addition to the default ones, you can add a
debian/tests/control.autodep8 file, or a autopkgtest target in
debian/rules (this is supported by the script in dh-golang).



On 26/05/17 04:35, Martín Ferrari wrote:
> An status update on autopkgtest for pkg-go.
> 
> I have been working on this since SunCamp, and I think it is ready for
> deployment.
> 
> I have created a much more polished script, and committed it to
> dh-golang. If nobody is against it, I will be uploading it during the
> weekend.
> 
> I have also added golang support to autodep8 (which uses the script in
> dh-golang) and pushed the changes to git master. (Antonio, please let me
> know if you think this change is OK).
> 
> Using these changes, I have already tested all packages that
> build-depend on golang, and I am satisfied that my scripts work for most
> of the cases out of the box, and that the majority of the team's
> packages pass the tests correctly.
> 
> If Antonio agrees, I would like to set up a whitelist for ci.debian.net
> to start testing these soon, so we don't need wait for the next upload
> to enable autopkgtest in 700+ packages.
> 
> 
> The stats from my tests are:
> 
>   694 Tests passing OK
>33 Tests failing, but maintainer decided to ignore these (FIXME!)
>16 Failures that seem like real problems.
>22 Failures because packaging incompatibilites with the script.
>17 Packages that already have autopkgtest support, or not relevant.
> 
> The names for each package are included in the attached lists.
> 
> 
> On 23/05/17 11:26, Martín Ferrari wrote:
>> Hey all!
>>
>> As I mentioned on IRC, I have been working on getting autopkgtest
>> coverage for all of our packages.
>>
>> I started by writing an script[1] that re-runs the same tests we use
>> during package build automatically. If all packages gets this testing in
>> ci.debian.net, we can automatically detect any regressions due to
>> changes in dependencies, which is IMHO one of our biggest problems nowadays.
>>
>> Later we can add more specialised tests, or re-enable tests that can't
>> run at buildtime because they require networking or root access.
>>
>>
>> To make this available to all packages, the first thing to do is to add
>> golang support to autodep8, which I am preparing.
>>
>> The second thing I need to do is to add the test running scripts to some
>> package we control. It could be an ad-hoc package like
>> pkg-perl-autopkgtest, or it could be integrated into something we
>> already have, like dh-golang.
>>
>> I am currently leaning towards adding this same script to dh-golang,
>> maybe adding detection for a pkgautotest target in debian/rules (which
>> would be run automatically too), and getting a whitelist in
>> ci.debian.net to start getting test reports for all our packages.
>>
>>
>> Ideas, comments?
>>
>>
>> [1]:
>> https://anonscm.debian.org/cgit/pkg-go/packages/prometheus-postgres-exporter.git/tree/debian/tests/build-test
>>
> 
> 


-- 
Martín Ferrari (Tincho)

___
Pkg-go-maintainers mailing list
Pkg-go-maintainers@lists.alioth.debian.org
http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/pkg-go-maintainers


Re: [pkg-go] RFC: autopkgtest

2017-05-26 Thread Martín Ferrari
Hi Antonio,

On 26/05/17 11:43, Antonio Terceiro wrote:
> On Fri, May 26, 2017 at 04:35:19AM +0100, Martín Ferrari wrote:
> [...]
>> I have also added golang support to autodep8 (which uses the script in
>> dh-golang) and pushed the changes to git master. (Antonio, please let me
>> know if you think this change is OK).
> 
> LGTM. I'm uploading autodep8 now.

Great!

>> Using these changes, I have already tested all packages that
>> build-depend on golang, and I am satisfied that my scripts work for most
>> of the cases out of the box, and that the majority of the team's
>> packages pass the tests correctly.
>>
>> If Antonio agrees, I would like to set up a whitelist for ci.debian.net
>> to start testing these soon, so we don't need wait for the next upload
>> to enable autopkgtest in 700+ packages.
> 
> Sure. Which list you want to use? PASS.txt?

Yes, please.

Thank you loads!

-- 
Martín Ferrari (Tincho)

___
Pkg-go-maintainers mailing list
Pkg-go-maintainers@lists.alioth.debian.org
http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/pkg-go-maintainers


Re: [pkg-go] RFC: autopkgtest

2017-05-25 Thread Martín Ferrari
An status update on autopkgtest for pkg-go.

I have been working on this since SunCamp, and I think it is ready for
deployment.

I have created a much more polished script, and committed it to
dh-golang. If nobody is against it, I will be uploading it during the
weekend.

I have also added golang support to autodep8 (which uses the script in
dh-golang) and pushed the changes to git master. (Antonio, please let me
know if you think this change is OK).

Using these changes, I have already tested all packages that
build-depend on golang, and I am satisfied that my scripts work for most
of the cases out of the box, and that the majority of the team's
packages pass the tests correctly.

If Antonio agrees, I would like to set up a whitelist for ci.debian.net
to start testing these soon, so we don't need wait for the next upload
to enable autopkgtest in 700+ packages.


The stats from my tests are:

  694 Tests passing OK
   33 Tests failing, but maintainer decided to ignore these (FIXME!)
   16 Failures that seem like real problems.
   22 Failures because packaging incompatibilites with the script.
   17 Packages that already have autopkgtest support, or not relevant.

The names for each package are included in the attached lists.


On 23/05/17 11:26, Martín Ferrari wrote:
> Hey all!
> 
> As I mentioned on IRC, I have been working on getting autopkgtest
> coverage for all of our packages.
> 
> I started by writing an script[1] that re-runs the same tests we use
> during package build automatically. If all packages gets this testing in
> ci.debian.net, we can automatically detect any regressions due to
> changes in dependencies, which is IMHO one of our biggest problems nowadays.
> 
> Later we can add more specialised tests, or re-enable tests that can't
> run at buildtime because they require networking or root access.
> 
> 
> To make this available to all packages, the first thing to do is to add
> golang support to autodep8, which I am preparing.
> 
> The second thing I need to do is to add the test running scripts to some
> package we control. It could be an ad-hoc package like
> pkg-perl-autopkgtest, or it could be integrated into something we
> already have, like dh-golang.
> 
> I am currently leaning towards adding this same script to dh-golang,
> maybe adding detection for a pkgautotest target in debian/rules (which
> would be run automatically too), and getting a whitelist in
> ci.debian.net to start getting test reports for all our packages.
> 
> 
> Ideas, comments?
> 
> 
> [1]:
> https://anonscm.debian.org/cgit/pkg-go/packages/prometheus-postgres-exporter.git/tree/debian/tests/build-test
> 


-- 
Martín Ferrari (Tincho)
consul
consul-migrate
gobgp
goiardi
golang-github-armon-gomdb
golang-github-azure-go-autorest
golang-github-hashicorp-hcl
golang-github-hashicorp-raft
golang-github-hashicorp-raft-mdb
golang-github-klauspost-compress
golang-github-lib-pq
golang-github-msteinert-pam
golang-github-nicksnyder-go-i18n
golang-github-rackspace-gophercloud
google-cloud-print-connector
nomad
docker-libkv
etcd-fs
golang-github-aanand-compose-file
golang-github-bugsnag-bugsnag-go
golang-github-cloudflare-cfssl
golang-github-cloudflare-go-metrics
golang-github-docker-go-connections
golang-github-docker-libnetwork
golang-github-elazarl-goproxy
golang-github-franela-goblin
golang-github-go-macaron-gzip
golang-github-go-macaron-session
golang-github-go-openapi-runtime
golang-github-go-openapi-validate
golang-github-hashicorp-go-getter
golang-github-hashicorp-memberlist
golang-github-hashicorp-serf
golang-github-hawkular-hawkular-client-go
golang-github-hpcloud-tail
golang-github-jeromer-syslogparser
golang-github-mitchellh-cli
golang-github-mitchellh-hashstructure
golang-github-racksec-srslog
golang-github-samalba-dockerclient
golang-github-smartystreets-assertions
golang-github-smartystreets-goconvey
golang-github-spacejam-loghisto
golang-github-unknwon-com
golang-github-vmware-photon-controller-go-sdk
golang-gopkg-dancannon-gorethink.v2
golang-gopkg-olivere-elastic.v2
runc
swarmkit
containerd
direnv
docker-swarm
gitlab-ci-multi-runner
golang-git2go
golang-github-docker-spdystream
golang-github-fsouza-go-dockerclient
golang-github-googleapis-proto-client-go
golang-github-gosexy-gettext
golang-github-jroimartin-gocui
golang-github-nsf-termbox-go
golang-github-revel-revel
golang-github-rfjakob-eme
golang-go.net-dev
golang-metrics
golang-vhost
golang-websocket
golang-xi2-x-xz
kubernetes-addon-heapster
notary
prometheus-alertmanager
restic
abci
acbuild
acmetool
cadvisor
certspotter
chasquid
codesearch
coyim
dh-make-golang
dnss
docker-registry
etcd
flannel
fuji
gitlab-workhorse
gocode
gocryptfs
golang-airbrake-go
golang-barcode
golang-bazil-fuse
golang-bindata
golang-bitbucket-pkg-inflect
golang-blackfriday
golang-bugsnag-panicwrap
golang-check.v1
golang-codegangsta-cli
golang-code.gitea-git
golang-code.gitea-sdk
golang-collectd
golang-context
golang-coreos-log
golang-dbus
golang-dns
golang-eclipse-paho
golang-ed25519-de

[pkg-go] RFC: autopkgtest

2017-05-23 Thread Martín Ferrari
Hey all!

As I mentioned on IRC, I have been working on getting autopkgtest
coverage for all of our packages.

I started by writing an script[1] that re-runs the same tests we use
during package build automatically. If all packages gets this testing in
ci.debian.net, we can automatically detect any regressions due to
changes in dependencies, which is IMHO one of our biggest problems nowadays.

Later we can add more specialised tests, or re-enable tests that can't
run at buildtime because they require networking or root access.


To make this available to all packages, the first thing to do is to add
golang support to autodep8, which I am preparing.

The second thing I need to do is to add the test running scripts to some
package we control. It could be an ad-hoc package like
pkg-perl-autopkgtest, or it could be integrated into something we
already have, like dh-golang.

I am currently leaning towards adding this same script to dh-golang,
maybe adding detection for a pkgautotest target in debian/rules (which
would be run automatically too), and getting a whitelist in
ci.debian.net to start getting test reports for all our packages.


Ideas, comments?


[1]:
https://anonscm.debian.org/cgit/pkg-go/packages/prometheus-postgres-exporter.git/tree/debian/tests/build-test
-- 
Martín Ferrari (Tincho)

___
Pkg-go-maintainers mailing list
Pkg-go-maintainers@lists.alioth.debian.org
http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/pkg-go-maintainers