Re: [pkg-go] RFC: autopkgtest
On 01/06/17 13:30, Antonio Terceiro wrote: > I have just installed the whitelist, and the packages in it started to > be tested. Excellent, thanks a lot! -- Martín Ferrari (Tincho) ___ Pkg-go-maintainers mailing list Pkg-go-maintainers@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/pkg-go-maintainers
Re: [pkg-go] RFC: autopkgtest
Update: autodep8 and dh-golang have been uploaded, and so we are ready to start using autopkgtest. Antonio will whitelist most of our packages in the CI infrastructure to help bootstrapping, but from now on, we should start adding this field to debian/control to explicitly enable the automatic tests: Testsuite: autopkgtest-pkg-go If you want a package to run other tests instead of the default tests, you can place a debian/tests/control file. If you want to run extra tests in addition to the default ones, you can add a debian/tests/control.autodep8 file, or a autopkgtest target in debian/rules (this is supported by the script in dh-golang). On 26/05/17 04:35, Martín Ferrari wrote: > An status update on autopkgtest for pkg-go. > > I have been working on this since SunCamp, and I think it is ready for > deployment. > > I have created a much more polished script, and committed it to > dh-golang. If nobody is against it, I will be uploading it during the > weekend. > > I have also added golang support to autodep8 (which uses the script in > dh-golang) and pushed the changes to git master. (Antonio, please let me > know if you think this change is OK). > > Using these changes, I have already tested all packages that > build-depend on golang, and I am satisfied that my scripts work for most > of the cases out of the box, and that the majority of the team's > packages pass the tests correctly. > > If Antonio agrees, I would like to set up a whitelist for ci.debian.net > to start testing these soon, so we don't need wait for the next upload > to enable autopkgtest in 700+ packages. > > > The stats from my tests are: > > 694 Tests passing OK >33 Tests failing, but maintainer decided to ignore these (FIXME!) >16 Failures that seem like real problems. >22 Failures because packaging incompatibilites with the script. >17 Packages that already have autopkgtest support, or not relevant. > > The names for each package are included in the attached lists. > > > On 23/05/17 11:26, Martín Ferrari wrote: >> Hey all! >> >> As I mentioned on IRC, I have been working on getting autopkgtest >> coverage for all of our packages. >> >> I started by writing an script[1] that re-runs the same tests we use >> during package build automatically. If all packages gets this testing in >> ci.debian.net, we can automatically detect any regressions due to >> changes in dependencies, which is IMHO one of our biggest problems nowadays. >> >> Later we can add more specialised tests, or re-enable tests that can't >> run at buildtime because they require networking or root access. >> >> >> To make this available to all packages, the first thing to do is to add >> golang support to autodep8, which I am preparing. >> >> The second thing I need to do is to add the test running scripts to some >> package we control. It could be an ad-hoc package like >> pkg-perl-autopkgtest, or it could be integrated into something we >> already have, like dh-golang. >> >> I am currently leaning towards adding this same script to dh-golang, >> maybe adding detection for a pkgautotest target in debian/rules (which >> would be run automatically too), and getting a whitelist in >> ci.debian.net to start getting test reports for all our packages. >> >> >> Ideas, comments? >> >> >> [1]: >> https://anonscm.debian.org/cgit/pkg-go/packages/prometheus-postgres-exporter.git/tree/debian/tests/build-test >> > > -- Martín Ferrari (Tincho) ___ Pkg-go-maintainers mailing list Pkg-go-maintainers@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/pkg-go-maintainers
Re: [pkg-go] RFC: autopkgtest
Hi Antonio, On 26/05/17 11:43, Antonio Terceiro wrote: > On Fri, May 26, 2017 at 04:35:19AM +0100, Martín Ferrari wrote: > [...] >> I have also added golang support to autodep8 (which uses the script in >> dh-golang) and pushed the changes to git master. (Antonio, please let me >> know if you think this change is OK). > > LGTM. I'm uploading autodep8 now. Great! >> Using these changes, I have already tested all packages that >> build-depend on golang, and I am satisfied that my scripts work for most >> of the cases out of the box, and that the majority of the team's >> packages pass the tests correctly. >> >> If Antonio agrees, I would like to set up a whitelist for ci.debian.net >> to start testing these soon, so we don't need wait for the next upload >> to enable autopkgtest in 700+ packages. > > Sure. Which list you want to use? PASS.txt? Yes, please. Thank you loads! -- Martín Ferrari (Tincho) ___ Pkg-go-maintainers mailing list Pkg-go-maintainers@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/pkg-go-maintainers
Re: [pkg-go] RFC: autopkgtest
An status update on autopkgtest for pkg-go. I have been working on this since SunCamp, and I think it is ready for deployment. I have created a much more polished script, and committed it to dh-golang. If nobody is against it, I will be uploading it during the weekend. I have also added golang support to autodep8 (which uses the script in dh-golang) and pushed the changes to git master. (Antonio, please let me know if you think this change is OK). Using these changes, I have already tested all packages that build-depend on golang, and I am satisfied that my scripts work for most of the cases out of the box, and that the majority of the team's packages pass the tests correctly. If Antonio agrees, I would like to set up a whitelist for ci.debian.net to start testing these soon, so we don't need wait for the next upload to enable autopkgtest in 700+ packages. The stats from my tests are: 694 Tests passing OK 33 Tests failing, but maintainer decided to ignore these (FIXME!) 16 Failures that seem like real problems. 22 Failures because packaging incompatibilites with the script. 17 Packages that already have autopkgtest support, or not relevant. The names for each package are included in the attached lists. On 23/05/17 11:26, Martín Ferrari wrote: > Hey all! > > As I mentioned on IRC, I have been working on getting autopkgtest > coverage for all of our packages. > > I started by writing an script[1] that re-runs the same tests we use > during package build automatically. If all packages gets this testing in > ci.debian.net, we can automatically detect any regressions due to > changes in dependencies, which is IMHO one of our biggest problems nowadays. > > Later we can add more specialised tests, or re-enable tests that can't > run at buildtime because they require networking or root access. > > > To make this available to all packages, the first thing to do is to add > golang support to autodep8, which I am preparing. > > The second thing I need to do is to add the test running scripts to some > package we control. It could be an ad-hoc package like > pkg-perl-autopkgtest, or it could be integrated into something we > already have, like dh-golang. > > I am currently leaning towards adding this same script to dh-golang, > maybe adding detection for a pkgautotest target in debian/rules (which > would be run automatically too), and getting a whitelist in > ci.debian.net to start getting test reports for all our packages. > > > Ideas, comments? > > > [1]: > https://anonscm.debian.org/cgit/pkg-go/packages/prometheus-postgres-exporter.git/tree/debian/tests/build-test > -- Martín Ferrari (Tincho) consul consul-migrate gobgp goiardi golang-github-armon-gomdb golang-github-azure-go-autorest golang-github-hashicorp-hcl golang-github-hashicorp-raft golang-github-hashicorp-raft-mdb golang-github-klauspost-compress golang-github-lib-pq golang-github-msteinert-pam golang-github-nicksnyder-go-i18n golang-github-rackspace-gophercloud google-cloud-print-connector nomad docker-libkv etcd-fs golang-github-aanand-compose-file golang-github-bugsnag-bugsnag-go golang-github-cloudflare-cfssl golang-github-cloudflare-go-metrics golang-github-docker-go-connections golang-github-docker-libnetwork golang-github-elazarl-goproxy golang-github-franela-goblin golang-github-go-macaron-gzip golang-github-go-macaron-session golang-github-go-openapi-runtime golang-github-go-openapi-validate golang-github-hashicorp-go-getter golang-github-hashicorp-memberlist golang-github-hashicorp-serf golang-github-hawkular-hawkular-client-go golang-github-hpcloud-tail golang-github-jeromer-syslogparser golang-github-mitchellh-cli golang-github-mitchellh-hashstructure golang-github-racksec-srslog golang-github-samalba-dockerclient golang-github-smartystreets-assertions golang-github-smartystreets-goconvey golang-github-spacejam-loghisto golang-github-unknwon-com golang-github-vmware-photon-controller-go-sdk golang-gopkg-dancannon-gorethink.v2 golang-gopkg-olivere-elastic.v2 runc swarmkit containerd direnv docker-swarm gitlab-ci-multi-runner golang-git2go golang-github-docker-spdystream golang-github-fsouza-go-dockerclient golang-github-googleapis-proto-client-go golang-github-gosexy-gettext golang-github-jroimartin-gocui golang-github-nsf-termbox-go golang-github-revel-revel golang-github-rfjakob-eme golang-go.net-dev golang-metrics golang-vhost golang-websocket golang-xi2-x-xz kubernetes-addon-heapster notary prometheus-alertmanager restic abci acbuild acmetool cadvisor certspotter chasquid codesearch coyim dh-make-golang dnss docker-registry etcd flannel fuji gitlab-workhorse gocode gocryptfs golang-airbrake-go golang-barcode golang-bazil-fuse golang-bindata golang-bitbucket-pkg-inflect golang-blackfriday golang-bugsnag-panicwrap golang-check.v1 golang-codegangsta-cli golang-code.gitea-git golang-code.gitea-sdk golang-collectd golang-context golang-coreos-log golang-dbus golang-dns golang-eclipse-paho golang-ed25519-de
[pkg-go] RFC: autopkgtest
Hey all! As I mentioned on IRC, I have been working on getting autopkgtest coverage for all of our packages. I started by writing an script[1] that re-runs the same tests we use during package build automatically. If all packages gets this testing in ci.debian.net, we can automatically detect any regressions due to changes in dependencies, which is IMHO one of our biggest problems nowadays. Later we can add more specialised tests, or re-enable tests that can't run at buildtime because they require networking or root access. To make this available to all packages, the first thing to do is to add golang support to autodep8, which I am preparing. The second thing I need to do is to add the test running scripts to some package we control. It could be an ad-hoc package like pkg-perl-autopkgtest, or it could be integrated into something we already have, like dh-golang. I am currently leaning towards adding this same script to dh-golang, maybe adding detection for a pkgautotest target in debian/rules (which would be run automatically too), and getting a whitelist in ci.debian.net to start getting test reports for all our packages. Ideas, comments? [1]: https://anonscm.debian.org/cgit/pkg-go/packages/prometheus-postgres-exporter.git/tree/debian/tests/build-test -- Martín Ferrari (Tincho) ___ Pkg-go-maintainers mailing list Pkg-go-maintainers@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/pkg-go-maintainers