Bug#886096: lintian: Emit warnings for Alioth URLs in packaging (migration to Salsa)

[Chris Lamb]

tags 886096 + pending
thanks

Thanks! Fixed in Git, pending upload:

  
https://anonscm.debian.org/git/lintian/lintian.git/commit/?id=04f5f6facceb9c51502f066f82a14d30472eb962


Regards,

-- 
  ,''`.
 : :'  : Chris Lamb
 `. `'`  la...@debian.org / chris-lamb.co.uk
   `-

Bug#886096: lintian: Emit warnings for Alioth URLs in packaging (migration to Salsa)

[Stuart Prescott]

On Sunday, 18 March 2018 17:15:12 AEDT Chris Lamb wrote:
> tags 886096 + patch
> thanks
> 
> Hi Stuart
> 
> > [..]
> 
> Thanks! Can you check the attached draft patch? Note that I hijacked
> te "vcs-deprecated-in-debian-infrastructure" tag instead of adding a
> new one as that had 99% overlap..
> 
> I'd love to get this applied and uploaded today. :)
> 
>   commit ce133079d182f8005e8896da2c9215057304bc89
>   Author: Chris Lamb 
>   Date:   Sun Mar 18 13:11:21 2018 -0400

Looks good to me!

thanks
Stuart

-- 
Stuart Prescotthttp://www.nanonanonano.net/   stu...@nanonanonano.net
Debian Developer   http://www.debian.org/ stu...@debian.org
GPG fingerprint90E2 D2C1 AD14 6A1B 7EBB 891D BBC1 7EBB 1396 F2F7

Bug#886096: lintian: Emit warnings for Alioth URLs in packaging (migration to Salsa)

[Chris Lamb]

tags 886096 + patch
thanks

Hi Stuart

> [..]

Thanks! Can you check the attached draft patch? Note that I hijacked
te "vcs-deprecated-in-debian-infrastructure" tag instead of adding a
new one as that had 99% overlap..

I'd love to get this applied and uploaded today. :)

  commit ce133079d182f8005e8896da2c9215057304bc89
  Author: Chris Lamb 
  Date:   Sun Mar 18 13:11:21 2018 -0400
  
  Upgrade vcs-deprecated-in-debian-infrastructure to "W:" from "P" due to 
Alioth becoming read-only from May 1st, as well as additionally checking 
Vcs-Browser fields hosted on Alioth and updating the tag description with more 
details. (Closes: #886096)
  
   checks/fields.desc | 31 
+-
   checks/fields.pm   | 11 
   debian/changelog   |  7 -
   .../debian/debian/control.in   |  1 +
   .../tags   |  3 ++-
   5 files changed, 39 insertions(+), 14 deletions(-)


Regards,

-- 
  ,''`.
 : :'  : Chris Lamb
 `. `'`  la...@debian.org / chris-lamb.co.uk
   `-
From ce133079d182f8005e8896da2c9215057304bc89 Mon Sep 17 00:00:00 2001
From: Chris Lamb 
Date: Sun, 18 Mar 2018 13:11:21 -0400
Subject: [PATCH] Upgrade vcs-deprecated-in-debian-infrastructure to "W:" from
 "P" due to Alioth becoming read-only from May 1st, as well as additionally
 checking Vcs-Browser fields hosted on Alioth and updating the tag description
 with more details. (Closes: #886096)

---
 checks/fields.desc | 31 +-
 checks/fields.pm   | 11 
 debian/changelog   |  7 -
 .../debian/debian/control.in   |  1 +
 .../tags   |  3 ++-
 5 files changed, 39 insertions(+), 14 deletions(-)

diff --git a/checks/fields.desc b/checks/fields.desc
index 54b6584b5..102183019 100644
--- a/checks/fields.desc
+++ b/checks/fields.desc
@@ -1386,19 +1386,18 @@ Info:
  the specified VCS field if it is otherwise wrong.
 
 Tag: vcs-deprecated-in-debian-infrastructure
-Severity: pedantic
+Severity: normal
 Certainty: certain
 Info: The specified Vcs-* field points to an area within the *.debian.org
  infrastructure but refers to a version control system that has been
  deprecated.
  .
  After 1st May 2018, Debian will not offer hosting for any version
- control system other than Git.
- .
- Whilst maintainers are free to use other version control systems hosted
- elsewhere, potential contributors are likely to be most familiar with
- Git.
-Ref: https://lists.debian.org/debian-devel-announce/2017/08/msg8.html
+ control system other than Git and the Alioth service will become
+ read-only in May 2018. Packages should migrate to Git hosting on
+ https://salsa.debian.org.
+Ref: https://lists.debian.org/debian-devel-announce/2017/08/msg8.html,
+ https://wiki.debian.org/Salsa
 
 Tag: co-maintained-package-with-no-vcs-fields
 Severity: pedantic
@@ -1496,3 +1495,21 @@ Info: This package has a relationship with a mail-transport-agent but
  Please change the dependency to:
  .
   default-mta | pkgname | mail-transport-agent
+
+Tag: vcs-field-uses-deprecated-infrastructure
+Severity: normal
+Certainty: certain
+Info: The specified Vcs-* field contains a URL pointing to the
+ deprecated alioth.debian.org infrastructure. This server will become
+ read-only in May 2018.
+ .
+ Packages should migrate to Git hosting on https://salsa.debian.org.
+ .
+ For further information about salsa.debian.org, please see:
+ .
+  https://wiki.debian.org/Salsa
+ .
+ HTTP redirects from alioth to salsa can be made by requesting that they
+ added to the AliothRewriter project:
+ .
+  https://salsa.debian.org/salsa/AliothRewriter/
diff --git a/checks/fields.pm b/checks/fields.pm
index e80ae24b8..50d41eb7f 100644
--- a/checks/fields.pm
+++ b/checks/fields.pm
@@ -1380,17 +1380,18 @@ sub run {
 if ($vcs eq 'browser') {
 tag 'vcs-browser-links-to-empty-view', $uri
   if $uri =~ m%rev=0=0%;
-next;
+} else {
+$seen_vcs{$vcs}++;
 }
-$seen_vcs{$vcs}++;
-if ($uri =~ m/\.debian\.org/) {
+if ($uri =~ m{//([^.]+)\.debian\.org/}) {
 tag 'vcs-deprecated-in-debian-infrastructure', "vcs-$vcs", $uri
-  if $vcs ne 'git';
+  if $1 ne 'salsa';
 } else {
 tag 'orphaned-package-not-maintained-in-debian-infrastructure',
   "vcs-$vcs", $uri
   if $info->field('maintainer', '')
-  =~ /packages\@qa.debian.org/;
+  =~ /packages\@qa.debian.org/
+  and $vcs ne 'browser';
 }
 }
 }
diff --git 

Bug#886096: lintian: Emit warnings for Alioth URLs in packaging (migration to Salsa)

[Stuart Prescott]

> Can you help by drafting the specific tag (or tags?) and giving some
> examples of Vcs-Foo it should trigger on?

sure, zeroth draft:


Bad Vcs-* URLs include:

Vcs-(Arch|Cvs|Darcs|Hg|Bzr|Svn):.*debian\.org

Vcs-Git:.*(git|anonscm|alioth)\.debian\.org

Vcs-Browser:.*(svn|bzr|darcs|git|hg|anoncms|anonscm|alioth)\.debian\.org



vcs-field-uses-deprecated-infrastructure

The VCS-* field contains a URI pointing at the deprecated alioth.debian.org 
infrastructure. The servers will become read-only in May 2018. Packages can 
migrate to git hosting on salsa.debian.org.

For further information about salsa.debian.org, see:
https://wiki.debian.org/Salsa

HTTP redirects from alioth to salsa can be made by requesting that they added 
to the AliothRewriter project:
https://salsa.debian.org/salsa/AliothRewriter/

Tools to help with the migration exist, including:
https://salsa.debian.org/mehdi/salsa-scripts/



For the others who have contributed to this bug -- anything else I've 
forgotten?

-- 
Stuart Prescotthttp://www.nanonanonano.net/   stu...@nanonanonano.net
Debian Developer   http://www.debian.org/ stu...@debian.org
GPG fingerprint90E2 D2C1 AD14 6A1B 7EBB 891D BBC1 7EBB 1396 F2F7

Bug#886096: lintian: Emit warnings for Alioth URLs in packaging (migration to Salsa)

[Chris Lamb]

[Adding Mattia to CC as he was did some work on this bug so inferring interest]

Hi Stuart et al.,

> > we now only have 6 time quanta until alioth is read-only
> 
> I like this "time quanta" concept :)  Okay, agreed, let's get onto
> this ASAP.
> 
> Can you help by drafting the specific tag (or tags?) and giving some
> examples of Vcs-Foo it should trigger on?

Gentle ping on this? I would love to upload another Lintian soon and
it would be a shame not to include these time-sensitive changes. 


Best wishes,

-- 
  ,''`.
 : :'  : Chris Lamb
 `. `'`  la...@debian.org / chris-lamb.co.uk
   `-

Bug#886096: lintian: Emit warnings for Alioth URLs in packaging (migration to Salsa)

[Chris Lamb]

Hi Stuart,

> we now only have 6 time quanta until alioth is read-only

I like this "time quanta" concept :)  Okay, agreed, let's get onto
this ASAP.

Can you help by drafting the specific tag (or tags?) and giving some
examples of Vcs-Foo it should trigger on?


Best wishes,

-- 
Chris Lamb
chris-lamb.co.uk / @lolamby

Bug#886096: lintian: Emit warnings for Alioth URLs in packaging (migration to Salsa)

[Stuart Prescott]

Control: tags 886096 - moreinfo

> > lintian: Emit warnings for Alioth URLs in packaging
> 
> I'm going to "park" this bug for the time being until we know the state
> of any redirection services — simply warning maintainers right now that
> they are using anonscm URIs is probably not tremendously helpful.

I think all the concerns and questions raised in this bug are now addressed.


salsa.debian.org is now out of beta and the threat of database drops has gone 
[1]; it is widely adopted by the project and impending close-down date of 
alioth draws closer [2].

It is now documented that the redirector is only for http/https URLs and it is 
not intended to be a replacement for changing the Vcs-* fields within debian/
control [3]. That ssh and git protocol connections are not redirected means 
that anonscm URLs from debian/control will play badly with existing magic from 
debcheckout or local pushInsteadOf config that was used to make those URLs 
usable for developers.

Maintainers need to move their repositories either individually or as part of 
teams and the lintian message could point towards a suitable tool like Mehdi's 
salsa-scripts to do this [4].

I think we're at the point where lintian should be giving very strong guidance 
on this matter. It's quite obviously not an ftp-master reject, but a very in-
the-face warning or error is appropriate. In a project where time is normally 
quantised in 'week' units and several quanta are normally allowed for any 
action, we now only have 6 time quanta until alioth is read-only.


[1] https://lists.debian.org/debian-devel-announce/2018/01/msg4.html

[2] Wheezy EOL, 2018-04-25 https://wiki.debian.org/
Alioth#Deprecation_of_Alioth https://wiki.debian.org/DebianWheezy

[3] https://salsa.debian.org/salsa/AliothRewriter/blob/master/README.md

[4] https://salsa.debian.org/mehdi/salsa-scripts/

-- 
Stuart Prescotthttp://www.nanonanonano.net/   stu...@nanonanonano.net
Debian Developer   http://www.debian.org/ stu...@debian.org
GPG fingerprint90E2 D2C1 AD14 6A1B 7EBB 891D BBC1 7EBB 1396 F2F7

Bug#886096: lintian: Emit warnings for Alioth URLs in packaging (migration to Salsa)

[Chris Lamb]

tags 886096 + moreinfo
thanks

Hi,

> lintian: Emit warnings for Alioth URLs in packaging

I'm going to "park" this bug for the time being until we know the state
of any redirection services — simply warning maintainers right now that
they are using anonscm URIs is probably not tremendously helpful.

(The "moreinfo" tag doesn't 100% fit this use-case, but it certainly
captures the idea that one cannot fix it but not being "wontfix". Please
adjust with something more suitable if you feel strongly enough.)


Best wishes,

-- 
  ,''`.
 : :'  : Chris Lamb
 `. `'`  la...@debian.org / chris-lamb.co.uk
   `-

Bug#886057: Bug#886096: lintian: Emit warnings for Alioth URLs in packaging (migration to Salsa)

[Chris Lamb]

Hi,

> IMHO these should emit a tag at at least warning/normal level.

Note to implementor that that we have some very similar wishlist
requests:

  #885974 "lintian: warn about non-git Vcs fields"
  #886057 "lintian: warn about 
orphaned-package-not-maintained-in-debian.org-infrastracture"
  #886096 "lintian: Emit warnings for Alioth URLs in packaging (migration to 
Salsa)"

(Thanks Mattia for the clone, etc!)


Regards,

-- 
  ,''`.
 : :'  : Chris Lamb
 `. `'`  la...@debian.org / chris-lamb.co.uk
   `-

Bug#886096: lintian: Emit warnings for Alioth URLs in packaging (migration to Salsa)

[Axel Beckert]

Hi Niels,

Niels Thykier wrote:
> My 2 cents: I do not think we should propose a mass-move until salsa is
> out of the beta.  Given that the admins have not commited to not wiping
> the database, nagging people to move seems counter productive
> (especially if a wipe is performed later).

Granted. Shouldn't though hurt to implement that now and maybe declare
it as experimental initially.

Nevertheless is the declared end of the beta phase (according to
https://lists.debian.org/debian-devel-announce/2017/12/msg3.html)
"around the end of January" while Alioth is said to be shutdown
(according to https://wiki.debian.org/Alioth/MailingListContinuation)
on 1st of February.

So IMHO we need checks of that kind rather sooner than later —
independent of Salsa still being in beta or not.

Regards, Axel
-- 
 ,''`.  |  Axel Beckert , https://people.debian.org/~abe/
: :' :  |  Debian Developer, ftp.ch.debian.org Admin
`. `'   |  4096R: 2517 B724 C5F6 CA99 5329  6E61 2FF9 CD59 6126 16B5
  `-|  1024D: F067 EA27 26B9 C3FC 1486  202E C09E 1D89 9593 0EDE

Bug#886096: lintian: Emit warnings for Alioth URLs in packaging (migration to Salsa)

[Niels Thykier]

Axel Beckert:
> Package: lintian
> Version: 2.5.67
> Severity: wishlist
> 
> According to https://wiki.debian.org/Salsa/Doc all packaging projects
> hosted on Alioth need to be moved away, preferably to Salsa.
> 
> This at least affects:
> 
> * ${VCS}.debian.org and anonscm.debian.org in Vcs-* headers and
>   debian/upstream/metadata (with ${VCS} in at least git, svn, hg, bzr,
>   darcs, arch, and cvs).
> 
> * http(s)://*.alioth.debian.org/ in Homepage headers,
>   debian/upstream/metadata, and maybe also debian/copyright.
> 
> IMHO these should emit a tag at at least warning/normal level.
> 
> According to https://wiki.debian.org/Alioth/MailingListContinuation the
> urgency for changing Maintainer fields, etc. which point to Alioth
> mailing lists is lower. Hence for now, I suggest to also emit a
> (separate) tag for these, but with severity informational/minor:
> 
> * lists.alioth.debian.org in the Maintainer and Uploaders fields
> 
> * Maybe also lists.alioth.debian.org elsewhere in packaging,
>   e.g. Mailing list archives in debian/upstream/metadata,
>   debian/copyright or so.
> 
> Additionally, some currently emitted tags can be probably removed or
> replaced with according salsa.debian.org-based tags:
> 
> * vcs-field-not-canoncial
> 
> Then there's another set which might need updates, too:
> 
> * vcs-field-bitrotted
> * vcs-field-uses-not-recommended-uri-format
> * vcs-git-uses-invalid-user-uri
> 
> Feel free to clone this bug report in case these things won't be
> implemented at the same time.
> 
> [...]

My 2 cents: I do not think we should propose a mass-move until salsa is
out of the beta.  Given that the admins have not commited to not wiping
the database, nagging people to move seems counter productive
(especially if a wipe is performed later).

Thanks,
~Niels

Bug#886096: lintian: Emit warnings for Alioth URLs in packaging (migration to Salsa)

[Axel Beckert]

Package: lintian
Version: 2.5.67
Severity: wishlist

According to https://wiki.debian.org/Salsa/Doc all packaging projects
hosted on Alioth need to be moved away, preferably to Salsa.

This at least affects:

* ${VCS}.debian.org and anonscm.debian.org in Vcs-* headers and
  debian/upstream/metadata (with ${VCS} in at least git, svn, hg, bzr,
  darcs, arch, and cvs).

* http(s)://*.alioth.debian.org/ in Homepage headers,
  debian/upstream/metadata, and maybe also debian/copyright.

IMHO these should emit a tag at at least warning/normal level.

According to https://wiki.debian.org/Alioth/MailingListContinuation the
urgency for changing Maintainer fields, etc. which point to Alioth
mailing lists is lower. Hence for now, I suggest to also emit a
(separate) tag for these, but with severity informational/minor:

* lists.alioth.debian.org in the Maintainer and Uploaders fields

* Maybe also lists.alioth.debian.org elsewhere in packaging,
  e.g. Mailing list archives in debian/upstream/metadata,
  debian/copyright or so.

Additionally, some currently emitted tags can be probably removed or
replaced with according salsa.debian.org-based tags:

* vcs-field-not-canoncial

Then there's another set which might need updates, too:

* vcs-field-bitrotted
* vcs-field-uses-not-recommended-uri-format
* vcs-git-uses-invalid-user-uri

Feel free to clone this bug report in case these things won't be
implemented at the same time.

-- System Information:
Debian Release: buster/sid
  APT prefers unstable
  APT policy: (990, 'unstable'), (600, 'testing'), (500, 'unstable-debug'), 
(500, 'buildd-unstable'), (110, 'experimental'), (1, 'experimental-debug'), (1, 
'buildd-experimental')
Architecture: amd64 (x86_64)

Kernel: Linux 4.13.0-rc7-amd64 (SMP w/8 CPU cores)
Locale: LANG=C.UTF-8, LC_CTYPE=C.UTF-8 (charmap=UTF-8), LANGUAGE=C.UTF-8 
(charmap=UTF-8)
Shell: /bin/sh linked to /bin/dash
Init: sysvinit (via /sbin/init)

Versions of packages lintian depends on:
ii  binutils  2.29.1-12
ii  bzip2 1.0.6-8.1
ii  diffstat  1.61-1+b1
ii  dpkg  1.19.0.4
ii  file  1:5.32-1
ii  gettext   0.19.8.1-4
ii  intltool-debian   0.35.0+20060710.4
ii  libapt-pkg-perl   0.1.33
ii  libarchive-zip-perl   1.60-1
ii  libclass-accessor-perl0.51-1
ii  libclone-perl 0.39-1
ii  libdigest-sha-perl6.01-1
ii  libdpkg-perl  1.19.0.4
ii  libemail-valid-perl   1.202-1
ii  libfile-basedir-perl  0.07-1
ii  libipc-run-perl   0.96-1
ii  liblist-moreutils-perl0.416-1+b3
ii  libparse-debianchangelog-perl 1.2.0-12
ii  libperl5.26 [libdigest-sha-perl]  5.26.1-3
ii  libtext-levenshtein-perl  0.13-1
ii  libtimedate-perl  2.3000-2
ii  liburi-perl   1.72-2
ii  libxml-simple-perl2.24-1
ii  libyaml-libyaml-perl  0.63-2+b2
ii  man-db2.7.6.1-4
ii  patchutils0.3.4-2
ii  perl  5.26.1-3
ii  t1utils   1.41-2
ii  xz-utils  5.2.2-1.3

Versions of packages lintian recommends:
ii  libperlio-gzip-perl  0.19-1+b4

Versions of packages lintian suggests:
ii  binutils-multiarch 2.29.1-12
ii  dpkg-dev   1.19.0.4
ii  libhtml-parser-perl3.72-3+b2
ii  libtext-template-perl  1.47-1

-- no debconf information