RE: Uscan no longer works with GitLab tags

[Thomas Ward]

Is there a specific reason you can't follow tags and their corresponding 
uploads? A long while ago I came up with a mechanism to force uscan to use the 
gitlab JSON API to get downloads, etc. but it was finnicky and I don't have the 
exact solutions here since that was long enough ago I lost a computer and its 
data and lost my solution to scrape the json with uscan.



Sent from my Galaxy



 Original message 
From: Michel Lind 
Date: 4/23/24 17:04 (GMT-05:00)
To: Fay Stegerman , debian-mentors@lists.debian.org
Subject: Re: Uscan no longer works with GitLab tags



On 4/3/24 5:19 PM, Fay Stegerman wrote:
> * Mike Gabriel  [2024-04-03 10:00]:
>>> On Tuesday, April 2, 2024 8:22:26 AM MST Mike Gabriel wrote:
>>>> https://salsa.debian.org/debian-ayatana-team/appmenu-gtk-module/-/blob/
>>> master/
>>>> debian/watch
>>>>
>>>> ```
>>>> version=3
>>>> https://gitlab.com/vala-panel-project/vala-panel-appmenu/-/tags/?([\d\.]+)
>>>> .*/uploads/.*/appmenu-gtk-module-?([\d\.]+)\.tar\.xz
>>>> ```
>>>>
>>>> The above used to successfully download the appmenu-gtk-module tarball
>>>> from the vala-panel-appmenu releases page:
>>>> https://gitlab.com/vala-panel-project/vala-panel-appmenu/-/releases
>>>
>>> Try the attached watch file.
>>
>> This works for vala-panel-appmenu, but not for appmenu-gtk-module. The
>> packaging of vala-panel-appmenu, appmenu-gtk-module and appmenu-registrar is
>> a bit special, because all three packages get built from the same source
>> code tree.
> [...]
>> The appmenu-gtk-module code is a subfolder in upstream vala-panel-appmenu
>> (subprojects/appmenu-gtk-module) and that subfolder was packaged as a
>> separate src:pkg in Debian at the time when it got introduced.
>>
>> For this the upstream maintainer provides appmenu-gtk-module as a separate
>> tarball  for download at [1].
>>
>> So the watch file should achieve downloading this exact tarball, i.e. 
>> https://gitlab.com/vala-panel-project/vala-panel-appmenu/uploads/6c0332e34c41e99de5a1db1fc4239de2/appmenu-gtk-module-24.02.tar.xz
>>
>> Only chew on this if you really want to nut-crack it. I have burnt quite a
>> few brain cells on it yesterday and failed (which does not mean you will
>> also, but be warned, the solution does not seem trivial, however, maybe it
>> is).
>
> GitLab asset download URLs are a bit of a mess, but the attached watch file
> seems to give me that exact URL.
>
Thanks for this! I have a similar problem with archlinux-keyring - which uses a
different GitLab instance, gitlab.archlinux.org

Using your watch file as a template, I managed to capture the exact URL needed
(tried using that and also matching on the releases URL), but I still get stuck
on the fact that uscan gets given an HTML page prompting to login instead.

wget handled the URL just fine. curl also gets an HTML page...

Best regards,

--
  _o) Michel Lind
_( ) identities: https://keyoxide.org/5dce2e7e9c3b1cffd335c1d78b229d2f7ccc04f2

Re: Uscan no longer works with GitLab tags

[Michel Lind]

Hi Thomas,

On 4/23/24 4:09 PM, Thomas Ward wrote:
Is there a specific reason you can't follow tags and their corresponding 
uploads? A long while ago I came up with a mechanism to force uscan to use the 
gitlab JSON API to get downloads, etc. but it was finnicky and I don't have the 
exact solutions here since that was long enough ago I lost a computer and its 
data and lost my solution to scrape the json with uscan.


No specific reason. From the discussion it seems that the JSON API is 
preferable, but is not recommended by default because of the need to mangle the 
owner/project URL


If it turns out to be more brittle I'll probably change it back - this project 
is finnicky anyway since like the Vala project Fay's example was for, I needed 
to get one of the manually uploaded tarballs instead of the auto-generated 
tag... and I also need to override the version (to prepend 0~).


Best regards,

--
 _o) Michel Lind
_( ) identities: https://keyoxide.org/5dce2e7e9c3b1cffd335c1d78b229d2f7ccc04f2


OpenPGP_0x8B229D2F7CCC04F2.asc
Description: OpenPGP public key


OpenPGP_signature.asc
Description: OpenPGP digital signature

Re: Uscan no longer works with GitLab tags

[Michel Lind]

On 4/23/24 4:02 PM, Michel Lind wrote:



On 4/3/24 5:19 PM, Fay Stegerman wrote:

GitLab asset download URLs are a bit of a mess, but the attached watch file
seems to give me that exact URL.

Thanks for this! I have a similar problem with archlinux-keyring - which uses a 
different GitLab instance, gitlab.archlinux.org


Using your watch file as a template, I managed to capture the exact URL needed 
(tried using that and also matching on the releases URL), but I still get stuck 
on the fact that uscan gets given an HTML page prompting to login instead.


Ah, turns out matching the upload found the .tar.gz.sig instead since it 
appeared earlier in the releases JSON, so the upload hash is wrong.


Finally ended up matching the release URL instead - luckily uscan (and wget) 
handles the redirection back to the uploads URL OK.


https://salsa.debian.org/michel/archlinux-keyring/-/commit/7c096ea23aad8e0c9f66819cdda2a7d7e094f196

Thanks for figuring out the API access. I'm using it from now on since hopefully 
it'll be more stable.


--
 _o) Michel Lind
_( ) identities: https://keyoxide.org/5dce2e7e9c3b1cffd335c1d78b229d2f7ccc04f2


OpenPGP_0x8B229D2F7CCC04F2.asc
Description: OpenPGP public key


OpenPGP_signature.asc
Description: OpenPGP digital signature

Re: Uscan no longer works with GitLab tags

[Michel Lind]

On 4/3/24 5:19 PM, Fay Stegerman wrote:

* Mike Gabriel  [2024-04-03 10:00]:

On Tuesday, April 2, 2024 8:22:26 AM MST Mike Gabriel wrote:

https://salsa.debian.org/debian-ayatana-team/appmenu-gtk-module/-/blob/

master/

debian/watch

```
version=3
https://gitlab.com/vala-panel-project/vala-panel-appmenu/-/tags/?([\d\.]+)
.*/uploads/.*/appmenu-gtk-module-?([\d\.]+)\.tar\.xz
```

The above used to successfully download the appmenu-gtk-module tarball
from the vala-panel-appmenu releases page:
https://gitlab.com/vala-panel-project/vala-panel-appmenu/-/releases


Try the attached watch file.


This works for vala-panel-appmenu, but not for appmenu-gtk-module. The
packaging of vala-panel-appmenu, appmenu-gtk-module and appmenu-registrar is
a bit special, because all three packages get built from the same source
code tree.

[...]

The appmenu-gtk-module code is a subfolder in upstream vala-panel-appmenu
(subprojects/appmenu-gtk-module) and that subfolder was packaged as a
separate src:pkg in Debian at the time when it got introduced.

For this the upstream maintainer provides appmenu-gtk-module as a separate
tarball  for download at [1].

So the watch file should achieve downloading this exact tarball, i.e. 
https://gitlab.com/vala-panel-project/vala-panel-appmenu/uploads/6c0332e34c41e99de5a1db1fc4239de2/appmenu-gtk-module-24.02.tar.xz

Only chew on this if you really want to nut-crack it. I have burnt quite a
few brain cells on it yesterday and failed (which does not mean you will
also, but be warned, the solution does not seem trivial, however, maybe it
is).


GitLab asset download URLs are a bit of a mess, but the attached watch file
seems to give me that exact URL.

Thanks for this! I have a similar problem with archlinux-keyring - which uses a 
different GitLab instance, gitlab.archlinux.org


Using your watch file as a template, I managed to capture the exact URL needed 
(tried using that and also matching on the releases URL), but I still get stuck 
on the fact that uscan gets given an HTML page prompting to login instead.


wget handled the URL just fine. curl also gets an HTML page...

Best regards,

--
 _o) Michel Lind
_( ) identities: https://keyoxide.org/5dce2e7e9c3b1cffd335c1d78b229d2f7ccc04f2


OpenPGP_0x8B229D2F7CCC04F2.asc
Description: OpenPGP public key


OpenPGP_signature.asc
Description: OpenPGP digital signature

Re: Uscan no longer works with GitLab tags

[Andrey Rakhmatullin]

On Thu, Apr 18, 2024 at 02:43:23AM +, Nilson Silva wrote:
> Hey!
> The purpose of this email is to contribute to the recent changes that 
> occurred in Gitlab with its 
> HTML that caused a series of errors in tracking new versions.
> 
> Reading the wiki more specifically at this point:
>  https://wiki.debian.org/debian/watch#GitLab,
>  and trying to apply it to my case, there was no success.
> 
> So, after some research, I came to this result:
> 
> version=4
> opts="searchmode=plain, uversionmangle=s/\.(\d+\.\d+.\d+)//" \
> https://gitlab.com///tags?sort=updated_desc 
> -/archive/(\d+.\d+.\d+)/-(\d+.\d+.\d+).tar .gz
Capturing the version twice and then removing the extra one via
uversionmangle is really wrong (you tried to discuss this on IRC yesterday
but ignored the suggestions).

-- 
WBR, wRAR


signature.asc
Description: PGP signature

Uscan no longer works with GitLab tags

[Nilson Silva]

Hey!
The purpose of this email is to contribute to the recent changes that occurred 
in Gitlab with its 
HTML that caused a series of errors in tracking new versions.

Reading the wiki more specifically at this point:
 https://wiki.debian.org/debian/watch#GitLab,
 and trying to apply it to my case, there was no success.

So, after some research, I came to this result:

version=4
opts="searchmode=plain, uversionmangle=s/\.(\d+\.\d+.\d+)//" \
https://gitlab.com///tags?sort=updated_desc 
-/archive/(\d+.\d+.\d+)/-(\d+.\d+.\d+).tar .gz

Which worked perfectly and from what I saw will work for many maintainers.

the corrected package was this:
https://salsa.debian.org/python-team/packages/python-public

I hope it gets added to this article.

Dear
Nilson F. Silva

Re: Uscan no longer works with GitLab tags

[Fay Stegerman]

* Mike Gabriel  [2024-04-03 10:00]:
> > On Tuesday, April 2, 2024 8:22:26 AM MST Mike Gabriel wrote:
> > > https://salsa.debian.org/debian-ayatana-team/appmenu-gtk-module/-/blob/
> > master/
> > > debian/watch
> > > 
> > > ```
> > > version=3
> > > https://gitlab.com/vala-panel-project/vala-panel-appmenu/-/tags/?([\d\.]+)
> > > .*/uploads/.*/appmenu-gtk-module-?([\d\.]+)\.tar\.xz
> > > ```
> > > 
> > > The above used to successfully download the appmenu-gtk-module tarball
> > > from the vala-panel-appmenu releases page:
> > > https://gitlab.com/vala-panel-project/vala-panel-appmenu/-/releases
> > 
> > Try the attached watch file.
> 
> This works for vala-panel-appmenu, but not for appmenu-gtk-module. The
> packaging of vala-panel-appmenu, appmenu-gtk-module and appmenu-registrar is
> a bit special, because all three packages get built from the same source
> code tree.
[...]
> The appmenu-gtk-module code is a subfolder in upstream vala-panel-appmenu
> (subprojects/appmenu-gtk-module) and that subfolder was packaged as a
> separate src:pkg in Debian at the time when it got introduced.
> 
> For this the upstream maintainer provides appmenu-gtk-module as a separate
> tarball  for download at [1].
> 
> So the watch file should achieve downloading this exact tarball, i.e. 
> https://gitlab.com/vala-panel-project/vala-panel-appmenu/uploads/6c0332e34c41e99de5a1db1fc4239de2/appmenu-gtk-module-24.02.tar.xz
> 
> Only chew on this if you really want to nut-crack it. I have burnt quite a
> few brain cells on it yesterday and failed (which does not mean you will
> also, but be warned, the solution does not seem trivial, however, maybe it
> is).

GitLab asset download URLs are a bit of a mess, but the attached watch file
seems to give me that exact URL.

- Fay
version=4
opts="searchmode=plain,downloadurlmangle=s!uploads!vala-panel-project/vala-panel-appmenu/uploads!"
 \
 
https://gitlab.com/api/v4/projects/vala-panel-project%2Fvala-panel-appmenu/releases
 \
 /uploads/[0-9a-f]+/appmenu-gtk-module-?([\d\.]+)\.tar\.xz

Re: Uscan no longer works with GitLab tags

[Mike Gabriel]

Hi Soren,

On  Mi 03 Apr 2024 14:00:03 CEST, Soren Stoutner wrote:


On Wednesday, April 3, 2024 1:00:28 AM MST Mike Gabriel wrote:

The appmenu-gtk-module code is a subfolder in upstream
vala-panel-appmenu (subprojects/appmenu-gtk-module) and that subfolder
was packaged as a separate src:pkg in Debian at the time when it got
introduced.

For this the upstream maintainer provides appmenu-gtk-module as a
separate tarball  for download at [1].

So the watch file should achieve downloading this exact tarball, i.e.
https://gitlab.com/vala-panel-project/vala-panel-appmenu/uploads/6c0332e34c41e
99de5a1db1fc4239de2/appmenu-gtk-module-24.02.tar.xz

Only chew on this if you really want to nut-crack it. I have burnt
quite a few brain cells on it yesterday and failed (which does not
mean you will also, but be warned, the solution does not seem trivial,
however, maybe it is).


That is indeed complex as the tarball is only available as an upload  
the developer has
manually added to GitLab and not as one of the standard release  
tarballs generated by

GitLab.

If GitLab weren’t being a punk and would present the same HTML a  
browser gets to uscan

(or wget) when pulling up:

https://gitlab.com/vala-panel-project/vala-panel-appmenu/-/releases[1]

it would be easy.  But GitLab is presenting what amounts to a login  
screen to uscan instead.


You might be able to resolve the situation by switching to using Git  
to download a tag and
then producing a tarball locally from that, but I don’t know how to  
have uscan produce a
tarball from just a subdirectory in Git.  Perhaps one of the people  
who are currently
advocating using git instead of pre-packaged tarballs on  
devian-devel would have an idea

of how to do that.


Thanks for taking a closer look. I'll need to find some quiet moment  
and then do a manual re-packaging in d/rules, I guess. Just like the  
really old way that @onlyjob in Debian wiki years ago.


Greets,
Mike
--

mike gabriel aka sunweaver (Debian Developer)
mobile: +49 (1520) 1976 148
landline: +49 (4351) 486 14 27

GnuPG Fingerprint: 9BFB AEE8 6C0A A5FF BF22  0782 9AF4 6B30 2577 1B31
mail: sunwea...@debian.org, http://sunweavers.net



pgptjwUKdht0l.pgp
Description: Digitale PGP-Signatur

Re: Uscan no longer works with GitLab tags

[Mike Gabriel]

On  Mi 03 Apr 2024 14:28:55 CEST, Soren Stoutner wrote:


On Wednesday, April 3, 2024 5:00:03 AM MST Soren Stoutner wrote:

That is indeed complex as the tarball is only available as an upload the
developer has manually added to GitLab and not as one of the  
standard release

tarballs generated by GitLab.

If GitLab weren’t being a punk and would present the same HTML a  
browser gets

to uscan (or wget) when pulling up:

https://gitlab.com/vala-panel-project/vala-panel-appmenu/-/releases[1]

it would be easy.  But GitLab is presenting what amounts to a login  
screen to

uscan instead.


I submitted a request to GitLab to correct this buggy behavior.

https://gitlab.com/gitlab-org/gitlab/-/issues/454301[1]


Thanks for that! Makes total sense complaining over at GitLab upstream.

Mike
--

mike gabriel aka sunweaver (Debian Developer)
mobile: +49 (1520) 1976 148
landline: +49 (4351) 486 14 27

GnuPG Fingerprint: 9BFB AEE8 6C0A A5FF BF22  0782 9AF4 6B30 2577 1B31
mail: sunwea...@debian.org, http://sunweavers.net



pgpsN6Rv8WC7O.pgp
Description: Digitale PGP-Signatur

Re: Uscan no longer works with GitLab tags

[Soren Stoutner]

On Wednesday, April 3, 2024 5:00:03 AM MST Soren Stoutner wrote:
> That is indeed complex as the tarball is only available as an upload the
> developer has manually added to GitLab and not as one of the standard release
> tarballs generated by GitLab.
> 
> If GitLab weren’t being a punk and would present the same HTML a browser gets
> to uscan (or wget) when pulling up:
> 
> https://gitlab.com/vala-panel-project/vala-panel-appmenu/-/releases[1]
> 
> it would be easy.  But GitLab is presenting what amounts to a login screen to
> uscan instead.

I submitted a request to GitLab to correct this buggy behavior.

https://gitlab.com/gitlab-org/gitlab/-/issues/454301[1]

-- 
Soren Stoutner
so...@debian.org


[1] https://gitlab.com/gitlab-org/gitlab/-/issues/454301


signature.asc
Description: This is a digitally signed message part.

Re: Uscan no longer works with GitLab tags

[Soren Stoutner]

On Wednesday, April 3, 2024 1:00:28 AM MST Mike Gabriel wrote:
> The appmenu-gtk-module code is a subfolder in upstream
> vala-panel-appmenu (subprojects/appmenu-gtk-module) and that subfolder
> was packaged as a separate src:pkg in Debian at the time when it got
> introduced.
> 
> For this the upstream maintainer provides appmenu-gtk-module as a
> separate tarball  for download at [1].
> 
> So the watch file should achieve downloading this exact tarball, i.e.
> https://gitlab.com/vala-panel-project/vala-panel-appmenu/uploads/6c0332e34c41e
> 99de5a1db1fc4239de2/appmenu-gtk-module-24.02.tar.xz
> 
> Only chew on this if you really want to nut-crack it. I have burnt
> quite a few brain cells on it yesterday and failed (which does not
> mean you will also, but be warned, the solution does not seem trivial,
> however, maybe it is).

That is indeed complex as the tarball is only available as an upload the 
developer has 
manually added to GitLab and not as one of the standard release tarballs 
generated by 
GitLab.

If GitLab weren’t being a punk and would present the same HTML a browser gets 
to uscan 
(or wget) when pulling up:

https://gitlab.com/vala-panel-project/vala-panel-appmenu/-/releases[1]

it would be easy.  But GitLab is presenting what amounts to a login screen to 
uscan instead.

You might be able to resolve the situation by switching to using Git to 
download a tag and 
then producing a tarball locally from that, but I don’t know how to have uscan 
produce a 
tarball from just a subdirectory in Git.  Perhaps one of the people who are 
currently 
advocating using git instead of pre-packaged tarballs on devian-devel would 
have an idea 
of how to do that.

-- 
Soren Stoutner
so...@debian.org


[1] https://gitlab.com/vala-panel-project/vala-panel-appmenu/-/releases


signature.asc
Description: This is a digitally signed message part.

Re: Uscan no longer works with GitLab tags

[Mike Gabriel]

Hi Soren,

On  Mi 03 Apr 2024 06:12:59 CEST, Soren Stoutner wrote:


Mike,

On Tuesday, April 2, 2024 8:22:26 AM MST Mike Gabriel wrote:

https://salsa.debian.org/debian-ayatana-team/appmenu-gtk-module/-/blob/

master/

debian/watch

```
version=3
https://gitlab.com/vala-panel-project/vala-panel-appmenu/-/tags/?([\d\.]+)
.*/uploads/.*/appmenu-gtk-module-?([\d\.]+)\.tar\.xz
```

The above used to successfully download the appmenu-gtk-module tarball
from the vala-panel-appmenu releases page:
https://gitlab.com/vala-panel-project/vala-panel-appmenu/-/releases


Try the attached watch file.


This works for vala-panel-appmenu, but not for appmenu-gtk-module. The  
packaging of vala-panel-appmenu, appmenu-gtk-module and  
appmenu-registrar is a bit special, because all three packages get  
built from the same source code tree.


[sunweaver@sunobo vala-panel-appmenu.upstream (master)]$ tree -L 1
.
??? applets
??? data
??? lib
??? LICENSE
??? meson.build
??? meson_options.txt
??? po
??? README.md
??? subprojects
??? vapi

7 directories, 4 files
[sunweaver@sunobo vala-panel-appmenu.upstream (master)]$ tree -L 1  
subprojects/

subprojects/
??? appmenu-glib-translator
??? appmenu-gtk-module
??? jayatana
??? registrar

5 directories, 0 files

The appmenu-gtk-module code is a subfolder in upstream  
vala-panel-appmenu (subprojects/appmenu-gtk-module) and that subfolder  
was packaged as a separate src:pkg in Debian at the time when it got  
introduced.


For this the upstream maintainer provides appmenu-gtk-module as a  
separate tarball  for download at [1].


So the watch file should achieve downloading this exact tarball, i.e.  
https://gitlab.com/vala-panel-project/vala-panel-appmenu/uploads/6c0332e34c41e99de5a1db1fc4239de2/appmenu-gtk-module-24.02.tar.xz


Only chew on this if you really want to nut-crack it. I have burnt  
quite a few brain cells on it yesterday and failed (which does not  
mean you will also, but be warned, the solution does not seem trivial,  
however, maybe it is).


Thanks,
Mike

[1] https://gitlab.com/vala-panel-project/vala-panel-appmenu/-/releases/24.02

--

mike gabriel aka sunweaver (Debian Developer)
mobile: +49 (1520) 1976 148
landline: +49 (4351) 486 14 27

GnuPG Fingerprint: 9BFB AEE8 6C0A A5FF BF22  0782 9AF4 6B30 2577 1B31
mail: sunwea...@debian.org, http://sunweavers.net



pgpQhidd9Un96.pgp
Description: Digitale PGP-Signatur

Re: Uscan no longer works with GitLab tags

[Soren Stoutner]

Mike,

On Tuesday, April 2, 2024 8:22:26 AM MST Mike Gabriel wrote:
> https://salsa.debian.org/debian-ayatana-team/appmenu-gtk-module/-/blob/
master/
> debian/watch
> 
> ```
> version=3
> https://gitlab.com/vala-panel-project/vala-panel-appmenu/-/tags/?([\d\.]+)
> .*/uploads/.*/appmenu-gtk-module-?([\d\.]+)\.tar\.xz
> ```
> 
> The above used to successfully download the appmenu-gtk-module tarball
> from the vala-panel-appmenu releases page:
> https://gitlab.com/vala-panel-project/vala-panel-appmenu/-/releases

Try the attached watch file.

-- 
Soren Stoutner
so...@debian.orgversion=4
opts="searchmode=plain" \
 
https://gitlab.com/vala-panel-project/vala-panel-appmenu/tags?sort=updated_desc 
-/archive/v?\d[\d.]+/vala-panel-appmenu-@any_vers...@.tar.gz


signature.asc
Description: This is a digitally signed message part.

Re: Uscan no longer works with GitLab tags

[Mike Gabriel]

Hi all,

On  Mo 25 Mär 2024 07:27:19 CET, Shriram Ravindranathan wrote:

It seems like a few other packages are also having similar troubles  
with uscan, for example lomiri 


For lomiri, I have fixed those watch files now (won't do a mass  
upload, fixes will trickle in slowly).


However, I have a challenge for uscan experts:

https://salsa.debian.org/debian-ayatana-team/appmenu-gtk-module/-/blob/master/debian/watch

```
version=3
https://gitlab.com/vala-panel-project/vala-panel-appmenu/-/tags/?([\d\.]+)  
.*/uploads/.*/appmenu-gtk-module-?([\d\.]+)\.tar\.xz

```

The above used to successfully download the appmenu-gtk-module tarball  
from the vala-panel-appmenu releases page:

https://gitlab.com/vala-panel-project/vala-panel-appmenu/-/releases

However, I fail to fiddle the new watch file together working with the  
new GitLab releases page rendering/delivery.


Any ideas?

Mike

PS: Make sure you Cc: me when replying, I am not following  
debian-mentos (sorry!!!) regularly.

--

mike gabriel aka sunweaver (Debian Developer)
mobile: +49 (1520) 1976 148
landline: +49 (4351) 486 14 27

GnuPG Fingerprint: 9BFB AEE8 6C0A A5FF BF22  0782 9AF4 6B30 2577 1B31
mail: sunwea...@debian.org, http://sunweavers.net



pgpyGi54tOqZ6.pgp
Description: Digitale PGP-Signatur

Re: Uscan no longer works with GitLab tags

[Sergio Durigan Junior]

On Tuesday, March 26 2024, Soren Stoutner wrote:

> https://wiki.debian.org/debian/watch#Gitlab[1]
>
> "From 2024/03 above solution do not work anymore since Gitlab changed their 
> tag 
> ordering. You can use direct API access:”
>
> version=4
> opts=\
> searchmode=plain,\
>https://gitlab.com/api/v4/projects/PROJECTID/releases 
> archive/v?\d[\d.]+//-v?
> ([\d.]+)@ARCHIVE_EXT@

My initial (successful) attempt was to use the API directly, but some
people might be confused about the PROJECTID part (it needs to be
URL-encoded unless you're specifying the actual integer ID directly).

Either way, here are the two options:

version=4
opts=searchmode=plain \
   https://gitlab.com/saalen/highlight/-/tags?sort=updated_desc 
/archive/@ANY_VERSION@/@PACKAGE@@ANY_VERSION@@ARCHIVE_EXT@
#  Using the API:
#  https://gitlab.com/api/v4/projects/saalen%2Fhighlight/releases 
/archive/@ANY_VERSION@/@PACKAGE@@ANY_VERSION@@ARCHIVE_EXT@

-- 
Sergio
GPG key ID: 237A 54B1 0287 28BF 00EF  31F4 D0EB 7628 65FC 5E36
Please send encrypted e-mail if possible
https://sergiodj.net/


signature.asc
Description: PGP signature

Re: Uscan no longer works with GitLab tags

[Soren Stoutner]

https://wiki.debian.org/debian/watch#Gitlab[1]

"From 2024/03 above solution do not work anymore since Gitlab changed their tag 
ordering. You can use direct API access:”

version=4
opts=\
searchmode=plain,\
   https://gitlab.com/api/v4/projects/PROJECTID/releases 
archive/v?\d[\d.]+//-v?
([\d.]+)@ARCHIVE_EXT@

Looks like you aren’t the only one with this problem.  ;)

On Tuesday, March 26, 2024 1:05:10 PM MST Sergio Durigan Junior wrote:
> On Monday, March 25 2024, Shriram Ravindranathan wrote:
> > Dear Mentors,
> > 
> > I noticed from the past couple of days, uscan seems to be having trouble
> > finding files from the gitlab tags page.
> > 
> > ```
> > $ uscan
> > uscan warn: In debian/watch no matching files for watch line
> > 
> >   https://gitlab.com/saalen/highlight/tags?sort=updated_desc
> >   .*/archive/(\d\S+)/.*\.tar\.gz.*> 
> > ```
> > 
> > Checking the same pattern with grep shows that it does find a match
> > 
> > ```
> > $ curl -L "https://gitlab.com/saalen/highlight/tags?sort=updated_desc; |
> > grep -E ".*/archive/([0-9]\S+)/.*\.tar\.gz.*"> 
> >   % Total% Received % Xferd  Average Speed   TimeTime Time 
> >   Current
> >   
> >  Dload  Upload   Total   SpentLeft 
> >  Speed
> > 
> > 100   126  100   1260 0287  0 --:--:-- --:--:-- --:--:--  
> > 287> 
> >   0 00 00 0  0  0 --:--:--  0:00:01 --:--:--
> >   0 >   data-download-artifacts="[]"
> >   data-download-links="[{text:zip,path:
> >   /saalen/highlight/-/archive/4.11/highlight-4.11.zip},{te
> >   xt:tar.gz,path:/saalen/highlight/-/arc
> >   hive/4.11/highlight-4.11.tar.gz},{text:tar.bz2
> >   t;,path:/saalen/highlight/-/archive/4.11/highlight-4.11.
> >   tar.bz2},{text:tar,path:/s
> >   aalen/highlight/-/archive/4.11/highlight-4.11.tar}]">> 
> > 100 85448  100 854480 0  51822  0  0:00:01  0:00:01 --:--:-- 
> > 401k ```
> > 
> > It seems like a few other packages are also having similar troubles with
> > uscan, for example lomiri 
> > 
> > Is there a way to fix this?
> 

signature.asc
Description: This is a digitally signed message part.

Re: Uscan no longer works with GitLab tags

[Sergio Durigan Junior]

On Monday, March 25 2024, Shriram Ravindranathan wrote:

> Dear Mentors,
>
> I noticed from the past couple of days, uscan seems to be having trouble 
> finding files from the gitlab tags page.
>
> ```
> $ uscan
> uscan warn: In debian/watch no matching files for watch line
>   https://gitlab.com/saalen/highlight/tags?sort=updated_desc 
> .*/archive/(\d\S+)/.*\.tar\.gz.*
> ```
>
> Checking the same pattern with grep shows that it does find a match
>
> ```
> $ curl -L "https://gitlab.com/saalen/highlight/tags?sort=updated_desc; | grep 
> -E ".*/archive/([0-9]\S+)/.*\.tar\.gz.*"
>   % Total% Received % Xferd  Average Speed   TimeTime Time  
> Current
>  Dload  Upload   Total   SpentLeft  Speed
> 100   126  100   1260 0287  0 --:--:-- --:--:-- --:--:--   287
>   0 00 00 0  0  0 --:--:--  0:00:01 --:--:-- 
> 0 data-download-artifacts="[]" 
> data-download-links="[{text:zip,path:/saalen/highlight/-/archive/4.11/highlight-4.11.zip},{text:tar.gz,path:/saalen/highlight/-/archive/4.11/highlight-4.11.tar.gz},{text:tar.bz2,path:/saalen/highlight/-/archive/4.11/highlight-4.11.tar.bz2},{text:tar,path:/saalen/highlight/-/archive/4.11/highlight-4.11.tar}]">
> 100 85448  100 854480 0  51822  0  0:00:01  0:00:01 --:--:--  401k
> ```
>
> It seems like a few other packages are also having similar troubles with 
> uscan, for example lomiri 
>
> Is there a way to fix this?

I don't know if there's an "official" way to fix this, but the problem
seems to be the fact the actual download link is embedded inside an
element of the page, like this:

  https://sergiodj.net/


signature.asc
Description: PGP signature

Re: Uscan no longer works with GitLab tags

[Andrey Rakhmatullin]

On Mon, Mar 25, 2024 at 11:57:19AM +0530, Shriram Ravindranathan wrote:
> Dear Mentors,
> 
> I noticed from the past couple of days, uscan seems to be having trouble 
> finding files from the gitlab tags page.
> 
> ```
> $ uscan
> uscan warn: In debian/watch no matching files for watch line
>   https://gitlab.com/saalen/highlight/tags?sort=updated_desc 
> .*/archive/(\d\S+)/.*\.tar\.gz.*
> ```
> 
> Checking the same pattern with grep shows that it does find a match
uscan doesn't use grep though (at least by default?), and the matches are
not links but embedded JSON.

> 100   126  100   1260 0287  0 --:--:-- --:--:-- --:--:--   287
>   0 00 00 0  0  0 --:--:--  0:00:01 --:--:-- 
> 0 data-download-artifacts="[]" 
> data-download-links="[{text:zip,path:/saalen/highlight/-/archive/4.11/highlight-4.11.zip},{text:tar.gz,path:/saalen/highlight/-/archive/4.11/highlight-4.11.tar.gz},{text:tar.bz2,path:/saalen/highlight/-/archive/4.11/highlight-4.11.tar.bz2},{text:tar,path:/saalen/highlight/-/archive/4.11/highlight-4.11.tar}]">
> 100 85448  100 854480 0  51822  0  0:00:01  0:00:01 --:--:--  401k


-- 
WBR, wRAR


signature.asc
Description: PGP signature

Re: Uscan no longer works with GitLab tags

[Soren Stoutner]

Shriram,

I am neither a uscan nor a regex expert, but when I was trying to troubleshoot 
problems with uscan for the upcoming fonts-adobe-sanssource3 package I found 
`uscan -vv` to be very helpful as it displays on the screen how it is 
attempting to process the page it is scraping.

On Sunday, March 24, 2024 11:27:19 PM MST Shriram Ravindranathan wrote:
> Dear Mentors,
> 
> I noticed from the past couple of days, uscan seems to be having trouble
> finding files from the gitlab tags page.
 
> ```
> $ uscan
> uscan warn: In debian/watch no matching files for watch line
>https://gitlab.com/saalen/highlight/tags?sort=updated_desc
> .*/archive/(\d\S+)/.*\.tar\.gz.*
 ```
> 
> Checking the same pattern with grep shows that it does find a match
> 
> ```
> $ curl -L "https://gitlab.com/saalen/highlight/tags?sort=updated_desc; | 
grep
> -E ".*/archive/([0-9]\S+)/.*\.tar\.gz.*"
 % Total% Received % Xferd 
> Average Speed   TimeTime Time  Current Dload  Upload   Total   Spent 
>   Left  Speed 100   126  100   1260 0287  0 --:--:-- 
--:--:--
> --:--:--   287 0 00 00 0  0  0 --:--:--  0:00:01
> --:--:-- 0 data-download-artifacts="[]"
> data-download-
links="[{text:zip,path:
> t;/saalen/highlight/-/archive/4.11/highlight-4.11.zip},
{text
> :tar.gz,path:/saalen/highlight/-/archive/4.11/
hi
> ghlight-4.11.tar.gz},
{text:tar.bz2,path
> ot;:/saalen/highlight/-/archive/4.11/highlight-4.11.tar.bz2},
{
> t;text:tar,path:/saalen/highlight/-/
archiv
> e/4.11/highlight-4.11.tar}]"> 100 85448  100 854480 0 
> 51822  0  0:00:01  0:00:01 --:--:--  401k ```
> 
> It seems like a few other packages are also having similar troubles with
> uscan, for example lomiri 
 
> Is there a way to fix this?
> 
> Regards,
> 
> -- 
> Shriram Ravindranathan
> 


-- 
Soren Stoutner
so...@debian.org

signature.asc
Description: This is a digitally signed message part.

Uscan no longer works with GitLab tags

[Shriram Ravindranathan]

Dear Mentors,

I noticed from the past couple of days, uscan seems to be having trouble 
finding files from the gitlab tags page.

```
$ uscan
uscan warn: In debian/watch no matching files for watch line
  https://gitlab.com/saalen/highlight/tags?sort=updated_desc 
.*/archive/(\d\S+)/.*\.tar\.gz.*
```

Checking the same pattern with grep shows that it does find a match

```
$ curl -L "https://gitlab.com/saalen/highlight/tags?sort=updated_desc; | grep -E 
".*/archive/([0-9]\S+)/.*\.tar\.gz.*"
  % Total% Received % Xferd  Average Speed   TimeTime Time  Current
 Dload  Upload   Total   SpentLeft  Speed
100   126  100   1260 0287  0 --:--:-- --:--:-- --:--:--   287
  0 00 00 0  0  0 --:--:--  0:00:01 --:--:-- 0
100 85448  100 854480 0  51822  0  0:00:01  0:00:01 --:--:--  401k
```

It seems like a few other packages are also having similar troubles with uscan, for 
example lomiri 

Is there a way to fix this?

Regards,

--
Shriram Ravindranathan



OpenPGP_signature.asc
Description: OpenPGP digital signature