Bug#1000984: ITP: highway -- Efficient and performance-portable SIMD wrapper

[Mathieu Malaterre]

Package: wnpp
Severity: wishlist
Owner: Mathieu Malaterre 
X-Debbugs-Cc: debian-devel@lists.debian.org

* Package name: highway
  Version : 0.15.0
  Upstream Author : JPEG XL Maintainers 
* URL : https://github.com/google/highway/
* License : Apache-2.0
  Programming Lang: C++
  Description : Efficient and performance-portable SIMD wrapper

 This library provides type-safe and source-code portable wrappers over
 existing platform-specific intrinsics. Its design aims for simplicity,
 reliable efficiency across platforms, and immediate usability with current
 compilers.

Answers:

* This is a dependency of jpeg-xl (currently under review).
* This will be team maintained (phototools)

Re: uscan roadmap

[Paul Wise]

On Wed, 2021-12-01 at 09:11 +0100, Yadd wrote:

> after few discussions with some devscripts maintainers, we decided to
> build a new "version=5" format for debian/watch.

It might be a idea to look at how other distributions do checking for
new upstream releases and adopt some of their improvements.

I note Fedora uses a service (that isn't Fedora specific) for this:

https://release-monitoring.org
https://docs.fedoraproject.org/en-US/package-maintainers/Upstream_Release_Monitoring/

Another idea would be to use the Repology service to notice when other
distros include a newer version of a package than Debian does. 

https://repology.org/

I also wonder if it is time to split debian/watch out of Debian source
packages, since upstream download locations generally change
independently of the Debian package and so information about upstream
download locations probably should be maintained independently.

Alternatively, perhaps we could workaround outdated debian/watch files
by having vcswatch extract debian/watch files from the repo specified
in the Vcs-* URLs.

-- 
bye,
pabs

https://wiki.debian.org/PaulWise


signature.asc
Description: This is a digitally signed message part

Re: uscan roadmap

[Paul Wise]

On Wed, 2021-12-01 at 12:53 +0100, Yadd wrote:

> sf.net because it needs JS interpretation

The sf.net redirector uses the RSS feed of the files.
This is documented at the top of the redirector HTML:

$ curl -s https://qa.debian.org/watch/sf.php/NSIS/ | grep -i rss
(https://sourceforge.net/projects/NSIS/rss?limit=1000";>RSS)
pages. File listing converted from the RSS.

-- 
bye,
pabs

https://wiki.debian.org/PaulWise


signature.asc
Description: This is a digitally signed message part

Re: uscan roadmap

[Paul Wise]

On Wed, 2021-12-01 at 12:53 +0100, Yadd wrote:

> Personally I dislike redirectors.

A redirector service is superior to including the redirector code
within uscan itself or within a debian/watch file, since when the
upstream website breaks the existing code, a service can be updated in
one place immediately, while uscan in Debian stable will be broken
until the next point release if it gets fixed at all and one in
debian/watch requires every package using the site to get updated.

-- 
bye,
pabs

https://wiki.debian.org/PaulWise


signature.asc
Description: This is a digitally signed message part

Re: uscan roadmap

[Yadd]

On 01/12/2021 21:07, Patrice wrote:


Really great!
And could the new uscan read a watch file from version 3/4/5 and output a
version 5 of it by its own (in-place or stdout)?
uscan --standardize
:-)


Yes but without optimization neither scheme (except some few fields). 
Example:


  version=4

  opts=uversionmangle=s/-/~/g,pgpmode=none \
   https://...  .*(\d[\d\.]*)@ARCHIVE_EXT@

will be translated into:

  Version: 5

  Source: https://...
  Regex: .*(\d[\d\.]*)@ARCHIVE_EXT@
  Uversionmangle: s/-/~/g

You'll have to manually modify it into

  Version: 5
  Scheme: stable

  Source: https://...

Re: uscan roadmap

[Patrice]

Really great!
And could the new uscan read a watch file from version 3/4/5 and output a
version 5 of it by its own (in-place or stdout)?
uscan --standardize
:-)

Wishes,
Patrice

Re: uscan roadmap

[Tomas Pospisek]

On 01.12.21 12:50, Mattia Rizzolo wrote:


Likewise, I would love if uscan could just learn how github, gitlab,
launchpad, etc are made so prople won't have to bother with sticking
urls into watchfiles, such as:

   Source: GitHub
   Source-Options:
namespace: trendmicro
project: tlsh
match-on: tags|releases


Excellent idea: +1

However at the very moment that abstractions get introduced (which is 
+1), please, please, please do keep the poor users in mind when stuff 
*does not* work. I.e. please make uscan trivially debuggable.


Something like:

$ uscan --verbose
... found GitHub source definition:
Source: GitHub
Source-Options:
  namespace: trendmicro
  project: tlsh
  match-on: tags|releases
... using that GitHub source definition to find new release under 
https://github.com/foo/bar/releases/bar-1.2.3.tar (uscan.py:line 3498)


One of the regularly recurring frustrations I am encountering is that 
I'm using some SW that has abstracted something and I know in principle 
how the thing it has abstracted works, but am completely unable to find 
out where the abstraction gets all wired up and falls on its face. (Hi 
k8s!). The famous "computer says no" moment.


OK, this is just me whining and not contributing any code at all, so 
please take it as just that: a wishlist item.

*t

Re: uscan roadmap

[Yadd]

On 01/12/2021 18:39, Thomas Goirand wrote:

Hi Yadd,

Thanks a lot for working on this. What you are proposing (ie: using a
mime thing, which is easy to parse instead of the dirty command-line
oriented thingy of version 3 and 4) feels much nicer than what we
currently have.

On 12/1/21 12:53 PM, Yadd wrote:

Fix: will be

   Version: 5

   Source: https://qa.debian.org/watch/sourceforge/
   Regex: -(.+)\.tar\.gz


That's much nicer than previous proposal!


And I don't think "uupdate" is still useful.


IMO, it is needed. That's what is nice with calling scripts: it can take
care with programming of things you didn't even think of. If you remove
it, the risk is that maintainers will continue to use version 3 or 4,
because they still need an update script.

How about:

Update-Script: uupdate

?


Looks good to me

Re: uscan roadmap

[Thomas Goirand]

Hi Yadd,

Thanks a lot for working on this. What you are proposing (ie: using a
mime thing, which is easy to parse instead of the dirty command-line
oriented thingy of version 3 and 4) feels much nicer than what we
currently have.

On 12/1/21 12:53 PM, Yadd wrote:
> Fix: will be
> 
>   Version: 5
> 
>   Source: https://qa.debian.org/watch/sourceforge/
>   Regex: -(.+)\.tar\.gz

That's much nicer than previous proposal!

> And I don't think "uupdate" is still useful.

IMO, it is needed. That's what is nice with calling scripts: it can take
care with programming of things you didn't even think of. If you remove
it, the risk is that maintainers will continue to use version 3 or 4,
because they still need an update script.

How about:

Update-Script: uupdate

?

Cheers,

Thomas Goirand (zigo)

Re: uscan roadmap

[Russ Allbery]

Yadd  writes:

> after few discussions with some devscripts maintainers, we decided to
> build a new "version=5" format for debian/watch.

> Principles:
>  * keep compatibility with versions 3 and 4, no need to change all
>debian/watch files
>  * new version 5 format using the same syntax than other debian/* files
>(rfc822 + "# comments")

I have no detailed feedback on your proposal, but just wanted to say thank
you so much for doing this.  Being able to write a uscan file using a
clear key/value syntax will make me very happy.

-- 
Russ Allbery (r...@debian.org)  

Debian Med video conference tomorrow, Thursday 2021-12-02 18:00 UTC

[Andreas Tille]

Hi,

this is the call for the next video conference of the Debian Med team
that are an established means to organise the tasks inside our team.
We do these conferences twice per month on every

   2th  and  17th

of a month.  Usually it takes us only 15-20min depending what we are
talking about and how many people are joining.  The next meeting is
tomorrow 18:00 UTC
   
 
https://www.timeanddate.com/worldclock/fixedtime.html?msg=Debian+CoViD-19+Biohackathon+Video+Conference&iso=20211202T18

The meeting is on the Debian Social channel

 https://jitsi.debian.social/DebianMedCovid19

These video meetings were started in the Debian Med Biohackathon.
The topic is what contributors have done in the past period and to
coordinate the work until the next meeting.

For those who are interested in hot topics we want to tackle, here
are some items:

  - This month we are doing our advent calendar bug squashing party
(even if its not displayed as nicely as in former years).[0]

  - Effort to package tensorflow and other ML software which is
used in several applications that are used to fight COVID-19.
The precondition bazel to package tensorflow was a direct
consequence of a Debian Med hackathon and it was even
acknowledged[1]

  - Package software that is used to fight COVID-19 which we are
listing in some spreadsheet[2].  It reaches from small tools
up to complex software packages.  There should be targets for
everybody who wants to join us.

  - General Debian Med issues not directly connected to COVID-19

Newcomers and bug squashing volunteers are always welcome.

Lets keep on the great work and see you tomorrow
 
   Andreas.

[0] https://lists.debian.org/debian-med/2021/12/msg0.html
[1] https://blog.bazel.build/2021/03/04/bazel-debian-packaging.html
[2] 
https://docs.google.com/spreadsheets/d/1tApLhVqxRZ2VOuMH_aPUgFENQJfbLlB_PFH_Ah_q7hM/edit#gid=543782716

-- 
http://fam-tille.de

Re: uscan roadmap

[Yadd]

On 01/12/2021 13:14, Jonas Smedegaard wrote:

Quoting Yadd (2021-12-01 13:04:09)

On 01/12/2021 12:50, Mattia Rizzolo wrote:

Possibly, I'm indeed kind of unimpressed that we grew a parse for
nodejs' package.json and perl's META.json.  Though I accepted it
because I saw some value, I'm totally in awe of universes where that
is actually needed..

This is very useful for package with components. This is the only way
to be able to `uscan --download-current-version`.


Speakin of components, it would be quite helpful if possible to handle
versions of components - e.g. upgrade all components except
SomeComponent like this:

   `uscan --download-current-version-SomeComponent`


Yes, maybe with a --download-compat-version-SomeComponent which will 
accept upgrades only if it is semver-compatible (no major updates)


Note that this will be in a separated MR (not related to version change)

Re: uscan roadmap

[Jonas Smedegaard]

Quoting Yadd (2021-12-01 13:04:09)
> On 01/12/2021 12:50, Mattia Rizzolo wrote:
> > Possibly, I'm indeed kind of unimpressed that we grew a parse for 
> > nodejs' package.json and perl's META.json.  Though I accepted it 
> > because I saw some value, I'm totally in awe of universes where that 
> > is actually needed..
> This is very useful for package with components. This is the only way 
> to be able to `uscan --download-current-version`.

Speakin of components, it would be quite helpful if possible to handle 
versions of components - e.g. upgrade all components except 
SomeComponent like this:

  `uscan --download-current-version-SomeComponent`


 - Jonas

-- 
 * Jonas Smedegaard - idealist & Internet-arkitekt
 * Tlf.: +45 40843136  Website: http://dr.jones.dk/

 [x] quote me freely  [ ] ask before reusing  [ ] keep private

signature.asc
Description: signature

Re: uscan roadmap

[Yadd]

On 01/12/2021 12:50, Mattia Rizzolo wrote:

On Wed, Dec 01, 2021 at 12:39:41PM +0100, Geert Stappers wrote:

Summary: unhide redirectors


And not only.


On Wed, Dec 01, 2021 at 09:11:17AM +0100, Yadd wrote:

after few discussions with some devscripts maintainers, we decided to build
a new "version=5" format for debian/watch.


To be clear, this is a *very* non-ready proposal that we are getting
through the wider community.  Nothing of this is implemented anywhere.


* URL and regex are separated
* Some default values change. For example, `dversionmangle` default
  value will be "auto" (drop +dfsg, ~ds,...), uversionmangle=s/-/~/g,
  filenamemangle=s/.*?(\d[\d\.]*@ARCHIVE_EXT@)/@PACKAGE@-$1/...


I honestly would like to add website-aware functionalities to uscan,
such as exactly this.


I think the move from v4 to v5 is an excellent opportunity
to express in the watch file that there is a dependency on a redirector.

[..]

  Version: 5
  Source: https://qa.debian.org/watch/sourceforge/ 
-(.+)\.tar\.gz debian uupdate


I would like something like:
   Source: qa-redirector
   Source-Options:
name: sourceforge
project: 


Likewise, I would love if uscan could just learn how github, gitlab,
launchpad, etc are made so prople won't have to bother with sticking
urls into watchfiles, such as:

   Source: GitHub
   Source-Options:
namespace: trendmicro
project: tlsh
match-on: tags|releases


OK, "Source" will be:
 * a keyword known by uscan DB
 * a URL for other websites

And what about default version value, "stable" (only digits), "semver" 
or "any" (== @ANY_VERSION@)



To go either matching on https://github.com/trendmicro/tlsh/tags or
https://github.com/trendmicro/tlsh/releases. using then Scheme (a name
that, tbh, I don't particularly like right now) for the tags or releases
regex.


And I think such change will allow removal of

bare
Disable all site specific special case code such as URL
redirector uses and page content alterations.

from the uscan code and uscan manual page  (they are in /usr/bin/uscan )


The goal is to have documented that there are extra components being used.
Avoiding nasty surprises.


this feels like the opposite direction I'm proposing above :D


Seconded


Awareness of redirectors will get us more redirectors.
Those redirectors will help us to prevent that `uscan`
must get a javascript interpreter.


Possibly, I'm indeed kind of unimpressed that we grew a parse for
nodejs' package.json and perl's META.json.  Though I accepted it because
I saw some value, I'm totally in awe of universes where that is actually
needed..
This is very useful for package with components. This is the only way to 
be able to `uscan --download-current-version`.

Re: uscan roadmap

[Yadd]

On 01/12/2021 12:39, Geert Stappers wrote:

Summary: unhide redirectors

On Wed, Dec 01, 2021 at 09:11:17AM +0100, Yadd wrote:

Hi,

after few discussions with some devscripts maintainers, we decided to build
a new "version=5" format for debian/watch.

Principles:
  * keep compatibility with versions 3 and 4, no need to change all
debian/watch files
  * new version 5 format using the same syntax than other debian/* files
(rfc822 + "# comments")
  * no option renaming (becomes case-insensitive to be compliant with
all formats)
  * Version 5:
* Main (first) paragraph contains "Version: 5" and optional options
  that change default values for source-paragraph
* URL and regex are separated
* Some default values change. For example, `dversionmangle` default
  value will be "auto" (drop +dfsg, ~ds,...), uversionmangle=s/-/~/g,
  filenamemangle=s/.*?(\d[\d\.]*@ARCHIVE_EXT@)/@PACKAGE@-$1/...

Example:

   Version: 5


   


Of course, comments are welcome!



I think the move from v4 to v5 is an excellent opportunity
to express in the watch file that there is a dependency on a redirector.


Example

  version=4
  https://sf.net// -(.+)\.tar\.gz debian uupdate


becomes something like

  Version: 5
  Source: https://qa.debian.org/watch/sourceforge/ 
-(.+)\.tar\.gz debian uupdate


Fix: will be

  Version: 5

  Source: https://qa.debian.org/watch/sourceforge/
  Regex: -(.+)\.tar\.gz

And I don't think "uupdate" is still useful.

Personally I dislike redirectors. For npmjs, I wrote a 
"serachMode=plain" to be able to parse npm JSON result, also because 
there are a lot of npmjs packages and redirector was often unable to 
respond. This is different of course for sf.net because it needs JS 
interpretation



And I think such change will allow removal of

bare
Disable all site specific special case code such as URL
redirector uses and page content alterations.

from the uscan code and uscan manual page  (they are in /usr/bin/uscan )


Yes, doc will be updated


The goal is to have documented that there are extra components being used.
Avoiding nasty surprises.

Groeten
Geert Stappers


Thanks!


P.S.
Awareness of redirectors will get us more redirectors.
Those redirectors will help us to prevent that `uscan`
must get a javascript interpreter

Re: uscan roadmap

[Mattia Rizzolo]

On Wed, Dec 01, 2021 at 12:39:41PM +0100, Geert Stappers wrote:
> Summary: unhide redirectors

And not only.

> On Wed, Dec 01, 2021 at 09:11:17AM +0100, Yadd wrote:
> > after few discussions with some devscripts maintainers, we decided to build
> > a new "version=5" format for debian/watch.

To be clear, this is a *very* non-ready proposal that we are getting
through the wider community.  Nothing of this is implemented anywhere.

> >* URL and regex are separated
> >* Some default values change. For example, `dversionmangle` default
> >  value will be "auto" (drop +dfsg, ~ds,...), uversionmangle=s/-/~/g,
> >  filenamemangle=s/.*?(\d[\d\.]*@ARCHIVE_EXT@)/@PACKAGE@-$1/...

I honestly would like to add website-aware functionalities to uscan,
such as exactly this.

> I think the move from v4 to v5 is an excellent opportunity
> to express in the watch file that there is a dependency on a redirector.
[..]
>  Version: 5
>  Source: https://qa.debian.org/watch/sourceforge/ 
> -(.+)\.tar\.gz debian uupdate

I would like something like:
  Source: qa-redirector
  Source-Options:
   name: sourceforge
   project: 


Likewise, I would love if uscan could just learn how github, gitlab,
launchpad, etc are made so prople won't have to bother with sticking
urls into watchfiles, such as:

  Source: GitHub
  Source-Options:
   namespace: trendmicro
   project: tlsh
   match-on: tags|releases

To go either matching on https://github.com/trendmicro/tlsh/tags or
https://github.com/trendmicro/tlsh/releases. using then Scheme (a name
that, tbh, I don't particularly like right now) for the tags or releases
regex.

> And I think such change will allow removal of
> 
>bare
>Disable all site specific special case code such as URL
>redirector uses and page content alterations.
> 
> from the uscan code and uscan manual page  (they are in /usr/bin/uscan )
> 
> 
> The goal is to have documented that there are extra components being used.
> Avoiding nasty surprises.

this feels like the opposite direction I'm proposing above :D

> Awareness of redirectors will get us more redirectors.
> Those redirectors will help us to prevent that `uscan`
> must get a javascript interpreter.

Possibly, I'm indeed kind of unimpressed that we grew a parse for
nodejs' package.json and perl's META.json.  Though I accepted it because
I saw some value, I'm totally in awe of universes where that is actually
needed..

-- 
regards,
Mattia Rizzolo

GPG Key: 66AE 2B4A FCCF 3F52 DA18  4D18 4B04 3FCD B944 4540  .''`.
More about me:  https://mapreri.org : :'  :
Launchpad user: https://launchpad.net/~mapreri  `. `'`
Debian QA page: https://qa.debian.org/developer.php?login=mattia  `-


signature.asc
Description: PGP signature

Re: uscan roadmap

[Geert Stappers]

Summary: unhide redirectors

On Wed, Dec 01, 2021 at 09:11:17AM +0100, Yadd wrote:
> Hi,
> 
> after few discussions with some devscripts maintainers, we decided to build
> a new "version=5" format for debian/watch.
> 
> Principles:
>  * keep compatibility with versions 3 and 4, no need to change all
>debian/watch files
>  * new version 5 format using the same syntax than other debian/* files
>(rfc822 + "# comments")
>  * no option renaming (becomes case-insensitive to be compliant with
>all formats)
>  * Version 5:
>* Main (first) paragraph contains "Version: 5" and optional options
>  that change default values for source-paragraph
>* URL and regex are separated
>* Some default values change. For example, `dversionmangle` default
>  value will be "auto" (drop +dfsg, ~ds,...), uversionmangle=s/-/~/g,
>  filenamemangle=s/.*?(\d[\d\.]*@ARCHIVE_EXT@)/@PACKAGE@-$1/...
> 
> Example:
> 
>   Version: 5
> 
  
> 
> Of course, comments are welcome!


I think the move from v4 to v5 is an excellent opportunity
to express in the watch file that there is a dependency on a redirector.


Example

 version=4
 https://sf.net// -(.+)\.tar\.gz debian uupdate


becomes something like

 Version: 5
 Source: https://qa.debian.org/watch/sourceforge/ 
-(.+)\.tar\.gz debian uupdate



And I think such change will allow removal of

   bare
   Disable all site specific special case code such as URL
   redirector uses and page content alterations.

from the uscan code and uscan manual page  (they are in /usr/bin/uscan )


The goal is to have documented that there are extra components being used.
Avoiding nasty surprises.




Groeten
Geert Stappers


P.S.
Awareness of redirectors will get us more redirectors.
Those redirectors will help us to prevent that `uscan`
must get a javascript interpreter.


-- 
Silence is hard to parse

Bug#1000951: ITP: ocaml-uri -- RFC3986 URI/URL parsing library in OCaml

[Stéphane Glondu]

Package: wnpp
Severity: wishlist
Owner: Stéphane Glondu 
X-Debbugs-Cc: debian-devel@lists.debian.org, debian-ocaml-ma...@lists.debian.org

* Package name: ocaml-uri
  Version : 4.2.0
  Upstream Author : David Sheets and Anil Madhavapeddy
* URL : https://github.com/mirage/ocaml-uri
* License : ISC
  Programming Lang: OCaml
  Description : RFC3986 URI/URL parsing library in OCaml

 This is an OCaml implementation of the RFC3986 specification for
 parsing URI or URLs.

This is a new dependency of ocsigenserver.

uscan roadmap

[Yadd]

Hi,

after few discussions with some devscripts maintainers, we decided to 
build a new "version=5" format for debian/watch.


Principles:
 * keep compatibility with versions 3 and 4, no need to change all
   debian/watch files
 * new version 5 format using the same syntax than other debian/* files
   (rfc822 + "# comments")
 * no option renaming (becomes case-insensitive to be compliant with
   all formats)
 * Version 5:
   * Main (first) paragraph contains "Version: 5" and optional options
 that change default values for source-paragraph
   * URL and regex are separated
   * Some default values change. For example, `dversionmangle` default
 value will be "auto" (drop +dfsg, ~ds,...), uversionmangle=s/-/~/g,
 filenamemangle=s/.*?(\d[\d\.]*@ARCHIVE_EXT@)/@PACKAGE@-$1/...

Example:

  Version: 5

  Source: https://...


Some new features to simplify debian/watch:
 * a scheme option with the following values:
   * "stable": digit-only versions (maybe default value)
   * "semver": semver versions only (see semver.org)
   * "any": version matching current @ANY_VERSION@ macro

Example:

  Version: 5
  Scheme: stable

  Source: https://...

  Component: foo
  Source: https://...

  Component: bar
  Source: https://...
  Scheme: semver

Then a current GitHub changes from:

  version=4
  opts=\
  dversionmangle=auto,\
  filenamemangle=s/.*?(\d[\d\.]*@ARCHIVE_EXT@/@PACKAGE@-$1/ \
https://github.com/foo/bar/tags .*/archive/.*/v?([\d\.]+).tar.gz

to:

  Version: 5
  Scheme: stable

  Source: https://github.com/foo/bar/tags

Then to include rc/beta releases:

  Version: 5
  Scheme: semver

  Source: https://github.com/foo/bar/tags

Version 5 adds also new macros (used by Scheme of course):
 * @STABLE_VERSION@
 * @SEMVER@

Uscan API will also provide a "write" function usable by a provided 
script to convert old formats to new.


Of course, comments are welcome!

Cheers,
Yadd