Re: Numpy migration?

[Mattia Rizzolo]

On Sat, Jan 05, 2019 at 08:30:28PM +0100, Ole Streicher wrote:
> >> Python-astropy is however going to be removed completely; it has
> >> however some cruft rdeps left in unstable. So, it cannot removed from
> >> unstable now, and therefore still remains in testing and
> >> (unnecessarily) blocks the numpy migration.  Python-astropy already
> >> has an RC bug, but its autoremoval from testing is still not even
> >> announced yet.
> >
> > Maybe you could ask the release team to hasten the removal of
> > python-astropy and rdeps from testing?  If the plan is to not relase
> > it in buster I don't see a reason to keep it further.
> 
> Couldn't we just also add a "breaks" to numpy? The important fact here
> is, that the new numpy and the current python-astropy don't work
> together; and this is independent of whether a fixed python-astropy
> version exists.

That wouldn't quite work, as without a working version of python-astropy
it would unilateraly block the migration of numpy as long as
python-astropy is in testing.

> This would remove one dependent party (release team) from the chain of
> blocking causes for the migration.

Given your email on -mentors a few minutes ago I see there are troubles
on removing python-astropy from unstable (I'll reply to that in a bit),
but for testing is quite easy.
Just file a bug against release.debian.org asking to remove it from
testing, and one against python-astropy at RC-level with "not release
with buster", and it will be out of testing beofore you even notice.

> >> The migration blocking CI tests seem to cause much more headaches than
> >> just "fix your bugs"...
> >
> > Well, from what I see, it has helped a lot in this half a year detect a
> > ton of regressions that would have otherwise bothered several people in
> > a later moment…
> 
> I usually regularly look into my packages and file bugs if a CI test
> fails caused by a buggy updated dependency. This works well without the
> need of blocks. It would also work, when a failing CI test would
> automatically trigger a bug report against the updated package, which
> then could be re-assigned to the rdep in case the problem was there.

There is elbrus that does that regularly (Except these last few weeks
when it was on vaction).  I think he said complete automation would miss
too many things that he is filtering out by hand for now.

-- 
regards,
Mattia Rizzolo

GPG Key: 66AE 2B4A FCCF 3F52 DA18  4D18 4B04 3FCD B944 4540  .''`.
more about me:  https://mapreri.org : :'  :
Launchpad user: https://launchpad.net/~mapreri  `. `'`
Debian QA page: https://qa.debian.org/developer.php?login=mattia  `-


signature.asc
Description: PGP signature

Re: Numpy migration?

[Scott Kitterman]

On Saturday, January 05, 2019 08:30:28 PM Ole Streicher wrote:
> Mattia Rizzolo  writes:
> > On Sat, Jan 05, 2019 at 04:02:35PM +0100, Ole Streicher wrote:
> >> I'll do tonight. It however looks a bit suboptimal: when the CI test
> >> with a new version fails for an old reverse dependency, then the new
> >> version obviously breaks that old package. So, the breakage could be
> >> detected (and taken into account) automatically. Why do we need a manual
> >> action then?
> > 
> > Let me try to suggest you see from another side: the goal of the Breaks
> > field is to prevent a given combination of known broken packages to be
> > installed.
> > Currently autopkgtest blocks the migration of numpy, but if there wasn't
> > that block a Breaks should still be added, as the astropy in testing is
> > not compatible with the new numpy.  The fact that it hints britney to
> > trigger the right tests is just a "side effects", the Breaks should be
> > added nonetheless, theoretically; in practice, we rarely did it before
> > autopkgtest started blocking the testing migration.
> 
> I agree (with the explanation in your other mail) here; it just states
> the fact that they really don't go together very well (at least in
> situation similar to the specific CI test).

For what it's worth, I hit (AIUI) the same situation when I updated python-
bleach.  It broke python-readme-renderer.  I reuploaded python-bleach with the 
breaks and a new version of python-readme-renderer (which worked with the new 
bleach) and both migrated two days later (since they both had autopkgtests).

Scott K

Re: Numpy migration?

[Ole Streicher]

Mattia Rizzolo  writes:

> On Sat, Jan 05, 2019 at 04:02:35PM +0100, Ole Streicher wrote:
>> I'll do tonight. It however looks a bit suboptimal: when the CI test
>> with a new version fails for an old reverse dependency, then the new
>> version obviously breaks that old package. So, the breakage could be
>> detected (and taken into account) automatically. Why do we need a manual
>> action then?
>
> Let me try to suggest you see from another side: the goal of the Breaks
> field is to prevent a given combination of known broken packages to be
> installed.
> Currently autopkgtest blocks the migration of numpy, but if there wasn't
> that block a Breaks should still be added, as the astropy in testing is
> not compatible with the new numpy.  The fact that it hints britney to
> trigger the right tests is just a "side effects", the Breaks should be
> added nonetheless, theoretically; in practice, we rarely did it before
> autopkgtest started blocking the testing migration.

I agree (with the explanation in your other mail) here; it just states
the fact that they really don't go together very well (at least in
situation similar to the specific CI test).

>> Python-astropy is however going to be removed completely; it has
>> however some cruft rdeps left in unstable. So, it cannot removed from
>> unstable now, and therefore still remains in testing and
>> (unnecessarily) blocks the numpy migration.  Python-astropy already
>> has an RC bug, but its autoremoval from testing is still not even
>> announced yet.
>
> Maybe you could ask the release team to hasten the removal of
> python-astropy and rdeps from testing?  If the plan is to not relase
> it in buster I don't see a reason to keep it further.

Couldn't we just also add a "breaks" to numpy? The important fact here
is, that the new numpy and the current python-astropy don't work
together; and this is independent of whether a fixed python-astropy
version exists.

This would remove one dependent party (release team) from the chain of
blocking causes for the migration.

>> The migration blocking CI tests seem to cause much more headaches than
>> just "fix your bugs"...
>
> Well, from what I see, it has helped a lot in this half a year detect a
> ton of regressions that would have otherwise bothered several people in
> a later moment…

I usually regularly look into my packages and file bugs if a CI test
fails caused by a buggy updated dependency. This works well without the
need of blocks. It would also work, when a failing CI test would
automatically trigger a bug report against the updated package, which
then could be re-assigned to the rdep in case the problem was there.

So, I value very much the CI tests by themself: they are the greatest
invention since sliced bread, but I still dislike the inflexible
handling of blocks. Often, a failing CI test makes a package only buggy
with respect to a certain feature, so it would correspond to a
"important" or "normal" bug severity. They are however always handled
like a "serious" bug by preventing migration. I miss the "That was not a
serious failure" button.

Thanks fur your help!

Best

Ole

Re: Numpy migration?

[Mattia Rizzolo]

On Sat, Jan 05, 2019 at 04:15:04PM +0100, Ole Streicher wrote:
> There is one more problem, which are transitional dependencies:
> 
> The new python3-numpy version breaks (f.e.) python3-pyregion because of
> the problem in python3-astropy. The new upload of python3-astropy fixes
> this, so in unstable everything is OK. Even when one now adds a
> 
> Breaks: python3-astropy (<< fixed-version)
> 
> to python-numpy, it will not migrate due to the test failure of
> python3-pyregion in testing. Adding a "Breaks: python3-pyregion" is
> however wrong, since pyregion is perfect for the new numpy version.

If I understood correctly, pyregion is perfectly fine and needs no
action.  In that case, it's not a problem: with the "Breaks:
python3-astropy (<< fixed-ver)" in place, the pyregion test would
install the fixed version of python3-astropy from unstable as the broken
version from testing would not be installable using the numpy from
unstable.

> So, how to handle this?

No action needed once both astropy and numpy are "fixed". :)

-- 
regards,
Mattia Rizzolo

GPG Key: 66AE 2B4A FCCF 3F52 DA18  4D18 4B04 3FCD B944 4540  .''`.
more about me:  https://mapreri.org : :'  :
Launchpad user: https://launchpad.net/~mapreri  `. `'`
Debian QA page: https://qa.debian.org/developer.php?login=mattia  `-


signature.asc
Description: PGP signature

Re: Numpy migration?

[Mattia Rizzolo]

On Sat, Jan 05, 2019 at 04:02:35PM +0100, Ole Streicher wrote:
> I'll do tonight. It however looks a bit suboptimal: when the CI test
> with a new version fails for an old reverse dependency, then the new
> version obviously breaks that old package. So, the breakage could be
> detected (and taken into account) automatically. Why do we need a manual
> action then?

Let me try to suggest you see from another side: the goal of the Breaks
field is to prevent a given combination of known broken packages to be
installed.
Currently autopkgtest blocks the migration of numpy, but if there wasn't
that block a Breaks should still be added, as the astropy in testing is
not compatible with the new numpy.  The fact that it hints britney to
trigger the right tests is just a "side effects", the Breaks should be
added nonetheless, theoretically; in practice, we rarely did it before
autopkgtest started blocking the testing migration.

> >> Another CI problem is python-astropy, which is the Python 2 version of
> >> Astropy. python-astropy is going to be removed as soon as there are no
> >> backward dependencies left; however there is still some cruft in
> >> unstable that depends on python-astropy. But this should not hinder
> >> numpy to migrate.
> >
> > I don't understand this, I don't see any python2-related issue right
> > now.  Could you please expand?
> 
> The new python-numpy breaks python-astropy in testing.

Oh, sorry.  I didn't realize python-astropy is a different source
package…

> Python-astropy is
> however going to be removed completely; it has however some cruft rdeps
> left in unstable. So, it cannot removed from unstable now, and therefore
> still remains in testing and (unnecessarily) blocks the numpy migration.
> Python-astropy already has an RC bug, but its autoremoval from testing
> is still not even announced yet.

Maybe you could ask the release team to hasten the removal of
python-astropy and rdeps from testing?  If the plan is to not relase it
in buster I don't see a reason to keep it further.

> The migration blocking CI tests seem to cause much more headaches than
> just "fix your bugs"...

Well, from what I see, it has helped a lot in this half a year detect a
ton of regressions that would have otherwise bothered several people in
a later moment…

For sure it's not just "fix your bugs" but more like "fix your bugs AND
all your rdeps", in fact you'd expect the maintainer of the package
breaking everything to be a tad more proactive, but in many cases like
this, it's not.

-- 
regards,
Mattia Rizzolo

GPG Key: 66AE 2B4A FCCF 3F52 DA18  4D18 4B04 3FCD B944 4540  .''`.
more about me:  https://mapreri.org : :'  :
Launchpad user: https://launchpad.net/~mapreri  `. `'`
Debian QA page: https://qa.debian.org/developer.php?login=mattia  `-


signature.asc
Description: PGP signature

Re: Numpy migration?

[Ole Streicher]

Mattia Rizzolo  writes:
> The way forward in cases like these is for the package that originally
> cuased the breakage (i.e. numpy) to declare a versioned Breaks on the
> borken and now fixed package (i.e. astropy (<< 3.1-1)). This way
> britney and debci will know they have to test numpy and astropy
> together, and will be able to correctly migrate to testing at the same
> time, and properly avoid a situation when two incompatible packages
> are installed.
> Maybe you could open a bug on numpy to get the maintainer to add the breaks.

There is one more problem, which are transitional dependencies:

The new python3-numpy version breaks (f.e.) python3-pyregion because of
the problem in python3-astropy. The new upload of python3-astropy fixes
this, so in unstable everything is OK. Even when one now adds a

Breaks: python3-astropy (<< fixed-version)

to python-numpy, it will not migrate due to the test failure of
python3-pyregion in testing. Adding a "Breaks: python3-pyregion" is
however wrong, since pyregion is perfect for the new numpy version.

So, how to handle this?

Cheers

Ole

Request to update the gnukhata-core new version

[Manas Kashyap]

Hola ,
I have updated the Gnukhata-core
 version to
5.50 and it has passed lintian and sbuild and is ready to be uploaded.
I would request mentors to please look into this and test it and update as
needed.
Thank you so much
Manas Kashyap 

Re: Numpy migration?

[Ole Streicher]

Mattia Rizzolo  writes:
> The way forward in cases like these is for the package that originally
> cuased the breakage (i.e. numpy) to declare a versioned Breaks on the
> borken and now fixed package (i.e. astropy (<< 3.1-1)).  This way
> britney and debci will know they have to test numpy and astropy
> together, and will be able to correctly migrate to testing at the same
> time, and properly avoid a situation when two incompatible packages
> are installed.
> Maybe you could open a bug on numpy to get the maintainer to add the
> breaks.

I'll do tonight. It however looks a bit suboptimal: when the CI test
with a new version fails for an old reverse dependency, then the new
version obviously breaks that old package. So, the breakage could be
detected (and taken into account) automatically. Why do we need a manual
action then?

>> Another CI problem is python-astropy, which is the Python 2 version of
>> Astropy. python-astropy is going to be removed as soon as there are no
>> backward dependencies left; however there is still some cruft in
>> unstable that depends on python-astropy. But this should not hinder
>> numpy to migrate.
>
> I don't understand this, I don't see any python2-related issue right
> now.  Could you please expand?

The new python-numpy breaks python-astropy in testing. Python-astropy is
however going to be removed completely; it has however some cruft rdeps
left in unstable. So, it cannot removed from unstable now, and therefore
still remains in testing and (unnecessarily) blocks the numpy migration.
Python-astropy already has an RC bug, but its autoremoval from testing
is still not even announced yet.

The migration blocking CI tests seem to cause much more headaches than
just "fix your bugs"...

Best

Ole

Re: Numpy migration?

[Mattia Rizzolo]

)On Sat, Jan 5, 2019 at 3:18 PM Ole Streicher  wrote:
> However, astropy cannot migrate now, to testing, since it depends on the
> new numpy version (and therefore can only migrate after numpy). And
> numpy is blocked by the CI failure of astropy ...
>
> Looks like a deadlock. Which will be resolved before the migration delay
> ends. Which is in the second half of february, and therefore will cause
> all packages that depend on numpy and are not in testing yet to be kept
> outside of Buster (due to the release timeline).

Starting from half a month ago CI regressions are de-facto release blockers.

The way forward in cases like these is for the package that originally
cuased the breakage (i.e. numpy) to declare a versioned Breaks on the
borken and now fixed package (i.e. astropy (<< 3.1-1)).  This way
britney and debci will know they have to test numpy and astropy
together, and will be able to correctly migrate to testing at the same
time, and properly avoid a situation when two incompatible packages
are installed.
Maybe you could open a bug on numpy to get the maintainer to add the breaks.

BTW, that numpy upload also blocked our effort to remove Python 3.6
from buster...

> Another CI problem is python-astropy, which is the Python 2 version of
> Astropy. python-astropy is going to be removed as soon as there are no
> backward dependencies left; however there is still some cruft in
> unstable that depends on python-astropy. But this should not hinder
> numpy to migrate.

I don't understand this, I don't see any python2-related issue right
now.  Could you please expand?

-- 
regards,
Mattia Rizzolo

GPG Key: 4096R/B9444540 http://goo.gl/I8TMB
more about me: http://mapreri.org
Launchpad User: https://launchpad.net/~mapreri
Ubuntu Wiki page: https://wiki.ubuntu.com/MattiaRizzolo

Numpy migration?

[Ole Streicher]

Hi all,

Since a few days, a new RC is stalled in unstable:

 testing: 1:1.15.4-2
unstable: 1:1.16.0~rc1-3

The reason that it can't migrate is that it breaks the CI tests of
several packages in testing. For example astropy (where I am the
uploader), and its dependencies. The CI test failures were caused by
astropy (#917466) and are fixed since a week in unstable.

However, astropy cannot migrate now, to testing, since it depends on the
new numpy version (and therefore can only migrate after numpy). And
numpy is blocked by the CI failure of astropy ...

Looks like a deadlock. Which will be resolved before the migration delay
ends. Which is in the second half of february, and therefore will cause
all packages that depend on numpy and are not in testing yet to be kept
outside of Buster (due to the release timeline).

What is the way to resolve those problems? These deadlocks probably
appear regularly when a package gets a major update: package A breaks
the CI test of the rdep B in testing, and a rebuild of B in unstable
depends on the newer version of A.

Another CI problem is python-astropy, which is the Python 2 version of
Astropy. python-astropy is going to be removed as soon as there are no
backward dependencies left; however there is still some cruft in
unstable that depends on python-astropy. But this should not hinder
numpy to migrate.

How to solve this?

Best

Ole

About to upload pelican

[Geert Stappers]

Control: tag +pending

On Fri, Jan 04, 2019 at 11:50:57PM +0100, Geert Stappers wrote:
> On Sat, Jan 05, 2019 at 12:58:49AM +0300, Dmitry Shachnev wrote:
> > On Fri, Jan 04, 2019 at 10:16:20PM +0100, Geert Stappers wrote:
> > >
> > > I would like to create 
> > > https://salsa.debian.org/python-team/applications/pelican
> > > or would like to have that git repo created and have write privilege to 
> > > it.
> > 
> > FWIW, that repo already exists.
> 
> What a nice surprise
> 

It even has the correct VCS fields.

So the thing to do is uploading.

Because it will enable `debcheckout pelican`  without errors.


Cheers
Geert Stappers