Re: Bug#891188: blends-dev: created d/control recommends packages not in main
Hi Ole, On Sat, Mar 24, 2018 at 09:28:32PM +0100, Ole Streicher wrote: > I just uploaded 0.6.101 to experimental (code is in the "experimental > branch); could everyone please have a look. UDD is not implemented yet. [..] > Debian Astro and Debian Edu (both without the Depends->Recommends > lowering) work, when the line > > Format: https://blends.debian.org/blends/1.1 > > was inserted as the first line in each task with > > $ sed -i '1i Format: https://blends.debian.org/blends/1.1' tasks/* > > In Debian Edu, the generated dependcy lists differ significantly from > what is checked in, but I have no idea why this is. Looking into the > tasks lists, they look fine. The education-* Buster meta-packages differ a lot from the Stretch ones. - A lot of superfluous Depends. (Only one additionally needed in Jessie: gosa-plugin-netgroups via some hackish trick.) - No evaluation of the missinglist packages. (Packages kept as recommended dispite beeing unavailable.) Just for testing I reworked the Debian Edu tasks files (basically s/Depends/Recommends/g). Using these files, this seem to work: - Using added (forced) depends. - Dropping the education-all package. - Correctly de-promoting non-available packages from Recommends to Suggests. - Correctly de-promoting contrib and non-free packages (only added for testing purposes) from Recommends to Suggests. I don't have more time atm (and the results should be confirmed by others). Thanks for your work. Wolfgang signature.asc Description: PGP signature
Re: Bug#891188: blends-dev: created d/control recommends packages not in main
Andreas Tille writes: > Please let me know if something is ready for testing. I just uploaded 0.6.101 to experimental (code is in the "experimental branch); could everyone please have a look. UDD is not implemented yet. I played with it on Debian Astro (ofcourse), Debian Science and Debian Edu. Debian Astro and Debian Edu (both without the Depends->Recommends lowering) work, when the line Format: https://blends.debian.org/blends/1.1 was inserted as the first line in each task with $ sed -i '1i Format: https://blends.debian.org/blends/1.1' tasks/* In Debian Edu, the generated dependcy lists differ significantly from what is checked in, but I have no idea why this is. Looking into the tasks lists, they look fine. Debian Science looks well; this is a blend where the dependency status should be lowered. BTW, blends-gen-control now has an option "-U" that updates the format of all tasks to the current version (and removes the backslashes at the end of the line). Feedback is very appreciated. Best regards Ole
Processed: Re: Bug#891188: blends-dev: created d/control recommends packages not in main
Processing control commands: > affects -1 src:debian-astro Bug #891188 [blends-dev] blends-dev: created d/control recommends packages not in main Added indication that 891188 affects src:debian-astro -- 891188: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=891188 Debian Bug Tracking System Contact ow...@bugs.debian.org with problems
Processed: Re: Bug#891188: blends-dev: created d/control recommends packages not in main
Processing control commands: > severity -1 important Bug #891188 [blends-dev] blends-dev: created d/control recommends packages not in main Severity set to 'important' from 'serious' -- 891188: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=891188 Debian Bug Tracking System Contact ow...@bugs.debian.org with problems
Re: Bug#891188: blends-dev: created d/control recommends packages not in main
Hi Ole,
On Fri, Mar 16, 2018 at 03:07:46PM +0100, Ole Streicher wrote:
> >
> > I can only repeat: Blends-dev *was* rewritten and creates correct
> > results:
> >
> > https://salsa.debian.org/blends-team/blends-gsoc
>
> I was not aware that the rewrite was done in Python.
I made probably even to less noise about it - despite I somehow felt to
report about it every three monthes.
> Anyway: That one is not much better. It is in Python, OK, but it is not
> more modular, and it implements everything by itself instead of re-using
> standard packages (namely debian.deb822 and apt). What I want to have is
> something like
>
> from debian.blends import Task
> t = Task('tasks/education')
>
> and t then "looks like" (duck-typing) the apt (versionized) package that
> one would get with
>
> import apt
> c = apt.Cache()
> t = c['astro-education'].version[0]
>
> so that "t.recommends" gets apt.package.Dependency-like dependencies etc.
Fair enough.
> In principle, one could do the same by generating the Task from UDD.
> But, I don't see why calling UDD instead of using the local tasks
> definition has an advantage for writing d/control. UDD makes it
> difficult for non-Debian blends/distributions: how f.e. would Mint Linux
> get it? Or how would a blend that is not in the UDD get the information
> about its metadata?
The point is: UDD has the information about all *architectures* and you
would need to parse a full matrix of Packages files to reach the same
effect. We have all information fully gathered in UDD.
> > I repeat again: This creates *architecture* dependent metapackages (I
> > explained here why only this makes sense).
>
> Could you point me to your explanation please? I missed it.
Since some time I have the impression that site searches on Debian
lists do not work as before. I'm using notmuch on my local mail
archive to uncover the links (in reverse time order):
1. https://lists.debian.org/debian-blends/2018/01/msg8.html
2. https://lists.debian.org/debian-blends/2017/08/msg00020.html
3. just a mention that rewrite should be done not worth reading
https://lists.debian.org/debian-blends/2016/10/msg3.html
4. https://lists.debian.org/debian-blends/2016/05/msg00049.html
In 4. its a direct reply to a mail of yours (you in CC ;-) )
In short: Our metapackages claim to be Architecture: all but they are
only granted to work on amd64 and are probably mostly all broken on any
other architecture.
> > I'm all for switching to
> > Python3 but a complete rewrite should not be necessary. There are some
> > glithes in the code (for instance line breaking in the auto generated
> > part of d/changelog) and it definitely needs testing.
>
> I would re-use parts of it, but I am a bit resistant of "just finishing"
> it. And the word "changelog" does not appear in its blends-gen-control?
Feel free to try a make dist. If you do not have a local UDD clone I'd
volunteer to redirect the code to the public mirror. It generates
automatic changelog entries - feel free to check debian/changelog of
debian-med and watch out for "start of automatic changelog entry".
> > I *really* appreciate your effort but may be 80% of the work is just
> > done and you try blends-gsoc first. As we all know the last 20% is
> > the harder part - but I urgently vote for using UDD as base for
> > anything we use. May me using the public UDD mirror can help to not
> > force everybody who wants to run blends-dev having a local clone.
>
> I will have a deeper look into the package. However, for package
> creation UDD looks to me the wrong tool (see above).
I see no sensible way around UDD - but feel free to prove me wrong. I'm
open to anything that will enable us to get rid of the old Perl code and
creates architecture dependant metapackages.
Kind regards
Andreas.
--
http://fam-tille.de
Re: Bug#891188: blends-dev: created d/control recommends packages not in main
Hi Andreas
(completely moving to d/blends, since this is not bug related)
On 16.03.2018 14:17, Andreas Tille wrote:
> On Fri, Mar 16, 2018 at 01:51:25PM +0100, Ole Streicher wrote:
>> In the moment, I would tend to rewrite blend-gen-control from scratch,
>> using Python 3 and the standard Debian Python packages (debian.deb822,
>> apt) in a modular fashion. This would make the script much more
>> maintainable (given the knowledge of Python is a bit better than Perl),
>> and could also lead to create a "debian.blends" Python package that
>> could be re-used for the Web pages.
>
> I can only repeat: Blends-dev *was* rewritten and creates correct
> results:
>
> https://salsa.debian.org/blends-team/blends-gsoc
I was not aware that the rewrite was done in Python.
Anyway: That one is not much better. It is in Python, OK, but it is not
more modular, and it implements everything by itself instead of re-using
standard packages (namely debian.deb822 and apt). What I want to have is
something like
from debian.blends import Task
t = Task('tasks/education')
and t then "looks like" (duck-typing) the apt (versionized) package that
one would get with
import apt
c = apt.Cache()
t = c['astro-education'].version[0]
so that "t.recommends" gets apt.package.Dependency-like dependencies etc.
In principle, one could do the same by generating the Task from UDD.
But, I don't see why calling UDD instead of using the local tasks
definition has an advantage for writing d/control. UDD makes it
difficult for non-Debian blends/distributions: how f.e. would Mint Linux
get it? Or how would a blend that is not in the UDD get the information
about its metadata?
> I repeat again: This creates *architecture* dependent metapackages (I
> explained here why only this makes sense).
Could you point me to your explanation please? I missed it.
> I'm all for switching to
> Python3 but a complete rewrite should not be necessary. There are some
> glithes in the code (for instance line breaking in the auto generated
> part of d/changelog) and it definitely needs testing.
I would re-use parts of it, but I am a bit resistant of "just finishing"
it. And the word "changelog" does not appear in its blends-gen-control?
> I *really* appreciate your effort but may be 80% of the work is just
> done and you try blends-gsoc first. As we all know the last 20% is
> the harder part - but I urgently vote for using UDD as base for
> anything we use. May me using the public UDD mirror can help to not
> force everybody who wants to run blends-dev having a local clone.
I will have a deeper look into the package. However, for package
creation UDD looks to me the wrong tool (see above).
Best regards
Ole
Re: Bug#891188: blends-dev: created d/control recommends packages not in main
Hi Ole,
On Fri, Mar 16, 2018 at 01:51:25PM +0100, Ole Streicher wrote:
> On 15.03.2018 19:41, Andreas Tille wrote:
> > I've commited a fix to Git which for me creates now sensible Debian
> > Astro metapackages.
>
> For me, it actually upgrades the "Recommends" with to "Depends". For
> example, the "education" task (shortened):
>
> ```
> Task: Education
> Install: true
> Description: Educational astronomy applications [...]
>
> Recommends: celestia-gnome, gpredict, kstars
> Suggests: sunclock, xtide
> ```
>
> becomes in d/control:
>
> ```
> Package: astro-education
> Section: metapackages
> Architecture: all
> Depends: ${misc:Depends},
> astro-tasks (= ${source:Version}),
> gpredict,
> kstars
> Recommends: celestia-gnome
> Suggests: sunclock,
> xtide
> Description: Educational astronomy applications [...]
> ```
This does not sound good. :-(
> celestia-gnome is a package that does not exist in unstable.
> GENCONTROL_DEPENDS is set to TRUE.
My short term recommendation would be to set this to FALSE.
> In the moment, I would tend to rewrite blend-gen-control from scratch,
> using Python 3 and the standard Debian Python packages (debian.deb822,
> apt) in a modular fashion. This would make the script much more
> maintainable (given the knowledge of Python is a bit better than Perl),
> and could also lead to create a "debian.blends" Python package that
> could be re-used for the Web pages.
I can only repeat: Blends-dev *was* rewritten and creates correct
results:
https://salsa.debian.org/blends-team/blends-gsoc
I repeat again: This creates *architecture* dependent metapackages (I
explained here why only this makes sense). I'm all for switching to
Python3 but a complete rewrite should not be necessary. There are some
glithes in the code (for instance line breaking in the auto generated
part of d/changelog) and it definitely needs testing.
> Any thoughts on that? I'd volunteer here.
I *really* appreciate your effort but may be 80% of the work is just
done and you try blends-gsoc first. As we all know the last 20% is
the harder part - but I urgently vote for using UDD as base for
anything we use. May me using the public UDD mirror can help to not
force everybody who wants to run blends-dev having a local clone.
> Best regards
Thanks a lot for pushing things
Andreas.
--
http://fam-tille.de
Re: Bug#891188: blends-dev: created d/control recommends packages not in main
Hi Wolfgang, thanks a lot for checking. On Fri, Mar 16, 2018 at 12:07:28PM +0100, Wolfgang Schweer wrote: > On Thu, Mar 15, 2018 at 07:41:30PM +0100, Andreas Tille wrote: > > I've commited a fix to Git which for me creates now sensible Debian > > Astro metapackages. > > I've checked how this fix would concern Debian Edu: > > (1) The Debian Edu Makefile has to be adjusted: > s/GENCONTROL_DEPENDS = true/#GENCONTROL_DEPENDS = true/ > > (2) Packages not available (any more) in testing are then correctly > demoted from Recommends to Suggests due to the $missinglist fix. (It > seems those have been wrongly kept as Recommends since the August > 2017 changes.) > > (3) The strong depends on gosa-plugin-netgroups needed due to Bug > #793667¹ (IMO cause for 'GENCONTROL_DEPENDS = true') are a non issue > since ages. I figure in a similar case adding information to the > Debian Edu upgrade chapter seems to be sufficient (if a smarter > solution isn't easy). I would summarise your check with: "Works for Debian Edu" Kind regards Andreas. -- http://fam-tille.de
Re: Bug#891188: blends-dev: created d/control recommends packages not in main
On Thu, Mar 15, 2018 at 07:41:30PM +0100, Andreas Tille wrote: > I've commited a fix to Git which for me creates now sensible Debian > Astro metapackages. I've checked how this fix would concern Debian Edu: (1) The Debian Edu Makefile has to be adjusted: s/GENCONTROL_DEPENDS = true/#GENCONTROL_DEPENDS = true/ (2) Packages not available (any more) in testing are then correctly demoted from Recommends to Suggests due to the $missinglist fix. (It seems those have been wrongly kept as Recommends since the August 2017 changes.) (3) The strong depends on gosa-plugin-netgroups needed due to Bug #793667¹ (IMO cause for 'GENCONTROL_DEPENDS = true') are a non issue since ages. I figure in a similar case adding information to the Debian Edu upgrade chapter seems to be sufficient (if a smarter solution isn't easy). Wolfgang ¹ https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=793667 signature.asc Description: PGP signature
Re: Bug#891188: blends-dev: created d/control recommends packages not in main
Hi Ole, On Thu, Mar 15, 2018 at 05:14:36PM +0100, Andreas Tille wrote: > > Feel free to lower the severity. > > I think there is no need to since I found the part of the code where the > issue occures. If I would only understand Perl a bit better I would be > ready with a fix. :-( I've commited a fix to Git which for me creates now sensible Debian Astro metapackages. I'd love some review / enhancement of my beginners Perl code before uploading. Kind regards Andreas. -- http://fam-tille.de
Re: Bug#891188: blends-dev: created d/control recommends packages not in main
Hi Ole,
On Thu, Mar 15, 2018 at 04:55:24PM +0100, Ole Streicher wrote:
> >
> >GENCONTROL_DEPENDS = true
> >
>
> Feel free to lower the severity.
I think there is no need to since I found the part of the code where the
issue occures. If I would only understand Perl a bit better I would be
ready with a fix. :-(
> However, that was a change of the
> default behaviour that we discussed a year ago: we wanted to remove the
> translation
>
> "Depends:" (tasks) -> "Recommends:" (d/control)
> "Recommends:" (tasks) -> "Suggests:" (d/control)
>
> in favour of a direct copy.
Its true that we agreed upon this and the code **seemed** to do this.
However, if there is no Depends -> Recommends the check whether the
missings need to be removed is simply not done.
> I am wondering why debian-astro is the only
> blend which finally used this, and I think we wanted to convert all.
I have not seen any reason to switch any behaviour since there are no
strict Depends in Debian Med. Thus I just substituted
s/Depends/Recommends/ inside the watch file "to be prepared" and be more
expclicit what we mean. But I had no reason to force any switch. Most
probably all other Blends did the same as I in Debian Med and thus you
were simply the only one who was stumbling upon this issue.
> > But I do not want to play severity ping-pong and will have a look
> > why GENCONTROL_DEPENDS is obviously breaking this.
>
> That is probably the code that was introduced for this purpose (August
> 2017).
It is
a452586c devtools/blend-gen-control(Mike Gabriel2017-08-11
23:39:29 -0400 267) my $pkglist;
a452586c devtools/blend-gen-control(Mike Gabriel2017-08-11
23:39:29 -0400 268) my $missinglist;
a452586c devtools/blend-gen-control(Mike Gabriel2017-08-11
23:39:29 -0400 269) if (defined $taskinfo{$task}{Depends})
a452586c devtools/blend-gen-control(Mike Gabriel2017-08-11
23:39:29 -0400 270) {
a452586c devtools/blend-gen-control(Mike Gabriel2017-08-11
23:39:29 -0400 271) ($pkglist, $missinglist) =
process_pkglist(join(",",@{$taskinfo{$task}{Depends}}));
36791f91 cdd/devtools/cdd-gen-control (Andreas Tille 2007-08-27
17:48:49 + 272) }
a452586c devtools/blend-gen-control(Mike Gabriel2017-08-11
23:39:29 -0400 273)
a452586c devtools/blend-gen-control(Mike Gabriel2017-08-11
23:39:29 -0400 274) my (@depends, @recommends, @suggests);
a452586c devtools/blend-gen-control(Mike Gabriel2017-08-11
23:39:29 -0400 275)
0fc23d84 devtools/blend-gen-control(Mike Gabriel2017-08-21
13:51:30 -0400 276) push @depends, $tasksname.' (=
${source:Version})';
a452586c devtools/blend-gen-control(Mike Gabriel2017-08-11
23:39:29 -0400 277) push @depends, '${misc:Depends}';
a452586c devtools/blend-gen-control(Mike Gabriel2017-08-11
23:39:29 -0400 278) push @depends, @{$pkglist}
a452586c devtools/blend-gen-control(Mike Gabriel2017-08-11
23:39:29 -0400 279) if defined $pkglist;
a452586c devtools/blend-gen-control(Mike Gabriel2017-08-11
23:39:29 -0400 280)
a452586c devtools/blend-gen-control(Mike Gabriel2017-08-11
23:39:29 -0400 281) push @recommends,
@{$taskinfo{$task}{Recommends}}
a452586c devtools/blend-gen-control(Mike Gabriel2017-08-11
23:39:29 -0400 282) if defined $taskinfo{$task}{Recommends};
a452586c devtools/blend-gen-control(Mike Gabriel2017-08-11
23:39:29 -0400 283)
a452586c devtools/blend-gen-control(Mike Gabriel2017-08-11
23:39:29 -0400 284) push @suggests, @{$missinglist}
a452586c devtools/blend-gen-control(Mike Gabriel2017-08-11
23:39:29 -0400 285) if defined $missinglist;
a452586c devtools/blend-gen-control(Mike Gabriel2017-08-11
23:39:29 -0400 286) push @suggests,
@{$taskinfo{$task}{Suggests}}
a452586c devtools/blend-gen-control(Mike Gabriel2017-08-11
23:39:29 -0400 287) if defined $taskinfo{$task}{Suggests};
a452586c devtools/blend-gen-control(Mike Gabriel2017-08-11
23:39:29 -0400 288)
a452586c devtools/blend-gen-control(Mike Gabriel2017-08-11
23:39:29 -0400 289) my @depends_sorted = sort_uniq(\%seenlist,
@depends);
a452586c devtools/blend-gen-control(Mike Gabriel2017-08-11
23:39:29 -0400 290) my @recommends_sorted =
sort_uniq(\%seenlist, @recommends);
a452586c devtools/blend-gen-control(Mike Gabriel2017-08-11
23:39:29 -0400 291) my @suggests_sorted = sort_uniq(\%seenlist,
@suggests);
a452
Re: Bug#891188: blends-dev: created d/control recommends packages not in main
Hi Ole, On Fri, Feb 23, 2018 at 01:32:17PM +0100, Ole Streicher wrote: > > No. As you can see from my last mail, the list of missing or avoided > packages seems to be generated properly. It is just not used to lower > the dependencies. Interestingly enough I can reproduce the issue for Debian Astro but in Debian Med and Debian Science the packages from contrib/non-free and up in Suggests. So I can not reproduce the issue on other examples which is really strange. Kind regards Andreas. -- http://fam-tille.de
Re: Bug#891188: blends-dev: created d/control recommends packages not in main
Hi Ole, On Fri, Feb 23, 2018 at 01:32:17PM +0100, Ole Streicher wrote: > could you Cc your mails to the bug? Ups, bounced ... > On 23.02.2018 13:23, Andreas Tille wrote: > > Did you somehow changed /etc/blends/sources.list or are you > > pointing to some different location with -s option? > > No. As you can see from my last mail, the list of missing or avoided > packages seems to be generated properly. It is just not used to lower > the dependencies. This would mean the behaviour has changed. :-( I admit I can not check since I'll travel the next four days and will be back to normal at 28.2. :-( May be some git bisecting (for this I use a symlink of /usr/share/blends-dev to my Git repository has proven quite usefull) will enable to find the problematic piece of code. The chancge from Depends to Recommends might seem suspicious, thought. Sorry for not beeing able to help more Andreas. -- http://fam-tille.de
Re: Bug#891188: blends-dev: created d/control recommends packages not in main
Hi Petter, On Fri, Feb 23, 2018 at 10:56:08AM +0100, Petter Reinholdtsen wrote: > > IMO this is a regression; it worked some time ago, right? > > As far as I know, this has never behaved differently. I am not aware of > blends-dev every looking at main/contrib status, only if the package > exists. Petter, if I might refresh your mind ;-): blends-dev has pulled Packages.gz as per /etc/blends/sources.list.* (by default /etc/blends/sources.list.testing since this will be our target release). If you provide a different sources.list file with -s option or if you change your local configuration by adding other resources, than you can tweak this behaviour. Blends-dev has checked, whether a package exists there (by default testing/main). Everything that did not existed there (may it be in contrib/non-free/somewhere-else) was put to Suggests. This is how it worked and how it should work to create policy conform packages. > In my view, it is the responsibility of the people writing the tasks to > decide if a package in contrib should use recommends or suggest, not the > blends build system. No. Recommends need to reside in main (per policy). Kind regards Andreas. -- http://fam-tille.de
Re: Bug#891188: blends-dev: created d/control recommends packages not in main
Hi Ole, On Fri, Feb 23, 2018 at 10:43:01AM +0100, Ole Streicher wrote: > > when I run "make" on a blend's package, it puts all that is in > "Recommends" in the tasks package into "Recommends" of d/control, > independent of the status of the package. > > Examples, from debian-astro > https://salsa.debian.org/debian-astro-team/debian-astro Did you somehow changed /etc/blends/sources.list or are you pointing to some different location with -s option? Kind regards Andreas. -- http://fam-tille.de
