Re: [gentoo-dev] Keywordreqs and slacking arch teams
On Sat, 28 Dec 2019 21:19:18 -0500 Aaron Bauman wrote: > Ah, you found the Achilles heel. It is much easier to postulate on the mailing > list, use big words, and then say you just won't do that thing because > tools/languages such. > > Perl though... FTR: Even though I'm good at Perl, I wouldn't use it for this task. There are a lot of reasons behind this. But at least one of them would be if I wrote it in Perl, I'd have nobody else contributing either. And IME, that's fair enough. pgpqWSZPaxxWp.pgp Description: OpenPGP digital signature
Re: [gentoo-dev] Keywordreqs and slacking arch teams
On Sat, Dec 28, 2019 at 10:05:02AM -0800, Alec Warner wrote: > On Sat, Dec 28, 2019 at 3:43 AM Kent Fredric wrote: > > > On Sat, 28 Dec 2019 11:35:49 + > > Michael 'veremitz' Everitt wrote: > > > > > Note: we're nnot acttually talking about replacing portage here, just > > > creating a tool thiink php script web tthingy)) that will do some of > > > the pre-screeninng wok that AT hate (eg what kensiington did > > with > > > stable-bbot) > > > > > > * with apologies for keyboard/remote-access la creating typo hell. > > > > But, doing that requires viewing realised copies of ebuilds, which > > requires interpreting eclasses and variable interpolation, which > > requires bash sourcing, which requires a mountain of portage hell. > > > > > Yes, sharing the stable-bot logic would probably be fine. > > > > But it doesn't use a database AFAIK, it would likely just be making use > > of the MD5Cache (either directly, or indirectly via portage APIs) > > > > But I won't be volunteering, because I won't touch python. > > > > You could of course work together with someone else to write the tool? > > -A Ah, you found the Achilles heel. It is much easier to postulate on the mailing list, use big words, and then say you just won't do that thing because tools/languages such. Perl though... -- Cheers, Aaron signature.asc Description: PGP signature
Re: [gentoo-dev] Keywordreqs and slacking arch teams
On Sat, Dec 28, 2019 at 3:43 AM Kent Fredric wrote: > On Sat, 28 Dec 2019 11:35:49 + > Michael 'veremitz' Everitt wrote: > > > Note: we're nnot acttually talking about replacing portage here, just > > creating a tool thiink php script web tthingy)) that will do some of > > the pre-screeninng wok that AT hate (eg what kensiington did > with > > stable-bbot) > > > > * with apologies for keyboard/remote-access la creating typo hell. > > But, doing that requires viewing realised copies of ebuilds, which > requires interpreting eclasses and variable interpolation, which > requires bash sourcing, which requires a mountain of portage hell. > > Yes, sharing the stable-bot logic would probably be fine. > > But it doesn't use a database AFAIK, it would likely just be making use > of the MD5Cache (either directly, or indirectly via portage APIs) > > But I won't be volunteering, because I won't touch python. > You could of course work together with someone else to write the tool? -A
Re: [gentoo-dev] Keywordreqs and slacking arch teams
On Sat, 28 Dec 2019 11:40:08 + James Le Cuirot wrote: > Unfortunately a3li used Elasticsearch, which no one understands, but > it's a start. And I've used ES enough to say I would rather never use it again. pgp7mS3HF7Z2A.pgp Description: OpenPGP digital signature
Re: [gentoo-dev] Keywordreqs and slacking arch teams
On Sat, 28 Dec 2019 11:35:49 + Michael 'veremitz' Everitt wrote: > Note: we're nnot acttually talking about replacing portage here, just > creating a tool thiink php script web tthingy)) that will do some of > the pre-screeninng wok that AT hate (eg what kensiington did with > stable-bbot) > > * with apologies for keyboard/remote-access la creating typo hell. But, doing that requires viewing realised copies of ebuilds, which requires interpreting eclasses and variable interpolation, which requires bash sourcing, which requires a mountain of portage hell. Yes, sharing the stable-bot logic would probably be fine. But it doesn't use a database AFAIK, it would likely just be making use of the MD5Cache (either directly, or indirectly via portage APIs) But I won't be volunteering, because I won't touch python. pgphawvJwBwAU.pgp Description: OpenPGP digital signature
Re: [gentoo-dev] Keywordreqs and slacking arch teams
On Sun, 29 Dec 2019 00:27:36 +1300 Kent Fredric wrote: > On Sat, 28 Dec 2019 11:14:15 + > Michael 'veremitz' Everitt wrote: > > > I know I'm gonna be shot down in flames, because $heresy, but here is where > > a package 'database' would actually work quite well, because you can > > trivially create a query that pulls this data out, and sorts it by package > > category or maintainer or whatever you like .. > > > > Ok, let the flamewars begin ... > > There's no real problem with a package database, however, the real > limitation is in the ebuild source format, which ultimately means any > such database needs a lot of bash-sourcing hell to simply stay > up-to-date ( any time an eclass changes, the interpretation of every > ebuild that uses it also changes, necessitating some pretty fun(1) code ) > > And that winds you up fighting with portage internals. > > So simply, in order for somebody like me to actually implement such a > thing, a precursory step is to rewrite enough portage to do just that. > > But I haven't (yet) gotten around to that. > > > 1: Not actually fun. Doesn't https://packages.gentoo.org already have such a database? Unfortunately a3li used Elasticsearch, which no one understands, but it's a start. -- James Le Cuirot (chewi) Gentoo Linux Developer pgpUl78_P3IXC.pgp Description: OpenPGP digital signature
Re: [gentoo-dev] Keywordreqs and slacking arch teams
On 28/12/19 11:32, Kent Fredric wrote: > On Sat, 28 Dec 2019 11:14:15 + > Michael 'veremitz' Everitt wrote: > >> I know I'm gonna be shot down in flames, because $heresy, but here is where >> a package 'database' would actually work quite well, because you can >> trivially create a query that pulls this data out, and sorts it by package >> category or maintainer or whatever you like . > Oh, and once upon a time, there was actually a trick you could do to > make portage keep its metadata caches in an SQLite database, which had > its benefits, and I even had a rough tool I wrote in ruby and shared on > the old (defunct) wiki that helped do quick database queries. > > But the trick actually makes portage slower in multiple ways, and it > never really got widespread love. > > ( Though I would argue how the data was stored was also inadequate for > a lot of other tasks one might want to do with a database, like, > keeping DEPEND stored in its string format ) Note: we're nnot acttually talking about replacing portage here, just creating a tool thiink php script web tthingy)) that will do some of the pre-screeninng wok that AT hate (eg what kensiington did with stable-bbot) * with apologies for keyboard/remote-access la creating typo hell. signature.asc Description: OpenPGP digital signature
Re: [gentoo-dev] Keywordreqs and slacking arch teams
On Sat, 28 Dec 2019 11:14:15 + Michael 'veremitz' Everitt wrote: > I know I'm gonna be shot down in flames, because $heresy, but here is where > a package 'database' would actually work quite well, because you can > trivially create a query that pulls this data out, and sorts it by package > category or maintainer or whatever you like . Oh, and once upon a time, there was actually a trick you could do to make portage keep its metadata caches in an SQLite database, which had its benefits, and I even had a rough tool I wrote in ruby and shared on the old (defunct) wiki that helped do quick database queries. But the trick actually makes portage slower in multiple ways, and it never really got widespread love. ( Though I would argue how the data was stored was also inadequate for a lot of other tasks one might want to do with a database, like, keeping DEPEND stored in its string format ) pgpG_kX027cdJ.pgp Description: OpenPGP digital signature
Re: [gentoo-dev] Keywordreqs and slacking arch teams
On Sat, 28 Dec 2019 11:14:15 + Michael 'veremitz' Everitt wrote: > I know I'm gonna be shot down in flames, because $heresy, but here is where > a package 'database' would actually work quite well, because you can > trivially create a query that pulls this data out, and sorts it by package > category or maintainer or whatever you like .. > > Ok, let the flamewars begin ... There's no real problem with a package database, however, the real limitation is in the ebuild source format, which ultimately means any such database needs a lot of bash-sourcing hell to simply stay up-to-date ( any time an eclass changes, the interpretation of every ebuild that uses it also changes, necessitating some pretty fun(1) code ) And that winds you up fighting with portage internals. So simply, in order for somebody like me to actually implement such a thing, a precursory step is to rewrite enough portage to do just that. But I haven't (yet) gotten around to that. 1: Not actually fun. pgppjeYD2_M91.pgp Description: OpenPGP digital signature
Re: [gentoo-dev] Keywordreqs and slacking arch teams
On 28/12/19 11:05, Kent Fredric wrote: > On Sat, 28 Dec 2019 10:35:09 +0100 > Fabian Groffen wrote: > >> Hmmm, interested to hear what kind of things you're thinking about here. > A lot of the "Work" of filing a keyword request is modelling all the > consequential keywordings that have to take place. > > If there was say, a web based UI, that: > > - Automatically determined which packages are ready for stabilization > due to all their dependencies already being stable (and maybe with > automatic cooldown-from-testing detection ) > > - Automatically determined which packages can be keyworded without > additional work due to all their dependencies being keyworded I know I'm gonna be shot down in flames, because $heresy, but here is where a package 'database' would actually work quite well, because you can trivially create a query that pulls this data out, and sorts it by package category or maintainer or whatever you like .. Ok, let the flamewars begin ... signature.asc Description: OpenPGP digital signature
Re: [gentoo-dev] Keywordreqs and slacking arch teams
On Sat, 28 Dec 2019 10:35:09 +0100 Fabian Groffen wrote: > Hmmm, interested to hear what kind of things you're thinking about here. A lot of the "Work" of filing a keyword request is modelling all the consequential keywordings that have to take place. If there was say, a web based UI, that: - Automatically determined which packages are ready for stabilization due to all their dependencies already being stable (and maybe with automatic cooldown-from-testing detection ) - Automatically determined which packages can be keyworded without additional work due to all their dependencies being keyworded - When requesting keywording/stabilization, automatically determined what all the consequences are and what else needs to be keyworded to satisfy it - Allowed a simple "Add keyword(s) for package " interface, that intelligently created an issue and a target list, and then once the list was built, constantly ensured the list to be valid, or determined automatically when sub-work was completed and reducing the published list automatically, and then responded to potential issues based on changes in git, ( as opposed to being only triggered when the bug was touched ) Most of the "pain" and legwork required by maintainers would go away. As it is, I feel a lot of us are reproducing a lot of logic that is rather routine and could be automated. But the overall idea here is to orient the point of keyword-requests in some way to focus on the primary objective, where the developer indicates their intent, and the system's job is to facilitate that intent coming to fruition, pointing out problems on its own. ( I have somewhat hacked together some perl scripts for myself for some of these tasks, but the command-line interface is not ideal for this workflow, and the code is not in a condition I can share it, and I don't think perl is the right language to address this problem with ) pgpBgVetvsNaY.pgp Description: OpenPGP digital signature
Re: [gentoo-dev] Keywordreqs and slacking arch teams
On 28-12-2019 22:27:02 +1300, Kent Fredric wrote: > On Sat, 28 Dec 2019 08:09:33 +0100 > Michał Górny wrote: > > > How can we improve this? > > Every time this kind of issue comes up, I can't help feeling we need > some sort of tool more advanced than what we currently have. > > Something that maintains persistence of keyword demands similar to how > the current bot does, but more ... > > Its the sort of thing I get tempted to implement myself, but get too > horrified about the prospect of working with portage internals to do it. Hmmm, interested to hear what kind of things you're thinking about here. I feel it would help if we would have the ability to at least compile/test ebuilds automatically. Not sure how helpful qemu could be there, but I know things like compiling for things like arm (RPi) works reasonably well. Thanks, Fabian -- Fabian Groffen Gentoo on a different level signature.asc Description: PGP signature
Re: [gentoo-dev] Keywordreqs and slacking arch teams
On Sat, 28 Dec 2019 08:09:33 +0100 Michał Górny wrote: > How can we improve this? Every time this kind of issue comes up, I can't help feeling we need some sort of tool more advanced than what we currently have. Something that maintains persistence of keyword demands similar to how the current bot does, but more ... Its the sort of thing I get tempted to implement myself, but get too horrified about the prospect of working with portage internals to do it. pgpRWoCmPMaBI.pgp Description: OpenPGP digital signature
