Re: State of Proposal to improving KDE Software Repository Organization?

2016-01-24 Thread Albert Astals Cid 
El Monday 18 January 2016, a les 20:27:16, Boudewijn Rempt va escriure:
> On Mon, 18 Jan 2016, Friedrich W. H. Kossebau wrote:
> > Reason that I ask is that due to the split of Calligra into several repos
> > (see background^) the layout in the repo structure does no longer
> > properly reflect the project organisation. Right now there are three
> > active repos in the calligra/ repo substructure:
> > "calligra" at "calligra/"
> > "krita"at "calligra/krita"
> > "kexi" at "calligra/kexi"
> 
> What I'm wondering is, where is this "structure" actually visible? Not in
> 
> https://quickgit.kde.org/
> 
> or
> 
> https://phabricator.kde.org/diffusion/
> 
> I see it reflected in the old, to be discarded
> 
> https://projects.kde.org/projects
> 
> But where else? And what is it actually needed for?
> 
> > (("calligra" at "calligra/" confuses at least kdesrc-build, sent an email
> > to mpyne about if moving it to "calligra/calligra" should fix it.))
> > 
> > Things that are not properly matching organization:
> > * Krita starting with 3.* no longer is part of Calligra project
> > 
> >  (screws e.g. api.kde.org/bundled-apps-api/calligra-apidocs/ and also
> >  what people think to which project Krita belongs)
> > 
> > * Calligra & Krita are nowhere different to KDevelop, Digikam & Co,
> > 
> >  so no reason to be in a complete own toplevel structure,
> >  rather should be in the same sub structure, i.e. "Extragear",
> >  like extragear/calligra/* and extragear/graphics/krita
> > 
> > More, not only Calligra & Krita related:
> > * "Extragear" is an awful grouping name for apps with individual
> > 
> >  release plans, a legacy term that no longer fits most of the apps
> >  in that substructure
> 
> It's ghastly -- almost insulting. It's perpetuating the fallacy that
> there are core KDE projects and peripheral projects.
> 
> > * "KDE Applications" is a misleading grouping name for apps with a
> > 
> >  central release plan, as if those with individual release plans
> >  are not "KDE" applications (as in, not done in the KDE community)
> 
> Horrible as well.

You're always welcome to suggest better names.

Cheers,
  Albert

Re: State of Proposal to improving KDE Software Repository Organization?

2016-01-20 Thread Ben Cooksley 
On Thu, Jan 21, 2016 at 7:00 AM, Friedrich W. H. Kossebau
<kosse...@kde.org> wrote:
> Am Dienstag, 19. Januar 2016, 13:57:10 schrieb Ben Cooksley:
>> On Tue, Jan 19, 2016 at 7:28 AM, Friedrich W. H. Kossebau
>>
>> <kosse...@kde.org> wrote:
>> > 4 months ago there was the thread "Proposal to improving KDE Software
>> > Repository Organization" on this mailinglist.
>> > What happened to that plan? Are people preparing its execution?
>>
>> That plan is tied up in other things taking priority / lack of time / etc.
>> We'll get there eventually. It is also in part related to the Phabricator
>> move.
>
> Okay, so still work in progress. Good.
>
>> > And would that be a time where some bigger reorganization of the repos is
>> > possible?
>> >
>> > Reason that I ask is that due to the split of Calligra into several repos
>> > (see background^) the layout in the repo structure does no longer
>> > properly reflect the project organisation. Right now there are three
>> > active repos in the calligra/ repo substructure:
>> > "calligra" at "calligra/"
>> > "krita"at "calligra/krita"
>> > "kexi" at "calligra/kexi"
>> >
>> > (("calligra" at "calligra/" confuses at least kdesrc-build, sent an email
>> > to mpyne about if moving it to "calligra/calligra" should fix it.))
>>
>> Repositories within repositories is a known bad thing to do, the
>> systems don't handle it properly at all (as it was never an intended
>> thing you should do). The proper fix is to move the repo to
>> calligra/calligra (ie. have a "calligra/" top level grouping project).
>
> This move is now requested on https://phabricator.kde.org/T1337 , the
> respective kde-build-metadata change locally prepared.

Okay. People have jumped on using that board a bit earlier than I
anticipated but we can work with that.

>
>> > Things that are not properly matching organization:
>> > * Krita starting with 3.* no longer is part of Calligra project
>> >
>> >   (screws e.g. api.kde.org/bundled-apps-api/calligra-apidocs/ and also
>> >   what people think to which project Krita belongs)
>> >
>> > * Calligra & Krita are nowhere different to KDevelop, Digikam & Co,
>> >
>> >   so no reason to be in a complete own toplevel structure,
>> >   rather should be in the same sub structure, i.e. "Extragear",
>> >   like extragear/calligra/* and extragear/graphics/krita
>>
>> In the Phabricator world I had envisioned Extragear as no longer existing.
>
> Okay, sounds good.
>
>> > More, not only Calligra & Krita related:
>> > * "Extragear" is an awful grouping name for apps with individual
>> >
>> >   release plans, a legacy term that no longer fits most of the apps
>> >   in that substructure
>> >
>> > * "KDE Applications" is a misleading grouping name for apps with a
>> >
>> >   central release plan, as if those with individual release plans
>> >   are not "KDE" applications (as in, not done in the KDE community)
>> >
>> > * a single category per app as needed by the current tree structure layout
>> >
>> >   of the repos, like "office", "graphics", "utils", is rather awkward,
>> >   many apps do not match exactly one or would match multiple categories
>>
>> Phabricator will allow multiple "categories" to be tagged to a repository...
>
> Nice, sounds even better.
>
>> > So IMHO some update of the repository organisation would be good, to
>> > reflect how things are these days.
>> > Renaming of "Extragear" and "KDE Applications" is surely something which
>> > needs care from promo/marketing/VDG people first to find if that makes
>> > sense at all and what a good solution would be.
>>
>> Extragear is really an internal structure, not part of marketing so I
>> think we can go ahead and just kill it...
>
> Seems we all pull on the same string then, glad to see that :)
>
>> > (Being both maintainer of Okteta, which is in "KDE Applications", and
>> > meta-co- maintainer of Calligra, which is not, but still done in the very
>> > same KDE community, that current naming seems so wrong to me).
>> >
>> > But the actual names and grouping aside, for the pure technical renewing
>> > (which also involves all infrast

Re: State of Proposal to improving KDE Software Repository Organization?

2016-01-20 Thread Friedrich W. H. Kossebau 
Am Dienstag, 19. Januar 2016, 13:57:10 schrieb Ben Cooksley:
> On Tue, Jan 19, 2016 at 7:28 AM, Friedrich W. H. Kossebau
> 
> <kosse...@kde.org> wrote:
> > 4 months ago there was the thread "Proposal to improving KDE Software
> > Repository Organization" on this mailinglist.
> > What happened to that plan? Are people preparing its execution?
> 
> That plan is tied up in other things taking priority / lack of time / etc.
> We'll get there eventually. It is also in part related to the Phabricator
> move.

Okay, so still work in progress. Good.

> > And would that be a time where some bigger reorganization of the repos is
> > possible?
> > 
> > Reason that I ask is that due to the split of Calligra into several repos
> > (see background^) the layout in the repo structure does no longer
> > properly reflect the project organisation. Right now there are three
> > active repos in the calligra/ repo substructure:
> > "calligra" at "calligra/"
> > "krita"at "calligra/krita"
> > "kexi" at "calligra/kexi"
> > 
> > (("calligra" at "calligra/" confuses at least kdesrc-build, sent an email
> > to mpyne about if moving it to "calligra/calligra" should fix it.))
> 
> Repositories within repositories is a known bad thing to do, the
> systems don't handle it properly at all (as it was never an intended
> thing you should do). The proper fix is to move the repo to
> calligra/calligra (ie. have a "calligra/" top level grouping project).

This move is now requested on https://phabricator.kde.org/T1337 , the 
respective kde-build-metadata change locally prepared.

> > Things that are not properly matching organization:
> > * Krita starting with 3.* no longer is part of Calligra project
> > 
> >   (screws e.g. api.kde.org/bundled-apps-api/calligra-apidocs/ and also
> >   what people think to which project Krita belongs)
> > 
> > * Calligra & Krita are nowhere different to KDevelop, Digikam & Co,
> > 
> >   so no reason to be in a complete own toplevel structure,
> >   rather should be in the same sub structure, i.e. "Extragear",
> >   like extragear/calligra/* and extragear/graphics/krita
> 
> In the Phabricator world I had envisioned Extragear as no longer existing.

Okay, sounds good.

> > More, not only Calligra & Krita related:
> > * "Extragear" is an awful grouping name for apps with individual
> > 
> >   release plans, a legacy term that no longer fits most of the apps
> >   in that substructure
> > 
> > * "KDE Applications" is a misleading grouping name for apps with a
> > 
> >   central release plan, as if those with individual release plans
> >   are not "KDE" applications (as in, not done in the KDE community)
> > 
> > * a single category per app as needed by the current tree structure layout
> > 
> >   of the repos, like "office", "graphics", "utils", is rather awkward,
> >   many apps do not match exactly one or would match multiple categories
> 
> Phabricator will allow multiple "categories" to be tagged to a repository...

Nice, sounds even better.

> > So IMHO some update of the repository organisation would be good, to
> > reflect how things are these days.
> > Renaming of "Extragear" and "KDE Applications" is surely something which
> > needs care from promo/marketing/VDG people first to find if that makes
> > sense at all and what a good solution would be.
> 
> Extragear is really an internal structure, not part of marketing so I
> think we can go ahead and just kill it...

Seems we all pull on the same string then, glad to see that :)

> > (Being both maintainer of Okteta, which is in "KDE Applications", and
> > meta-co- maintainer of Calligra, which is not, but still done in the very
> > same KDE community, that current naming seems so wrong to me).
> > 
> > But the actual names and grouping aside, for the pure technical renewing
> > (which also involves all infrastructure like translation system,
> > documentation, phabricator, etc), who is currently planning or working on
> > what?
> 
> Like most things in this department, Sysadmin...

So in good hands, I leave it there :P Nah, ready to also do some share of work 
on this, as I would like this itch scratched as well. Please call me where you 
see fit.

> > So does it makes sense to wait some more, or should we assume the current
> > organization stays for longer, and Calligra & Krita repos should be moved
> &

Re: State of Proposal to improving KDE Software Repository Organization?

2016-01-20 Thread Friedrich W. H. Kossebau 
Am Dienstag, 19. Januar 2016, 08:07:11 schrieb Elvis Angelaccio:
> 2016-01-19 2:05 GMT+01:00 Nicolás Alvarez :
> > 2016-01-18 21:57 GMT-03:00 Ben Cooksley :
> > > On Tue, Jan 19, 2016 at 7:28 AM, Friedrich W. H. Kossebau
> > > 
> > >  wrote:
> > >> So IMHO some update of the repository organisation would be good, to
> > 
> > reflect
> > 
> > >> how things are these days.
> > >> Renaming of "Extragear" and "KDE Applications" is surely something
> > 
> > which needs
> > 
> > >> care from promo/marketing/VDG people first to find if that makes sense
> > 
> > at all
> > 
> > >> and what a good solution would be.
> > > 
> > > Extragear is really an internal structure, not part of marketing so I
> > > think we can go ahead and just kill it...
> > 
> > Then the first thing to kill is https://extragear.kde.org/ (careful
> > with potential incoming broken links though).
> 
> This should probably have happened some time ago, but for some reason it
> still hasn't.
> See also this thread started by Helio:
> https://mail.kde.org/pipermail/kde-extra-gear/2015-August/002075.html

Picked that thread up, time to make that a "Done".

Cheers
Friedrich

Re: State of Proposal to improving KDE Software Repository Organization?

2016-01-19 Thread Elvis Angelaccio 
2016-01-19 2:05 GMT+01:00 Nicolás Alvarez :

> 2016-01-18 21:57 GMT-03:00 Ben Cooksley :
> > On Tue, Jan 19, 2016 at 7:28 AM, Friedrich W. H. Kossebau
> >  wrote:
> >
> >> So IMHO some update of the repository organisation would be good, to
> reflect
> >> how things are these days.
> >> Renaming of "Extragear" and "KDE Applications" is surely something
> which needs
> >> care from promo/marketing/VDG people first to find if that makes sense
> at all
> >> and what a good solution would be.
> >
> > Extragear is really an internal structure, not part of marketing so I
> > think we can go ahead and just kill it...
>
> Then the first thing to kill is https://extragear.kde.org/ (careful
> with potential incoming broken links though).
>
>
This should probably have happened some time ago, but for some reason it
still hasn't.
See also this thread started by Helio:
https://mail.kde.org/pipermail/kde-extra-gear/2015-August/002075.html


> --
> Nicolás
>

Cheers,
Elvis

State of Proposal to improving KDE Software Repository Organization?

2016-01-18 Thread Friedrich W. H. Kossebau 
Hi,

(calligra-devel, kexi-devel, kimageshop mailinglists only for heads-up,
 please remove from reply, discussion only on kde-core-devel should be fine)

4 months ago there was the thread "Proposal to improving KDE Software 
Repository Organization" on this mailinglist.
What happened to that plan? Are people preparing its execution?

And would that be a time where some bigger reorganization of the repos is 
possible?

Reason that I ask is that due to the split of Calligra into several repos (see 
background^) the layout in the repo structure does no longer properly reflect 
the project organisation. Right now there are three active repos in the 
calligra/ repo substructure:
"calligra" at "calligra/"
"krita"at "calligra/krita"
"kexi" at "calligra/kexi"

(("calligra" at "calligra/" confuses at least kdesrc-build, sent an email to 
mpyne about if moving it to "calligra/calligra" should fix it.))

Things that are not properly matching organization:
* Krita starting with 3.* no longer is part of Calligra project
  (screws e.g. api.kde.org/bundled-apps-api/calligra-apidocs/ and also
  what people think to which project Krita belongs)
* Calligra & Krita are nowhere different to KDevelop, Digikam & Co,
  so no reason to be in a complete own toplevel structure,
  rather should be in the same sub structure, i.e. "Extragear",
  like extragear/calligra/* and extragear/graphics/krita

More, not only Calligra & Krita related:
* "Extragear" is an awful grouping name for apps with individual
  release plans, a legacy term that no longer fits most of the apps
  in that substructure
* "KDE Applications" is a misleading grouping name for apps with a
  central release plan, as if those with individual release plans
  are not "KDE" applications (as in, not done in the KDE community)
* a single category per app as needed by the current tree structure layout
  of the repos, like "office", "graphics", "utils", is rather awkward,
  many apps do not match exactly one or would match multiple categories

So IMHO some update of the repository organisation would be good, to reflect 
how things are these days.
Renaming of "Extragear" and "KDE Applications" is surely something which needs 
care from promo/marketing/VDG people first to find if that makes sense at all 
and what a good solution would be.
(Being both maintainer of Okteta, which is in "KDE Applications", and meta-co-
maintainer of Calligra, which is not, but still done in the very same KDE 
community, that current naming seems so wrong to me).

But the actual names and grouping aside, for the pure technical renewing 
(which also involves all infrastructure like translation system, 
documentation, phabricator, etc), who is currently planning or working on 
what?
So does it makes sense to wait some more, or should we assume the current 
organization stays for longer, and Calligra & Krita repos should be moved 
inside that organization for now?


^Some background about Calligra repo split, as things are slightly 
complicated:

KRITA)  The "krita" repo was split off, because Krita has finally become a 
full project of its own, separate from Calligra. A logical place for the krita 
repo in the KDE repo structure would perhaps have been somewhere in extragear, 
but at least due to the translators preferring to keep the string catalogs of 
Krita in the "calligra" module as before, for less work, the krita repo was 
for now put as submodule of "calligra/".

KEXI)  Kexi continues to be part of the Calligra project/subcommunity, but the 
Kexi developers preferred a small simple repo "kexi" of their own (for build 
time and size). So the placement at "calligra/kexi" makes perfect sense.

OTHERAPPS)  As the other Calligra apps (Braindump, Karbon, Sheets, Words, 
Stage, etc.) are more tightly coupled and the binary interfaces between libs, 
plugins & apps can still change every other week, for now no further repo 
splitting is planned (to ensure atomic commits on API changes), and they all 
stay in the existing "calligra" repo.


Cheers
Friedrich

Re: State of Proposal to improving KDE Software Repository Organization?

2016-01-18 Thread Ben Cooksley 
On Tue, Jan 19, 2016 at 7:28 AM, Friedrich W. H. Kossebau
<kosse...@kde.org> wrote:
> Hi,

Hi,

>
> (calligra-devel, kexi-devel, kimageshop mailinglists only for heads-up,
>  please remove from reply, discussion only on kde-core-devel should be fine)
>
> 4 months ago there was the thread "Proposal to improving KDE Software
> Repository Organization" on this mailinglist.
> What happened to that plan? Are people preparing its execution?

That plan is tied up in other things taking priority / lack of time / etc.
We'll get there eventually. It is also in part related to the Phabricator move.

>
> And would that be a time where some bigger reorganization of the repos is
> possible?
>
> Reason that I ask is that due to the split of Calligra into several repos (see
> background^) the layout in the repo structure does no longer properly reflect
> the project organisation. Right now there are three active repos in the
> calligra/ repo substructure:
> "calligra" at "calligra/"
> "krita"at "calligra/krita"
> "kexi" at "calligra/kexi"
>
> (("calligra" at "calligra/" confuses at least kdesrc-build, sent an email to
> mpyne about if moving it to "calligra/calligra" should fix it.))

Repositories within repositories is a known bad thing to do, the
systems don't handle it properly at all (as it was never an intended
thing you should do). The proper fix is to move the repo to
calligra/calligra (ie. have a "calligra/" top level grouping project).

>
> Things that are not properly matching organization:
> * Krita starting with 3.* no longer is part of Calligra project
>   (screws e.g. api.kde.org/bundled-apps-api/calligra-apidocs/ and also
>   what people think to which project Krita belongs)
> * Calligra & Krita are nowhere different to KDevelop, Digikam & Co,
>   so no reason to be in a complete own toplevel structure,
>   rather should be in the same sub structure, i.e. "Extragear",
>   like extragear/calligra/* and extragear/graphics/krita

In the Phabricator world I had envisioned Extragear as no longer existing.

>
> More, not only Calligra & Krita related:
> * "Extragear" is an awful grouping name for apps with individual
>   release plans, a legacy term that no longer fits most of the apps
>   in that substructure
> * "KDE Applications" is a misleading grouping name for apps with a
>   central release plan, as if those with individual release plans
>   are not "KDE" applications (as in, not done in the KDE community)
> * a single category per app as needed by the current tree structure layout
>   of the repos, like "office", "graphics", "utils", is rather awkward,
>   many apps do not match exactly one or would match multiple categories

Phabricator will allow multiple "categories" to be tagged to a repository...

>
> So IMHO some update of the repository organisation would be good, to reflect
> how things are these days.
> Renaming of "Extragear" and "KDE Applications" is surely something which needs
> care from promo/marketing/VDG people first to find if that makes sense at all
> and what a good solution would be.

Extragear is really an internal structure, not part of marketing so I
think we can go ahead and just kill it...

> (Being both maintainer of Okteta, which is in "KDE Applications", and meta-co-
> maintainer of Calligra, which is not, but still done in the very same KDE
> community, that current naming seems so wrong to me).
>
> But the actual names and grouping aside, for the pure technical renewing
> (which also involves all infrastructure like translation system,
> documentation, phabricator, etc), who is currently planning or working on
> what?

Like most things in this department, Sysadmin...

> So does it makes sense to wait some more, or should we assume the current
> organization stays for longer, and Calligra & Krita repos should be moved
> inside that organization for now?

Not sure how long things are going to take sorry.
Chances are the existing tree will survive in some form (even if it is
only in the XML file various things use) so you may as well do it now.

>
>
> ^Some background about Calligra repo split, as things are slightly
> complicated:
>
> KRITA)  The "krita" repo was split off, because Krita has finally become a
> full project of its own, separate from Calligra. A logical place for the krita
> repo in the KDE repo structure would perhaps have been somewhere in extragear,
> but at least due to the translators preferring to keep the string catalogs of
> Krita in the "calligra" module as before, for less work, the

Re: State of Proposal to improving KDE Software Repository Organization?

2016-01-18 Thread Ben Cooksley 
On Tue, Jan 19, 2016 at 8:27 AM, Boudewijn Rempt  wrote:
> On Mon, 18 Jan 2016, Friedrich W. H. Kossebau wrote:
>
>> Reason that I ask is that due to the split of Calligra into several repos
>> (see background^) the layout in the repo structure does no longer properly
>> reflect the project organisation. Right now there are three active repos in
>> the calligra/ repo substructure:
>> "calligra" at "calligra/"
>> "krita"at "calligra/krita"
>> "kexi" at "calligra/kexi"
>
>
> What I'm wondering is, where is this "structure" actually visible? Not in
>
> https://quickgit.kde.org/

Quickgit shows the raw git repository structure, which deliberately
does not include the tree in it.

>
> or
>
> https://phabricator.kde.org/diffusion/

Eventually we'll have projects for each broader category (Multimedia,
Graphics, etc) and repositories will be tagged for those.
Phabricator will never provide a repository tree though (nor should
it, the existing tree is a hell of a maintenance nightmare).

>
> I see it reflected in the old, to be discarded
>
> https://projects.kde.org/projects
>
> But where else? And what is it actually needed for?

The build metadata depends on it, and it is used by:

- The CI system
- API / LXR / EBN
- Scripty.
- kdesrc-build

>
>>
>> (("calligra" at "calligra/" confuses at least kdesrc-build, sent an email
>> to mpyne about if moving it to "calligra/calligra" should fix it.))
>>
>> Things that are not properly matching organization:
>> * Krita starting with 3.* no longer is part of Calligra project
>>  (screws e.g. api.kde.org/bundled-apps-api/calligra-apidocs/ and also
>>  what people think to which project Krita belongs)
>> * Calligra & Krita are nowhere different to KDevelop, Digikam & Co,
>>  so no reason to be in a complete own toplevel structure,
>>  rather should be in the same sub structure, i.e. "Extragear",
>>  like extragear/calligra/* and extragear/graphics/krita
>>
>> More, not only Calligra & Krita related:
>> * "Extragear" is an awful grouping name for apps with individual
>>  release plans, a legacy term that no longer fits most of the apps
>>  in that substructure
>
>
> It's ghastly -- almost insulting. It's perpetuating the fallacy that
> there are core KDE projects and peripheral projects.

I agree. Which is why i'd like to see the Extragear moniker dropped.
If repositories are part of some broader release unit (like KDE
Applications - even if this does have a better name) then they still
need to visible as belonging to it though I think.

>
>> * "KDE Applications" is a misleading grouping name for apps with a
>>  central release plan, as if those with individual release plans
>>  are not "KDE" applications (as in, not done in the KDE community)
>
>
> Horrible as well.
>
>> * a single category per app as needed by the current tree structure layout
>>  of the repos, like "office", "graphics", "utils", is rather awkward,
>>  many apps do not match exactly one or would match multiple categories
>
>
> Honestly, the need to group repositories is, to me, so weird that I think it
> would be best to adopt the following scheme:

Note that "Frameworks", "KDevelop" and "KDE Telepathy" are all fairly
logical groupings of repositories (and things would be completely
unmanagable in so many ways if we didn't have these).

>
> a/amarok
> a/...
> ...
> c/calligra
> g/gcompris
> k/krita
>
> And so on. It's meaningless as it is; it would be better to make that clear,
> if grouping is really needed.
>
>> So IMHO some update of the repository organisation would be good, to
>> reflect how things are these days.
>> Renaming of "Extragear" and "KDE Applications" is surely something which
>> needs care from promo/marketing/VDG people first to find if that makes sense
>> at all and what a good solution would be.
>
>
> That again begs the question: where is the "organization" which apparently
> has
> purely technical reasons visible to contributors and users?
>
>> (Being both maintainer of Okteta, which is in "KDE Applications", and
>> meta-co-
>> maintainer of Calligra, which is not, but still done in the very same KDE
>> community, that current naming seems so wrong to me).
>>
>> But the actual names and grouping aside, for the pure technical renewing
>> (which also involves all infrastructure like translation system,
>> documentation, phabricator, etc), who is currently planning or working on
>> what?
>> So does it makes sense to wait some more, or should we assume the current
>> organization stays for longer, and Calligra & Krita repos should be moved
>> inside that organization for now?
>>
>>
>> ^Some background about Calligra repo split, as things are slightly
>> complicated:
>>
>> KRITA)  The "krita" repo was split off, because Krita has finally become a
>> full project of its own, separate from Calligra. A logical place for the
>> krita repo in the KDE repo structure would perhaps have been somewhere in
>> extragear, but at least due to the translators preferring to keep

Re: State of Proposal to improving KDE Software Repository Organization?

2016-01-18 Thread Jaroslaw Staniek 
On 18 January 2016 at 20:27, Boudewijn Rempt  wrote:

> On Mon, 18 Jan 2016, Friedrich W. H. Kossebau wrote:
>
> Reason that I ask is that due to the split of Calligra into several repos
>> (see background^) the layout in the repo structure does no longer properly
>> reflect the project organisation. Right now there are three active repos in
>> the calligra/ repo substructure:
>> "calligra" at "calligra/"
>> "krita"at "calligra/krita"
>> "kexi" at "calligra/kexi"
>>
>
​Thanks ​Friedrich, as always you introduce great amount of structure and
logic to KDE projects :)

It's even a bit more interesting that that; there are sub-sub-projects
playground/libs/kdb
playground/libs/kproperty
playground/libs/kreport​


- all historically and by heart ​being part of Calligra. In the future
Calligra _initiative_ can contain repos from any "category", why not?
Everyone can. See below for simple solutions to have that.


> What I'm wondering is, where is this "structure" actually visible? Not in
> ​​
>
> https://quickgit.kde.org/
>
> or
>
> https://phabricator.kde.org/diffusion/
>
> I see it reflected in the old, to be discarded
>
> https://projects.kde.org/projects
>
> But where else? And what is it actually needed for?
>

​build.kde.org's config (I remember that only because ​

​today I edited it: https://git.reviewboard.kde.org/r/126797 )​
Is this visible on the web page? No idea.
e.g. https://build.kde.org/view/Calligra/ groups some Calligra builds with
direct deps, that's all.

I know chiliproject.org that's used for projects.kde.org would better be
not patched. I hope this can be solved somehow and we can model our KDE
structures by our tools, not the other way round.


>
>> (("calligra" at "calligra/" confuses at least kdesrc-build, sent an email
>> to mpyne about if moving it to "calligra/calligra" should fix it.))
>>
>> Things that are not properly matching organization:
>> * Krita starting with 3.* no longer is part of Calligra project
>>  (screws e.g. api.kde.org/bundled-apps-api/calligra-apidocs/ and also
>>  what people think to which project Krita belongs)
>> * Calligra & Krita are nowhere different to KDevelop, Digikam & Co,
>>  so no reason to be in a complete own toplevel structure,
>>  rather should be in the same sub structure, i.e. "Extragear",
>>  like extragear/calligra/* and extragear/graphics/krita
>>
>> More, not only Calligra & Krita related:
>> * "Extragear" is an awful grouping name for apps with individual
>>  release plans, a legacy term that no longer fits most of the apps
>>  in that substructure
>>
>
> It's ghastly -- almost insulting. It's perpetuating the fallacy that
> there are core KDE projects and peripheral projects.
>

​Trees are dead. ​

I'd suggest a flat structure:
- relations between apps is a graph not a tree anymore (Kexi can be both in
office /productivity category as well as software development like KDevelop)
- it's 2016, people search, not browse

Or if categorization is needed, on top of the flat structure, tags are the
real means for that that people understand

There are app description formats: .lsm, .appdata.xml... I use them for
Kexi. Some others too. .lsm supports keywords.

I think semantic tags would be best (or do we use them already?).
A repo can be in a subproject and also belong to a number of categories.


>
> * "KDE Applications" is a misleading grouping name for apps with a
>>  central release plan, as if those with individual release plans
>>  are not "KDE" applications (as in, not done in the KDE community)
>>
>
> Horrible as well.
>

Yes, it's a surprise even to KDE people. Here, Calligra and KDevelop, to
name just them, are not KDE Apps to normal people.


>
> * a single category per app as needed by the current tree structure layout
>>  of the repos, like "office", "graphics", "utils", is rather awkward,
>>  many apps do not match exactly one or would match multiple categories
>>
>
> Honestly, the need to group repositories is, to me, so weird that I think
> it would be best to adopt the following scheme:
>
> a/amarok
> a/...
> ...
> c/calligra
> g/gcompris
> k/krita
>
> And so on. It's meaningless as it is; it would be better to make that
> clear,
> if grouping is really needed.
>
> So IMHO some update of the repository organisation would be good, to
>> reflect how things are these days.
>> Renaming of "Extragear" and "KDE Applications" is surely something which
>> needs care from promo/marketing/VDG people first to find if that makes
>> sense at all and what a good solution would be.
>>
>
> That again begs the question: where is the "organization" which apparently
> has
> purely technical reasons visible to contributors and users?
>
> (Being both maintainer of Okteta, which is in "KDE Applications", and
>> meta-co-
>> maintainer of Calligra, which is not, but still done in the very same KDE
>> community, that current naming seems so wrong to me).
>>
>> But the actual names and grouping aside, for the pure technical

Re: Proposal to improving KDE Software Repository Organization

2015-08-18 Thread Ben Cooksley 
On Mon, Aug 17, 2015 at 8:15 PM, David Faure fa...@kde.org wrote:
 On Sunday 16 August 2015 23:36:33 Luigi Toscano wrote:
 David Faure ha scritto:
  On Sunday 16 August 2015 13:51:29 Michael Pyne wrote:
  There's no reason even with our current build metadata that we'd *have* to
  have project hierarchies, as long as each underlying git repository name
  remains unique. It might even make things easier since there would be no 
  way
  for a sub-path in our project hierarchy (such as kde/kdelibs/kactivities) 
  to
  mask a git repository name (kdelibs in this case).
 
  Ben and I discussed it today and we think there is usefulness in one level 
  of subtree within the
  Applications product, to be able to keep the 'groups' like kdegraphics, 
  kdemultimedia etc. which
  are useful in order to have a maintainer per 'group' (as reinforced by the 
  release team recently).
 
  But yes, only one level, and AFAICS only useful in Applications.
  kactivities (to pick your example) would be at the root of Frameworks, 
  no sub-path needed.
 
 Does it mean a giant big blob for extragear and playground? Translation-wise,
 having the 'groups' is really useful to not get lost.

 Also, when phabricator support subproject, using groups would be useful again
 to not have a big blob of projects (it was one of the few complains I 
 recorded
 for phabricator, the big list of projects).

 OK, so more products with repos organized in sub-paths, makes sense to me.

Note that i'm not sure it makes sense for playground repositories to
be in there - as they're alpha code we're not yet distributing.

As a general matter of policy we don't allow Playground projects to be
released - they have to go through review (and end up in
Extragear/Applications/Plasma/Frameworks) first.
We don't offer CI coverage usually either.

In terms of Extragear, divisions / products is more of a release
unit so locking them all together in one item is bound to cause
problems but i'm on the fence as to how to handle these as having one
for each would be a bit overkill I think. We probably need to go
through Extragear at some point and sort the maintained from the
unmaintained...


 --
 David Faure, fa...@kde.org, http://www.davidfaure.fr
 Working on KDE Frameworks 5

Cheers,
Ben


 ___
 Kde-frameworks-devel mailing list
 kde-frameworks-de...@kde.org
 https://mail.kde.org/mailman/listinfo/kde-frameworks-devel

Re: Proposal to improving KDE Software Repository Organization

2015-08-18 Thread Ben Cooksley 
On Mon, Aug 17, 2015 at 9:36 AM, Luigi Toscano luigi.tosc...@tiscali.it wrote:
 David Faure ha scritto:
 On Sunday 16 August 2015 13:51:29 Michael Pyne wrote:
 There's no reason even with our current build metadata that we'd *have* to
 have project hierarchies, as long as each underlying git repository name
 remains unique. It might even make things easier since there would be no way
 for a sub-path in our project hierarchy (such as kde/kdelibs/kactivities) to
 mask a git repository name (kdelibs in this case).

 Ben and I discussed it today and we think there is usefulness in one level 
 of subtree within the
 Applications product, to be able to keep the 'groups' like kdegraphics, 
 kdemultimedia etc. which
 are useful in order to have a maintainer per 'group' (as reinforced by the 
 release team recently).

 But yes, only one level, and AFAICS only useful in Applications.
 kactivities (to pick your example) would be at the root of Frameworks, no 
 sub-path needed.

 Does it mean a giant big blob for extragear and playground? Translation-wise,
 having the 'groups' is really useful to not get lost.

 Also, when phabricator support subproject, using groups would be useful again
 to not have a big blob of projects (it was one of the few complains I recorded
 for phabricator, the big list of projects).

Note that we won't be pulling any form of metadata for any structure
or organisation from Phabricator.
It'll be stored in a separate file and referenced appropriately.

As for Phabricator, i'd prefer using Group and Product projects to
organise things. You can then execute searches for all repositories
that are in a given set of projects.

eg: Kdenlive would be in #Extragear and #Multimedia. So you could find
all #Multimedia applications by searching for repositories which
belong to that project (and so forth).

Subprojects are not necessary here from my perspective. Subprojects
become useful when you have an overall project (see Plasma / Krita for
instance) which have the need for several boards to organise their
work - and thus have several projects.


 Ciao
 --
 Luigi

Regards,
Ben

Re: Proposal to improving KDE Software Repository Organization

2015-08-17 Thread David Faure 
On Sunday 16 August 2015 23:36:33 Luigi Toscano wrote:
 David Faure ha scritto:
  On Sunday 16 August 2015 13:51:29 Michael Pyne wrote:
  There's no reason even with our current build metadata that we'd *have* to 
  have project hierarchies, as long as each underlying git repository name 
  remains unique. It might even make things easier since there would be no 
  way 
  for a sub-path in our project hierarchy (such as kde/kdelibs/kactivities) 
  to 
  mask a git repository name (kdelibs in this case).
  
  Ben and I discussed it today and we think there is usefulness in one level 
  of subtree within the
  Applications product, to be able to keep the 'groups' like kdegraphics, 
  kdemultimedia etc. which
  are useful in order to have a maintainer per 'group' (as reinforced by the 
  release team recently).
  
  But yes, only one level, and AFAICS only useful in Applications.
  kactivities (to pick your example) would be at the root of Frameworks, no 
  sub-path needed.
  
 Does it mean a giant big blob for extragear and playground? Translation-wise,
 having the 'groups' is really useful to not get lost.
 
 Also, when phabricator support subproject, using groups would be useful again
 to not have a big blob of projects (it was one of the few complains I recorded
 for phabricator, the big list of projects).

OK, so more products with repos organized in sub-paths, makes sense to me.

-- 
David Faure, fa...@kde.org, http://www.davidfaure.fr
Working on KDE Frameworks 5

Re: Proposal to improving KDE Software Repository Organization

2015-08-16 Thread David Faure 
On Monday 18 August 2014 21:54:40 Michael Pyne wrote:
 
 Overview of Proposed Fix
 
 
 What we would like to do instead is the classic Comp. Sci. fix: Another layer
 of indirection.
 
 In this case, we'd like to re-organize the `kde-build-metadata` to map to the
 same types of project divisions that we already intuitively utilize ourselves
 (i.e.  the repositories that make up Plasma 5 are a different grouping than
 those that make up KDE Frameworks 5, which are different from those that make
 up KDevelop for KF5, etc.).
 
 Under this scheme, the universe of all (KDE.org) git repositories would fall
 into this outline:
 
 + Division (e.g. KF5)
  + Track   (e.g. devel)
   + Repositories + Git branches
 
 The following would be true of these divisions:
 
 * Each division/track combination could depend on a different division (e.g.
   Plasma5/Devel could depend on KF5/Devel).
 
 * Each division/track combination would list all git repositories that make up
   that track (wildcards will continue to be permitted), along with the git
   branch of that repository. E.g. Plasma5/Devel could include
   kde/workspace/plasma-workspace: master, while Plasma5/Stable might include
   kde/workspace/plasma-workspace: Plasma/5.0.
 
 * The branch group concept will be retained (both for backwards compat for
   kdesrc-build users and for ease of Jenkins implementation), and is the most
   global grouping (but now, of divisions, not repositories directly).  Each
   division will map global branch group names to one of its tracks, if
   appropriate.
 
   So kf5-qt5 might mean KF5/Devel, Plasma5/Devel, etc. while
   kf5-qt5-stable might mean KF5/Devel, Plasma5/Stable, etc.. If CI builds
   kf5-qt5-stable and then builds kf5-qt5, it would be able to skip 
 building
   KF5/Devel the second time as it's stated to be compatible with both 
 Plasma5
   tracks.
 
 * Any given repository in a branch group would map to 0-1 divisions. 0, since 
 a
   repository simply might not be present at all (and might even be in 
 different
   divisions for different global branch groups...). 1, since there must be 
 only
   1 possible git branch name for a repository.
 
 * Instead of using a separate dependency file, intra-division dependencies
   would be listed along with the rest of the division/track details.
 
 * Likewise, inter-division dependencies would be supported (but the dependency
   would only be on the repository names, since the branches for that 
 repository
   would be controlled by the division/track combination). This is to allow for
   smaller applications that depend on only a couple of Tier 1 KF5 repositories
   to be tested without building all 50+ KF5 modules too.
 
 * You can also simply depend on a division/track combo as a whole, without
   listing each individual dependency (similar to how many apps now depend on
   the virtual kf5umbrella repository).
 
 * A division can specify that certain of its tracks are equivalent. For
   instance, FooApp/stable might only require Plasma5/stable, but work 
 perfectly
   fine with Plasma5/devel if it's already available, which is something 
 Plasma5
   can specify.  This helps reduce combinatorial explosion for the CI
   infrastructure.
 
 * Every repository would need to be a member of *some* Division/Track
   combination to be seen by CI, even small apps.

Re-reading all this, I feel that this would be very beneficial to have, in the 
light of more recent issues.

E.g. I struggle to make it possible to build all of KDE SC 4 from git, because 
every new
qt5-kf5-only repo is picked up by kdesrc-build until it's blacklisted with an 
empty branch name
in logical-module-structure. This shows the need to separate things some more, 
e.g.
with a KDE SC 4 division (*).

In addition, moves in projects.kde.org (e.g. to frameworks/*) affect (sometimes 
break)
the kde4 build as well, which shows the need for a per-product tree rather than 
global
(or even per-product per-track). I'm unsure about moving from everything gets 
built
to you need to add the new module to the right file for it to be built, 
though. People will
forget to do that, and kdesrc-build (including lxr) and CI will just ignore 
that module...
well I guess we just need to make it part of the procedure for new modules.

(*) I keep finding the division term a bit obscure, and I wonder if this 
shouldn't be
called product instead. I.e. matching how we release things. Nowadays we
basically have 4 products (frameworks, plasma, applications, extragear?),
previously we had 2 (KDE SC 4, extragear). If this is what you had in mind
with divisions, then I suggest to call it product for clarity.
The reason why I think it's the same concept is that the one reason to have
different tracks is exactly because of different release schedules (e.g. plasma
could be tested against KF5/Stable (last release) and KF5/Devel (more recent 
code))

-- 
David Faure, fa...@kde.org, http://www.davidfaure.fr
Working on

Re: Proposal to improving KDE Software Repository Organization

2015-08-16 Thread John Layt 
On 16 August 2015 at 11:14, David Faure fa...@kde.org wrote:

 (*) I keep finding the division term a bit obscure, and I wonder if this 
 shouldn't be
 called product instead. I.e. matching how we release things. Nowadays we
 basically have 4 products (frameworks, plasma, applications, extragear?),
 previously we had 2 (KDE SC 4, extragear). If this is what you had in mind
 with divisions, then I suggest to call it product for clarity.
 The reason why I think it's the same concept is that the one reason to have
 different tracks is exactly because of different release schedules (e.g. 
 plasma
 could be tested against KF5/Stable (last release) and KF5/Devel (more recent 
 code))

[Rambling naming bikeshed alert !]

TLDR:

We have a Marketing View and we have a Technical Build View and I
think they are different enough to have different names and
structures. How about we call 'division' something like 'build-group'
or 'build-unit' instead? And while we're busy regrouping and renaming
things, lets get rid of the application Modules...

Longer:

I like Product too, which is why I'm using it in the new TechBase KF5
Getting Started docs I started writing today [1] and especially [2],
but I don't see it as being the same thing as 'division' here, and I
think it exposes again a problem in our organisation and naming of
Applications.

My marketing-oriented view is we are organised as 3 Products:
* KDE Frameworks
* KDE Plasma
* KDE Applications

(We could add other products like 'KDE Infrastructure' and KDE
Servers', but they're mostly internal).

Within KDE Applications we then have a confusing mixture of Modules,
Projects and standalone Applications with different porting status and
release cycles:
* Edu Module
* Games Module
* PIM Module
* Playground Module
* Review Module
* Extragear Module
* Calligra Suite
* GCompris
* etc...

Now, some may argue that some of those are actually Products in their
own right, e.g. Calligra and Extragear and GCompris, but they are
Applications developed by the KDE Community within the KDE
Infrastructure and adhering to the KDE Manifesto, they are all by
definition KDE Applications, just with different development status or
release cycles.

'division' appears slightly different to me, it is about grouping
things into standalone build dependency units, so Calligra and
GCompris do appear to belong at that level. OTOH, do we really want
every repo in Extragear or Playground to have their own top-level
'division' just because they have different release cycles or
different porting status or different version dependencies? Even the
official KDE SC4 modules all have different porting status and thus
build dependencies and thus conceivably different 'divisions'. That
would just be too messy. I think how they get grouped into 'divisions'
is always going to be different than the marketing Products and
Modules. Alternative names that spring to mind are 'build-unit' or
'build-group'.

Personally I think the whole
Playground/Review/Applications/Extragear/Unmaintained module level
split needs to go away and all Applications be considered equal, just
with a different release status metadata tag based on the official
lifecycle [3]. It's just reflecting the reality of the git-based split
up of modules, different release cycles, death of a number of our
sub-communities, and a more-focussed view on what makes up the
workspace/applications split and their different release cycles.

So, a proposal: we drop modules and extragear and playground and
unmaintained altogether for organising our repos and paths and build
process and instead just have Applications with a 'release-status' tag
marking where they are in our official KDE Application Lifecycle [3].
A second 'release-cycle' tag could identify those that are to be
included in the regular KDE Applications release cycle.

We can still sort-of keep module, but now it's more a category tag, so
all edu apps live under the path apps/edu/*. This could also remove
some hassle when apps move around, their place in the namespace
hierarchy and path doesn't change, just their release status tag.

John.

[1] https://techbase.kde.org/KF5/Getting_Started
[2] https://techbase.kde.org/KF5/Getting_Started/Source_Code
[3] https://techbase.kde.org/Policies/Application_Lifecycle

Re: Proposal to improving KDE Software Repository Organization

2015-08-16 Thread Michael Pyne 
On Sun, August 16, 2015 17:48:59 John Layt wrote:
 On 16 August 2015 at 11:14, David Faure fa...@kde.org wrote:
  (*) I keep finding the division term a bit obscure, and I wonder if this
  shouldn't be called product instead. I.e. matching how we release
  things. Nowadays we basically have 4 products (frameworks, plasma,
  applications, extragear?), previously we had 2 (KDE SC 4, extragear). If
  this is what you had in mind with divisions, then I suggest to call it
  product for clarity.
  The reason why I think it's the same concept is that the one reason to
  have
  different tracks is exactly because of different release schedules (e.g.
  plasma could be tested against KF5/Stable (last release) and KF5/Devel
  (more recent code))

 [Rambling naming bikeshed alert !]

Just to be clear I agree with David that product would better match the 
intent of my proposal than division; I've spent the past year or so not 
being very satisfied with division.

 TLDR:
 
 We have a Marketing View and we have a Technical Build View and I
 think they are different enough to have different names and
 structures. How about we call 'division' something like 'build-group'
 or 'build-unit' instead? And while we're busy regrouping and renaming
 things, lets get rid of the application Modules...
 
snip
 So, a proposal: we drop modules and extragear and playground and
 unmaintained altogether for organising our repos and paths and build
 process and instead just have Applications with a 'release-status' tag
 marking where they are in our official KDE Application Lifecycle [3].
 A second 'release-cycle' tag could identify those that are to be
 included in the regular KDE Applications release cycle.

I think this is probably a good conversation to have as well but I do want to 
re-emphasize that the proposal David is referring to relates to how we build 
KDE software, and how we organize to build that software easily.

This involves grappling with at least a couple of things:

- What underlying source repositories make up a given major product that we 
ship? (e.g. what comprised Plasma 5? what makes up KF5?). We have to care at 
least about this level of granularity since each of these may have their own 
release cycles.

- For each of those repositories, what is the appropriate branch to build 
from, for a given development branch (e.g. stable or dev)? One of the 
weaknesses of the current branch-group scheme is that we list all KDE 
repositories *first* and only then try to map to a branch... but not every 
repository necessarily belongs with every major product.

But beyond that we already really don't care about extragear/, playground/, 
etc., at least as far as build infrastructure goes. The reason we'd split 
playground or Extragear in the new scheme would be because they operate on 
different release schedules from Plasma, Applications or KF5 (and therefore 
probably have different ideas of 'Release', 'Dev', 'Stable', etc.). For 
playground in particular it would also give us a way to lower the priority for 
those from a CI perspective. But we don't have to make that part of our 
marketing efforts and I'm not aware that we do even today.

 We can still sort-of keep module, but now it's more a category tag, so
 all edu apps live under the path apps/edu/*. This could also remove
 some hassle when apps move around, their place in the namespace
 hierarchy and path doesn't change, just their release status tag.

Well, having a project hierarchy is also a slightly different question. 
There's no reason even with our current build metadata that we'd *have* to 
have project hierarchies, as long as each underlying git repository name 
remains unique. It might even make things easier since there would be no way 
for a sub-path in our project hierarchy (such as kde/kdelibs/kactivities) to 
mask a git repository name (kdelibs in this case).

If we got rid of this it would take away the ability to easily refer 
collectively to related modules though, and I don't think we'd want to 
reimplement that with the high-level Project scheme being talked about.

Regards,
 - Michael Pyne

Re: Proposal to improving KDE Software Repository Organization

2015-08-16 Thread Allen Winter 
On Sunday, August 16, 2015 11:21:00 PM David Faure wrote:
 On Sunday 16 August 2015 13:51:29 Michael Pyne wrote:
  There's no reason even with our current build metadata that we'd *have* to 
  have project hierarchies, as long as each underlying git repository name 
  remains unique. It might even make things easier since there would be no 
  way 
  for a sub-path in our project hierarchy (such as kde/kdelibs/kactivities) 
  to 
  mask a git repository name (kdelibs in this case).
 
 Ben and I discussed it today and we think there is usefulness in one level of 
 subtree within the
 Applications product, to be able to keep the 'groups' like kdegraphics, 
 kdemultimedia etc. which
 are useful in order to have a maintainer per 'group' (as reinforced by the 
 release team recently).
 
This is indeed what we use to call modules, no?
And what was earlier in this thread referred to as Products is what we used 
to call Components
Only in Applications I suppose, but nevertheless this is coming back around to 
what we had. which I liked.


 But yes, only one level, and AFAICS only useful in Applications.
 kactivities (to pick your example) would be at the root of Frameworks, no 
 sub-path needed.

Re: Proposal to improving KDE Software Repository Organization

2015-08-16 Thread David Faure 
On Sunday 16 August 2015 13:51:29 Michael Pyne wrote:
 There's no reason even with our current build metadata that we'd *have* to 
 have project hierarchies, as long as each underlying git repository name 
 remains unique. It might even make things easier since there would be no way 
 for a sub-path in our project hierarchy (such as kde/kdelibs/kactivities) to 
 mask a git repository name (kdelibs in this case).

Ben and I discussed it today and we think there is usefulness in one level of 
subtree within the
Applications product, to be able to keep the 'groups' like kdegraphics, 
kdemultimedia etc. which
are useful in order to have a maintainer per 'group' (as reinforced by the 
release team recently).

But yes, only one level, and AFAICS only useful in Applications.
kactivities (to pick your example) would be at the root of Frameworks, no 
sub-path needed.

-- 
David Faure, fa...@kde.org, http://www.davidfaure.fr
Working on KDE Frameworks 5

Re: Proposal to improving KDE Software Repository Organization

2014-08-19 Thread David Faure 
Nice work.

Just one thing:

On Monday 18 August 2014 21:54:40 Michael Pyne wrote:
   So kf5-qt5 might mean KF5/Devel, Plasma5/Devel, etc. while
   kf5-qt5-stable might mean KF5/Devel, Plasma5/Stable, etc..

This looks like an attempt to keep the current branch-group naming for 
compatibility and ease of transition. But I fear it makes things harder to 
understand in the longer term.
Would who guess that kf5-qt5 means plasma devel, since the name says nothing 
about plasma?

I think the concept works, but the actual naming of the divisions should be 
improved, even if it means we need to update our kdesrc-buildrc files (or some 
compat code maps from old names to new names).

Plasma5Devel-KF5 and Plasma5Stable-KF5 would already be better names for the 
divisions  but then what happens with apps on top? :)
I guess the divisions will include apps in the same state as plasma?
So this is really AppsAndPlasmaDevel-KF5 and AppsAndPlasmaStable-KF5,
both quite unreadable :)
(Just Devel-KF5 and Stable-KF5 would make one thing the adjective applies to 
KF5, so no go)

Sorry for starting a naming bikeshed, but I fear that the divison concept 
won't work out well if we can't find proper names for the divisions, because 
then people will keep getting confused about what's in them.

-- 
David Faure, fa...@kde.org, http://www.davidfaure.fr
Working on KDE Frameworks 5

Re: Proposal to improving KDE Software Repository Organization

2014-08-19 Thread Ben Cooksley 
On Tue, Aug 19, 2014 at 6:55 PM, David Faure fa...@kde.org wrote:
 Nice work.

Thanks.


 Just one thing:

 On Monday 18 August 2014 21:54:40 Michael Pyne wrote:
   So kf5-qt5 might mean KF5/Devel, Plasma5/Devel, etc. while
   kf5-qt5-stable might mean KF5/Devel, Plasma5/Stable, etc..

 This looks like an attempt to keep the current branch-group naming for
 compatibility and ease of transition. But I fear it makes things harder to
 understand in the longer term.

It is for compatibility - yes.
The old kf5-qt5 / latest-qt4 names are being mapped to division /
track combinations. They are otherwise not used.

 Would who guess that kf5-qt5 means plasma devel, since the name says nothing
 about plasma?

 I think the concept works, but the actual naming of the divisions should be
 improved, even if it means we need to update our kdesrc-buildrc files (or some
 compat code maps from old names to new names).

 Plasma5Devel-KF5 and Plasma5Stable-KF5 would already be better names for the
 divisions  but then what happens with apps on top? :)

Each application grouping would have it's own division.

Just a clarification though: there would only be two divisions for the
above scenario: Plasma5 and KF5.
Plasma5 would then have two tracks: stable and devel. KF5 would have
it's single track.

 I guess the divisions will include apps in the same state as plasma?
 So this is really AppsAndPlasmaDevel-KF5 and AppsAndPlasmaStable-KF5,
 both quite unreadable :)
 (Just Devel-KF5 and Stable-KF5 would make one thing the adjective applies to
 KF5, so no go)

 Sorry for starting a naming bikeshed, but I fear that the divison concept
 won't work out well if we can't find proper names for the divisions, because
 then people will keep getting confused about what's in them.

KF5 will have the frameworks in it.
Plasma5 will have those repositories which form part of the Plasma 5
workspace, and are released as Plasma 5.
Everything else will go in other divisions.

KDevelop for instance would probably have it's own division. As would
Calligra, even though it is only one repository. I imagine PIM would
have it's own as well (covering kdepim and kdepim-runtime).

How the current generation modules group themselves as divisions would
be up to them ultimately.


 --
 David Faure, fa...@kde.org, http://www.davidfaure.fr
 Working on KDE Frameworks 5


Thanks,
Ben

Re: Proposal to improving KDE Software Repository Organization

2014-08-19 Thread David Faure 
On Tuesday 19 August 2014 19:10:14 Ben Cooksley wrote:
 The old kf5-qt5 / latest-qt4 names are being mapped to division /
 track combinations. They are otherwise not used.

Ah!

 Just a clarification though: there would only be two divisions for the
 above scenario: Plasma5 and KF5.
 Plasma5 would then have two tracks: stable and devel. KF5 would have
 it's single track.

Ah!

OK it's a lot clearer to me now.

I thought a division was an overall selection of everything
(like kf5-qt5 was).
Now I see, it's a group of modules, e.g. just KF5, or just Plasma5.
This makes a lot more sense, and leads to understandable naming, I like it :)

I assume a future kdesrc-buildrc will look like a selection of *one or many*
divisions, with a track for each one, then.

All sounds very good, +1.

-- 
David Faure, fa...@kde.org, http://www.davidfaure.fr
Working on KDE Frameworks 5

Re: Proposal to improving KDE Software Repository Organization

2014-08-19 Thread Aleix Pol 
On Tue, Aug 19, 2014 at 3:54 AM, Michael Pyne mp...@kde.org wrote:

 Hi all,

 Ben Cooksley and I would like to get some feedback on further evolutions to
 the organization structure we employ for the repositories at git.kde.org,
 to
 allow our current usage of CI even as we move farther into the KF5-based
 world.

 TL;DR: More indirection in our JSON in kde-build-metadata, not a lot of
 end-
 user visible change, new org. terms: Division and Track for multi-repo
 organization, tracking inter-repo dependencies would change too (sayonara
 dependency-data-$branch_group), less CI servers turned into melting piles
 of
 slag. +1?

 The proposal follows for those who like reading excessively wordy text.

 Regards,
  - Michael Pyne

 Improving KDE Project Organization: A Proposal
 ==

 18 Aug 2014

 Michael Pyne mp...@kde.org and Ben Cooksley bcooks...@kde.org

 This is a proposal to evolve the current method of organizing our mass of
 KDE
 source code repositories, and their dependencies, as contained in the
 `kde-build-metadata` repository and used by kdesrc-build and build.kde.org
 (referred to as CI). This is needed in order to correct some
 deficiencies in
 the [current
 specification](https://community.kde.org/Infrastructure/Project_Metadata),
 and
 to help better support changing trends in developer workflow.

 Current Situation
 =

 If you're familiar with the current organization of KDE build metadata
 you
 should skip to the next section.

 Currently, the git-based source code repositories that make up KDE.org's
 software releases are each given a project path that fully specifies the
 name
 of the module in a virtual hierarchy. For instance, kdesrc-build itself is
 really extragear/utils/kdesrc-build, and KDE 4's kdelibs is
 kde/kdelibs.

 Since many modules support KDE4 and Qt5/KF5 (or may in the future), some
 developers associated with KDE source code repositories introduced the
 branch
 group construct, that maps the git repository branch for the majority of
 repositories into a few broad groupings, such as stable-qt4, latest-qt4
 and
 kf5-qt5. Developers and users using kdesrc-build could then use these
 groups
 to easily build the appropriate git branch of the many repositories needed
 for
 current releases of KDE.org software. This also allowed the CI
 infrastructure
 to support testing the development branches of both software using both
 KDE4 and KF5, in addition to the libraries/Frameworks themselves.

 Current Issues
 ==

 Things have gone fairly well with branch groups, but there have been minor
 issues with the construct:

 1. The existing metadata listing dependencies between git repositories
 could
not support multiple branch groups, as the dependencies were not
 necessarily
identical for a given repository, for every possible branch group it
belonged to. We worked around this by forking the metadata such that
 each
different branch group used a separate dependency file.

 2. Compounding that issue, different branch groups would have different
 sets
of repositories. For instance some repositories will never have a
 KF5-based
release due to ongoing reorganization, and many repositories were born
 for
KF5. By common agreement, software using `kde-build-metadata` now
 recognize
empty git branch names to mean that a repository doesn't actually
 belong to
the given branch group. This is still a workaround, however; if we
 forget
to manually specify an empty branch, then CI and kdesrc-build will both
try to build that repository as part of that branch group (using a
 default
branch).

 Upcoming Problems
 =

 A larger concern (and what instigated this effort) is that the KF5 era will
 introduce multiple development models that are difficult for the CI
 infrastructure to efficiently support.

 For example, testing the KF5-based Plasma 5 Workspace will eventually need
 to
 test both the stable and development tracks for Plasma 5. Under the branch
 group concept, this would lead to branch groups kf5-qt5 and
 kf5-qt5-stable
 (or similar names).

 However the KF5 repositories that Plasma 5 requires do not have a split
 between
 stable and devel: They use a review-required process by which there's only
 one
 development track. In other words, Plasma 5's two development tracks will
 only
 depend on 1 KF5 track.

 At this time, that means CI will have to build 56 KF5 modules to test
 Plasme
 5-stable, and then re-build, re-install, etc. the exact same 56 modules to
 then
 test Plasma 5-devel. This re-build is required because experience has shown
 that built repositories cannot be assumed to be compatible between
 different
 branch groups (in fact many repositories are significantly different
 on-disk
 between branch groups). There's simply no data recorded at this point that
 delimits the ways in which repositories would remain compatible (or not)
 between different

Re: Proposal to improving KDE Software Repository Organization

2014-08-19 Thread Michael Pyne 
On Tue, August 19, 2014 10:18:17 David Faure wrote:
 On Tuesday 19 August 2014 19:10:14 Ben Cooksley wrote:
  The old kf5-qt5 / latest-qt4 names are being mapped to division /
  track combinations. They are otherwise not used.
 
 Ah!
 
  Just a clarification though: there would only be two divisions for the
  above scenario: Plasma5 and KF5.
  Plasma5 would then have two tracks: stable and devel. KF5 would have
  it's single track.
 
 Ah!
 
 OK it's a lot clearer to me now.
 
 I thought a division was an overall selection of everything
 (like kf5-qt5 was).
 Now I see, it's a group of modules, e.g. just KF5, or just Plasma5.
 This makes a lot more sense, and leads to understandable naming, I like it
 :)
 
 I assume a future kdesrc-buildrc will look like a selection of *one or many*
 divisions, with a track for each one, then.

Yes, and branch groups simply become a pre-tested set of divisions (I would 
imagine it simply defines the various combinations CI would test in the 
future, though that concept is still up to Ben).

In the kdesrc-build world you'd still be able to pick divisions (and tracks), 
individual modules, etc.

There's two feasible ways to go with the branch-group option though: It is 
used to pick the right branch or track if not specified, or it is used as CI 
would use it, to pull in an entire set of divisions in one fell swoop. I'm 
open to other ideas, but that's a topic for a separate thread in any event.

Regards,
 - Michael Pyne

Re: Proposal to improving KDE Software Repository Organization

2014-08-19 Thread Ben Cooksley 
On Wed, Aug 20, 2014 at 9:39 AM, Michael Pyne mp...@kde.org wrote:
 On Tue, August 19, 2014 10:18:17 David Faure wrote:
 On Tuesday 19 August 2014 19:10:14 Ben Cooksley wrote:
  The old kf5-qt5 / latest-qt4 names are being mapped to division /
  track combinations. They are otherwise not used.

 Ah!

  Just a clarification though: there would only be two divisions for the
  above scenario: Plasma5 and KF5.
  Plasma5 would then have two tracks: stable and devel. KF5 would have
  it's single track.

 Ah!

 OK it's a lot clearer to me now.

 I thought a division was an overall selection of everything
 (like kf5-qt5 was).
 Now I see, it's a group of modules, e.g. just KF5, or just Plasma5.
 This makes a lot more sense, and leads to understandable naming, I like it
 :)

 I assume a future kdesrc-buildrc will look like a selection of *one or many*
 divisions, with a track for each one, then.

 Yes, and branch groups simply become a pre-tested set of divisions (I would
 imagine it simply defines the various combinations CI would test in the
 future, though that concept is still up to Ben).

 In the kdesrc-build world you'd still be able to pick divisions (and tracks),
 individual modules, etc.

 There's two feasible ways to go with the branch-group option though: It is
 used to pick the right branch or track if not specified, or it is used as CI
 would use it, to pull in an entire set of divisions in one fell swoop. I'm
 open to other ideas, but that's a topic for a separate thread in any event.

The CI system will probably end up relying on divisions directly in
the long run from what I can see at this point.
As an interim measure though, it is likely the branch group concept
will continue to be used though.

I have some other questions for those working on Windows/OS X/Android
CI, but that is a topic for another thread :)


 Regards,
  - Michael Pyne

Thanks,
Ben

Re: Proposal to improving KDE Software Repository Organization

2014-08-19 Thread Àlex Fiestas 
On Monday 18 August 2014 21:54:40 Michael Pyne wrote:
 We await your comments, suggestions, clarification requests, and other
 feedback.
The proposed solution will clearly help to improve the situation, so +1!

Something I would like to explore is the possibility of putting on each 
repository the information relative to it, and then maybe via jenkins generate 
the unified file.

The pros of this is that we have the information about each project contain on 
itself making it easier for maintainers to keep the information up to date and 
also making it easier to discover this information (not many people know about 
kde-build-metadata and surely not third party developers).

Cheers!

signature.asc
Description: This is a digitally signed message part.

Re: Proposal to improving KDE Software Repository Organization

2014-08-19 Thread Michael Pyne 
On Wed, August 20, 2014 00:48:36 Àlex Fiestas wrote:
 On Monday 18 August 2014 21:54:40 Michael Pyne wrote:
  We await your comments, suggestions, clarification requests, and other
  feedback.
 
 The proposed solution will clearly help to improve the situation, so +1!
 
 Something I would like to explore is the possibility of putting on each
 repository the information relative to it, and then maybe via jenkins
 generate the unified file.
 
 The pros of this is that we have the information about each project contain
 on itself making it easier for maintainers to keep the information up to
 date and also making it easier to discover this information (not many
 people know about kde-build-metadata and surely not third party

Actually this was my initial idea. I had even tried to sell the sysadmins on 
the idea of expanding the information recorded about a project in the 
projects.kde.org interface to allow for this.

However we ended up noting some issues. Eventually, the idea at the creation 
of kde-build-metadata was that this information would be centralized there 
(not at the repo) so that maintainer action would not be required to make 
things work. Not to say that maintainer action is not desired; just that it 
wouldn't be mandated to make things work right.

Although it's true that moving data to the repo doesn't mean errors couldn't 
be fixed by KDE developers who happen to notice it, that would require cloning 
the whole repo (since it's fairly likely that it's not already be checked 
out), making the fix, pushing it, you know the drill. With kde-build-metadata 
you already have it checked out by definition, and the ones most likely to 
notice a test failure know where to make the fix (or at least where to point 
the repo's maintainer).

Additionally there's actually a couple of scripts in kde-build-metadata's 
tools/ subdirectory that would be much less useful if they had to wait for 
Jenkins to re-generate the centralized file, and these tools are useful for 
validating the JSON format used and proving that the dependencies are as 
expected, so I think that would increase the odds of introducing errors by 
accident.

As it stands I don't think discoverability is a big concern here. If a 
maintainer is making a repo from a template, we could add a pointer to the 
right repo in the template itself. If a maintainer is already maintaining a 
repository, they'd have to hear the news to know to add a new metadata file, 
but if they're able to receive that news it's easy enough to point them to 
kde-build-metadata instead.

The one problem is developers making a new repository by simply copying the 
structure of an old one, but then they'd hear about dependency data when their 
new repo doesn't work on Jenkins. ;)

With all that said there's no reason we can't make the data in kde-build-
metadata more easily available to developers in a more-preferred location 
(maybe at projects.kde.org?), but I don't think I'll be the one implementing 
it.

Regards,
 - Michael Pyne

signature.asc
Description: This is a digitally signed message part.

Proposal to improving KDE Software Repository Organization

2014-08-18 Thread Michael Pyne 
Hi all,

Ben Cooksley and I would like to get some feedback on further evolutions to 
the organization structure we employ for the repositories at git.kde.org, to 
allow our current usage of CI even as we move farther into the KF5-based 
world.

TL;DR: More indirection in our JSON in kde-build-metadata, not a lot of end-
user visible change, new org. terms: Division and Track for multi-repo 
organization, tracking inter-repo dependencies would change too (sayonara 
dependency-data-$branch_group), less CI servers turned into melting piles of 
slag. +1?

The proposal follows for those who like reading excessively wordy text.

Regards,
 - Michael Pyne

Improving KDE Project Organization: A Proposal
==

18 Aug 2014

Michael Pyne mp...@kde.org and Ben Cooksley bcooks...@kde.org

This is a proposal to evolve the current method of organizing our mass of KDE
source code repositories, and their dependencies, as contained in the
`kde-build-metadata` repository and used by kdesrc-build and build.kde.org
(referred to as CI). This is needed in order to correct some deficiencies in
the [current
specification](https://community.kde.org/Infrastructure/Project_Metadata), and
to help better support changing trends in developer workflow.

Current Situation
=

If you're familiar with the current organization of KDE build metadata you
should skip to the next section.

Currently, the git-based source code repositories that make up KDE.org's
software releases are each given a project path that fully specifies the 
name
of the module in a virtual hierarchy. For instance, kdesrc-build itself is
really extragear/utils/kdesrc-build, and KDE 4's kdelibs is kde/kdelibs.

Since many modules support KDE4 and Qt5/KF5 (or may in the future), some
developers associated with KDE source code repositories introduced the branch
group construct, that maps the git repository branch for the majority of
repositories into a few broad groupings, such as stable-qt4, latest-qt4 
and
kf5-qt5. Developers and users using kdesrc-build could then use these groups
to easily build the appropriate git branch of the many repositories needed for
current releases of KDE.org software. This also allowed the CI infrastructure
to support testing the development branches of both software using both
KDE4 and KF5, in addition to the libraries/Frameworks themselves.

Current Issues
==

Things have gone fairly well with branch groups, but there have been minor
issues with the construct:

1. The existing metadata listing dependencies between git repositories could
   not support multiple branch groups, as the dependencies were not 
necessarily
   identical for a given repository, for every possible branch group it
   belonged to. We worked around this by forking the metadata such that each
   different branch group used a separate dependency file.

2. Compounding that issue, different branch groups would have different sets
   of repositories. For instance some repositories will never have a KF5-based
   release due to ongoing reorganization, and many repositories were born for
   KF5. By common agreement, software using `kde-build-metadata` now recognize
   empty git branch names to mean that a repository doesn't actually belong to
   the given branch group. This is still a workaround, however; if we forget
   to manually specify an empty branch, then CI and kdesrc-build will both
   try to build that repository as part of that branch group (using a default
   branch).

Upcoming Problems
=

A larger concern (and what instigated this effort) is that the KF5 era will
introduce multiple development models that are difficult for the CI
infrastructure to efficiently support.

For example, testing the KF5-based Plasma 5 Workspace will eventually need to
test both the stable and development tracks for Plasma 5. Under the branch
group concept, this would lead to branch groups kf5-qt5 and kf5-qt5-stable
(or similar names).

However the KF5 repositories that Plasma 5 requires do not have a split 
between
stable and devel: They use a review-required process by which there's only one
development track. In other words, Plasma 5's two development tracks will only
depend on 1 KF5 track.

At this time, that means CI will have to build 56 KF5 modules to test Plasme
5-stable, and then re-build, re-install, etc. the exact same 56 modules to 
then
test Plasma 5-devel. This re-build is required because experience has shown
that built repositories cannot be assumed to be compatible between different
branch groups (in fact many repositories are significantly different on-disk
between branch groups). There's simply no data recorded at this point that
delimits the ways in which repositories would remain compatible (or not)
between different branch group combinations.

Solving this (so that the right 56 modules are retained and re-used) would
require quite some manual hackery, and it's uncertain how easy