Re: Sysadmin Load Reduction: Code Related Services
пн, 18 нояб. 2019 г. в 15:35, Harald Sitter : > > On Sat, Nov 16, 2019 at 9:40 PM Ben Cooksley wrote: > > > > On Sun, Nov 17, 2019 at 5:19 AM Carl Schwan wrote: > > > > > > Hi all, > > > > Hi Carl, > > > > > > > > Can the gitlab api be of useful in the future? > > > > > > e.g https://invent.kde.org/api/v4/groups/7 > > > > While for many purposes Gitlab's API wil be perfectly fine, it doesn't > > provide us with the i18n branch information which some users will > > require. > > Something to perhaps consider at this point is to revise the > repo-metadata format in general and offload data to gitlab? > > Notably I'd kick the following yaml properties in favor of their > literal replacements in gitlab: description, icon, members, name. On a > related note I guess there'd be need to figure out how to map from a > "kde project" to a gitlab project. Currently the paths between > repo-metadata and invent do not line up. > > Additionally we may able able to remove: > > - hasrepo (repopath==null implies this) > - repoactive (totally not sure what this communicates) > > And lastly, fold the i18n data into the yaml. Which I think means we > could drop the excessive use of directories and just have one file per > project, e.g. projects/kde/workspace/sddm-kcm.yaml or perhaps even > more ideally projects/sddm-kcm.yaml (because the current dir structure > duplicates information encoded in repopath; so I would think either we > should drop the property or flatten projects/). I think it's too early to offload properties from repo-metadata.git to gitlab because we are still far from migrating all project to gitlab. Here is how the i18n data can be folded into yaml (example of Konsole [1]): description: KDE's terminal emulator. [...] projectpath: kde/applications/konsole repoactive: true repopath: konsole type: project i18n: branches: trunk: none stable: none stable_kf5: release/19.12 trunk_kf5: master The idea is to insert the data from i18n.json into the main YAML under i18n.branches. Also, the "none" values can be omitted, I guess. [1] https://cgit.kde.org/sysadmin/repo-metadata.git/tree/projects/kde/applications/konsole -- Alexander Potashev
Re: Sysadmin Load Reduction: Code Related Services
On Tue, Nov 19, 2019 at 6:53 AM Luigi Toscano wrote: > > Ben Cooksley ha scritto: > > On Tue, Nov 19, 2019 at 6:20 AM Luigi Toscano > > wrote: > >> > >> Ben Cooksley ha scritto: > >>> On Tue, Nov 19, 2019 at 1:34 AM Harald Sitter wrote: > > On Sat, Nov 16, 2019 at 9:40 PM Ben Cooksley wrote: > > > > On Sun, Nov 17, 2019 at 5:19 AM Carl Schwan wrote: > >> > >> Hi all, > > > > Hi Carl, > > > >> > >> Can the gitlab api be of useful in the future? > >> > >> e.g https://invent.kde.org/api/v4/groups/7 > > > > While for many purposes Gitlab's API wil be perfectly fine, it doesn't > > provide us with the i18n branch information which some users will > > require. > > Something to perhaps consider at this point is to revise the > repo-metadata format in general and offload data to gitlab? > >>> > >>> Once we have transitioned repository hosting and code review to > >>> Gitlab, restructuring how repo-metadata works was one of the items I > >>> wanted to look into (if anything because i'd like to automate updates > >>> to repo-metadata so we don't have to create them on Gitlab and then > >>> register them in repo-metadata as a second separate step) > >> > >> Uhm, does gitlab allow to define custom properties? That may solve the > >> problem. (but probably it doesn't, or it would have already proposed.) > > > > It doesn't allow us to define custom properties i'm afraid. > > Uhm, there is something, but it requires administrative access and I'm not > sure whether it's available in the community edition: > > https://docs.gitlab.com/ee/api/custom_attributes.html >From what I can tell this is available in the Community Edition. It is however though only available via the API (and isn't shown anywhere in the Gitlab UI), and of course can only be changed by an administrator (ie. Sysadmin) > > project, e.g. projects/kde/workspace/sddm-kcm.yaml or perhaps even > more ideally projects/sddm-kcm.yaml (because the current dir structure > duplicates information encoded in repopath; so I would think either we > should drop the property or flatten projects/). > >>> > >>> We should mirror the repopath as in the Gitlab world it will be > >>> possible for us to have two repositories that have the same name, just > >>> in different parts of the tree. > >> > >> Can we keep the rule of having unique repository names, even if we could > >> use > >> them? One of the plans to reduce the moves around for translations is to > >> flatten the structure without namespaces. Allowing duplicates would make > >> this > >> impossible and introduce more complication (and it may complicate our life > >> as > >> well if a duplicated repository ever moves to plasma or to frameworks). > > > > It'll be hard to tell whether a name is unique or not when creating a > > repository unfortunately. > > For the most part I do not expect collisions to occur though and we > > certainly won't aim to create duplicate names. > > I understand that, but still it means that we can't flatten the translation > repositories removing the namespaces. > Won't you still have issues though when repositories are renamed though? Moves into Plasma and Frameworks from already reviewed repositories are fairly uncommon from my recollection (about as common as repository renames) > Unless you use the future flattened translation repository as a way to see the > existing names. In fact, it should be possible to periodically (every two > days?) generate a list of all repository names; that should allow to check for > duplicates. Maybe this could reduce the minimum amount of uncertainty when > creating new repositories. > > -- > Luigi Cheers, Ben
Re: Sysadmin Load Reduction: Code Related Services
On Tue, Nov 19, 2019 at 7:27 AM Albert Astals Cid wrote: > > El dilluns, 18 de novembre de 2019, a les 13:34:19 CET, Harald Sitter va > escriure: > > And lastly, fold the i18n data into the yaml. > > So to know which is the stable i18n branch of okular i have to checkout it's > master branch, read a file and then change to whatever branch it is said > there to be the stable branch? > > Feels a bit weird that the master branch has that kind of knowledge to be > honest. I think Harald was referring to the 'metadata.yaml' files in sysadmin/repo-metadata in this case. > > Cheers, > Albert > > Cheers, Ben
Re: Sysadmin Load Reduction: Code Related Services
El dilluns, 18 de novembre de 2019, a les 13:34:19 CET, Harald Sitter va escriure: > And lastly, fold the i18n data into the yaml. So to know which is the stable i18n branch of okular i have to checkout it's master branch, read a file and then change to whatever branch it is said there to be the stable branch? Feels a bit weird that the master branch has that kind of knowledge to be honest. Cheers, Albert
Re: Sysadmin Load Reduction: Code Related Services
Ben Cooksley ha scritto: > On Tue, Nov 19, 2019 at 6:20 AM Luigi Toscano > wrote: >> >> Ben Cooksley ha scritto: >>> On Tue, Nov 19, 2019 at 1:34 AM Harald Sitter wrote: On Sat, Nov 16, 2019 at 9:40 PM Ben Cooksley wrote: > > On Sun, Nov 17, 2019 at 5:19 AM Carl Schwan wrote: >> >> Hi all, > > Hi Carl, > >> >> Can the gitlab api be of useful in the future? >> >> e.g https://invent.kde.org/api/v4/groups/7 > > While for many purposes Gitlab's API wil be perfectly fine, it doesn't > provide us with the i18n branch information which some users will > require. Something to perhaps consider at this point is to revise the repo-metadata format in general and offload data to gitlab? >>> >>> Once we have transitioned repository hosting and code review to >>> Gitlab, restructuring how repo-metadata works was one of the items I >>> wanted to look into (if anything because i'd like to automate updates >>> to repo-metadata so we don't have to create them on Gitlab and then >>> register them in repo-metadata as a second separate step) >> >> Uhm, does gitlab allow to define custom properties? That may solve the >> problem. (but probably it doesn't, or it would have already proposed.) > > It doesn't allow us to define custom properties i'm afraid. Uhm, there is something, but it requires administrative access and I'm not sure whether it's available in the community edition: https://docs.gitlab.com/ee/api/custom_attributes.html project, e.g. projects/kde/workspace/sddm-kcm.yaml or perhaps even more ideally projects/sddm-kcm.yaml (because the current dir structure duplicates information encoded in repopath; so I would think either we should drop the property or flatten projects/). >>> >>> We should mirror the repopath as in the Gitlab world it will be >>> possible for us to have two repositories that have the same name, just >>> in different parts of the tree. >> >> Can we keep the rule of having unique repository names, even if we could use >> them? One of the plans to reduce the moves around for translations is to >> flatten the structure without namespaces. Allowing duplicates would make this >> impossible and introduce more complication (and it may complicate our life as >> well if a duplicated repository ever moves to plasma or to frameworks). > > It'll be hard to tell whether a name is unique or not when creating a > repository unfortunately. > For the most part I do not expect collisions to occur though and we > certainly won't aim to create duplicate names. I understand that, but still it means that we can't flatten the translation repositories removing the namespaces. Unless you use the future flattened translation repository as a way to see the existing names. In fact, it should be possible to periodically (every two days?) generate a list of all repository names; that should allow to check for duplicates. Maybe this could reduce the minimum amount of uncertainty when creating new repositories. -- Luigi
Re: Sysadmin Load Reduction: Code Related Services
On Tue, Nov 19, 2019 at 6:20 AM Luigi Toscano wrote: > > Ben Cooksley ha scritto: > > On Tue, Nov 19, 2019 at 1:34 AM Harald Sitter wrote: > >> > >> On Sat, Nov 16, 2019 at 9:40 PM Ben Cooksley wrote: > >>> > >>> On Sun, Nov 17, 2019 at 5:19 AM Carl Schwan wrote: > > Hi all, > >>> > >>> Hi Carl, > >>> > > Can the gitlab api be of useful in the future? > > e.g https://invent.kde.org/api/v4/groups/7 > >>> > >>> While for many purposes Gitlab's API wil be perfectly fine, it doesn't > >>> provide us with the i18n branch information which some users will > >>> require. > >> > >> Something to perhaps consider at this point is to revise the > >> repo-metadata format in general and offload data to gitlab? > > > > Once we have transitioned repository hosting and code review to > > Gitlab, restructuring how repo-metadata works was one of the items I > > wanted to look into (if anything because i'd like to automate updates > > to repo-metadata so we don't have to create them on Gitlab and then > > register them in repo-metadata as a second separate step) > > Uhm, does gitlab allow to define custom properties? That may solve the > problem. (but probably it doesn't, or it would have already proposed.) It doesn't allow us to define custom properties i'm afraid. > > >> project, e.g. projects/kde/workspace/sddm-kcm.yaml or perhaps even > >> more ideally projects/sddm-kcm.yaml (because the current dir structure > >> duplicates information encoded in repopath; so I would think either we > >> should drop the property or flatten projects/). > > > > We should mirror the repopath as in the Gitlab world it will be > > possible for us to have two repositories that have the same name, just > > in different parts of the tree. > > Can we keep the rule of having unique repository names, even if we could use > them? One of the plans to reduce the moves around for translations is to > flatten the structure without namespaces. Allowing duplicates would make this > impossible and introduce more complication (and it may complicate our life as > well if a duplicated repository ever moves to plasma or to frameworks). It'll be hard to tell whether a name is unique or not when creating a repository unfortunately. For the most part I do not expect collisions to occur though and we certainly won't aim to create duplicate names. Note that I do not expect repositories to ever really move beyond the module they're created in (as a multimedia application will always be a multimedia application for instance). The release component (playground, extragear, release service) will not be encoded in the path which should greatly minimize the number of moves. Plasma and Frameworks will be two exceptions there, but that is because they're both modules and release units. > > -- > Luigi Cheers, Ben
Re: Sysadmin Load Reduction: Code Related Services
Ben Cooksley ha scritto: > On Tue, Nov 19, 2019 at 1:34 AM Harald Sitter wrote: >> >> On Sat, Nov 16, 2019 at 9:40 PM Ben Cooksley wrote: >>> >>> On Sun, Nov 17, 2019 at 5:19 AM Carl Schwan wrote: Hi all, >>> >>> Hi Carl, >>> Can the gitlab api be of useful in the future? e.g https://invent.kde.org/api/v4/groups/7 >>> >>> While for many purposes Gitlab's API wil be perfectly fine, it doesn't >>> provide us with the i18n branch information which some users will >>> require. >> >> Something to perhaps consider at this point is to revise the >> repo-metadata format in general and offload data to gitlab? > > Once we have transitioned repository hosting and code review to > Gitlab, restructuring how repo-metadata works was one of the items I > wanted to look into (if anything because i'd like to automate updates > to repo-metadata so we don't have to create them on Gitlab and then > register them in repo-metadata as a second separate step) Uhm, does gitlab allow to define custom properties? That may solve the problem. (but probably it doesn't, or it would have already proposed.) >> project, e.g. projects/kde/workspace/sddm-kcm.yaml or perhaps even >> more ideally projects/sddm-kcm.yaml (because the current dir structure >> duplicates information encoded in repopath; so I would think either we >> should drop the property or flatten projects/). > > We should mirror the repopath as in the Gitlab world it will be > possible for us to have two repositories that have the same name, just > in different parts of the tree. Can we keep the rule of having unique repository names, even if we could use them? One of the plans to reduce the moves around for translations is to flatten the structure without namespaces. Allowing duplicates would make this impossible and introduce more complication (and it may complicate our life as well if a duplicated repository ever moves to plasma or to frameworks). -- Luigi
Re: Sysadmin Load Reduction: Code Related Services
On Tue, Nov 19, 2019 at 1:34 AM Harald Sitter wrote: > > On Sat, Nov 16, 2019 at 9:40 PM Ben Cooksley wrote: > > > > On Sun, Nov 17, 2019 at 5:19 AM Carl Schwan wrote: > > > > > > Hi all, > > > > Hi Carl, > > > > > > > > Can the gitlab api be of useful in the future? > > > > > > e.g https://invent.kde.org/api/v4/groups/7 > > > > While for many purposes Gitlab's API wil be perfectly fine, it doesn't > > provide us with the i18n branch information which some users will > > require. > > Something to perhaps consider at this point is to revise the > repo-metadata format in general and offload data to gitlab? Once we have transitioned repository hosting and code review to Gitlab, restructuring how repo-metadata works was one of the items I wanted to look into (if anything because i'd like to automate updates to repo-metadata so we don't have to create them on Gitlab and then register them in repo-metadata as a second separate step) > > Notably I'd kick the following yaml properties in favor of their > literal replacements in gitlab: description, icon, members, name. On a Nothing uses members, so once the XML is gone we can get rid of that completely. Likewise, nothing uses icon either to my knowledge. Description and Name should definitely both come from Gitlab. > related note I guess there'd be need to figure out how to map from a > "kde project" to a gitlab project. Currently the paths between > repo-metadata and invent do not line up. The 'repopath' key in repo-metadata will serve this purpose (it might be missing the kde/ prefix though depending on how the implementation ends up working out) This fits into an overall discussion i'd like to have around playground/release service/extragear in the Git world though which will be brought up separately. > > Additionally we may able able to remove: > > - hasrepo (repopath==null implies this) This can go yes. > - repoactive (totally not sure what this communicates) This is meant to indicate whether a given entry/repository should be processed by CI and scripty. We have a number of unmaintained and other infrastructural repositories which both of those should ignore. > > And lastly, fold the i18n data into the yaml. Which I think means we > could drop the excessive use of directories and just have one file per I've no objection to this. > project, e.g. projects/kde/workspace/sddm-kcm.yaml or perhaps even > more ideally projects/sddm-kcm.yaml (because the current dir structure > duplicates information encoded in repopath; so I would think either we > should drop the property or flatten projects/). We should mirror the repopath as in the Gitlab world it will be possible for us to have two repositories that have the same name, just in different parts of the tree. > > HS Cheers, Ben
Re: Sysadmin Load Reduction: Code Related Services
On Sat, Nov 16, 2019 at 9:40 PM Ben Cooksley wrote: > > On Sun, Nov 17, 2019 at 5:19 AM Carl Schwan wrote: > > > > Hi all, > > Hi Carl, > > > > > Can the gitlab api be of useful in the future? > > > > e.g https://invent.kde.org/api/v4/groups/7 > > While for many purposes Gitlab's API wil be perfectly fine, it doesn't > provide us with the i18n branch information which some users will > require. Something to perhaps consider at this point is to revise the repo-metadata format in general and offload data to gitlab? Notably I'd kick the following yaml properties in favor of their literal replacements in gitlab: description, icon, members, name. On a related note I guess there'd be need to figure out how to map from a "kde project" to a gitlab project. Currently the paths between repo-metadata and invent do not line up. Additionally we may able able to remove: - hasrepo (repopath==null implies this) - repoactive (totally not sure what this communicates) And lastly, fold the i18n data into the yaml. Which I think means we could drop the excessive use of directories and just have one file per project, e.g. projects/kde/workspace/sddm-kcm.yaml or perhaps even more ideally projects/sddm-kcm.yaml (because the current dir structure duplicates information encoded in repopath; so I would think either we should drop the property or flatten projects/). HS
Re: Sysadmin Load Reduction: Code Related Services
On Sat, Nov 16, 2019 at 10:14 AM Ben Cooksley wrote: > (Harald, I assume it's only requirement is a copy of the > sysadmin/repo-metadata repository locally, and it doesn't require the > actual Git repositories?) Correct.
