Re: Sysadmin Load Reduction: Code Related Services
On Wed, Nov 20, 2019 at 1:48 AM Harald Sitter wrote: > > On Mon, Nov 18, 2019 at 6:29 PM Ben Cooksley wrote: > > It'll be hard to tell whether a name is unique or not when creating a > > repository unfortunately. > > For the most part I do not expect collisions to occur though and we > > certainly won't aim to create duplicate names. > > Wouldn't creating a repo also include creating the repo-metadata file > for it? Through that it should be easy to assert uniqueness via a > server-side hook on repo-metadata. Not necessarily. It will definitely be possible for a repository to exist on Gitlab but not in the repo-metadata repository. Any sync to repo-metadata would be a step taken after the repository is created on Gitlab. Ideally we would have an automated script sync the contents of repo-metadata after they're changed on Gitlab (ie. Gitlab is the authoritative source here) Having the repository come into existence after the repo-metadata entry can cause issues, although they appear fairly rarely as there is generally a delay between Sysadmin creating the repository and Developers starting to use a repository - hence why i'd like for the repository to appear first. > > In point of fact, if we fully flatten out the files I expect uniquness > would be implicit. > > e.g. > > repo-metadata/projects/blinken.yaml (repopath: kde/blinken) > repo-metadata/projects/kinfocenter.yaml (repopath: kde/kinfocenter > repo-metadata/projects/www-kde-org.yaml (repopath: sysadmin/www-kde-org) Website repositories, much like Sysadmin repositories, will continue to live in their own namespaces and therefore should not be flattened (if they're even included in repo-metadata, as they won't be under kde/) Please note that the repository paths for Blinken and KInfoCenter will be kde/edu/blinken and kde/plasma/kinfocenter respectively, as a degree of grouping is required on Gitlab for ease of visibility over merge requests and tasks (modules rarely change, and Gitlab handles redirecting old names to new names for us in any event even for Git clients) > > HS Cheers, Ben
Re: Sysadmin Load Reduction: Code Related Services
On Mon, Nov 18, 2019 at 6:29 PM Ben Cooksley wrote: > It'll be hard to tell whether a name is unique or not when creating a > repository unfortunately. > For the most part I do not expect collisions to occur though and we > certainly won't aim to create duplicate names. Wouldn't creating a repo also include creating the repo-metadata file for it? Through that it should be easy to assert uniqueness via a server-side hook on repo-metadata. In point of fact, if we fully flatten out the files I expect uniquness would be implicit. e.g. repo-metadata/projects/blinken.yaml (repopath: kde/blinken) repo-metadata/projects/kinfocenter.yaml (repopath: kde/kinfocenter repo-metadata/projects/www-kde-org.yaml (repopath: sysadmin/www-kde-org) HS
Re: Sysadmin Load Reduction: Code Related Services
On Mon, Nov 18, 2019 at 7:33 PM Ben Cooksley wrote: > > On Tue, Nov 19, 2019 at 7:27 AM Albert Astals Cid wrote: > > > > El dilluns, 18 de novembre de 2019, a les 13:34:19 CET, Harald Sitter va > > escriure: > > > And lastly, fold the i18n data into the yaml. > > > > So to know which is the stable i18n branch of okular i have to checkout > > it's master branch, read a file and then change to whatever branch it is > > said there to be the stable branch? > > > > Feels a bit weird that the master branch has that kind of knowledge to be > > honest. > > I think Harald was referring to the 'metadata.yaml' files in > sysadmin/repo-metadata in this case. Yep.
Re: Sysadmin Load Reduction: Code Related Services
пн, 18 нояб. 2019 г. в 15:35, Harald Sitter : > > On Sat, Nov 16, 2019 at 9:40 PM Ben Cooksley wrote: > > > > On Sun, Nov 17, 2019 at 5:19 AM Carl Schwan wrote: > > > > > > Hi all, > > > > Hi Carl, > > > > > > > > Can the gitlab api be of useful in the future? > > > > > > e.g https://invent.kde.org/api/v4/groups/7 > > > > While for many purposes Gitlab's API wil be perfectly fine, it doesn't > > provide us with the i18n branch information which some users will > > require. > > Something to perhaps consider at this point is to revise the > repo-metadata format in general and offload data to gitlab? > > Notably I'd kick the following yaml properties in favor of their > literal replacements in gitlab: description, icon, members, name. On a > related note I guess there'd be need to figure out how to map from a > "kde project" to a gitlab project. Currently the paths between > repo-metadata and invent do not line up. > > Additionally we may able able to remove: > > - hasrepo (repopath==null implies this) > - repoactive (totally not sure what this communicates) > > And lastly, fold the i18n data into the yaml. Which I think means we > could drop the excessive use of directories and just have one file per > project, e.g. projects/kde/workspace/sddm-kcm.yaml or perhaps even > more ideally projects/sddm-kcm.yaml (because the current dir structure > duplicates information encoded in repopath; so I would think either we > should drop the property or flatten projects/). I think it's too early to offload properties from repo-metadata.git to gitlab because we are still far from migrating all project to gitlab. Here is how the i18n data can be folded into yaml (example of Konsole [1]): description: KDE's terminal emulator. [...] projectpath: kde/applications/konsole repoactive: true repopath: konsole type: project i18n: branches: trunk: none stable: none stable_kf5: release/19.12 trunk_kf5: master The idea is to insert the data from i18n.json into the main YAML under i18n.branches. Also, the "none" values can be omitted, I guess. [1] https://cgit.kde.org/sysadmin/repo-metadata.git/tree/projects/kde/applications/konsole -- Alexander Potashev
Re: Sysadmin Load Reduction: Code Related Services
On Tue, Nov 19, 2019 at 6:53 AM Luigi Toscano wrote: > > Ben Cooksley ha scritto: > > On Tue, Nov 19, 2019 at 6:20 AM Luigi Toscano > > wrote: > >> > >> Ben Cooksley ha scritto: > >>> On Tue, Nov 19, 2019 at 1:34 AM Harald Sitter wrote: > > On Sat, Nov 16, 2019 at 9:40 PM Ben Cooksley wrote: > > > > On Sun, Nov 17, 2019 at 5:19 AM Carl Schwan wrote: > >> > >> Hi all, > > > > Hi Carl, > > > >> > >> Can the gitlab api be of useful in the future? > >> > >> e.g https://invent.kde.org/api/v4/groups/7 > > > > While for many purposes Gitlab's API wil be perfectly fine, it doesn't > > provide us with the i18n branch information which some users will > > require. > > Something to perhaps consider at this point is to revise the > repo-metadata format in general and offload data to gitlab? > >>> > >>> Once we have transitioned repository hosting and code review to > >>> Gitlab, restructuring how repo-metadata works was one of the items I > >>> wanted to look into (if anything because i'd like to automate updates > >>> to repo-metadata so we don't have to create them on Gitlab and then > >>> register them in repo-metadata as a second separate step) > >> > >> Uhm, does gitlab allow to define custom properties? That may solve the > >> problem. (but probably it doesn't, or it would have already proposed.) > > > > It doesn't allow us to define custom properties i'm afraid. > > Uhm, there is something, but it requires administrative access and I'm not > sure whether it's available in the community edition: > > https://docs.gitlab.com/ee/api/custom_attributes.html >From what I can tell this is available in the Community Edition. It is however though only available via the API (and isn't shown anywhere in the Gitlab UI), and of course can only be changed by an administrator (ie. Sysadmin) > > project, e.g. projects/kde/workspace/sddm-kcm.yaml or perhaps even > more ideally projects/sddm-kcm.yaml (because the current dir structure > duplicates information encoded in repopath; so I would think either we > should drop the property or flatten projects/). > >>> > >>> We should mirror the repopath as in the Gitlab world it will be > >>> possible for us to have two repositories that have the same name, just > >>> in different parts of the tree. > >> > >> Can we keep the rule of having unique repository names, even if we could > >> use > >> them? One of the plans to reduce the moves around for translations is to > >> flatten the structure without namespaces. Allowing duplicates would make > >> this > >> impossible and introduce more complication (and it may complicate our life > >> as > >> well if a duplicated repository ever moves to plasma or to frameworks). > > > > It'll be hard to tell whether a name is unique or not when creating a > > repository unfortunately. > > For the most part I do not expect collisions to occur though and we > > certainly won't aim to create duplicate names. > > I understand that, but still it means that we can't flatten the translation > repositories removing the namespaces. > Won't you still have issues though when repositories are renamed though? Moves into Plasma and Frameworks from already reviewed repositories are fairly uncommon from my recollection (about as common as repository renames) > Unless you use the future flattened translation repository as a way to see the > existing names. In fact, it should be possible to periodically (every two > days?) generate a list of all repository names; that should allow to check for > duplicates. Maybe this could reduce the minimum amount of uncertainty when > creating new repositories. > > -- > Luigi Cheers, Ben
Re: Sysadmin Load Reduction: Code Related Services
On Tue, Nov 19, 2019 at 7:27 AM Albert Astals Cid wrote: > > El dilluns, 18 de novembre de 2019, a les 13:34:19 CET, Harald Sitter va > escriure: > > And lastly, fold the i18n data into the yaml. > > So to know which is the stable i18n branch of okular i have to checkout it's > master branch, read a file and then change to whatever branch it is said > there to be the stable branch? > > Feels a bit weird that the master branch has that kind of knowledge to be > honest. I think Harald was referring to the 'metadata.yaml' files in sysadmin/repo-metadata in this case. > > Cheers, > Albert > > Cheers, Ben
Re: Sysadmin Load Reduction: Code Related Services
El dilluns, 18 de novembre de 2019, a les 13:34:19 CET, Harald Sitter va escriure: > And lastly, fold the i18n data into the yaml. So to know which is the stable i18n branch of okular i have to checkout it's master branch, read a file and then change to whatever branch it is said there to be the stable branch? Feels a bit weird that the master branch has that kind of knowledge to be honest. Cheers, Albert
Re: Sysadmin Load Reduction: Code Related Services
Ben Cooksley ha scritto: > On Tue, Nov 19, 2019 at 6:20 AM Luigi Toscano > wrote: >> >> Ben Cooksley ha scritto: >>> On Tue, Nov 19, 2019 at 1:34 AM Harald Sitter wrote: On Sat, Nov 16, 2019 at 9:40 PM Ben Cooksley wrote: > > On Sun, Nov 17, 2019 at 5:19 AM Carl Schwan wrote: >> >> Hi all, > > Hi Carl, > >> >> Can the gitlab api be of useful in the future? >> >> e.g https://invent.kde.org/api/v4/groups/7 > > While for many purposes Gitlab's API wil be perfectly fine, it doesn't > provide us with the i18n branch information which some users will > require. Something to perhaps consider at this point is to revise the repo-metadata format in general and offload data to gitlab? >>> >>> Once we have transitioned repository hosting and code review to >>> Gitlab, restructuring how repo-metadata works was one of the items I >>> wanted to look into (if anything because i'd like to automate updates >>> to repo-metadata so we don't have to create them on Gitlab and then >>> register them in repo-metadata as a second separate step) >> >> Uhm, does gitlab allow to define custom properties? That may solve the >> problem. (but probably it doesn't, or it would have already proposed.) > > It doesn't allow us to define custom properties i'm afraid. Uhm, there is something, but it requires administrative access and I'm not sure whether it's available in the community edition: https://docs.gitlab.com/ee/api/custom_attributes.html project, e.g. projects/kde/workspace/sddm-kcm.yaml or perhaps even more ideally projects/sddm-kcm.yaml (because the current dir structure duplicates information encoded in repopath; so I would think either we should drop the property or flatten projects/). >>> >>> We should mirror the repopath as in the Gitlab world it will be >>> possible for us to have two repositories that have the same name, just >>> in different parts of the tree. >> >> Can we keep the rule of having unique repository names, even if we could use >> them? One of the plans to reduce the moves around for translations is to >> flatten the structure without namespaces. Allowing duplicates would make this >> impossible and introduce more complication (and it may complicate our life as >> well if a duplicated repository ever moves to plasma or to frameworks). > > It'll be hard to tell whether a name is unique or not when creating a > repository unfortunately. > For the most part I do not expect collisions to occur though and we > certainly won't aim to create duplicate names. I understand that, but still it means that we can't flatten the translation repositories removing the namespaces. Unless you use the future flattened translation repository as a way to see the existing names. In fact, it should be possible to periodically (every two days?) generate a list of all repository names; that should allow to check for duplicates. Maybe this could reduce the minimum amount of uncertainty when creating new repositories. -- Luigi
Re: Sysadmin Load Reduction: Code Related Services
On Tue, Nov 19, 2019 at 6:20 AM Luigi Toscano wrote: > > Ben Cooksley ha scritto: > > On Tue, Nov 19, 2019 at 1:34 AM Harald Sitter wrote: > >> > >> On Sat, Nov 16, 2019 at 9:40 PM Ben Cooksley wrote: > >>> > >>> On Sun, Nov 17, 2019 at 5:19 AM Carl Schwan wrote: > > Hi all, > >>> > >>> Hi Carl, > >>> > > Can the gitlab api be of useful in the future? > > e.g https://invent.kde.org/api/v4/groups/7 > >>> > >>> While for many purposes Gitlab's API wil be perfectly fine, it doesn't > >>> provide us with the i18n branch information which some users will > >>> require. > >> > >> Something to perhaps consider at this point is to revise the > >> repo-metadata format in general and offload data to gitlab? > > > > Once we have transitioned repository hosting and code review to > > Gitlab, restructuring how repo-metadata works was one of the items I > > wanted to look into (if anything because i'd like to automate updates > > to repo-metadata so we don't have to create them on Gitlab and then > > register them in repo-metadata as a second separate step) > > Uhm, does gitlab allow to define custom properties? That may solve the > problem. (but probably it doesn't, or it would have already proposed.) It doesn't allow us to define custom properties i'm afraid. > > >> project, e.g. projects/kde/workspace/sddm-kcm.yaml or perhaps even > >> more ideally projects/sddm-kcm.yaml (because the current dir structure > >> duplicates information encoded in repopath; so I would think either we > >> should drop the property or flatten projects/). > > > > We should mirror the repopath as in the Gitlab world it will be > > possible for us to have two repositories that have the same name, just > > in different parts of the tree. > > Can we keep the rule of having unique repository names, even if we could use > them? One of the plans to reduce the moves around for translations is to > flatten the structure without namespaces. Allowing duplicates would make this > impossible and introduce more complication (and it may complicate our life as > well if a duplicated repository ever moves to plasma or to frameworks). It'll be hard to tell whether a name is unique or not when creating a repository unfortunately. For the most part I do not expect collisions to occur though and we certainly won't aim to create duplicate names. Note that I do not expect repositories to ever really move beyond the module they're created in (as a multimedia application will always be a multimedia application for instance). The release component (playground, extragear, release service) will not be encoded in the path which should greatly minimize the number of moves. Plasma and Frameworks will be two exceptions there, but that is because they're both modules and release units. > > -- > Luigi Cheers, Ben
Re: Sysadmin Load Reduction: Code Related Services
Ben Cooksley ha scritto: > On Tue, Nov 19, 2019 at 1:34 AM Harald Sitter wrote: >> >> On Sat, Nov 16, 2019 at 9:40 PM Ben Cooksley wrote: >>> >>> On Sun, Nov 17, 2019 at 5:19 AM Carl Schwan wrote: Hi all, >>> >>> Hi Carl, >>> Can the gitlab api be of useful in the future? e.g https://invent.kde.org/api/v4/groups/7 >>> >>> While for many purposes Gitlab's API wil be perfectly fine, it doesn't >>> provide us with the i18n branch information which some users will >>> require. >> >> Something to perhaps consider at this point is to revise the >> repo-metadata format in general and offload data to gitlab? > > Once we have transitioned repository hosting and code review to > Gitlab, restructuring how repo-metadata works was one of the items I > wanted to look into (if anything because i'd like to automate updates > to repo-metadata so we don't have to create them on Gitlab and then > register them in repo-metadata as a second separate step) Uhm, does gitlab allow to define custom properties? That may solve the problem. (but probably it doesn't, or it would have already proposed.) >> project, e.g. projects/kde/workspace/sddm-kcm.yaml or perhaps even >> more ideally projects/sddm-kcm.yaml (because the current dir structure >> duplicates information encoded in repopath; so I would think either we >> should drop the property or flatten projects/). > > We should mirror the repopath as in the Gitlab world it will be > possible for us to have two repositories that have the same name, just > in different parts of the tree. Can we keep the rule of having unique repository names, even if we could use them? One of the plans to reduce the moves around for translations is to flatten the structure without namespaces. Allowing duplicates would make this impossible and introduce more complication (and it may complicate our life as well if a duplicated repository ever moves to plasma or to frameworks). -- Luigi
Re: Sysadmin Load Reduction: Code Related Services
On Tue, Nov 19, 2019 at 1:34 AM Harald Sitter wrote: > > On Sat, Nov 16, 2019 at 9:40 PM Ben Cooksley wrote: > > > > On Sun, Nov 17, 2019 at 5:19 AM Carl Schwan wrote: > > > > > > Hi all, > > > > Hi Carl, > > > > > > > > Can the gitlab api be of useful in the future? > > > > > > e.g https://invent.kde.org/api/v4/groups/7 > > > > While for many purposes Gitlab's API wil be perfectly fine, it doesn't > > provide us with the i18n branch information which some users will > > require. > > Something to perhaps consider at this point is to revise the > repo-metadata format in general and offload data to gitlab? Once we have transitioned repository hosting and code review to Gitlab, restructuring how repo-metadata works was one of the items I wanted to look into (if anything because i'd like to automate updates to repo-metadata so we don't have to create them on Gitlab and then register them in repo-metadata as a second separate step) > > Notably I'd kick the following yaml properties in favor of their > literal replacements in gitlab: description, icon, members, name. On a Nothing uses members, so once the XML is gone we can get rid of that completely. Likewise, nothing uses icon either to my knowledge. Description and Name should definitely both come from Gitlab. > related note I guess there'd be need to figure out how to map from a > "kde project" to a gitlab project. Currently the paths between > repo-metadata and invent do not line up. The 'repopath' key in repo-metadata will serve this purpose (it might be missing the kde/ prefix though depending on how the implementation ends up working out) This fits into an overall discussion i'd like to have around playground/release service/extragear in the Git world though which will be brought up separately. > > Additionally we may able able to remove: > > - hasrepo (repopath==null implies this) This can go yes. > - repoactive (totally not sure what this communicates) This is meant to indicate whether a given entry/repository should be processed by CI and scripty. We have a number of unmaintained and other infrastructural repositories which both of those should ignore. > > And lastly, fold the i18n data into the yaml. Which I think means we > could drop the excessive use of directories and just have one file per I've no objection to this. > project, e.g. projects/kde/workspace/sddm-kcm.yaml or perhaps even > more ideally projects/sddm-kcm.yaml (because the current dir structure > duplicates information encoded in repopath; so I would think either we > should drop the property or flatten projects/). We should mirror the repopath as in the Gitlab world it will be possible for us to have two repositories that have the same name, just in different parts of the tree. > > HS Cheers, Ben
Re: Sysadmin Load Reduction: Code Related Services
On Sat, Nov 16, 2019 at 9:40 PM Ben Cooksley wrote: > > On Sun, Nov 17, 2019 at 5:19 AM Carl Schwan wrote: > > > > Hi all, > > Hi Carl, > > > > > Can the gitlab api be of useful in the future? > > > > e.g https://invent.kde.org/api/v4/groups/7 > > While for many purposes Gitlab's API wil be perfectly fine, it doesn't > provide us with the i18n branch information which some users will > require. Something to perhaps consider at this point is to revise the repo-metadata format in general and offload data to gitlab? Notably I'd kick the following yaml properties in favor of their literal replacements in gitlab: description, icon, members, name. On a related note I guess there'd be need to figure out how to map from a "kde project" to a gitlab project. Currently the paths between repo-metadata and invent do not line up. Additionally we may able able to remove: - hasrepo (repopath==null implies this) - repoactive (totally not sure what this communicates) And lastly, fold the i18n data into the yaml. Which I think means we could drop the excessive use of directories and just have one file per project, e.g. projects/kde/workspace/sddm-kcm.yaml or perhaps even more ideally projects/sddm-kcm.yaml (because the current dir structure duplicates information encoded in repopath; so I would think either we should drop the property or flatten projects/). HS
Re: Sysadmin Load Reduction: Code Related Services
On Sun, Nov 17, 2019 at 5:19 AM Carl Schwan wrote: > > Hi all, Hi Carl, > > Can the gitlab api be of useful in the future? > > e.g https://invent.kde.org/api/v4/groups/7 While for many purposes Gitlab's API wil be perfectly fine, it doesn't provide us with the i18n branch information which some users will require. > > Cheers, > Carl > Cheers, Ben > ‐‐‐ Original Message ‐‐‐ > On Saturday, November 16, 2019 9:51 AM, Ben Cooksley > wrote: > > > On Sat, Nov 16, 2019 at 10:39 PM Albert Astals Cid aa...@kde.org wrote: > > > > > > El dissabte, 16 de novembre de 2019, a les 10:14:15 CET, Ben Cooksley va > > > escriure: > > > > > > > > On Sat, Nov 16, 2019 at 3:58 PM Alexander Potashev aspotas...@gmail.com > > > > wrote: > > > > > > > > > > пт, 15 нояб. 2019 г. в 22:05, Ben Cooksley bcooks...@kde.org: > > > > > > > > > > > > On Sat, Nov 16, 2019 at 3:27 AM Alexander Potashev > > > > > > aspotas...@gmail.com wrote: > > > > > > > > > > > > > > пт, 15 нояб. 2019 г. в 15:46, Harald Sitter sit...@kde.org: > > > > > > > > > > > > > > > > On Sat, Nov 9, 2019 at 11:37 PM Alexander Potashev > > > > > > > > aspotas...@gmail.com wrote: > > > > > > > > > > > > > > > > > > сб, 9 нояб. 2019 г. в 02:51, Ben Cooksley bcooks...@kde.org: > > > > > > > > > > > > > > > > > > > > In the category of no longer in use, we have the > > > > > > > > > > compatibility > > > > > > > > > > generator for the kde_projects.xml file. This was > > > > > > > > > > introduced when we > > > > > > > > > > shutdown Redmine/Chiliproject and migrated to Phabricator, > > > > > > > > > > as a way of > > > > > > > > > > keeping services that needed to discover a list of KDE > > > > > > > > > > Projects > > > > > > > > > > functional. > > > > > > > > > > As we've since migrated to using YAML files within the > > > > > > > > > > sysadmin/repo-metadata repository for both the CI System and > > > > > > > > > > kdesrc-build (and with LXR using kdesrc-build to do it's > > > > > > > > > > code > > > > > > > > > > checkouts) there shouldn't to my knowledge be anything > > > > > > > > > > still relying > > > > > > > > > > on this (aside from perhaps scripty). > > > > > > > > > > I'd therefore like to shut this generator down as well, > > > > > > > > > > along with the > > > > > > > > > > compaibility redirector running at projects.kde.org (given > > > > > > > > > > that it has > > > > > > > > > > been some time since we were using that site, and many > > > > > > > > > > projects have > > > > > > > > > > moved around in the virtual structure since then, making > > > > > > > > > > the redirects > > > > > > > > > > it is able to offer useless) > > > > > > > > > > > > > > > > > > > Hi Ben, > > > > > > > > > I am developing a new version of the "opensrc" plugin for > > > > > > > > > Lokalize [1] > > > > > > > > > and it currently depends on kde_projects.xml. Of course I can > > > > > > > > > add new > > > > > > > > > code to scan the Git repo instead of just fetching > > > > > > > > > kde_projects.xml, > > > > > > > > > however it would be more complicated. > > > > > > > > > > > > > > > > > https://projects.kde.org/api/doc/ specifically deals with this > > > > > > > > problem > > > > > > > > by abstracting the repo away behind a micro service. > > > > > > > > > > > > > > > This looks like another view of the data available in > > > > > > > kde_projects.xml, however the API is very limited. For example I > > > > > > > can't > > > > > > > query repo descriptions using this API. Thus not helpful. > > > > > > > However if we were going to kill kde_projects.xml, are you sure > > > > > > > projects.kde.org/api/ would be still available and not shut down > > > > > > > as > > > > > > > well? > > > > > > > > > > > > > The API that Harald mentions is based off the YAML files, and is not > > > > > > reliant on the legacy kde_projects.xml file. > > > > > > > > > > > I can implement kde_projects.xml or a modernized version of it as part > > > > > of projects.kde.org/api/. Does this sound like a good idea? > > > > > We don't need to support the exact same XML format, so I would prefer > > > > > a JSON listing all projects with all their properties, at something > > > > > like GET https://projects.kde.org/api/v1/export or may be return all > > > > > project properties in GET https://projects.kde.org/api/v1/projects > > > > > > > > > I'll leave it to Harald to comment on whether he'd be happy having > > > > additional capabilities in the Projects Microservice API he maintains. > > > > (Harald, I assume it's only requirement is a copy of the > > > > sysadmin/repo-metadata repository locally, and it doesn't require the > > > > actual Git repositories?) > > > > We really should avoid continuing to keep legacy endpoints alive > > > > though (as it is just shifting the maintenance effort of porting away > > > > from it further down the road), especially for something that is going > > > > to end up on end user systems. > > > > > > > Wait, are you saying we
Re: Sysadmin Load Reduction: Code Related Services
Hi all, Can the gitlab api be of useful in the future? e.g https://invent.kde.org/api/v4/groups/7 Cheers, Carl ‐‐‐ Original Message ‐‐‐ On Saturday, November 16, 2019 9:51 AM, Ben Cooksley wrote: > On Sat, Nov 16, 2019 at 10:39 PM Albert Astals Cid aa...@kde.org wrote: > > > El dissabte, 16 de novembre de 2019, a les 10:14:15 CET, Ben Cooksley va > > escriure: > > > > > On Sat, Nov 16, 2019 at 3:58 PM Alexander Potashev aspotas...@gmail.com > > > wrote: > > > > > > > пт, 15 нояб. 2019 г. в 22:05, Ben Cooksley bcooks...@kde.org: > > > > > > > > > On Sat, Nov 16, 2019 at 3:27 AM Alexander Potashev > > > > > aspotas...@gmail.com wrote: > > > > > > > > > > > пт, 15 нояб. 2019 г. в 15:46, Harald Sitter sit...@kde.org: > > > > > > > > > > > > > On Sat, Nov 9, 2019 at 11:37 PM Alexander Potashev > > > > > > > aspotas...@gmail.com wrote: > > > > > > > > > > > > > > > сб, 9 нояб. 2019 г. в 02:51, Ben Cooksley bcooks...@kde.org: > > > > > > > > > > > > > > > > > In the category of no longer in use, we have the compatibility > > > > > > > > > generator for the kde_projects.xml file. This was introduced > > > > > > > > > when we > > > > > > > > > shutdown Redmine/Chiliproject and migrated to Phabricator, as > > > > > > > > > a way of > > > > > > > > > keeping services that needed to discover a list of KDE > > > > > > > > > Projects > > > > > > > > > functional. > > > > > > > > > As we've since migrated to using YAML files within the > > > > > > > > > sysadmin/repo-metadata repository for both the CI System and > > > > > > > > > kdesrc-build (and with LXR using kdesrc-build to do it's code > > > > > > > > > checkouts) there shouldn't to my knowledge be anything still > > > > > > > > > relying > > > > > > > > > on this (aside from perhaps scripty). > > > > > > > > > I'd therefore like to shut this generator down as well, along > > > > > > > > > with the > > > > > > > > > compaibility redirector running at projects.kde.org (given > > > > > > > > > that it has > > > > > > > > > been some time since we were using that site, and many > > > > > > > > > projects have > > > > > > > > > moved around in the virtual structure since then, making the > > > > > > > > > redirects > > > > > > > > > it is able to offer useless) > > > > > > > > > > > > > > > > Hi Ben, > > > > > > > > I am developing a new version of the "opensrc" plugin for > > > > > > > > Lokalize [1] > > > > > > > > and it currently depends on kde_projects.xml. Of course I can > > > > > > > > add new > > > > > > > > code to scan the Git repo instead of just fetching > > > > > > > > kde_projects.xml, > > > > > > > > however it would be more complicated. > > > > > > > > > > > > > > https://projects.kde.org/api/doc/ specifically deals with this > > > > > > > problem > > > > > > > by abstracting the repo away behind a micro service. > > > > > > > > > > > > This looks like another view of the data available in > > > > > > kde_projects.xml, however the API is very limited. For example I > > > > > > can't > > > > > > query repo descriptions using this API. Thus not helpful. > > > > > > However if we were going to kill kde_projects.xml, are you sure > > > > > > projects.kde.org/api/ would be still available and not shut down as > > > > > > well? > > > > > > > > > > The API that Harald mentions is based off the YAML files, and is not > > > > > reliant on the legacy kde_projects.xml file. > > > > > > > > I can implement kde_projects.xml or a modernized version of it as part > > > > of projects.kde.org/api/. Does this sound like a good idea? > > > > We don't need to support the exact same XML format, so I would prefer > > > > a JSON listing all projects with all their properties, at something > > > > like GET https://projects.kde.org/api/v1/export or may be return all > > > > project properties in GET https://projects.kde.org/api/v1/projects > > > > > > I'll leave it to Harald to comment on whether he'd be happy having > > > additional capabilities in the Projects Microservice API he maintains. > > > (Harald, I assume it's only requirement is a copy of the > > > sysadmin/repo-metadata repository locally, and it doesn't require the > > > actual Git repositories?) > > > We really should avoid continuing to keep legacy endpoints alive > > > though (as it is just shifting the maintenance effort of porting away > > > from it further down the road), especially for something that is going > > > to end up on end user systems. > > > > Wait, are you saying we shouldn't be using that endpoint to solve our > > dependency on kde_projects.xml on scripty either? > > I just asked Adrián to have a look at it, since it seemed easier than > > having to worry about downloading and keeping an up to date copy of another > > repo. > > I was referring to the "implement kde_projects.xml" part of his email there. > > The modern components of the projects.kde.org/api/ endpoint are of course > fine. > > > Cheers, > > Albert > >
Re: Sysadmin Load Reduction: Code Related Services
On Sat, Nov 16, 2019 at 10:39 PM Albert Astals Cid wrote: > > El dissabte, 16 de novembre de 2019, a les 10:14:15 CET, Ben Cooksley va > escriure: > > On Sat, Nov 16, 2019 at 3:58 PM Alexander Potashev > > wrote: > > > > > > пт, 15 нояб. 2019 г. в 22:05, Ben Cooksley : > > > > > > > > On Sat, Nov 16, 2019 at 3:27 AM Alexander Potashev > > > > wrote: > > > > > > > > > > пт, 15 нояб. 2019 г. в 15:46, Harald Sitter : > > > > > > > > > > > > On Sat, Nov 9, 2019 at 11:37 PM Alexander Potashev > > > > > > wrote: > > > > > > > > > > > > > > сб, 9 нояб. 2019 г. в 02:51, Ben Cooksley : > > > > > > > > In the category of no longer in use, we have the compatibility > > > > > > > > generator for the kde_projects.xml file. This was introduced > > > > > > > > when we > > > > > > > > shutdown Redmine/Chiliproject and migrated to Phabricator, as a > > > > > > > > way of > > > > > > > > keeping services that needed to discover a list of KDE Projects > > > > > > > > functional. > > > > > > > > > > > > > > > > As we've since migrated to using YAML files within the > > > > > > > > sysadmin/repo-metadata repository for both the CI System and > > > > > > > > kdesrc-build (and with LXR using kdesrc-build to do it's code > > > > > > > > checkouts) there shouldn't to my knowledge be anything still > > > > > > > > relying > > > > > > > > on this (aside from perhaps scripty). > > > > > > > > > > > > > > > > I'd therefore like to shut this generator down as well, along > > > > > > > > with the > > > > > > > > compaibility redirector running at projects.kde.org (given that > > > > > > > > it has > > > > > > > > been some time since we were using that site, and many projects > > > > > > > > have > > > > > > > > moved around in the virtual structure since then, making the > > > > > > > > redirects > > > > > > > > it is able to offer useless) > > > > > > > > > > > > > > Hi Ben, > > > > > > > > > > > > > > I am developing a new version of the "opensrc" plugin for > > > > > > > Lokalize [1] > > > > > > > and it currently depends on kde_projects.xml. Of course I can add > > > > > > > new > > > > > > > code to scan the Git repo instead of just fetching > > > > > > > kde_projects.xml, > > > > > > > however it would be more complicated. > > > > > > > > > > > > https://projects.kde.org/api/doc/ specifically deals with this > > > > > > problem > > > > > > by abstracting the repo away behind a micro service. > > > > > > > > > > This looks like another view of the data available in > > > > > kde_projects.xml, however the API is very limited. For example I can't > > > > > query repo descriptions using this API. Thus not helpful. > > > > > > > > > > However if we were going to kill kde_projects.xml, are you sure > > > > > projects.kde.org/api/ would be still available and not shut down as > > > > > well? > > > > > > > > The API that Harald mentions is based off the YAML files, and is not > > > > reliant on the legacy kde_projects.xml file. > > > > > > I can implement kde_projects.xml or a modernized version of it as part > > > of projects.kde.org/api/. Does this sound like a good idea? > > > > > > We don't need to support the exact same XML format, so I would prefer > > > a JSON listing all projects with all their properties, at something > > > like GET https://projects.kde.org/api/v1/export or may be return all > > > project properties in GET https://projects.kde.org/api/v1/projects > > > > I'll leave it to Harald to comment on whether he'd be happy having > > additional capabilities in the Projects Microservice API he maintains. > > (Harald, I assume it's only requirement is a copy of the > > sysadmin/repo-metadata repository locally, and it doesn't require the > > actual Git repositories?) > > > > We really should avoid continuing to keep legacy endpoints alive > > though (as it is just shifting the maintenance effort of porting away > > from it further down the road), especially for something that is going > > to end up on end user systems. > > Wait, are you saying we shouldn't be using that endpoint to solve our > dependency on kde_projects.xml on scripty either? > > I just asked Adrián to have a look at it, since it seemed easier than having > to worry about downloading and keeping an up to date copy of another repo. I was referring to the "implement kde_projects.xml" part of his email there. The modern components of the projects.kde.org/api/ endpoint are of course fine. > > Cheers, > Albert Cheers, Ben > > > > > On that note, should you be using QNetworkAccessManager, please ensure > > you forcibly and explicitly enable handling of redirects. > > > > > > > > -- > > > Alexander Potashev > > > > Cheers, > > Ben > > > > > >
Re: Sysadmin Load Reduction: Code Related Services
El dissabte, 16 de novembre de 2019, a les 10:14:15 CET, Ben Cooksley va escriure: > On Sat, Nov 16, 2019 at 3:58 PM Alexander Potashev > wrote: > > > > пт, 15 нояб. 2019 г. в 22:05, Ben Cooksley : > > > > > > On Sat, Nov 16, 2019 at 3:27 AM Alexander Potashev > > > wrote: > > > > > > > > пт, 15 нояб. 2019 г. в 15:46, Harald Sitter : > > > > > > > > > > On Sat, Nov 9, 2019 at 11:37 PM Alexander Potashev > > > > > wrote: > > > > > > > > > > > > сб, 9 нояб. 2019 г. в 02:51, Ben Cooksley : > > > > > > > In the category of no longer in use, we have the compatibility > > > > > > > generator for the kde_projects.xml file. This was introduced when > > > > > > > we > > > > > > > shutdown Redmine/Chiliproject and migrated to Phabricator, as a > > > > > > > way of > > > > > > > keeping services that needed to discover a list of KDE Projects > > > > > > > functional. > > > > > > > > > > > > > > As we've since migrated to using YAML files within the > > > > > > > sysadmin/repo-metadata repository for both the CI System and > > > > > > > kdesrc-build (and with LXR using kdesrc-build to do it's code > > > > > > > checkouts) there shouldn't to my knowledge be anything still > > > > > > > relying > > > > > > > on this (aside from perhaps scripty). > > > > > > > > > > > > > > I'd therefore like to shut this generator down as well, along > > > > > > > with the > > > > > > > compaibility redirector running at projects.kde.org (given that > > > > > > > it has > > > > > > > been some time since we were using that site, and many projects > > > > > > > have > > > > > > > moved around in the virtual structure since then, making the > > > > > > > redirects > > > > > > > it is able to offer useless) > > > > > > > > > > > > Hi Ben, > > > > > > > > > > > > I am developing a new version of the "opensrc" plugin for Lokalize > > > > > > [1] > > > > > > and it currently depends on kde_projects.xml. Of course I can add > > > > > > new > > > > > > code to scan the Git repo instead of just fetching kde_projects.xml, > > > > > > however it would be more complicated. > > > > > > > > > > https://projects.kde.org/api/doc/ specifically deals with this problem > > > > > by abstracting the repo away behind a micro service. > > > > > > > > This looks like another view of the data available in > > > > kde_projects.xml, however the API is very limited. For example I can't > > > > query repo descriptions using this API. Thus not helpful. > > > > > > > > However if we were going to kill kde_projects.xml, are you sure > > > > projects.kde.org/api/ would be still available and not shut down as > > > > well? > > > > > > The API that Harald mentions is based off the YAML files, and is not > > > reliant on the legacy kde_projects.xml file. > > > > I can implement kde_projects.xml or a modernized version of it as part > > of projects.kde.org/api/. Does this sound like a good idea? > > > > We don't need to support the exact same XML format, so I would prefer > > a JSON listing all projects with all their properties, at something > > like GET https://projects.kde.org/api/v1/export or may be return all > > project properties in GET https://projects.kde.org/api/v1/projects > > I'll leave it to Harald to comment on whether he'd be happy having > additional capabilities in the Projects Microservice API he maintains. > (Harald, I assume it's only requirement is a copy of the > sysadmin/repo-metadata repository locally, and it doesn't require the > actual Git repositories?) > > We really should avoid continuing to keep legacy endpoints alive > though (as it is just shifting the maintenance effort of porting away > from it further down the road), especially for something that is going > to end up on end user systems. Wait, are you saying we shouldn't be using that endpoint to solve our dependency on kde_projects.xml on scripty either? I just asked Adrián to have a look at it, since it seemed easier than having to worry about downloading and keeping an up to date copy of another repo. Cheers, Albert > > On that note, should you be using QNetworkAccessManager, please ensure > you forcibly and explicitly enable handling of redirects. > > > > > -- > > Alexander Potashev > > Cheers, > Ben >
Re: Sysadmin Load Reduction: Code Related Services
On Sat, Nov 16, 2019 at 3:58 PM Alexander Potashev wrote: > > пт, 15 нояб. 2019 г. в 22:05, Ben Cooksley : > > > > On Sat, Nov 16, 2019 at 3:27 AM Alexander Potashev > > wrote: > > > > > > пт, 15 нояб. 2019 г. в 15:46, Harald Sitter : > > > > > > > > On Sat, Nov 9, 2019 at 11:37 PM Alexander Potashev > > > > wrote: > > > > > > > > > > сб, 9 нояб. 2019 г. в 02:51, Ben Cooksley : > > > > > > In the category of no longer in use, we have the compatibility > > > > > > generator for the kde_projects.xml file. This was introduced when we > > > > > > shutdown Redmine/Chiliproject and migrated to Phabricator, as a way > > > > > > of > > > > > > keeping services that needed to discover a list of KDE Projects > > > > > > functional. > > > > > > > > > > > > As we've since migrated to using YAML files within the > > > > > > sysadmin/repo-metadata repository for both the CI System and > > > > > > kdesrc-build (and with LXR using kdesrc-build to do it's code > > > > > > checkouts) there shouldn't to my knowledge be anything still relying > > > > > > on this (aside from perhaps scripty). > > > > > > > > > > > > I'd therefore like to shut this generator down as well, along with > > > > > > the > > > > > > compaibility redirector running at projects.kde.org (given that it > > > > > > has > > > > > > been some time since we were using that site, and many projects have > > > > > > moved around in the virtual structure since then, making the > > > > > > redirects > > > > > > it is able to offer useless) > > > > > > > > > > Hi Ben, > > > > > > > > > > I am developing a new version of the "opensrc" plugin for Lokalize [1] > > > > > and it currently depends on kde_projects.xml. Of course I can add new > > > > > code to scan the Git repo instead of just fetching kde_projects.xml, > > > > > however it would be more complicated. > > > > > > > > https://projects.kde.org/api/doc/ specifically deals with this problem > > > > by abstracting the repo away behind a micro service. > > > > > > This looks like another view of the data available in > > > kde_projects.xml, however the API is very limited. For example I can't > > > query repo descriptions using this API. Thus not helpful. > > > > > > However if we were going to kill kde_projects.xml, are you sure > > > projects.kde.org/api/ would be still available and not shut down as > > > well? > > > > The API that Harald mentions is based off the YAML files, and is not > > reliant on the legacy kde_projects.xml file. > > I can implement kde_projects.xml or a modernized version of it as part > of projects.kde.org/api/. Does this sound like a good idea? > > We don't need to support the exact same XML format, so I would prefer > a JSON listing all projects with all their properties, at something > like GET https://projects.kde.org/api/v1/export or may be return all > project properties in GET https://projects.kde.org/api/v1/projects I'll leave it to Harald to comment on whether he'd be happy having additional capabilities in the Projects Microservice API he maintains. (Harald, I assume it's only requirement is a copy of the sysadmin/repo-metadata repository locally, and it doesn't require the actual Git repositories?) We really should avoid continuing to keep legacy endpoints alive though (as it is just shifting the maintenance effort of porting away from it further down the road), especially for something that is going to end up on end user systems. On that note, should you be using QNetworkAccessManager, please ensure you forcibly and explicitly enable handling of redirects. > > -- > Alexander Potashev Cheers, Ben
Re: Sysadmin Load Reduction: Code Related Services
пт, 15 нояб. 2019 г. в 22:05, Ben Cooksley : > > On Sat, Nov 16, 2019 at 3:27 AM Alexander Potashev > wrote: > > > > пт, 15 нояб. 2019 г. в 15:46, Harald Sitter : > > > > > > On Sat, Nov 9, 2019 at 11:37 PM Alexander Potashev > > > wrote: > > > > > > > > сб, 9 нояб. 2019 г. в 02:51, Ben Cooksley : > > > > > In the category of no longer in use, we have the compatibility > > > > > generator for the kde_projects.xml file. This was introduced when we > > > > > shutdown Redmine/Chiliproject and migrated to Phabricator, as a way of > > > > > keeping services that needed to discover a list of KDE Projects > > > > > functional. > > > > > > > > > > As we've since migrated to using YAML files within the > > > > > sysadmin/repo-metadata repository for both the CI System and > > > > > kdesrc-build (and with LXR using kdesrc-build to do it's code > > > > > checkouts) there shouldn't to my knowledge be anything still relying > > > > > on this (aside from perhaps scripty). > > > > > > > > > > I'd therefore like to shut this generator down as well, along with the > > > > > compaibility redirector running at projects.kde.org (given that it has > > > > > been some time since we were using that site, and many projects have > > > > > moved around in the virtual structure since then, making the redirects > > > > > it is able to offer useless) > > > > > > > > Hi Ben, > > > > > > > > I am developing a new version of the "opensrc" plugin for Lokalize [1] > > > > and it currently depends on kde_projects.xml. Of course I can add new > > > > code to scan the Git repo instead of just fetching kde_projects.xml, > > > > however it would be more complicated. > > > > > > https://projects.kde.org/api/doc/ specifically deals with this problem > > > by abstracting the repo away behind a micro service. > > > > This looks like another view of the data available in > > kde_projects.xml, however the API is very limited. For example I can't > > query repo descriptions using this API. Thus not helpful. > > > > However if we were going to kill kde_projects.xml, are you sure > > projects.kde.org/api/ would be still available and not shut down as > > well? > > The API that Harald mentions is based off the YAML files, and is not > reliant on the legacy kde_projects.xml file. I can implement kde_projects.xml or a modernized version of it as part of projects.kde.org/api/. Does this sound like a good idea? We don't need to support the exact same XML format, so I would prefer a JSON listing all projects with all their properties, at something like GET https://projects.kde.org/api/v1/export or may be return all project properties in GET https://projects.kde.org/api/v1/projects -- Alexander Potashev
Re: Sysadmin Load Reduction: Code Related Services
On Sat, Nov 16, 2019 at 3:27 AM Alexander Potashev wrote: > > пт, 15 нояб. 2019 г. в 15:46, Harald Sitter : > > > > On Sat, Nov 9, 2019 at 11:37 PM Alexander Potashev > > wrote: > > > > > > сб, 9 нояб. 2019 г. в 02:51, Ben Cooksley : > > > > In the category of no longer in use, we have the compatibility > > > > generator for the kde_projects.xml file. This was introduced when we > > > > shutdown Redmine/Chiliproject and migrated to Phabricator, as a way of > > > > keeping services that needed to discover a list of KDE Projects > > > > functional. > > > > > > > > As we've since migrated to using YAML files within the > > > > sysadmin/repo-metadata repository for both the CI System and > > > > kdesrc-build (and with LXR using kdesrc-build to do it's code > > > > checkouts) there shouldn't to my knowledge be anything still relying > > > > on this (aside from perhaps scripty). > > > > > > > > I'd therefore like to shut this generator down as well, along with the > > > > compaibility redirector running at projects.kde.org (given that it has > > > > been some time since we were using that site, and many projects have > > > > moved around in the virtual structure since then, making the redirects > > > > it is able to offer useless) > > > > > > Hi Ben, > > > > > > I am developing a new version of the "opensrc" plugin for Lokalize [1] > > > and it currently depends on kde_projects.xml. Of course I can add new > > > code to scan the Git repo instead of just fetching kde_projects.xml, > > > however it would be more complicated. > > > > https://projects.kde.org/api/doc/ specifically deals with this problem > > by abstracting the repo away behind a micro service. > > This looks like another view of the data available in > kde_projects.xml, however the API is very limited. For example I can't > query repo descriptions using this API. Thus not helpful. > > However if we were going to kill kde_projects.xml, are you sure > projects.kde.org/api/ would be still available and not shut down as > well? The API that Harald mentions is based off the YAML files, and is not reliant on the legacy kde_projects.xml file. > > -- > Alexander Potashev Cheers, Ben
Re: Sysadmin Load Reduction: Code Related Services
пт, 15 нояб. 2019 г. в 15:46, Harald Sitter : > > On Sat, Nov 9, 2019 at 11:37 PM Alexander Potashev > wrote: > > > > сб, 9 нояб. 2019 г. в 02:51, Ben Cooksley : > > > In the category of no longer in use, we have the compatibility > > > generator for the kde_projects.xml file. This was introduced when we > > > shutdown Redmine/Chiliproject and migrated to Phabricator, as a way of > > > keeping services that needed to discover a list of KDE Projects > > > functional. > > > > > > As we've since migrated to using YAML files within the > > > sysadmin/repo-metadata repository for both the CI System and > > > kdesrc-build (and with LXR using kdesrc-build to do it's code > > > checkouts) there shouldn't to my knowledge be anything still relying > > > on this (aside from perhaps scripty). > > > > > > I'd therefore like to shut this generator down as well, along with the > > > compaibility redirector running at projects.kde.org (given that it has > > > been some time since we were using that site, and many projects have > > > moved around in the virtual structure since then, making the redirects > > > it is able to offer useless) > > > > Hi Ben, > > > > I am developing a new version of the "opensrc" plugin for Lokalize [1] > > and it currently depends on kde_projects.xml. Of course I can add new > > code to scan the Git repo instead of just fetching kde_projects.xml, > > however it would be more complicated. > > https://projects.kde.org/api/doc/ specifically deals with this problem > by abstracting the repo away behind a micro service. This looks like another view of the data available in kde_projects.xml, however the API is very limited. For example I can't query repo descriptions using this API. Thus not helpful. However if we were going to kill kde_projects.xml, are you sure projects.kde.org/api/ would be still available and not shut down as well? -- Alexander Potashev
Re: Sysadmin Load Reduction: Code Related Services
On Sat, Nov 9, 2019 at 11:37 PM Alexander Potashev wrote: > > сб, 9 нояб. 2019 г. в 02:51, Ben Cooksley : > > In the category of no longer in use, we have the compatibility > > generator for the kde_projects.xml file. This was introduced when we > > shutdown Redmine/Chiliproject and migrated to Phabricator, as a way of > > keeping services that needed to discover a list of KDE Projects > > functional. > > > > As we've since migrated to using YAML files within the > > sysadmin/repo-metadata repository for both the CI System and > > kdesrc-build (and with LXR using kdesrc-build to do it's code > > checkouts) there shouldn't to my knowledge be anything still relying > > on this (aside from perhaps scripty). > > > > I'd therefore like to shut this generator down as well, along with the > > compaibility redirector running at projects.kde.org (given that it has > > been some time since we were using that site, and many projects have > > moved around in the virtual structure since then, making the redirects > > it is able to offer useless) > > Hi Ben, > > I am developing a new version of the "opensrc" plugin for Lokalize [1] > and it currently depends on kde_projects.xml. Of course I can add new > code to scan the Git repo instead of just fetching kde_projects.xml, > however it would be more complicated. https://projects.kde.org/api/doc/ specifically deals with this problem by abstracting the repo away behind a micro service. Yet another example of the poor service discoverability we noticed at the sysadmin bof I guess :/ HS
Re: Sysadmin Load Reduction: Code Related Services
On Sun, Nov 10, 2019 at 1:24 PM Albert Astals Cid wrote: > > El diumenge, 10 de novembre de 2019, a les 1:04:04 CET, Ben Cooksley va > escriure: > > On Sun, Nov 10, 2019 at 11:22 AM Albert Astals Cid wrote: > > > > > > El dissabte, 9 de novembre de 2019, a les 21:52:13 CET, Ben Cooksley va > > > escriure: > > > > On Sun, Nov 10, 2019 at 8:43 AM Albert Astals Cid wrote: > > > > > > > > > > El dissabte, 9 de novembre de 2019, a les 18:47:57 CET, Ben Cooksley > > > > > va escriure: > > > > > > On Sun, Nov 10, 2019 at 12:29 AM Luigi Toscano > > > > > > wrote: > > > > > > > > > > > > > > Ben Cooksley ha scritto: > > > > > > > > Hi all, > > > > > > > > > > > > > > > > In the category of code related services, Sysadmin currently > > > > > > > > supports > > > > > > > > a wide-ranging number of services (which makes sense given the > > > > > > > > nature > > > > > > > > of the community). Some of these however may no longer be in > > > > > > > > use or > > > > > > > > will be duplicative of other services following the transition > > > > > > > > to > > > > > > > > Gitlab. > > > > > > > > > > > > > > > > In the category of duplicative, we have our two CGit instances > > > > > > > > at > > > > > > > > cgit.kde.org and packaging.neon.kde.org. Prior to Gitlab, these > > > > > > > > were > > > > > > > > justifiable as there was no other way of browsing scratch/clone > > > > > > > > repositories over the web. > > > > > > > > > > > > > > > > With the migration to Gitlab however all repositories will > > > > > > > > become > > > > > > > > browsable through it, meaning it no longer makes sense to offer > > > > > > > > two > > > > > > > > browsers that provide the exact same information (with Gitlab > > > > > > > > having > > > > > > > > greater capabilities). I'd therefore like to shut both of these > > > > > > > > down > > > > > > > > once Code Hosting has transitioned to Gitlab. > > > > > > > > > > > > > > Luckily we tried to use commits.kde.org as generic redirect. Will > > > > > > > the generic > > > > > > > redirect commits.kde.org be updated to point to the proper > > > > > > > repository? It may > > > > > > > be complicated if the new structure contains the namespaces for > > > > > > > each > > > > > > > repository, but we need to keep it working. > > > > > > > > > > > > The commits.kde.org redirector is intended to provide permanent > > > > > > links, > > > > > > so yes it will be updated to redirect people to Gitlab instead once > > > > > > the switch to Gitlab is completed. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > In the category of no longer in use, we have the compatibility > > > > > > > > generator for the kde_projects.xml file. This was introduced > > > > > > > > when we > > > > > > > > shutdown Redmine/Chiliproject and migrated to Phabricator, as a > > > > > > > > way of > > > > > > > > keeping services that needed to discover a list of KDE Projects > > > > > > > > functional. > > > > > > > > > > > > > > > > As we've since migrated to using YAML files within the > > > > > > > > sysadmin/repo-metadata repository for both the CI System and > > > > > > > > kdesrc-build (and with LXR using kdesrc-build to do it's code > > > > > > > > checkouts) there shouldn't to my knowledge be anything still > > > > > > > > relying > > > > > > > > on this (aside from perhaps scripty).> > > > > > > > > I'd therefore like to shut this generator down as well, along > > > > > > > > with the > > > > > > > > compaibility redirector running at projects.kde.org (given that > > > > > > > > it has > > > > > > > > been some time since we were using that site, and many projects > > > > > > > > have > > > > > > > > moved around in the virtual structure since then, making the > > > > > > > > redirects > > > > > > > > it is able to offer useless) > > > > > > > > > > > > > > Two points: > > > > > > > - scripty still uses the XML file and we may need some time to > > > > > > > migrate. > > > > > > > > > > > > I feared this may have been the case. What sort of timeline are we > > > > > > looking at in terms of switching scripty over? > > > > > > > > > > Over to what? > > > > > > > > To using the YAML files within sysadmin/repo-metadata, which is what > > > > the legacy kde_projects.xml file is generated from currently. > > > > > > Well, someone has to change the code that parses the xml to code that > > > gets stuff from git and parses yaml, it's not hard, but has to be done. > > > > *nod* > > > > I do recall that at one point in our conversations that scripty only > > used the kde_projects.xml file as a check against it's own internal > > data. > > Yes and no, we have our i18n branch data hardcoded, but we don't have a list > of repos, so we need the list of repos from somewhere (and we use the branch > data from kde_projects.xml to compare it to our hardcoded one to see if > there's some mismatch). > Thanks for confirming how that is used. > Cheers, > Albert > Cheers, Ben > > Has that
Re: Sysadmin Load Reduction: Code Related Services
El diumenge, 10 de novembre de 2019, a les 1:04:04 CET, Ben Cooksley va escriure: > On Sun, Nov 10, 2019 at 11:22 AM Albert Astals Cid wrote: > > > > El dissabte, 9 de novembre de 2019, a les 21:52:13 CET, Ben Cooksley va > > escriure: > > > On Sun, Nov 10, 2019 at 8:43 AM Albert Astals Cid wrote: > > > > > > > > El dissabte, 9 de novembre de 2019, a les 18:47:57 CET, Ben Cooksley va > > > > escriure: > > > > > On Sun, Nov 10, 2019 at 12:29 AM Luigi Toscano > > > > > wrote: > > > > > > > > > > > > Ben Cooksley ha scritto: > > > > > > > Hi all, > > > > > > > > > > > > > > In the category of code related services, Sysadmin currently > > > > > > > supports > > > > > > > a wide-ranging number of services (which makes sense given the > > > > > > > nature > > > > > > > of the community). Some of these however may no longer be in use > > > > > > > or > > > > > > > will be duplicative of other services following the transition to > > > > > > > Gitlab. > > > > > > > > > > > > > > In the category of duplicative, we have our two CGit instances at > > > > > > > cgit.kde.org and packaging.neon.kde.org. Prior to Gitlab, these > > > > > > > were > > > > > > > justifiable as there was no other way of browsing scratch/clone > > > > > > > repositories over the web. > > > > > > > > > > > > > > With the migration to Gitlab however all repositories will become > > > > > > > browsable through it, meaning it no longer makes sense to offer > > > > > > > two > > > > > > > browsers that provide the exact same information (with Gitlab > > > > > > > having > > > > > > > greater capabilities). I'd therefore like to shut both of these > > > > > > > down > > > > > > > once Code Hosting has transitioned to Gitlab. > > > > > > > > > > > > Luckily we tried to use commits.kde.org as generic redirect. Will > > > > > > the generic > > > > > > redirect commits.kde.org be updated to point to the proper > > > > > > repository? It may > > > > > > be complicated if the new structure contains the namespaces for each > > > > > > repository, but we need to keep it working. > > > > > > > > > > The commits.kde.org redirector is intended to provide permanent links, > > > > > so yes it will be updated to redirect people to Gitlab instead once > > > > > the switch to Gitlab is completed. > > > > > > > > > > > > > > > > > > > > > > > > > In the category of no longer in use, we have the compatibility > > > > > > > generator for the kde_projects.xml file. This was introduced when > > > > > > > we > > > > > > > shutdown Redmine/Chiliproject and migrated to Phabricator, as a > > > > > > > way of > > > > > > > keeping services that needed to discover a list of KDE Projects > > > > > > > functional. > > > > > > > > > > > > > > As we've since migrated to using YAML files within the > > > > > > > sysadmin/repo-metadata repository for both the CI System and > > > > > > > kdesrc-build (and with LXR using kdesrc-build to do it's code > > > > > > > checkouts) there shouldn't to my knowledge be anything still > > > > > > > relying > > > > > > > on this (aside from perhaps scripty).> > > > > > > > I'd therefore like to shut this generator down as well, along > > > > > > > with the > > > > > > > compaibility redirector running at projects.kde.org (given that > > > > > > > it has > > > > > > > been some time since we were using that site, and many projects > > > > > > > have > > > > > > > moved around in the virtual structure since then, making the > > > > > > > redirects > > > > > > > it is able to offer useless) > > > > > > > > > > > > Two points: > > > > > > - scripty still uses the XML file and we may need some time to > > > > > > migrate. > > > > > > > > > > I feared this may have been the case. What sort of timeline are we > > > > > looking at in terms of switching scripty over? > > > > > > > > Over to what? > > > > > > To using the YAML files within sysadmin/repo-metadata, which is what > > > the legacy kde_projects.xml file is generated from currently. > > > > Well, someone has to change the code that parses the xml to code that gets > > stuff from git and parses yaml, it's not hard, but has to be done. > > *nod* > > I do recall that at one point in our conversations that scripty only > used the kde_projects.xml file as a check against it's own internal > data. Yes and no, we have our i18n branch data hardcoded, but we don't have a list of repos, so we need the list of repos from somewhere (and we use the branch data from kde_projects.xml to compare it to our hardcoded one to see if there's some mismatch). Cheers, Albert > Has that since been changed? > > > > > > (Originally the kde_projects.xml file was generated from the > > > Redmine/Chiliproject database - those YAML files just replaced it) > > > > Yes i know :) > > > > Cheers, > > Albert > > Cheers, > Ben > > > > > > > > > > > > > > Cheers, > > > > Albert > > > > > > > > > > > > > > Thanks, > > > Ben > > > > > > > > > > > >
Re: Sysadmin Load Reduction: Code Related Services
On Sun, Nov 10, 2019 at 11:22 AM Albert Astals Cid wrote: > > El dissabte, 9 de novembre de 2019, a les 21:52:13 CET, Ben Cooksley va > escriure: > > On Sun, Nov 10, 2019 at 8:43 AM Albert Astals Cid wrote: > > > > > > El dissabte, 9 de novembre de 2019, a les 18:47:57 CET, Ben Cooksley va > > > escriure: > > > > On Sun, Nov 10, 2019 at 12:29 AM Luigi Toscano > > > > wrote: > > > > > > > > > > Ben Cooksley ha scritto: > > > > > > Hi all, > > > > > > > > > > > > In the category of code related services, Sysadmin currently > > > > > > supports > > > > > > a wide-ranging number of services (which makes sense given the > > > > > > nature > > > > > > of the community). Some of these however may no longer be in use or > > > > > > will be duplicative of other services following the transition to > > > > > > Gitlab. > > > > > > > > > > > > In the category of duplicative, we have our two CGit instances at > > > > > > cgit.kde.org and packaging.neon.kde.org. Prior to Gitlab, these were > > > > > > justifiable as there was no other way of browsing scratch/clone > > > > > > repositories over the web. > > > > > > > > > > > > With the migration to Gitlab however all repositories will become > > > > > > browsable through it, meaning it no longer makes sense to offer two > > > > > > browsers that provide the exact same information (with Gitlab having > > > > > > greater capabilities). I'd therefore like to shut both of these down > > > > > > once Code Hosting has transitioned to Gitlab. > > > > > > > > > > Luckily we tried to use commits.kde.org as generic redirect. Will the > > > > > generic > > > > > redirect commits.kde.org be updated to point to the proper > > > > > repository? It may > > > > > be complicated if the new structure contains the namespaces for each > > > > > repository, but we need to keep it working. > > > > > > > > The commits.kde.org redirector is intended to provide permanent links, > > > > so yes it will be updated to redirect people to Gitlab instead once > > > > the switch to Gitlab is completed. > > > > > > > > > > > > > > > > > > > > > In the category of no longer in use, we have the compatibility > > > > > > generator for the kde_projects.xml file. This was introduced when we > > > > > > shutdown Redmine/Chiliproject and migrated to Phabricator, as a way > > > > > > of > > > > > > keeping services that needed to discover a list of KDE Projects > > > > > > functional. > > > > > > > > > > > > As we've since migrated to using YAML files within the > > > > > > sysadmin/repo-metadata repository for both the CI System and > > > > > > kdesrc-build (and with LXR using kdesrc-build to do it's code > > > > > > checkouts) there shouldn't to my knowledge be anything still relying > > > > > > on this (aside from perhaps scripty).> > > > > > > I'd therefore like to shut this generator down as well, along with > > > > > > the > > > > > > compaibility redirector running at projects.kde.org (given that it > > > > > > has > > > > > > been some time since we were using that site, and many projects have > > > > > > moved around in the virtual structure since then, making the > > > > > > redirects > > > > > > it is able to offer useless) > > > > > > > > > > Two points: > > > > > - scripty still uses the XML file and we may need some time to > > > > > migrate. > > > > > > > > I feared this may have been the case. What sort of timeline are we > > > > looking at in terms of switching scripty over? > > > > > > Over to what? > > > > To using the YAML files within sysadmin/repo-metadata, which is what > > the legacy kde_projects.xml file is generated from currently. > > Well, someone has to change the code that parses the xml to code that gets > stuff from git and parses yaml, it's not hard, but has to be done. *nod* I do recall that at one point in our conversations that scripty only used the kde_projects.xml file as a check against it's own internal data. Has that since been changed? > > > (Originally the kde_projects.xml file was generated from the > > Redmine/Chiliproject database - those YAML files just replaced it) > > Yes i know :) > > Cheers, > Albert Cheers, Ben > > > > > > > > > Cheers, > > > Albert > > > > > > > > > > Thanks, > > Ben > > > > > >
Re: Sysadmin Load Reduction: Code Related Services
On Sun, Nov 10, 2019 at 11:37 AM Alexander Potashev wrote: > > сб, 9 нояб. 2019 г. в 02:51, Ben Cooksley : > > In the category of no longer in use, we have the compatibility > > generator for the kde_projects.xml file. This was introduced when we > > shutdown Redmine/Chiliproject and migrated to Phabricator, as a way of > > keeping services that needed to discover a list of KDE Projects > > functional. > > > > As we've since migrated to using YAML files within the > > sysadmin/repo-metadata repository for both the CI System and > > kdesrc-build (and with LXR using kdesrc-build to do it's code > > checkouts) there shouldn't to my knowledge be anything still relying > > on this (aside from perhaps scripty). > > > > I'd therefore like to shut this generator down as well, along with the > > compaibility redirector running at projects.kde.org (given that it has > > been some time since we were using that site, and many projects have > > moved around in the virtual structure since then, making the redirects > > it is able to offer useless) > > Hi Ben, Hi Alexander, > > I am developing a new version of the "opensrc" plugin for Lokalize [1] > and it currently depends on kde_projects.xml. Of course I can add new > code to scan the Git repo instead of just fetching kde_projects.xml, > however it would be more complicated. > > > Is there any good reason for killing kde_projects.xml? Would that > really reduce the workload on Sysadmin team? > I can't suspect any "moving parts" in the generator that may require > maintenance. > This is mostly a case of another (small) thing that needs to be kept running, and fixed should it break. All of the reductions i've proposed will have a quite small effect, however cumulatively they will have a much larger effect. Given that it is duplicative of what we have in the YAML files (which are substantially easier to maintain) it makes more sense to standardise on the original source of the information and remove the legacy compatibility XML file. > > [1] https://cgit.kde.org/scratch/aspotashev/lokalize-opensrc-py.git/ > > -- > Alexander Potashev Cheers, Ben
Re: Sysadmin Load Reduction: Code Related Services
сб, 9 нояб. 2019 г. в 02:51, Ben Cooksley : > In the category of no longer in use, we have the compatibility > generator for the kde_projects.xml file. This was introduced when we > shutdown Redmine/Chiliproject and migrated to Phabricator, as a way of > keeping services that needed to discover a list of KDE Projects > functional. > > As we've since migrated to using YAML files within the > sysadmin/repo-metadata repository for both the CI System and > kdesrc-build (and with LXR using kdesrc-build to do it's code > checkouts) there shouldn't to my knowledge be anything still relying > on this (aside from perhaps scripty). > > I'd therefore like to shut this generator down as well, along with the > compaibility redirector running at projects.kde.org (given that it has > been some time since we were using that site, and many projects have > moved around in the virtual structure since then, making the redirects > it is able to offer useless) Hi Ben, I am developing a new version of the "opensrc" plugin for Lokalize [1] and it currently depends on kde_projects.xml. Of course I can add new code to scan the Git repo instead of just fetching kde_projects.xml, however it would be more complicated. Is there any good reason for killing kde_projects.xml? Would that really reduce the workload on Sysadmin team? I can't suspect any "moving parts" in the generator that may require maintenance. [1] https://cgit.kde.org/scratch/aspotashev/lokalize-opensrc-py.git/ -- Alexander Potashev
Re: Sysadmin Load Reduction: Code Related Services
El dissabte, 9 de novembre de 2019, a les 21:52:13 CET, Ben Cooksley va escriure: > On Sun, Nov 10, 2019 at 8:43 AM Albert Astals Cid wrote: > > > > El dissabte, 9 de novembre de 2019, a les 18:47:57 CET, Ben Cooksley va > > escriure: > > > On Sun, Nov 10, 2019 at 12:29 AM Luigi Toscano > > > wrote: > > > > > > > > Ben Cooksley ha scritto: > > > > > Hi all, > > > > > > > > > > In the category of code related services, Sysadmin currently supports > > > > > a wide-ranging number of services (which makes sense given the nature > > > > > of the community). Some of these however may no longer be in use or > > > > > will be duplicative of other services following the transition to > > > > > Gitlab. > > > > > > > > > > In the category of duplicative, we have our two CGit instances at > > > > > cgit.kde.org and packaging.neon.kde.org. Prior to Gitlab, these were > > > > > justifiable as there was no other way of browsing scratch/clone > > > > > repositories over the web. > > > > > > > > > > With the migration to Gitlab however all repositories will become > > > > > browsable through it, meaning it no longer makes sense to offer two > > > > > browsers that provide the exact same information (with Gitlab having > > > > > greater capabilities). I'd therefore like to shut both of these down > > > > > once Code Hosting has transitioned to Gitlab. > > > > > > > > Luckily we tried to use commits.kde.org as generic redirect. Will the > > > > generic > > > > redirect commits.kde.org be updated to point to the proper repository? > > > > It may > > > > be complicated if the new structure contains the namespaces for each > > > > repository, but we need to keep it working. > > > > > > The commits.kde.org redirector is intended to provide permanent links, > > > so yes it will be updated to redirect people to Gitlab instead once > > > the switch to Gitlab is completed. > > > > > > > > > > > > > > > > > In the category of no longer in use, we have the compatibility > > > > > generator for the kde_projects.xml file. This was introduced when we > > > > > shutdown Redmine/Chiliproject and migrated to Phabricator, as a way of > > > > > keeping services that needed to discover a list of KDE Projects > > > > > functional. > > > > > > > > > > As we've since migrated to using YAML files within the > > > > > sysadmin/repo-metadata repository for both the CI System and > > > > > kdesrc-build (and with LXR using kdesrc-build to do it's code > > > > > checkouts) there shouldn't to my knowledge be anything still relying > > > > > on this (aside from perhaps scripty).> > > > > > I'd therefore like to shut this generator down as well, along with the > > > > > compaibility redirector running at projects.kde.org (given that it has > > > > > been some time since we were using that site, and many projects have > > > > > moved around in the virtual structure since then, making the redirects > > > > > it is able to offer useless) > > > > > > > > Two points: > > > > - scripty still uses the XML file and we may need some time to migrate. > > > > > > I feared this may have been the case. What sort of timeline are we > > > looking at in terms of switching scripty over? > > > > Over to what? > > To using the YAML files within sysadmin/repo-metadata, which is what > the legacy kde_projects.xml file is generated from currently. Well, someone has to change the code that parses the xml to code that gets stuff from git and parses yaml, it's not hard, but has to be done. > (Originally the kde_projects.xml file was generated from the > Redmine/Chiliproject database - those YAML files just replaced it) Yes i know :) Cheers, Albert > > > > > Cheers, > > Albert > > > > > > Thanks, > Ben >
Re: Sysadmin Load Reduction: Code Related Services
On Sun, Nov 10, 2019 at 8:43 AM Albert Astals Cid wrote: > > El dissabte, 9 de novembre de 2019, a les 18:47:57 CET, Ben Cooksley va > escriure: > > On Sun, Nov 10, 2019 at 12:29 AM Luigi Toscano > > wrote: > > > > > > Ben Cooksley ha scritto: > > > > Hi all, > > > > > > > > In the category of code related services, Sysadmin currently supports > > > > a wide-ranging number of services (which makes sense given the nature > > > > of the community). Some of these however may no longer be in use or > > > > will be duplicative of other services following the transition to > > > > Gitlab. > > > > > > > > In the category of duplicative, we have our two CGit instances at > > > > cgit.kde.org and packaging.neon.kde.org. Prior to Gitlab, these were > > > > justifiable as there was no other way of browsing scratch/clone > > > > repositories over the web. > > > > > > > > With the migration to Gitlab however all repositories will become > > > > browsable through it, meaning it no longer makes sense to offer two > > > > browsers that provide the exact same information (with Gitlab having > > > > greater capabilities). I'd therefore like to shut both of these down > > > > once Code Hosting has transitioned to Gitlab. > > > > > > Luckily we tried to use commits.kde.org as generic redirect. Will the > > > generic > > > redirect commits.kde.org be updated to point to the proper repository? It > > > may > > > be complicated if the new structure contains the namespaces for each > > > repository, but we need to keep it working. > > > > The commits.kde.org redirector is intended to provide permanent links, > > so yes it will be updated to redirect people to Gitlab instead once > > the switch to Gitlab is completed. > > > > > > > > > > > > > In the category of no longer in use, we have the compatibility > > > > generator for the kde_projects.xml file. This was introduced when we > > > > shutdown Redmine/Chiliproject and migrated to Phabricator, as a way of > > > > keeping services that needed to discover a list of KDE Projects > > > > functional. > > > > > > > > As we've since migrated to using YAML files within the > > > > sysadmin/repo-metadata repository for both the CI System and > > > > kdesrc-build (and with LXR using kdesrc-build to do it's code > > > > checkouts) there shouldn't to my knowledge be anything still relying > > > > on this (aside from perhaps scripty).> > > > > I'd therefore like to shut this generator down as well, along with the > > > > compaibility redirector running at projects.kde.org (given that it has > > > > been some time since we were using that site, and many projects have > > > > moved around in the virtual structure since then, making the redirects > > > > it is able to offer useless) > > > > > > Two points: > > > - scripty still uses the XML file and we may need some time to migrate. > > > > I feared this may have been the case. What sort of timeline are we > > looking at in terms of switching scripty over? > > Over to what? To using the YAML files within sysadmin/repo-metadata, which is what the legacy kde_projects.xml file is generated from currently. (Originally the kde_projects.xml file was generated from the Redmine/Chiliproject database - those YAML files just replaced it) > > Cheers, > Albert > > Thanks, Ben
Re: Sysadmin Load Reduction: Code Related Services
El dissabte, 9 de novembre de 2019, a les 18:47:57 CET, Ben Cooksley va escriure: > On Sun, Nov 10, 2019 at 12:29 AM Luigi Toscano > wrote: > > > > Ben Cooksley ha scritto: > > > Hi all, > > > > > > In the category of code related services, Sysadmin currently supports > > > a wide-ranging number of services (which makes sense given the nature > > > of the community). Some of these however may no longer be in use or > > > will be duplicative of other services following the transition to > > > Gitlab. > > > > > > In the category of duplicative, we have our two CGit instances at > > > cgit.kde.org and packaging.neon.kde.org. Prior to Gitlab, these were > > > justifiable as there was no other way of browsing scratch/clone > > > repositories over the web. > > > > > > With the migration to Gitlab however all repositories will become > > > browsable through it, meaning it no longer makes sense to offer two > > > browsers that provide the exact same information (with Gitlab having > > > greater capabilities). I'd therefore like to shut both of these down > > > once Code Hosting has transitioned to Gitlab. > > > > Luckily we tried to use commits.kde.org as generic redirect. Will the > > generic > > redirect commits.kde.org be updated to point to the proper repository? It > > may > > be complicated if the new structure contains the namespaces for each > > repository, but we need to keep it working. > > The commits.kde.org redirector is intended to provide permanent links, > so yes it will be updated to redirect people to Gitlab instead once > the switch to Gitlab is completed. > > > > > > > > > In the category of no longer in use, we have the compatibility > > > generator for the kde_projects.xml file. This was introduced when we > > > shutdown Redmine/Chiliproject and migrated to Phabricator, as a way of > > > keeping services that needed to discover a list of KDE Projects > > > functional. > > > > > > As we've since migrated to using YAML files within the > > > sysadmin/repo-metadata repository for both the CI System and > > > kdesrc-build (and with LXR using kdesrc-build to do it's code > > > checkouts) there shouldn't to my knowledge be anything still relying > > > on this (aside from perhaps scripty).> > > > I'd therefore like to shut this generator down as well, along with the > > > compaibility redirector running at projects.kde.org (given that it has > > > been some time since we were using that site, and many projects have > > > moved around in the virtual structure since then, making the redirects > > > it is able to offer useless) > > > > Two points: > > - scripty still uses the XML file and we may need some time to migrate. > > I feared this may have been the case. What sort of timeline are we > looking at in terms of switching scripty over? Over to what? Cheers, Albert
Re: Sysadmin Load Reduction: Code Related Services
On Sun, Nov 10, 2019 at 12:29 AM Luigi Toscano wrote: > > Ben Cooksley ha scritto: > > Hi all, > > > > In the category of code related services, Sysadmin currently supports > > a wide-ranging number of services (which makes sense given the nature > > of the community). Some of these however may no longer be in use or > > will be duplicative of other services following the transition to > > Gitlab. > > > > In the category of duplicative, we have our two CGit instances at > > cgit.kde.org and packaging.neon.kde.org. Prior to Gitlab, these were > > justifiable as there was no other way of browsing scratch/clone > > repositories over the web. > > > > With the migration to Gitlab however all repositories will become > > browsable through it, meaning it no longer makes sense to offer two > > browsers that provide the exact same information (with Gitlab having > > greater capabilities). I'd therefore like to shut both of these down > > once Code Hosting has transitioned to Gitlab. > > Luckily we tried to use commits.kde.org as generic redirect. Will the generic > redirect commits.kde.org be updated to point to the proper repository? It may > be complicated if the new structure contains the namespaces for each > repository, but we need to keep it working. The commits.kde.org redirector is intended to provide permanent links, so yes it will be updated to redirect people to Gitlab instead once the switch to Gitlab is completed. > > > > > In the category of no longer in use, we have the compatibility > > generator for the kde_projects.xml file. This was introduced when we > > shutdown Redmine/Chiliproject and migrated to Phabricator, as a way of > > keeping services that needed to discover a list of KDE Projects > > functional. > > > > As we've since migrated to using YAML files within the > > sysadmin/repo-metadata repository for both the CI System and > > kdesrc-build (and with LXR using kdesrc-build to do it's code > > checkouts) there shouldn't to my knowledge be anything still relying > > on this (aside from perhaps scripty).> > > I'd therefore like to shut this generator down as well, along with the > > compaibility redirector running at projects.kde.org (given that it has > > been some time since we were using that site, and many projects have > > moved around in the virtual structure since then, making the redirects > > it is able to offer useless) > > Two points: > - scripty still uses the XML file and we may need some time to migrate. I feared this may have been the case. What sort of timeline are we looking at in terms of switching scripty over? > - we may have a few links around which still points to projects.kde.org (for > example, the "Consistency" goal listed a few of them), so we may need to go > through all of them before doing that. > Do we have measures that shows how much projects.kde.org is hit by services > which are not scripty? I've reviewed the logs for the past 24 hours and it would appear the majority of the legitimate hits now are for the RSS/ATOM feeds that Chiliproject used to offer and which have no corresponding replacement (amazingly, these are still configured in people's feed readers). After excluding bots and spiders, and people running automated tools against old URL sets, very little in the way of hits remained. The remainder are a very small proportion of hits from places such as the Archlinux wiki, which we can likely request to be updated (referencing Bluedevil in particular it would seem). > > -- > Luigi Cheers, Ben
Re: Sysadmin Load Reduction: Code Related Services
Ben Cooksley ha scritto: > Hi all, > > In the category of code related services, Sysadmin currently supports > a wide-ranging number of services (which makes sense given the nature > of the community). Some of these however may no longer be in use or > will be duplicative of other services following the transition to > Gitlab. > > In the category of duplicative, we have our two CGit instances at > cgit.kde.org and packaging.neon.kde.org. Prior to Gitlab, these were > justifiable as there was no other way of browsing scratch/clone > repositories over the web. > > With the migration to Gitlab however all repositories will become > browsable through it, meaning it no longer makes sense to offer two > browsers that provide the exact same information (with Gitlab having > greater capabilities). I'd therefore like to shut both of these down > once Code Hosting has transitioned to Gitlab. Luckily we tried to use commits.kde.org as generic redirect. Will the generic redirect commits.kde.org be updated to point to the proper repository? It may be complicated if the new structure contains the namespaces for each repository, but we need to keep it working. > > In the category of no longer in use, we have the compatibility > generator for the kde_projects.xml file. This was introduced when we > shutdown Redmine/Chiliproject and migrated to Phabricator, as a way of > keeping services that needed to discover a list of KDE Projects > functional. > > As we've since migrated to using YAML files within the > sysadmin/repo-metadata repository for both the CI System and > kdesrc-build (and with LXR using kdesrc-build to do it's code > checkouts) there shouldn't to my knowledge be anything still relying > on this (aside from perhaps scripty).> > I'd therefore like to shut this generator down as well, along with the > compaibility redirector running at projects.kde.org (given that it has > been some time since we were using that site, and many projects have > moved around in the virtual structure since then, making the redirects > it is able to offer useless) Two points: - scripty still uses the XML file and we may need some time to migrate. - we may have a few links around which still points to projects.kde.org (for example, the "Consistency" goal listed a few of them), so we may need to go through all of them before doing that. Do we have measures that shows how much projects.kde.org is hit by services which are not scripty? -- Luigi
