Re: Sysadmin Load Reduction: Legacy Compatibility Redirects

2019-11-16 Thread Ben Cooksley
On Sat, Nov 9, 2019 at 1:02 PM Ben Cooksley  wrote:
>
> Hi all,

Hi all,

>
> One of the more smaller things that Sysadmin currently looks after is
> a large number of legacy compatibility redirects, which keep a variety
> of subdomains under KDE.org functional.
>
> For the most part these refer to dead projects, and have been legacy
> compatibility redirects for many years (5+) now.
>
> Given that sites should have now had a chance to update themselves,
> i'd like to go ahead and remove a number of these redirects.
>
> These redirects, whilst appearing relatively minor in nature, do
> require a certain degree of custom logic on the server side to handle
> them and therefore collectively create maintenance burden that in many
> cases probably outweighs the value they provide.
>
> We therefore should only retain them if there are places we are still
> unable to update which someone may need to follow (and not simply
> because 'somewhere might still link there') given that most people
> find things through their preferred search engine now.
>
> Below are a list of all the redirect candidates:
> --
> kate.kde.org
> accessibility.kde.org
> books.kde.org
> buzz.kde.org
> de.kde.org
> dolphin.kde.org
> es.kde.org
> evolve.kde.org
> gwenview.kde.org
> kaffeine.kde.org
> kmail.kde.org
> kmobiletools.kde.org
> kopete.kde.org
> korganizer.kde.org
> korganizer.org
> lokalize.kde.org
> people.kde.org
> phonon.kde.org
> pim.kde.org
> kdepim.org
> kdepim.com
> plasma.kde.org
> solaris.kde.org
> themes.kde.org
> usability.kde.org
> vdesign.kde.org
> windows.kde.org
> women.kde.org
> yakuake.kde.org
> contour.kde.org
> brainstorm.forum.kde.org
> research.kde.org
> akregator.kde.org
> nepomuk.kde.org
> in.kde.org
> people.kde.org
> mac.kde.org
> kdevelop.kde.org
> ar.kde.org
> --
>
> Any comments?
>
> I'd also like to be able to recommend to the KDE e.V. Board that we
> permit kdepim.org, kdepim.com and korganizer.org to expire at the end
> of their current registration period.

I have now gone ahead and removed those various redirect hosts from our systems.

In total 44 redirect hosts were removed, which reduced the size of our
zone file (excluding the various *.*.l10n.kde.org entries) from 432
lines to 388 lines (a reduction of 10%).
Additionally, for one system it has now made it substantially easier
to see the services it does operate.

>
> Thanks,
> Ben

Cheers,
Ben


Re: Sysadmin Load Reduction: Communication Services

2019-11-16 Thread Ben Cooksley
On Sat, Nov 9, 2019 at 12:49 PM Ben Cooksley  wrote:
>
> Hi all,

Hi all,

>
> In the category of Communication Services, we currently run a couple
> of things which appear to be of limited use now to the broader
> community.

Given the lack of response to this, we're going to assume all is well
for us to proceed with the below.

>
> The items i'd like to remove here are:
>
> - KDETalk.net Jabber Server: This was subject to registration abuse
> sometime back, and due to a lack of anti-spam measures within Jabber
> servers in general, we were forced to disable public registration on
> this service.
>
> Given that the community in general leans to preferring Email, IRC,
> Telegram and Matrix (in no particular order) for it's communications,
> it does not appear to make sense for us to continue to operate this.
>
> - 'insanity' and 'Amarok' IRC Bots: These are hosted on behalf of the
> Amarok project. It would appear that some time back the 'Amarok' bot
> crashed, and given that we've yet to receive a report regarding this,
> it appears that the bot is no longer in use. Given that Amarok has yet
> to make a KF5 based release and activity is very minimal, i'd like to
> shutdown and archive both bots in this instance.

These have both been shutdown, along with 'Kikibot' which didn't
appear to be running but which did have an account on the same system
used to host these two bots.

>
> Regards,
> Ben

Cheers,
Ben


Re: Sysadmin Load Reduction: Project Specific Sites

2019-11-16 Thread Ben Cooksley
On Sat, Nov 9, 2019 at 12:50 PM Ben Cooksley  wrote:
>
> Hi all,

Hi all,

>
> In terms of project specific sites, we are in a reasonably good state
> here, with most sites being based off either a standard CMS or static
> site generator (which are relatively easy to maintain in bulk with the
> incremental effort being much lower).
>
> There are some exceptions though, and in the case where the service no
> longer appears to be in use, or the project in question now inactive,
> it makes more sense for us to render the sites into either static
> copies, or to discontinue the service.
>
> I'd therefore like to propose that the following websites be converted
> into static copies:

As we've not received any commentary on the below, with the exception
of RKWard, I have now gone ahead and started the process of actioning
this.

>
> - Commit Digest (commit-digest.org): The last time this was updated
> was back in 2014, with the most recent issue there being nearly 5
> years old.
>

The process of converting this to a static site is now underway.

> - RKWard (rkward.kde.org): This site is currently based on Mediawiki,
> which is substantially more difficult to maintain and keep updated
> compared to a site built using Hugo, Jekyll, Wordpress or Drupal.
>
> Given that the site was last updated back in early 2018, i'd like to
> make this static, and should updates need to be made in the future the
> content can then be converted at that time into something more easy to
> maintain.
>

Following discussion with the RKWard developers and thanks to the help
of Carl, this has now been converted to a Jekyll site.

> - Simon (simon.kde.org): While this site is based on Drupal, the
> project itself hasn't seen a release now since 2013, more than 5 years
> ago. Given that this seems to be unlikely to change soon, it makes
> more sense at this stage to remove even the incremental cost of
> maintaining it's Drupal instance, and convert it to a static site.
>

This has now been made static, and the Drupal instance archived.

> - Marble (marble.kde.org): Whilst this site is for the most part a
> minimally dynamic site, it has a component to it which essentially
> replicates Planet, just having Marble only postings. This necessitates
> custom logic on the server side, which is something no other site
> (including www.kde.org) has.
>
> I'd therefore like to eliminate this logic (leaving the rest of the
> site as it is)
>

This is in the process of being scheduled currently.

> - Vvave Stream (vvave-stream.kde.org): This site was originally
> created as an "online platform for the Babe music player" (which was
> subsequently renamed to Vvave).
>
> Based on server logs however it appears to be essentially unused, and
> given that it is Python based (which is one of the more maintenance
> intensive forms of hosting we provide) i'd like to shut this down.
>

This has now been archived and shutdown.

> Any comments regarding the above?
>
> Cheers,
> Ben

Thanks,
Ben


Re: Sysadmin Load Reduction: Code Related Services

2019-11-16 Thread Ben Cooksley
On Sun, Nov 17, 2019 at 5:19 AM Carl Schwan  wrote:
>
> Hi all,

Hi Carl,

>
> Can the gitlab api be of useful in the future?
>
> e.g https://invent.kde.org/api/v4/groups/7

While for many purposes Gitlab's API wil be perfectly fine, it doesn't
provide us with the i18n branch information which some users will
require.

>
> Cheers,
> Carl
>

Cheers,
Ben

> ‐‐‐ Original Message ‐‐‐
> On Saturday, November 16, 2019 9:51 AM, Ben Cooksley  
> wrote:
>
> > On Sat, Nov 16, 2019 at 10:39 PM Albert Astals Cid aa...@kde.org wrote:
> >
>
> > > El dissabte, 16 de novembre de 2019, a les 10:14:15 CET, Ben Cooksley va 
> > > escriure:
> > >
>
> > > > On Sat, Nov 16, 2019 at 3:58 PM Alexander Potashev aspotas...@gmail.com 
> > > > wrote:
> > > >
>
> > > > > пт, 15 нояб. 2019 г. в 22:05, Ben Cooksley bcooks...@kde.org:
> > > > >
>
> > > > > > On Sat, Nov 16, 2019 at 3:27 AM Alexander Potashev 
> > > > > > aspotas...@gmail.com wrote:
> > > > > >
>
> > > > > > > пт, 15 нояб. 2019 г. в 15:46, Harald Sitter sit...@kde.org:
> > > > > > >
>
> > > > > > > > On Sat, Nov 9, 2019 at 11:37 PM Alexander Potashev 
> > > > > > > > aspotas...@gmail.com wrote:
> > > > > > > >
>
> > > > > > > > > сб, 9 нояб. 2019 г. в 02:51, Ben Cooksley bcooks...@kde.org:
> > > > > > > > >
>
> > > > > > > > > > In the category of no longer in use, we have the 
> > > > > > > > > > compatibility
> > > > > > > > > > generator for the kde_projects.xml file. This was 
> > > > > > > > > > introduced when we
> > > > > > > > > > shutdown Redmine/Chiliproject and migrated to Phabricator, 
> > > > > > > > > > as a way of
> > > > > > > > > > keeping services that needed to discover a list of KDE 
> > > > > > > > > > Projects
> > > > > > > > > > functional.
> > > > > > > > > > As we've since migrated to using YAML files within the
> > > > > > > > > > sysadmin/repo-metadata repository for both the CI System and
> > > > > > > > > > kdesrc-build (and with LXR using kdesrc-build to do it's 
> > > > > > > > > > code
> > > > > > > > > > checkouts) there shouldn't to my knowledge be anything 
> > > > > > > > > > still relying
> > > > > > > > > > on this (aside from perhaps scripty).
> > > > > > > > > > I'd therefore like to shut this generator down as well, 
> > > > > > > > > > along with the
> > > > > > > > > > compaibility redirector running at projects.kde.org (given 
> > > > > > > > > > that it has
> > > > > > > > > > been some time since we were using that site, and many 
> > > > > > > > > > projects have
> > > > > > > > > > moved around in the virtual structure since then, making 
> > > > > > > > > > the redirects
> > > > > > > > > > it is able to offer useless)
> > > > > > > > >
>
> > > > > > > > > Hi Ben,
> > > > > > > > > I am developing a new version of the "opensrc" plugin for 
> > > > > > > > > Lokalize [1]
> > > > > > > > > and it currently depends on kde_projects.xml. Of course I can 
> > > > > > > > > add new
> > > > > > > > > code to scan the Git repo instead of just fetching 
> > > > > > > > > kde_projects.xml,
> > > > > > > > > however it would be more complicated.
> > > > > > > >
>
> > > > > > > > https://projects.kde.org/api/doc/ specifically deals with this 
> > > > > > > > problem
> > > > > > > > by abstracting the repo away behind a micro service.
> > > > > > >
>
> > > > > > > This looks like another view of the data available in
> > > > > > > kde_projects.xml, however the API is very limited. For example I 
> > > > > > > can't
> > > > > > > query repo descriptions using this API. Thus not helpful.
> > > > > > > However if we were going to kill kde_projects.xml, are you sure
> > > > > > > projects.kde.org/api/ would be still available and not shut down 
> > > > > > > as
> > > > > > > well?
> > > > > >
>
> > > > > > The API that Harald mentions is based off the YAML files, and is not
> > > > > > reliant on the legacy kde_projects.xml file.
> > > > >
>
> > > > > I can implement kde_projects.xml or a modernized version of it as part
> > > > > of projects.kde.org/api/. Does this sound like a good idea?
> > > > > We don't need to support the exact same XML format, so I would prefer
> > > > > a JSON listing all projects with all their properties, at something
> > > > > like GET https://projects.kde.org/api/v1/export or may be return all
> > > > > project properties in GET https://projects.kde.org/api/v1/projects
> > > >
>
> > > > I'll leave it to Harald to comment on whether he'd be happy having
> > > > additional capabilities in the Projects Microservice API he maintains.
> > > > (Harald, I assume it's only requirement is a copy of the
> > > > sysadmin/repo-metadata repository locally, and it doesn't require the
> > > > actual Git repositories?)
> > > > We really should avoid continuing to keep legacy endpoints alive
> > > > though (as it is just shifting the maintenance effort of porting away
> > > > from it further down the road), especially for something that is going
> > > > to end up on end user systems.
> > >
>
> > > Wait, are you saying we 

Re: Sysadmin Load Reduction: Code Related Services

2019-11-16 Thread Carl Schwan
Hi all,

Can the gitlab api be of useful in the future?

e.g https://invent.kde.org/api/v4/groups/7

Cheers,
Carl

‐‐‐ Original Message ‐‐‐
On Saturday, November 16, 2019 9:51 AM, Ben Cooksley  wrote:

> On Sat, Nov 16, 2019 at 10:39 PM Albert Astals Cid aa...@kde.org wrote:
> 

> > El dissabte, 16 de novembre de 2019, a les 10:14:15 CET, Ben Cooksley va 
> > escriure:
> > 

> > > On Sat, Nov 16, 2019 at 3:58 PM Alexander Potashev aspotas...@gmail.com 
> > > wrote:
> > > 

> > > > пт, 15 нояб. 2019 г. в 22:05, Ben Cooksley bcooks...@kde.org:
> > > > 

> > > > > On Sat, Nov 16, 2019 at 3:27 AM Alexander Potashev 
> > > > > aspotas...@gmail.com wrote:
> > > > > 

> > > > > > пт, 15 нояб. 2019 г. в 15:46, Harald Sitter sit...@kde.org:
> > > > > > 

> > > > > > > On Sat, Nov 9, 2019 at 11:37 PM Alexander Potashev 
> > > > > > > aspotas...@gmail.com wrote:
> > > > > > > 

> > > > > > > > сб, 9 нояб. 2019 г. в 02:51, Ben Cooksley bcooks...@kde.org:
> > > > > > > > 

> > > > > > > > > In the category of no longer in use, we have the compatibility
> > > > > > > > > generator for the kde_projects.xml file. This was introduced 
> > > > > > > > > when we
> > > > > > > > > shutdown Redmine/Chiliproject and migrated to Phabricator, as 
> > > > > > > > > a way of
> > > > > > > > > keeping services that needed to discover a list of KDE 
> > > > > > > > > Projects
> > > > > > > > > functional.
> > > > > > > > > As we've since migrated to using YAML files within the
> > > > > > > > > sysadmin/repo-metadata repository for both the CI System and
> > > > > > > > > kdesrc-build (and with LXR using kdesrc-build to do it's code
> > > > > > > > > checkouts) there shouldn't to my knowledge be anything still 
> > > > > > > > > relying
> > > > > > > > > on this (aside from perhaps scripty).
> > > > > > > > > I'd therefore like to shut this generator down as well, along 
> > > > > > > > > with the
> > > > > > > > > compaibility redirector running at projects.kde.org (given 
> > > > > > > > > that it has
> > > > > > > > > been some time since we were using that site, and many 
> > > > > > > > > projects have
> > > > > > > > > moved around in the virtual structure since then, making the 
> > > > > > > > > redirects
> > > > > > > > > it is able to offer useless)
> > > > > > > > 

> > > > > > > > Hi Ben,
> > > > > > > > I am developing a new version of the "opensrc" plugin for 
> > > > > > > > Lokalize [1]
> > > > > > > > and it currently depends on kde_projects.xml. Of course I can 
> > > > > > > > add new
> > > > > > > > code to scan the Git repo instead of just fetching 
> > > > > > > > kde_projects.xml,
> > > > > > > > however it would be more complicated.
> > > > > > > 

> > > > > > > https://projects.kde.org/api/doc/ specifically deals with this 
> > > > > > > problem
> > > > > > > by abstracting the repo away behind a micro service.
> > > > > > 

> > > > > > This looks like another view of the data available in
> > > > > > kde_projects.xml, however the API is very limited. For example I 
> > > > > > can't
> > > > > > query repo descriptions using this API. Thus not helpful.
> > > > > > However if we were going to kill kde_projects.xml, are you sure
> > > > > > projects.kde.org/api/ would be still available and not shut down as
> > > > > > well?
> > > > > 

> > > > > The API that Harald mentions is based off the YAML files, and is not
> > > > > reliant on the legacy kde_projects.xml file.
> > > > 

> > > > I can implement kde_projects.xml or a modernized version of it as part
> > > > of projects.kde.org/api/. Does this sound like a good idea?
> > > > We don't need to support the exact same XML format, so I would prefer
> > > > a JSON listing all projects with all their properties, at something
> > > > like GET https://projects.kde.org/api/v1/export or may be return all
> > > > project properties in GET https://projects.kde.org/api/v1/projects
> > > 

> > > I'll leave it to Harald to comment on whether he'd be happy having
> > > additional capabilities in the Projects Microservice API he maintains.
> > > (Harald, I assume it's only requirement is a copy of the
> > > sysadmin/repo-metadata repository locally, and it doesn't require the
> > > actual Git repositories?)
> > > We really should avoid continuing to keep legacy endpoints alive
> > > though (as it is just shifting the maintenance effort of porting away
> > > from it further down the road), especially for something that is going
> > > to end up on end user systems.
> > 

> > Wait, are you saying we shouldn't be using that endpoint to solve our 
> > dependency on kde_projects.xml on scripty either?
> > I just asked Adrián to have a look at it, since it seemed easier than 
> > having to worry about downloading and keeping an up to date copy of another 
> > repo.
> 

> I was referring to the "implement kde_projects.xml" part of his email there.
> 

> The modern components of the projects.kde.org/api/ endpoint are of course 
> fine.
> 

> > Cheers,
> > Albert
> 

> 

Re: Sysadmin Load Reduction: Code Related Services

2019-11-16 Thread Ben Cooksley
On Sat, Nov 16, 2019 at 10:39 PM Albert Astals Cid  wrote:
>
> El dissabte, 16 de novembre de 2019, a les 10:14:15 CET, Ben Cooksley va 
> escriure:
> > On Sat, Nov 16, 2019 at 3:58 PM Alexander Potashev  
> > wrote:
> > >
> > > пт, 15 нояб. 2019 г. в 22:05, Ben Cooksley :
> > > >
> > > > On Sat, Nov 16, 2019 at 3:27 AM Alexander Potashev 
> > > >  wrote:
> > > > >
> > > > > пт, 15 нояб. 2019 г. в 15:46, Harald Sitter :
> > > > > >
> > > > > > On Sat, Nov 9, 2019 at 11:37 PM Alexander Potashev 
> > > > > >  wrote:
> > > > > > >
> > > > > > > сб, 9 нояб. 2019 г. в 02:51, Ben Cooksley :
> > > > > > > > In the category of no longer in use, we have the compatibility
> > > > > > > > generator for the kde_projects.xml file. This was introduced 
> > > > > > > > when we
> > > > > > > > shutdown Redmine/Chiliproject and migrated to Phabricator, as a 
> > > > > > > > way of
> > > > > > > > keeping services that needed to discover a list of KDE Projects
> > > > > > > > functional.
> > > > > > > >
> > > > > > > > As we've since migrated to using YAML files within the
> > > > > > > > sysadmin/repo-metadata repository for both the CI System and
> > > > > > > > kdesrc-build (and with LXR using kdesrc-build to do it's code
> > > > > > > > checkouts) there shouldn't to my knowledge be anything still 
> > > > > > > > relying
> > > > > > > > on this (aside from perhaps scripty).
> > > > > > > >
> > > > > > > > I'd therefore like to shut this generator down as well, along 
> > > > > > > > with the
> > > > > > > > compaibility redirector running at projects.kde.org (given that 
> > > > > > > > it has
> > > > > > > > been some time since we were using that site, and many projects 
> > > > > > > > have
> > > > > > > > moved around in the virtual structure since then, making the 
> > > > > > > > redirects
> > > > > > > > it is able to offer useless)
> > > > > > >
> > > > > > > Hi Ben,
> > > > > > >
> > > > > > > I am developing a new version of the "opensrc" plugin for 
> > > > > > > Lokalize [1]
> > > > > > > and it currently depends on kde_projects.xml. Of course I can add 
> > > > > > > new
> > > > > > > code to scan the Git repo instead of just fetching 
> > > > > > > kde_projects.xml,
> > > > > > > however it would be more complicated.
> > > > > >
> > > > > > https://projects.kde.org/api/doc/ specifically deals with this 
> > > > > > problem
> > > > > > by abstracting the repo away behind a micro service.
> > > > >
> > > > > This looks like another view of the data available in
> > > > > kde_projects.xml, however the API is very limited. For example I can't
> > > > > query repo descriptions using this API. Thus not helpful.
> > > > >
> > > > > However if we were going to kill kde_projects.xml, are you sure
> > > > > projects.kde.org/api/ would be still available and not shut down as
> > > > > well?
> > > >
> > > > The API that Harald mentions is based off the YAML files, and is not
> > > > reliant on the legacy kde_projects.xml file.
> > >
> > > I can implement kde_projects.xml or a modernized version of it as part
> > > of projects.kde.org/api/. Does this sound like a good idea?
> > >
> > > We don't need to support the exact same XML format, so I would prefer
> > > a JSON listing all projects with all their properties, at something
> > > like GET https://projects.kde.org/api/v1/export or may be return all
> > > project properties in GET https://projects.kde.org/api/v1/projects
> >
> > I'll leave it to Harald to comment on whether he'd be happy having
> > additional capabilities in the Projects Microservice API he maintains.
> > (Harald, I assume it's only requirement is a copy of the
> > sysadmin/repo-metadata repository locally, and it doesn't require the
> > actual Git repositories?)
> >
> > We really should avoid continuing to keep legacy endpoints alive
> > though (as it is just shifting the maintenance effort of porting away
> > from it further down the road), especially for something that is going
> > to end up on end user systems.
>
> Wait, are you saying we shouldn't be using that endpoint to solve our 
> dependency on kde_projects.xml on scripty either?
>
> I just asked Adrián to have a look at it, since it seemed easier than having 
> to worry about downloading and keeping an up to date copy of another repo.

I was referring to the "implement kde_projects.xml" part of his email there.

The modern components of the projects.kde.org/api/ endpoint are of course fine.

>
> Cheers,
>   Albert

Cheers,
Ben

>
> >
> > On that note, should you be using QNetworkAccessManager, please ensure
> > you forcibly and explicitly enable handling of redirects.
> >
> > >
> > > --
> > > Alexander Potashev
> >
> > Cheers,
> > Ben
> >
>
>
>
>


Re: Sysadmin Load Reduction: Code Related Services

2019-11-16 Thread Albert Astals Cid
El dissabte, 16 de novembre de 2019, a les 10:14:15 CET, Ben Cooksley va 
escriure:
> On Sat, Nov 16, 2019 at 3:58 PM Alexander Potashev  
> wrote:
> >
> > пт, 15 нояб. 2019 г. в 22:05, Ben Cooksley :
> > >
> > > On Sat, Nov 16, 2019 at 3:27 AM Alexander Potashev  
> > > wrote:
> > > >
> > > > пт, 15 нояб. 2019 г. в 15:46, Harald Sitter :
> > > > >
> > > > > On Sat, Nov 9, 2019 at 11:37 PM Alexander Potashev 
> > > > >  wrote:
> > > > > >
> > > > > > сб, 9 нояб. 2019 г. в 02:51, Ben Cooksley :
> > > > > > > In the category of no longer in use, we have the compatibility
> > > > > > > generator for the kde_projects.xml file. This was introduced when 
> > > > > > > we
> > > > > > > shutdown Redmine/Chiliproject and migrated to Phabricator, as a 
> > > > > > > way of
> > > > > > > keeping services that needed to discover a list of KDE Projects
> > > > > > > functional.
> > > > > > >
> > > > > > > As we've since migrated to using YAML files within the
> > > > > > > sysadmin/repo-metadata repository for both the CI System and
> > > > > > > kdesrc-build (and with LXR using kdesrc-build to do it's code
> > > > > > > checkouts) there shouldn't to my knowledge be anything still 
> > > > > > > relying
> > > > > > > on this (aside from perhaps scripty).
> > > > > > >
> > > > > > > I'd therefore like to shut this generator down as well, along 
> > > > > > > with the
> > > > > > > compaibility redirector running at projects.kde.org (given that 
> > > > > > > it has
> > > > > > > been some time since we were using that site, and many projects 
> > > > > > > have
> > > > > > > moved around in the virtual structure since then, making the 
> > > > > > > redirects
> > > > > > > it is able to offer useless)
> > > > > >
> > > > > > Hi Ben,
> > > > > >
> > > > > > I am developing a new version of the "opensrc" plugin for Lokalize 
> > > > > > [1]
> > > > > > and it currently depends on kde_projects.xml. Of course I can add 
> > > > > > new
> > > > > > code to scan the Git repo instead of just fetching kde_projects.xml,
> > > > > > however it would be more complicated.
> > > > >
> > > > > https://projects.kde.org/api/doc/ specifically deals with this problem
> > > > > by abstracting the repo away behind a micro service.
> > > >
> > > > This looks like another view of the data available in
> > > > kde_projects.xml, however the API is very limited. For example I can't
> > > > query repo descriptions using this API. Thus not helpful.
> > > >
> > > > However if we were going to kill kde_projects.xml, are you sure
> > > > projects.kde.org/api/ would be still available and not shut down as
> > > > well?
> > >
> > > The API that Harald mentions is based off the YAML files, and is not
> > > reliant on the legacy kde_projects.xml file.
> >
> > I can implement kde_projects.xml or a modernized version of it as part
> > of projects.kde.org/api/. Does this sound like a good idea?
> >
> > We don't need to support the exact same XML format, so I would prefer
> > a JSON listing all projects with all their properties, at something
> > like GET https://projects.kde.org/api/v1/export or may be return all
> > project properties in GET https://projects.kde.org/api/v1/projects
> 
> I'll leave it to Harald to comment on whether he'd be happy having
> additional capabilities in the Projects Microservice API he maintains.
> (Harald, I assume it's only requirement is a copy of the
> sysadmin/repo-metadata repository locally, and it doesn't require the
> actual Git repositories?)
> 
> We really should avoid continuing to keep legacy endpoints alive
> though (as it is just shifting the maintenance effort of porting away
> from it further down the road), especially for something that is going
> to end up on end user systems.

Wait, are you saying we shouldn't be using that endpoint to solve our 
dependency on kde_projects.xml on scripty either?

I just asked Adrián to have a look at it, since it seemed easier than having to 
worry about downloading and keeping an up to date copy of another repo.

Cheers,
  Albert

> 
> On that note, should you be using QNetworkAccessManager, please ensure
> you forcibly and explicitly enable handling of redirects.
> 
> >
> > --
> > Alexander Potashev
> 
> Cheers,
> Ben
> 






Re: Sysadmin Load Reduction: Code Related Services

2019-11-16 Thread Ben Cooksley
On Sat, Nov 16, 2019 at 3:58 PM Alexander Potashev  wrote:
>
> пт, 15 нояб. 2019 г. в 22:05, Ben Cooksley :
> >
> > On Sat, Nov 16, 2019 at 3:27 AM Alexander Potashev  
> > wrote:
> > >
> > > пт, 15 нояб. 2019 г. в 15:46, Harald Sitter :
> > > >
> > > > On Sat, Nov 9, 2019 at 11:37 PM Alexander Potashev 
> > > >  wrote:
> > > > >
> > > > > сб, 9 нояб. 2019 г. в 02:51, Ben Cooksley :
> > > > > > In the category of no longer in use, we have the compatibility
> > > > > > generator for the kde_projects.xml file. This was introduced when we
> > > > > > shutdown Redmine/Chiliproject and migrated to Phabricator, as a way 
> > > > > > of
> > > > > > keeping services that needed to discover a list of KDE Projects
> > > > > > functional.
> > > > > >
> > > > > > As we've since migrated to using YAML files within the
> > > > > > sysadmin/repo-metadata repository for both the CI System and
> > > > > > kdesrc-build (and with LXR using kdesrc-build to do it's code
> > > > > > checkouts) there shouldn't to my knowledge be anything still relying
> > > > > > on this (aside from perhaps scripty).
> > > > > >
> > > > > > I'd therefore like to shut this generator down as well, along with 
> > > > > > the
> > > > > > compaibility redirector running at projects.kde.org (given that it 
> > > > > > has
> > > > > > been some time since we were using that site, and many projects have
> > > > > > moved around in the virtual structure since then, making the 
> > > > > > redirects
> > > > > > it is able to offer useless)
> > > > >
> > > > > Hi Ben,
> > > > >
> > > > > I am developing a new version of the "opensrc" plugin for Lokalize [1]
> > > > > and it currently depends on kde_projects.xml. Of course I can add new
> > > > > code to scan the Git repo instead of just fetching kde_projects.xml,
> > > > > however it would be more complicated.
> > > >
> > > > https://projects.kde.org/api/doc/ specifically deals with this problem
> > > > by abstracting the repo away behind a micro service.
> > >
> > > This looks like another view of the data available in
> > > kde_projects.xml, however the API is very limited. For example I can't
> > > query repo descriptions using this API. Thus not helpful.
> > >
> > > However if we were going to kill kde_projects.xml, are you sure
> > > projects.kde.org/api/ would be still available and not shut down as
> > > well?
> >
> > The API that Harald mentions is based off the YAML files, and is not
> > reliant on the legacy kde_projects.xml file.
>
> I can implement kde_projects.xml or a modernized version of it as part
> of projects.kde.org/api/. Does this sound like a good idea?
>
> We don't need to support the exact same XML format, so I would prefer
> a JSON listing all projects with all their properties, at something
> like GET https://projects.kde.org/api/v1/export or may be return all
> project properties in GET https://projects.kde.org/api/v1/projects

I'll leave it to Harald to comment on whether he'd be happy having
additional capabilities in the Projects Microservice API he maintains.
(Harald, I assume it's only requirement is a copy of the
sysadmin/repo-metadata repository locally, and it doesn't require the
actual Git repositories?)

We really should avoid continuing to keep legacy endpoints alive
though (as it is just shifting the maintenance effort of porting away
from it further down the road), especially for something that is going
to end up on end user systems.

On that note, should you be using QNetworkAccessManager, please ensure
you forcibly and explicitly enable handling of redirects.

>
> --
> Alexander Potashev

Cheers,
Ben