Re: github, phabricator: a new threadZ

[Ben Cooksley]

On Thu, Aug 10, 2017 at 11:57 PM, Harald Sitter  wrote:
> On Thu, Aug 10, 2017 at 11:06 AM, Adriaan de Groot  wrote:
>> Perhaps we should ask sitter what kinds of scratch repo's those were, and
>> whether he would have created them on KDE infra, if there was a similarly
>> simple one-click way to create scratch repo's.
>
> - experimental snap build tooling for neon's jenkins
> - experimental os-autoinst (openqa) tooling
> - experimental jenkins plugins (later migrated to git.kde for rollout)
> - experimental api.projects.kde.org (later migrated to git.kde for rollout)
> - experimental appstream health to junit converter
> - experimental filelight-ming32 build using only cmake
> - experimental plasmoid to track packagekit activity (later migtated to 
> git.kde)
> - experimental dbus traffic recording and playback system for mocking
> services on a dbus level
> - various rbot plugins (later migrated to git.kde)
> - assistive service for logind session cleanup for racnoss (later
> migrated to git.kde)
> - sftp-to-http bridging service for neon's jenkins to get access to
> pre-release tarballs
> - experimental snap bundle running an entire plasma desktop
> - experimental apache config example for transparent fallback from
> drupal to a static set of pages on 404
> - migration script for kanboard to phabricator (which was used to
> migrate KDE todos)
> - some other experimental helpers mostly to assist with one of the
> aforementioned things
>
> Everything denoted experimental was created not knowing if this is
> production viable or even worth seeing through to production quality.
> Barring a minor problem with git.kde not being able to support https
> clones with `go get` I'd probably have created them all as
> "throw-away" repos on KDE infrastructure since there is no immediate
> benefit to these things being on github.

Please note that the anongit system does support https and can be used for this.

Due to the needs of those behind restrictive networks, such as Italian
research labs and Indian universities we have to offer SSH over port
443, so git.kde.org cannot provide HTTPS services.

When we transition to Phabricator hosting we will likely re-arrange
the setup of addresses and offer something like scm-ssh.kde.org for
those who are operating behind these crippled networks. Once that is
done, we should be able to provide https://git.kde.org/ access
(although it will likely just redirect Git clients to anongit and web
browsers to Phabricator)

As an aside, i'll note that these organisations must not be members of
Eduroam, as part of the agreement for being an Eduroam member includes
port 22 (SSH) being permitted for outbound traffic. If your research
lab or university is a member of Eduroam but is not allowing SSH
traffic please file a complaint regarding that.

>
> For good measure in the same time period I've forked or worked on
> forks of the following on github:
>
> - aptly (repo server used by neon)
> - appstream & appstream-generator
> - various jenkins plugins
> - the bot service which closes PRs against KDE's github mirror repos
> - repology (distro-package-version-tracking website)
> - various ruby gems
> - snapd & snapcraft
> - packagekit
>
> On that note: anecdotal evidence suggests that a lot of jenkins
> plugins suffer from the "dead repo" problem often mentioned. 3/3
> plugins I submitted PRs against have not gotten a review or only after
> months sitting there. They are all under the shared community
> jenkinsci organization though.
>
> And on a further note since I write a lot of ruby: I'd have to think
> twice or trice about putting "vast/complex/more than a bunch of lines"
> ruby software on KDE infrastructure, even if the software was very
> related to a KDE project. I'd miss out on easy access to the vast ruby
> ecosystem centered around github. That's not necessarily an
> infrastructure fault on our side, but I thought I'd mention it. Food
> for thought: maybe KDE infra should feel similarly like a want-have
> when writing qt software.
>
> HS

Cheers,
Ben

Re: github, phabricator: a new threadZ

[Harald Sitter]

On Thu, Aug 10, 2017 at 11:06 AM, Adriaan de Groot  wrote:
> Perhaps we should ask sitter what kinds of scratch repo's those were, and
> whether he would have created them on KDE infra, if there was a similarly
> simple one-click way to create scratch repo's.

- experimental snap build tooling for neon's jenkins
- experimental os-autoinst (openqa) tooling
- experimental jenkins plugins (later migrated to git.kde for rollout)
- experimental api.projects.kde.org (later migrated to git.kde for rollout)
- experimental appstream health to junit converter
- experimental filelight-ming32 build using only cmake
- experimental plasmoid to track packagekit activity (later migtated to git.kde)
- experimental dbus traffic recording and playback system for mocking
services on a dbus level
- various rbot plugins (later migrated to git.kde)
- assistive service for logind session cleanup for racnoss (later
migrated to git.kde)
- sftp-to-http bridging service for neon's jenkins to get access to
pre-release tarballs
- experimental snap bundle running an entire plasma desktop
- experimental apache config example for transparent fallback from
drupal to a static set of pages on 404
- migration script for kanboard to phabricator (which was used to
migrate KDE todos)
- some other experimental helpers mostly to assist with one of the
aforementioned things

Everything denoted experimental was created not knowing if this is
production viable or even worth seeing through to production quality.
Barring a minor problem with git.kde not being able to support https
clones with `go get` I'd probably have created them all as
"throw-away" repos on KDE infrastructure since there is no immediate
benefit to these things being on github.

For good measure in the same time period I've forked or worked on
forks of the following on github:

- aptly (repo server used by neon)
- appstream & appstream-generator
- various jenkins plugins
- the bot service which closes PRs against KDE's github mirror repos
- repology (distro-package-version-tracking website)
- various ruby gems
- snapd & snapcraft
- packagekit

On that note: anecdotal evidence suggests that a lot of jenkins
plugins suffer from the "dead repo" problem often mentioned. 3/3
plugins I submitted PRs against have not gotten a review or only after
months sitting there. They are all under the shared community
jenkinsci organization though.

And on a further note since I write a lot of ruby: I'd have to think
twice or trice about putting "vast/complex/more than a bunch of lines"
ruby software on KDE infrastructure, even if the software was very
related to a KDE project. I'd miss out on easy access to the vast ruby
ecosystem centered around github. That's not necessarily an
infrastructure fault on our side, but I thought I'd mention it. Food
for thought: maybe KDE infra should feel similarly like a want-have
when writing qt software.

HS

Re: github, phabricator: a new threadZ

[Adriaan de Groot]

On Thursday 10 August 2017 10:45:14 Harald Sitter wrote:
> Seems to me y'all aren't appreciating that I am telling you my point
> of view. I have created some 20 repos on github last year and 0
> scratch repos on our infrastructure. I thought you might want to know
> why. Feel free to find my reasons silly, that doesn't change them
> though.

So, since this is the thread for requirements-collecting on github, 
phabricator, KDE git infrastructure, we have some data points:

 - KDE scratch repo's are not widely known within the KDE community.
 - KDE scratch repo's take more effort to create, and for scratchj / throwaway 
repo's, every bit of extra effort is one.

I suppose if I had a browser window open, and logged in to KDE phab and github 
at the same time, and the one tab gives me instant gratification (and a scratch 
repo) while the other gives me gratification four clicks and a few minutes 
away, then the one it is.

Buut .. that presupposes that people create scratch repo's wherever is 
convenient, not wherever they make sense. So the initial question that kicked 
off this thread, (I can't spot who asked it though)

>>> >> should also think about. Why many KDE developers choose github instead
>>> >> of scratch KDE repositories to start new software, where it could
>>> >> happily be hosted within KDE infrastructure?

presupposes that those scratch repo's could be created and hosted within KDE 
infrastructure (I personally would only create scratch repo's on KDE 
infrastructure for things that have something to do with KDE and can fall 
within the KDE manifesto).

Perhaps we should ask sitter what kinds of scratch repo's those were, and 
whether he would have created them on KDE infra, if there was a similarly 
simple one-click way to create scratch repo's.

[ade]

Re: github, phabricator: a new threadZ

[Luigi Toscano]

Il 10 agosto 2017 11:45:14 EEST, Harald Sitter  ha scritto:
>Seems to me y'all aren't appreciating that I am telling you my point
>of view. 


One thing is say "adding an easier workflow for scratch repo to our infra is 
much required for this and that", which I can agree too; another is going 
straight to something that seems more like to "ready to kill our infra anytime".

> I have created some 20 repos on github last year and 0
>scratch repos on our infrastructure. I thought you might want to know
>why. Feel free to find my reasons silly, that doesn't change them
>though.

I don't find your reasoning silly, as I wrote above. Hyperbole(s?) are fine but 
maybe distracting in an already complicated discussion.


-- 
Luigi

Re: github, phabricator: a new threadZ

[Harald Sitter]

Seems to me y'all aren't appreciating that I am telling you my point
of view. I have created some 20 repos on github last year and 0
scratch repos on our infrastructure. I thought you might want to know
why. Feel free to find my reasons silly, that doesn't change them
though.

I am did not mean to argue any side.

On Thu, Aug 10, 2017 at 9:02 AM, Harald Sitter  wrote:
> I for one would pick the alternative which is not using our
> infrastructure and instead click two buttons so I don't have to file
> paperwork manually nor have to wait for said paperwork to get faxed to
> HQ for approval. i.e. not waiting trumps waiting in my world.
>
> On Wed, Aug 9, 2017 at 8:05 PM, Albert Astals Cid  wrote:
>> El dimecres, 9 d’agost de 2017, a les 13:50:46 CEST, Harald Sitter va
>> escriure:
>>> On Wed, Aug 9, 2017 at 1:39 PM, Boudewijn Rempt  wrote:
>>> > On Wed, 9 Aug 2017, Riccardo Iaconelli wrote:
>>> >> This is a new thread entirely, but incidentally also something we
>>> >> should also think about. Why many KDE developers choose github instead
>>> >> of scratch KDE repositories to start new software, where it could
>>> >> happily be hosted within KDE infrastructure?
>>> >
>>> > Our infra doesn't offer scratch repos anymore, does it?
>>>
>>> It does, they were slated for removal with the transition to phab,
>>> since that hasn't happend yet though I assume scratch repos are still
>>> a thing. Albeit, a thing that is meant to be removed with (currently)
>>> no replacement planned
>>
>> That was not the plan, the plan (unless it has changed since last time i
>> asked) is to have a team of "trusted people" that can create repos on
>> phabricator, so basically you'd open a ticket and you'd get a repo created in
>> a matter of minutes, not automagic, but surely you can continue coding for 15
>> minutes while you wait for your repository to be created.
>>
>> Cheers,
>>   Albert
>>
>>> which is why for example I do not create any
>>> new ones and instead use github. Although TBH the UX of creating a
>>> scratch has also left me wanting as it entails me googling how to
>>> setup a scratch every single time.
>>>
>>> HS
>>
>>

Re: github, phabricator: a new threadZ

[Luigi Toscano]

Il 10 agosto 2017 10:02:43 EEST, Harald Sitter  ha scritto:
>I for one would pick the alternative which is not using our
>infrastructure and instead click two buttons so I don't have to file
>paperwork manually nor have to wait for said paperwork to get faxed to
>HQ for approval. i.e. not waiting trumps waiting in my world.
>

Nothing technically prevents this from implementing this in oir infrastructure, 
nor what Albert wrote hints at this: he just reported the current plan. Plan 
can be changed.

I don't understand the need of "let's trow everything out of.the window" 
instead of "let's extend what we have" (same goes for the proper IM discussion)

Let me requote:

>On Wed, Aug 9, 2017 at 8:05 PM, Albert Astals Cid 
>wrote:
>> That was not the plan, the plan (unless it has changed since last
>time i
>> asked) is to have a team of "trusted people" that can create repos on
>> phabricator, so basically you'd open a ticket and you'd get a repo
>created in
>> a matter of minutes, not automagic, but surely you can continue
>coding for 15
>> minutes while you wait for your repository to be created.
>>
>> Cheers,
>>   Albert


-- 
Luigi

Re: github, phabricator: a new threadZ

[Ben Cooksley]

On Thu, Aug 10, 2017 at 7:02 PM, Harald Sitter  wrote:
> I for one would pick the alternative which is not using our
> infrastructure and instead click two buttons so I don't have to file
> paperwork manually nor have to wait for said paperwork to get faxed to
> HQ for approval. i.e. not waiting trumps waiting in my world.

The reason why it's restricted right now is because creating a
repository with Gitolite requires the ability to push to
gitolite-admin, which also means you can change SSH Keys and thus
become them as far as the system is concerned.

I dislike your use of terminology there regarding paperwork being
faxed to HQ - Sysadmin hasn't ever declined a developers request for a
repository. We have on occasion suggested different names, to keep to
conventions (for Plasmoids and packaging repositories for instance) or
where a developer has chosen a heavily generic and non-descriptive
names.

Repository names that don't follow conventions or which are
non-descriptive are harmful as they interfere with the discoverability
of code.

Oh, and you're neglecting to mention that scratch repositories very
much still exist, and require basically no effort to use. Not even a
Sysadmin ticket.

>
> On Wed, Aug 9, 2017 at 8:05 PM, Albert Astals Cid  wrote:
>> El dimecres, 9 d’agost de 2017, a les 13:50:46 CEST, Harald Sitter va
>> escriure:
>>> On Wed, Aug 9, 2017 at 1:39 PM, Boudewijn Rempt  wrote:
>>> > On Wed, 9 Aug 2017, Riccardo Iaconelli wrote:
>>> >> This is a new thread entirely, but incidentally also something we
>>> >> should also think about. Why many KDE developers choose github instead
>>> >> of scratch KDE repositories to start new software, where it could
>>> >> happily be hosted within KDE infrastructure?
>>> >
>>> > Our infra doesn't offer scratch repos anymore, does it?
>>>
>>> It does, they were slated for removal with the transition to phab,
>>> since that hasn't happend yet though I assume scratch repos are still
>>> a thing. Albeit, a thing that is meant to be removed with (currently)
>>> no replacement planned
>>
>> That was not the plan, the plan (unless it has changed since last time i
>> asked) is to have a team of "trusted people" that can create repos on

That group has for the most part been established already, and is
known as #Community_Admins.
It's full membership can be seen at
https://phabricator.kde.org/project/members/27/

Once we move to using Phabricator for hosting repositories they'll
have the power to create a repository in about 4 clicks.

They currently have the power to fully edit tasks (including
visibility and edit rights), create and edit all projects, setup short
urls (go.kde.org), manage global Herald rules and setup top level Wiki
namespaces on Phabricator among other things.

>> phabricator, so basically you'd open a ticket and you'd get a repo created in
>> a matter of minutes, not automagic, but surely you can continue coding for 15
>> minutes while you wait for your repository to be created.

In terms of a scratch repositories, we just haven't figured out how
those should be handled in Phabricator yet. Given the fact people have
issue with how they work now, one of the possible solutions floated at
the time was that we would retire them.

It is possible a more limited version of the create repository flow
will be offered to developers who aren't community admins which will
set repositories up (but not allow editing them), but this would
require custom code (likely as a Phabricator extension).

>>
>> Cheers,
>>   Albert
>>
>>> which is why for example I do not create any
>>> new ones and instead use github. Although TBH the UX of creating a
>>> scratch has also left me wanting as it entails me googling how to
>>> setup a scratch every single time.
>>>
>>> HS
>>
>>

Regards,
Ben

Re: github, phabricator: a new threadZ

[Eike Hein]

On August 9, 2017 9:10:15 PM GMT+09:00, Jos van den Oever 
 wrote:
>Op woensdag 9 augustus 2017 13:50:46 CEST schreef Harald Sitter:
>> On Wed, Aug 9, 2017 at 1:39 PM, Boudewijn Rempt 
>wrote:
>> > On Wed, 9 Aug 2017, Riccardo Iaconelli wrote:
>> >> This is a new thread entirely, but incidentally also something we
>> >> should also think about. Why many KDE developers choose github
>instead
>> >> of scratch KDE repositories to start new software, where it could
>> >> happily be hosted within KDE infrastructure?
>> > 
>> > Our infra doesn't offer scratch repos anymore, does it?
>> 
>> It does, they were slated for removal with the transition to phab,
>> since that hasn't happend yet though I assume scratch repos are still
>> a thing. Albeit, a thing that is meant to be removed with (currently)
>> no replacement planned which is why for example I do not create any
>> new ones and instead use github. Although TBH the UX of creating a
>> scratch has also left me wanting as it entails me googling how to
>> setup a scratch every single time.
>
>Scatch repos are awesome! No need to open a browser, just push to a
>url. I was 
>very happy when I discovered KDE infra had such a lovely feature.
>
>Cheers,
>Jos

Yeah, I use them all the time, too.

Cheers,
Eike
-- 
Plasma, apps developer
KDE e.V. vice president, treasurer
Seoul, South Korea

Re: github, phabricator: a new threadZ

[Jos van den Oever]

Op woensdag 9 augustus 2017 13:50:46 CEST schreef Harald Sitter:
> On Wed, Aug 9, 2017 at 1:39 PM, Boudewijn Rempt  wrote:
> > On Wed, 9 Aug 2017, Riccardo Iaconelli wrote:
> >> This is a new thread entirely, but incidentally also something we
> >> should also think about. Why many KDE developers choose github instead
> >> of scratch KDE repositories to start new software, where it could
> >> happily be hosted within KDE infrastructure?
> > 
> > Our infra doesn't offer scratch repos anymore, does it?
> 
> It does, they were slated for removal with the transition to phab,
> since that hasn't happend yet though I assume scratch repos are still
> a thing. Albeit, a thing that is meant to be removed with (currently)
> no replacement planned which is why for example I do not create any
> new ones and instead use github. Although TBH the UX of creating a
> scratch has also left me wanting as it entails me googling how to
> setup a scratch every single time.

Scatch repos are awesome! No need to open a browser, just push to a url. I was 
very happy when I discovered KDE infra had such a lovely feature.

Cheers,
Jos


signature.asc
Description: This is a digitally signed message part.

Re: github, phabricator: a new threadZ

[Harald Sitter]

On Wed, Aug 9, 2017 at 1:39 PM, Boudewijn Rempt  wrote:
> On Wed, 9 Aug 2017, Riccardo Iaconelli wrote:
>
>> This is a new thread entirely, but incidentally also something we
>> should also think about. Why many KDE developers choose github instead
>> of scratch KDE repositories to start new software, where it could
>> happily be hosted within KDE infrastructure?
>
> Our infra doesn't offer scratch repos anymore, does it?

It does, they were slated for removal with the transition to phab,
since that hasn't happend yet though I assume scratch repos are still
a thing. Albeit, a thing that is meant to be removed with (currently)
no replacement planned which is why for example I do not create any
new ones and instead use github. Although TBH the UX of creating a
scratch has also left me wanting as it entails me googling how to
setup a scratch every single time.

HS