Re: [Wikimedia-l] Fwd: Re: [Wikitech-l] CI and Code Review

[Grant Ingersoll]

Apologies for the cross-post, but doing so because the thread was
forwarded, also apologies for the length.

On Wed, Jul 8, 2020 at 5:01 PM Maarten Dammers  wrote:

> Of interest to the wider community. I really hope this is not part of a
> larger pattern of the WMF ignoring community.
>
>
"Never attribute to malice that which is adequately explained by
stupidity." [1]


In this case, my own stupidity (I'm the new CTO here at the WMF, for
context), or perhaps to be a little kinder to myself, a combination of bias
and naivety: my engineering bias towards wanting to solve a problem I felt
was important to take action on (context provided shortly) which got in the
way of taking a user-centric approach first in trying to understand what
the needs and wants are of the people using the system.  As I said on the
talk page, I mistakenly thought that the main feedback loops would be about
porting workflows and not about the tool itself.


Even though many have publicly said that moving from Gerrit might still be
the right decision, how we go about deciding that is just as important as
what we do and I messed that up. Given that perspective, I've asked the
team to pause with moving forward on changes to our Code Review (CR) tools
and to begin a consultation that includes the option of sticking with what
we have for CR. I've also asked my team to update some of our decision
making processes relative to topics like this to make sure we properly hear
from stakeholders (e.g. in this case, both staff developers and our broader
community of developers) along the way.


For some more context, if it is helpful:


I'm ~11 months in here and still learning every day.  While I've worked in
open source for a long time, this community is new to me and different
enough that I have and continue to need to update and adjust the way I
think and the way I direct my teams to do their work.


Coming in and talking with our tech teams and folks in the community, I see
a few themes that have emerged that contributed to me wanting to move
forward faster on this decision:


1. We have a lot of tech debt[2].  In many cases, I think software,
especially software that is successful, can collapse under its own weight
if people are not careful in servicing that tech debt.  The work required
to both maintain existing infrastructure, products and services while at
the same time improving what we offer is a delicate balancing act. At our
scale, there is a significant and justified bias towards production, but it
has come at a cost that has compounded over the years and has a very real
human toll. Much of this debt was created because we had to invent things
that didn't exist.  Now some of those things do exist and we should check
to see whether we can replace those older, albeit well-understood-by-us
systems, with newer ones that have become standards or best in class and
are still in line with our open source values.


2. The tech debt and the sheer number of services we support (many of which
aren't fully maintained[3]) is compounded by the scale at which we support
them. The result is that a number of people, especially those on the front
line of caring for that software, are either burnt out, or approaching that
point. A global pandemic hasn't helped. I view much of my role here early
on as one of trying to help somehow reduce that burnout. Modernizing and
upgrading our processes and toolchains can, I think, help fight this, even
if there is some short term pain in the shift.


All that being said, in this particular case, we have a team of people who
work on maintaining our CI (Continuous Integration) and CR systems who have
long been looking at replacing our CI system. This system runs on an
end-of-lifed version of Python and on an end-of-lifed version of Zuul, and
it’s critical we correct this since end-of-lifed software doesn’t receive
security updates. This is primarily behind the scenes work that most people
don't have to think about. There is also a growing sense of desire by some
of our developers to adopt more mainstream, well understood toolchains like
Gitlab/Github for development, combined with my own view that CI/CR is
*not* somewhere we should be deviating from broad industry norms on
ourselves and that we should adopt workflows that are (de facto) standards
(e.g. Gitlab/Github, with Gitlab being the open one of the two) amongst
developers irrespective of their backgrounds. Those two things led to my
biased thinking that it was obvious it needed to be changed and that the
primary feedback needed would therefore be on the workflows, not the tool
itself.


While I still think it needs to be changed, I completely missed, as I said
above, the stakeholder angle  here and basic community laws of not
surprising people.  For that I apologize.  We are now working to correct
this, even if it means it's going to take longer or we end up sticking with
the status quo on CR.


Thanks,

Grant



[1] 

Re: [Wikimedia-l] Fwd: Re: [Wikitech-l] CI and Code Review

[Aron Manning]

*Cross-posting to wikitech-l as it's a topic related to development.*

On Wed, 8 Jul 2020 at 23:01, Maarten Dammers  wrote:

> Of interest to the wider community. I really hope this is not part of a
> larger pattern of the WMF ignoring community.
>
> Maarten


I've had a great experience in the "Discussion Tools" (WMF's talk pages
reply tool) project. Feedback is properly documented, considered and worked
on. I recall having quite some input on the UI design and I'm happy that
resulted in a clean, focused UI.

That is an example to follow, but there's a lot to improve in other areas.
Although I favor GitLab, I was surprised that this introduction didn't have
a community round other than the developer feedback survey. I assume
the non-public results of the surcey justify this move and I'm happy
with it, so no complaints, but it came as a surprise to me that this is
happening as such discussions were promptly shut down on phabricator: I've
given this suggestion once in a related topic and rude comments told me
this is off-topic and basically to keep it to myself. The discussion ended
abruptly and the ticket was closed within a day.

There was another discussion  closed
and declined after 4 comments from 3 participants in 12 minutes. This
discussion was later referred to as the official decision to avoid GitLab
:-)
And prior to the current announcement, an attempt was made to rename this
discussion to "RFC: ..."
I have a hard time to understand these actions and correlate it to any
consensus process and transparent communication.


The recent update to the history diff font
 also could have been
communicated better. Informing editors about a simple solution (to change
your editor font setting or add some CSS to common.css/global.css) would
have avoided disruption and community backlash. Instead the latter solution
was only shared reactively after complaints and the simple solution was not
suggested.

I'd hope the developer team gives more attention to communication and
transparency. I think this has improved in the last year, primarily in the
preparation of Desktop Improvements and Talk pages project, but it's still
a long path to improve engagement generally with the community and in
particular development matters.


Aron (Demian)
___
Wikimedia-l mailing list, guidelines at: 
https://meta.wikimedia.org/wiki/Mailing_lists/Guidelines and 
https://meta.wikimedia.org/wiki/Wikimedia-l
New messages to: Wikimedia-l@lists.wikimedia.org
Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/wikimedia-l, 

[Wikimedia-l] Fwd: Re: [Wikitech-l] CI and Code Review

[Maarten Dammers]

Of interest to the wider community. I really hope this is not part of a 
larger pattern of the WMF ignoring community.


Maarten

 Forwarded Message 
Subject:Re: [Wikitech-l] CI and Code Review
Date:   Wed, 8 Jul 2020 22:40:38 +0200
From:   Maarten Dammers 
Reply-To: 	For developers discussing technical aspects and organization 
of Wikimedia projects 

To: wikitec...@lists.wikimedia.org



Hi Greg,

On 06-07-2020 19:39, Greg Grossmeier wrote:

First, apologies for not announcing this last week. A short work week
coupled with a new fiscal year delayed this until today.

tl;dr: Wikimedia will be moving to a self-hosted (in our datacenter(s))
GitLab Community Edition (CE) installation for both code review and
continuous integration (CI).


tl;dr: WMF decides to do a major change without any community 
consultation. Community members are upset.

More at https://www.mediawiki.org/wiki/Topic:Vpbt50rwxgb2r6qn

Maarten


___
Wikitech-l mailing list
wikitec...@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
___
Wikimedia-l mailing list, guidelines at: 
https://meta.wikimedia.org/wiki/Mailing_lists/Guidelines and 
https://meta.wikimedia.org/wiki/Wikimedia-l
New messages to: Wikimedia-l@lists.wikimedia.org
Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/wikimedia-l,