When I see what the current fires are, I see them switching from C++
support to ranges to DLLs to allocators to GC and back and forth. I'm
never sure why the thing that you and Walter say is the most important
really is the most important, and I see the community questioning this
as well. If anything, it is quite clear that there are as many agendas
as contributors.
What I don't see anywhere is something as simple as a product backlog.
This doesn't even need fancy software to maintain. It could be a wiki
page. But the point of it is to make clear how all the agendas interact,
intersect, and conflict. Obviously, this can become a highly political
issue, which is likely why it hasn't been attempted.
It should also expose how the priorities shift over time, as well as the
extent to which the community has influence over what is considered
important. If there's one bit of process that is almost universally
valuable, it is transparency. I see D development as more like black
art. Shining a light on everyone's priorities would lend credibility to
the process and likely make people more willing to compromise on them.
What I sense is instead a community that sometimes resents having
priorities dictated to them.
D has real users, and by that I mean people who are betting their money
and their jobs on it. The hobbyists are real users too, but don't have
quite so much skin in the game. These people deserve more than black
magic when it comes to setting priorities. If I'm wrong and a backlog or
equivalent does exist and is easily discoverable, and contains rationale
for the ordering, then just point me to it so I can understand the
process better.
I'm not married to Jira. If folks prefer Trello, I'm fine with that. But
the odd thing about the D community is the excitement over the
possibilities mixed with the frustration over dysfunction. I don't see
why trying to bring some clarity to the process is a bad thing.
Now, let's talk about why I gave up on contributing to D. To be
perfectly honest, it's because D development lacks a certain
professional maturity. I wanted to add unit tests to the compiler (dmd).
You and Walter said that was a waste of time. The important tests are
higher level (in some unspecified way). I find it highly ironic that the
D language added inline unit tests because they are so important all
code needs them, but the D compiler itself is somehow immune from bugs.
Frankly, this is bullshit. I can hardly believe a professional software
engineer could make such a claim, let alone one who has written several
acclaimed books.
Ok, so I wrote some tests anyway. Everyone complained. Literally every
comment on the tests were a complaint. No "hey unit tests in dmd...it's
about time!" Just complaints about the testing library. Complaints about
the layout. Complaints about the formatting. For all the singing of
praises of the unittest keyword, the dmd maintainers were almost
universally hostile to actual unit tests in the compiler. This still
blows my mind. This is so far from professional software development
that it's not even funny.
I wanted to add names to unit test blocks. "Why?" Why? Why?!? Have you
ever RUN unit test before? How about in an IDE? In a build farm? In a
continuous integration pipeline? The idea that unit tests should have
names is so blindingly obvious to anyone who has used Junit for 5
minutes that the best answer I can give is literally "watch my tests run
in junit for a minute." I could give excruciating detail on why unit
test names are essential, but really, watching unit tests run in a
professional environment is far more educational. Having to justify
something this basic exceeded my patience, and my hope.
I assumed that the dmd community was a bunch of hobbyists who were
professional coders by day and who would mostly see eye to eye on a lot
of these basic issues. I still don't know why I was so wrong about this,
but I've talked to enough people that I believe that most of them are,
in fact, competent professional software engineers. That's why I like
the conferences.
Unfortunately, I think much of the friction in the community is due to
mixing professionals with hobbyists and academics. Each has a very
different world view. This often drives the divergent priorities, or so
I speculate. But it also leads to fundamental disagreements and
misunderstandings which lead to folks talking past and over each other.
Because software engineers have far more confidence than they need to
get the job done, nobody wants to admit that anyone else is right. It's
pathological Microsoft politics played out tragically in an OSS
community. What the community needs is less talking down from on high
and more listening from the bottom up. I'm actually a little surprised
that the community hasn't completely imploded already. I think it is a
testament to how much folks believe in the product, even if they have
doubts about how it is produced.
People not following orders is called "disagreeing with the dictats in a
passive-aggressive manner." It's a strong signal that the leadership
does not have buy-in from the led. It is, unfortunately, a vote of no
confidence.
I was going to offer to maintain Jira in parallel with BugZilla, but I
see that the opposition is to the idea of a backlog with history in the
first place. This is a bad sign. It means that transparency itself is
what is contentious. I can't fix that. So sad.
Dave
-------- Original message --------
From: Andrei Alexandrescu <[email protected]>
Date: 05/30/2015 10:16 PM (GMT-08:00)
To: David Held <[email protected]>, Discuss the internals of DMD
<[email protected]>
Subject: Re: [dmd-internals] Jira?
I'll be curious what Martin thinks of this.
My thoughts - as one who's thought of this for years and tried various
angles - are generally in disagreement with yours, though not vehemently so.
* This looks like a random change of a tool that works, for benefits
that seem small and an overhaul that seems large.
* We are trying trello for management. This is the second attempt to
introduce it, by Martin; first, by me, was a failure. Sadly this one
also doesn't seem to fare very well.
* Don't take this personally, but we've had a pattern of strong
up-and-coming folks without a track record and proposing a large change
with "trust me it'll be better" written all over it. Invariably those
have been flashes in the pan. If I remember correctly, one of such
initiatives was from yourself. Again, don't take this personally but it
is my opinion that if you or anyone wants to impact the community, the
one way is to consistently get work done on issues that matter.
* I believed D has had a management problem for a good while, until I
figured it doesn't. We are in need of talent. I understand how being at
DConf - it's a good conference and those are inspiring - instills one
with the impression that there's just a lot of folks ready to work but
are not organized properly. Monitoring the forums may give a similar
impression. The reality in the trenches is we have problems that are
very well known, often for years, small and large - often unbelievably
trivial - and there are very few people out there to work on them. Those
who do, do make progress (e.g. look at Martin's work on the GC, Unique,
RefCounted, etc., or Walter's work on weaning Phobos off GC). Walter and
I have been trying to lead both by example and by simply telling what
needs to be done, several time, with virtually no success.
Andrei
On 5/30/15 6:01 PM, David Held via dmd-internals wrote:
> Although BugZilla has served well for many years, I would like to
> propose a move to Jira. I see the following benefits:
>
> * BugZilla is a bug tracking system; Jira is bug tracking + project
> management
> * Jira has a more mature UI, due to commercial licensing paying for
> development
> * Jira is free for OSS!
>
> So why does D need project management? I would suggest that work on D
> is somewhat disorganized and haphazard. That is partly just the nature
> of OSS/volunteer work, but also partly due to a lack of organizational
> structure. Although language changes are neatly encapsulated by D
> Improvement Proposals, the vast majority of work which would benefit D
> falls far short of a proposed language change, and is not neatly and
> clearly tracked. From what I can tell, this is where someone would look
> to find out where to make a contribution:
>
> http://wiki.dlang.org/How_You_Can_Help
>
> This is better than nothing, but just barely. With a proper project
> management tool, we can capture issues which arise on the newsgroups,
> informal discussions, conferences, IRC, etc., along with associated
> communication and any artifacts produced. We can order work by
> importance and track progress. We can set goals and gain visibility
> into how and where contributions are made.
>
> None of this means that the current D community workflow needs to
> change, other than using Jira instead of BugTracker. While this does
> incur a user cost to switch, I think most people will find that Jira is
> generally easy to use and relatively painless to pick up. They should
> also find it much easier to run reports and view dashboards.
>
> If the community agrees to make the switch, I will volunteer to apply
> for the Jira license and work with the current BugZilla maintainer(s) to
> migrate the DB (Jira has a tool for BugZilla to do this). I will also
> volunteer to capture information from the newsgroups into tasks and
> other relevant project-tracking data.
>
> Although it is possible to track non-bug issues in BugZilla, this is not
> the kind of use case for which BugZilla was designed. Since Atlassian
> is making their tools available to us for free, I think we should take
> advantage of this offer. For the record, I think we should stick with
> MediaWiki rather than move to Confluence. And I do not know of anyone
> who thinks that Stash is better than GitHub.
>
> Dave
>
>
> P.S. For those less familiar with the Agile methodology, what I propose
> to do is to create a master backlog of work. The community can
> collaborate to define the ordering of the backlog, but the backlog would
> represent the priority of work, such that someone who wanted to
> contribute would know that the top item on the backlog is considered the
> most important task to work on next, and they could just scan from the
> top down until they find something they think they can work on.
>
> We can debate whether there should be a backlog per work type (website
> maintenance, compiler maintenance, documentation, etc.) or a global
> backlog, but these things only make sense to discuss when we have a tool
> that makes maintaining a backlog easy to do. The backlog can also
> capture work which was only partially done. So someone might pick up a
> task, work on it for a while, and then update the work item to explain
> how far they got, even if they couldn't close it out.
>
> The project tracking system can help produce reports on task progress,
> bug vs. non-bug work rate, etc. The idea is that by having easy access
> to this reporting information, the community can have greater visibility
> into the overall workings and accomplishments of the "D machine" and see
> the areas which are deemed important but neglected in a quantified way.
> Ideally, this would result in self-directed behavior which would
> optimize towards the high-value work rather than just the easy work.
> Even if it only does so in, say, 20% of the contributions, this would
> still be a significant improvement in return-on-contribution-invested.
>
> Furthermore, contributors could voluntarily report on status, which
> would give a sense of what people were actively working on, and help
> answer the question: "Why isn't issue X being addressed?" Folks could
> just look at the capacity dashboard and say: "Oh, I see people are
> working on Y and Z, which is much higher in the backlog." Then they
> could either lobby to have the backlog reordered, or they could admit
> that the highest value-add was being worked on, and pitch in to the
> tasks that they deem important.
>
> Of course, I have done a lot of hand-waving about "community-ordered
> backlog". In reality, I expect this to be a major point of contention
> and source of more than a few flame wars. But having a single place to
> witness the debate over priority unfold should be valuable all by
> itself. As long as justifications are provided as to why the backlog is
> ordered the way it is, I think the community can come together and agree
> that it is reasonable, even if it doesn't fit every person's exact
> preferences (nor could such an ordering be produced). And, of course,
> the backlog doesn't determine what anyone works on. It just represents
> what the community thinks is important, in rank order. Anyone is still
> free to pick something from the bottom and work on it. However, finding
> that a lot of people are working on low-ranked tasks should communicate
> something important about the community in and of itself. That's why
> these kinds of tools are important.
>
> A unified backlog should also make it easier for people to discover
> overlapping and/or duplicated work. This probably doesn't happen often,
> but may save a lot of time and effort for folks when it does. At the
> least it should make people aware of each other's work so they can
> proceed deliberately rather than in ignorance. These kinds of things
> are done informally now via the newsgroups, but are a kind of waste of
> contributor cycles, because humans do not need to be the web servers for
> this kind of mundane information. By offloading these overhead tasks to
> the proper software, the forums can be optimized for discussions which
> cannot be automated. That should make everyone happier and more
efficient.
>
> _______________________________________________
> dmd-internals mailing list
> [email protected]
> http://lists.puremagic.com/mailman/listinfo/dmd-internals
