On Wednesday, Sep 24, 2003, at 20:34 Europe/Rome, Timothy Larson wrote:
Maybe we are having a hard time finding the "right" word because we are mixing concerns.
I came to the same conclusion.
I can think of roughly four separate things the user of a module would want to know:
(1) Is the module stable?
(i.e. is it considered to generally work properly with no critical bugs?)
Stability should be divided in two different concerns:
1a) functional stability 1b) contract stability
the two are orthogonal and both, well, used as an artificial safety that is sooooooo appreciated by managers.
At the same time, estimating those is painful, if ever possible. Note that 1b) somewhat overlaps with 3) and below
(2) Will code written against it will work with future releases?
(i.e. will there be a back-compatibility effort made in new releases?)
this question overlap greatly with 3) so much that I think it's the same question: if there is a community around it, the community will decide what is better for the evolution, including deciding if it's worth to keep back compatibility or not.
I personally like an active and balanced community more than an artificial safety on back-compatibility... because if I trust a community, I trust the fact that many people will be having the same problems that I'm facing, thus, will probably want the community to behave as I would. Community pressure keeps project honest... but, unlike, marketing doesn't hinter evolution for legacy.
(3) Will there likely be any future versions?
(i.e. is there still an active development person or community?)
this is, IMO, the most significant question: is there a community around it?
(4) Are they the only guinea pigs actually using the module?
(i.e. is there an active user community?)
this is part of 3)
The concerns you outlined are not so separate, IMO. So, in fact they are:
1) is the code stable? 2) is the API stable? 3) is the community healthy?
we could come up with a multiple rating for the above:
1) code stability status
- under development (alpha)
- under final testing (beta)
- ready for production (final) 2) contract stability status
- under design (alpha)
- under final testing (beta)
- ready for production (final)these two can be signed without creating community friction, the third question is the one that worries me the most.
On another tack, this information is mostly dynamic:
While case (1) is usually static, but see the planned cocoon-2.1.1 -> 2.1.2
release for an example of this being a little dynamic. For a stronger
example, think of a crypto module after a security hole is found and
fixed in a new release. The old version transitions from stable to condemed.
The above suggests one simple, but really important thing:
the block 'health' metadata should *NOT* be included in the block, but should be looked up from a centralized 'weather reporter' part of the block librarian.
Case (2) starts out as only an expression of intent and then transitions
when a new release is made to being an attribute of the new release that
reflects back on the release in question.
Cases (3) and (4) are not direct attributes of the module, but rather are
attributes of the community that reflect back on the module.
In all these cases, any meta-info included in a release can only represent
a snapshot in time. This is useful, but it would be better if it could be
combined with live meta-info retrieved from the module's source site upon
download, to account for reality drift over time.
You raise a very good point: health information drifts with time and this information cannot be contained with the block.
<side-note>
Eventually it would be helpful for the source website to include the static
meta-info, live meta-info, and some pretty, graphical data from some community
data miners like Agora, etc. to help evaluate the liveliness of a module and
its developer and user communities. This could even be used by the developers
to help track what is used, what is causing problems, what can be retired, etc.
</side-note>
The ability to make sense out of complex datasets (such as mailboxes or CVS commits) is the holy grail of datamining. Agora is one attempt to provide useful information but without "pointing" fingers.
One thing that I would be *seriously* against is the establishment of any kind of mechanical "metric" about the health of a community.
This is asking for trouble. A *LOT* of trouble.
While alpha/beta/final status don't hurt the community because nobody really cares if one block was released as final and later a serious bug was found, having users scared away from your block because there is no community will make it even harder for that block to have a community around.
The 'winner takes all' effect. One of the first behaviors exhibited by power-law distributions (and each social community is one of them).
Power-law distributions are natural and cannot be avoided, but what we can do is to 'reduce' the risk of aggregation, lowering the coefficient of that power-law curve.
In Agora, for example, I didn't have any number shown. Nothing that can be used to say "I'm more inflential than you are". Everything is fuzzy and undefined, just like in real life. You get the "impression" but impressions are not measurable and, even more, are not 'confrontable' in a ordering way.
But using data-emergence patterns to estimate the health of a community, well, it's a nightmare. Agora works on a few mechanical severely strong assumptions:
1) one mail list -> one community 2) a reply -> social relationship between the two individuals
the rest is lots of math. Doing the same for CVS is much harder. I think it is possible to estimate the health of a particular code by looking at "patterns" of checkins, convolved with fourier analysis of the sequences in time, confronted with the reply mail patterns.
But no matter *how* complex is the algorithm that comes out with some numbers, those numbers will make someone happy and somebody unhappy... and friction is created.
So, while accademically intriguing (I will continue to spend time thinking at ways to create useful social datamining tools and algorithms that come up with reasonble patterns), they are a social minefield and I would like this community to stay away from them as much as possible as everyday tools to judge people.
- o -
The best way to judge is to make a vote.
And the vote should not, in any circumstance, make the block being voted bad if the vote doesn't pass.
So, the answer to
3) is the community healthy?
is misposed. I would like to have somethign a little less judging: something like
3) is the cocoon community officially supporting this block?
The risk is to come up with something which is not really meaningful. Because "official support" doesn't really mean anything.
But, at least, it's a distinction and a label of 'community health', without necessarely implying that the community around (yet?) unsupported blocks is not healthy.
Note that I'm in favor of changing the wording to whatever we might find more appropriate, but one thing must be clear: we must try to reduce the friction that a negative vote might create and we should try to reduce the change that users will not want to try out un-stamped blocks because scared by them.
Thoughts?
-- Stefano.
