I'd like to expand a bit on the phrase "opportunity cost" to try and make
it more concrete: delaying a release means that the community is *not*
receiving various bug fixes (and features). Just as a particular example,
the wait for 2.3.2 delayed a fix for the Py3.7 iterator breaking change
that was also causing a correctness bug. It also delays community feedback
from running new releases. That in and of itself does not give an answer
to block/not-block for any specific case, but it's another way of saying
that blocking a release *prevents* people from getting bug fixes, as well
as potentially fixing bugs.
On Thu, Oct 25, 2018 at 7:39 AM Sean Owen <sro...@gmail.com> wrote:
> What does "PMC members aren't saying its a block for reasons other then
> the actual impact the jira has" mean that isn't already widely agreed?
> Likewise "Committers and PMC members should not be saying its not a
> blocker because they personally or their company doesn't care about this
> feature or api". It sounds like insinuation, and I'd rather make it
> explicit -- call out the bad actions -- or keep it to observable technical
> issues.
>
> Likewise one could say there's a problem just because A thinks X should be
> a blocker and B disagrees. I see no bad faith, process problem, or obvious
> errors. Do you? I see disagreement, and it's tempting to suspect motives. I
> have seen what I think are actual bad-faith decisions in the past in this
> project, too. I don't see it here though and want to stick to 'now'.
>
> (Aside: the implication is that those representing vendors are
> steam-rolling a release. Actually, the cynical incentives cut the other way
> here. Blessing the latest changes as OSS Apache Spark is predominantly
> beneficial to users of OSS, not distros. In fact, it forces distros to make
> changes. And broadly, vendors have much more accountability for quality of
> releases, because they're paid to.)
>
>
> I'm still not sure what specifically the objection is to what here? I
> understand a lot is in flight and nobody agrees with every decision made,
> but, what else is new?
> Concretely: the release is held again to fix a few issues, in the end. For
> the map_filter issue, that seems like the right call, and there are a few
> other important issues that could be quickly fixed too. All is well there,
> yes?
>
> This has surfaced some implicit reasoning about releases that we could
> make explicit, like:
>
> (Sure, if you want to write down things like, release blockers should be
> decided in the interests of the project by the PMC, OK)
>
> We have a time-based release schedule, so time matters. There is an
> opportunity cost to not releasing. The bar for blockers goes up over time.
>
> Not all regressions are blockers. Would you hold a release over a trivial
> regression? but then which must or should block? There's no objective
> answer, but a reasonable rule is: non-trivial regressions from minor
> release x.y to x.{y+1} block releases. Regressions from x.{y-1} to x.{y+1}
> should, but not necessarily, block the release. We try hard to avoid
> regressions in x.y.0 releases because these are generally consumed by
> aggressive upgraders, on x.{y-1}.z now. If a bug exists in x.{y-1}, they're
> not affected or worked around it. The cautious upgrader goes from maybe
> x.{y-2}.z to x.y.1 later. They're affected, but not before, maybe, a
> maintenance release. A crude argument, and it's not an argument that
> regressions are OK. It's an argument that 'old' regressions matter less.
> And maybe it's reasonable to draw the "must" vs "should" line between them.
>
>
>
> On Thu, Oct 25, 2018 at 8:51 AM Tom Graves <tgraves...@yahoo.com> wrote:
>
>> So just to clarify a few things in case people didn't read the entire
>> thread in the PR, the discussion is what is the criteria for a blocker and
>> really my concerns are what people are using as criteria for not marking a
>> jira as a blocker.
>>
>> The only thing we have documented to mark a jira as a blocker is for
>> correctness issues: http://spark.apache.org/contributing.html. And
>> really I think that is initially mark it as a blocker to bring attention to
>> it.
>> The final decision as to whether something is a blocker is up to the PMC
>> who votes on whether a release passes. I think it would be impossible to
>> properly define what a blocker is with strict rules.
>>
>> Personally from this thread I would like to make sure committers and PMC
>> members aren't saying its a block for reasons other then the actual impact
>> the jira has and if its at all in question it should be brought to the
>> PMC's attention for a vote. I agree with others that if its during an RC
>> it should be talked about on the RC thread.
>>
>> A few specific things that were said that I disagree with are:
>> - its not a blocker because it was also an issue in the last release
>> (meaning feature release). ie the bug was introduced in 2.2 and now we are
>> doing 2.4 so its automatically not a blocker. This to me is just wrong.
>> Lots of things are not found immediately, or aren't reported immediately.
>> Now I do believe the timeframe its been in there does affect the decision
>> on the impact but just making the decision on this to me is to strict.
>> - Committers and PMC members should not be saying its not a blocker
>> because they personally or their company doesn't care about this feature or
>> api, or state that the Spark project as a whole doesn't care about this
>> feature unless that was specifically voted on at the project level. They
>> need to follow the api compatibility we have documented. This is really a
>> broader issue then just marking a jira, it goes to anything checked in and
>> perhaps need to be a separate thread.
>>
>>
>> For the verbiage of what a regression is, it seems like that should be
>> defined by our versioning documents. It states what we do in maintenance,
>> feature, and major releases (
>> http://spark.apache.org/versioning-policy.html), if its not defined by
>> that we probably need to clarify. There was a good example we might want
>> to clarify about things like scala or java compatibility in feature
>> releases.
>>
>> Obviously this is my opinion and its here for everyone to discuss and
>> come to a consensus on.
>>
>> Tom
>>
>>
