Re: Project management (JIRA fix version tracking) is in crisis
PRs can be opened for backports and cherry picks and you can have all of
the desired precommit reports available before any merging happens.
Publishing changes should be relatively atomic. (Is "relatively" better? I
mean on the same day.) The JIRAs that linger open in a half open state with
some commits on some branches is a bad practice that should be eliminated,
IMHO.
On Tue, Jan 19, 2021 at 8:21 AM Nick Dimiduk wrote:
> It sounds like we have rough consensus at least on removing the CHANGES and
> RELEASENOTES files from source control. I filed HBASE-25520 to track this
> work.
>
> > What is the common thing here? That the merge action is atomic! All are
> ready or none are ready! Don't stuff half done work in some branches
> leaving dangling open JIRAs and random PRs for other branches!
>
> I don't think the action of landing a patch can be atomic so long as we
> start with master and then backport. For example, I happen to prefer to use
> pre-commit checking of a PR on the backport to earlier branches. (Y'all
> should prefer this too -- it's why we have pre-commit checking!) This
> results in a workflow of (1) land PR vs master (2) cherry-pick commit to
> backport branch (3) publish PR vs backport branch (4) wait 4-24 hours for
> pre-commit check.
>
> In order to have something close to atomic commits AND get the
> peace-of-mind of a successful PR build vs. the backport branch(es), I would
> change the above workflow to prepare all backport PRs at once. Then I would
> only merge all of them once all backport PRs pass their checks.
>
> Then there's the issue of human error. People will make mistakes, post
> addendums ; need to re-open a ticket.
>
> In short, there are several reasons for a commit (even to a single branch)
> to be non-atomic. I think any process that doesn't account for an issue
> being open over the course of several days is not realistic.
>
> On Mon, Jan 18, 2021 at 4:55 PM Andrew Purtell
> wrote:
>
> > Thank you Wellington. We do need for committers to pay attention to JIRA.
> > We still use JIRA for project management so we can't ignore it or allow
> it
> > to get out of sync with reality anyway. As long as we keep fix versions
> in
> > sync, we can continue to use today's create-release script and Yetus'
> > nice releasedocmaker tool for generating the change log and release
> notes.
> > (There is no substitute for these tools that use git history.)
> >
> > The release audit tool proposed on HBASE-22853 will also be really
> helpful,
> > but it is orthogonal to committers maintaining JIRA discipline. If
> > committers don't do that, the audit tool will give the RM a lot of
> warnings
> > that the RM will have to spend time cleaning up. If committers maintain
> > good JIRA discipline, the audit tool will quickly confirm the consistency
> > of JIRA fix versions and release branch commit history and the RM will
> > continue to have a pleasant day.
> >
> > My take is this:
> >
> > You have a change in mind. You will also have a target branch in mind -
> > master and/or branch-2 and/or branch-1 - depending on when and where you
> > need it to end up.
> >
> > Develop your change for your target branch. Remember that changes are
> > committed back in order like so: master -> branch-2 ( -> maybe
> branch-2.x
> > ( -> maybe branch-1 ) ). So if you are planning to commit to branch-2.4,
> > we will need PRs or patches for master, branch-2, and branch-2.4. Prepare
> > all of these patches before raising the PRs!
> >
> > Want to commit only to master? Open PR for master, merge, set fix
> version {
> > 3.0.0-alpha }, resolve, done!
> >
> > Want to commit to branch-2?: Open PR for master, cherry pick to branch-2
> > (or PR for branch-2), merge/commit all at once, set fix version {
> > 3.0.0-alpha, 2.5.0 }, resolve, done!
> >
> > Want to commit to branch-2.4? Open PR for master, cherry pick to branch-2
> > (or PR for branch-2), cherry pick to branch-2.4 (or PR for branch-2.4),
> > merge/commit all at once, set fix versions { 3.0.0-alpha, 2.5.0, 2.4.2 },
> > resolve, done!
> >
> > Want to commit to all branch-2.x? Open PR for master, cherry pick to
> > branch-2 (or PR for branch-2), cherry pick to branch-2.4 (or PR for
> > branch-2.4), cherry pick to branch-2.3 (or PR for branch-2.3),
> merge/commit
> > all at once, set fix versions { 3.0.0-alpha, 2.5.0, 2.4.2, 2.3.7 },
> > resolve, done!
> >
> > What is the common thing here? That the merge action is atomic! All are
> > ready or none are ready! Don't stuff half done work in some branches
> > leaving dangling open JIRAs and random PRs for other branches!
> >
> > If your target branch is branch-1, that can be a problem, because
> > backporting to branch-1 can be difficult. (Related: I think this year I
> > will propose EOL of branch-1, maybe in December.) So you might want to
> > select branch-2 as your initial target branch, do the above, and after
> > resolving the JIRA open a new JIRA to track the branch-1 backport.
> >
> >
> >
> > On
Re: Project management (JIRA fix version tracking) is in crisis
It sounds like we have rough consensus at least on removing the CHANGES and
RELEASENOTES files from source control. I filed HBASE-25520 to track this
work.
> What is the common thing here? That the merge action is atomic! All are
ready or none are ready! Don't stuff half done work in some branches
leaving dangling open JIRAs and random PRs for other branches!
I don't think the action of landing a patch can be atomic so long as we
start with master and then backport. For example, I happen to prefer to use
pre-commit checking of a PR on the backport to earlier branches. (Y'all
should prefer this too -- it's why we have pre-commit checking!) This
results in a workflow of (1) land PR vs master (2) cherry-pick commit to
backport branch (3) publish PR vs backport branch (4) wait 4-24 hours for
pre-commit check.
In order to have something close to atomic commits AND get the
peace-of-mind of a successful PR build vs. the backport branch(es), I would
change the above workflow to prepare all backport PRs at once. Then I would
only merge all of them once all backport PRs pass their checks.
Then there's the issue of human error. People will make mistakes, post
addendums ; need to re-open a ticket.
In short, there are several reasons for a commit (even to a single branch)
to be non-atomic. I think any process that doesn't account for an issue
being open over the course of several days is not realistic.
On Mon, Jan 18, 2021 at 4:55 PM Andrew Purtell wrote:
> Thank you Wellington. We do need for committers to pay attention to JIRA.
> We still use JIRA for project management so we can't ignore it or allow it
> to get out of sync with reality anyway. As long as we keep fix versions in
> sync, we can continue to use today's create-release script and Yetus'
> nice releasedocmaker tool for generating the change log and release notes.
> (There is no substitute for these tools that use git history.)
>
> The release audit tool proposed on HBASE-22853 will also be really helpful,
> but it is orthogonal to committers maintaining JIRA discipline. If
> committers don't do that, the audit tool will give the RM a lot of warnings
> that the RM will have to spend time cleaning up. If committers maintain
> good JIRA discipline, the audit tool will quickly confirm the consistency
> of JIRA fix versions and release branch commit history and the RM will
> continue to have a pleasant day.
>
> My take is this:
>
> You have a change in mind. You will also have a target branch in mind -
> master and/or branch-2 and/or branch-1 - depending on when and where you
> need it to end up.
>
> Develop your change for your target branch. Remember that changes are
> committed back in order like so: master -> branch-2 ( -> maybe branch-2.x
> ( -> maybe branch-1 ) ). So if you are planning to commit to branch-2.4,
> we will need PRs or patches for master, branch-2, and branch-2.4. Prepare
> all of these patches before raising the PRs!
>
> Want to commit only to master? Open PR for master, merge, set fix version {
> 3.0.0-alpha }, resolve, done!
>
> Want to commit to branch-2?: Open PR for master, cherry pick to branch-2
> (or PR for branch-2), merge/commit all at once, set fix version {
> 3.0.0-alpha, 2.5.0 }, resolve, done!
>
> Want to commit to branch-2.4? Open PR for master, cherry pick to branch-2
> (or PR for branch-2), cherry pick to branch-2.4 (or PR for branch-2.4),
> merge/commit all at once, set fix versions { 3.0.0-alpha, 2.5.0, 2.4.2 },
> resolve, done!
>
> Want to commit to all branch-2.x? Open PR for master, cherry pick to
> branch-2 (or PR for branch-2), cherry pick to branch-2.4 (or PR for
> branch-2.4), cherry pick to branch-2.3 (or PR for branch-2.3), merge/commit
> all at once, set fix versions { 3.0.0-alpha, 2.5.0, 2.4.2, 2.3.7 },
> resolve, done!
>
> What is the common thing here? That the merge action is atomic! All are
> ready or none are ready! Don't stuff half done work in some branches
> leaving dangling open JIRAs and random PRs for other branches!
>
> If your target branch is branch-1, that can be a problem, because
> backporting to branch-1 can be difficult. (Related: I think this year I
> will propose EOL of branch-1, maybe in December.) So you might want to
> select branch-2 as your initial target branch, do the above, and after
> resolving the JIRA open a new JIRA to track the branch-1 backport.
>
>
>
> On Mon, Jan 18, 2021 at 2:32 AM Wellington Chevreuil <
> wellington.chevre...@gmail.com> wrote:
>
> > While no perfect script/tool is available to solve this problem, we may
> > agree on a preferable commiting process to make RM's life less difficult.
> > My take from this thread is that something like below would help:
> >
> > 1) Once a commit happens on any branch, resolve the jira and update
> > fixVersions field with value corresponding the committed branch;
> > 2) Open sub-task jiras for each subsequent commit on different branches,
> > repeating the step #1 above on each of these subtasks;
> > - How about
Re: Project management (JIRA fix version tracking) is in crisis
Thank you Wellington. We do need for committers to pay attention to JIRA.
We still use JIRA for project management so we can't ignore it or allow it
to get out of sync with reality anyway. As long as we keep fix versions in
sync, we can continue to use today's create-release script and Yetus'
nice releasedocmaker tool for generating the change log and release notes.
(There is no substitute for these tools that use git history.)
The release audit tool proposed on HBASE-22853 will also be really helpful,
but it is orthogonal to committers maintaining JIRA discipline. If
committers don't do that, the audit tool will give the RM a lot of warnings
that the RM will have to spend time cleaning up. If committers maintain
good JIRA discipline, the audit tool will quickly confirm the consistency
of JIRA fix versions and release branch commit history and the RM will
continue to have a pleasant day.
My take is this:
You have a change in mind. You will also have a target branch in mind -
master and/or branch-2 and/or branch-1 - depending on when and where you
need it to end up.
Develop your change for your target branch. Remember that changes are
committed back in order like so: master -> branch-2 ( -> maybe branch-2.x
( -> maybe branch-1 ) ). So if you are planning to commit to branch-2.4,
we will need PRs or patches for master, branch-2, and branch-2.4. Prepare
all of these patches before raising the PRs!
Want to commit only to master? Open PR for master, merge, set fix version {
3.0.0-alpha }, resolve, done!
Want to commit to branch-2?: Open PR for master, cherry pick to branch-2
(or PR for branch-2), merge/commit all at once, set fix version {
3.0.0-alpha, 2.5.0 }, resolve, done!
Want to commit to branch-2.4? Open PR for master, cherry pick to branch-2
(or PR for branch-2), cherry pick to branch-2.4 (or PR for branch-2.4),
merge/commit all at once, set fix versions { 3.0.0-alpha, 2.5.0, 2.4.2 },
resolve, done!
Want to commit to all branch-2.x? Open PR for master, cherry pick to
branch-2 (or PR for branch-2), cherry pick to branch-2.4 (or PR for
branch-2.4), cherry pick to branch-2.3 (or PR for branch-2.3), merge/commit
all at once, set fix versions { 3.0.0-alpha, 2.5.0, 2.4.2, 2.3.7 },
resolve, done!
What is the common thing here? That the merge action is atomic! All are
ready or none are ready! Don't stuff half done work in some branches
leaving dangling open JIRAs and random PRs for other branches!
If your target branch is branch-1, that can be a problem, because
backporting to branch-1 can be difficult. (Related: I think this year I
will propose EOL of branch-1, maybe in December.) So you might want to
select branch-2 as your initial target branch, do the above, and after
resolving the JIRA open a new JIRA to track the branch-1 backport.
On Mon, Jan 18, 2021 at 2:32 AM Wellington Chevreuil <
wellington.chevre...@gmail.com> wrote:
> While no perfect script/tool is available to solve this problem, we may
> agree on a preferable commiting process to make RM's life less difficult.
> My take from this thread is that something like below would help:
>
> 1) Once a commit happens on any branch, resolve the jira and update
> fixVersions field with value corresponding the committed branch;
> 2) Open sub-task jiras for each subsequent commit on different branches,
> repeating the step #1 above on each of these subtasks;
> - How about commit messages on these subtask jiras? Should it mention both
> parent and subtask jira ids? Or maybe just parent id, considering most
> commits would be simple cherry-picks?
> 3) If a commit is reverted, related jira should be reopened and fixVersions
> field should be cleaned up.
>
>
>
> Em seg., 18 de jan. de 2021 às 02:18, 张铎(Duo Zhang) >
> escreveu:
>
> > What about a Jenkins job for detecting the diff between jira and git?
> >
> > And I agree that, let's not include CHANGES.md and RELEASENOTES.md in our
> > code base directly which means we need to sink an RC only if these two
> > files are incorrect, not the actual code.
> >
> > Andrew Purtell 于2021年1月16日周六 上午11:11写道：
> >
> > > I think the discussion has already moved forward a lot with mention of
> > > scripts I did not know existed. One well developed git<->jira diff
> tool,
> > > documented as part of the RM procedure, would be much better than we
> have
> > > right now, with various RMs doing ad hoc things.
> > >
> > >
> > > On Fri, Jan 15, 2021 at 6:04 PM 张铎(Duo Zhang)
> > > wrote:
> > >
> > > > Haven't fully read all the comments but I think this is the duty for
> > the
> > > > release manager? This is what I use to get the difference between
> jira
> > > and
> > > > git
> > > >
> > > > git log --oneline
> > 0167558eb31ff48308d592ef70b6d005ba6d21fb...3ee8b0c75b |
> > > > grep -o -E "^[0-9a-z]*\s*HBASE-[0-9]*"|awk '{print $2}' | sort -u >
> > > > in_git_2.0.0.txt
> > > > cat CHANGELOG.md | grep -o -E "\[HBASE-[0-9]*\]" | grep -o -E
> > > > "HBASE-[0-9]*" | sort -u > CHANGELOG_jiras.txt
> > > >
> > > > And then
Re: Project management (JIRA fix version tracking) is in crisis
While no perfect script/tool is available to solve this problem, we may
agree on a preferable commiting process to make RM's life less difficult.
My take from this thread is that something like below would help:
1) Once a commit happens on any branch, resolve the jira and update
fixVersions field with value corresponding the committed branch;
2) Open sub-task jiras for each subsequent commit on different branches,
repeating the step #1 above on each of these subtasks;
- How about commit messages on these subtask jiras? Should it mention both
parent and subtask jira ids? Or maybe just parent id, considering most
commits would be simple cherry-picks?
3) If a commit is reverted, related jira should be reopened and fixVersions
field should be cleaned up.
Em seg., 18 de jan. de 2021 às 02:18, 张铎(Duo Zhang)
escreveu:
> What about a Jenkins job for detecting the diff between jira and git?
>
> And I agree that, let's not include CHANGES.md and RELEASENOTES.md in our
> code base directly which means we need to sink an RC only if these two
> files are incorrect, not the actual code.
>
> Andrew Purtell 于2021年1月16日周六 上午11:11写道：
>
> > I think the discussion has already moved forward a lot with mention of
> > scripts I did not know existed. One well developed git<->jira diff tool,
> > documented as part of the RM procedure, would be much better than we have
> > right now, with various RMs doing ad hoc things.
> >
> >
> > On Fri, Jan 15, 2021 at 6:04 PM 张铎(Duo Zhang)
> > wrote:
> >
> > > Haven't fully read all the comments but I think this is the duty for
> the
> > > release manager? This is what I use to get the difference between jira
> > and
> > > git
> > >
> > > git log --oneline
> 0167558eb31ff48308d592ef70b6d005ba6d21fb...3ee8b0c75b |
> > > grep -o -E "^[0-9a-z]*\s*HBASE-[0-9]*"|awk '{print $2}' | sort -u >
> > > in_git_2.0.0.txt
> > > cat CHANGELOG.md | grep -o -E "\[HBASE-[0-9]*\]" | grep -o -E
> > > "HBASE-[0-9]*" | sort -u > CHANGELOG_jiras.txt
> > >
> > > And then you could use comm to get the difference.
> > >
> > > We all make mistakes so this process can not be automatic, trust me.
> > > Sometimes a commit is reverted and never applied again, sometimes we
> > > include the wrong jira id in commit message.
> > > This is why I always use a jira issue to land the CHANGES and
> > RELEASENOTES,
> > > tag it, and then use the release script to generate the RC.
> > >
> > > For me, it will cost me a lot of time when creating a new minor release
> > > because there will be a lot of commits, but for a patch release usually
> > it
> > > is fine.
> > >
> > > Maybe things will be easier if we move all the development process to
> > > github?
> > >
> > > Thanks.
> > >
> > > Sean Busbey 于2021年1月16日周六 上午8:05写道：
> > >
> > > > What would the logistics look like for git as the source of truth?
> > > >
> > > > Every release candidate I've ever done I've had to do the jira and
> git
> > > > reconciliation. There are always errors in both. Folks commit stuff
> > with
> > > > the wrong JIRA ID or wrong message.
> > > >
> > > > Would reconciliation for a RM then require revert and reapply of any
> > such
> > > > changes?
> > > >
> > > > On Fri, Jan 15, 2021, 17:54 Stack wrote:
> > > >
> > > > > On Fri, Jan 15, 2021 at 10:06 AM Andrew Purtell <
> apurt...@apache.org
> > >
> > > > > wrote:
> > > > >
> > > > > > I have now had to sink two 2.4.1 RCs because of errors in the
> > change
> > > > log.
> > > > > >
> > > > > > I made a pass over git history and ensured every commit was
> > > included. I
> > > > > had
> > > > > > also made a pass over JIRA to move out any unresolved issues or
> > > > complete
> > > > > > the resolution of same. What I did not do is check that every
> > > resolved
> > > > > JIRA
> > > > > > corresponded to an actual commit. This is not something RMs have
> > had
> > > to
> > > > > do
> > > > > > in the past and it asks a lot of them.
> > > > > >
> > > > > >
> > > > > Just to say, that making RCs, I've the done the JIRA<->GIT
> > > reconciliation
> > > > > described (ugly hack scripting, sorting, and comparing). Others
> have
> > > too
> > > > > I've noticed. It is awful, yes, especially when issues like those
> > found
> > > > by
> > > > > Viraj in yours and Huaxiangs RCs this week. The refguide is vague
> on
> > > what
> > > > > is involved; it needs more detail. The issue HBASE-22853
> description
> > on
> > > > the
> > > > > need for a tool to do the reconcile is a bit better.
> > > > >
> > > > >
> > > > >
> > > > > > I know NOW that as RM I cannot currently trust committers to get
> > fix
> > > > > > versions right or care about this.
> > > > > >
> > > > > > That's right... Commtters cannot be trusted to correctly maintain
> > > issue
> > > > > > metadata in JIRA.
> > > > > >
> > > > > >
> > > > > Yes (says a key offender).
> > > > >
> > > > >
> > > > >
> > > > > > That is not a good situation for the project to be in. Up until
> now
> > > it
> > > > > has
> > > > > > not been the
Re: Project management (JIRA fix version tracking) is in crisis
What about a Jenkins job for detecting the diff between jira and git?
And I agree that, let's not include CHANGES.md and RELEASENOTES.md in our
code base directly which means we need to sink an RC only if these two
files are incorrect, not the actual code.
Andrew Purtell 于2021年1月16日周六 上午11:11写道：
> I think the discussion has already moved forward a lot with mention of
> scripts I did not know existed. One well developed git<->jira diff tool,
> documented as part of the RM procedure, would be much better than we have
> right now, with various RMs doing ad hoc things.
>
>
> On Fri, Jan 15, 2021 at 6:04 PM 张铎(Duo Zhang)
> wrote:
>
> > Haven't fully read all the comments but I think this is the duty for the
> > release manager? This is what I use to get the difference between jira
> and
> > git
> >
> > git log --oneline 0167558eb31ff48308d592ef70b6d005ba6d21fb...3ee8b0c75b |
> > grep -o -E "^[0-9a-z]*\s*HBASE-[0-9]*"|awk '{print $2}' | sort -u >
> > in_git_2.0.0.txt
> > cat CHANGELOG.md | grep -o -E "\[HBASE-[0-9]*\]" | grep -o -E
> > "HBASE-[0-9]*" | sort -u > CHANGELOG_jiras.txt
> >
> > And then you could use comm to get the difference.
> >
> > We all make mistakes so this process can not be automatic, trust me.
> > Sometimes a commit is reverted and never applied again, sometimes we
> > include the wrong jira id in commit message.
> > This is why I always use a jira issue to land the CHANGES and
> RELEASENOTES,
> > tag it, and then use the release script to generate the RC.
> >
> > For me, it will cost me a lot of time when creating a new minor release
> > because there will be a lot of commits, but for a patch release usually
> it
> > is fine.
> >
> > Maybe things will be easier if we move all the development process to
> > github?
> >
> > Thanks.
> >
> > Sean Busbey 于2021年1月16日周六 上午8:05写道：
> >
> > > What would the logistics look like for git as the source of truth?
> > >
> > > Every release candidate I've ever done I've had to do the jira and git
> > > reconciliation. There are always errors in both. Folks commit stuff
> with
> > > the wrong JIRA ID or wrong message.
> > >
> > > Would reconciliation for a RM then require revert and reapply of any
> such
> > > changes?
> > >
> > > On Fri, Jan 15, 2021, 17:54 Stack wrote:
> > >
> > > > On Fri, Jan 15, 2021 at 10:06 AM Andrew Purtell >
> > > > wrote:
> > > >
> > > > > I have now had to sink two 2.4.1 RCs because of errors in the
> change
> > > log.
> > > > >
> > > > > I made a pass over git history and ensured every commit was
> > included. I
> > > > had
> > > > > also made a pass over JIRA to move out any unresolved issues or
> > > complete
> > > > > the resolution of same. What I did not do is check that every
> > resolved
> > > > JIRA
> > > > > corresponded to an actual commit. This is not something RMs have
> had
> > to
> > > > do
> > > > > in the past and it asks a lot of them.
> > > > >
> > > > >
> > > > Just to say, that making RCs, I've the done the JIRA<->GIT
> > reconciliation
> > > > described (ugly hack scripting, sorting, and comparing). Others have
> > too
> > > > I've noticed. It is awful, yes, especially when issues like those
> found
> > > by
> > > > Viraj in yours and Huaxiangs RCs this week. The refguide is vague on
> > what
> > > > is involved; it needs more detail. The issue HBASE-22853 description
> on
> > > the
> > > > need for a tool to do the reconcile is a bit better.
> > > >
> > > >
> > > >
> > > > > I know NOW that as RM I cannot currently trust committers to get
> fix
> > > > > versions right or care about this.
> > > > >
> > > > > That's right... Commtters cannot be trusted to correctly maintain
> > issue
> > > > > metadata in JIRA.
> > > > >
> > > > >
> > > > Yes (says a key offender).
> > > >
> > > >
> > > >
> > > > > That is not a good situation for the project to be in. Up until now
> > it
> > > > has
> > > > > not been the responsibility of the RM to check each and every JIRA
> > > > status.
> > > > > It has been the collective responsibility of committers to care
> about
> > > the
> > > > > project's release tracking insofar as to correctly update fix
> > versions
> > > in
> > > > > JIRA. For releases containing relatively few changes, like 2.4.1,
> > with
> > > > ~50
> > > > > changes, I suppose it is possible for the RM to remove all 2.4.1
> fix
> > > > > versions, walk the commit history, and set back fix versions on
> JIRA
> > to
> > > > > actually correspond with what was truly committed. However, for
> minor
> > > > > releases, with hundreds of commits, this will not be possible.
> > > > >
> > > > > I think the root cause is GitHub and JIRA are two separate change
> > > > tracking
> > > > > systems with only a minimal amount of integration. It requires
> manual
> > > > > effort. More and more, new committers are familiar with GitHub and
> > PRs
> > > > and
> > > > > are not familiar with JIRA and the Apache way of using JIRA to
> build
> > > > change
> > > > > logs. We need to better educate new and existing
Re: Project management (JIRA fix version tracking) is in crisis
I think the discussion has already moved forward a lot with mention of
scripts I did not know existed. One well developed git<->jira diff tool,
documented as part of the RM procedure, would be much better than we have
right now, with various RMs doing ad hoc things.
On Fri, Jan 15, 2021 at 6:04 PM 张铎(Duo Zhang) wrote:
> Haven't fully read all the comments but I think this is the duty for the
> release manager? This is what I use to get the difference between jira and
> git
>
> git log --oneline 0167558eb31ff48308d592ef70b6d005ba6d21fb...3ee8b0c75b |
> grep -o -E "^[0-9a-z]*\s*HBASE-[0-9]*"|awk '{print $2}' | sort -u >
> in_git_2.0.0.txt
> cat CHANGELOG.md | grep -o -E "\[HBASE-[0-9]*\]" | grep -o -E
> "HBASE-[0-9]*" | sort -u > CHANGELOG_jiras.txt
>
> And then you could use comm to get the difference.
>
> We all make mistakes so this process can not be automatic, trust me.
> Sometimes a commit is reverted and never applied again, sometimes we
> include the wrong jira id in commit message.
> This is why I always use a jira issue to land the CHANGES and RELEASENOTES,
> tag it, and then use the release script to generate the RC.
>
> For me, it will cost me a lot of time when creating a new minor release
> because there will be a lot of commits, but for a patch release usually it
> is fine.
>
> Maybe things will be easier if we move all the development process to
> github?
>
> Thanks.
>
> Sean Busbey 于2021年1月16日周六 上午8:05写道：
>
> > What would the logistics look like for git as the source of truth?
> >
> > Every release candidate I've ever done I've had to do the jira and git
> > reconciliation. There are always errors in both. Folks commit stuff with
> > the wrong JIRA ID or wrong message.
> >
> > Would reconciliation for a RM then require revert and reapply of any such
> > changes?
> >
> > On Fri, Jan 15, 2021, 17:54 Stack wrote:
> >
> > > On Fri, Jan 15, 2021 at 10:06 AM Andrew Purtell
> > > wrote:
> > >
> > > > I have now had to sink two 2.4.1 RCs because of errors in the change
> > log.
> > > >
> > > > I made a pass over git history and ensured every commit was
> included. I
> > > had
> > > > also made a pass over JIRA to move out any unresolved issues or
> > complete
> > > > the resolution of same. What I did not do is check that every
> resolved
> > > JIRA
> > > > corresponded to an actual commit. This is not something RMs have had
> to
> > > do
> > > > in the past and it asks a lot of them.
> > > >
> > > >
> > > Just to say, that making RCs, I've the done the JIRA<->GIT
> reconciliation
> > > described (ugly hack scripting, sorting, and comparing). Others have
> too
> > > I've noticed. It is awful, yes, especially when issues like those found
> > by
> > > Viraj in yours and Huaxiangs RCs this week. The refguide is vague on
> what
> > > is involved; it needs more detail. The issue HBASE-22853 description on
> > the
> > > need for a tool to do the reconcile is a bit better.
> > >
> > >
> > >
> > > > I know NOW that as RM I cannot currently trust committers to get fix
> > > > versions right or care about this.
> > > >
> > > > That's right... Commtters cannot be trusted to correctly maintain
> issue
> > > > metadata in JIRA.
> > > >
> > > >
> > > Yes (says a key offender).
> > >
> > >
> > >
> > > > That is not a good situation for the project to be in. Up until now
> it
> > > has
> > > > not been the responsibility of the RM to check each and every JIRA
> > > status.
> > > > It has been the collective responsibility of committers to care about
> > the
> > > > project's release tracking insofar as to correctly update fix
> versions
> > in
> > > > JIRA. For releases containing relatively few changes, like 2.4.1,
> with
> > > ~50
> > > > changes, I suppose it is possible for the RM to remove all 2.4.1 fix
> > > > versions, walk the commit history, and set back fix versions on JIRA
> to
> > > > actually correspond with what was truly committed. However, for minor
> > > > releases, with hundreds of commits, this will not be possible.
> > > >
> > > > I think the root cause is GitHub and JIRA are two separate change
> > > tracking
> > > > systems with only a minimal amount of integration. It requires manual
> > > > effort. More and more, new committers are familiar with GitHub and
> PRs
> > > and
> > > > are not familiar with JIRA and the Apache way of using JIRA to build
> > > change
> > > > logs. We need to better educate new and existing committers on their
> > > > responsibilities with regards to maintaining JIRA metadata correctly.
> > > >
> > > >
> > > Agree.
> > >
> > > Other issues are that the release process has been evolving. In the old
> > > days, JIRA was the source of truth; changelog was a JIRA report. The
> > > CHANGES.md/RELEASENOTES.md have come to the fore perhaps making
> > difference
> > > between JIRA and GIT more pronounced.
> > >
> > > I like the suggestion made by you above (and Nick in an internal
> > > discussion) that git be the source of truth. That the CHANGES.md NOT be
> > >
Re: Project management (JIRA fix version tracking) is in crisis
Haven't fully read all the comments but I think this is the duty for the
release manager? This is what I use to get the difference between jira and
git
git log --oneline 0167558eb31ff48308d592ef70b6d005ba6d21fb...3ee8b0c75b |
grep -o -E "^[0-9a-z]*\s*HBASE-[0-9]*"|awk '{print $2}' | sort -u >
in_git_2.0.0.txt
cat CHANGELOG.md | grep -o -E "\[HBASE-[0-9]*\]" | grep -o -E
"HBASE-[0-9]*" | sort -u > CHANGELOG_jiras.txt
And then you could use comm to get the difference.
We all make mistakes so this process can not be automatic, trust me.
Sometimes a commit is reverted and never applied again, sometimes we
include the wrong jira id in commit message.
This is why I always use a jira issue to land the CHANGES and RELEASENOTES,
tag it, and then use the release script to generate the RC.
For me, it will cost me a lot of time when creating a new minor release
because there will be a lot of commits, but for a patch release usually it
is fine.
Maybe things will be easier if we move all the development process to
github?
Thanks.
Sean Busbey 于2021年1月16日周六 上午8:05写道：
> What would the logistics look like for git as the source of truth?
>
> Every release candidate I've ever done I've had to do the jira and git
> reconciliation. There are always errors in both. Folks commit stuff with
> the wrong JIRA ID or wrong message.
>
> Would reconciliation for a RM then require revert and reapply of any such
> changes?
>
> On Fri, Jan 15, 2021, 17:54 Stack wrote:
>
> > On Fri, Jan 15, 2021 at 10:06 AM Andrew Purtell
> > wrote:
> >
> > > I have now had to sink two 2.4.1 RCs because of errors in the change
> log.
> > >
> > > I made a pass over git history and ensured every commit was included. I
> > had
> > > also made a pass over JIRA to move out any unresolved issues or
> complete
> > > the resolution of same. What I did not do is check that every resolved
> > JIRA
> > > corresponded to an actual commit. This is not something RMs have had to
> > do
> > > in the past and it asks a lot of them.
> > >
> > >
> > Just to say, that making RCs, I've the done the JIRA<->GIT reconciliation
> > described (ugly hack scripting, sorting, and comparing). Others have too
> > I've noticed. It is awful, yes, especially when issues like those found
> by
> > Viraj in yours and Huaxiangs RCs this week. The refguide is vague on what
> > is involved; it needs more detail. The issue HBASE-22853 description on
> the
> > need for a tool to do the reconcile is a bit better.
> >
> >
> >
> > > I know NOW that as RM I cannot currently trust committers to get fix
> > > versions right or care about this.
> > >
> > > That's right... Commtters cannot be trusted to correctly maintain issue
> > > metadata in JIRA.
> > >
> > >
> > Yes (says a key offender).
> >
> >
> >
> > > That is not a good situation for the project to be in. Up until now it
> > has
> > > not been the responsibility of the RM to check each and every JIRA
> > status.
> > > It has been the collective responsibility of committers to care about
> the
> > > project's release tracking insofar as to correctly update fix versions
> in
> > > JIRA. For releases containing relatively few changes, like 2.4.1, with
> > ~50
> > > changes, I suppose it is possible for the RM to remove all 2.4.1 fix
> > > versions, walk the commit history, and set back fix versions on JIRA to
> > > actually correspond with what was truly committed. However, for minor
> > > releases, with hundreds of commits, this will not be possible.
> > >
> > > I think the root cause is GitHub and JIRA are two separate change
> > tracking
> > > systems with only a minimal amount of integration. It requires manual
> > > effort. More and more, new committers are familiar with GitHub and PRs
> > and
> > > are not familiar with JIRA and the Apache way of using JIRA to build
> > change
> > > logs. We need to better educate new and existing committers on their
> > > responsibilities with regards to maintaining JIRA metadata correctly.
> > >
> > >
> > Agree.
> >
> > Other issues are that the release process has been evolving. In the old
> > days, JIRA was the source of truth; changelog was a JIRA report. The
> > CHANGES.md/RELEASENOTES.md have come to the fore perhaps making
> difference
> > between JIRA and GIT more pronounced.
> >
> > I like the suggestion made by you above (and Nick in an internal
> > discussion) that git be the source of truth. That the CHANGES.md NOT be
> > checked-in is also a good idea so it can be regenerated w/o need of a
> > change to the RC if JIRA is changed (I can work on this part if wanted).
> >
> > S
> >
> >
> > > --
> > > Best regards,
> > > Andrew
> > >
> > > Words like orphans lost among the crosstalk, meaning torn from truth's
> > > decrepit hands
> > >- A23, Crosstalk
> > >
> >
>
Re: Project management (JIRA fix version tracking) is in crisis
What would the logistics look like for git as the source of truth? Every release candidate I've ever done I've had to do the jira and git reconciliation. There are always errors in both. Folks commit stuff with the wrong JIRA ID or wrong message. Would reconciliation for a RM then require revert and reapply of any such changes? On Fri, Jan 15, 2021, 17:54 Stack wrote: > On Fri, Jan 15, 2021 at 10:06 AM Andrew Purtell > wrote: > > > I have now had to sink two 2.4.1 RCs because of errors in the change log. > > > > I made a pass over git history and ensured every commit was included. I > had > > also made a pass over JIRA to move out any unresolved issues or complete > > the resolution of same. What I did not do is check that every resolved > JIRA > > corresponded to an actual commit. This is not something RMs have had to > do > > in the past and it asks a lot of them. > > > > > Just to say, that making RCs, I've the done the JIRA<->GIT reconciliation > described (ugly hack scripting, sorting, and comparing). Others have too > I've noticed. It is awful, yes, especially when issues like those found by > Viraj in yours and Huaxiangs RCs this week. The refguide is vague on what > is involved; it needs more detail. The issue HBASE-22853 description on the > need for a tool to do the reconcile is a bit better. > > > > > I know NOW that as RM I cannot currently trust committers to get fix > > versions right or care about this. > > > > That's right... Commtters cannot be trusted to correctly maintain issue > > metadata in JIRA. > > > > > Yes (says a key offender). > > > > > That is not a good situation for the project to be in. Up until now it > has > > not been the responsibility of the RM to check each and every JIRA > status. > > It has been the collective responsibility of committers to care about the > > project's release tracking insofar as to correctly update fix versions in > > JIRA. For releases containing relatively few changes, like 2.4.1, with > ~50 > > changes, I suppose it is possible for the RM to remove all 2.4.1 fix > > versions, walk the commit history, and set back fix versions on JIRA to > > actually correspond with what was truly committed. However, for minor > > releases, with hundreds of commits, this will not be possible. > > > > I think the root cause is GitHub and JIRA are two separate change > tracking > > systems with only a minimal amount of integration. It requires manual > > effort. More and more, new committers are familiar with GitHub and PRs > and > > are not familiar with JIRA and the Apache way of using JIRA to build > change > > logs. We need to better educate new and existing committers on their > > responsibilities with regards to maintaining JIRA metadata correctly. > > > > > Agree. > > Other issues are that the release process has been evolving. In the old > days, JIRA was the source of truth; changelog was a JIRA report. The > CHANGES.md/RELEASENOTES.md have come to the fore perhaps making difference > between JIRA and GIT more pronounced. > > I like the suggestion made by you above (and Nick in an internal > discussion) that git be the source of truth. That the CHANGES.md NOT be > checked-in is also a good idea so it can be regenerated w/o need of a > change to the RC if JIRA is changed (I can work on this part if wanted). > > S > > > > -- > > Best regards, > > Andrew > > > > Words like orphans lost among the crosstalk, meaning torn from truth's > > decrepit hands > >- A23, Crosstalk > > >
Re: Project management (JIRA fix version tracking) is in crisis
On Fri, Jan 15, 2021 at 10:06 AM Andrew Purtell wrote: > I have now had to sink two 2.4.1 RCs because of errors in the change log. > > I made a pass over git history and ensured every commit was included. I had > also made a pass over JIRA to move out any unresolved issues or complete > the resolution of same. What I did not do is check that every resolved JIRA > corresponded to an actual commit. This is not something RMs have had to do > in the past and it asks a lot of them. > > Just to say, that making RCs, I've the done the JIRA<->GIT reconciliation described (ugly hack scripting, sorting, and comparing). Others have too I've noticed. It is awful, yes, especially when issues like those found by Viraj in yours and Huaxiangs RCs this week. The refguide is vague on what is involved; it needs more detail. The issue HBASE-22853 description on the need for a tool to do the reconcile is a bit better. > I know NOW that as RM I cannot currently trust committers to get fix > versions right or care about this. > > That's right... Commtters cannot be trusted to correctly maintain issue > metadata in JIRA. > > Yes (says a key offender). > That is not a good situation for the project to be in. Up until now it has > not been the responsibility of the RM to check each and every JIRA status. > It has been the collective responsibility of committers to care about the > project's release tracking insofar as to correctly update fix versions in > JIRA. For releases containing relatively few changes, like 2.4.1, with ~50 > changes, I suppose it is possible for the RM to remove all 2.4.1 fix > versions, walk the commit history, and set back fix versions on JIRA to > actually correspond with what was truly committed. However, for minor > releases, with hundreds of commits, this will not be possible. > > I think the root cause is GitHub and JIRA are two separate change tracking > systems with only a minimal amount of integration. It requires manual > effort. More and more, new committers are familiar with GitHub and PRs and > are not familiar with JIRA and the Apache way of using JIRA to build change > logs. We need to better educate new and existing committers on their > responsibilities with regards to maintaining JIRA metadata correctly. > > Agree. Other issues are that the release process has been evolving. In the old days, JIRA was the source of truth; changelog was a JIRA report. The CHANGES.md/RELEASENOTES.md have come to the fore perhaps making difference between JIRA and GIT more pronounced. I like the suggestion made by you above (and Nick in an internal discussion) that git be the source of truth. That the CHANGES.md NOT be checked-in is also a good idea so it can be regenerated w/o need of a change to the RC if JIRA is changed (I can work on this part if wanted). S > -- > Best regards, > Andrew > > Words like orphans lost among the crosstalk, meaning torn from truth's > decrepit hands >- A23, Crosstalk >
Re: Project management (JIRA fix version tracking) is in crisis
I basically agree with all of this, except: - I believe committers are going to have a hard time with fix version management and we should not trust this - Git as source of truth was discussed in the Yetus context IIRC but docmaker uses the JIRA API, so JIRA remains the effective source of truth without a good replacement (there is no replacement for docmaker), unless there is a GitHub API I'm not aware of (there could be). - I would prefer to resolve API compatibility or changelog inconsistencies before tagging the RC and publishing this tag - Unless the git branches are locked there is a race between prep and RC/tag publish. This part is fine as long as the RM can detect the collision (git rejects the push) and so ultimately this requires all prep happen locally with the tag/push/publish as final atomic step. On Fri, Jan 15, 2021 at 3:19 PM Nick Dimiduk wrote: > > I have now had to sink two 2.4.1 RCs because of errors in the change log. > > IIRC, it's a relatively new phenomenon that we sink an RC because the > CHANGES file is incomplete. > > > That's right... Commtters cannot be trusted to correctly maintain issue > metadata in JIRA. > > I wouldn't go so far as to make a blanket statement such as this... I > suspect there's a couple new people who missed this message. In my > experience, a polite nudge from the RM on the affected JIRA issue to the > offending committer resolves the knowledge gap. > > > I think the root cause is GitHub and JIRA are two separate change > tracking systems with only a minimal amount of integration. It requires > manual effort. > > This I 100% agree with. But then, even before GitHub, Jira and SVN were two > separate systems, and we managed fine. Maybe, as you say, the committer > mentality around attention to detail has laxed. Or maybe, as I suggest, > there's just an education gap involving a couple people, which can be > addressed. > > > We do not have a single source of truth as to what are active branches > and what was the most recent release by version number made from each > branch. > > Ah, now the discussion gets interesting! IMHO, we do have a source of truth > re: this information -- it's git. > > > Perhaps something can be created that takes the union of git branch > listing and reporter.a.o release version history. > > I did create such a tool, it scrapes git and Jira and produces a database > containing this information, from which reports are generated and ad-hoc > queries are issued. I committed it (nearly a year ago to the day) via > HBASE-22853 (Viraj, you signed off on this commit...). As I mentioned > earlier, it's still young, but it was invaluable for me in establishing a > baseline for 2.3.0, and includes a readme with lots of examples. > > I am of the opinion that the git repository is the authority on what is in > a release. After all, a release is defined as a permanent "rel" tag in git > (plus some ASF ceremony). IMHO, Jira needs to follow git, not the other way > around. Likewise, tracking the CHANGES file in git -- a file generated from > a Jira query -- is very silly. Instead, I propose that the changes file be > generated on demand, perhaps as part building the tarball. The only > alternative, if you want to retain the CHANGES file in git, is to include > an update to the CHANGES file with every git commit, which I think is > completely untenable. > > One other stumbling point is that our release notes are generated from a > field in Jira. We could generate release notes from git commit messages, > but I think that is an unrealistic burden to place on committers at the > time of commit. > > Following the logic that git is the source of truth, I propose a change to > our release generation process so that it looks something like > 1. set the version number in poms, commit. > 2. apply the RC tag. > 3. run the audit tool to generate a report. > 4. resolve inconsistencies (manually for now, eventually the tool could > update Jira -- it has an API) > 5. build artifacts, including generating any files from metadata in Jira. > > Step 3, when automated, can even be run during our nightly branch builds. > At that point, a committer doesn't need to worry at all about a fixVersion, > they just commit to their branches and automation takes care of the rest. > > The above has the issue around when to close a ticket. IMHO, a ticket > should be closed as soon as there's a commit to any branch, but maybe there > are other complexities to consider. > > Thanks for bringing this up, Andrew. It's bothered me since 2.3.0 :) > > -n > -- Best regards, Andrew Words like orphans lost among the crosstalk, meaning torn from truth's decrepit hands - A23, Crosstalk
Re: Project management (JIRA fix version tracking) is in crisis
If it's possible, I would also appreciate it if we could add a pre-commit hook to our repository that rejects a commit whose title does not match our prescribed pattern. Doing so would go a long way toward simplifying the parsing tools. -n On Fri, Jan 15, 2021 at 3:18 PM Nick Dimiduk wrote: > > I have now had to sink two 2.4.1 RCs because of errors in the change log. > > IIRC, it's a relatively new phenomenon that we sink an RC because the > CHANGES file is incomplete. > > > That's right... Commtters cannot be trusted to correctly maintain issue > metadata in JIRA. > > I wouldn't go so far as to make a blanket statement such as this... I > suspect there's a couple new people who missed this message. In my > experience, a polite nudge from the RM on the affected JIRA issue to the > offending committer resolves the knowledge gap. > > > I think the root cause is GitHub and JIRA are two separate change > tracking systems with only a minimal amount of integration. It requires > manual effort. > > This I 100% agree with. But then, even before GitHub, Jira and SVN were > two separate systems, and we managed fine. Maybe, as you say, the committer > mentality around attention to detail has laxed. Or maybe, as I suggest, > there's just an education gap involving a couple people, which can be > addressed. > > > We do not have a single source of truth as to what are active branches > and what was the most recent release by version number made from each > branch. > > Ah, now the discussion gets interesting! IMHO, we do have a source of > truth re: this information -- it's git. > > > Perhaps something can be created that takes the union of git branch > listing and reporter.a.o release version history. > > I did create such a tool, it scrapes git and Jira and produces a database > containing this information, from which reports are generated and ad-hoc > queries are issued. I committed it (nearly a year ago to the day) via > HBASE-22853 (Viraj, you signed off on this commit...). As I mentioned > earlier, it's still young, but it was invaluable for me in establishing a > baseline for 2.3.0, and includes a readme with lots of examples. > > I am of the opinion that the git repository is the authority on what is in > a release. After all, a release is defined as a permanent "rel" tag in git > (plus some ASF ceremony). IMHO, Jira needs to follow git, not the other way > around. Likewise, tracking the CHANGES file in git -- a file generated from > a Jira query -- is very silly. Instead, I propose that the changes file be > generated on demand, perhaps as part building the tarball. The only > alternative, if you want to retain the CHANGES file in git, is to include > an update to the CHANGES file with every git commit, which I think is > completely untenable. > > One other stumbling point is that our release notes are generated from a > field in Jira. We could generate release notes from git commit messages, > but I think that is an unrealistic burden to place on committers at the > time of commit. > > Following the logic that git is the source of truth, I propose a change to > our release generation process so that it looks something like > 1. set the version number in poms, commit. > 2. apply the RC tag. > 3. run the audit tool to generate a report. > 4. resolve inconsistencies (manually for now, eventually the tool could > update Jira -- it has an API) > 5. build artifacts, including generating any files from metadata in Jira. > > Step 3, when automated, can even be run during our nightly branch builds. > At that point, a committer doesn't need to worry at all about a fixVersion, > they just commit to their branches and automation takes care of the rest. > > The above has the issue around when to close a ticket. IMHO, a ticket > should be closed as soon as there's a commit to any branch, but maybe there > are other complexities to consider. > > Thanks for bringing this up, Andrew. It's bothered me since 2.3.0 :) > > -n >
Re: Project management (JIRA fix version tracking) is in crisis
> I have now had to sink two 2.4.1 RCs because of errors in the change log. IIRC, it's a relatively new phenomenon that we sink an RC because the CHANGES file is incomplete. > That's right... Commtters cannot be trusted to correctly maintain issue metadata in JIRA. I wouldn't go so far as to make a blanket statement such as this... I suspect there's a couple new people who missed this message. In my experience, a polite nudge from the RM on the affected JIRA issue to the offending committer resolves the knowledge gap. > I think the root cause is GitHub and JIRA are two separate change tracking systems with only a minimal amount of integration. It requires manual effort. This I 100% agree with. But then, even before GitHub, Jira and SVN were two separate systems, and we managed fine. Maybe, as you say, the committer mentality around attention to detail has laxed. Or maybe, as I suggest, there's just an education gap involving a couple people, which can be addressed. > We do not have a single source of truth as to what are active branches and what was the most recent release by version number made from each branch. Ah, now the discussion gets interesting! IMHO, we do have a source of truth re: this information -- it's git. > Perhaps something can be created that takes the union of git branch listing and reporter.a.o release version history. I did create such a tool, it scrapes git and Jira and produces a database containing this information, from which reports are generated and ad-hoc queries are issued. I committed it (nearly a year ago to the day) via HBASE-22853 (Viraj, you signed off on this commit...). As I mentioned earlier, it's still young, but it was invaluable for me in establishing a baseline for 2.3.0, and includes a readme with lots of examples. I am of the opinion that the git repository is the authority on what is in a release. After all, a release is defined as a permanent "rel" tag in git (plus some ASF ceremony). IMHO, Jira needs to follow git, not the other way around. Likewise, tracking the CHANGES file in git -- a file generated from a Jira query -- is very silly. Instead, I propose that the changes file be generated on demand, perhaps as part building the tarball. The only alternative, if you want to retain the CHANGES file in git, is to include an update to the CHANGES file with every git commit, which I think is completely untenable. One other stumbling point is that our release notes are generated from a field in Jira. We could generate release notes from git commit messages, but I think that is an unrealistic burden to place on committers at the time of commit. Following the logic that git is the source of truth, I propose a change to our release generation process so that it looks something like 1. set the version number in poms, commit. 2. apply the RC tag. 3. run the audit tool to generate a report. 4. resolve inconsistencies (manually for now, eventually the tool could update Jira -- it has an API) 5. build artifacts, including generating any files from metadata in Jira. Step 3, when automated, can even be run during our nightly branch builds. At that point, a committer doesn't need to worry at all about a fixVersion, they just commit to their branches and automation takes care of the rest. The above has the issue around when to close a ticket. IMHO, a ticket should be closed as soon as there's a commit to any branch, but maybe there are other complexities to consider. Thanks for bringing this up, Andrew. It's bothered me since 2.3.0 :) -n
Re: Project management (JIRA fix version tracking) is in crisis
Regarding a tool, I’ve already committed one that I used for 2.3.0. IIRC, Busbey suggested on its original PR the idea of using it during our nightlies as well. I’m pretty sure I’ve mentioned it explicit already to Viraj, Andrew, and Huaxiang. Check dev-support/git-jira-release-audit on master. There’s a readme in the directory. Of course, if Viraj’s tool is better, let’s move to that one. But please, take a look here first. I found it very useful for the new minor branch scenario with 100’s of commits that Andrew mentioned earlier in this thread. On Fri, Jan 15, 2021 at 11:20 Viraj Jasani wrote: > Thanks Andrew and Zach. > The script tries to identify these issues: > > 1. commit is reverted as per commit message > 2. commit does not contain Jira number format HBASE- in message > 3. Jira does not have expected fixVersion > 4. Jira has expected fixVersion, but it is not yet resolved > 5. Jiras that are marked resolved with given fixVersion but have no > corresponding > commit present > > IMHO, issues like > 1. reverted commit and relevant Jira being resolved with > expected fixVersion which it is actually reverted but Jira is > not reopened > 2. some false alarm like preparation of snapshot version > which is not tied to any Jira > > might make it bit difficult to automate this script with > create-release or nightly build because these kind of discrepancies > are for committers/RMs to manually check and resolve. > Having said that, I am sure we can make something better and > meaningful out of this that will make our life easier going forward. > > > On 2021/01/15 19:07:35, Zach York wrote: > > Awesome Viraj! Also if we have confidence in the script we could probably > > add it to run during a nightly run to try to catch issues earlier and > > remove the burden on the RM (if there isn't too much noise from open > > issues). > > > > On Fri, Jan 15, 2021 at 10:55 AM Andrew Purtell > wrote: > > > > > Thanks, Viraj. A script like that will be super helpful. > > > > > > On Fri, Jan 15, 2021 at 10:50 AM Viraj Jasani > wrote: > > > > > > > > This is not something RMs have had to do > > > > > in the past and it asks a lot of them. > > > > > > > > +1. It is too much for RM to do if RM alone is responsible for > > > > maintaining Git/Jira in sync. > > > > > > > > The reason why I was able to get the diffs for 2.3.4 and 2.4.1 > > > > RCs is because I used a small script that a) goes through all > > > > commits, b) gets Jira number, c) search for the Jira fields using > it's > > > > API and d) finds the diff. And similarly using JQL, it finds all > Jiras > > > > with expected fixVersion but has no git commit present. Mostly I > > > > will raise PR for this script tomorrow early as part of HBASE-25514. > > > > > > > > If we have enough trust on this, when RM given heads up on > > > > dev mailing list for a new release, more committers can be > > > > encouraged to start using this script to find any discrepancy, > resolve > > > > it if possible and keep in mind the upcoming release version while > > > > resolving a Jira. > > > > However, this is just a script that can help specifically just before > > > > spinning RCs. More precautionary steps would be bringing > > > > awareness of all active branches and how they relate to active > > > > releases as mentioned by Andrew. > > > > > > > > > > > > On 2021/01/15 18:39:18, Andrew Purtell wrote: > > > > > We do not have a single source of truth as to what are active > branches > > > > and > > > > > what was the most recent release by version number made from each > > > branch. > > > > > If we had such a resource, then committers could refer to it > whenever > > > > > merging PRs or cherry picking commits backward. Perhaps something > can > > > be > > > > > created that takes the union of git branch listing and reporter.a.o > > > > release > > > > > version history. The downside of course it this would be something > new > > > to > > > > > maintain. Committers would also need to be educated about the > existence > > > > of > > > > > this tool and the requirement to use it, but that could be solved > with > > > an > > > > > update to the How To Release section of the online book. I filed > > > > > HBASE-25515 for this idea. > > > > > > > > > > > > > > > > > > > > On Fri, Jan 15, 2021 at 10:05 AM Andrew Purtell < > apurt...@apache.org> > > > > wrote: > > > > > > > > > > > I have now had to sink two 2.4.1 RCs because of errors in the > change > > > > log. > > > > > > > > > > > > I made a pass over git history and ensured every commit was > > > included. I > > > > > > had also made a pass over JIRA to move out any unresolved issues > or > > > > > > complete the resolution of same. What I did not do is check that > > > every > > > > > > resolved JIRA corresponded to an actual commit. This is not > something > > > > RMs > > > > > > have had to do in the past and it asks a lot of them. > > > > > > > > > > > > I know NOW that as RM I cannot currently trust committers
Re: Project management (JIRA fix version tracking) is in crisis
Yeah, after multiple release efforts of 2.3.4 failed, I finally did a manual check. 1. got git logs for jiras from 2.3.3. 2. did a dry-run to generate CHANGES.md. I compared these two outputs and removed the common ones. The one left in 1) are issues in git and fixVersion not correctly setup in jira. The one left in 2) is fixVersion set to 2.3.4 but patch is not committed to the branch. The logic is essentially based on the fact that git is the only source of truth. On Fri, Jan 15, 2021 at 11:30 AM Nick Dimiduk wrote: > Hi Andrew, > > I had a conversation on this same topic yesterday with Huaxiang and, I > believe, came to a similar conclusion. I was advocating that git be the > source of truth and Jira follow, and there be no CHANGES file in the > repository at all. > > I’m on mobile at the moment. Let me spend some time this afternoon to catch > up this thread and write up my own thoughts. I actually planned to raise a > discuss thread today anyway. > > Thank you all for caring about this! > -n > > On Fri, Jan 15, 2021 at 11:20 Viraj Jasani wrote: > > > Thanks Andrew and Zach. > > The script tries to identify these issues: > > > > 1. commit is reverted as per commit message > > 2. commit does not contain Jira number format HBASE- in message > > 3. Jira does not have expected fixVersion > > 4. Jira has expected fixVersion, but it is not yet resolved > > 5. Jiras that are marked resolved with given fixVersion but have no > > corresponding > > commit present > > > > IMHO, issues like > > 1. reverted commit and relevant Jira being resolved with > > expected fixVersion which it is actually reverted but Jira is > > not reopened > > 2. some false alarm like preparation of snapshot version > > which is not tied to any Jira > > > > might make it bit difficult to automate this script with > > create-release or nightly build because these kind of discrepancies > > are for committers/RMs to manually check and resolve. > > Having said that, I am sure we can make something better and > > meaningful out of this that will make our life easier going forward. > > > > > > On 2021/01/15 19:07:35, Zach York wrote: > > > Awesome Viraj! Also if we have confidence in the script we could > probably > > > add it to run during a nightly run to try to catch issues earlier and > > > remove the burden on the RM (if there isn't too much noise from open > > > issues). > > > > > > On Fri, Jan 15, 2021 at 10:55 AM Andrew Purtell > > wrote: > > > > > > > Thanks, Viraj. A script like that will be super helpful. > > > > > > > > On Fri, Jan 15, 2021 at 10:50 AM Viraj Jasani > > wrote: > > > > > > > > > > This is not something RMs have had to do > > > > > > in the past and it asks a lot of them. > > > > > > > > > > +1. It is too much for RM to do if RM alone is responsible for > > > > > maintaining Git/Jira in sync. > > > > > > > > > > The reason why I was able to get the diffs for 2.3.4 and 2.4.1 > > > > > RCs is because I used a small script that a) goes through all > > > > > commits, b) gets Jira number, c) search for the Jira fields using > > it's > > > > > API and d) finds the diff. And similarly using JQL, it finds all > > Jiras > > > > > with expected fixVersion but has no git commit present. Mostly I > > > > > will raise PR for this script tomorrow early as part of > HBASE-25514. > > > > > > > > > > If we have enough trust on this, when RM given heads up on > > > > > dev mailing list for a new release, more committers can be > > > > > encouraged to start using this script to find any discrepancy, > > resolve > > > > > it if possible and keep in mind the upcoming release version while > > > > > resolving a Jira. > > > > > However, this is just a script that can help specifically just > before > > > > > spinning RCs. More precautionary steps would be bringing > > > > > awareness of all active branches and how they relate to active > > > > > releases as mentioned by Andrew. > > > > > > > > > > > > > > > On 2021/01/15 18:39:18, Andrew Purtell > wrote: > > > > > > We do not have a single source of truth as to what are active > > branches > > > > > and > > > > > > what was the most recent release by version number made from each > > > > branch. > > > > > > If we had such a resource, then committers could refer to it > > whenever > > > > > > merging PRs or cherry picking commits backward. Perhaps something > > can > > > > be > > > > > > created that takes the union of git branch listing and > reporter.a.o > > > > > release > > > > > > version history. The downside of course it this would be > something > > new > > > > to > > > > > > maintain. Committers would also need to be educated about the > > existence > > > > > of > > > > > > this tool and the requirement to use it, but that could be solved > > with > > > > an > > > > > > update to the How To Release section of the online book. I filed > > > > > > HBASE-25515 for this idea. > > > > > > > > > > > > > > > > > > > > > > > > On Fri, Jan 15, 2021 at 10:05 AM Andrew
Re: Project management (JIRA fix version tracking) is in crisis
Hi Andrew, I had a conversation on this same topic yesterday with Huaxiang and, I believe, came to a similar conclusion. I was advocating that git be the source of truth and Jira follow, and there be no CHANGES file in the repository at all. I’m on mobile at the moment. Let me spend some time this afternoon to catch up this thread and write up my own thoughts. I actually planned to raise a discuss thread today anyway. Thank you all for caring about this! -n On Fri, Jan 15, 2021 at 11:20 Viraj Jasani wrote: > Thanks Andrew and Zach. > The script tries to identify these issues: > > 1. commit is reverted as per commit message > 2. commit does not contain Jira number format HBASE- in message > 3. Jira does not have expected fixVersion > 4. Jira has expected fixVersion, but it is not yet resolved > 5. Jiras that are marked resolved with given fixVersion but have no > corresponding > commit present > > IMHO, issues like > 1. reverted commit and relevant Jira being resolved with > expected fixVersion which it is actually reverted but Jira is > not reopened > 2. some false alarm like preparation of snapshot version > which is not tied to any Jira > > might make it bit difficult to automate this script with > create-release or nightly build because these kind of discrepancies > are for committers/RMs to manually check and resolve. > Having said that, I am sure we can make something better and > meaningful out of this that will make our life easier going forward. > > > On 2021/01/15 19:07:35, Zach York wrote: > > Awesome Viraj! Also if we have confidence in the script we could probably > > add it to run during a nightly run to try to catch issues earlier and > > remove the burden on the RM (if there isn't too much noise from open > > issues). > > > > On Fri, Jan 15, 2021 at 10:55 AM Andrew Purtell > wrote: > > > > > Thanks, Viraj. A script like that will be super helpful. > > > > > > On Fri, Jan 15, 2021 at 10:50 AM Viraj Jasani > wrote: > > > > > > > > This is not something RMs have had to do > > > > > in the past and it asks a lot of them. > > > > > > > > +1. It is too much for RM to do if RM alone is responsible for > > > > maintaining Git/Jira in sync. > > > > > > > > The reason why I was able to get the diffs for 2.3.4 and 2.4.1 > > > > RCs is because I used a small script that a) goes through all > > > > commits, b) gets Jira number, c) search for the Jira fields using > it's > > > > API and d) finds the diff. And similarly using JQL, it finds all > Jiras > > > > with expected fixVersion but has no git commit present. Mostly I > > > > will raise PR for this script tomorrow early as part of HBASE-25514. > > > > > > > > If we have enough trust on this, when RM given heads up on > > > > dev mailing list for a new release, more committers can be > > > > encouraged to start using this script to find any discrepancy, > resolve > > > > it if possible and keep in mind the upcoming release version while > > > > resolving a Jira. > > > > However, this is just a script that can help specifically just before > > > > spinning RCs. More precautionary steps would be bringing > > > > awareness of all active branches and how they relate to active > > > > releases as mentioned by Andrew. > > > > > > > > > > > > On 2021/01/15 18:39:18, Andrew Purtell wrote: > > > > > We do not have a single source of truth as to what are active > branches > > > > and > > > > > what was the most recent release by version number made from each > > > branch. > > > > > If we had such a resource, then committers could refer to it > whenever > > > > > merging PRs or cherry picking commits backward. Perhaps something > can > > > be > > > > > created that takes the union of git branch listing and reporter.a.o > > > > release > > > > > version history. The downside of course it this would be something > new > > > to > > > > > maintain. Committers would also need to be educated about the > existence > > > > of > > > > > this tool and the requirement to use it, but that could be solved > with > > > an > > > > > update to the How To Release section of the online book. I filed > > > > > HBASE-25515 for this idea. > > > > > > > > > > > > > > > > > > > > On Fri, Jan 15, 2021 at 10:05 AM Andrew Purtell < > apurt...@apache.org> > > > > wrote: > > > > > > > > > > > I have now had to sink two 2.4.1 RCs because of errors in the > change > > > > log. > > > > > > > > > > > > I made a pass over git history and ensured every commit was > > > included. I > > > > > > had also made a pass over JIRA to move out any unresolved issues > or > > > > > > complete the resolution of same. What I did not do is check that > > > every > > > > > > resolved JIRA corresponded to an actual commit. This is not > something > > > > RMs > > > > > > have had to do in the past and it asks a lot of them. > > > > > > > > > > > > I know NOW that as RM I cannot currently trust committers to get > fix > > > > > > versions right or care about this. > > > > > > > > > > > > That's
Re: Project management (JIRA fix version tracking) is in crisis
Thanks Andrew and Zach. The script tries to identify these issues: 1. commit is reverted as per commit message 2. commit does not contain Jira number format HBASE- in message 3. Jira does not have expected fixVersion 4. Jira has expected fixVersion, but it is not yet resolved 5. Jiras that are marked resolved with given fixVersion but have no corresponding commit present IMHO, issues like 1. reverted commit and relevant Jira being resolved with expected fixVersion which it is actually reverted but Jira is not reopened 2. some false alarm like preparation of snapshot version which is not tied to any Jira might make it bit difficult to automate this script with create-release or nightly build because these kind of discrepancies are for committers/RMs to manually check and resolve. Having said that, I am sure we can make something better and meaningful out of this that will make our life easier going forward. On 2021/01/15 19:07:35, Zach York wrote: > Awesome Viraj! Also if we have confidence in the script we could probably > add it to run during a nightly run to try to catch issues earlier and > remove the burden on the RM (if there isn't too much noise from open > issues). > > On Fri, Jan 15, 2021 at 10:55 AM Andrew Purtell wrote: > > > Thanks, Viraj. A script like that will be super helpful. > > > > On Fri, Jan 15, 2021 at 10:50 AM Viraj Jasani wrote: > > > > > > This is not something RMs have had to do > > > > in the past and it asks a lot of them. > > > > > > +1. It is too much for RM to do if RM alone is responsible for > > > maintaining Git/Jira in sync. > > > > > > The reason why I was able to get the diffs for 2.3.4 and 2.4.1 > > > RCs is because I used a small script that a) goes through all > > > commits, b) gets Jira number, c) search for the Jira fields using it's > > > API and d) finds the diff. And similarly using JQL, it finds all Jiras > > > with expected fixVersion but has no git commit present. Mostly I > > > will raise PR for this script tomorrow early as part of HBASE-25514. > > > > > > If we have enough trust on this, when RM given heads up on > > > dev mailing list for a new release, more committers can be > > > encouraged to start using this script to find any discrepancy, resolve > > > it if possible and keep in mind the upcoming release version while > > > resolving a Jira. > > > However, this is just a script that can help specifically just before > > > spinning RCs. More precautionary steps would be bringing > > > awareness of all active branches and how they relate to active > > > releases as mentioned by Andrew. > > > > > > > > > On 2021/01/15 18:39:18, Andrew Purtell wrote: > > > > We do not have a single source of truth as to what are active branches > > > and > > > > what was the most recent release by version number made from each > > branch. > > > > If we had such a resource, then committers could refer to it whenever > > > > merging PRs or cherry picking commits backward. Perhaps something can > > be > > > > created that takes the union of git branch listing and reporter.a.o > > > release > > > > version history. The downside of course it this would be something new > > to > > > > maintain. Committers would also need to be educated about the existence > > > of > > > > this tool and the requirement to use it, but that could be solved with > > an > > > > update to the How To Release section of the online book. I filed > > > > HBASE-25515 for this idea. > > > > > > > > > > > > > > > > On Fri, Jan 15, 2021 at 10:05 AM Andrew Purtell > > > wrote: > > > > > > > > > I have now had to sink two 2.4.1 RCs because of errors in the change > > > log. > > > > > > > > > > I made a pass over git history and ensured every commit was > > included. I > > > > > had also made a pass over JIRA to move out any unresolved issues or > > > > > complete the resolution of same. What I did not do is check that > > every > > > > > resolved JIRA corresponded to an actual commit. This is not something > > > RMs > > > > > have had to do in the past and it asks a lot of them. > > > > > > > > > > I know NOW that as RM I cannot currently trust committers to get fix > > > > > versions right or care about this. > > > > > > > > > > That's right... Commtters cannot be trusted to correctly maintain > > issue > > > > > metadata in JIRA. > > > > > > > > > > That is not a good situation for the project to be in. Up until now > > it > > > has > > > > > not been the responsibility of the RM to check each and every JIRA > > > status. > > > > > It has been the collective responsibility of committers to care about > > > the > > > > > project's release tracking insofar as to correctly update fix > > versions > > > in > > > > > JIRA. For releases containing relatively few changes, like 2.4.1, > > with > > > ~50 > > > > > changes, I suppose it is possible for the RM to remove all 2.4.1 fix > > > > > versions, walk the commit history, and set back fix versions on JIRA > > to > > > > > actually
Re: Project management (JIRA fix version tracking) is in crisis
Awesome Viraj! Also if we have confidence in the script we could probably add it to run during a nightly run to try to catch issues earlier and remove the burden on the RM (if there isn't too much noise from open issues). On Fri, Jan 15, 2021 at 10:55 AM Andrew Purtell wrote: > Thanks, Viraj. A script like that will be super helpful. > > On Fri, Jan 15, 2021 at 10:50 AM Viraj Jasani wrote: > > > > This is not something RMs have had to do > > > in the past and it asks a lot of them. > > > > +1. It is too much for RM to do if RM alone is responsible for > > maintaining Git/Jira in sync. > > > > The reason why I was able to get the diffs for 2.3.4 and 2.4.1 > > RCs is because I used a small script that a) goes through all > > commits, b) gets Jira number, c) search for the Jira fields using it's > > API and d) finds the diff. And similarly using JQL, it finds all Jiras > > with expected fixVersion but has no git commit present. Mostly I > > will raise PR for this script tomorrow early as part of HBASE-25514. > > > > If we have enough trust on this, when RM given heads up on > > dev mailing list for a new release, more committers can be > > encouraged to start using this script to find any discrepancy, resolve > > it if possible and keep in mind the upcoming release version while > > resolving a Jira. > > However, this is just a script that can help specifically just before > > spinning RCs. More precautionary steps would be bringing > > awareness of all active branches and how they relate to active > > releases as mentioned by Andrew. > > > > > > On 2021/01/15 18:39:18, Andrew Purtell wrote: > > > We do not have a single source of truth as to what are active branches > > and > > > what was the most recent release by version number made from each > branch. > > > If we had such a resource, then committers could refer to it whenever > > > merging PRs or cherry picking commits backward. Perhaps something can > be > > > created that takes the union of git branch listing and reporter.a.o > > release > > > version history. The downside of course it this would be something new > to > > > maintain. Committers would also need to be educated about the existence > > of > > > this tool and the requirement to use it, but that could be solved with > an > > > update to the How To Release section of the online book. I filed > > > HBASE-25515 for this idea. > > > > > > > > > > > > On Fri, Jan 15, 2021 at 10:05 AM Andrew Purtell > > wrote: > > > > > > > I have now had to sink two 2.4.1 RCs because of errors in the change > > log. > > > > > > > > I made a pass over git history and ensured every commit was > included. I > > > > had also made a pass over JIRA to move out any unresolved issues or > > > > complete the resolution of same. What I did not do is check that > every > > > > resolved JIRA corresponded to an actual commit. This is not something > > RMs > > > > have had to do in the past and it asks a lot of them. > > > > > > > > I know NOW that as RM I cannot currently trust committers to get fix > > > > versions right or care about this. > > > > > > > > That's right... Commtters cannot be trusted to correctly maintain > issue > > > > metadata in JIRA. > > > > > > > > That is not a good situation for the project to be in. Up until now > it > > has > > > > not been the responsibility of the RM to check each and every JIRA > > status. > > > > It has been the collective responsibility of committers to care about > > the > > > > project's release tracking insofar as to correctly update fix > versions > > in > > > > JIRA. For releases containing relatively few changes, like 2.4.1, > with > > ~50 > > > > changes, I suppose it is possible for the RM to remove all 2.4.1 fix > > > > versions, walk the commit history, and set back fix versions on JIRA > to > > > > actually correspond with what was truly committed. However, for minor > > > > releases, with hundreds of commits, this will not be possible. > > > > > > > > I think the root cause is GitHub and JIRA are two separate change > > tracking > > > > systems with only a minimal amount of integration. It requires manual > > > > effort. More and more, new committers are familiar with GitHub and > PRs > > and > > > > are not familiar with JIRA and the Apache way of using JIRA to build > > change > > > > logs. We need to better educate new and existing committers on their > > > > responsibilities with regards to maintaining JIRA metadata correctly. > > > > > > > > -- > > > > Best regards, > > > > Andrew > > > > > > > > Words like orphans lost among the crosstalk, meaning torn from > truth's > > > > decrepit hands > > > >- A23, Crosstalk > > > > > > > > > > > > > -- > > > Best regards, > > > Andrew > > > > > > Words like orphans lost among the crosstalk, meaning torn from truth's > > > decrepit hands > > >- A23, Crosstalk > > > > > > > > -- > Best regards, > Andrew > > Words like orphans lost among the crosstalk, meaning torn from truth's > decrepit hands >- A23,
Re: Project management (JIRA fix version tracking) is in crisis
Thanks, Viraj. A script like that will be super helpful. On Fri, Jan 15, 2021 at 10:50 AM Viraj Jasani wrote: > > This is not something RMs have had to do > > in the past and it asks a lot of them. > > +1. It is too much for RM to do if RM alone is responsible for > maintaining Git/Jira in sync. > > The reason why I was able to get the diffs for 2.3.4 and 2.4.1 > RCs is because I used a small script that a) goes through all > commits, b) gets Jira number, c) search for the Jira fields using it's > API and d) finds the diff. And similarly using JQL, it finds all Jiras > with expected fixVersion but has no git commit present. Mostly I > will raise PR for this script tomorrow early as part of HBASE-25514. > > If we have enough trust on this, when RM given heads up on > dev mailing list for a new release, more committers can be > encouraged to start using this script to find any discrepancy, resolve > it if possible and keep in mind the upcoming release version while > resolving a Jira. > However, this is just a script that can help specifically just before > spinning RCs. More precautionary steps would be bringing > awareness of all active branches and how they relate to active > releases as mentioned by Andrew. > > > On 2021/01/15 18:39:18, Andrew Purtell wrote: > > We do not have a single source of truth as to what are active branches > and > > what was the most recent release by version number made from each branch. > > If we had such a resource, then committers could refer to it whenever > > merging PRs or cherry picking commits backward. Perhaps something can be > > created that takes the union of git branch listing and reporter.a.o > release > > version history. The downside of course it this would be something new to > > maintain. Committers would also need to be educated about the existence > of > > this tool and the requirement to use it, but that could be solved with an > > update to the How To Release section of the online book. I filed > > HBASE-25515 for this idea. > > > > > > > > On Fri, Jan 15, 2021 at 10:05 AM Andrew Purtell > wrote: > > > > > I have now had to sink two 2.4.1 RCs because of errors in the change > log. > > > > > > I made a pass over git history and ensured every commit was included. I > > > had also made a pass over JIRA to move out any unresolved issues or > > > complete the resolution of same. What I did not do is check that every > > > resolved JIRA corresponded to an actual commit. This is not something > RMs > > > have had to do in the past and it asks a lot of them. > > > > > > I know NOW that as RM I cannot currently trust committers to get fix > > > versions right or care about this. > > > > > > That's right... Commtters cannot be trusted to correctly maintain issue > > > metadata in JIRA. > > > > > > That is not a good situation for the project to be in. Up until now it > has > > > not been the responsibility of the RM to check each and every JIRA > status. > > > It has been the collective responsibility of committers to care about > the > > > project's release tracking insofar as to correctly update fix versions > in > > > JIRA. For releases containing relatively few changes, like 2.4.1, with > ~50 > > > changes, I suppose it is possible for the RM to remove all 2.4.1 fix > > > versions, walk the commit history, and set back fix versions on JIRA to > > > actually correspond with what was truly committed. However, for minor > > > releases, with hundreds of commits, this will not be possible. > > > > > > I think the root cause is GitHub and JIRA are two separate change > tracking > > > systems with only a minimal amount of integration. It requires manual > > > effort. More and more, new committers are familiar with GitHub and PRs > and > > > are not familiar with JIRA and the Apache way of using JIRA to build > change > > > logs. We need to better educate new and existing committers on their > > > responsibilities with regards to maintaining JIRA metadata correctly. > > > > > > -- > > > Best regards, > > > Andrew > > > > > > Words like orphans lost among the crosstalk, meaning torn from truth's > > > decrepit hands > > >- A23, Crosstalk > > > > > > > > > -- > > Best regards, > > Andrew > > > > Words like orphans lost among the crosstalk, meaning torn from truth's > > decrepit hands > >- A23, Crosstalk > > > -- Best regards, Andrew Words like orphans lost among the crosstalk, meaning torn from truth's decrepit hands - A23, Crosstalk
Re: Project management (JIRA fix version tracking) is in crisis
> This is not something RMs have had to do > in the past and it asks a lot of them. +1. It is too much for RM to do if RM alone is responsible for maintaining Git/Jira in sync. The reason why I was able to get the diffs for 2.3.4 and 2.4.1 RCs is because I used a small script that a) goes through all commits, b) gets Jira number, c) search for the Jira fields using it's API and d) finds the diff. And similarly using JQL, it finds all Jiras with expected fixVersion but has no git commit present. Mostly I will raise PR for this script tomorrow early as part of HBASE-25514. If we have enough trust on this, when RM given heads up on dev mailing list for a new release, more committers can be encouraged to start using this script to find any discrepancy, resolve it if possible and keep in mind the upcoming release version while resolving a Jira. However, this is just a script that can help specifically just before spinning RCs. More precautionary steps would be bringing awareness of all active branches and how they relate to active releases as mentioned by Andrew. On 2021/01/15 18:39:18, Andrew Purtell wrote: > We do not have a single source of truth as to what are active branches and > what was the most recent release by version number made from each branch. > If we had such a resource, then committers could refer to it whenever > merging PRs or cherry picking commits backward. Perhaps something can be > created that takes the union of git branch listing and reporter.a.o release > version history. The downside of course it this would be something new to > maintain. Committers would also need to be educated about the existence of > this tool and the requirement to use it, but that could be solved with an > update to the How To Release section of the online book. I filed > HBASE-25515 for this idea. > > > > On Fri, Jan 15, 2021 at 10:05 AM Andrew Purtell wrote: > > > I have now had to sink two 2.4.1 RCs because of errors in the change log. > > > > I made a pass over git history and ensured every commit was included. I > > had also made a pass over JIRA to move out any unresolved issues or > > complete the resolution of same. What I did not do is check that every > > resolved JIRA corresponded to an actual commit. This is not something RMs > > have had to do in the past and it asks a lot of them. > > > > I know NOW that as RM I cannot currently trust committers to get fix > > versions right or care about this. > > > > That's right... Commtters cannot be trusted to correctly maintain issue > > metadata in JIRA. > > > > That is not a good situation for the project to be in. Up until now it has > > not been the responsibility of the RM to check each and every JIRA status. > > It has been the collective responsibility of committers to care about the > > project's release tracking insofar as to correctly update fix versions in > > JIRA. For releases containing relatively few changes, like 2.4.1, with ~50 > > changes, I suppose it is possible for the RM to remove all 2.4.1 fix > > versions, walk the commit history, and set back fix versions on JIRA to > > actually correspond with what was truly committed. However, for minor > > releases, with hundreds of commits, this will not be possible. > > > > I think the root cause is GitHub and JIRA are two separate change tracking > > systems with only a minimal amount of integration. It requires manual > > effort. More and more, new committers are familiar with GitHub and PRs and > > are not familiar with JIRA and the Apache way of using JIRA to build change > > logs. We need to better educate new and existing committers on their > > responsibilities with regards to maintaining JIRA metadata correctly. > > > > -- > > Best regards, > > Andrew > > > > Words like orphans lost among the crosstalk, meaning torn from truth's > > decrepit hands > >- A23, Crosstalk > > > > > -- > Best regards, > Andrew > > Words like orphans lost among the crosstalk, meaning torn from truth's > decrepit hands >- A23, Crosstalk >
Re: Project management (JIRA fix version tracking) is in crisis
A related factor is the practice of holding a JIRA open until all backports are complete. This is something RMs will already be familiar with, so isn't more than an inconvenience, but it does lend itself to fix version inaccuracy. Say you merge a PR or commit a cherry pick to some of the active branches, but not all, and do not resolve the JIRA. If someone wants to release from one of the branches where the commit was committed, the change will not appear in the change log. The JIRA status for the issue must be set to resolved in order for the issue to be included in the change log. What I do as RM, and what I expect all RMs do, is run a report of all issues with a fix version of what I am about to release, and if any are in unresolved state, will resolve the JIRA at that time and advise further commits be made on subtasks or new issues. It would be better, when you are considering holding an issue open for a while for some backport to come in, to not hold the issue open, to resolve it at the time the first commits are complete. Open new issues for any additional backporting work. On Fri, Jan 15, 2021 at 10:39 AM Andrew Purtell wrote: > We do not have a single source of truth as to what are active branches and > what was the most recent release by version number made from each branch. > If we had such a resource, then committers could refer to it whenever > merging PRs or cherry picking commits backward. Perhaps something can be > created that takes the union of git branch listing and reporter.a.o release > version history. The downside of course it this would be something new to > maintain. Committers would also need to be educated about the existence of > this tool and the requirement to use it, but that could be solved with an > update to the How To Release section of the online book. I filed > HBASE-25515 for this idea. > > > > On Fri, Jan 15, 2021 at 10:05 AM Andrew Purtell > wrote: > >> I have now had to sink two 2.4.1 RCs because of errors in the change log. >> >> I made a pass over git history and ensured every commit was included. I >> had also made a pass over JIRA to move out any unresolved issues or >> complete the resolution of same. What I did not do is check that every >> resolved JIRA corresponded to an actual commit. This is not something RMs >> have had to do in the past and it asks a lot of them. >> >> I know NOW that as RM I cannot currently trust committers to get fix >> versions right or care about this. >> >> That's right... Commtters cannot be trusted to correctly maintain issue >> metadata in JIRA. >> >> That is not a good situation for the project to be in. Up until now it >> has not been the responsibility of the RM to check each and every JIRA >> status. It has been the collective responsibility of committers to care >> about the project's release tracking insofar as to correctly update fix >> versions in JIRA. For releases containing relatively few changes, like >> 2.4.1, with ~50 changes, I suppose it is possible for the RM to remove all >> 2.4.1 fix versions, walk the commit history, and set back fix versions on >> JIRA to actually correspond with what was truly committed. However, for >> minor releases, with hundreds of commits, this will not be possible. >> >> I think the root cause is GitHub and JIRA are two separate change >> tracking systems with only a minimal amount of integration. It requires >> manual effort. More and more, new committers are familiar with GitHub and >> PRs and are not familiar with JIRA and the Apache way of using JIRA to >> build change logs. We need to better educate new and existing committers on >> their responsibilities with regards to maintaining JIRA metadata correctly. >> >> -- >> Best regards, >> Andrew >> >> Words like orphans lost among the crosstalk, meaning torn from truth's >> decrepit hands >>- A23, Crosstalk >> > > > -- > Best regards, > Andrew > > Words like orphans lost among the crosstalk, meaning torn from truth's > decrepit hands >- A23, Crosstalk > -- Best regards, Andrew Words like orphans lost among the crosstalk, meaning torn from truth's decrepit hands - A23, Crosstalk
Re: Project management (JIRA fix version tracking) is in crisis
We do not have a single source of truth as to what are active branches and what was the most recent release by version number made from each branch. If we had such a resource, then committers could refer to it whenever merging PRs or cherry picking commits backward. Perhaps something can be created that takes the union of git branch listing and reporter.a.o release version history. The downside of course it this would be something new to maintain. Committers would also need to be educated about the existence of this tool and the requirement to use it, but that could be solved with an update to the How To Release section of the online book. I filed HBASE-25515 for this idea. On Fri, Jan 15, 2021 at 10:05 AM Andrew Purtell wrote: > I have now had to sink two 2.4.1 RCs because of errors in the change log. > > I made a pass over git history and ensured every commit was included. I > had also made a pass over JIRA to move out any unresolved issues or > complete the resolution of same. What I did not do is check that every > resolved JIRA corresponded to an actual commit. This is not something RMs > have had to do in the past and it asks a lot of them. > > I know NOW that as RM I cannot currently trust committers to get fix > versions right or care about this. > > That's right... Commtters cannot be trusted to correctly maintain issue > metadata in JIRA. > > That is not a good situation for the project to be in. Up until now it has > not been the responsibility of the RM to check each and every JIRA status. > It has been the collective responsibility of committers to care about the > project's release tracking insofar as to correctly update fix versions in > JIRA. For releases containing relatively few changes, like 2.4.1, with ~50 > changes, I suppose it is possible for the RM to remove all 2.4.1 fix > versions, walk the commit history, and set back fix versions on JIRA to > actually correspond with what was truly committed. However, for minor > releases, with hundreds of commits, this will not be possible. > > I think the root cause is GitHub and JIRA are two separate change tracking > systems with only a minimal amount of integration. It requires manual > effort. More and more, new committers are familiar with GitHub and PRs and > are not familiar with JIRA and the Apache way of using JIRA to build change > logs. We need to better educate new and existing committers on their > responsibilities with regards to maintaining JIRA metadata correctly. > > -- > Best regards, > Andrew > > Words like orphans lost among the crosstalk, meaning torn from truth's > decrepit hands >- A23, Crosstalk > -- Best regards, Andrew Words like orphans lost among the crosstalk, meaning torn from truth's decrepit hands - A23, Crosstalk
Project management (JIRA fix version tracking) is in crisis
I have now had to sink two 2.4.1 RCs because of errors in the change log. I made a pass over git history and ensured every commit was included. I had also made a pass over JIRA to move out any unresolved issues or complete the resolution of same. What I did not do is check that every resolved JIRA corresponded to an actual commit. This is not something RMs have had to do in the past and it asks a lot of them. I know NOW that as RM I cannot currently trust committers to get fix versions right or care about this. That's right... Commtters cannot be trusted to correctly maintain issue metadata in JIRA. That is not a good situation for the project to be in. Up until now it has not been the responsibility of the RM to check each and every JIRA status. It has been the collective responsibility of committers to care about the project's release tracking insofar as to correctly update fix versions in JIRA. For releases containing relatively few changes, like 2.4.1, with ~50 changes, I suppose it is possible for the RM to remove all 2.4.1 fix versions, walk the commit history, and set back fix versions on JIRA to actually correspond with what was truly committed. However, for minor releases, with hundreds of commits, this will not be possible. I think the root cause is GitHub and JIRA are two separate change tracking systems with only a minimal amount of integration. It requires manual effort. More and more, new committers are familiar with GitHub and PRs and are not familiar with JIRA and the Apache way of using JIRA to build change logs. We need to better educate new and existing committers on their responsibilities with regards to maintaining JIRA metadata correctly. -- Best regards, Andrew Words like orphans lost among the crosstalk, meaning torn from truth's decrepit hands - A23, Crosstalk
