On Sat, Sep 28, 2019 at 4:19 PM Sean Busbey <[email protected]> wrote:
> There's a lot more in dev-support than just the release tooling; we > can't delete the entirety of that folder from all branches currently. > Each branch needs to have several Jenkinsfiles for our jobs that use > the Jenkins Multibranch Pipeline for our various automated testing (I > think there are ~4 of them). Each branch also needs to have the > dockerfile needed for Yetus to run nightly and precommit against that > branch. > > Even with those things left in place it'll take a non-trivial amount > of effort to get the nightly jobs on all branches to work off of > support scripts out of the master branch. It's not clear to me that > anyone thus far in this thread has volunteered to do that work. > > Thanks for the reminder. For now will just move the create-release dir to master branch. Will delete from elsewhere. S > On Sat, Sep 28, 2019 at 6:06 PM Stack <[email protected]> wrote: > > > > On Sat, Sep 28, 2019 at 11:13 AM Nick Dimiduk <[email protected]> > wrote: > > > > > On Sat, Sep 28, 2019 at 10:58 Stack <[email protected]> wrote: > > > > > > > hbase-operator-tools/create-release or a new repo altogether? > > > > > > > > > > If we’re going to have one tool to release them all, I’m okay with it > > > staying in the main repo under dev-support or similar, so long as it’s > on > > > only one branch (probably master, maybe it’s own). If the tools is > itself > > > is going to be released and made installable (on a developer laptop or > a > > > Jenkins worker) then I think it should have its own repo. > > > > > > > > Sounds good. Lets start with master branch of core for now. Can do a > > dedicated repo later. I can delete the dev-tools from older branches (and > > from other repos where it exists) so only the one instance in one > location > > going forward. > > > > S > > > > > > > > > > On Wed, Sep 25, 2019 at 8:56 AM Peter Somogyi <[email protected]> > > > wrote: > > > > > > > > > > > > It is great that you brought up this topic Nick! I agree that the > > > > optimal > > > > > > solution would be to host all the release related scripts (RC > build, > > > > > > verifier) in a common place. > > > > > > > > > > > > The RC making scrip in hbase-operator-tools that Stack finetuned > is > > > > meant > > > > > > to work with different artifacts. The current version there > gives the > > > > RM > > > > > a > > > > > > smooth experience. It emerged from HBase's create-release script > and > > > > > > hopefully it can be used on other releases as well. There are > some > > > > > > constraints of the tool like Jira versions should use > > > > > > `hbase-operator-tools` prefix. > > > > > > > > > > > > > This is a good point. I wonder if `dev-support` should be > pruned or > > > > > purged > > > > > > from all branches other than master > > > > > > > > > > > > When we create branch-3 out of master then this becomes a problem > > > > again. > > > > > > Would it work if we use a specific branch for such tooling (e.g. > > > > > > dev-tools)? In this case RMs can just clone that branch and don't > > > need > > > > > the > > > > > > whole HBase repository on their local machine. > > > > > > > > > > > > On Thu, Sep 19, 2019 at 5:40 PM Nick Dimiduk < > [email protected]> > > > > > wrote: > > > > > > > > > > > > > On Wed, Sep 18, 2019 at 10:30 AM Sean Busbey < > [email protected]> > > > > > wrote: > > > > > > > > > > > > > > > If the tooling is in one place where will it live? > > > > > > > > > > > > > > > > As an RM I like not needing to checkout more then the repo > I'm > > > > > trying to > > > > > > > > get a release out on. > > > > > > > > > > > > > > > > > > > > > > My initial thinking was all would be in the main repo, but > this is > > > > > contrary > > > > > > > to your above statement. As I see it, everyone working on HBase > > > has a > > > > > > > checkout of the main repo handy, so for releases spun on > developer > > > > > machines > > > > > > > it's "no big deal." If we can ever get to releases spun through > > > > > automated > > > > > > > environments, it's all scripted anyway and thus "no big deal." > > > > > > > > > > > > > > In particular my release machine is slow on disk and so > updates to > > > > the > > > > > main > > > > > > > > git repo when trying to release something not in the main > repo > > > will > > > > > be > > > > > > > > painful. > > > > > > > > > > > > > > > > > > > > > "slow on disk" as in iops rate or "low on disk" as in capacity? > > > > Either > > > > > way, > > > > > > > I'm surprised to hear about this as a barrier. Is there a > > > > > side-conversation > > > > > > > we can have about trimming the fat out of our repo(s)? Some git > > > > > maintenance > > > > > > > that can/needs to be done? I was recently shocked by the girth > of > > > our > > > > > repos > > > > > > > -- nearly 0.5g on the main repo and a whopping 4g on the site! > > > > > > > > > > > > > > For the most part this is also why I usually manually build RCs > > > > > > > > that I run for the main repo, because I can do a shallow > clone of > > > > the > > > > > > > > release branch instead of needing to get updates to all the > > > active > > > > > > > > branches. > > > > > > > > > > > > > > > > > > > > > > This makes me sad. Automate more, not less! Altering the > automation > > > > to > > > > > make > > > > > > > shallow clones of targeted branches should be simple enough, > no? > > > > > > > > > > > > > > For testing RCs I guess it's currently all some combination of > the > > > > > > > > foundation policies that should be the same and maybe maven > > > > profiles? > > > > > > > > > > > > > > > > > > > > > > I agree that codification of foundation policy is the baseline. > > > That > > > > > would > > > > > > > be enough for me as a first pass. After that, perhaps layers of > > > > > increasing > > > > > > > sophistication, perhaps repo-dependent. I don't follow your > meaning > > > > re: > > > > > > > maven profiles. > > > > > > > > > > > > > > Iirc there was already some confusion about using the testing > > > script > > > > > that > > > > > > > > came in the main repo source bundle vs the one on master. > > > > > > > > > > > > > > > > > > > > > > This is a good point. I wonder if `dev-support` should be > pruned or > > > > > purged > > > > > > > from all branches other than master. Maybe the CI related stuff > > > > > branches > > > > > > > into it's own directory or directories, and we keep everything > else > > > > > limited > > > > > > > to a single canonical copy on master. This strategy would > eliminate > > > > > > > confusion re: what is authority in any given situation but > perhaps > > > is > > > > > too > > > > > > > constraining, given the number of active release branches we > > > > maintain. > > > > > > > > > > > > > > What issue are we trying to solve? > > > > > > > > > > > > > > > > > > > > > > Minimize contributor friction. Make it easy for every > subscriber to > > > > > dev@ > > > > > > > to > > > > > > > say "There's a new RC posted and I have an idle machine for an > > > hour / > > > > > for > > > > > > > the evening / for the weekend, let's just kick the tires." > Make it > > > > > easy for > > > > > > > everyone who's learned the RC tooling on one branch to > pinch-hit in > > > > on > > > > > > > another branch or another repo. I hear a constant complaint of > > > > > "scarcity of > > > > > > > volunteer hours" and "I wish we had more people voting in RCs" > and > > > "I > > > > > wish > > > > > > > we could keep a monthly release cadence on every branch we're > > > > > maintaining". > > > > > > > So I'd like to see a focused effort on maximizing the volunteer > > > human > > > > > and > > > > > > > machine time that's thrown our way. > > > > > > > > > > > > > > Thanks, > > > > > > > Nick > > > > > > > > > > > > > > On Wed, Sep 18, 2019, 11:50 Nick Dimiduk <[email protected]> > > > > wrote: > > > > > > > > > > > > > > > > > Heya, > > > > > > > > > > > > > > > > > > Looks like we have quite a few repos now, each of which > must > > > > > produce > > > > > > > > > artifacts that follow the Apache protocols. I also see we > have > > > > some > > > > > > > nice > > > > > > > > > tools built up in dev-support in the main repo for RM's who > > > build > > > > > > > release > > > > > > > > > candidates and community members to vote on them. I tried > to > > > use > > > > > our > > > > > > > > > hbase-vote.sh on this operator-tools RC and found it mostly > > > > works. > > > > > I > > > > > > > > think > > > > > > > > > a few adjustments on each end would allow these tools to > work > > > > > across > > > > > > > > repos, > > > > > > > > > so I filed HBASE-23048. However, I now see that > operator-tools > > > > has > > > > > its > > > > > > > > own > > > > > > > > > dev-support/create-release, so I wonder what direction we > want > > > to > > > > > take > > > > > > > > with > > > > > > > > > this automation, so here I come to the list. > > > > > > > > > > > > > > > > > > Do we want to have independent tooling for each repo? Are > the > > > > > processes > > > > > > > > of > > > > > > > > > building RC's different enough to warrant separate tools? > Is a > > > > > single > > > > > > > > tool > > > > > > > > > that can build an RC for all of them not worth the > trouble? At > > > > the > > > > > very > > > > > > > > > least, I think the hbase-vote.sh can be made to work with > > > > releases > > > > > > > > > generated from each of the repos, as it's not doing all > that > > > > much. > > > > > > > > > > > > > > > > > > Thoughts? > > > > > > > > > > > > > > > > > > Thanks, > > > > > > > > > Nick > > > > > > > > > > > > > > > > > > On Wed, Sep 18, 2019 at 9:42 AM Nick Dimiduk < > > > > [email protected]> > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > +1 > > > > > > > > > > > > > > > > > > > > - Verified signatures. > > > > > > > > > > - Verified checksums. > > > > > > > > > > - Built from source tarball successfully. > > > > > > > > > > - Ran unit tests from source tarball, pass. > > > > > > > > > > - Ran rat check from source tarball, pass. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Mon, Sep 16, 2019 at 11:31 AM Peter Somogyi < > > > > > [email protected]> > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > >> Please vote on this Apache HBase Operator Tools Release > > > > > Candidate > > > > > > > > (RC), > > > > > > > > > >> 1.0.0. > > > > > > > > > >> > > > > > > > > > >> The VOTE will remain open for at least 72 hours. > > > > > > > > > >> > > > > > > > > > >> [ ] +1 Release this package as Apache HBase Operator > Tools > > > > 1.0.0 > > > > > > > > > >> [ ] -1 Do not release this package because ... > > > > > > > > > >> > > > > > > > > > >> The tag to be voted on is 1.0.0RC0: > > > > > > > > > >> > > > > > > > > > >> > > > https://github.com/apache/hbase-operator-tools/tree/1.0.0RC0 > > > > > > > > > >> > > > > > > > > > >> The release files, including signatures, digests, as > well as > > > > > > > > CHANGES.md > > > > > > > > > >> and RELEASENOTES.md included in this RC can be found at: > > > > > > > > > >> > > > > > > > > > >> https://dist.apache.org/repos/dist/dev/hbase/1.0.0RC0/ > > > > > > > > > >> > > > > > > > > > >> Maven artifacts are available in a staging repository > at: > > > > > > > > > >> > > > > > > > > > >> > > > > > > > > > > > > > > > > > > > > > > > > > https://repository.apache.org/content/repositories/orgapachehbase-1348/ > > > > > > > > > >> > > > > > > > > > >> Artifacts were signed with the [email protected] key > > > which > > > > > can be > > > > > > > > > found > > > > > > > > > >> in: > > > > > > > > > >> > > > > > > > > > >> https://dist.apache.org/repos/dist/release/hbase/KEYS > > > > > > > > > >> > > > > > > > > > >> To learn more about Apache HBase Operator Tools, please > see > > > > > > > > > >> http://hbase.apache.org/ > > > > > > > > > >> > > > > > > > > > >> Thanks, > > > > > > > > > >> Your HBase Release Manager > > > > > > > > > >> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > >
