Hi Nick, I support the version naming in Jira as hbase-kustomize- and the initial version as 0.1.
- Peter On Wed, Jun 14, 2023 at 3:35 PM Nick Dimiduk <[email protected]> wrote: > Heya, > > Now that we have this repo, we need to establish some basic release > processes around it. I propose: > - We need a new version numbering scheme for releases from this > repository. For hbase-third-party, we have simply "thirdparty-". For all > the other repos, we use the entire repository name as the release version > prefix. I propose that we adopt the prefix "hbase-kustomize-" for this > repository's releases. > - We need a release scheme for this repository. We also need to define what > a release means in terms of compatibility guidelines. This is an > orchestration definition -- essentially, a data structure that represents > the serialized state of a Kubernetes cluster's API -- so its definitions of > compatibility will likely deviate a bit from our existing definitions. > Until we've sorted this out (a dedicated DISCUSS thread, once someone can > make a proposal), I propose we use a simple incrementing 0.x versioning > scheme for any release tags we produce. > > Based on these first two points, I propose that we start with an initial > JIRA fixVersion of "hbase-kustomize-0.1". > > (As a separate topic, we may also want to unify this version number prefix > across our repositories, by using "hbase-thirdparty-" as that repo's > version prefix going forward.) > > What do you think? > > Thanks, > Nick > > On Mon, Jun 12, 2023 at 7:18 PM Nick Dimiduk <[email protected]> wrote: > > > I’ve created this repository. > > > > Unless there is an objection, I’ll push an initial commit with a LICENSE > > file so that there’s a baseline to make PRs against. > > > > Thanks, > > Nick > > > > On Fri, 9 Jun 2023 at 18:30, Nick Dimiduk <[email protected]> wrote: > > > >> Thanks everyone who has replied. > >> > >> If there are no further concerns raised over the weekend, I’ll get > >> started on Monday with Infrastructure to provision the new repository. > >> > >> Thanks, > >> Nick > >> > >> On Fri, 9 Jun 2023 at 18:08, Peter Somogyi <[email protected]> wrote: > >> > >>> +1 on the new repository. > >>> > >>> On Wed, Jun 7, 2023 at 5:33 PM Viraj Jasani <[email protected]> > wrote: > >>> > >>> > +1 for hbase-kustomize > >>> > > >>> > On Mon, Jun 5, 2023 at 4:44 AM Nick Dimiduk <[email protected]> > >>> wrote: > >>> > > >>> > > Hello, > >>> > > > >>> > > As I work through the integration of these kustomize definitions > >>> into the > >>> > > existing java project structure that is hbase-operator-tools, I'm > >>> > > increasingly of the opinion that this is too much of a clash of > >>> > concerns. I > >>> > > think that this contribution would make better sense as its own > >>> > repository > >>> > > with its own release cycle. I'm neither aware of nor can I imagine > a > >>> > > technical coupling between the kustomize resources and the rest of > >>> the > >>> > > utilities in operator-tools. Likewise, this change set introduces > new > >>> > > requirements (docker, buildx, KinD and/or minikube) to the build > and > >>> test > >>> > > environment that are not otherwise needed by operator-tools. > >>> > > > >>> > > What do you think? SHould we request a new repository for the > >>> > > kustomize files? I propose hbase-kustomize.git. > >>> > > > >>> > > Thanks, > >>> > > Nick > >>> > > > >>> > > On Mon, May 22, 2023 at 4:33 PM Nick Dimiduk <[email protected]> > >>> > wrote: > >>> > > > >>> > > > I went ahead and rebuilt the Hadoop image module in the same > >>> style. I > >>> > > > rebased the zookeeper-single and hdfs kustimize implementations > >>> onto > >>> > the > >>> > > > same structure. So, PR’s #118, #119, #120, and #121 are all in > this > >>> > > style. > >>> > > > I don’t have a place for running integration tests, but unit > tests > >>> are > >>> > > now > >>> > > > running in Jenkins. > >>> > > > > >>> > > > I appreciate any attention you can provide. > >>> > > > > >>> > > > Thanks, > >>> > > > Nick > >>> > > > > >>> > > > On Mon, 22 May 2023 at 16:02, TAK-LON WU <[email protected]> > >>> wrote: > >>> > > > > >>> > > >> Sorry that I’m on vacation and will be back online after 06/06 , > >>> but > >>> > > >> thanks > >>> > > >> for putting the PR out and I believe someone on our side will > >>> review > >>> > it > >>> > > . > >>> > > >> ( > >>> > > >> or when I come back I will review them) > >>> > > >> > >>> > > >> -Stephen > >>> > > >> > >>> > > >> On Mon, May 22, 2023 at 6:08 PM Nick Dimiduk < > [email protected] > >>> > > >>> > > wrote: > >>> > > >> > >>> > > >> > Heya team, > >>> > > >> > > >>> > > >> > I have rebuilt one of the early PRs so that the docker image > >>> build > >>> > > >> pieces > >>> > > >> > are integrated with the maven build. If this is acceptable to > >>> the > >>> > > >> > reviewers, I'll go forward with integrating the other images > and > >>> > > >> > kustomize/kuttl tests in the same way. > >>> > > >> > > >>> > > >> > Please take a look. > >>> > > >> > > >>> > > >> > Thanks, > >>> > > >> > Nick > >>> > > >> > > >>> > > >> > https://github.com/apache/hbase-operator-tools/pull/118 > >>> > > >> > > >>> > > >> > On Fri, May 12, 2023 at 4:19 PM Nick Dimiduk < > >>> [email protected]> > >>> > > >> wrote: > >>> > > >> > > >>> > > >> > > Heya team, > >>> > > >> > > > >>> > > >> > > I have created individual pull-requests for each of the > major > >>> > > >> functional > >>> > > >> > > pieces outlined in the initial branch. These await review. > >>> > > >> > > > >>> > > >> > > I've now started working integrating the test harness into > the > >>> > maven > >>> > > >> > > build. After a brief detour for a Yetus plugin, I'm now > >>> looking > >>> > > >> instead > >>> > > >> > at > >>> > > >> > > maven integration via exec-maven-plugin. I'm also > >>> investigating > >>> > how > >>> > > to > >>> > > >> > pull > >>> > > >> > > the container image build up into maven as well. > >>> > > >> > > > >>> > > >> > > Thanks, > >>> > > >> > > Nic, > >>> > > >> > > > >>> > > >> > > On Thu, May 4, 2023 at 2:42 PM Nick Dimiduk < > >>> [email protected]> > >>> > > >> wrote: > >>> > > >> > > > >>> > > >> > >> Heya team, > >>> > > >> > >> > >>> > > >> > >> I've created a feature ticket [0] from which this > >>> contribution > >>> > can > >>> > > >> hang. > >>> > > >> > >> I've created an associated release version [1] and feature > >>> branch > >>> > > [2] > >>> > > >> > >> against which we can target PRs while things take shape. > I've > >>> > > >> published > >>> > > >> > my > >>> > > >> > >> initial extraction of this feature as a whole for your > >>> review [3] > >>> > > -- > >>> > > >> > take a > >>> > > >> > >> look at the big picture there. For each commit on that > >>> branch, > >>> > I've > >>> > > >> > created > >>> > > >> > >> a sub-task on HBASE-27827. Probably reviewers will find > other > >>> > items > >>> > > >> we > >>> > > >> > need > >>> > > >> > >> to peel off as sub-tasks. I'll start turning each of these > >>> > commits > >>> > > >> into > >>> > > >> > PRs > >>> > > >> > >> suitable for the Apache repo and your perusal. > >>> > > >> > >> > >>> > > >> > >> I think we're getting due for the 1.3 release of Operator > >>> Tools, > >>> > > so I > >>> > > >> > >> expect this will land in 1.4. > >>> > > >> > >> > >>> > > >> > >> Thanks, > >>> > > >> > >> Nick > >>> > > >> > >> > >>> > > >> > >> [0]: https://issues.apache.org/jira/browse/HBASE-27827 > >>> > > >> > >> [1]: > >>> > > >> > >>> https://issues.apache.org/jira/browse/HBASE/fixforversion/12353199 > >>> > > >> > >> [2]: > >>> > > >> > >> > >>> > > >> > > >>> > > >> > >>> > > > >>> > > >>> > https://github.com/apache/hbase-operator-tools/tree/HBASE-27827-kubernetes-deployment > >>> > > >> > >> [3]: > >>> > > >> > >> > >>> > > >> > > >>> > > >> > >>> > > > >>> > > >>> > https://github.com/apache/hbase-operator-tools/compare/HBASE-27827-kubernetes-deployment...ndimiduk:hbase-operator-tools:HBASE-27827-kubernetes-deployment > >>> > > >> > >> > >>> > > >> > >> > >>> > > >> > >> > >>> > > >> > >> On Mon, Mar 13, 2023 at 11:28 AM Nick Dimiduk < > >>> > [email protected] > >>> > > > > >>> > > >> > >> wrote: > >>> > > >> > >> > >>> > > >> > >>> Heya team, > >>> > > >> > >>> > >>> > > >> > >>> Over here at $dayjob, we have an increasing reliance on > >>> > Kubernetes > >>> > > >> for > >>> > > >> > >>> both development and production workloads. Our tools are > >>> > maturing > >>> > > >> and > >>> > > >> > >>> we're hoping that they might be of interest to the wider > >>> > > community. > >>> > > >> > >>> I'd like to see if there's community interest in receiving > >>> > > some/any > >>> > > >> of > >>> > > >> > >>> them as a contribution. I think we'll also need a plan > from > >>> ASF > >>> > > >> Infra > >>> > > >> > >>> that makes kubernetes available to us as a project. > >>> > > >> > >>> > >>> > > >> > >>> We have implemented a basic stack of tools for > >>> orchestrating ZK > >>> > + > >>> > > >> HDFS > >>> > > >> > >>> + HBase on Kubernetes. We use this for running a small > >>> local dev > >>> > > >> > >>> cluster via MiniKube/KIND ; for ITBLL on smallish > >>> distributed > >>> > > >> clusters > >>> > > >> > >>> in a public cloud ; and in production for running clusters > >>> of > >>> > ~100 > >>> > > >> > >>> Data Nodes/Region Servers in a public cloud. There was an > >>> > earlier > >>> > > >> > >>> discussion about using our donation of test hardware for > >>> running > >>> > > >> more > >>> > > >> > >>> thorough tests in our CI, but one of the limiting factors > is > >>> > full > >>> > > >> > >>> cluster deployment. I hope that the community might be > >>> > interested > >>> > > in > >>> > > >> > >>> receiving this tooling as a foundation for more rigorous > >>> > > correctness > >>> > > >> > >>> and maybe even performance tests in the open. Furthermore, > >>> > perhaps > >>> > > >> the > >>> > > >> > >>> wider community has interest in an Apache licensed cluster > >>> > > >> > >>> orchestration tool for other uses. > >>> > > >> > >>> > >>> > > >> > >>> Now for some details: The implementation is built on > >>> Kustomize, > >>> > so > >>> > > >> > >>> it's fundamentally transparent resource specification with > >>> yaml > >>> > > >> > >>> patches for composability; this is in contrast to a > solution > >>> > using > >>> > > >> > >>> templates with defined capabilities and interfaces. There > >>> is no > >>> > > >> > >>> operator ; it's all coordinated via init/bootstrap > >>> containers, > >>> > > shell > >>> > > >> > >>> scripts, shared volumes for state, &c. For now. > >>> > > >> > >>> > >>> > > >> > >>> Such a donation will amount to a code drop, which will > have > >>> its > >>> > > >> > >>> challenges. I'm motivated via internal processes to carve > it > >>> > into > >>> > > >> > >>> smaller pieces, and I think that will benefit community > >>> review > >>> > as > >>> > > >> > >>> well. Perhaps this approach could be used to make the > >>> > contribution > >>> > > >> via > >>> > > >> > >>> a feature branch. > >>> > > >> > >>> > >>> > > >> > >>> Is there community interest in adding such a capability to > >>> our > >>> > > >> > >>> maintained responsibilities? I'd hope that we have several > >>> > > >> volunteers > >>> > > >> > >>> to work with me through the contribution process, and who > >>> are > >>> > > >> > >>> reasonably confident that they'll be able to help maintain > >>> such > >>> > a > >>> > > >> > >>> capability going forward. We'll also need someone who can > >>> work > >>> > > with > >>> > > >> > >>> Infra to get us access to Kubernetes cluster(s), via > >>> whatever > >>> > > means. > >>> > > >> > >>> > >>> > > >> > >>> What do you think? > >>> > > >> > >>> > >>> > > >> > >>> Thanks, > >>> > > >> > >>> Nick & the HBase team at Apple > >>> > > >> > >>> > >>> > > >> > >> > >>> > > >> > > >>> > > >> > >>> > > > > >>> > > > >>> > > >>> > >> >
