I’ve created this repository. Unless there is an objection, I’ll push an initial commit with a LICENSE file so that there’s a baseline to make PRs against.
Thanks, Nick On Fri, 9 Jun 2023 at 18:30, Nick Dimiduk <[email protected]> wrote: > Thanks everyone who has replied. > > If there are no further concerns raised over the weekend, I’ll get started > on Monday with Infrastructure to provision the new repository. > > Thanks, > Nick > > On Fri, 9 Jun 2023 at 18:08, Peter Somogyi <[email protected]> wrote: > >> +1 on the new repository. >> >> On Wed, Jun 7, 2023 at 5:33 PM Viraj Jasani <[email protected]> wrote: >> >> > +1 for hbase-kustomize >> > >> > On Mon, Jun 5, 2023 at 4:44 AM Nick Dimiduk <[email protected]> >> wrote: >> > >> > > Hello, >> > > >> > > As I work through the integration of these kustomize definitions into >> the >> > > existing java project structure that is hbase-operator-tools, I'm >> > > increasingly of the opinion that this is too much of a clash of >> > concerns. I >> > > think that this contribution would make better sense as its own >> > repository >> > > with its own release cycle. I'm neither aware of nor can I imagine a >> > > technical coupling between the kustomize resources and the rest of the >> > > utilities in operator-tools. Likewise, this change set introduces new >> > > requirements (docker, buildx, KinD and/or minikube) to the build and >> test >> > > environment that are not otherwise needed by operator-tools. >> > > >> > > What do you think? SHould we request a new repository for the >> > > kustomize files? I propose hbase-kustomize.git. >> > > >> > > Thanks, >> > > Nick >> > > >> > > On Mon, May 22, 2023 at 4:33 PM Nick Dimiduk <[email protected]> >> > wrote: >> > > >> > > > I went ahead and rebuilt the Hadoop image module in the same style. >> I >> > > > rebased the zookeeper-single and hdfs kustimize implementations onto >> > the >> > > > same structure. So, PR’s #118, #119, #120, and #121 are all in this >> > > style. >> > > > I don’t have a place for running integration tests, but unit tests >> are >> > > now >> > > > running in Jenkins. >> > > > >> > > > I appreciate any attention you can provide. >> > > > >> > > > Thanks, >> > > > Nick >> > > > >> > > > On Mon, 22 May 2023 at 16:02, TAK-LON WU <[email protected]> >> wrote: >> > > > >> > > >> Sorry that I’m on vacation and will be back online after 06/06 , >> but >> > > >> thanks >> > > >> for putting the PR out and I believe someone on our side will >> review >> > it >> > > . >> > > >> ( >> > > >> or when I come back I will review them) >> > > >> >> > > >> -Stephen >> > > >> >> > > >> On Mon, May 22, 2023 at 6:08 PM Nick Dimiduk <[email protected]> >> > > wrote: >> > > >> >> > > >> > Heya team, >> > > >> > >> > > >> > I have rebuilt one of the early PRs so that the docker image >> build >> > > >> pieces >> > > >> > are integrated with the maven build. If this is acceptable to the >> > > >> > reviewers, I'll go forward with integrating the other images and >> > > >> > kustomize/kuttl tests in the same way. >> > > >> > >> > > >> > Please take a look. >> > > >> > >> > > >> > Thanks, >> > > >> > Nick >> > > >> > >> > > >> > https://github.com/apache/hbase-operator-tools/pull/118 >> > > >> > >> > > >> > On Fri, May 12, 2023 at 4:19 PM Nick Dimiduk < >> [email protected]> >> > > >> wrote: >> > > >> > >> > > >> > > Heya team, >> > > >> > > >> > > >> > > I have created individual pull-requests for each of the major >> > > >> functional >> > > >> > > pieces outlined in the initial branch. These await review. >> > > >> > > >> > > >> > > I've now started working integrating the test harness into the >> > maven >> > > >> > > build. After a brief detour for a Yetus plugin, I'm now looking >> > > >> instead >> > > >> > at >> > > >> > > maven integration via exec-maven-plugin. I'm also investigating >> > how >> > > to >> > > >> > pull >> > > >> > > the container image build up into maven as well. >> > > >> > > >> > > >> > > Thanks, >> > > >> > > Nic, >> > > >> > > >> > > >> > > On Thu, May 4, 2023 at 2:42 PM Nick Dimiduk < >> [email protected]> >> > > >> wrote: >> > > >> > > >> > > >> > >> Heya team, >> > > >> > >> >> > > >> > >> I've created a feature ticket [0] from which this contribution >> > can >> > > >> hang. >> > > >> > >> I've created an associated release version [1] and feature >> branch >> > > [2] >> > > >> > >> against which we can target PRs while things take shape. I've >> > > >> published >> > > >> > my >> > > >> > >> initial extraction of this feature as a whole for your review >> [3] >> > > -- >> > > >> > take a >> > > >> > >> look at the big picture there. For each commit on that branch, >> > I've >> > > >> > created >> > > >> > >> a sub-task on HBASE-27827. Probably reviewers will find other >> > items >> > > >> we >> > > >> > need >> > > >> > >> to peel off as sub-tasks. I'll start turning each of these >> > commits >> > > >> into >> > > >> > PRs >> > > >> > >> suitable for the Apache repo and your perusal. >> > > >> > >> >> > > >> > >> I think we're getting due for the 1.3 release of Operator >> Tools, >> > > so I >> > > >> > >> expect this will land in 1.4. >> > > >> > >> >> > > >> > >> Thanks, >> > > >> > >> Nick >> > > >> > >> >> > > >> > >> [0]: https://issues.apache.org/jira/browse/HBASE-27827 >> > > >> > >> [1]: >> > > >> https://issues.apache.org/jira/browse/HBASE/fixforversion/12353199 >> > > >> > >> [2]: >> > > >> > >> >> > > >> > >> > > >> >> > > >> > >> https://github.com/apache/hbase-operator-tools/tree/HBASE-27827-kubernetes-deployment >> > > >> > >> [3]: >> > > >> > >> >> > > >> > >> > > >> >> > > >> > >> https://github.com/apache/hbase-operator-tools/compare/HBASE-27827-kubernetes-deployment...ndimiduk:hbase-operator-tools:HBASE-27827-kubernetes-deployment >> > > >> > >> >> > > >> > >> >> > > >> > >> >> > > >> > >> On Mon, Mar 13, 2023 at 11:28 AM Nick Dimiduk < >> > [email protected] >> > > > >> > > >> > >> wrote: >> > > >> > >> >> > > >> > >>> Heya team, >> > > >> > >>> >> > > >> > >>> Over here at $dayjob, we have an increasing reliance on >> > Kubernetes >> > > >> for >> > > >> > >>> both development and production workloads. Our tools are >> > maturing >> > > >> and >> > > >> > >>> we're hoping that they might be of interest to the wider >> > > community. >> > > >> > >>> I'd like to see if there's community interest in receiving >> > > some/any >> > > >> of >> > > >> > >>> them as a contribution. I think we'll also need a plan from >> ASF >> > > >> Infra >> > > >> > >>> that makes kubernetes available to us as a project. >> > > >> > >>> >> > > >> > >>> We have implemented a basic stack of tools for orchestrating >> ZK >> > + >> > > >> HDFS >> > > >> > >>> + HBase on Kubernetes. We use this for running a small local >> dev >> > > >> > >>> cluster via MiniKube/KIND ; for ITBLL on smallish distributed >> > > >> clusters >> > > >> > >>> in a public cloud ; and in production for running clusters of >> > ~100 >> > > >> > >>> Data Nodes/Region Servers in a public cloud. There was an >> > earlier >> > > >> > >>> discussion about using our donation of test hardware for >> running >> > > >> more >> > > >> > >>> thorough tests in our CI, but one of the limiting factors is >> > full >> > > >> > >>> cluster deployment. I hope that the community might be >> > interested >> > > in >> > > >> > >>> receiving this tooling as a foundation for more rigorous >> > > correctness >> > > >> > >>> and maybe even performance tests in the open. Furthermore, >> > perhaps >> > > >> the >> > > >> > >>> wider community has interest in an Apache licensed cluster >> > > >> > >>> orchestration tool for other uses. >> > > >> > >>> >> > > >> > >>> Now for some details: The implementation is built on >> Kustomize, >> > so >> > > >> > >>> it's fundamentally transparent resource specification with >> yaml >> > > >> > >>> patches for composability; this is in contrast to a solution >> > using >> > > >> > >>> templates with defined capabilities and interfaces. There is >> no >> > > >> > >>> operator ; it's all coordinated via init/bootstrap >> containers, >> > > shell >> > > >> > >>> scripts, shared volumes for state, &c. For now. >> > > >> > >>> >> > > >> > >>> Such a donation will amount to a code drop, which will have >> its >> > > >> > >>> challenges. I'm motivated via internal processes to carve it >> > into >> > > >> > >>> smaller pieces, and I think that will benefit community >> review >> > as >> > > >> > >>> well. Perhaps this approach could be used to make the >> > contribution >> > > >> via >> > > >> > >>> a feature branch. >> > > >> > >>> >> > > >> > >>> Is there community interest in adding such a capability to >> our >> > > >> > >>> maintained responsibilities? I'd hope that we have several >> > > >> volunteers >> > > >> > >>> to work with me through the contribution process, and who are >> > > >> > >>> reasonably confident that they'll be able to help maintain >> such >> > a >> > > >> > >>> capability going forward. We'll also need someone who can >> work >> > > with >> > > >> > >>> Infra to get us access to Kubernetes cluster(s), via whatever >> > > means. >> > > >> > >>> >> > > >> > >>> What do you think? >> > > >> > >>> >> > > >> > >>> Thanks, >> > > >> > >>> Nick & the HBase team at Apple >> > > >> > >>> >> > > >> > >> >> > > >> > >> > > >> >> > > > >> > > >> > >> >
