Thanks everyone who has replied. If there are no further concerns raised over the weekend, I’ll get started on Monday with Infrastructure to provision the new repository.
Thanks, Nick On Fri, 9 Jun 2023 at 18:08, Peter Somogyi <psomo...@apache.org> wrote: > +1 on the new repository. > > On Wed, Jun 7, 2023 at 5:33 PM Viraj Jasani <vjas...@apache.org> wrote: > > > +1 for hbase-kustomize > > > > On Mon, Jun 5, 2023 at 4:44 AM Nick Dimiduk <ndimi...@apache.org> wrote: > > > > > Hello, > > > > > > As I work through the integration of these kustomize definitions into > the > > > existing java project structure that is hbase-operator-tools, I'm > > > increasingly of the opinion that this is too much of a clash of > > concerns. I > > > think that this contribution would make better sense as its own > > repository > > > with its own release cycle. I'm neither aware of nor can I imagine a > > > technical coupling between the kustomize resources and the rest of the > > > utilities in operator-tools. Likewise, this change set introduces new > > > requirements (docker, buildx, KinD and/or minikube) to the build and > test > > > environment that are not otherwise needed by operator-tools. > > > > > > What do you think? SHould we request a new repository for the > > > kustomize files? I propose hbase-kustomize.git. > > > > > > Thanks, > > > Nick > > > > > > On Mon, May 22, 2023 at 4:33 PM Nick Dimiduk <ndimi...@apache.org> > > wrote: > > > > > > > I went ahead and rebuilt the Hadoop image module in the same style. I > > > > rebased the zookeeper-single and hdfs kustimize implementations onto > > the > > > > same structure. So, PR’s #118, #119, #120, and #121 are all in this > > > style. > > > > I don’t have a place for running integration tests, but unit tests > are > > > now > > > > running in Jenkins. > > > > > > > > I appreciate any attention you can provide. > > > > > > > > Thanks, > > > > Nick > > > > > > > > On Mon, 22 May 2023 at 16:02, TAK-LON WU <taklo...@gmail.com> wrote: > > > > > > > >> Sorry that I’m on vacation and will be back online after 06/06 , but > > > >> thanks > > > >> for putting the PR out and I believe someone on our side will review > > it > > > . > > > >> ( > > > >> or when I come back I will review them) > > > >> > > > >> -Stephen > > > >> > > > >> On Mon, May 22, 2023 at 6:08 PM Nick Dimiduk <ndimi...@apache.org> > > > wrote: > > > >> > > > >> > Heya team, > > > >> > > > > >> > I have rebuilt one of the early PRs so that the docker image build > > > >> pieces > > > >> > are integrated with the maven build. If this is acceptable to the > > > >> > reviewers, I'll go forward with integrating the other images and > > > >> > kustomize/kuttl tests in the same way. > > > >> > > > > >> > Please take a look. > > > >> > > > > >> > Thanks, > > > >> > Nick > > > >> > > > > >> > https://github.com/apache/hbase-operator-tools/pull/118 > > > >> > > > > >> > On Fri, May 12, 2023 at 4:19 PM Nick Dimiduk <ndimi...@apache.org > > > > > >> wrote: > > > >> > > > > >> > > Heya team, > > > >> > > > > > >> > > I have created individual pull-requests for each of the major > > > >> functional > > > >> > > pieces outlined in the initial branch. These await review. > > > >> > > > > > >> > > I've now started working integrating the test harness into the > > maven > > > >> > > build. After a brief detour for a Yetus plugin, I'm now looking > > > >> instead > > > >> > at > > > >> > > maven integration via exec-maven-plugin. I'm also investigating > > how > > > to > > > >> > pull > > > >> > > the container image build up into maven as well. > > > >> > > > > > >> > > Thanks, > > > >> > > Nic, > > > >> > > > > > >> > > On Thu, May 4, 2023 at 2:42 PM Nick Dimiduk < > ndimi...@apache.org> > > > >> wrote: > > > >> > > > > > >> > >> Heya team, > > > >> > >> > > > >> > >> I've created a feature ticket [0] from which this contribution > > can > > > >> hang. > > > >> > >> I've created an associated release version [1] and feature > branch > > > [2] > > > >> > >> against which we can target PRs while things take shape. I've > > > >> published > > > >> > my > > > >> > >> initial extraction of this feature as a whole for your review > [3] > > > -- > > > >> > take a > > > >> > >> look at the big picture there. For each commit on that branch, > > I've > > > >> > created > > > >> > >> a sub-task on HBASE-27827. Probably reviewers will find other > > items > > > >> we > > > >> > need > > > >> > >> to peel off as sub-tasks. I'll start turning each of these > > commits > > > >> into > > > >> > PRs > > > >> > >> suitable for the Apache repo and your perusal. > > > >> > >> > > > >> > >> I think we're getting due for the 1.3 release of Operator > Tools, > > > so I > > > >> > >> expect this will land in 1.4. > > > >> > >> > > > >> > >> Thanks, > > > >> > >> Nick > > > >> > >> > > > >> > >> [0]: https://issues.apache.org/jira/browse/HBASE-27827 > > > >> > >> [1]: > > > >> https://issues.apache.org/jira/browse/HBASE/fixforversion/12353199 > > > >> > >> [2]: > > > >> > >> > > > >> > > > > >> > > > > > > https://github.com/apache/hbase-operator-tools/tree/HBASE-27827-kubernetes-deployment > > > >> > >> [3]: > > > >> > >> > > > >> > > > > >> > > > > > > https://github.com/apache/hbase-operator-tools/compare/HBASE-27827-kubernetes-deployment...ndimiduk:hbase-operator-tools:HBASE-27827-kubernetes-deployment > > > >> > >> > > > >> > >> > > > >> > >> > > > >> > >> On Mon, Mar 13, 2023 at 11:28 AM Nick Dimiduk < > > ndimi...@apache.org > > > > > > > >> > >> wrote: > > > >> > >> > > > >> > >>> Heya team, > > > >> > >>> > > > >> > >>> Over here at $dayjob, we have an increasing reliance on > > Kubernetes > > > >> for > > > >> > >>> both development and production workloads. Our tools are > > maturing > > > >> and > > > >> > >>> we're hoping that they might be of interest to the wider > > > community. > > > >> > >>> I'd like to see if there's community interest in receiving > > > some/any > > > >> of > > > >> > >>> them as a contribution. I think we'll also need a plan from > ASF > > > >> Infra > > > >> > >>> that makes kubernetes available to us as a project. > > > >> > >>> > > > >> > >>> We have implemented a basic stack of tools for orchestrating > ZK > > + > > > >> HDFS > > > >> > >>> + HBase on Kubernetes. We use this for running a small local > dev > > > >> > >>> cluster via MiniKube/KIND ; for ITBLL on smallish distributed > > > >> clusters > > > >> > >>> in a public cloud ; and in production for running clusters of > > ~100 > > > >> > >>> Data Nodes/Region Servers in a public cloud. There was an > > earlier > > > >> > >>> discussion about using our donation of test hardware for > running > > > >> more > > > >> > >>> thorough tests in our CI, but one of the limiting factors is > > full > > > >> > >>> cluster deployment. I hope that the community might be > > interested > > > in > > > >> > >>> receiving this tooling as a foundation for more rigorous > > > correctness > > > >> > >>> and maybe even performance tests in the open. Furthermore, > > perhaps > > > >> the > > > >> > >>> wider community has interest in an Apache licensed cluster > > > >> > >>> orchestration tool for other uses. > > > >> > >>> > > > >> > >>> Now for some details: The implementation is built on > Kustomize, > > so > > > >> > >>> it's fundamentally transparent resource specification with > yaml > > > >> > >>> patches for composability; this is in contrast to a solution > > using > > > >> > >>> templates with defined capabilities and interfaces. There is > no > > > >> > >>> operator ; it's all coordinated via init/bootstrap containers, > > > shell > > > >> > >>> scripts, shared volumes for state, &c. For now. > > > >> > >>> > > > >> > >>> Such a donation will amount to a code drop, which will have > its > > > >> > >>> challenges. I'm motivated via internal processes to carve it > > into > > > >> > >>> smaller pieces, and I think that will benefit community review > > as > > > >> > >>> well. Perhaps this approach could be used to make the > > contribution > > > >> via > > > >> > >>> a feature branch. > > > >> > >>> > > > >> > >>> Is there community interest in adding such a capability to our > > > >> > >>> maintained responsibilities? I'd hope that we have several > > > >> volunteers > > > >> > >>> to work with me through the contribution process, and who are > > > >> > >>> reasonably confident that they'll be able to help maintain > such > > a > > > >> > >>> capability going forward. We'll also need someone who can work > > > with > > > >> > >>> Infra to get us access to Kubernetes cluster(s), via whatever > > > means. > > > >> > >>> > > > >> > >>> What do you think? > > > >> > >>> > > > >> > >>> Thanks, > > > >> > >>> Nick & the HBase team at Apple > > > >> > >>> > > > >> > >> > > > >> > > > > >> > > > > > > > > > >
