Heya team, I've created a feature ticket [0] from which this contribution can hang. I've created an associated release version [1] and feature branch [2] against which we can target PRs while things take shape. I've published my initial extraction of this feature as a whole for your review [3] -- take a look at the big picture there. For each commit on that branch, I've created a sub-task on HBASE-27827. Probably reviewers will find other items we need to peel off as sub-tasks. I'll start turning each of these commits into PRs suitable for the Apache repo and your perusal.
I think we're getting due for the 1.3 release of Operator Tools, so I expect this will land in 1.4. Thanks, Nick [0]: https://issues.apache.org/jira/browse/HBASE-27827 [1]: https://issues.apache.org/jira/browse/HBASE/fixforversion/12353199 [2]: https://github.com/apache/hbase-operator-tools/tree/HBASE-27827-kubernetes-deployment [3]: https://github.com/apache/hbase-operator-tools/compare/HBASE-27827-kubernetes-deployment...ndimiduk:hbase-operator-tools:HBASE-27827-kubernetes-deployment On Mon, Mar 13, 2023 at 11:28 AM Nick Dimiduk <[email protected]> wrote: > Heya team, > > Over here at $dayjob, we have an increasing reliance on Kubernetes for > both development and production workloads. Our tools are maturing and > we're hoping that they might be of interest to the wider community. > I'd like to see if there's community interest in receiving some/any of > them as a contribution. I think we'll also need a plan from ASF Infra > that makes kubernetes available to us as a project. > > We have implemented a basic stack of tools for orchestrating ZK + HDFS > + HBase on Kubernetes. We use this for running a small local dev > cluster via MiniKube/KIND ; for ITBLL on smallish distributed clusters > in a public cloud ; and in production for running clusters of ~100 > Data Nodes/Region Servers in a public cloud. There was an earlier > discussion about using our donation of test hardware for running more > thorough tests in our CI, but one of the limiting factors is full > cluster deployment. I hope that the community might be interested in > receiving this tooling as a foundation for more rigorous correctness > and maybe even performance tests in the open. Furthermore, perhaps the > wider community has interest in an Apache licensed cluster > orchestration tool for other uses. > > Now for some details: The implementation is built on Kustomize, so > it's fundamentally transparent resource specification with yaml > patches for composability; this is in contrast to a solution using > templates with defined capabilities and interfaces. There is no > operator ; it's all coordinated via init/bootstrap containers, shell > scripts, shared volumes for state, &c. For now. > > Such a donation will amount to a code drop, which will have its > challenges. I'm motivated via internal processes to carve it into > smaller pieces, and I think that will benefit community review as > well. Perhaps this approach could be used to make the contribution via > a feature branch. > > Is there community interest in adding such a capability to our > maintained responsibilities? I'd hope that we have several volunteers > to work with me through the contribution process, and who are > reasonably confident that they'll be able to help maintain such a > capability going forward. We'll also need someone who can work with > Infra to get us access to Kubernetes cluster(s), via whatever means. > > What do you think? > > Thanks, > Nick & the HBase team at Apple >
