Hey Ron, As a part of MapR (now HPE), I've created a native operator for Apache Drill and this works on multiple variants of Kubernetes including OpenShift. With this, we introduce a new Kind called "DrillCluster" via a Custom Resource Definition (CRD) and a Custom Controller (logic to manage this DrillCluster kind - written in Golang) for the same. Using this, users can easily deploy Drill clusters by submitting Custom Resource YAML files (CRs) for the DrillCluster kind. It supports creation of multiple Drill clusters (multiple Drillbits launched in distributed mode), multiple versions (such as 1.15.0 and 1.16.0), auto-scaling the number of Drillbits (based on CPU utilization) and more. I can share more details of this if anyone's interested.
While Vanilla K8S, and GKE worked out of the box, I had to make some changes to support OpenShift (related to Service Accounts, Security Context Constraints, etc). Perhaps you ran into similar issues (I'm yet to read this thread fully). We recently had a v1.0.0 GA release [1], [2] & [3]. One thing to note is that the current release has dependencies and integrations with MapR's distribution of Apache Drill and is close sourced at the moment (there is plan to open source that in the near future). I have an open source variant of this in the works - to support vanilla Apache Drill. In the current state, it has all similar features , it removes the MapR specific integration (reliance on MapR-FS instead of HDFS, MapR ZooKeeper and such). I shortly plan to add Apache HDFS and ZooKeeper integration instead. Let me know if you're interested - and I can share the GitHub branch. Regards, Abhishek [1] https://mapr.com/blog/mapr-releases-kubernetes-ecosystem-operators-for-apache-spark-and-apache-drill/ [2] https://mapr.com/docs/home/PersistentStorage/running_drillbits_in_compute_space.html [3] https://github.com/mapr/mapr-operators On Wed, Jan 29, 2020 at 11:11 AM Ron Cecchini <[email protected]> wrote: > > Hi, all. Drill and OpenShift newbie here. > > Has anyone successfully deployed a Drill Docker container to an OpenShift > environment? > > While there is information about Drill Docker, there seems to be zero > information about OpenShift in particular. > > Per the instructions at drill.apache.org/docs/running-drill-on-docker, I > pulled the Drill Docker image from Docker Hub, and then pushed it to our > OpenShift environment. But when I tried to deploy it, I immediately ran > into an error about /opt/drill/conf/drill-override.conf not being readable. > > I understand why the problem is happening (because of who OpenShift runs > the container as), so I downloaded the source from GitHub and modified the > Dockerfile to include: > > RUN chgrp -R 0 /opt/drill && chmod -R g=u /opt/drill > > so that all of /opt/drill would be available to everyone. But then > 'docker build' kept failing, giving the error: > > Non-resolvable parent POM for > org.apache.drill:drill-root:1.18.0-SNAPSHOT: > Could not transfer artifact org.apache:apache:pom:21 > > I tried researching that error but couldn't figure out what was going on. > So I finally decided to start trying to mount persistent volumes, creating > one PV for /opt/drill/conf (and then copying the default > drill-override.conf there) and one PV for /opt/drill/log. > > Now the container gets much further, but eventually fails on something > Hadoop related. I'm not trying to do anything with Hadoop, so I don't know > what that's about, but it says I don't have HADOOP_HOME set. > > Hopefully I can figure out the remaining steps I need (an environment > variable? more configs?), but I was wondering if anyone else had already > successfully figured out how to deploy to OpenShift, or might know why the > 'docker build' fails with that error? > > For what it's worth, I copied over only that drill-override.conf and > nothing else. And I did not set any Drill environment variables in > OpenShift. I'm basically trying to run the "vanilla" Drill Docker as-is. > > Thanks for any help! > > Ron >
