+1 on this plan. Thanks for wrapping this up Mike!
I ran the test locally on my Mac (10.14.2) with no issues.
I also ran the tests in a docker image for the following operating systems:
- centos:6
- centos:7
- debian:jessie
- debian:stretch
- ubuntu:trusty
- ubuntu:xenial
- ubuntu:bionic
I ran into a few hiccups during testing, but they were all easy to resolve:
- I needed to upgrade the default Maven version on ubuntu:trusty and
centos.
- I ran into a open-jdk/maven-surefire bug on debian:stretch
-
https://stackoverflow.com/questions/53010200/maven-surefire-could-not-find-forkedbooter-class
- Fixed with:
>
>
>
>
>
> *<plugin> <groupId>org.apache.maven.plugins</groupId>
> <artifactId>maven-surefire-plugin</artifactId> <configuration>
> <useSystemClassLoader>false</useSystemClassLoader>
> </configuration></plugin>*
The setup varies a bit based on the OS but my basic test script was
effectively this:
*docker run -it --rm --privileged ubuntu:xenial /bin/bash*
> *apt-get update -y*
*apt-get install -y git maven ntp openjdk-8-jdk*
> *echo "listen on *" >> /etc/ntp.conf*
*service ntp restart*
> *git clone https://github.com/mpercy/kudu-binary-jar-test/
> <https://github.com/mpercy/kudu-binary-jar-test/>*
*cd kudu-binary-jar-test*
*mvn test*
On Tue, Mar 5, 2019 at 2:57 AM Tim Robertson <[email protected]>
wrote:
> Thank you for progressing all this Mike (and others).
> I will endeavour to test this soon and replace the Beam unit tests with
> this - I've been sidelined a bit recently with other tasks but getting back
> into Beam development again.
>
>
>
>
> On Tue, Mar 5, 2019 at 9:16 AM Mike Percy <[email protected]> wrote:
>
> > Hi devs,
> > For the 1.9.0 release, we (Grant Henke, Brian McDevitt, and I)
> implemented
> > the capability to start a Kudu minicluster from a JVM environment without
> > having to first build Kudu locally (work tracked as KUDU-2411
> > <https://issues.apache.org/jira/browse/KUDU-2411>). This was made
> possible
> > by uploading relocatable Kudu binaries (Linux and Mac) to Maven central,
> > exposing the Kudu MiniCluster libraries for use by external projects, and
> > adding logic to the MiniCluster to search the classpath for the Kudu
> > binaries.
> >
> > The completion of this work landed a little late in the 1.9.0 release
> > cycle, just before we started creating release candidates, and as a
> result
> > the binaries didn't make it into the RC process for a vote. I propose we
> > release binary convenience artifacts based on the source tag without
> going
> > through another vote because (1) they were built with the 1.9.0 RC2
> source
> > code as voted upon, (2) they are provided simply for convenience and are
> > not needed to use Kudu, and (3) they are currently considered
> experimental.
> >
> > I have already audited the binary artifacts for licensing issues and put
> in
> > place automation
> > <
> >
> https://github.com/apache/kudu/blob/1.9.0-RC2/build-support/mini-cluster/check-license.pl
> > >
> > for
> > the license checks at artifact build time. README, NOTICE, and LICENSE
> are
> > all included.
> >
> > If you're interested in these artifacts and would like to see if they
> work
> > for you, or would like to take a closer look for other reasons, I've
> > uploaded binary jar artifacts for Linux and Mac (h/t to Alexey for the El
> > Capitan build) to the Maven staging repository at
> > https://repository.apache.org/content/repositories/orgapachekudu-1033/
> >
> > We also have an example of using the above binary artifacts, built from
> the
> > 1.9.0-RC2 tag and using 1.9.0-RC2 Java client libraries at
> > https://github.com/mpercy/kudu-binary-jar-test/ -- you can see if the
> > artifact works in your environment by simply cloning that Git repository
> > and running "mvn test". (This is just a copy/pasted and modified version
> of
> > java-example from the Kudu code examples repo; h/t to Brian for getting
> it
> > to work with the binary jar.)
> >
> > Please note that the binary jar Maven integration (using os-maven-plugin)
> > requires Maven 3.1 or later. In addition, it must be run on macOS El
> > Capitan or later, or a modern flavor of Linux (any Linux distribution
> that
> > Kudu supports
> > <
> >
> https://kudu.apache.org/docs/installation.html#_prerequisites_and_requirements
> > >
> > should work).
> >
> > A draft of user docs for how to integrate the Kudu binary jar into a Java
> > project can be found here
> > <
> >
> https://docs.google.com/document/d/1n9flFdHEnULX3_wxmy__gU7WKK-FIYFxvufWbvEHu-c/
> > >
> > and
> > if we have consensus on including this as outlined above then I'll work
> > with Andrew to get the relevant docs posted to the web site in time for
> the
> > 1.9.0 release announcement.
> >
> > Thanks to those who helped get this done (some early prototyping was also
> > done by Tim Robertson) and I hope that folks find this new capability
> > useful!
> >
> > Please let me know if you have questions, concerns, or feedback on this
> > functionality.
> >
> > Regards,
> > Mike
> >
>
--
Grant Henke
Software Engineer | Cloudera
[email protected] | twitter.com/gchenke | linkedin.com/in/granthenke
