Thanks to you for making this test possible Mike.
I was approaching this emulating protoc where they put the binaries as
artifacts for Maven [1].
A slight difference is that protoc is a single file while your tarball has
several. I notice that the protoc build plugin for maven also copies out
from the jar to a local filesystem [2] so I just copied that approach.
What I could imagine is we have a jar with the binaries and a single class
(say EmbeddedKudu) that copies the binaries into a temporary directory (as
my hack in GH).
A user would then do the following:
<!-- finds the environment -->
<build>
<extensions>
<extension>
<groupId>kr.motd.maven</groupId>
<artifactId>os-maven-plugin</artifactId>
<version>1.6.0</version>
</extension>
</extensions>
</build>
....
<!-- Adds mini cluster -->
<dependency>
<groupId>org.apache.kudu</groupId>
<artifactId>kudu-client</artifactId>
<version>1.7.0</version>
<classifier>tests</classifier>
</dependency>
<!-- Adds an embedded Kudu -->
<dependency>
<groupId>org.apache.kudu</groupId>
<artifactId>kudu-test-server</artifactId>
<version>1.7.0</version>
<classifier>${os.detected.classifier}</classifier>
</dependency>
The detect classifier stuff is copied from protoc, but perhaps we'd just
state that available options are (?) linux / OSX and ignore autodetection.
And in code users would do something like:
EmbeddedKudu.prepare(); // copies kudu from the jar and sets the
system variable (as per my example)
MiniKuduCluster miniCluster =
new
MiniKuduCluster.MiniKuduClusterBuilder().numMasters(1).numTservers(1).build();
What this would mean:
- no change to the existing Kudu code packkaging
- leverage caching of the binaries as any Java artifact
- download would be at build time not test runtime - only copying out from
the jar each time
- simple to include in most build environments (not tied to maven as a
plugin)
I can't comment if it makes sense to do this WRT the binaries though and it
would mean building and releasing binaries into a jar on each Kudu release
(as protoc does).
WDYT?
Thanks,
Tim
[1] http://repo1.maven.org/maven2/com/google/protobuf/protoc/3.6.1/
[2]
https://github.com/os72/protoc-jar-maven-plugin/blob/master/src/main/java/com/github/os72/protocjar/maven/ProtocJarMojo.java#L664
On Thu, Aug 2, 2018 at 10:28 PM, Mike Percy <[email protected]> wrote:
> Ha, neat, thanks for posting this, Tim. It's a nice proof of concept.
>
> I was imagining that we would try to implement the downloading part as a
> Maven plugin, but maybe it could work to try to download the artifacts at
> runtime with a JUnit test. Do you think we could cache the artifacts
> somewhere, maybe in the Maven repo somehow, so we don't have to download
> the artifact every time we want to use it? I was hoping to simply be able
> to ship a tarball or a jar of the binaries separately from the test
> framework code.
>
> Mike
>
> On Thu, Aug 2, 2018 at 8:42 AM Tim Robertson <[email protected]>
> wrote:
>
> > Hi folks
> >
> > I've not had too much time, but I threw this together using Mike's
> > binaries:
> > https://github.com/timrobertson100/kudu-test-server
> >
> > I think this shows that running a mini cluster is possible using the
> > binaries Mike prepared when they are included in a jar (on CentOS 7.4 at
> > least).
> >
> > Please don't flame me for the code - it was a rush job - but perhaps you
> > could verify it works for you Mike?
> >
> >
> >
> >
> >
> >
> > On Thu, Jul 19, 2018 at 12:35 AM, Mike Percy <[email protected]> wrote:
> >
> > > On Wed, Jul 18, 2018 at 1:24 PM Tim Robertson <
> [email protected]
> > >
> > > wrote:
> > >
> > > > > I'm pretty busy this week and would be happy to get some help on
> this
> > > if
> > > > anybody has cycles to spare.
> > > >
> > > > Thanks Mike - I'll look into 3) and if we get it working I'm happy to
> > > offer
> > > > a PR for 4).
> > > > This is an evening project for me so I might be a little slow too -
> if
> > > > someone else is keen and has time please feel free to jump in.
> > > >
> > >
> > > Sounds great! I think Grant started working on #4 but I don't know how
> > far
> > > he got.
> > >
> > > Mike
> > >
> >
>
