Thanks, Allen! On 8/2/18 2:56 PM, Allen Wittenauer wrote:
On Aug 2, 2018, at 10:58 AM, Josh Elser <[email protected]> wrote: The fun part is that this change breaks Precommit because the maven plugin is using the "test-compile" Maven lifecycle phase for the "unit", "findbugs", and "compile" Yetus tests. It breaks precommit because the shade-plugin doesn't run until the package lifecycle phase, and thus downstream Maven modules from this one miss all of the third-party code because the shaded jar wasn't built (the Maven reactor only finds the compiled protobuf classes). I think the long term solution is to pull out the shading of third-party deps into its own jar and pull it out of the Ratis day-to-day build.Maybe I’m missing something here, but is the shaded module not listed as a dependency of the other modules? (It sounds like “no”, which means all kinds of problems.) But if it is, then this should be trivially fixed by just changing the phase that the shaded plugin runs. The other thing that sticks out is that (at least by default) mvn install gets run as one of the first things, so the shaded jar should already be in the maven repo long before any test-compile-based plugins happen. Are classpath shenanigans happening? It sounds like something more fundamental is wrong here.
Moving maven-jar-plugin and the maven-shade-plugin to the requisite lifecycle phase should do the trick, yes. Feels similarly dirty :)
A related point is that Ratis' personality just enqueues all modules, not the ones modified. I think that's why compile and unit end up pulling from the reactor instead of the installed copy.
They dependency on the shaded jar is marked as provided, but I don't think that's the reason cause of the problem.
That said...I am hoping to come up with a workaround for now. I could change the shade-plugin to bind to the compilation phase instead of package but that goes against the "maven way". I was hoping to find any way that would let me override `maven_modules_worker`, shy of copying/modifying the entire plugin. Any suggestions?Personality files have a bunch of abilities that are great for situations where a dependency needs to be in place that maven doesn’t know. There are two ways to handle this particular problem, depending upon the particular nuances: * A _precompile step (available to PROJECT_NAME, BUILDTOOL, TESTTYPES, and TESTFORMATS) Fairly self explanatory. Just write a routine called ratis_precompile (or whatever) that does what needs to be done. It will get executed prior to *every* compile phase * Special handling in personsonality_modules This is a routine that called prior to (almost) every individual action and allows the personality to provide overrides, extra flags, etc. Hadoop, of course, has a nice and complex example due to it’s overly complex build process. But the highlight for this use case is almost certainly the unit) clause and the hadoop_unittest_prereqs routine.
Thanks, I'll look closer here. It's still going over my head from terminology only, but maybe a reread on the hadoop personality will do me some good.
