This is a known issue in maven that we have to do a more advanced build phase (test-compile) to run a simpler one. The main reason behind this is that maven isn't smart enough at the moment to recognize when it needs to do more work for another, dependent, module. For instance, it needs the hadoop-compat.jar since it is a dependent project but it can't get it because it doesn't get run in the correct phase.
This is a known issue with maven and there are comments in the pom (or at least there were, when I moved the poms) to the effect that there are filed maven issues and its a known bug. It seemed a minor overhead to workaround the bug by ordering your tasks correctly in maven such that things build correctly. Note that you can run maven with the -o option to run offline and any previously installed jars will be used and any changes your made will overwrite only the necessary jars. There has also been some discussion around moving the test sources around for certain classes, such that they are correctly inheritable. On one hand, this solves the problem, but is a 'dirty' way to enable the sources to be accessible. In fact, we discussed this option with the hbase-common module ( https://issues.apache.org/jira/browse/HBASE-6162). If I recall correctly, we ended up going with the idea of moving the test-utilites to a src/main directory, but leaving the actual tests in the test directory. Even better, would be creating an hbase-test-util class that has the utilities in the src/main and is usable by all dependent projects (though this could just as easily stay in hbase-common). The problem with building the jar at generate-test-sources (or -resources) is that this is actually before the compilation step (test-compile), inherently breaking the logical ordering of these things. All that said, any ideas to make this better (in a comprehensive solution) would be great! -Jesse ------------------- Jesse Yates @jesse_yates jyates.github.com On Wed, Aug 1, 2012 at 12:33 PM, Alex Baranau <[email protected]>wrote: > Hello, > > I've been working on https://issues.apache.org/jira/browse/HBASE-6411. > There's a patch almost ready for commit (I guess). But we faced the > following problem. Would be great if someone can share his ideas. Thank > you! > > Zhihong Ted Yu< > https://issues.apache.org/jira/secure/ViewProfile.jspa?name=zhihyu%40ebaysf.com > > > added > a comment - 01/Aug/12 16:43 > > For findbugs: > > [ERROR] Failed to execute goal > org.apache.maven.plugins:maven-remote-resources-plugin:1.1:process > (default) on project hbase-hadoop1-compat: Failed to resolve > dependencies for one or more projects in the reactor. Reason: Missing: > [ERROR] ---------- > [ERROR] 1) > org.apache.hbase:hbase-hadoop-compat:test-jar:tests:0.95-SNAPSHOT > [ERROR] > [ERROR] Try downloading the file manually from the project website. > [ERROR] > [ERROR] Then, install it using the command: > [ERROR] mvn install:install-file -DgroupId=org.apache.hbase > -DartifactId=hbase-hadoop-compat -Dversion=0.95-SNAPSHOT > -Dclassifier=tests -Dpackaging=test-jar -Dfile=/path/to/file > [ERROR] > [ERROR] Alternatively, if you host your own repository you can deploy > the file there: > [ERROR] mvn deploy:deploy-file -DgroupId=org.apache.hbase > -DartifactId=hbase-hadoop-compat -Dversion=0.95-SNAPSHOT > -Dclassifier=tests -Dpackaging=test-jar -Dfile=/path/to/file > -Durl=[url] -DrepositoryId=[id] > [ERROR] > [ERROR] Path to dependency: > [ERROR] 1) org.apache.hbase:hbase-hadoop1-compat:jar:0.95-SNAPSHOT > [ERROR] 2) > org.apache.hbase:hbase-hadoop-compat:test-jar:tests:0.95-SNAPSHOT > [ERROR] > [ERROR] ---------- > [ERROR] 1 required artifact is missing. > [ERROR] > > > Alex Baranau: > > Heh, > > Take a look at http://jira.codehaus.org/browse/MRRESOURCES-53. I guess we > are facing this problem. I tried to fix by moving remote plugin from > execution phase as it was suggested by adding [1] to parent pom. Though it > managed to compile hadoop1-compat, hbase-server compilation failed with > same error. Haven't dig a lot there, but what I tried didn't help. > > Using test-copile instead of compile does work. Can we for findbugs > execution change compile to test-compile? I believe it will compile the > main sources as well. But it looks like we'd have to adjust a lot more > than just this command... > > > After digging more, I isolated the problem with hbase-server compilation > when trying to fix things with adding [1] in parent pom. It turned out that > maven antrun plugin [2] is the reason. It requires test dependencies to be > resolved and currently configured to be executed on "generate-sources" > phase. And fails to resolve test-jar dependencies. E.g. if you change it to > run at "generate-test-sources" phase, build completes. But it looks like we > don't want to do it. Otherwise we'd have to move dependent plugin > executions to this phase as well. If I'm not mistaken. > > > I'd may be ask someone who configured HBase poms if he/they have ideas for > workaround. > > As a solution (even if not nice) I guess we could move those classes from > test sources into main sources. But this would mean that we need to have a > dependency from main sources in hadoop*-compat on hadoop test sources (as > those classes implemented with the help of them). > > Thoughts? > > Alex > > [1] > <plugin> > <groupId>org.apache.maven.plugins</groupId> > <artifactId>maven-remote-resources-plugin</artifactId> > <version>1.1</version> > <executions> > <execution> > <phase>none</phase> > </execution> > </executions> > <goals> > <goal>process</goal> > </goals> > </plugin> > > [2] > > from hbase-server pom: > <plugin> > <artifactId>maven-antrun-plugin</artifactId> > <executions> > <!-- Generate web app sources --> > <execution> > <id>generate</id> > <phase>generate-sources</phase> > <configuration> > <target> > <property name="build.webapps" > location="${project.build.directory}/hbase-webapps"/> > <property name="src.webapps" > location="${basedir}/src/main/resources/hbase-webapps"/> > <property name="generated.sources" > location="${project.build.directory}/generated-sources"/> > <mkdir dir="${build.webapps}"/> > <copy todir="${build.webapps}"> > <fileset dir="${src.webapps}"> > <exclude name="**/*.jsp"/> > <exclude name="**/.*"/> > <exclude name="**/*~"/> > </fileset> > </copy> > <!--The compile.classpath is passed in by maven --> > <taskdef classname="org.apache.jasper.JspC" > name="jspcompiler" classpathref="maven.compile.classpath"/> > <mkdir dir="${build.webapps}/master/WEB-INF"/> > <jspcompiler uriroot="${src.webapps}/master" > outputdir="${generated.sources}/java" > package="org.apache.hadoop.hbase.generated.master" > webxml="${build.webapps}/master/WEB-INF/web.xml"/> > <mkdir dir="${build.webapps}/regionserver/WEB-INF"/> > <jspcompiler uriroot="${src.webapps}/regionserver" > outputdir="${generated.sources}/java" > package="org.apache.hadoop.hbase.generated.regionserver" > webxml="${build.webapps}/regionserver/WEB-INF/web.xml"/> > <exec executable="sh"> > <arg line="${basedir}/src/saveVersion.sh > ${project.version} ${generated.sources}/java"/> > </exec> > </target> > </configuration> > <goals> > <goal>run</goal> > </goals> > </execution> > </executions> > </plugin> > > -- > Alex Baranau > ------ > Sematext :: http://blog.sematext.com/ :: Hadoop - HBase - ElasticSearch - > Solr >
