Repository: incubator-ratis Updated Branches: refs/heads/master 768d7caa4 -> 930f4c019
RATIS-49. Add a clean profile to remove shaded source. Contributed by Tsz Wo Nicholas Sze. Project: http://git-wip-us.apache.org/repos/asf/incubator-ratis/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-ratis/commit/930f4c01 Tree: http://git-wip-us.apache.org/repos/asf/incubator-ratis/tree/930f4c01 Diff: http://git-wip-us.apache.org/repos/asf/incubator-ratis/diff/930f4c01 Branch: refs/heads/master Commit: 930f4c019e3d8acc81376182c941ef887cc8a865 Parents: 768d7ca Author: Jing Zhao <[email protected]> Authored: Tue Mar 28 19:40:01 2017 -0700 Committer: Jing Zhao <[email protected]> Committed: Tue Mar 28 19:40:01 2017 -0700 ---------------------------------------------------------------------- BUILDING.md | 55 +++++++++++++++++++++++++--------------- pom.xml | 2 +- ratis-hadoop-shaded/pom.xml | 42 +++++++++++++++++++++++------- ratis-proto-shaded/pom.xml | 54 +++++++++++++++++++++------------------ 4 files changed, 99 insertions(+), 54 deletions(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/incubator-ratis/blob/930f4c01/BUILDING.md ---------------------------------------------------------------------- diff --git a/BUILDING.md b/BUILDING.md index 4c634a4..42a6f08 100644 --- a/BUILDING.md +++ b/BUILDING.md @@ -1,38 +1,54 @@ # Building -Ratis uses Apache Maven for the builds. A 3.0+ version of Maven is required as well as +Apache Ratis uses Apache Maven for the builds. A 3.2+ version of Maven is required as well as at least Java-1.8. -When Ratis is build for the first time, shaded protobuf files needs to be generated first -which happens only in the mvn package phase. That is why you should run: +When building Ratis the first time, shaded files need to be generated by the following command: ``` $ mvn package -DskipTests ``` -the first time the sources are checked out first. After doing that mvn compile or mvn test -can be used as normal. +After that, `mvn compile` or `mvn test` can be used as normal. +For example, we may run the basic tests by +``` +$ mvn test -Dtest=TestRaftWith\* +``` -# Raft Proto Shaded +# Shading -We shade protos, protobuf and other libraries such as Netty, gRPC and Hadoop so that -applications using Raft may use protobuf and other libraries with versions different -from the versions used here. +We shade protos, protobuf and other libraries such as Netty, gRPC, Guava and Hadoop +so that applications using Ratis may use protobuf and other libraries with versions +different from the versions used here. -The library requires the shaded sources for compilation. The generated sources are stored in -`ratis-proto-shaded/src/main/java/`. They are not checked-in git though. +Ratis requires the shaded sources for compilation. +The generated sources are stored in the following directories. +``` +ratis-proto-shaded/src/main/java/ +ratis-hadoop-shaded/src/main/java/ +``` +They are not checked-in to git though. -If you want to force-compile the proto files (for example after changing them), you should -run with +If you want to force compiling the proto files (for example after changing them), +you should run ``` $ mvn package -DskipTests -Dcompile-protobuf ``` +For removing the shaded sources, run +``` +$ mvn clean -Pclean-shade +``` + +Note that `mvn clean` alone does not remove the shaded sources. + ## What are shaded? -| Original packages | Shaded packages | -| ----------------------------------|----------------------------------------------------------| -| `com.google.protobuf` | `org.apache.ratis.shaded.com.google.protobuf` | -| `io.grpc` | `org.apache.ratis.shaded.io.grpc` | -| `io.netty.handler.codec.protobuf` | `org.apache.ratis.shaded.io.netty.handler.codec.protobuf` | -| `org.apache.hadoop.ipc.protobuf` | `org.apache.ratis.shaded.org.apache.hadoop.ipc.protobuf` | +| Original packages | Shaded packages | +| ------------------------------------|--------------------------------------------------------------| +| `com.google.common` | `org.apache.ratis.shaded.com.google.common` | +| `com.google.protobuf` | `org.apache.ratis.shaded.com.google.protobuf` | +| `com.google.thirdparty.publicsuffix`| `org.apache.ratis.shaded.com.google.thirdparty.publicsuffix` | +| `io.grpc` | `org.apache.ratis.shaded.io.grpc` | +| `io.netty` | `org.apache.ratis.shaded.io.netty` | +| `org.apache.hadoop.ipc.protobuf` | `org.apache.ratis.shaded.org.apache.hadoop.ipc.protobuf` | The protos defined in this project are stored in the `org.apache.ratis.shaded.proto` package. @@ -53,7 +69,6 @@ To publish, use the following settings.xml file ( placed in ~/.m2/settings.xml ) <username>ratis_committer</username> <password>********</password> </server> - </servers> </settings> ``` http://git-wip-us.apache.org/repos/asf/incubator-ratis/blob/930f4c01/pom.xml ---------------------------------------------------------------------- diff --git a/pom.xml b/pom.xml index 7e0869f..ea4f72b 100644 --- a/pom.xml +++ b/pom.xml @@ -134,7 +134,7 @@ <maven-assembly-plugin.version>2.4</maven-assembly-plugin.version> <maven-bundle-plugin.version>2.5.3</maven-bundle-plugin.version> <maven-checkstyle-plugin.version>2.15</maven-checkstyle-plugin.version> - <maven-clean-plugin.version>2.5</maven-clean-plugin.version> + <maven-clean-plugin.version>3.0.0</maven-clean-plugin.version> <maven-clover2-plugin.version>3.3.0</maven-clover2-plugin.version> <maven-compiler-plugin.version>3.2</maven-compiler-plugin.version> <maven-dependency-plugin.version>2.10</maven-dependency-plugin.version> http://git-wip-us.apache.org/repos/asf/incubator-ratis/blob/930f4c01/ratis-hadoop-shaded/pom.xml ---------------------------------------------------------------------- diff --git a/ratis-hadoop-shaded/pom.xml b/ratis-hadoop-shaded/pom.xml index cb74fa2..0321a22 100644 --- a/ratis-hadoop-shaded/pom.xml +++ b/ratis-hadoop-shaded/pom.xml @@ -32,7 +32,7 @@ <!--The Default target dir--> <classes.dir>${project.build.directory}/classes</classes.dir> <!--The Default location for sources--> - <sources.dir>src/main/java</sources.dir> + <shaded.sources.dir>src/main/java</shaded.sources.dir> <!--Version of Hadoop to be shaded --> <shaded.hadoop.version>${hadoop.version}</shaded.hadoop.version> @@ -41,7 +41,7 @@ <build> <!--I want to override these in profile so define them with variables up here--> - <sourceDirectory>${sources.dir}</sourceDirectory> + <sourceDirectory>${shaded.sources.dir}</sourceDirectory> <outputDirectory>${classes.dir}</outputDirectory> <plugins> <!-- Make a jar and put the sources in the jar --> @@ -72,17 +72,20 @@ <groupId>com.google.protobuf</groupId> <artifactId>protobuf-java</artifactId> <version>${shaded.protobuf.version}</version> + <optional>true</optional> </dependency> <dependency> <groupId>com.google.protobuf.nano</groupId> <artifactId>protobuf-javanano</artifactId> <version>${shaded.protobuf.version}</version> + <optional>true</optional> </dependency> <dependency> <groupId>org.apache.hadoop</groupId> <artifactId>hadoop-common</artifactId> <version>${shaded.hadoop.version}</version> + <optional>true</optional> </dependency> </dependencies> @@ -101,19 +104,41 @@ </profile> <profile> - <id>skipShade</id> + <id>skipCompileProto</id> <activation> <file> - <exists>${sources.dir}</exists> + <exists>${shaded.sources.dir}</exists> </file> <property> - <name>!shade-hadoop</name> + <name>!compile-protobuf</name> </property> </activation> </profile> <profile> - <id>shade-hadoop</id> + <id>clean-shade</id> + <activation> + <activeByDefault>false</activeByDefault> + </activation> + <build> + <plugins> + <plugin> + <artifactId>maven-clean-plugin</artifactId> + <configuration> + <filesets> + <fileset> + <directory>${shaded.sources.dir}</directory> + <followSymlinks>false</followSymlinks> + </fileset> + </filesets> + </configuration> + </plugin> + </plugins> + </build> + </profile> + + <profile> + <id>compile-protobuf</id> <!-- Shade and drop the generated java files under src/main/java. --> @@ -121,7 +146,7 @@ <activeByDefault>true</activeByDefault> </activation> <properties> - <profile.id>shade-hadoop</profile.id> + <profile.id>compile-protobuf</profile.id> <!--When the compile for this profile runs, make sure it makes jars that can be related back to this shading profile. Give them a shading prefix. --> @@ -273,8 +298,7 @@ <classifier>sources</classifier> <type>jar</type> <overWrite>true</overWrite> - <outputDirectory>${basedir}/src/main/java - </outputDirectory> + <outputDirectory>${basedir}/${shaded.sources.dir}</outputDirectory> <includes>**/*.java</includes> </artifactItem> </artifactItems> http://git-wip-us.apache.org/repos/asf/incubator-ratis/blob/930f4c01/ratis-proto-shaded/pom.xml ---------------------------------------------------------------------- diff --git a/ratis-proto-shaded/pom.xml b/ratis-proto-shaded/pom.xml index 76148c4..dc32403 100644 --- a/ratis-proto-shaded/pom.xml +++ b/ratis-proto-shaded/pom.xml @@ -32,8 +32,7 @@ <!--The Default target dir--> <classes.dir>${project.build.directory}/classes</classes.dir> <!--The Default location for sources--> - <sources.dir>src/main/java</sources.dir> - + <shaded.sources.dir>src/main/java</shaded.sources.dir> <!--Version of grpc to be shaded --> <shaded.grpc.version>1.0.1</shaded.grpc.version> @@ -44,7 +43,7 @@ <build> <!--I want to override these in profile so define them with variables up here--> - <sourceDirectory>${sources.dir}</sourceDirectory> + <sourceDirectory>${shaded.sources.dir}</sourceDirectory> <outputDirectory>${classes.dir}</outputDirectory> <plugins> <!-- Make a jar and put the sources in the jar --> @@ -372,10 +371,10 @@ <profiles> <!-- Skip the tests in this module --> <profile> - <id>skip-proto-shaded-tests</id> + <id>skip-shaded-tests</id> <activation> <property> - <name>skip-proto-shaded-tests</name> + <name>skip-shaded-tests</name> </property> </activation> <properties> @@ -387,7 +386,7 @@ <id>skipCompileProto</id> <activation> <file> - <exists>${sources.dir}</exists> + <exists>${shaded.sources.dir}</exists> </file> <property> <name>!compile-protobuf</name> @@ -396,25 +395,33 @@ </profile> <profile> + <id>clean-shade</id> + <activation> + <activeByDefault>false</activeByDefault> + </activation> + <build> + <plugins> + <plugin> + <artifactId>maven-clean-plugin</artifactId> + <configuration> + <filesets> + <fileset> + <directory>${shaded.sources.dir}</directory> + <followSymlinks>false</followSymlinks> + </fileset> + </filesets> + </configuration> + </plugin> + </plugins> + </build> + </profile> + + <profile> <id>compile-protobuf</id> <!-- - Generate and shade proto files. Drops generated java files - under src/main/java. Check in the generated files so available - at build time. Run this profile/step everytime you change proto + Shade and drop the generated java files under src/main/java. + Run this profile/step everytime you change proto files or update the protobuf version. - - The below does a bunch of ugly stuff. It purges current content - of the generated and shaded com.google.protobuf java files first. - It does this because later we apply patches later and patches - fail they've already been applied. We remove too because we - overlay the shaded protobuf and if files have been removed or - added, it'll be more plain if we have first done this delete. - - Next up we generate proto, build a jar, shade it (which - includes the referenced protobuf), undo it over the src/main/java - directory, and then apply patches. - - The result needs to be checked in. --> <activation> <activeByDefault>true</activeByDefault> @@ -594,8 +601,7 @@ <classifier>sources</classifier> <type>jar</type> <overWrite>true</overWrite> - <outputDirectory>${basedir}/src/main/java - </outputDirectory> + <outputDirectory>${basedir}/${shaded.sources.dir}</outputDirectory> <includes>**/*.java</includes> </artifactItem> </artifactItems>
