Curtis,
I was able to get it working. Here is the final pom.xml. Thanks for all
of your help.
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0";
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance";
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0
http://maven.apache.org/xsd/maven-4.0.0.xsd";>
<modelVersion>4.0.0</modelVersion>
<artifactId>aws-java-sdk-s3-shaded</artifactId>
<groupId>com.amazonaws</groupId>
<version>1.10.2-SNAPSHOT</version>
<properties>
<aws.version>1.10.2</aws.version>
<shade.prefix>shaded</shade.prefix>
</properties>
<dependencies>
<dependency>
<groupId>com.amazonaws</groupId>
<artifactId>aws-java-sdk-s3</artifactId>
<version>${aws.version}</version>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<artifactId>maven-shade-plugin</artifactId>
<version>2.4</version>
<configuration>
<artifactSet>
<includes>
<include>com.amazonaws:*</include>
<include>org.apache.httpcomponents:*</include>
</includes>
</artifactSet>
<relocations>
<relocation>
<pattern>org.apache</pattern>
<shadedPattern>shaded.org.apache</shadedPattern>
</relocation>
</relocations>
</configuration>
<executions>
<execution>
<phase>package</phase>
<goals>
<goal>shade</goal>
</goals>
</execution>
</executions>
</plugin>
</plugins>
</build>
</project>
Michael
On 7/2/2015 2:37 PM, Curtis Rueden wrote:
Hi Michael,
Sorry, I've been too busy to follow up on this in detail.
My strong intuition is that you don't need to hack the Amazon S3 source
code or POMs at all. Rather, you can create another project downstream
whose whole purpose is to shade the contents of
com.amazonaws:aws-java-sdk-s3 as you desire, producing an uber-jar artifact
with GAV of your choosing.
This is how I shaded org.python:jython-standalone to fix some of its
embedded libraries. In your case, you'd be creating an uber-jar for
aws-java-sdk-s3 that includes all (or some) of its dependencies, some of
which (of your choosing) get shaded/renamed.
I pushed the start of such a project:
https://github.com/ctrueden/aws-java-sdk-s3-shaded
Unfortunately, I don't have time to get it working right now. But maybe you
can solve it. I am not convinced this is a problem with Shade (though I
suppose it might be).
Regards,
Curtis
On Sat, Jun 27, 2015 at 12:22 PM, [email protected] <
[email protected]> wrote:
Curtis,
I have checked in my code to github: [email protected]:
tribloom/aws-sdk-java.git.
You can check out my shaded version by doing a "git clone
[email protected]:tribloom/aws-sdk-java.git"
followed by a "git checkout -b test origin/1.10.2" ("test" can be changed
to whatever branch name you would like to use locally) in the newly created
repository directory (aws-java-sdk).
You should see three modified files, pom.xml, ./aws-java-sdk-core/pom.xml,
and ./aws-java-sdk-s3/pom.xml.
Then run maven to build the project (narrowed down to just
aws-java-sdk-core, aws-java-sdk-kms, and aws-java-sdk-s3 using the top
level pom.xml):
mvn install -DskipTests=true
You should get the core and kms to build properly and then s3 will fail
with:
[INFO] AWS SDK for Java .................................. SUCCESS [0.380s]
[INFO] AWS SDK for Java - Core ........................... SUCCESS
[10.438s]
[INFO] AWS Java SDK for AWS KMS .......................... SUCCESS [2.942s]
[INFO] AWS Java SDK for Amazon S3 ........................ FAILURE [3.733s]
[INFO]
------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO]
------------------------------------------------------------------------
[INFO] Total time: 17.646s
[INFO] Finished at: Sat Jun 27 11:12:55 MDT 2015
[INFO] Final Memory: 33M/518M
[INFO]
------------------------------------------------------------------------
[ERROR] Failed to execute goal
org.apache.maven.plugins:maven-compiler-plugin:2.3:compile
(default-compile) on project aws-java-sdk-s3: Compilation failure
[ERROR]
\dev\CSJ\aws\temp\aws-sdk-java\aws-java-sdk-s3\src\main\java\com\amazonaws\services\s3\internal\S3ObjectResponseHandler.java:[54,26]
error: no suitable constructor found for
S3ObjectInputStream(InputStream,org.shaded.http.client.methods.HttpRequestBase)
The way I understood the shade plugin to work, it would relocate the
dependencies and "rewrite" the code that called those dependencies. I
thought it would "fix" the constructor for S3ObjectInputStream to match the
newly relocated org.apache.http.client.methods.HttoRequestBase class. If
this isn't the way it should work, how then do I get the shaded project to
build?
Thanks,
Michael
On 6/26/2015 5:52 AM, Curtis Rueden wrote:
Hi Michael,
Your error message refers to the maven-compiler-plugin. Are you sure the
shade plugin is the culprit? Normally shade is bound to a later phase. If
you need further detailed assistance, share your project (or an MCVE) with
us.
Regards,
Curtis
On Jun 25, 2015 5:16 PM, "[email protected]" <[email protected]
wrote:
Matt/Curtis,
I tried using both packages and a higher package with no luck, I get the
same error message:
[ERROR] Failed to execute goal
org.apache.maven.plugins:maven-compiler-plugin:2.3:compile
(default-compile) on project aws-java-sdk-s3: Compilation failure[ERROR]
\dev\CSJ\aws\aws-sdk-java-1.10.1\aws-java-sdk-s3\src\main\java\com\amazonaws\services\s3\internal\S3ObjectResponseHandler.java:[54,26]
error: no suitable constructor found for
S3ObjectInputStream(InputStream,org.shaded.http.client.methods.HttpRequestBase)
I tried:
<relocations>
<relocation>
<pattern>org</pattern>
<shadedPattern>shaded</shadedPattern>
</relocation>
</relocations>
and:
<relocations>
<relocation>
<pattern>org.apache</pattern>
<shadedPattern>org.shaded</shadedPattern>
</relocation>
<relocation>
<pattern>org.apache.http</pattern>
<shadedPattern>org.shaded.http</shadedPattern>
</relocation>
<relocation>
<pattern>org.apache.http.methods</pattern>
<shadedPattern>org.shaded.http.methods</shadedPattern>
</relocation>
</relocations>
Has anyone else had luck with trying to shade the AWS Java SDK? Can
anyone
with more experience try and build the project given my configuration and
tell me the results?
Thanks,
Michael
On 6/25/2015 10:27 AM, Curtis Rueden wrote:
Hi Matt and Michael,
Matt Benson wrote:
In my experience you must relocate each individual package; simply
relocating org.apache.http is not sufficient.
In my experience, the maven-shade-plugin will process subpackages which
match the given patterns. Here is a working example that shades all of
Jython's dependencies:
https://github.com/scijava/jython-shaded/blob/jython-shaded-2.5.3/deps/pom.xml#L56-L93
Regards,
Curtis
On Thu, Jun 25, 2015 at 7:25 AM, Matt Benson <[email protected]>
wrote:
In my experience you must relocate each individual package; simply
relocating org.apache.http is not sufficient.
Matt
On Jun 25, 2015 6:11 AM, "[email protected]" <
[email protected]
wrote:
I have been attempting to "shade"[1] the AWS Java SDK[2] version
1.10.1,
specifically the S3 SDK, to relocate the Apache HTTPCompontents[3]
libraries. I am attempting to do this in order to resolve a dependency
conflict with another more complicated project which is also dependent
on
an older version of HTTPComponents. So far I have been unsuccessful and
can't determine if it is misconfiguration or an issue with the plugin.
I
hope it is the former.
First I added a Maven Shade configuration to the aws-java-sdk-core
pom.xml:
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-shade-plugin</artifactId>
<version>2.4</version>
<configuration>
<artifactSet>
<includes>
<include>com.amazonaws:*</include>
<include>org.apache.httpcomponents:*</include>
</includes>
</artifactSet>
<relocations>
<relocation>
<pattern>org.apache.http</pattern>
<shadedPattern>org.shaded.http</shadedPattern>
</relocation>
</relocations>
</configuration>
<executions>
<execution>
<phase>package</phase>
<goals>
<goal>shade</goal>
</goals>
</execution>
</executions>
</plugin>
This seems to work fine and produces a jar that seems to be as expected
when I build the project using "mvn clean install -Dgpg.skip=true
-DskipTests=true", although I haven't tested it. Second, I modify the
aws-java-sdk-s3 pom.xml as follows:
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-shade-plugin</artifactId>
<version>2.4</version>
<configuration>
<shadeSourcesContent>true</shadeSourcesContent>
<createSourcesJar>true</createSourcesJar>
<artifactSet>
<includes>
<include>com.amazonaws:*</include>
<include>org.apache.httpcomponents:*</include>
</includes>
</artifactSet>
<relocations>
<relocation>
<pattern>org.apache.http</pattern>
<shadedPattern>org.shaded.http</shadedPattern>
</relocation>
</relocations>
</configuration>
<executions>
<execution>
<phase>package</phase>
<goals>
<goal>shade</goal>
</goals>
</execution>
</executions>
</plugin>
By itself this produces build errors like this:
[ERROR]
\dev\CSJ\aws\aws-sdk-java-1.10.1\aws-java-sdk-s3\src\main\java\com\amazonaws\services\s3\model\S3ObjectInputStream.java:[20,29]
error: package org.apache.http.client does not exist
So I added an explicit dependency on HTTPComponents to the S3 pom.xml:
<dependency>
<groupId>org.apache.httpcomponents</groupId>
<artifactId>httpclient</artifactId>
<version>4.3.6</version>
</dependency>
When I build I get only one error:
[ERROR]
\dev\CSJ\aws\aws-sdk-java-1.10.1\aws-java-sdk-s3\src\main\java\com\amazonaws\services\s3\internal\S3ObjectResponseHandler.java:[54,26]
error: no suitable constructor found for
S3ObjectInputStream(InputStream,org.shaded.http.client.methods.HttpRequestBase)
It appears to me that the "shading" is not actually relocating the
http.client.methods.HttpRequestBase argument to the S3ObjectInputStream
class within the code. Running javap confirms this:
$ javap
target/classes/com/amazonaws/services/s3/model/S3ObjectInputStream.class
Picked up JAVA_TOOL_OPTIONS: -Dfile.encoding=UTF8
Compiled from "S3ObjectInputStream.java"
public class com.amazonaws.services.s3.model.S3ObjectInputStream
extends
com.amazonaws.internal.SdkFilterInputStream {
public
com.amazonaws.services.s3.model.S3ObjectInputStream(java.io.InputStream,
org.apache.http.client.methods.HttpRequestBase);
public
com.amazonaws.services.s3.model.S3ObjectInputStream(java.io.InputStream,
org.apache.http.client.methods.HttpRequestBase, boolean);
public void abort();
public org.apache.http.client.methods.HttpRequestBase
getHttpRequest();
public int available() throws java.io.IOException;
}
Can anyone point out if my configuration is wrong, my methodology is
wrong, or the Maven Shade plugin is actually not behaving as expected?
[1] http://maven.apache.org/plugins/maven-shade-plugin/index.html
[2] https://github.com/aws/aws-sdk-java
[3] https://hc.apache.org/
Thanks,
--
Michael
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
