I have found that the shade plugin is not completing the job. I manually
updated one class, com.amazonaws.services.s3.model.S3ObjectInputStream,
updating the imports to use the shaded versions of the org.apache.http.*
imports:
import org.shaded.http.client.HttpClient;
import org.shaded.http.client.methods.HttpRequestBase;
import org.shaded.http.conn.EofSensorInputStream;
After making this change, the AWS Java SDK builds fine. I intend to file
an issue against the shade plugin unless anyone can explain what I have
done wrong in my configuration.
Thanks,
Michael
On 6/27/2015 11:22 AM, [email protected] wrote:
Curtis,
I have checked in my code to github:
[email protected]:tribloom/aws-sdk-java.git.
You can check out my shaded version by doing a "git clone
[email protected]:tribloom/aws-sdk-java.git" followed by a "git checkout
-b test origin/1.10.2" ("test" can be changed to whatever branch name
you would like to use locally) in the newly created repository
directory (aws-java-sdk).
You should see three modified files, pom.xml,
./aws-java-sdk-core/pom.xml, and ./aws-java-sdk-s3/pom.xml.
Then run maven to build the project (narrowed down to just
aws-java-sdk-core, aws-java-sdk-kms, and aws-java-sdk-s3 using the top
level pom.xml):
mvn install -DskipTests=true
You should get the core and kms to build properly and then s3 will
fail with:
[INFO] AWS SDK for Java .................................. SUCCESS
[0.380s]
[INFO] AWS SDK for Java - Core ........................... SUCCESS
[10.438s]
[INFO] AWS Java SDK for AWS KMS .......................... SUCCESS
[2.942s]
[INFO] AWS Java SDK for Amazon S3 ........................ FAILURE
[3.733s]
[INFO]
------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO]
------------------------------------------------------------------------
[INFO] Total time: 17.646s
[INFO] Finished at: Sat Jun 27 11:12:55 MDT 2015
[INFO] Final Memory: 33M/518M
[INFO]
------------------------------------------------------------------------
[ERROR] Failed to execute goal
org.apache.maven.plugins:maven-compiler-plugin:2.3:compile
(default-compile) on project aws-java-sdk-s3: Compilation failure
[ERROR]
\dev\CSJ\aws\temp\aws-sdk-java\aws-java-sdk-s3\src\main\java\com\amazonaws\services\s3\internal\S3ObjectResponseHandler.java:[54,26]
error: no suitable constructor found for
S3ObjectInputStream(InputStream,org.shaded.http.client.methods.HttpRequestBase)
The way I understood the shade plugin to work, it would relocate the
dependencies and "rewrite" the code that called those dependencies. I
thought it would "fix" the constructor for S3ObjectInputStream to
match the newly relocated
org.apache.http.client.methods.HttoRequestBase class. If this isn't
the way it should work, how then do I get the shaded project to build?
Thanks,
Michael
On 6/26/2015 5:52 AM, Curtis Rueden wrote:
Hi Michael,
Your error message refers to the maven-compiler-plugin. Are you sure the
shade plugin is the culprit? Normally shade is bound to a later
phase. If
you need further detailed assistance, share your project (or an MCVE)
with
us.
Regards,
Curtis
On Jun 25, 2015 5:16 PM, "[email protected]"
<[email protected]>
wrote:
Matt/Curtis,
I tried using both packages and a higher package with no luck, I get
the
same error message:
[ERROR] Failed to execute goal
org.apache.maven.plugins:maven-compiler-plugin:2.3:compile
(default-compile) on project aws-java-sdk-s3: Compilation
failure[ERROR]
\dev\CSJ\aws\aws-sdk-java-1.10.1\aws-java-sdk-s3\src\main\java\com\amazonaws\services\s3\internal\S3ObjectResponseHandler.java:[54,26]
error: no suitable constructor found for
S3ObjectInputStream(InputStream,org.shaded.http.client.methods.HttpRequestBase)
I tried:
<relocations>
<relocation>
<pattern>org</pattern>
<shadedPattern>shaded</shadedPattern>
</relocation>
</relocations>
and:
<relocations>
<relocation>
<pattern>org.apache</pattern>
<shadedPattern>org.shaded</shadedPattern>
</relocation>
<relocation>
<pattern>org.apache.http</pattern>
<shadedPattern>org.shaded.http</shadedPattern>
</relocation>
<relocation>
<pattern>org.apache.http.methods</pattern>
<shadedPattern>org.shaded.http.methods</shadedPattern>
</relocation>
</relocations>
Has anyone else had luck with trying to shade the AWS Java SDK? Can
anyone
with more experience try and build the project given my
configuration and
tell me the results?
Thanks,
Michael
On 6/25/2015 10:27 AM, Curtis Rueden wrote:
Hi Matt and Michael,
Matt Benson wrote:
In my experience you must relocate each individual package; simply
relocating org.apache.http is not sufficient.
In my experience, the maven-shade-plugin will process subpackages
which
match the given patterns. Here is a working example that shades all of
Jython's dependencies:
https://github.com/scijava/jython-shaded/blob/jython-shaded-2.5.3/deps/pom.xml#L56-L93
Regards,
Curtis
On Thu, Jun 25, 2015 at 7:25 AM, Matt Benson <[email protected]>
wrote:
In my experience you must relocate each individual package; simply
relocating org.apache.http is not sufficient.
Matt
On Jun 25, 2015 6:11 AM, "[email protected]"
<[email protected]
wrote:
I have been attempting to "shade"[1] the AWS Java SDK[2] version
1.10.1,
specifically the S3 SDK, to relocate the Apache HTTPCompontents[3]
libraries. I am attempting to do this in order to resolve a
dependency
conflict with another more complicated project which is also
dependent
on
an older version of HTTPComponents. So far I have been
unsuccessful and
can't determine if it is misconfiguration or an issue with the
plugin. I
hope it is the former.
First I added a Maven Shade configuration to the aws-java-sdk-core
pom.xml:
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-shade-plugin</artifactId>
<version>2.4</version>
<configuration>
<artifactSet>
<includes>
<include>com.amazonaws:*</include>
<include>org.apache.httpcomponents:*</include>
</includes>
</artifactSet>
<relocations>
<relocation>
<pattern>org.apache.http</pattern>
<shadedPattern>org.shaded.http</shadedPattern>
</relocation>
</relocations>
</configuration>
<executions>
<execution>
<phase>package</phase>
<goals>
<goal>shade</goal>
</goals>
</execution>
</executions>
</plugin>
This seems to work fine and produces a jar that seems to be as
expected
when I build the project using "mvn clean install -Dgpg.skip=true
-DskipTests=true", although I haven't tested it. Second, I modify the
aws-java-sdk-s3 pom.xml as follows:
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-shade-plugin</artifactId>
<version>2.4</version>
<configuration>
<shadeSourcesContent>true</shadeSourcesContent>
<createSourcesJar>true</createSourcesJar>
<artifactSet>
<includes>
<include>com.amazonaws:*</include>
<include>org.apache.httpcomponents:*</include>
</includes>
</artifactSet>
<relocations>
<relocation>
<pattern>org.apache.http</pattern>
<shadedPattern>org.shaded.http</shadedPattern>
</relocation>
</relocations>
</configuration>
<executions>
<execution>
<phase>package</phase>
<goals>
<goal>shade</goal>
</goals>
</execution>
</executions>
</plugin>
By itself this produces build errors like this:
[ERROR]
\dev\CSJ\aws\aws-sdk-java-1.10.1\aws-java-sdk-s3\src\main\java\com\amazonaws\services\s3\model\S3ObjectInputStream.java:[20,29]
error: package org.apache.http.client does not exist
So I added an explicit dependency on HTTPComponents to the S3
pom.xml:
<dependency>
<groupId>org.apache.httpcomponents</groupId>
<artifactId>httpclient</artifactId>
<version>4.3.6</version>
</dependency>
When I build I get only one error:
[ERROR]
\dev\CSJ\aws\aws-sdk-java-1.10.1\aws-java-sdk-s3\src\main\java\com\amazonaws\services\s3\internal\S3ObjectResponseHandler.java:[54,26]
error: no suitable constructor found for
S3ObjectInputStream(InputStream,org.shaded.http.client.methods.HttpRequestBase)
It appears to me that the "shading" is not actually relocating the
http.client.methods.HttpRequestBase argument to the
S3ObjectInputStream
class within the code. Running javap confirms this:
$ javap
target/classes/com/amazonaws/services/s3/model/S3ObjectInputStream.class
Picked up JAVA_TOOL_OPTIONS: -Dfile.encoding=UTF8
Compiled from "S3ObjectInputStream.java"
public class com.amazonaws.services.s3.model.S3ObjectInputStream
extends
com.amazonaws.internal.SdkFilterInputStream {
public
com.amazonaws.services.s3.model.S3ObjectInputStream(java.io.InputStream,
org.apache.http.client.methods.HttpRequestBase);
public
com.amazonaws.services.s3.model.S3ObjectInputStream(java.io.InputStream,
org.apache.http.client.methods.HttpRequestBase, boolean);
public void abort();
public org.apache.http.client.methods.HttpRequestBase
getHttpRequest();
public int available() throws java.io.IOException;
}
Can anyone point out if my configuration is wrong, my methodology is
wrong, or the Maven Shade plugin is actually not behaving as
expected?
[1] http://maven.apache.org/plugins/maven-shade-plugin/index.html
[2] https://github.com/aws/aws-sdk-java
[3] https://hc.apache.org/
Thanks,
--
Michael
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
