Hi Michael, Sorry, I've been too busy to follow up on this in detail.
My strong intuition is that you don't need to hack the Amazon S3 source code or POMs at all. Rather, you can create another project downstream whose whole purpose is to shade the contents of com.amazonaws:aws-java-sdk-s3 as you desire, producing an uber-jar artifact with GAV of your choosing. This is how I shaded org.python:jython-standalone to fix some of its embedded libraries. In your case, you'd be creating an uber-jar for aws-java-sdk-s3 that includes all (or some) of its dependencies, some of which (of your choosing) get shaded/renamed. I pushed the start of such a project: https://github.com/ctrueden/aws-java-sdk-s3-shaded Unfortunately, I don't have time to get it working right now. But maybe you can solve it. I am not convinced this is a problem with Shade (though I suppose it might be). Regards, Curtis On Sat, Jun 27, 2015 at 12:22 PM, mmccar...@tribloom.com < mmccar...@tribloom.com> wrote: > Curtis, > > I have checked in my code to github: g...@github.com: > tribloom/aws-sdk-java.git. > > You can check out my shaded version by doing a "git clone > g...@github.com:tribloom/aws-sdk-java.git" > followed by a "git checkout -b test origin/1.10.2" ("test" can be changed > to whatever branch name you would like to use locally) in the newly created > repository directory (aws-java-sdk). > > You should see three modified files, pom.xml, ./aws-java-sdk-core/pom.xml, > and ./aws-java-sdk-s3/pom.xml. > > Then run maven to build the project (narrowed down to just > aws-java-sdk-core, aws-java-sdk-kms, and aws-java-sdk-s3 using the top > level pom.xml): > mvn install -DskipTests=true > > You should get the core and kms to build properly and then s3 will fail > with: > [INFO] AWS SDK for Java .................................. SUCCESS [0.380s] > [INFO] AWS SDK for Java - Core ........................... SUCCESS > [10.438s] > [INFO] AWS Java SDK for AWS KMS .......................... SUCCESS [2.942s] > [INFO] AWS Java SDK for Amazon S3 ........................ FAILURE [3.733s] > [INFO] > ------------------------------------------------------------------------ > [INFO] BUILD FAILURE > [INFO] > ------------------------------------------------------------------------ > [INFO] Total time: 17.646s > [INFO] Finished at: Sat Jun 27 11:12:55 MDT 2015 > [INFO] Final Memory: 33M/518M > [INFO] > ------------------------------------------------------------------------ > [ERROR] Failed to execute goal > org.apache.maven.plugins:maven-compiler-plugin:2.3:compile > (default-compile) on project aws-java-sdk-s3: Compilation failure > [ERROR] > \dev\CSJ\aws\temp\aws-sdk-java\aws-java-sdk-s3\src\main\java\com\amazonaws\services\s3\internal\S3ObjectResponseHandler.java:[54,26] > error: no suitable constructor found for > S3ObjectInputStream(InputStream,org.shaded.http.client.methods.HttpRequestBase) > > The way I understood the shade plugin to work, it would relocate the > dependencies and "rewrite" the code that called those dependencies. I > thought it would "fix" the constructor for S3ObjectInputStream to match the > newly relocated org.apache.http.client.methods.HttoRequestBase class. If > this isn't the way it should work, how then do I get the shaded project to > build? > > Thanks, > Michael > > On 6/26/2015 5:52 AM, Curtis Rueden wrote: > >> Hi Michael, >> >> Your error message refers to the maven-compiler-plugin. Are you sure the >> shade plugin is the culprit? Normally shade is bound to a later phase. If >> you need further detailed assistance, share your project (or an MCVE) with >> us. >> >> Regards, >> Curtis >> On Jun 25, 2015 5:16 PM, "mmccar...@tribloom.com" <mmccar...@tribloom.com >> > >> wrote: >> >> Matt/Curtis, >>> >>> I tried using both packages and a higher package with no luck, I get the >>> same error message: >>> [ERROR] Failed to execute goal >>> org.apache.maven.plugins:maven-compiler-plugin:2.3:compile >>> (default-compile) on project aws-java-sdk-s3: Compilation failure[ERROR] >>> >>> \dev\CSJ\aws\aws-sdk-java-1.10.1\aws-java-sdk-s3\src\main\java\com\amazonaws\services\s3\internal\S3ObjectResponseHandler.java:[54,26] >>> error: no suitable constructor found for >>> >>> S3ObjectInputStream(InputStream,org.shaded.http.client.methods.HttpRequestBase) >>> >>> I tried: >>> <relocations> >>> <relocation> >>> <pattern>org</pattern> >>> <shadedPattern>shaded</shadedPattern> >>> </relocation> >>> </relocations> >>> and: >>> <relocations> >>> <relocation> >>> <pattern>org.apache</pattern> >>> <shadedPattern>org.shaded</shadedPattern> >>> </relocation> >>> <relocation> >>> <pattern>org.apache.http</pattern> >>> >>> <shadedPattern>org.shaded.http</shadedPattern> >>> </relocation> >>> <relocation> >>> <pattern>org.apache.http.methods</pattern> >>> >>> <shadedPattern>org.shaded.http.methods</shadedPattern> >>> </relocation> >>> </relocations> >>> >>> Has anyone else had luck with trying to shade the AWS Java SDK? Can >>> anyone >>> with more experience try and build the project given my configuration and >>> tell me the results? >>> >>> Thanks, >>> Michael >>> On 6/25/2015 10:27 AM, Curtis Rueden wrote: >>> >>> Hi Matt and Michael, >>> >>> Matt Benson wrote: >>> >>>> In my experience you must relocate each individual package; simply >>>> relocating org.apache.http is not sufficient. >>>> >>> In my experience, the maven-shade-plugin will process subpackages which >>> match the given patterns. Here is a working example that shades all of >>> Jython's dependencies: >>> >>> >>> >>> https://github.com/scijava/jython-shaded/blob/jython-shaded-2.5.3/deps/pom.xml#L56-L93 >>> >>> Regards, >>> Curtis >>> >>> On Thu, Jun 25, 2015 at 7:25 AM, Matt Benson <gudnabr...@gmail.com> >>> wrote: >>> >>> In my experience you must relocate each individual package; simply >>>> relocating org.apache.http is not sufficient. >>>> >>>> Matt >>>> On Jun 25, 2015 6:11 AM, "mmccar...@tribloom.com" < >>>> mmccar...@tribloom.com >>>> wrote: >>>> >>>> I have been attempting to "shade"[1] the AWS Java SDK[2] version >>>>> 1.10.1, >>>>> specifically the S3 SDK, to relocate the Apache HTTPCompontents[3] >>>>> libraries. I am attempting to do this in order to resolve a dependency >>>>> conflict with another more complicated project which is also dependent >>>>> >>>> on >>>> >>>>> an older version of HTTPComponents. So far I have been unsuccessful and >>>>> can't determine if it is misconfiguration or an issue with the plugin. >>>>> I >>>>> hope it is the former. >>>>> >>>>> First I added a Maven Shade configuration to the aws-java-sdk-core >>>>> >>>> pom.xml: >>>> >>>>> <plugin> >>>>> <groupId>org.apache.maven.plugins</groupId> >>>>> <artifactId>maven-shade-plugin</artifactId> >>>>> <version>2.4</version> >>>>> <configuration> >>>>> <artifactSet> >>>>> <includes> >>>>> <include>com.amazonaws:*</include> >>>>> <include>org.apache.httpcomponents:*</include> >>>>> </includes> >>>>> </artifactSet> >>>>> <relocations> >>>>> <relocation> >>>>> <pattern>org.apache.http</pattern> >>>>> <shadedPattern>org.shaded.http</shadedPattern> >>>>> </relocation> >>>>> </relocations> >>>>> </configuration> >>>>> <executions> >>>>> <execution> >>>>> <phase>package</phase> >>>>> <goals> >>>>> <goal>shade</goal> >>>>> </goals> >>>>> </execution> >>>>> </executions> >>>>> </plugin> >>>>> >>>>> This seems to work fine and produces a jar that seems to be as expected >>>>> when I build the project using "mvn clean install -Dgpg.skip=true >>>>> -DskipTests=true", although I haven't tested it. Second, I modify the >>>>> aws-java-sdk-s3 pom.xml as follows: >>>>> <plugin> >>>>> <groupId>org.apache.maven.plugins</groupId> >>>>> <artifactId>maven-shade-plugin</artifactId> >>>>> <version>2.4</version> >>>>> <configuration> >>>>> <shadeSourcesContent>true</shadeSourcesContent> >>>>> <createSourcesJar>true</createSourcesJar> >>>>> <artifactSet> >>>>> <includes> >>>>> <include>com.amazonaws:*</include> >>>>> <include>org.apache.httpcomponents:*</include> >>>>> </includes> >>>>> </artifactSet> >>>>> <relocations> >>>>> <relocation> >>>>> <pattern>org.apache.http</pattern> >>>>> <shadedPattern>org.shaded.http</shadedPattern> >>>>> </relocation> >>>>> </relocations> >>>>> </configuration> >>>>> <executions> >>>>> <execution> >>>>> <phase>package</phase> >>>>> <goals> >>>>> <goal>shade</goal> >>>>> </goals> >>>>> </execution> >>>>> </executions> >>>>> </plugin> >>>>> >>>>> By itself this produces build errors like this: >>>>> [ERROR] >>>>> >>>>> >>>> \dev\CSJ\aws\aws-sdk-java-1.10.1\aws-java-sdk-s3\src\main\java\com\amazonaws\services\s3\model\S3ObjectInputStream.java:[20,29] >>>> >>>>> error: package org.apache.http.client does not exist >>>>> >>>>> So I added an explicit dependency on HTTPComponents to the S3 pom.xml: >>>>> <dependency> >>>>> <groupId>org.apache.httpcomponents</groupId> >>>>> <artifactId>httpclient</artifactId> >>>>> <version>4.3.6</version> >>>>> </dependency> >>>>> >>>>> When I build I get only one error: >>>>> [ERROR] >>>>> >>>>> >>>> \dev\CSJ\aws\aws-sdk-java-1.10.1\aws-java-sdk-s3\src\main\java\com\amazonaws\services\s3\internal\S3ObjectResponseHandler.java:[54,26] >>>> >>>>> error: no suitable constructor found for >>>>> >>>>> >>>> S3ObjectInputStream(InputStream,org.shaded.http.client.methods.HttpRequestBase) >>>> >>>>> It appears to me that the "shading" is not actually relocating the >>>>> http.client.methods.HttpRequestBase argument to the S3ObjectInputStream >>>>> class within the code. Running javap confirms this: >>>>> $ javap >>>>> >>>>> target/classes/com/amazonaws/services/s3/model/S3ObjectInputStream.class >>>>> Picked up JAVA_TOOL_OPTIONS: -Dfile.encoding=UTF8 >>>>> Compiled from "S3ObjectInputStream.java" >>>>> public class com.amazonaws.services.s3.model.S3ObjectInputStream >>>>> extends >>>>> com.amazonaws.internal.SdkFilterInputStream { >>>>> public >>>>> >>>>> com.amazonaws.services.s3.model.S3ObjectInputStream(java.io.InputStream, >>>>> org.apache.http.client.methods.HttpRequestBase); >>>>> public >>>>> >>>>> com.amazonaws.services.s3.model.S3ObjectInputStream(java.io.InputStream, >>>>> org.apache.http.client.methods.HttpRequestBase, boolean); >>>>> public void abort(); >>>>> public org.apache.http.client.methods.HttpRequestBase >>>>> >>>> getHttpRequest(); >>>> >>>>> public int available() throws java.io.IOException; >>>>> } >>>>> >>>>> Can anyone point out if my configuration is wrong, my methodology is >>>>> wrong, or the Maven Shade plugin is actually not behaving as expected? >>>>> >>>>> [1] http://maven.apache.org/plugins/maven-shade-plugin/index.html >>>>> [2] https://github.com/aws/aws-sdk-java >>>>> [3] https://hc.apache.org/ >>>>> >>>>> Thanks, >>>>> -- >>>>> Michael >>>>> >>>>> --------------------------------------------------------------------- >>>>> To unsubscribe, e-mail: users-unsubscr...@maven.apache.org >>>>> For additional commands, e-mail: users-h...@maven.apache.org >>>>> >>>>> >>>>> >>> >>> >
