[jira] [Commented] (FLINK-1605) Create a shaded Hadoop fat jar to resolve library version conflicts
[ https://issues.apache.org/jira/browse/FLINK-1605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15509539#comment-15509539 ] Renkai Ge commented on FLINK-1605: -- Hi [~rmetzger], Could you help take a view at this issue https://issues.apache.org/jira/browse/FLINK-4587?filter=-2 and this problem? http://stackoverflow.com/questions/39610268/different-result-of-mvn-clean-package-when-execute-in-different-directory Thanks a lot! > Create a shaded Hadoop fat jar to resolve library version conflicts > --- > > Key: FLINK-1605 > URL: https://issues.apache.org/jira/browse/FLINK-1605 > Project: Flink > Issue Type: Improvement > Components: Build System >Affects Versions: 0.9 >Reporter: Robert Metzger >Assignee: Robert Metzger > Fix For: 0.9 > > > As per mailing list discussion: > http://apache-flink-incubator-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Create-a-shaded-Hadoop-fat-jar-to-resolve-library-version-conflicts-td3881.html -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (FLINK-1605) Create a shaded Hadoop fat jar to resolve library version conflicts
[ https://issues.apache.org/jira/browse/FLINK-1605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14360144#comment-14360144 ] ASF GitHub Bot commented on FLINK-1605: --- Github user rmetzger commented on the pull request: https://github.com/apache/flink/pull/454#issuecomment-78889015 Hey Henry, I'm sorry that the change is making your builds so much slower. My linux machine needs 5 minutes to build the current master. I suspect there is something wrong on your MBP setup. Create a shaded Hadoop fat jar to resolve library version conflicts --- Key: FLINK-1605 URL: https://issues.apache.org/jira/browse/FLINK-1605 Project: Flink Issue Type: Improvement Components: Build System Affects Versions: 0.9 Reporter: Robert Metzger Assignee: Robert Metzger As per mailing list discussion: http://apache-flink-incubator-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Create-a-shaded-Hadoop-fat-jar-to-resolve-library-version-conflicts-td3881.html -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (FLINK-1605) Create a shaded Hadoop fat jar to resolve library version conflicts
[
https://issues.apache.org/jira/browse/FLINK-1605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14356656#comment-14356656
]
ASF GitHub Bot commented on FLINK-1605:
---
Github user tillrohrmann commented on a diff in the pull request:
https://github.com/apache/flink/pull/454#discussion_r26199913
--- Diff: pom.xml ---
@@ -1159,6 +726,55 @@ under the License.
/execution
/executions
/plugin
+
+ !-- We use shading in all packages for relocating some
classes, such as
+ Guava and ASM.
+ By doing so, users adding Flink as a dependency
won't run into conflicts.
+ (For example users can use whatever guava
version they want, because we don't
+ expose our guava dependency)
+ --
+ plugin
+ groupIdorg.apache.maven.plugins/groupId
+ artifactIdmaven-shade-plugin/artifactId
+ version2.3/version
+ executions
+ execution
+ idshade-flink/id
+ phasepackage/phase
+ goals
+ goalshade/goal
+ /goals
+ configuration
+
shadeTestJartrue/shadeTestJar
+
shadedArtifactAttachedfalse/shadedArtifactAttached
+
createDependencyReducedPomtrue/createDependencyReducedPom
+
dependencyReducedPomLocation${project.basedir}/target/dependency-reduced-pom.xml/dependencyReducedPomLocation
+ artifactSet
+ includes
+
includecom.google.guava:*/include
+
includeorg.ow2.asm:*/include
+ !--
includeorg.apache.flume:*/include Required because flume depends on guava.
This only affects the flink-streaming connector which is not shipped through
flink-dist by default. So its only about users adding flume as a dependency. --
--- End diff --
Why is it commented out rather than removed?
Create a shaded Hadoop fat jar to resolve library version conflicts
---
Key: FLINK-1605
URL: https://issues.apache.org/jira/browse/FLINK-1605
Project: Flink
Issue Type: Improvement
Components: Build System
Affects Versions: 0.9
Reporter: Robert Metzger
Assignee: Robert Metzger
As per mailing list discussion:
http://apache-flink-incubator-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Create-a-shaded-Hadoop-fat-jar-to-resolve-library-version-conflicts-td3881.html
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (FLINK-1605) Create a shaded Hadoop fat jar to resolve library version conflicts
[ https://issues.apache.org/jira/browse/FLINK-1605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14359139#comment-14359139 ] ASF GitHub Bot commented on FLINK-1605: --- Github user hsaputra commented on the pull request: https://github.com/apache/flink/pull/454#issuecomment-78560058 How much time does this new shading add to the total compile? It used to be around 16-18mins for me using mvn clean install -DskipTests. I just did merge today and it has been more than 25 mins and has not complete the build =( Create a shaded Hadoop fat jar to resolve library version conflicts --- Key: FLINK-1605 URL: https://issues.apache.org/jira/browse/FLINK-1605 Project: Flink Issue Type: Improvement Components: Build System Affects Versions: 0.9 Reporter: Robert Metzger Assignee: Robert Metzger As per mailing list discussion: http://apache-flink-incubator-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Create-a-shaded-Hadoop-fat-jar-to-resolve-library-version-conflicts-td3881.html -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (FLINK-1605) Create a shaded Hadoop fat jar to resolve library version conflicts
[ https://issues.apache.org/jira/browse/FLINK-1605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14356717#comment-14356717 ] ASF GitHub Bot commented on FLINK-1605: --- Github user asfgit closed the pull request at: https://github.com/apache/flink/pull/454 Create a shaded Hadoop fat jar to resolve library version conflicts --- Key: FLINK-1605 URL: https://issues.apache.org/jira/browse/FLINK-1605 Project: Flink Issue Type: Improvement Components: Build System Affects Versions: 0.9 Reporter: Robert Metzger Assignee: Robert Metzger As per mailing list discussion: http://apache-flink-incubator-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Create-a-shaded-Hadoop-fat-jar-to-resolve-library-version-conflicts-td3881.html -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (FLINK-1605) Create a shaded Hadoop fat jar to resolve library version conflicts
[
https://issues.apache.org/jira/browse/FLINK-1605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14356650#comment-14356650
]
ASF GitHub Bot commented on FLINK-1605:
---
Github user tillrohrmann commented on a diff in the pull request:
https://github.com/apache/flink/pull/454#discussion_r26199162
--- Diff: flink-shaded-hadoop/pom.xml ---
@@ -60,19 +73,36 @@ under the License.
shadedArtifactAttachedfalse/shadedArtifactAttached
createDependencyReducedPomtrue/createDependencyReducedPom
dependencyReducedPomLocation${project.basedir}/target/dependency-reduced-pom.xml/dependencyReducedPomLocation
- artifactSet
- includes
-
includecom.google.guava:guava/include
- /includes
- /artifactSet
+ filters
+ filter
+ !--
maybe slf4j-jdk14 --
+
artifactorg.slf4j:*/artifact
+
excludes
+
excludeorg/slf4j/impl/StaticLoggerBinder*/exclude
+
/excludes
+ /filter
+ /filters
+ transformers
+ !-- The
service transformer is needed to merge META-INF/services files --
+ transformer
implementation=org.apache.maven.plugins.shade.resource.ServicesResourceTransformer/
+ /transformers
relocations
relocation
patterncom.google/pattern
-
shadedPatternorg.apache.flink.shaded.com.google/shadedPattern
+
shadedPatternorg.apache.flink.hadoop.shaded.com.google/shadedPattern
excludes
-
excludecom.google.protobuf.**/exclude
+
!-- excludecom.google.protobuf.**/exclude --
--- End diff --
Why commenting out and not removing?
Create a shaded Hadoop fat jar to resolve library version conflicts
---
Key: FLINK-1605
URL: https://issues.apache.org/jira/browse/FLINK-1605
Project: Flink
Issue Type: Improvement
Components: Build System
Affects Versions: 0.9
Reporter: Robert Metzger
Assignee: Robert Metzger
As per mailing list discussion:
http://apache-flink-incubator-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Create-a-shaded-Hadoop-fat-jar-to-resolve-library-version-conflicts-td3881.html
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (FLINK-1605) Create a shaded Hadoop fat jar to resolve library version conflicts
[
https://issues.apache.org/jira/browse/FLINK-1605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14354600#comment-14354600
]
ASF GitHub Bot commented on FLINK-1605:
---
Github user StephanEwen commented on a diff in the pull request:
https://github.com/apache/flink/pull/454#discussion_r26108263
--- Diff: flink-shaded-hadoop/flink-shaded-hadoop2/pom.xml ---
@@ -0,0 +1,183 @@
+?xml version=1.0 encoding=UTF-8?
+!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+License); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+AS IS BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+--
+project xmlns=http://maven.apache.org/POM/4.0.0;
xmlns:xsi=http://www.w3.org/2001/XMLSchema-instance;
+ xsi:schemaLocation=http://maven.apache.org/POM/4.0.0
http://maven.apache.org/maven-v4_0_0.xsd;
+
+ modelVersion4.0.0/modelVersion
+
+ parent
+ groupIdorg.apache.flink/groupId
+ artifactIdflink-shaded-hadoop/artifactId
+ version0.9-SNAPSHOT/version
+ relativePath../relativePath
+ /parent
+
+ artifactIdflink-shaded-hadoop2/artifactId
+ nameflink-shaded-hadoop2/name
+
+ packagingjar/packaging
+
+ dependencies
+ dependency
+ groupIdorg.apache.hadoop/groupId
+ artifactIdhadoop-common/artifactId
+ version${hadoop.version}/version
+ exclusions
+ exclusion
+ groupIdasm/groupId
+ artifactIdasm/artifactId
+ /exclusion
+ exclusion
+ groupIdtomcat/groupId
+ artifactIdjasper-compiler/artifactId
+ /exclusion
+ exclusion
+ groupIdtomcat/groupId
+ artifactIdjasper-runtime/artifactId
+ /exclusion
+ exclusion
+ groupIdorg.mortbay.jetty/groupId
+ artifactIdjetty/artifactId
+ /exclusion
+ exclusion
+ groupIdorg.mortbay.jetty/groupId
+ artifactIdjsp-api-2.1/artifactId
+ /exclusion
+ exclusion
+ groupIdorg.mortbay.jetty/groupId
+ artifactIdjsp-2.1/artifactId
+ /exclusion
+ exclusion
+ groupIdorg.mortbay.jetty/groupId
+ artifactIdjetty-util/artifactId
+ /exclusion
+ exclusion
+ groupIdorg.eclipse.jdt/groupId
+ artifactIdcore/artifactId
+ /exclusion
+ exclusion
+ groupIdjavax.servlet/groupId
+ artifactIdservlet-api/artifactId
+ /exclusion
+ exclusion
+ groupIdjavax.servlet.jsp/groupId
+ artifactIdjsp-api/artifactId
+ /exclusion
+ /exclusions
+ /dependency
+
+ dependency
+ groupIdorg.apache.hadoop/groupId
+ artifactIdhadoop-hdfs/artifactId
+ version${hadoop.version}/version
+ exclusions
+ exclusion
+ groupIdasm/groupId
--- End diff --
I this groupId correct, or should it be `org.ow2.asm` ?
Create a shaded
[jira] [Commented] (FLINK-1605) Create a shaded Hadoop fat jar to resolve library version conflicts
[
https://issues.apache.org/jira/browse/FLINK-1605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14354601#comment-14354601
]
ASF GitHub Bot commented on FLINK-1605:
---
Github user StephanEwen commented on a diff in the pull request:
https://github.com/apache/flink/pull/454#discussion_r26108269
--- Diff: flink-shaded-hadoop/flink-shaded-hadoop2/pom.xml ---
@@ -0,0 +1,183 @@
+?xml version=1.0 encoding=UTF-8?
+!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+License); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+AS IS BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+--
+project xmlns=http://maven.apache.org/POM/4.0.0;
xmlns:xsi=http://www.w3.org/2001/XMLSchema-instance;
+ xsi:schemaLocation=http://maven.apache.org/POM/4.0.0
http://maven.apache.org/maven-v4_0_0.xsd;
+
+ modelVersion4.0.0/modelVersion
+
+ parent
+ groupIdorg.apache.flink/groupId
+ artifactIdflink-shaded-hadoop/artifactId
+ version0.9-SNAPSHOT/version
+ relativePath../relativePath
+ /parent
+
+ artifactIdflink-shaded-hadoop2/artifactId
+ nameflink-shaded-hadoop2/name
+
+ packagingjar/packaging
+
+ dependencies
+ dependency
+ groupIdorg.apache.hadoop/groupId
+ artifactIdhadoop-common/artifactId
+ version${hadoop.version}/version
+ exclusions
+ exclusion
+ groupIdasm/groupId
+ artifactIdasm/artifactId
+ /exclusion
+ exclusion
+ groupIdtomcat/groupId
+ artifactIdjasper-compiler/artifactId
+ /exclusion
+ exclusion
+ groupIdtomcat/groupId
+ artifactIdjasper-runtime/artifactId
+ /exclusion
+ exclusion
+ groupIdorg.mortbay.jetty/groupId
+ artifactIdjetty/artifactId
+ /exclusion
+ exclusion
+ groupIdorg.mortbay.jetty/groupId
+ artifactIdjsp-api-2.1/artifactId
+ /exclusion
+ exclusion
+ groupIdorg.mortbay.jetty/groupId
+ artifactIdjsp-2.1/artifactId
+ /exclusion
+ exclusion
+ groupIdorg.mortbay.jetty/groupId
+ artifactIdjetty-util/artifactId
+ /exclusion
+ exclusion
+ groupIdorg.eclipse.jdt/groupId
+ artifactIdcore/artifactId
+ /exclusion
+ exclusion
+ groupIdjavax.servlet/groupId
+ artifactIdservlet-api/artifactId
+ /exclusion
+ exclusion
+ groupIdjavax.servlet.jsp/groupId
+ artifactIdjsp-api/artifactId
+ /exclusion
+ /exclusions
+ /dependency
+
+ dependency
+ groupIdorg.apache.hadoop/groupId
+ artifactIdhadoop-hdfs/artifactId
+ version${hadoop.version}/version
+ exclusions
+ exclusion
+ groupIdasm/groupId
+ artifactIdasm/artifactId
+
[jira] [Commented] (FLINK-1605) Create a shaded Hadoop fat jar to resolve library version conflicts
[
https://issues.apache.org/jira/browse/FLINK-1605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14354599#comment-14354599
]
ASF GitHub Bot commented on FLINK-1605:
---
Github user StephanEwen commented on a diff in the pull request:
https://github.com/apache/flink/pull/454#discussion_r26108259
--- Diff: flink-shaded-hadoop/flink-shaded-hadoop2/pom.xml ---
@@ -0,0 +1,183 @@
+?xml version=1.0 encoding=UTF-8?
+!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+License); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+AS IS BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+--
+project xmlns=http://maven.apache.org/POM/4.0.0;
xmlns:xsi=http://www.w3.org/2001/XMLSchema-instance;
+ xsi:schemaLocation=http://maven.apache.org/POM/4.0.0
http://maven.apache.org/maven-v4_0_0.xsd;
+
+ modelVersion4.0.0/modelVersion
+
+ parent
+ groupIdorg.apache.flink/groupId
+ artifactIdflink-shaded-hadoop/artifactId
+ version0.9-SNAPSHOT/version
+ relativePath../relativePath
+ /parent
+
+ artifactIdflink-shaded-hadoop2/artifactId
+ nameflink-shaded-hadoop2/name
+
+ packagingjar/packaging
+
+ dependencies
+ dependency
+ groupIdorg.apache.hadoop/groupId
+ artifactIdhadoop-common/artifactId
+ version${hadoop.version}/version
+ exclusions
+ exclusion
+ groupIdasm/groupId
--- End diff --
I this groupId correct, or should it be `org.ow2.asm` ?
Create a shaded Hadoop fat jar to resolve library version conflicts
---
Key: FLINK-1605
URL: https://issues.apache.org/jira/browse/FLINK-1605
Project: Flink
Issue Type: Improvement
Components: Build System
Affects Versions: 0.9
Reporter: Robert Metzger
Assignee: Robert Metzger
As per mailing list discussion:
http://apache-flink-incubator-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Create-a-shaded-Hadoop-fat-jar-to-resolve-library-version-conflicts-td3881.html
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (FLINK-1605) Create a shaded Hadoop fat jar to resolve library version conflicts
[
https://issues.apache.org/jira/browse/FLINK-1605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14354603#comment-14354603
]
ASF GitHub Bot commented on FLINK-1605:
---
Github user StephanEwen commented on a diff in the pull request:
https://github.com/apache/flink/pull/454#discussion_r26108281
--- Diff: flink-shaded-hadoop/flink-shaded-include-yarn/pom.xml ---
@@ -0,0 +1,286 @@
+?xml version=1.0 encoding=UTF-8?
+!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+License); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+AS IS BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+--
+project xmlns=http://maven.apache.org/POM/4.0.0;
xmlns:xsi=http://www.w3.org/2001/XMLSchema-instance;
+ xsi:schemaLocation=http://maven.apache.org/POM/4.0.0
http://maven.apache.org/maven-v4_0_0.xsd;
+
+ modelVersion4.0.0/modelVersion
+
+ parent
+ groupIdorg.apache.flink/groupId
+ artifactIdflink-shaded-hadoop/artifactId
+ version0.9-SNAPSHOT/version
+ relativePath../relativePath
+ /parent
+
+ artifactIdflink-shaded-include-yarn/artifactId
+ nameflink-shaded-include-yarn/name
+
+ packagingjar/packaging
+
+ dependencies
+ dependency
+ groupIdorg.apache.hadoop/groupId
+ artifactIdhadoop-common/artifactId
+ version${hadoop.version}/version
+ exclusions
+ exclusion
+ groupIdasm/groupId
+ artifactIdasm/artifactId
+ /exclusion
+ exclusion
+ groupIdtomcat/groupId
+ artifactIdjasper-compiler/artifactId
+ /exclusion
+ exclusion
+ groupIdtomcat/groupId
+ artifactIdjasper-runtime/artifactId
+ /exclusion
+ exclusion
+ groupIdorg.mortbay.jetty/groupId
+ artifactIdjetty/artifactId
+ /exclusion
+ exclusion
+ groupIdorg.mortbay.jetty/groupId
+ artifactIdjsp-api-2.1/artifactId
+ /exclusion
+ exclusion
+ groupIdorg.mortbay.jetty/groupId
+ artifactIdjsp-2.1/artifactId
+ /exclusion
+ exclusion
+ groupIdorg.mortbay.jetty/groupId
+ artifactIdjetty-util/artifactId
+ /exclusion
+ exclusion
+ groupIdorg.eclipse.jdt/groupId
+ artifactIdcore/artifactId
+ /exclusion
+ exclusion
+ groupIdjavax.servlet/groupId
+ artifactIdservlet-api/artifactId
+ /exclusion
+ exclusion
+ groupIdjavax.servlet.jsp/groupId
+ artifactIdjsp-api/artifactId
+ /exclusion
+ /exclusions
+ /dependency
+
+ dependency
+ groupIdorg.apache.hadoop/groupId
+ artifactIdhadoop-hdfs/artifactId
+ version${hadoop.version}/version
+ exclusions
+ exclusion
+ groupIdasm/groupId
--- End diff --
I this groupId correct, or should it be `org.ow2.asm` ?
[jira] [Commented] (FLINK-1605) Create a shaded Hadoop fat jar to resolve library version conflicts
[
https://issues.apache.org/jira/browse/FLINK-1605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14354602#comment-14354602
]
ASF GitHub Bot commented on FLINK-1605:
---
Github user StephanEwen commented on a diff in the pull request:
https://github.com/apache/flink/pull/454#discussion_r26108278
--- Diff: flink-shaded-hadoop/flink-shaded-include-yarn/pom.xml ---
@@ -0,0 +1,286 @@
+?xml version=1.0 encoding=UTF-8?
+!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+License); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+AS IS BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+--
+project xmlns=http://maven.apache.org/POM/4.0.0;
xmlns:xsi=http://www.w3.org/2001/XMLSchema-instance;
+ xsi:schemaLocation=http://maven.apache.org/POM/4.0.0
http://maven.apache.org/maven-v4_0_0.xsd;
+
+ modelVersion4.0.0/modelVersion
+
+ parent
+ groupIdorg.apache.flink/groupId
+ artifactIdflink-shaded-hadoop/artifactId
+ version0.9-SNAPSHOT/version
+ relativePath../relativePath
+ /parent
+
+ artifactIdflink-shaded-include-yarn/artifactId
+ nameflink-shaded-include-yarn/name
+
+ packagingjar/packaging
+
+ dependencies
+ dependency
+ groupIdorg.apache.hadoop/groupId
+ artifactIdhadoop-common/artifactId
+ version${hadoop.version}/version
+ exclusions
+ exclusion
+ groupIdasm/groupId
--- End diff --
I this groupId correct, or should it be `org.ow2.asm` ?
Create a shaded Hadoop fat jar to resolve library version conflicts
---
Key: FLINK-1605
URL: https://issues.apache.org/jira/browse/FLINK-1605
Project: Flink
Issue Type: Improvement
Components: Build System
Affects Versions: 0.9
Reporter: Robert Metzger
Assignee: Robert Metzger
As per mailing list discussion:
http://apache-flink-incubator-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Create-a-shaded-Hadoop-fat-jar-to-resolve-library-version-conflicts-td3881.html
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (FLINK-1605) Create a shaded Hadoop fat jar to resolve library version conflicts
[
https://issues.apache.org/jira/browse/FLINK-1605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14354605#comment-14354605
]
ASF GitHub Bot commented on FLINK-1605:
---
Github user StephanEwen commented on a diff in the pull request:
https://github.com/apache/flink/pull/454#discussion_r26108289
--- Diff: flink-shaded-hadoop/flink-shaded-include-yarn/pom.xml ---
@@ -0,0 +1,286 @@
+?xml version=1.0 encoding=UTF-8?
+!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+License); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+AS IS BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+--
+project xmlns=http://maven.apache.org/POM/4.0.0;
xmlns:xsi=http://www.w3.org/2001/XMLSchema-instance;
+ xsi:schemaLocation=http://maven.apache.org/POM/4.0.0
http://maven.apache.org/maven-v4_0_0.xsd;
+
+ modelVersion4.0.0/modelVersion
+
+ parent
+ groupIdorg.apache.flink/groupId
+ artifactIdflink-shaded-hadoop/artifactId
+ version0.9-SNAPSHOT/version
+ relativePath../relativePath
+ /parent
+
+ artifactIdflink-shaded-include-yarn/artifactId
+ nameflink-shaded-include-yarn/name
+
+ packagingjar/packaging
+
+ dependencies
+ dependency
+ groupIdorg.apache.hadoop/groupId
+ artifactIdhadoop-common/artifactId
+ version${hadoop.version}/version
+ exclusions
+ exclusion
+ groupIdasm/groupId
+ artifactIdasm/artifactId
+ /exclusion
+ exclusion
+ groupIdtomcat/groupId
+ artifactIdjasper-compiler/artifactId
+ /exclusion
+ exclusion
+ groupIdtomcat/groupId
+ artifactIdjasper-runtime/artifactId
+ /exclusion
+ exclusion
+ groupIdorg.mortbay.jetty/groupId
+ artifactIdjetty/artifactId
+ /exclusion
+ exclusion
+ groupIdorg.mortbay.jetty/groupId
+ artifactIdjsp-api-2.1/artifactId
+ /exclusion
+ exclusion
+ groupIdorg.mortbay.jetty/groupId
+ artifactIdjsp-2.1/artifactId
+ /exclusion
+ exclusion
+ groupIdorg.mortbay.jetty/groupId
+ artifactIdjetty-util/artifactId
+ /exclusion
+ exclusion
+ groupIdorg.eclipse.jdt/groupId
+ artifactIdcore/artifactId
+ /exclusion
+ exclusion
+ groupIdjavax.servlet/groupId
+ artifactIdservlet-api/artifactId
+ /exclusion
+ exclusion
+ groupIdjavax.servlet.jsp/groupId
+ artifactIdjsp-api/artifactId
+ /exclusion
+ /exclusions
+ /dependency
+
+ dependency
+ groupIdorg.apache.hadoop/groupId
+ artifactIdhadoop-hdfs/artifactId
+ version${hadoop.version}/version
+ exclusions
+ exclusion
+ groupIdasm/groupId
+ artifactIdasm/artifactId
+
[jira] [Commented] (FLINK-1605) Create a shaded Hadoop fat jar to resolve library version conflicts
[
https://issues.apache.org/jira/browse/FLINK-1605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14354604#comment-14354604
]
ASF GitHub Bot commented on FLINK-1605:
---
Github user StephanEwen commented on a diff in the pull request:
https://github.com/apache/flink/pull/454#discussion_r26108286
--- Diff: flink-shaded-hadoop/flink-shaded-include-yarn/pom.xml ---
@@ -0,0 +1,286 @@
+?xml version=1.0 encoding=UTF-8?
+!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+License); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+AS IS BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+--
+project xmlns=http://maven.apache.org/POM/4.0.0;
xmlns:xsi=http://www.w3.org/2001/XMLSchema-instance;
+ xsi:schemaLocation=http://maven.apache.org/POM/4.0.0
http://maven.apache.org/maven-v4_0_0.xsd;
+
+ modelVersion4.0.0/modelVersion
+
+ parent
+ groupIdorg.apache.flink/groupId
+ artifactIdflink-shaded-hadoop/artifactId
+ version0.9-SNAPSHOT/version
+ relativePath../relativePath
+ /parent
+
+ artifactIdflink-shaded-include-yarn/artifactId
+ nameflink-shaded-include-yarn/name
+
+ packagingjar/packaging
+
+ dependencies
+ dependency
+ groupIdorg.apache.hadoop/groupId
+ artifactIdhadoop-common/artifactId
+ version${hadoop.version}/version
+ exclusions
+ exclusion
+ groupIdasm/groupId
+ artifactIdasm/artifactId
+ /exclusion
+ exclusion
+ groupIdtomcat/groupId
+ artifactIdjasper-compiler/artifactId
+ /exclusion
+ exclusion
+ groupIdtomcat/groupId
+ artifactIdjasper-runtime/artifactId
+ /exclusion
+ exclusion
+ groupIdorg.mortbay.jetty/groupId
+ artifactIdjetty/artifactId
+ /exclusion
+ exclusion
+ groupIdorg.mortbay.jetty/groupId
+ artifactIdjsp-api-2.1/artifactId
+ /exclusion
+ exclusion
+ groupIdorg.mortbay.jetty/groupId
+ artifactIdjsp-2.1/artifactId
+ /exclusion
+ exclusion
+ groupIdorg.mortbay.jetty/groupId
+ artifactIdjetty-util/artifactId
+ /exclusion
+ exclusion
+ groupIdorg.eclipse.jdt/groupId
+ artifactIdcore/artifactId
+ /exclusion
+ exclusion
+ groupIdjavax.servlet/groupId
+ artifactIdservlet-api/artifactId
+ /exclusion
+ exclusion
+ groupIdjavax.servlet.jsp/groupId
+ artifactIdjsp-api/artifactId
+ /exclusion
+ /exclusions
+ /dependency
+
+ dependency
+ groupIdorg.apache.hadoop/groupId
+ artifactIdhadoop-hdfs/artifactId
+ version${hadoop.version}/version
+ exclusions
+ exclusion
+ groupIdasm/groupId
+ artifactIdasm/artifactId
+
[jira] [Commented] (FLINK-1605) Create a shaded Hadoop fat jar to resolve library version conflicts
[
https://issues.apache.org/jira/browse/FLINK-1605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14354606#comment-14354606
]
ASF GitHub Bot commented on FLINK-1605:
---
Github user StephanEwen commented on a diff in the pull request:
https://github.com/apache/flink/pull/454#discussion_r26108294
--- Diff: flink-shaded-hadoop/flink-shaded-include-yarn/pom.xml ---
@@ -0,0 +1,286 @@
+?xml version=1.0 encoding=UTF-8?
+!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+License); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+AS IS BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+--
+project xmlns=http://maven.apache.org/POM/4.0.0;
xmlns:xsi=http://www.w3.org/2001/XMLSchema-instance;
+ xsi:schemaLocation=http://maven.apache.org/POM/4.0.0
http://maven.apache.org/maven-v4_0_0.xsd;
+
+ modelVersion4.0.0/modelVersion
+
+ parent
+ groupIdorg.apache.flink/groupId
+ artifactIdflink-shaded-hadoop/artifactId
+ version0.9-SNAPSHOT/version
+ relativePath../relativePath
+ /parent
+
+ artifactIdflink-shaded-include-yarn/artifactId
+ nameflink-shaded-include-yarn/name
+
+ packagingjar/packaging
+
+ dependencies
+ dependency
+ groupIdorg.apache.hadoop/groupId
+ artifactIdhadoop-common/artifactId
+ version${hadoop.version}/version
+ exclusions
+ exclusion
+ groupIdasm/groupId
+ artifactIdasm/artifactId
+ /exclusion
+ exclusion
+ groupIdtomcat/groupId
+ artifactIdjasper-compiler/artifactId
+ /exclusion
+ exclusion
+ groupIdtomcat/groupId
+ artifactIdjasper-runtime/artifactId
+ /exclusion
+ exclusion
+ groupIdorg.mortbay.jetty/groupId
+ artifactIdjetty/artifactId
+ /exclusion
+ exclusion
+ groupIdorg.mortbay.jetty/groupId
+ artifactIdjsp-api-2.1/artifactId
+ /exclusion
+ exclusion
+ groupIdorg.mortbay.jetty/groupId
+ artifactIdjsp-2.1/artifactId
+ /exclusion
+ exclusion
+ groupIdorg.mortbay.jetty/groupId
+ artifactIdjetty-util/artifactId
+ /exclusion
+ exclusion
+ groupIdorg.eclipse.jdt/groupId
+ artifactIdcore/artifactId
+ /exclusion
+ exclusion
+ groupIdjavax.servlet/groupId
+ artifactIdservlet-api/artifactId
+ /exclusion
+ exclusion
+ groupIdjavax.servlet.jsp/groupId
+ artifactIdjsp-api/artifactId
+ /exclusion
+ /exclusions
+ /dependency
+
+ dependency
+ groupIdorg.apache.hadoop/groupId
+ artifactIdhadoop-hdfs/artifactId
+ version${hadoop.version}/version
+ exclusions
+ exclusion
+ groupIdasm/groupId
+ artifactIdasm/artifactId
+
[jira] [Commented] (FLINK-1605) Create a shaded Hadoop fat jar to resolve library version conflicts
[ https://issues.apache.org/jira/browse/FLINK-1605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14354608#comment-14354608 ] ASF GitHub Bot commented on FLINK-1605: --- Github user StephanEwen commented on the pull request: https://github.com/apache/flink/pull/454#issuecomment-78022132 Here are some points to double check - You exclude all the `flink-clients` web docs from the uberjar. I would guess this breaks the webclient, because it uses the JAR file as its web root, or am I overlooking something? - The uberjar has the YARN session as the main class. Seems strange to make this one the main class of the Uberjar, when there are several main classes (JobManager, TaskManager, CliFrontend, WebClient, AppMaster, ...) - Previously, the shade plugin was deactivated for the quickstarts. Is that no longer necessary? - Comments inline about the `asm` groupId If none of these points is actually an issue, then +1 to merge Create a shaded Hadoop fat jar to resolve library version conflicts --- Key: FLINK-1605 URL: https://issues.apache.org/jira/browse/FLINK-1605 Project: Flink Issue Type: Improvement Components: Build System Affects Versions: 0.9 Reporter: Robert Metzger Assignee: Robert Metzger As per mailing list discussion: http://apache-flink-incubator-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Create-a-shaded-Hadoop-fat-jar-to-resolve-library-version-conflicts-td3881.html -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (FLINK-1605) Create a shaded Hadoop fat jar to resolve library version conflicts
[
https://issues.apache.org/jira/browse/FLINK-1605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14354598#comment-14354598
]
ASF GitHub Bot commented on FLINK-1605:
---
Github user StephanEwen commented on a diff in the pull request:
https://github.com/apache/flink/pull/454#discussion_r26108253
--- Diff: flink-shaded-hadoop/flink-shaded-hadoop1/pom.xml ---
@@ -0,0 +1,79 @@
+?xml version=1.0 encoding=UTF-8?
+!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+License); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+AS IS BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+--
+project xmlns=http://maven.apache.org/POM/4.0.0;
xmlns:xsi=http://www.w3.org/2001/XMLSchema-instance;
+ xsi:schemaLocation=http://maven.apache.org/POM/4.0.0
http://maven.apache.org/maven-v4_0_0.xsd;
+
+ modelVersion4.0.0/modelVersion
+
+ parent
+ groupIdorg.apache.flink/groupId
+ artifactIdflink-shaded-hadoop/artifactId
+ version0.9-SNAPSHOT/version
+ relativePath../relativePath
+ /parent
+
+ artifactIdflink-shaded-hadoop1/artifactId
+ nameflink-shaded-hadoop1/name
+
+ packagingjar/packaging
+
+ dependencies
+ dependency
+ groupIdorg.apache.hadoop/groupId
+ artifactIdhadoop-core/artifactId
+ version${hadoop.version}/version
+ exclusions
+ exclusion
+ groupIdasm/groupId
--- End diff --
I this groupId correct, or should it be `org.ow2.asm` ?
Create a shaded Hadoop fat jar to resolve library version conflicts
---
Key: FLINK-1605
URL: https://issues.apache.org/jira/browse/FLINK-1605
Project: Flink
Issue Type: Improvement
Components: Build System
Affects Versions: 0.9
Reporter: Robert Metzger
Assignee: Robert Metzger
As per mailing list discussion:
http://apache-flink-incubator-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Create-a-shaded-Hadoop-fat-jar-to-resolve-library-version-conflicts-td3881.html
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (FLINK-1605) Create a shaded Hadoop fat jar to resolve library version conflicts
[ https://issues.apache.org/jira/browse/FLINK-1605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14354844#comment-14354844 ] ASF GitHub Bot commented on FLINK-1605: --- Github user StephanEwen commented on the pull request: https://github.com/apache/flink/pull/454#issuecomment-78047798 Looks good to me. +1 to merge, if Travis (or Bob the Builder) agrees Create a shaded Hadoop fat jar to resolve library version conflicts --- Key: FLINK-1605 URL: https://issues.apache.org/jira/browse/FLINK-1605 Project: Flink Issue Type: Improvement Components: Build System Affects Versions: 0.9 Reporter: Robert Metzger Assignee: Robert Metzger As per mailing list discussion: http://apache-flink-incubator-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Create-a-shaded-Hadoop-fat-jar-to-resolve-library-version-conflicts-td3881.html -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (FLINK-1605) Create a shaded Hadoop fat jar to resolve library version conflicts
[ https://issues.apache.org/jira/browse/FLINK-1605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14353414#comment-14353414 ] ASF GitHub Bot commented on FLINK-1605: --- Github user rmetzger commented on the pull request: https://github.com/apache/flink/pull/454#issuecomment-77921349 The pull request has been rebased to master. I'm looking for some feedback for this change. Create a shaded Hadoop fat jar to resolve library version conflicts --- Key: FLINK-1605 URL: https://issues.apache.org/jira/browse/FLINK-1605 Project: Flink Issue Type: Improvement Components: Build System Affects Versions: 0.9 Reporter: Robert Metzger Assignee: Robert Metzger As per mailing list discussion: http://apache-flink-incubator-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Create-a-shaded-Hadoop-fat-jar-to-resolve-library-version-conflicts-td3881.html -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (FLINK-1605) Create a shaded Hadoop fat jar to resolve library version conflicts
[ https://issues.apache.org/jira/browse/FLINK-1605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14347502#comment-14347502 ] ASF GitHub Bot commented on FLINK-1605: --- Github user StephanEwen commented on the pull request: https://github.com/apache/flink/pull/454#issuecomment-77237532 Looks good, but also like a lot of magic. Can you briefly summarize here (and copy it to the wiki) how the setup works? I am especially confused by the extra YARN shaded versions (two of them). Some remaining questions: - Why give up the `ship` directory? I think it was good to differentiate between `ship` and `lib`. - What is included in the final flink-dist fat jar? All streaming connectors? The hbase connector and hadoop compatibility? Create a shaded Hadoop fat jar to resolve library version conflicts --- Key: FLINK-1605 URL: https://issues.apache.org/jira/browse/FLINK-1605 Project: Flink Issue Type: Improvement Components: Build System Affects Versions: 0.9 Reporter: Robert Metzger Assignee: Robert Metzger As per mailing list discussion: http://apache-flink-incubator-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Create-a-shaded-Hadoop-fat-jar-to-resolve-library-version-conflicts-td3881.html -- This message was sent by Atlassian JIRA (v6.3.4#6332)
