[ https://issues.apache.org/jira/browse/FLINK-13945?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Chesnay Schepler reassigned FLINK-13945: ---------------------------------------- Assignee: Chesnay Schepler > Vendor-repos Maven profile doesn't exist in flink-shaded > -------------------------------------------------------- > > Key: FLINK-13945 > URL: https://issues.apache.org/jira/browse/FLINK-13945 > Project: Flink > Issue Type: Bug > Components: BuildSystem / Shaded > Affects Versions: shaded-7.0, shaded-8.0, shaded-9.0 > Reporter: Elise Ramé > Assignee: Chesnay Schepler > Priority: Major > > According to > [documentation|https://ci.apache.org/projects/flink/flink-docs-release-1.9/flinkDev/building.html#custom--vendor-specific-versions], > to build Flink against a vendor specific Hadoop version it is necessary to > build flink-shaded against this version first : > {code:bash} > mvn clean install -DskipTests -Pvendor-repos -Dhadoop.version=<hadoop_version> > {code} > vendor-repos profile has to be activated to include Hadoop vendors > repositories. > But Maven cannot find expected Hadoop dependencies and returns an error > because vendor-repos profile isn't defined in flink-shaded. > Example using flink-shaded 8.0 and HDP 2.6.5 Hadoop version : > {code:bash} > mvn clean install -DskipTests -Pvendor-repos > -Dhadoop.version=2.7.3.2.6.5.0-292 > {code} > {code:bash} > [INFO] ---------------< org.apache.flink:flink-shaded-hadoop-2 > >--------------- > [INFO] Building flink-shaded-hadoop-2 2.7.3.2.6.5.0-292-8.0 > [10/11] > [INFO] --------------------------------[ jar > ]--------------------------------- > [WARNING] The POM for org.apache.hadoop:hadoop-common:jar:2.7.3.2.6.5.0-292 > is missing, no dependency information available > [WARNING] The POM for org.apache.hadoop:hadoop-hdfs:jar:2.7.3.2.6.5.0-292 is > missing, no dependency information available > [WARNING] The POM for > org.apache.hadoop:hadoop-mapreduce-client-core:jar:2.7.3.2.6.5.0-292 is > missing, no dependency information available > [WARNING] The POM for > org.apache.hadoop:hadoop-yarn-client:jar:2.7.3.2.6.5.0-292 is missing, no > dependency information available > [WARNING] The POM for > org.apache.hadoop:hadoop-yarn-common:jar:2.7.3.2.6.5.0-292 is missing, no > dependency information available > [INFO] > ------------------------------------------------------------------------ > [INFO] Reactor Summary: > [INFO] > [INFO] flink-shaded 8.0 ................................... SUCCESS [ 2.122 > s] > [INFO] flink-shaded-force-shading 8.0 ..................... SUCCESS [ 0.607 > s] > [INFO] flink-shaded-asm-7 7.1-8.0 ......................... SUCCESS [ 0.667 > s] > [INFO] flink-shaded-guava-18 18.0-8.0 ..................... SUCCESS [ 1.452 > s] > [INFO] flink-shaded-netty-4 4.1.39.Final-8.0 .............. SUCCESS [ 4.597 > s] > [INFO] flink-shaded-netty-tcnative-dynamic 2.0.25.Final-8.0 SUCCESS [ 0.620 > s] > [INFO] flink-shaded-jackson-parent 2.9.8-8.0 .............. SUCCESS [ 0.018 > s] > [INFO] flink-shaded-jackson-2 2.9.8-8.0 ................... SUCCESS [ 0.914 > s] > [INFO] flink-shaded-jackson-module-jsonSchema-2 2.9.8-8.0 . SUCCESS [ 0.627 > s] > [INFO] flink-shaded-hadoop-2 2.7.3.2.6.5.0-292-8.0 ........ FAILURE [ 0.047 > s] > [INFO] flink-shaded-hadoop-2-uber 2.7.3.2.6.5.0-292-8.0 ... SKIPPED > [INFO] > ------------------------------------------------------------------------ > [INFO] BUILD FAILURE > [INFO] > ------------------------------------------------------------------------ > [INFO] Total time: 11.947 s > [INFO] Finished at: 2019-09-03T16:52:59+02:00 > [INFO] > ------------------------------------------------------------------------ > [WARNING] The requested profile "vendor-repos" could not be activated because > it does not exist. > [ERROR] Failed to execute goal on project flink-shaded-hadoop-2: Could not > resolve dependencies for project > org.apache.flink:flink-shaded-hadoop-2:jar:2.7.3.2.6.5.0-292-8.0: The > following artifacts could not be resolved: > org.apache.hadoop:hadoop-common:jar:2.7.3.2.6.5.0-292, > org.apache.hadoop:hadoop-hdfs:jar:2.7.3.2.6.5.0-292, > org.apache.hadoop:hadoop-mapreduce-client-core:jar:2.7.3.2.6.5.0-292, > org.apache.hadoop:hadoop-yarn-client:jar:2.7.3.2.6.5.0-292, > org.apache.hadoop:hadoop-yarn-common:jar:2.7.3.2.6.5.0-292: Failure to find > org.apache.hadoop:hadoop-common:jar:2.7.3.2.6.5.0-292 in > https://repo.maven.apache.org/maven2 was cached in the local repository, > resolution will not be reattempted until the update interval of central has > elapsed or updates are forced -> [Help 1] > [ERROR] > [ERROR] To see the full stack trace of the errors, re-run Maven with the -e > switch. > [ERROR] Re-run Maven using the -X switch to enable full debug logging. > [ERROR] > [ERROR] For more information about the errors and possible solutions, please > read the following articles: > [ERROR] [Help 1] > http://cwiki.apache.org/confluence/display/MAVEN/DependencyResolutionException > [ERROR] > [ERROR] After correcting the problems, you can resume the build with the > command > [ERROR] mvn <goals> -rf :flink-shaded-hadoop-2 > {code} > vendor-repos profile exists in Flink pom.xml file : > [https://github.com/apache/flink/blob/3079d11913f153ec40c75afb5356fd3be1a1e550/pom.xml#L1037] -- This message was sent by Atlassian Jira (v8.3.4#803005)