Pei He created BEAM-2295:
----------------------------
Summary: Hadoop IO connectors require additional repositories
Key: BEAM-2295
URL: https://issues.apache.org/jira/browse/BEAM-2295
Project: Beam
Issue Type: Bug
Components: sdk-java-core
Reporter: Pei He
Assignee: Jean-Baptiste Onofré
Several dependencies are not in https://repo.maven.apache.org/maven2.
404 when trying to visit https://repo.maven.apache.org/maven2/cascading/
My workaround is to add conjars.org repository to my local profile.
<repository>
<id>clojars</id>
<url>https://clojars.org/repo/</url>
<releases>
<enabled>true</enabled>
</releases>
<snapshots>
<enabled>true</enabled>
</snapshots>
</repository>
The question is how to make the io connector modules have the additional repo
by default? I can see this issue will come up when more io connectors are
added, and when they need additional repos.
[INFO] ------------------------------------------------------------------------
[INFO] Building Apache Beam :: SDKs :: Java :: IO :: Hadoop :: jdk1.8-tests
2.0.0
[INFO] ------------------------------------------------------------------------
[INFO]
[INFO] --- maven-clean-plugin:3.0.0:clean (default-clean) @
beam-sdks-java-io-hadoop-jdk1.8-tests ---
[INFO] Deleting
/Users/peihe/Downloads/apache-beam-2.0.0/sdks/java/io/hadoop/jdk1.8-tests
(includes = [**/*.pyc, **/*.egg-info/, **/sdks/python/LICENSE,
**/sdks/python/NOTICE, **/sdks/python/README.md], excludes = [])
[INFO]
[INFO] --- maven-enforcer-plugin:1.4.1:enforce (enforce) @
beam-sdks-java-io-hadoop-jdk1.8-tests ---
Downloading:
https://repo.maven.apache.org/maven2/org/pentaho/pentaho-aggdesigner-algorithm/5.1.5-jhyde/pentaho-aggdesigner-algorithm-5.1.5-jhyde.jar
[WARNING] Could not find artifact
org.pentaho:pentaho-aggdesigner-algorithm:jar:5.1.5-jhyde in central
(https://repo.maven.apache.org/maven2)
Try downloading the file manually from the project website.
Then, install it using the command:
mvn install:install-file -DgroupId=org.pentaho
-DartifactId=pentaho-aggdesigner-algorithm -Dversion=5.1.5-jhyde
-Dpackaging=jar -Dfile=/path/to/file
Alternatively, if you host your own repository you can deploy the file there:
mvn deploy:deploy-file -DgroupId=org.pentaho
-DartifactId=pentaho-aggdesigner-algorithm -Dversion=5.1.5-jhyde
-Dpackaging=jar -Dfile=/path/to/file -Durl=[url] -DrepositoryId=[id]
Path to dependency:
1) org.apache.beam:beam-sdks-java-io-hadoop-jdk1.8-tests:jar:2.0.0
2) org.elasticsearch:elasticsearch-hadoop:jar:5.0.0
3) org.apache.hive:hive-service:jar:1.2.1
4) org.apache.hive:hive-exec:jar:1.2.1
5) org.apache.calcite:calcite-core:jar:1.2.0-incubating
6) org.pentaho:pentaho-aggdesigner-algorithm:jar:5.1.5-jhyde
org.pentaho:pentaho-aggdesigner-algorithm:jar:5.1.5-jhyde
from the specified remote repositories:
apache.snapshots (https://repository.apache.org/snapshots, releases=false,
snapshots=true),
central (https://repo.maven.apache.org/maven2, releases=true, snapshots=false)
Downloading:
https://repo.maven.apache.org/maven2/cascading/cascading-hadoop/2.6.3/cascading-hadoop-2.6.3.jar
[WARNING] Could not find artifact cascading:cascading-hadoop:jar:2.6.3 in
central (https://repo.maven.apache.org/maven2)
Try downloading the file manually from the project website.
Then, install it using the command:
mvn install:install-file -DgroupId=cascading -DartifactId=cascading-hadoop
-Dversion=2.6.3 -Dpackaging=jar -Dfile=/path/to/file
Alternatively, if you host your own repository you can deploy the file there:
mvn deploy:deploy-file -DgroupId=cascading -DartifactId=cascading-hadoop
-Dversion=2.6.3 -Dpackaging=jar -Dfile=/path/to/file -Durl=[url]
-DrepositoryId=[id]
Path to dependency:
1) org.apache.beam:beam-sdks-java-io-hadoop-jdk1.8-tests:jar:2.0.0
2) org.elasticsearch:elasticsearch-hadoop:jar:5.0.0
3) cascading:cascading-hadoop:jar:2.6.3
cascading:cascading-hadoop:jar:2.6.3
from the specified remote repositories:
apache.snapshots (https://repository.apache.org/snapshots, releases=false,
snapshots=true),
central (https://repo.maven.apache.org/maven2, releases=true, snapshots=false)
Downloading:
https://repo.maven.apache.org/maven2/cascading/cascading-local/2.6.3/cascading-local-2.6.3.jar
[WARNING] Could not find artifact cascading:cascading-local:jar:2.6.3 in
central (https://repo.maven.apache.org/maven2)
Try downloading the file manually from the project website.
Then, install it using the command:
mvn install:install-file -DgroupId=cascading -DartifactId=cascading-local
-Dversion=2.6.3 -Dpackaging=jar -Dfile=/path/to/file
Alternatively, if you host your own repository you can deploy the file there:
mvn deploy:deploy-file -DgroupId=cascading -DartifactId=cascading-local
-Dversion=2.6.3 -Dpackaging=jar -Dfile=/path/to/file -Durl=[url]
-DrepositoryId=[id]
Path to dependency:
1) org.apache.beam:beam-sdks-java-io-hadoop-jdk1.8-tests:jar:2.0.0
2) org.elasticsearch:elasticsearch-hadoop:jar:5.0.0
3) cascading:cascading-local:jar:2.6.3
cascading:cascading-local:jar:2.6.3
from the specified remote repositories:
apache.snapshots (https://repository.apache.org/snapshots, releases=false,
snapshots=true),
central (https://repo.maven.apache.org/maven2, releases=true, snapshots=false)
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
