[
https://issues.apache.org/jira/browse/BEAM-4481?focusedWorklogId=111251&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-111251
]
ASF GitHub Bot logged work on BEAM-4481:
----------------------------------------
Author: ASF GitHub Bot
Created on: 12/Jun/18 20:15
Start Date: 12/Jun/18 20:15
Worklog Time Spent: 10m
Work Description: lukecwik commented on a change in pull request #5594:
[BEAM-4481, BEAM-4484] Start vendoring portability dependencies to not have
dependency conflicts
URL: https://github.com/apache/beam/pull/5594#discussion_r194861219
##########
File path: build_rules.gradle
##########
@@ -1253,6 +1254,125 @@ ext.applyGrpcNature = {
/*************************************************************************************************/
+ext.applyPortabilityNature = {
+ println "applyPortabilityNature with " + (it ? "$it" : "default
configuration") + " for project $project.name"
+ applyJavaNature(enableFindbugs: false, enableErrorProne: false,
shadowClosure: {
+ // guava uses the com.google.common and com.google.thirdparty package
namespaces
+ relocate "com.google.common",
"org.apache.beam.vendor.guava.v20.com.google.common"
+ relocate "com.google.thirdparty",
"org.apache.beam.vendor.guava.v20.com.google.thirdparty"
+
+ relocate "com.google.protobuf",
"org.apache.beam.vendor.protobuf.v3.com.google.protobuf"
+ relocate "com.google.gson",
"org.apache.beam.vendor.gson.v2.com.google.gson"
+ relocate "io.grpc", "org.apache.beam.vendor.grpc.v1.io.grpc"
+ relocate "com.google.auth",
"org.apache.beam.vendor.google_auth_library_credentials.v0_9_1.com.google.auth"
+ relocate "com.google.api",
"org.apache.beam.vendor.proto_google_common_protos.v1.com.google.api"
+ relocate "com.google.cloud",
"org.apache.beam.vendor.proto_google_common_protos.v1.com.google.cloud"
+ relocate "com.google.logging",
"org.apache.beam.vendor.proto_google_common_protos.v1.com.google.logging"
+ relocate "com.google.longrunning",
"org.apache.beam.vendor.proto_google_common_protos.v1.com.google.longrunning"
+ relocate "com.google.rpc",
"org.apache.beam.vendor.proto_google_common_protos.v1.com.google.rpc"
+ relocate "com.google.type",
"org.apache.beam.vendor.proto_google_common_protos.v1.com.google.type"
+ relocate "io.opencensus",
"org.apache.beam.vendor.opencensus.v0_11.io.opencensus"
+
+ // Adapted from
https://github.com/grpc/grpc-java/blob/e283f70ad91f99c7fee8b31b605ef12a4f9b1690/netty/shaded/build.gradle#L41
+ relocate "io.netty", "org.apache.beam.vendor.netty.v4.io.netty"
+ // We have to be careful with these replacements as they must not match any
+ // string in NativeLibraryLoader, else they cause corruption. Note that
+ // this includes concatenation of string literals and constants.
+ relocate 'META-INF/native/libnetty',
'META-INF/native/liborg_apache_beam_vendor_netty_v4_netty'
+ relocate 'META-INF/native/netty',
'META-INF/native/org_apache_beam_vendor_netty_v4_netty'
+
+ // Don't include errorprone, JDK8 annotations, objenesis, junit, and
mockito in the bundled jar
+ exclude "com/google/errorprone/**"
+ exclude "com/google/instrumentation/**"
+ exclude "javax/annotation/**"
+ exclude "junit/**"
+ exclude "org/hamcrest/**"
+ exclude "org/junit/**"
+ exclude "org/mockito/**"
+ exclude "org/objenesis/**"
+ })
+
+ // Don't force modules here because we don't want to take the shared
declarations in build_rules.gradle
+ // because we would like to have the freedom to choose which versions of
dependencies we
+ // are using for the portability APIs separate from what is being used
inside other modules such as GCP.
+ configurations.all { config ->
+ config.resolutionStrategy {
+ forcedModules = []
+ }
+ }
+
+ apply plugin: "com.google.protobuf"
+ protobuf {
+ protoc {
+ // The artifact spec for the Protobuf Compiler
+ artifact = "com.google.protobuf:protoc:3.5.1"
+ }
+
+ // Configure the codegen plugins
+ plugins {
+ // An artifact spec for a protoc plugin, with "grpc" as
+ // the identifier, which can be referred to in the "plugins"
+ // container of the "generateProtoTasks" closure.
+ grpc {
+ artifact = "io.grpc:protoc-gen-grpc-java:1.12.0"
+ }
+ }
+
+ generateProtoTasks {
+ ofSourceSet("main")*.plugins {
+ // Apply the "grpc" plugin whose spec is defined above, without
+ // options. Note the braces cannot be omitted, otherwise the
+ // plugin will not be added. This is because of the implicit way
+ // NamedDomainObjectContainer binds the methods.
+ grpc { }
+ }
+ }
+ }
+
+ dependencies {
+ compile 'com.google.guava:guava:20.0'
Review comment:
With vendoring, version drift only leads to larger jars since we will
possibly be packaging multiple versions of the same dependency, other then that
it doesn't impact the user. I believe for non-vendored packages, using
`ext.library` is the right way to go since we want to have a consistent set of
dependencies across the project so users don't have to deal with diamond
dependency issues.
In this case I specifically choose different versions because I also wanted
to perform a partial upgrade moving us to the latest `protobuf` and `grpc`
libraries. It was difficult enough to do this once on a single set of
dependencies so I didn't want to repeat this again. Its likely that Chamikara
will be able to use this set of dependencies when going GCP packages.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 111251)
Time Spent: 5h (was: 4h 50m)
> Remove duplicate dependency declarations from runners/direct-java
> -----------------------------------------------------------------
>
> Key: BEAM-4481
> URL: https://issues.apache.org/jira/browse/BEAM-4481
> Project: Beam
> Issue Type: Bug
> Components: runner-direct
> Reporter: Luke Cwik
> Assignee: Luke Cwik
> Priority: Major
> Time Spent: 5h
> Remaining Estimate: 0h
>
> beam-model-pipeline and others are duplicated in the dependency list
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
