lukecwik commented on code in PR #24264:
URL: https://github.com/apache/beam/pull/24264#discussion_r1033927030
##########
CHANGES.md:
##########
@@ -78,6 +78,8 @@
dependency of the Java SDK Harness. Some users of a portable runner (such as
Dataflow Runner v2)
may have an undeclared dependency on this package (for example using GCS with
TextIO) and will now need to declare the dependency.
+* `beam-sdks-java-core` is no longer a dependency of the Java SDK Harness.
Users of a portable
+ runner (such as Dataflow Runner v2) will need to provide this package and
it's dependecies.
Review Comment:
```suggestion
runner (such as Dataflow Runner v2) will need to provide this package and
its dependencies.
```
##########
sdks/java/harness/build.gradle:
##########
@@ -18,35 +18,51 @@
plugins { id 'org.apache.beam.module' }
-// We specifically enumerate all the projects that we depend on since
-// the list is used in both defining the included set for the uber jar
-// and also the set of project level dependencies.
-def dependOnShadedProjects = [":model:pipeline", ":model:fn-execution",
":sdks:java:core"]
-def dependOnProjects = [":sdks:java:fn-execution",
- ":runners:core-java",
":runners:core-construction-java"]
+configurations {
+ provided
Review Comment:
I was thinking that we would be able to decouple this from using `provided`
if we declared an arbitrary configuration here like `resolveForShadowJar` and
then specify the dependencies twice, once before the `applyJavaNature` in the
`resolveForShadowJar` configuration and once in the `provided` configuration
after the `applyJavaNature`. This way don't force this limitation on to
`BeamModulePlugin.groovy`
##########
sdks/java/harness/build.gradle:
##########
@@ -18,35 +18,51 @@
plugins { id 'org.apache.beam.module' }
-// We specifically enumerate all the projects that we depend on since
-// the list is used in both defining the included set for the uber jar
-// and also the set of project level dependencies.
-def dependOnShadedProjects = [":model:pipeline", ":model:fn-execution",
":sdks:java:core"]
-def dependOnProjects = [":sdks:java:fn-execution",
- ":runners:core-java",
":runners:core-construction-java"]
+configurations {
+ provided
+}
+
+dependencies {
+ // provided dependencies are declared early so they can be resolved in
shadowClosure
+
+ // :sdks:java:core and transitive dependencies
+ provided project(path: ":model:pipeline", configuration: "shadow")
+ provided project(path: ":sdks:java:core", configuration: "shadow")
+ provided library.java.joda_time
+ provided library.java.slf4j_api
+ provided library.java.vendored_grpc_1_48_1
+ provided library.java.vendored_guava_26_0_jre
+
+ // :sdks:java:fn-execution and transitive dependencies
+ provided project(path: ":model:fn-execution", configuration: "shadow")
+ provided project(":sdks:java:fn-execution")
Review Comment:
I think your right that we need to keep this as `implementation`. Some
future change can deal with moving more and more dependencies out of the
shadowJar
##########
sdks/java/harness/build.gradle:
##########
@@ -18,35 +18,44 @@
plugins { id 'org.apache.beam.module' }
-// We specifically enumerate all the projects that we depend on since
-// the list is used in both defining the included set for the uber jar
-// and also the set of project level dependencies.
-def dependOnShadedProjects = [":model:pipeline", ":model:fn-execution",
":sdks:java:core"]
-def dependOnProjects = [":sdks:java:fn-execution",
- ":runners:core-java",
":runners:core-construction-java"]
+configurations {
+ provided
+}
+
+dependencies {
+ // provided dependencies are declared early so they can be resolved in
shadowClosure
+ provided project(path: ":sdks:java:core", configuration: "shadow")
+}
applyJavaNature(
classesTriggerCheckerBugs: [
'AssignWindowsRunner':
'https://github.com/typetools/checker-framework/issues/3794',
'WindowMergingFnRunner':
'https://github.com/typetools/checker-framework/issues/3794',
],
automaticModuleName: 'org.apache.beam.fn.harness',
- validateShadowJar: false,
testShadowJar: true,
+ shadowJarValidationExcludes: [
+ "io/github/classgraph/**",
+ "nonapi/io/github/classgraph/**",
+ "org/apache/beam/fn/**",
Review Comment:
Please narrow to `org/apache/beam/fn/harness/**`
##########
sdks/java/harness/build.gradle:
##########
@@ -18,35 +18,51 @@
plugins { id 'org.apache.beam.module' }
-// We specifically enumerate all the projects that we depend on since
-// the list is used in both defining the included set for the uber jar
-// and also the set of project level dependencies.
-def dependOnShadedProjects = [":model:pipeline", ":model:fn-execution",
":sdks:java:core"]
-def dependOnProjects = [":sdks:java:fn-execution",
- ":runners:core-java",
":runners:core-construction-java"]
+configurations {
+ provided
+}
+
+dependencies {
+ // provided dependencies are declared early so they can be resolved in
shadowClosure
Review Comment:
Can we set the GCP libraries BOM as the enforced platform for `provided` and
`implementation` configurations
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
