Good example about Guava deps, let me go a bit deeper. > $ find . -name build.gradle | xargs grep library.java.guava > ./sdks/java/core/build.gradle: shadowTest library.java.guava_testlib > ./sdks/java/io/kinesis/build.gradle: testCompile library.java.guava_testlib
Regarding using non-vendored Guava in KinesisIO (and "java/core" as well), it’s all about “library.java.guava_testlib” and “com.google.common.testing.EqualsTester” only in particular, which is used for tests. Do we need to vendor “com.google.guava:guava-testlib” for this in this case? > - KinesisIO does depend on Guava at compile scope but has incorrect > dependencies (Kinesis libs have Guava on API surface so it is OK here, but > should be correctly declared) Sorry, but I didn’t understand what do you mean by “but should be correctly declared”. Since Kinesis client libs have own Guava deps and we shade our own guava, so it should be fine, no? > On 11 Nov 2019, at 22:29, Kenneth Knowles <[email protected]> wrote: > > BeamModulePlugin just contains lists of versions to ease coordination across > Beam modules, but mostly does not create dependencies. Most of Beam's modules > only depend on a few things there. For example Guava is not a core > dependency, but here is where it is actually depended upon: > > $ find . -name build.gradle | xargs grep library.java.guava > ./sdks/java/core/build.gradle: shadowTest library.java.guava_testlib > ./sdks/java/extensions/sql/jdbc/build.gradle: compile library.java.guava > ./sdks/java/io/google-cloud-platform/build.gradle: compile library.java.guava > ./sdks/java/io/kinesis/build.gradle: testCompile library.java.guava_testlib > > These results appear to be misleading. Grepping for 'import > com.google.common', I see this as the actual state of things: > > - GCP connector does not appear to actually depend on Guava in compile scope > - The Beam SQL JDBC driver does not appear to actually depend on Guava in > compile scope > - The Dataflow Java worker does depend on Guava at compile scope but has > incorrect dependencies (and it probably shouldn't) > - KinesisIO does depend on Guava at compile scope but has incorrect > dependencies (Kinesis libs have Guava on API surface so it is OK here, but > should be correctly declared) > - ZetaSQL translator does depend on Guava at compile scope but has incorrect > dependencies (ZetaSQL has it on API surface so it is OK here, but should be > correctly declared) > > We used to have an analysis that prevented this class of error. > > Once the errors are fixed, the guava_version is simply a version that we have > discovered that seems to work for both Kinesis and ZetaSQL, libraries we do > not control. Kinesis producer is built against 18.0. Kinesis client against > 26.0-jre. ZetaSQL against 26.0-android. > > (or maybe I messed up in my analysis) > > Kenn > > On Mon, Nov 11, 2019 at 12:07 PM Tomo Suzuki <[email protected] > <mailto:[email protected]>> wrote: > > Chamikara and Yifan, > Thank you for the responses! Looking forward to hearing the investigation > result. > In the meantime, I'll explore .test-infra/jenkins/dependency_check directory. >
