On Thu, Nov 14, 2019 at 8:04 AM Alexey Romanenko <[email protected]> wrote:
> Good example about Guava deps, let me go a bit deeper. > > $ find . -name build.gradle | xargs grep library.java.guava > ./sdks/java/core/build.gradle: shadowTest library.java.guava_testlib > > ./sdks/java/io/kinesis/build.gradle: testCompile > library.java.guava_testlib > > > Regarding using non-vendored Guava in KinesisIO (and "java/core" as well), > it’s all about *“library.java.guava_testlib” *and > *“com.google.common.testing.EqualsTester”* only in particular, which is > used for tests. > Do we need to vendor “*com.google.guava:guava-testlib*” for this in this > case? > I didn't worry about test scopes because they don't ship to users so much. It could be useful if there is a runner with a conflict and they want to run the integration tests. > - KinesisIO does depend on Guava at compile scope but has incorrect > dependencies (Kinesis libs have Guava on API surface so it is OK here, but > should be correctly declared) > > > Sorry, but I didn’t understand what do you mean by “*but should be > correctly declared*”. > Since Kinesis client libs have own Guava deps and we shade our own guava, > so it should be fine, no? > I mean that any module with an "import X" should have a dependency on X in its build.gradle. When you leave it off, the dep analysis (such as Maven's) calls it out as "used undeclared dependency". Kenn > > On 11 Nov 2019, at 22:29, Kenneth Knowles <[email protected]> wrote: > > BeamModulePlugin just contains lists of versions to ease coordination > across Beam modules, but mostly does not create dependencies. Most of > Beam's modules only depend on a few things there. For example Guava is not > a core dependency, but here is where it is actually depended upon: > > $ find . -name build.gradle | xargs grep library.java.guava > ./sdks/java/core/build.gradle: shadowTest library.java.guava_testlib > ./sdks/java/extensions/sql/jdbc/build.gradle: compile library.java.guava > ./sdks/java/io/google-cloud-platform/build.gradle: compile > library.java.guava > ./sdks/java/io/kinesis/build.gradle: testCompile > library.java.guava_testlib > > These results appear to be misleading. Grepping for 'import > com.google.common', I see this as the actual state of things: > > - GCP connector does not appear to actually depend on Guava in compile > scope > - The Beam SQL JDBC driver does not appear to actually depend on Guava in > compile scope > - The Dataflow Java worker does depend on Guava at compile scope but has > incorrect dependencies (and it probably shouldn't) > - KinesisIO does depend on Guava at compile scope but has incorrect > dependencies (Kinesis libs have Guava on API surface so it is OK here, but > should be correctly declared) > - ZetaSQL translator does depend on Guava at compile scope but has > incorrect dependencies (ZetaSQL has it on API surface so it is OK here, but > should be correctly declared) > > We used to have an analysis that prevented this class of error. > > Once the errors are fixed, the guava_version is simply a version that we > have discovered that seems to work for both Kinesis and ZetaSQL, libraries > we do not control. Kinesis producer is built against 18.0. Kinesis client > against 26.0-jre. ZetaSQL against 26.0-android. > > (or maybe I messed up in my analysis) > > Kenn > > On Mon, Nov 11, 2019 at 12:07 PM Tomo Suzuki <[email protected]> wrote: > >> >> Chamikara and Yifan, >> Thank you for the responses! Looking forward to hearing the investigation >> result. >> In the meantime, I'll explore .test-infra/jenkins/dependency_check >> directory. >> >> >
