This is really cool and should cut down our artifact size significantly!
Thanks Luke!

I am running into one issue after this: builds with the publishing flag no
longer work. (We run './gradlew -Ppublishing shadowJar' to generate release
artifacts for the Beam SQL shell.) I get a bunch of errors like this:

error: no suitable method found for

Is there something I need to change in my build?


On Tue, Jul 10, 2018 at 2:10 PM Lukasz Cwik <> wrote:

> With the merge of PR #5594[1], we started shading all gRPC / Protobuf
> dependencies within all the modules that depended on the model/*
> dependencies by vendoring them. The vendored versions are built and
> packaged into the model jars (they should be separated out once I figure
> out how to generate proto code using a shaded import path). Note that this
> cleaned up several issues where we were incorrectly built shaded jars
> without repackaging in some locations or the shading process was corrupting
> the contents of some of the jars.
> Note that the majority of the code base (especially related to
> portability) should be using imports under the
> org.apache.beam.vendor.protobuf.v3 or org.apache.beam.vendor.grpc.v1 paths.
> I have yet to figure out a clean way to get Intellij to recognize these
> vendored paths. My only solution so far has been to manually add one of the
> built model jars to the compile classpath of the module being worked on in
> Intellij as described here[2]. I would greatly appreciate some ideas on how
> to improve this integration because from a few attempts configuring the
> intellij gradle pluglin scope sections didn't produce the result that I was
> expecting.
> I also added a simple test task
> *validateShadedJarDoesntLeakNonOrgApacheBeamClasses* that validates the
> shaded jar doesn't contain classes without repackaging which we should
> apply to any module that performs shading to ensure that classes are
> relocated and we don't accidentally expose stuff. I filed BEAM-4753[3] to
> this end.
> Note, these are the following modules that still depend on protobuf that
> are shaded away and could move to use a vendored variant of protobuf:
> * sdks/java/core
> * sdks/java/extensions/sql
> These modules expose protobuf because it is part of the API surface:
> * sdks/java/extensions/protobuf
> * sdks/java/io/google-cloud-platform (I believe that gRPC could be shaded
> here but preferrably the IO module would do it so we wouldn't have this
> maintenance burden.)
> 1:
> 2:
> 3:

Reply via email to