On Fri, Jan 17, 2020 at 10:21 AM Elliotte Rusty Harold <elh...@ibiblio.org> wrote:
> Thanks. That's a very clear explanation. I should think about how to > incorporate that into https://jlbp.dev Might want to give it some time before calling it a best practice. Maybe we'll come to regret it :-). But I'm willing to say it is a better alternative to shading in most cases, with the major exception being generated code (like proto libs) that depend on libraries (like protobuf-java). Ideally, proto compiler would vendor the dep into each generated lib so we don't have to deal with the issue at all. Instead we do it ourselves. Kenn > > > On Fri, Jan 17, 2020 at 12:12 PM Aaron Dixon <atdi...@gmail.com> wrote: > > > > @Tomo Suzuki Thanks for looking at this and your thorough analysis// let > me know if I can help with any regression testing, CRs, or more context > anything else. > > > > --- > > > > @Elliotte I was a little confused at first re: vendoring vs shading but > here is how I understand it (happy to be corrected if anything I say here > is a mischaracterization); I tend to over-explain for my own clarity > fwiw--using Guava as an example: > > > > With SHADING you have your project (e.g., Beam) depend directly on Guava > and its public package names. Your build will then compile and link to > these package names and only after-the-fact [1] rewrite the byte-code to > rename the guava package. This entails both byte-code rewriting the Guava > class files (to rename the packages of the Guava code itself) and byte-code > rewriting all of your project class files that import Guava (to change the > import to use the new package name). > > > > With VENDORING you release a renamed Guava ("vendor") JAR and depend on > that directly from your project. So your project's source code itself > imports the (repackated) vendor JAR and depends on the renamed packages > directly. Your project build then has no need for shading or byte code > rewriting of its own classes. I see vendoring making it very clear what is > going on when looking at a project's source code and its dependencies, and > there are likely other tangible brass-tacks benefits to vendoring besides > the conceptual clarity over shading. > > > > [1] You can see that Maven shade:shade goal binds by default to Maven's > package phase (which is well after after compilation). > > > > On Fri, Jan 17, 2020 at 9:16 AM Tomo Suzuki <suzt...@google.com> wrote: > >> > >> Ismaël and I think the short-term solution for Aaron's issue is to > >> define Beam's own class for Avro's TimestampConversion. Luckily it's > >> the only blocker for Avro 1.9 I found. > >> I created a ticket https://issues.apache.org/jira/browse/BEAM-9144 and > >> PR https://github.com/apache/beam/pull/10628 . > >> > >> Regards, > >> Tomo > >> > >> On Fri, Jan 17, 2020 at 7:32 AM Elliotte Rusty Harold > >> <elh...@ibiblio.org> wrote: > >> > > >> > On Thu, Jan 16, 2020 at 7:08 PM Kenneth Knowles <k...@apache.org> > wrote: > >> > > > >> > > Regarding Google's advice about shading: don't go to a "one version > rule" monorepo for advice about solving diamond dependencies in the wild. > >> > > >> > FWIW these docs came out of the open source, Github, multirepo, many > >> > versions part of Google Cloud, not the mono repo world of google3. > >> > Though I will say the more time I spend trying to deal with these > >> > issues the more I love the monorepo. > >> > > >> > > It is a useful description of the pitfalls. We (and Flink before > us, and likely many more) are already doing something that avoids many of > them or makes them less likely. Building a separate vendored library is > simpler and more robust than the "shade during build". > >> > > > >> > > >> > I've only encountered "vendoring" as distinct from shading in Beam. > >> > Are there more details about this anywhere? A quick Google search only > >> > turned up one Beam doc that pointed to Go docs and a cloud endpoints > >> > doc that seemed to use it as a synonym for shading. > >> > > >> > > >> > -- > >> > Elliotte Rusty Harold > >> > elh...@ibiblio.org > >> > >> > >> > >> -- > >> Regards, > >> Tomo > > > > -- > Elliotte Rusty Harold > elh...@ibiblio.org >
