The new protobuf plugin related issues have all been pushed to trunk(though I think we'd better port them to all active branches).
So what's the next step? Shade and relocated protobuf? HBase has already done this before so I do not think it will take too much time. If we all agree on the solution, I think we can finish this in one week. But maybe a problem is that, is it OK to upgrade protobuf in a minor release? Of course if we shade and relocate protobuf it will less hurt to users as they can depend on protobuf 2.5 explicitly if they want, but still a bit uncomfortable. Thanks. Wangda Tan <wheele...@gmail.com> 于2019年9月24日周二 上午2:29写道： > Hi Vinay, > > Thanks for the clarification. > > Do you have a timeline about all you described works w.r.t. the > compatibility will be completed? I'm asking this is because we need to > release 3.3.0 earlier if possible since there're 1k+ of patches in 3.3.0 > already, we should get it out earlier. > > If the PB work will take more time, do you think if we should create a > branch for 3.3, revert PB changes from branch-3.3, and keep on working on > PB for the next minor release? (or major release if we do see some > compatibility issue in the future). > > Just my $0.02 > > Thanks, > Wangda > > On Mon, Sep 23, 2019 at 5:43 AM Steve Loughran <ste...@cloudera.com.invalid > > > wrote: > > > aah, that makes sense > > > > On Sun, Sep 22, 2019 at 6:11 PM Vinayakumar B <vinayakum...@apache.org> > > wrote: > > > > > Thanks Steve. > > > > > > Idea is not to shade all artifacts. > > > Instead maintain one artifact ( hadoop-thirdparty) which have all such > > > dependencies ( com.google.* may be), add this artifact as dependency > in > > > hadoop modules. Use shaded classes directly in the code of hadoop > modules > > > instead of shading at package phase. > > > > > > Hbase, ozone and ratis already following this way. The artifact ( > > > hadoop-thirdparty) with shaded dependencies can be maintained in a > > separate > > > repo as suggested by stack on HADOOP-13363 or could be maintained as a > > > separate module in Hadoop repo. If maintained in separate repo, need to > > > build this only when there are changes related to shaded dependencies. > > > > > > > > > -Vinay > > > > > > On Sun, 22 Sep 2019, 10:11 pm Steve Loughran, <ste...@cloudera.com> > > wrote: > > > > > > > > > > > > > > > On Sun, Sep 22, 2019 at 3:22 PM Vinayakumar B < > vinayakum...@apache.org > > > > > > > wrote: > > > > > > > >> Protobuf provides Wire compatibility between releases.. but not > > > >> guarantees the source compatibility in generated sources. There will > > be > > > a > > > >> problem in compatibility if anyone uses generated protobuf message > > > outside > > > >> of Hadoop modules. Which ideally shouldn't be as generated sources > are > > > not > > > >> public APIs. > > > >> > > > >> There should not be any compatibility problems between releases > in > > > >> terms > > > >> of communication provided both uses same syntax (proto2) of proto > > > message. > > > >> This I have verified by communication between protobuf 2.5.0 client > > with > > > >> protobuf 3.7.1 server. > > > >> > > > >> To avoid the downstream transitive dependency classpath problem, > > who > > > >> might be using protobuf 2.5.0 classes, planning to shade the 3.7.1 > > > classes > > > >> and its usages in all hadoop modules.. and keep 2.5.0 jar back in > > hadoop > > > >> classpath. > > > >> > > > >> Hope I have answered your question. > > > >> > > > >> -Vinay > > > >> > > > >> > > > > While I support the move and CP isolation, this is going to (finally) > > > > force us to make shaded versions of all artifacts which we publish > with > > > the > > > > intent of them being loaded on the classpath of other applications > > > > > > > > > >
