It sounds like there is consensus around separating the languages into separate releases and possibly projects. Should we consider a separate discuss & vote thread for that alone? I don't think there's a need for an improvement proposal there.
On Sun, May 3, 2020 at 7:47 AM Andy Le <[email protected]> wrote: > +1 the project structure and more documentation. > > I'm really impressed with the way Vertx documentation organized. > > Finally, so what should we do? IMHO, we should: > > - Clear about the organization > - Brainstorming what to do in a plan > - Publicly show the plan & call for contributions > > I'm a newcomer here. But it's great if I can help any thing. > > On 2020/05/03 07:29:02, "Driesprong, Fokko" <[email protected]> wrote: > > Yes, I'd love to see the AEPs being revived. This would enable it to make > > more structural/fundamental changes, such as the one mentioned above. > This > > also enables others to have a complete overview of changes and the > impact. > > > > I also agree with Michael when it comes to the project structure. Maybe > we > > can look at other projects, such as Thrift and Protobuf, how they handle > > this. > > > > Cheers, Fokko > > > > Op wo 29 apr. 2020 om 05:01 schreef Michael A. Smith < > [email protected]>: > > > > > Definitely in favor of the AEPs. > > > > > > I have mixed feelings about the notion of a single language binding. > These > > > types of packages are commonplace enough in Python, but installing a > Python > > > package that requires a build time clib is often troublesome, and I'm > often > > > grateful to find a pure Python alternative. > > > > > > But I also think the project suffers from the current language > structure – > > > it encourages implementations to be similar to one another over being > > > idiomatic in their own language, and we can't do a release for just one > > > language. If boiling down the project into a single core lib for > multiple > > > languages is what it takes to fix that, then it's probably for the > best. > > > > > > If we do go down that road, I'd like to see us make the build process > > > extremely smooth. > > > > > > All the best, > > > Michael A. Smith > > > > > > On Tue, Apr 28, 2020 at 04:01 Ismaël Mejía <[email protected]> wrote: > > > > > > > Huge +1 to recover the Avro Enhancement Proposals (AEP) > > > > > > > > The experimental features Ryan mentioned definitely merit(ed) to be > > > > part of it, and in particular the procedure to decide when they will > > > > become ‘stable’ or default, for example for fastread. Also other > > > > proposals/discussions like the split release or semantic versioning > > > > should be part of it. > > > > > > > > About Avro 2.0.0 I think breaking binary compatibility of the format > > > > is going to prove to be a hard sell (are named unions valuable enough > > > > to break backwards compatibility?), if we can extend the binary > format > > > > in a compatible way there is no reason to have 2.0.0 so I agree that > > > > there is a delicate balance we should avoid because strict stability > > > > could let us also ostracized. > > > > > > > > What I personally would like is to make Avro as lean and efficient as > > > > possible and focus mostly in the binary format part and tools > probably > > > > removing the less used parts (IPC/RPC/trevni) so it is good to see > > > > that other people are starting to agree on that. > > > > > > > > One more radical idea I would like is to try is to unify a bit the > > > > implementations probably having a robust low level one in one systems > > > > language (C or Rust) and bindings for all the languages that rely on > > > > it but this is probably more because of my frustration of seeing > > > > projects that take this approach becoming slowly the standard and > > > > Apacho Avro relegated (this is already happening on the python > front). > > > > > > > > In general the critical issue with Avro are the downstream > > > > consequences of our actions, and of course we will always have > > > > incomplete information, but we can investigate and see if changes are > > > > worth. > > > > > > > > Regards, > > > > Ismaël > > > > > > > > On Mon, Apr 27, 2020 at 6:51 PM Ryan Skraba <[email protected]> wrote: > > > > > > > > > > Hello! > > > > > > > > > > You bring up some good points -- I'm glad Avro is so widely used, > but > > > > > it does make me nervous to see any changes that might break other > > > > > projects, or change any behaviour. > > > > > > > > > > Currently, we've talked about managing developer expectations with > > > > > semantic versioning (especially with the necessary Jackson API > cleanup > > > > > that happened in 1.9.x), or versioning artifacts separately. > > > > > > > > > > We also have a couple of experimental/feature flags for some > behaviour > > > > > changes: > > > > > > > > https://cwiki.apache.org/confluence/display/AVRO/Experimental+features+in+Avro > > > > > > > > > > And there is already a page for Avro Enhancement Proposals that > look > > > > > largely out of date: > > > > > > > > > > > > > https://cwiki.apache.org/confluence/display/AVRO/Avro+Enhancement+Proposals > > > > > > > > > > Moving some of the extras to a separate repo brings many of the > same > > > > > problems as versioning artifacts separately (nobody wants to deal > with > > > > > a compatibility matrix). I'm definitely not against it, but I'm > not > > > > > sure how it would improve the situation. > > > > > > > > > > There's a fine line between being extremely stable and being > > > > > paralyzed! I would be enthusiastic about any process changes that > > > > > would help us encourage and adopt new features (and fixes) more > > > > > quickly. > > > > > > > > > > All my best, Ryan > > > > > > > > > > > > > > > On Sun, Apr 26, 2020 at 11:18 AM Driesprong, Fokko > > > <[email protected]> > > > > wrote: > > > > > > > > > > > > Hi Andy, > > > > > > > > > > > > Thanks for reaching out. Sorry for not being so active in the > > > community > > > > > > lately. > > > > > > > > > > > > Since Avro 1.8.2 there has been some activity on the repository > > > again, > > > > > > fixing stuff like security issues and migrating to later > versions of > > > > Java. > > > > > > Avro has been around for 10 years now, and I would like to keep > > > (some) > > > > > > backward compatibility to make sure that people are still going > to > > > use > > > > it > > > > > > for another 10 years :) In the past, the idea was to keep the > format > > > > > > backward compatibility, this excludes the Java API to. So we did > some > > > > > > changes to the API, such as removing Jackson from the public API > and > > > > > > aggressively migrating from Joda Time to Java JSR-310. This > caused a > > > > lot of > > > > > > issues because Avro is deeply nested in a lot of projects. For > > > > example, it > > > > > > is a huge task to update Avro in Hive or Hadoop. Therefore we > believe > > > > that > > > > > > backward compatibility is very important. > > > > > > > > > > > > And I agree that we should mainly focus on the Avro spec itself, > and > > > > not > > > > > > too much on File I/O and Network etc :) However, if we decide to > > > break > > > > an > > > > > > API, we should do it for a good reason. > > > > > > > > > > > > Cheers, Fokko > > > > > > > > > > > > Op wo 22 apr. 2020 om 16:09 schreef Andy Le <[email protected]>: > > > > > > > > > > > > > Hi guys, > > > > > > > > > > > > > > I'm new to this vibrant open source community. My story with > Avro > > > > can be > > > > > > > found here [1] > > > > > > > > > > > > > > While implementing the feature, I got stuck and had various > > > > discussions > > > > > > > with Dough Cutting, Fokko Driesprong.... You may see here [2] > > > > > > > > > > > > > > Here my (bias) observations about our current Avro 1.9.x: > > > > > > > > > > > > > > - Some improvements can't be made due to fear of backward > > > > > > > incompatibilities. For example: specifications about named > Union. > > > > > > > > > > > > > > - If `Apache Avro™ is a data serialization system.` then the > > > > repository > > > > > > > `apache/avro` should solely focus on (de)serialization, right? > > > > Currently > > > > > > > our repository contains many nice-to-have-but-not-critical > things > > > > like: > > > > > > > File I/O, Network I/O.... > > > > > > > > > > > > > > IMHO, I think: > > > > > > > > > > > > > > - We should publicly gather RFCs for Avro 2.x > > > > > > > > > > > > > > - We should move such nice things out of Avro 2.x (may be to > other > > > > > > > dedicated repositories) > > > > > > > > > > > > > > What do you think about my suggestions. Pls kindly let me know. > > > > > > > > > > > > > > Thank you & be strong. > > > > > > > > > > > > > > [1] My fork: > https://github.com/anhldbk/avro-fork#why-this-fork > > > > > > > [2] My opened issue: > > > > > > > > > > > > > > > https://issues.apache.org/jira/browse/AVRO-2808?jql=reporter%3Danhldbk%20AND%20resolution%20is%20EMPTY > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- Ryan Blue Software Engineer Netflix
