Consensus is a big word for this case. I am not really convinced this will get us to a better state for the reasons already exposed before. In my opinion we are having a BIG case of wishful thinking.
We had a really hard time to put together a new release for 1.9.0 and one of the reasons was the lack of people contributing/actively maintaining the different languages and reviewing the PRs (and yes this was even for Java aka our best maintained implementation), so I cannot imagine that separating languages will improve this 'au contraire' I expect that neglect will become bigger for less maintained languages, compatibility issues will be more present and confusion for users with multiple versioning will arise. Maybe we should re evaluate what is separating releases per language trying to solve that we cannot solve by producing unified releases more regularly, for example doing a full release every 2 months? Sorry I went into the discussion, maybe better to do this in a separate thread. On Sun, May 3, 2020 at 8:45 PM Ryan Blue <[email protected]> wrote: > > It sounds like there is consensus around separating the languages into > separate releases and possibly projects. Should we consider a separate > discuss & vote thread for that alone? I don't think there's a need for an > improvement proposal there. > > On Sun, May 3, 2020 at 7:47 AM Andy Le <[email protected]> wrote: > > > +1 the project structure and more documentation. > > > > I'm really impressed with the way Vertx documentation organized. > > > > Finally, so what should we do? IMHO, we should: > > > > - Clear about the organization > > - Brainstorming what to do in a plan > > - Publicly show the plan & call for contributions > > > > I'm a newcomer here. But it's great if I can help any thing. > > > > On 2020/05/03 07:29:02, "Driesprong, Fokko" <[email protected]> wrote: > > > Yes, I'd love to see the AEPs being revived. This would enable it to make > > > more structural/fundamental changes, such as the one mentioned above. > > This > > > also enables others to have a complete overview of changes and the > > impact. > > > > > > I also agree with Michael when it comes to the project structure. Maybe > > we > > > can look at other projects, such as Thrift and Protobuf, how they handle > > > this. > > > > > > Cheers, Fokko > > > > > > Op wo 29 apr. 2020 om 05:01 schreef Michael A. Smith < > > [email protected]>: > > > > > > > Definitely in favor of the AEPs. > > > > > > > > I have mixed feelings about the notion of a single language binding. > > These > > > > types of packages are commonplace enough in Python, but installing a > > Python > > > > package that requires a build time clib is often troublesome, and I'm > > often > > > > grateful to find a pure Python alternative. > > > > > > > > But I also think the project suffers from the current language > > structure – > > > > it encourages implementations to be similar to one another over being > > > > idiomatic in their own language, and we can't do a release for just one > > > > language. If boiling down the project into a single core lib for > > multiple > > > > languages is what it takes to fix that, then it's probably for the > > best. > > > > > > > > If we do go down that road, I'd like to see us make the build process > > > > extremely smooth. > > > > > > > > All the best, > > > > Michael A. Smith > > > > > > > > On Tue, Apr 28, 2020 at 04:01 Ismaël Mejía <[email protected]> wrote: > > > > > > > > > Huge +1 to recover the Avro Enhancement Proposals (AEP) > > > > > > > > > > The experimental features Ryan mentioned definitely merit(ed) to be > > > > > part of it, and in particular the procedure to decide when they will > > > > > become ‘stable’ or default, for example for fastread. Also other > > > > > proposals/discussions like the split release or semantic versioning > > > > > should be part of it. > > > > > > > > > > About Avro 2.0.0 I think breaking binary compatibility of the format > > > > > is going to prove to be a hard sell (are named unions valuable enough > > > > > to break backwards compatibility?), if we can extend the binary > > format > > > > > in a compatible way there is no reason to have 2.0.0 so I agree that > > > > > there is a delicate balance we should avoid because strict stability > > > > > could let us also ostracized. > > > > > > > > > > What I personally would like is to make Avro as lean and efficient as > > > > > possible and focus mostly in the binary format part and tools > > probably > > > > > removing the less used parts (IPC/RPC/trevni) so it is good to see > > > > > that other people are starting to agree on that. > > > > > > > > > > One more radical idea I would like is to try is to unify a bit the > > > > > implementations probably having a robust low level one in one systems > > > > > language (C or Rust) and bindings for all the languages that rely on > > > > > it but this is probably more because of my frustration of seeing > > > > > projects that take this approach becoming slowly the standard and > > > > > Apacho Avro relegated (this is already happening on the python > > front). > > > > > > > > > > In general the critical issue with Avro are the downstream > > > > > consequences of our actions, and of course we will always have > > > > > incomplete information, but we can investigate and see if changes are > > > > > worth. > > > > > > > > > > Regards, > > > > > Ismaël > > > > > > > > > > On Mon, Apr 27, 2020 at 6:51 PM Ryan Skraba <[email protected]> wrote: > > > > > > > > > > > > Hello! > > > > > > > > > > > > You bring up some good points -- I'm glad Avro is so widely used, > > but > > > > > > it does make me nervous to see any changes that might break other > > > > > > projects, or change any behaviour. > > > > > > > > > > > > Currently, we've talked about managing developer expectations with > > > > > > semantic versioning (especially with the necessary Jackson API > > cleanup > > > > > > that happened in 1.9.x), or versioning artifacts separately. > > > > > > > > > > > > We also have a couple of experimental/feature flags for some > > behaviour > > > > > > changes: > > > > > > > > > > > https://cwiki.apache.org/confluence/display/AVRO/Experimental+features+in+Avro > > > > > > > > > > > > And there is already a page for Avro Enhancement Proposals that > > look > > > > > > largely out of date: > > > > > > > > > > > > > > > > > https://cwiki.apache.org/confluence/display/AVRO/Avro+Enhancement+Proposals > > > > > > > > > > > > Moving some of the extras to a separate repo brings many of the > > same > > > > > > problems as versioning artifacts separately (nobody wants to deal > > with > > > > > > a compatibility matrix). I'm definitely not against it, but I'm > > not > > > > > > sure how it would improve the situation. > > > > > > > > > > > > There's a fine line between being extremely stable and being > > > > > > paralyzed! I would be enthusiastic about any process changes that > > > > > > would help us encourage and adopt new features (and fixes) more > > > > > > quickly. > > > > > > > > > > > > All my best, Ryan > > > > > > > > > > > > > > > > > > On Sun, Apr 26, 2020 at 11:18 AM Driesprong, Fokko > > > > <[email protected]> > > > > > wrote: > > > > > > > > > > > > > > Hi Andy, > > > > > > > > > > > > > > Thanks for reaching out. Sorry for not being so active in the > > > > community > > > > > > > lately. > > > > > > > > > > > > > > Since Avro 1.8.2 there has been some activity on the repository > > > > again, > > > > > > > fixing stuff like security issues and migrating to later > > versions of > > > > > Java. > > > > > > > Avro has been around for 10 years now, and I would like to keep > > > > (some) > > > > > > > backward compatibility to make sure that people are still going > > to > > > > use > > > > > it > > > > > > > for another 10 years :) In the past, the idea was to keep the > > format > > > > > > > backward compatibility, this excludes the Java API to. So we did > > some > > > > > > > changes to the API, such as removing Jackson from the public API > > and > > > > > > > aggressively migrating from Joda Time to Java JSR-310. This > > caused a > > > > > lot of > > > > > > > issues because Avro is deeply nested in a lot of projects. For > > > > > example, it > > > > > > > is a huge task to update Avro in Hive or Hadoop. Therefore we > > believe > > > > > that > > > > > > > backward compatibility is very important. > > > > > > > > > > > > > > And I agree that we should mainly focus on the Avro spec itself, > > and > > > > > not > > > > > > > too much on File I/O and Network etc :) However, if we decide to > > > > break > > > > > an > > > > > > > API, we should do it for a good reason. > > > > > > > > > > > > > > Cheers, Fokko > > > > > > > > > > > > > > Op wo 22 apr. 2020 om 16:09 schreef Andy Le <[email protected]>: > > > > > > > > > > > > > > > Hi guys, > > > > > > > > > > > > > > > > I'm new to this vibrant open source community. My story with > > Avro > > > > > can be > > > > > > > > found here [1] > > > > > > > > > > > > > > > > While implementing the feature, I got stuck and had various > > > > > discussions > > > > > > > > with Dough Cutting, Fokko Driesprong.... You may see here [2] > > > > > > > > > > > > > > > > Here my (bias) observations about our current Avro 1.9.x: > > > > > > > > > > > > > > > > - Some improvements can't be made due to fear of backward > > > > > > > > incompatibilities. For example: specifications about named > > Union. > > > > > > > > > > > > > > > > - If `Apache Avro™ is a data serialization system.` then the > > > > > repository > > > > > > > > `apache/avro` should solely focus on (de)serialization, right? > > > > > Currently > > > > > > > > our repository contains many nice-to-have-but-not-critical > > things > > > > > like: > > > > > > > > File I/O, Network I/O.... > > > > > > > > > > > > > > > > IMHO, I think: > > > > > > > > > > > > > > > > - We should publicly gather RFCs for Avro 2.x > > > > > > > > > > > > > > > > - We should move such nice things out of Avro 2.x (may be to > > other > > > > > > > > dedicated repositories) > > > > > > > > > > > > > > > > What do you think about my suggestions. Pls kindly let me know. > > > > > > > > > > > > > > > > Thank you & be strong. > > > > > > > > > > > > > > > > [1] My fork: > > https://github.com/anhldbk/avro-fork#why-this-fork > > > > > > > > [2] My opened issue: > > > > > > > > > > > > > > > > > > > https://issues.apache.org/jira/browse/AVRO-2808?jql=reporter%3Danhldbk%20AND%20resolution%20is%20EMPTY > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > Ryan Blue > Software Engineer > Netflix
