I disagree that Avro is heading towards the Attic. Its rate of contribution has been slow but steady for years. That's the nature of this project. It had a larger number of contributions in its first few years, when new languages and substantial features were being added, but since then, we see a relatively steady stream of bug fixes and minor features. Here are issues fixed per year:
2009 213 2010 345 2011 240 2012 183 2013 130 2014 103 2015 71 2016 107 This year is actually an uptick from the past few. Projects go to the Attic when there are zero commits and zero releases for years, with fewer than 3 PMC members who'll respond to emails at all. We're committing several contributions per week. It sometimes takes a little prodding to get enough PMC members to vote on a release, but they're out there and will eventually vote and help get a release out. Pre-commit tests would be great to have, facilitating development & releases. +1 Breaking the project into multiple products, each released separately, could be a significant task, especially when you include getting the first release out for each. On one hand, it would make per-language releases easier, but it might also let some build problems languish undetected. Build issues are sometimes the result of other projects and distributions changing and may only be detected when you re-build, even if code hasn't changed. Lastly, getting PMC reviews & votes for more-frequent, single-language releases might be harder than less-frequent, multi-language releases. To be clear, I don't strongly oppose splitting things up, but I don't think it will be easy & may create some new problems as it resolves existing ones. I installed a new version of Linux on my laptop a few months ago and was able to recreate Avro's docker image relatively quickly and painlessly. I just now started it for the first time since, and it took 10 minutes to update. Building all languages took another 5 minutes. Each step required only a single command and proceeded without failure. It's a little slow, but not prohibitive. Overall the process is much better than before we had docker. Sure, it could be improved, but it's not unworkable. I find other steps of releases more tiresome. Would it be nice to be able to get a Java release out more quickly and easily? For sure, but getting there might not be that simple. Are you proposing to drive this effort, following it through to completion, nursing it if it stumbles, perhaps reverting it if it somehow fails? (Hadoop was once split & reunified.) To some degree, splitting assumes that each implementation has a sufficient community to maintain itself independently. We have a viable community combined, but I worry that breaking the project up could fragment it into projects that are too tiny to make releases. Right now we're all forced to care a bit about other implementations. Doug On Sun, Nov 13, 2016 at 12:21 PM, Ryan Blue <[email protected]> wrote: > Hi everyone, > > We tried to release Avro 1.8.2 this week, but the release vote failed > because only two PMC members voted on the candidate and we didn't have > enough binding votes to pass. There was a minor problem (in my opinion) > with the candidate that could have been the reason why there weren't more > votes. If there's anyone out there that didn't vote because of this, please > say so. Otherwise, this appears to be related to declining engagement in > the community and that's a major problem I want to discuss. > > Right now, we aren't getting contributions reviewed, committed, and > released in time for new contributors to become part of the community and > refresh the set of active committers and PMC members. If that continues, > this community is heading for the Attic. I think we can build back an > active committer base and I'd love to discuss how to do that on this thread. > > To get us started, I have a couple of ideas. > > I think we need to make it easier to participate. I've brought up these > ideas before when we moved to git: I think we need to separate the > implementations into their own repositories and set up CI tests for each > one. > > Right now, a contributor has to wait for a review to get feedback and a > committer has to build a contribution and run tests. If we set up CI, then > the contributor gets automated feedback and committers can spend their time > on substantive review, rather than making sure the patch builds and tests > pass. Making it easier for contributors and committers will help increase > participation. > > Similarly, the build and release process is too difficult. It took me hours > to get the docker image built so I could make a release candidate, because > of a failure rate of about 1/500 downloading and installing packages. I had > to try ~20 times before one happily completed. While the docker image helps > a lot, the real problem we need to solve is how difficult it is to build > all of Avro. The docker image helps, but no one really uses it until it's > time to check a release. Instead, we all build and test how we are used to > for a particular language implementation: maven for Java, Rake for ruby, > etc. That's why the build.sh scripts get broken and we don't notice, and > why the only problem with the latest RC was that it didn't pass C# tests > outside of docker. > > The current build also makes implementation releases dependent on one > another. Last release, C and ruby problems caused a multi-week delay, and > this release we want to get the C# test environment fixed before the next > candidate. All of this makes it take longer for contributions to make it > out, which undermines the motivation for people to contribute. > > Separating the implementations will allow us to structure each repository > how the contributors and committers for that language expect it to be. We > can also set up per-implementation CI easily through Travis CI. And the > biggest benefit is separating the releases, so that Python, for example, > can release a bug fix without waiting months for unrelated changes in Java. > > From my perspective, these two things are a good place to start. To > everyone still reading, what do you think? > > rb > > > -- > Ryan Blue
