So I have developed for a similar scenario where we had APIs in 3 different
languages that needed to be kept in sync. The releases were split, and we only
had some high level tests to verify compatibility and therefore had to be very
careful what changes we made to avoid failures across many different use cases
of our APIs. There were a few people who were required to review any changes to
this API. I feel that in a project with as many implementations as Avro, it
will be difficult to have this same level of review. If it's possible to write
some linked tests for most of the implementations like Niels suggested, it
should be ok to split the releases. We would still have to pay close attention
to changes in these test cases.
Sam
On Saturday, November 14, 2015 7:54 AM, Niels Basjes <[email protected]>
wrote:
Hi,
First of all a +1 from me to move to git.
Regarding the "how many parts and the release cycle";
In general I'm in the "release often" camp.
Yet I understand the needless confusion if a certain language has not
changed.
How about this idea:
We create a separate project (avro-spec) with the specification/testcases
and such that guard the language specification. All language specific
implementations must reference this (perhaps by using the git feature of a
"linked submodule") and run the tests during the language specific deploy.
The version of this module therefor the version of the Avro format
specification.
I think it would be good if these modules use a version that essentially is
<avro-spec version>-<library version>
This way the libraries can have a separate release cycle and still clearly
indicate the supported Avro specifcation.
My 2ct
Niels Basjes
On Thu, Nov 12, 2015 at 6:34 PM, Ryan Blue <[email protected]> wrote:
> My responses are inline.
>
> rb
>
> On 11/06/2015 07:32 AM, Tom White wrote:
>
>> I'm not sure that moving to a model where there are releases of individual
>> components will increase the frequency of releases. There will still need
>> to be a release manager for each component, and then there's a danger that
>> the less maintained components will not get released at all.
>>
>
> My concern is that new components, like Miki's fastavro (python) and
> Matthieu's js implementation, already have significantly faster release
> cycles (see [1] and [2]). We want those to be successful in the Avro
> community and I think that blocking their fast release cycles on languages
> without maintainers is a community problem. Not delivering timely releases
> discourages participation.
>
> I think a release manager per release, even if it is a python-only
> release, is fine. As I said, we've had less than one release per year
> lately so it isn't going to be that much overhead. And to your point about
> making releases easier, I think it is much easier to learn how to do
> releases with a small project.
>
> We will need to discuss what to do about languages that aren't maintained.
> (Maybe in a separate thread?) I don't think we need to deprecate them as
> Phil suggested, but I also don't think it is right to continue releasing
> code with new version numbers that hasn't been updated in a year or more. I
> think separating the releases is the right thing to do here too: it signals
> to users that the component is still "current" but hasn't been released in
> a while. We're neither discouraging use or participation by deprecating it,
> nor are we making it appear more active than it is.
>
> [1]: https://github.com/tebeka/fastavro/releases
> [2]: https://github.com/mtth/avsc/releases
>
> I would rather continue to make the release process easier (Docker helps a
>> lot) so that any committer can do it. We should be able to use the Docker
>> work to run tests for all components with Jenkins to ensure that trunk is
>> always in a releasable state.
>>
>
> Docker does help quite a lot, but it doesn't help when the problem isn't
> build-related. If Ruby contributors want to get a feature into a release or
> perl has a blocking bug or the C# implementation has license issues, we end
> up blocking all components.
>
> Where are we with the licensing issues? If we can get those worked out then
>> I'd like to make a release (of all components).
>>
>
> The source tarball is nearly ready and a few implementations have updated
> license docs in the binary artifacts. We have a couple days work to go.
>
>
> I'm +1 on moving to git.
>>
>> Cheers,
>> Tom
>>
>> On Thu, Nov 5, 2015 at 12:45 PM, Ryan Blue <[email protected]> wrote:
>>
>> It isn't just license problems, either. Releases that include all of the
>>> languages can be blocked by bugs that need to be fixed in those languages
>>> that are suggested during release planning.
>>>
>>> It is also necessary to make sure the older language implementations
>>> still
>>> build and pass tests, which can mean, for example, installing php and
>>> fixing any tests that currently break. Tom's recent work to port the
>>> build
>>> to docker really helps this situation, but that took patches to
>>> unmaintained implementations and will still require maintenance.
>>>
>>> I also disagree that it's always okay to re-release artifacts. Everything
>>> is moving toward semantic versioning and I think that Avro should as
>>> well.
>>> It is confusing to users to have an identical library released with a
>>> version number that indicates a breaking change (though it appears not to
>>> be by semver rules).
>>>
>>> Each language should adopt a release cadence that works for its
>>> contributors so that those contributors are able to use their work in
>>> timely releases. Otherwise, I'm afraid that we will see fewer
>>> contributions
>>> because of the long release cycle we currently have.
>>>
>>> rb
>>>
>>> On 11/05/2015 10:09 AM, Sean Busbey wrote:
>>>
>>> we are currently blocked on all releases because of licensing errors
>>>> in under-maintained libraries.
>>>>
>>>> https://issues.apache.org/jira/browse/AVRO-1722
>>>>
>>>> essentially Ryan and I slowly work our way through understanding each
>>>> code base enough to do an evaluation and update things.
>>>>
>>>> It's been over 2 months now and it's a crappy situation to put our
>>>> contributors in.
>>>>
>>>>
>>>> On Thu, Nov 5, 2015 at 11:59 AM, Philip Zeyliger <[email protected]>
>>>> wrote:
>>>>
>>>> I think it's always ok to re-release artifacts where nothing's changed.
>>>>> So, how can you be blocked on another language's implementation if you
>>>>> simply change the version number and re-release?
>>>>>
>>>>> -- Philip
>>>>>
>>>>>
>>>>
>>> --
>>> Ryan Blue
>>> Software Engineer
>>> Cloudera, Inc.
>>>
>>>
>>
>
> --
> Ryan Blue
> Software Engineer
> Cloudera, Inc.
>
--
Best regards / Met vriendelijke groeten,
Niels Basjes
