Hi, I already proposed release per module while ago. It doesn't need any change on the existing module, just a different versioning.
It's a little bit more work for the release manager to do a full release, but it gives more flexibility. It allows us also do use several branches for the same IO for instance. I already doing Apache ServiceMix Bundles and Specs release per release. So I can prepare some details about that. Regards JB On 25/06/2018 17:48, Ismaël Mejía wrote: > Agree with Kenn, granularity would be key for per-module Beam releases. > > We can easily think today of releasing separately IOs and extensions > because they depend only on the public API of the SDK which has not > changed much since 2.0.0. So scio will probably fit this category. > > A different story would be for runners at least at this moment with all > the ongoing work on portability, there could be important benefits on > having fixes and refactors applied globally when multiple internal > changes are happening. > > In the end separating modules for releases has its pros/cons. In the pro > side it encourages the project to be more stable and robust, but adds > the weight of guaranteeing that everything works together given the > multiple combination of versions released which has of course a price > (this can be a burden for some users if we consider that users should > also deal with the other dependencies of their own). Today Beam users > get this for free with the monolithic release approach. > > As usual with software trade-offs trade-offs. > > > > > > On Fri, Jun 22, 2018 at 7:53 PM Kenneth Knowles <[email protected] > <mailto:[email protected]>> wrote: > > I'm generally in favor of greater decoupling / decentralization > where possible. It is easy to imagine a world in which Beam consists > of a few fairly autonomous projects, some centralized policy, opt-in > infrastructure, just like ~all non-tiny software > projects/foundations/companies. > > The granularity matters quite a lot. But in my experience if you > choose subprojects to match clear self-sustaining subcommunities > then each one moves faster and has a stronger community than when > you try to erase those distinctions in a giant monoproject. > > Starting with an active project that already exists and has users > and maintainers works perfectly for getting the granularity right. > I'm for trying it if/when the opportunity arises. There are > technical challenges around shared infrastructure and global quality > assurance. It would be an opportunity to make them concrete and > address them. > > Kenn > > > > On Fri, Jun 22, 2018 at 10:24 AM Rafael Fernandez > <[email protected] <mailto:[email protected]>> wrote: > > I like the idea of per-module releases for Beam. I know Henning > and others have thought about that space as well. > > BTW, I'm a big fan of scio, and happy to help in any way > possible if they are interested in turning "de facto" into "de > jure" :D > > In such a world, Scio could be a very good first use case to > drive the mechanisms to enable per-module releases. I think it > allows us to scale better and sets a healthy path for special > interest groups to naturally emerge and collaborate with their > own scope in mind. > > > On Thu, Jun 21, 2018 at 11:48 PM [email protected] > <mailto:[email protected]> > <[email protected] > <mailto:[email protected]>> wrote: > > > > On 2018/06/21 17:17:36, Reuven Lax <[email protected]> wrote: > > In that case things have changed since I talked to Neville > about it last > > November. > > > > On Thu, Jun 21, 2018 at 10:16 AM Rafal Wojdyla > <[email protected]> wrote: > > > > > Nope - it uses standard runners and is fully Beam compliant. > > > > > > On Thu, Jun 21, 2018 at 1:12 PM, Reuven Lax > <[email protected]> wrote: > > > > > >> My understanding was that under the covers it used the > low-level Dataflow > > >> service API to run the evaluations. > > >> > > >> On Thu, Jun 21, 2018 at 10:10 AM Rafal Wojdyla > <[email protected]> wrote: > > >> > > >>> Hi. > > >>> Reuven - sorry to hijack the thread - regarding REPL - > what do you mean > > >>> by it being very Dataflow specific? > > >>> > > >>> On Thu, Jun 21, 2018 at 12:04 PM, Jean-Baptiste Onofré > <[email protected]> > > >>> wrote: > > >>> > > >>>> As the code is not hosted at Apache as Beam, I would > not consider SCIO > > >>>> as the "official" Scala DSL. > > >>>> > > >>>> However, I agree that it's "de facto" Scala DSL for > Beam ;) > > >>>> > > >>>> Just "wording" ;) > > >>>> > > >>>> Regards > > >>>> JB > > >>>> > > >>>> On 21/06/2018 18:00, Robert Bradshaw wrote: > > >>>> > I might go so far as to say Scio *is* the official > Scala API for Beam. > > >>>> > We point to it on our website, and have no plans to > create another. It > > >>>> > just happens to not be maintained and released by us. > > >>>> > On Thu, Jun 21, 2018 at 7:37 AM Jean-Baptiste > Onofré <[email protected]> > > >>>> wrote: > > >>>> >> > > >>>> >> Hi Alistair, > > >>>> >> > > >>>> >> we discussed several times in the past with SCIO > guys (especially > > >>>> >> Neville), but it seems there's no strong plan > right now about a > > >>>> donation > > >>>> >> of SCIO in Beam. > > >>>> >> I think one of the concern is the release cycle, > but I think it makes > > >>>> >> sense to think about a release per module in Beam. > It would allow > > >>>> use to > > >>>> >> release DSLs, IOs/extensions independently. But > that's another story > > >>>> ;) > > >>>> >> > > >>>> >> Regards > > >>>> >> JB > > >>>> >> > > >>>> >> On 21/06/2018 16:34, [email protected] > wrote: > > >>>> >>> Hi, > > >>>> >>> > > >>>> >>> Is there any plan to make scio an official scala > API for beam? If > > >>>> not, is there any plan to have a scala API? > > >>>> >>> > > >>>> >>> Thanks, > > >>>> >>> Alistair > > >>>> >>> > > >>>> >> > > >>>> >> -- > > >>>> >> Jean-Baptiste Onofré > > >>>> >> [email protected] > > >>>> >> http://blog.nanthrax.net > > >>>> >> Talend - http://www.talend.com > > >>>> > > >>>> -- > > >>>> Jean-Baptiste Onofré > > >>>> [email protected] > > >>>> http://blog.nanthrax.net > > >>>> Talend - http://www.talend.com > > >>>> > > >>> > > >>> > > > > > Thanks for the responses everyone. I'm essentially looking > for some reassurance scio is still going to be supported in > the long term. We'd like to use it for a big project. > > It states in the scio repo readme that from v0.3.0 it > depends on beam and not on dataflow. > > Thanks, > Alistair >
