It seems there is support in the community - what is the next step and who is on it? How can I help?
On Mon, Jun 25, 2018 at 9:25 AM Jean-Baptiste Onofré <j...@nanthrax.net> wrote: > Hi, > > I already proposed release per module while ago. It doesn't need any > change on the existing module, just a different versioning. > > It's a little bit more work for the release manager to do a full > release, but it gives more flexibility. It allows us also do use several > branches for the same IO for instance. > > I already doing Apache ServiceMix Bundles and Specs release per release. > So I can prepare some details about that. > > Regards > JB > > On 25/06/2018 17:48, Ismaël Mejía wrote: > > Agree with Kenn, granularity would be key for per-module Beam releases. > > > > We can easily think today of releasing separately IOs and extensions > > because they depend only on the public API of the SDK which has not > > changed much since 2.0.0. So scio will probably fit this category. > > > > A different story would be for runners at least at this moment with all > > the ongoing work on portability, there could be important benefits on > > having fixes and refactors applied globally when multiple internal > > changes are happening. > > > > In the end separating modules for releases has its pros/cons. In the pro > > side it encourages the project to be more stable and robust, but adds > > the weight of guaranteeing that everything works together given the > > multiple combination of versions released which has of course a price > > (this can be a burden for some users if we consider that users should > > also deal with the other dependencies of their own). Today Beam users > > get this for free with the monolithic release approach. > > > > As usual with software trade-offs trade-offs. > > > > > > > > > > > > On Fri, Jun 22, 2018 at 7:53 PM Kenneth Knowles <k...@google.com > > <mailto:k...@google.com>> wrote: > > > > I'm generally in favor of greater decoupling / decentralization > > where possible. It is easy to imagine a world in which Beam consists > > of a few fairly autonomous projects, some centralized policy, opt-in > > infrastructure, just like ~all non-tiny software > > projects/foundations/companies. > > > > The granularity matters quite a lot. But in my experience if you > > choose subprojects to match clear self-sustaining subcommunities > > then each one moves faster and has a stronger community than when > > you try to erase those distinctions in a giant monoproject. > > > > Starting with an active project that already exists and has users > > and maintainers works perfectly for getting the granularity right. > > I'm for trying it if/when the opportunity arises. There are > > technical challenges around shared infrastructure and global quality > > assurance. It would be an opportunity to make them concrete and > > address them. > > > > Kenn > > > > > > > > On Fri, Jun 22, 2018 at 10:24 AM Rafael Fernandez > > <rfern...@google.com <mailto:rfern...@google.com>> wrote: > > > > I like the idea of per-module releases for Beam. I know Henning > > and others have thought about that space as well. > > > > BTW, I'm a big fan of scio, and happy to help in any way > > possible if they are interested in turning "de facto" into "de > > jure" :D > > > > In such a world, Scio could be a very good first use case to > > drive the mechanisms to enable per-module releases. I think it > > allows us to scale better and sets a healthy path for special > > interest groups to naturally emerge and collaborate with their > > own scope in mind. > > > > > > On Thu, Jun 21, 2018 at 11:48 PM alistair.m...@googlemail.com > > <mailto:alistair.m...@googlemail.com> > > <alistair.m...@googlemail.com > > <mailto:alistair.m...@googlemail.com>> wrote: > > > > > > > > On 2018/06/21 17:17:36, Reuven Lax <re...@google.com> wrote: > > > In that case things have changed since I talked to Neville > > about it last > > > November. > > > > > > On Thu, Jun 21, 2018 at 10:16 AM Rafal Wojdyla > > <r...@spotify.com> wrote: > > > > > > > Nope - it uses standard runners and is fully Beam > compliant. > > > > > > > > On Thu, Jun 21, 2018 at 1:12 PM, Reuven Lax > > <re...@google.com> wrote: > > > > > > > >> My understanding was that under the covers it used the > > low-level Dataflow > > > >> service API to run the evaluations. > > > >> > > > >> On Thu, Jun 21, 2018 at 10:10 AM Rafal Wojdyla > > <r...@spotify.com> wrote: > > > >> > > > >>> Hi. > > > >>> Reuven - sorry to hijack the thread - regarding REPL - > > what do you mean > > > >>> by it being very Dataflow specific? > > > >>> > > > >>> On Thu, Jun 21, 2018 at 12:04 PM, Jean-Baptiste Onofré > > <j...@nanthrax.net> > > > >>> wrote: > > > >>> > > > >>>> As the code is not hosted at Apache as Beam, I would > > not consider SCIO > > > >>>> as the "official" Scala DSL. > > > >>>> > > > >>>> However, I agree that it's "de facto" Scala DSL for > > Beam ;) > > > >>>> > > > >>>> Just "wording" ;) > > > >>>> > > > >>>> Regards > > > >>>> JB > > > >>>> > > > >>>> On 21/06/2018 18:00, Robert Bradshaw wrote: > > > >>>> > I might go so far as to say Scio *is* the official > > Scala API for Beam. > > > >>>> > We point to it on our website, and have no plans to > > create another. It > > > >>>> > just happens to not be maintained and released by > us. > > > >>>> > On Thu, Jun 21, 2018 at 7:37 AM Jean-Baptiste > > Onofré <j...@nanthrax.net> > > > >>>> wrote: > > > >>>> >> > > > >>>> >> Hi Alistair, > > > >>>> >> > > > >>>> >> we discussed several times in the past with SCIO > > guys (especially > > > >>>> >> Neville), but it seems there's no strong plan > > right now about a > > > >>>> donation > > > >>>> >> of SCIO in Beam. > > > >>>> >> I think one of the concern is the release cycle, > > but I think it makes > > > >>>> >> sense to think about a release per module in Beam. > > It would allow > > > >>>> use to > > > >>>> >> release DSLs, IOs/extensions independently. But > > that's another story > > > >>>> ;) > > > >>>> >> > > > >>>> >> Regards > > > >>>> >> JB > > > >>>> >> > > > >>>> >> On 21/06/2018 16:34, alistair.m...@googlemail.com > > wrote: > > > >>>> >>> Hi, > > > >>>> >>> > > > >>>> >>> Is there any plan to make scio an official scala > > API for beam? If > > > >>>> not, is there any plan to have a scala API? > > > >>>> >>> > > > >>>> >>> Thanks, > > > >>>> >>> Alistair > > > >>>> >>> > > > >>>> >> > > > >>>> >> -- > > > >>>> >> Jean-Baptiste Onofré > > > >>>> >> jbono...@apache.org > > > >>>> >> http://blog.nanthrax.net > > > >>>> >> Talend - http://www.talend.com > > > >>>> > > > >>>> -- > > > >>>> Jean-Baptiste Onofré > > > >>>> jbono...@apache.org > > > >>>> http://blog.nanthrax.net > > > >>>> Talend - http://www.talend.com > > > >>>> > > > >>> > > > >>> > > > > > > > Thanks for the responses everyone. I'm essentially looking > > for some reassurance scio is still going to be supported in > > the long term. We'd like to use it for a big project. > > > > It states in the scio repo readme that from v0.3.0 it > > depends on beam and not on dataflow. > > > > Thanks, > > Alistair > > >
smime.p7s
Description: S/MIME Cryptographic Signature
