On Wed, Oct 10, 2018 at 10:35 AM Romain Manni-Bucau <[email protected]> wrote:
> Also we can get a more adapted build tool by area and not break the repo > for each build. Go and python build always need a git clean for java users > which is a big issue so let's build each subproject - that is what beam is > today - as they should with an adapted tool. > If this is the case, that should be fixed. I can't remember the last time I did a git clean, so clearly things are not working as well for you as for I. > It requires very few validations byt it is trivial to add unit tests to > ensure it is not broken on these contact points. > > Le mer. 10 oct. 2018 11:29, Jean-Baptiste Onofré <[email protected]> a > écrit : > >> The purpose is that we have a monolithic core today mostly providing >> abstract classes. >> >> The idea is to have something more API oriented with interface/SPI. >> >> Our users would then be able to pick the part of the core they want, >> resulting with lighter artifacts, and for us, it gives a more flexible >> approach. >> >> Regards >> JB >> >> On 10/10/2018 10:26, Robert Bradshaw wrote: >> > My question was not whether we should split the repo, but why? (Dividing >> > things into more (or fewer) modules withing a single repo is a separate >> > question.) Maybe I'm just not following what you mean by "more API >> > oriented." It would force stabler APIs. >> > >> > On Wed, Oct 10, 2018 at 10:18 AM Jean-Baptiste Onofré <[email protected] >> > <mailto:[email protected]>> wrote: >> > >> > Hi, >> > >> > +1, even I think we could split the core even deeper. >> > >> > I discussed with Luke and Reuven to introduce core-sql, core-schema, >> > core-sdf, ... >> > >> > It's not a huge effort, and would allow us to move forward on Beam >> "more >> > API oriented" approach. >> > >> > Regards >> > JB >> > >> > On 10/10/2018 10:12, Robert Bradshaw wrote: >> > > Hi everyone, >> > > >> > > While IMHO it's too early to even be able to split the repo, it's >> > not to >> > > early to talk about it, and I wanted to spin this off to keep the >> > other >> > > thread focused. >> > > >> > > In particular, I am trying to figure out exactly what is hoped to >> be >> > > gained by splitting things up. In my experience, a single project >> that >> > > spans multiple repos has always come with excessive overhead and >> pain. >> > > Of note, we recently merged the website and dataflow-worker into >> the >> > > main repo *exactly* to avoid this pain (though the latter was >> > > particularly bad due to one of the repos being private). >> > > >> > > If need be, I don't see any reason we can't have a single repo >> with >> > > directories >> > > >> > > model/ >> > > website/ >> > > java/ >> > > go/ >> > > ... >> > > >> > > possibly even with their own build system (unified only through a >> > > top-level "build everything" script that descends into each >> subdir and >> > > runs the appropriate command). I'm not saying we should do this >> (there >> > > is value in having a single consistent build system, etc.) but >> it's >> > > possible. We could probably even make separate releases out of >> this >> > > single repo (if we wanted, though given that our releases are >> > time-based >> > > rather than feature-based, I don't see much advantage here). >> > > >> > > Also, there was the comment. >> > > >> > > On Wed, Oct 10, 2018 at 7:35 AM Romain Manni-Bucau >> > > <[email protected] <mailto:[email protected]> >> > <mailto:[email protected] <mailto:[email protected]>>> >> wrote: >> > >> >> > >> Side note: beam portability would be saner if added on top of >> others >> > > than the opposite which is done today. >> > > >> > > I think you brought this up before, Romain. I'm still trying to >> > wrap my >> > > head around what you mean here. Could you elaborate what such a >> > > structure would look like? >> > >> > -- >> > Jean-Baptiste Onofré >> > [email protected] <mailto:[email protected]> >> > http://blog.nanthrax.net >> > Talend - http://www.talend.com >> > >> >> -- >> Jean-Baptiste Onofré >> [email protected] >> http://blog.nanthrax.net >> Talend - http://www.talend.com >> >
