Hi Frances, Thank you for the documents. The structure of the repository looks good. I wonder if "core" could even be divided further, e.g. in API and runtime related modules. For the CI, we could checkout Apache's Jenkins or Travis CI. As for the /develop branch, I would suggest to make it mandatory to have it in a usable state at all times.
Considering the rework of the Dataflow SDK, it would be great if you could give some status updates while you're doing that. This could help us to prepare the runners to any major breaking changes (particularly thinking about the Flink runner because I've been working on it). Best, Max On Tue, Feb 9, 2016 at 5:01 PM, Jean-Baptiste Onofré <[email protected]> wrote: > Hi Frances, > > and thanks for the update. > > The repository structure looks good to me. > > Maybe we can add a section about the PR workflow (PR/review/push). WDYT ? > > For the Jira, no problem. I will add some tasks as well related to the > roadmap (especially the DSLs, new IO, and DataIntegration part). > > Thanks ! > Regards > JB > > > On 02/09/2016 04:46 PM, Frances Perry wrote: >> >> Hi Beamers! >> >> Here’s the Apache Beam: Technical Vision >> >> <https://docs.google.com/document/d/1UyAeugHxZmVlQ5cEWo_eOPgXNQA1oD-rGooWOSwAqh8/edit?pref=2&pli=1#heading=h.e5s64nliyukh> >> document I shared last week with a number of you. (Now we have a dev@ list >> to share it more widely -- yay!) >> >> I just wanted to give you a little visibility into some of the work we’ve >> been doing within Google over the last week: >> >> * Refactoring the DataflowJavaSDK: We’re hard at work separating out the >> user-facing portions of the DataflowJavaSDK from the Google-specific >> worker >> harness. This will ensure that all runners (Cloud Dataflow, Spark, Flink) >> are on equal footing with clear APIs to implement. Due to the >> complications >> that come with doing that while supporting our current users, we won’t be >> able to push those changes to GitHub for a couple of weeks or so. >> >> * Repository structure: As we get ready to start moving different chunks >> of >> code into the new repo, we need to figure out the right way to structure >> it. Here’s a proposal >> >> <https://docs.google.com/document/d/1mTeZED33Famq25XedbKeDlGIJRvtzCXjSfwH9NKQYUE/edit?usp=sharing> >> -- please provide feedback! >> >> * Issue tracking: Thanks to JB for getting the Beam JIRA >> <https://issues.apache.org/jira/browse/BEAM/> set up. We were thinking >> that >> it makes sense to put in components that match the repository structure >> (see above). And then we’ll go ahead and start transitioning our internal >> Google bug tracking into JIRA. >> >> Frances >> > > -- > Jean-Baptiste Onofré > [email protected] > http://blog.nanthrax.net > Talend - http://www.talend.com
