+1 for a separate repo On Wed, Jan 16, 2019 at 2:34 PM Ning Wang <[email protected]> wrote:
> +Siming > > On Tue, Jan 15, 2019 at 11:35 PM Ning Wang <[email protected]> wrote: > > > Hi, all, > > > > A few of us (Spencer, Saikat, Siming, Karthik, Josh, Sree) discussed > today > > in our general slack channel that we should have spouts code somewhere so > > that people can reuse them (spouts are highly reusable in general) and > > contribute improvements. This is just a recap of the idea and some > updates. > > > > We have two options: > > 1. add a spouts/ dir in heron project. > > 2. create a new project in github. > > > > For option 1, it is easy to start. But the iteration and release will be > > coupled with Heron project itself. It is likely there will be quite some > > activities around spouts time by time when new spouts are added. Also, > > Heron itself is basically the engine itself plus APIs and tooling, while > > there could be quite some spouts in future with many new dependencies > like > > Kafka, pubsub, neo4j and neptune, etc. It is debatable to have spout > > implementations in Heron project, and these extra dependencies could add > > some unnecessary complexity. > > > > For option 2, there will be some work up front. but it will be much > easier > > to manage and evolve. And here will be less concerns about new spouts (in > > different languages) and dependencies because spouts are relatively > > independent to each other and we may generate artifacts per spout. > > > > Overall most people prefer option 2 for its cleanness. > > > > I talked with Twitter OSS team. They are happy to support the initiative > > and suggest us to check with Apache team and see what is the best > process. > > First question is that should this new side project be under Apache or > not? > > This might be a question to mentors. What do you think/suggest? > > > > Another topic being discussed is the build tool in case we decide to > > create a new side project. Maven is more mature for sure, but we will > > likely need multi language support so currently Bazel seems to be the > > winner (I personally vote for Bazel 1.0 because the backward > compatibility > > has been bad so far). > > > > Any ideas or suggestions, please feel free to reply. > > > > Regards, > > --ning > > >
