Re: Heron Spouts Code

2019-01-17 Thread Ning Wang
It does make sense for each spout has its own version number. We should define a guide about the versions and a good way to track them. For Heron version, it is a good question. Spouts are not that dependent on Heron version so I feel the lib shouldn't need to have this dependency. However we may

Re: Heron Spouts Code

2019-01-17 Thread Saikat Kanjilal
As an analogy I was looking at Hadoop, yarn and spark as a comparison related to get some ideas and it seems that these components work together pretty seamlessly and have independent versioning. I really feel like it’s up to the main engineers of each spout project on how to version things,

Re: Heron Spouts Code

2019-01-17 Thread Simon Weng
This is a good question. Each version spout must maintain a compatibility matrix {Spout Version, external SDK version, Heron API version}. It’s more of a documentation effort so that user haves enough information to determine which one to pick, isn’t it? On Thu, Jan 17, 2019 at 7:48 AM Josh

Re: Heron Spouts Code

2019-01-17 Thread Simon Weng
As long as it does not have to be a single release train for all of the spout, I’m good with a single separate repo hosting all of spout. We just need a bit upfront effort to setup Bazel for the project. However, I expect once one Spout project folder is setup, it can serve as sample for other

Re: Heron Spouts Code

2019-01-16 Thread Ning Wang
This is an option. I have a few concerns about it: - There will be a lot of repos and it will be messy to manage and it might be harder for users to find it. I am expecting at least more than ten (different services times different languages). - There will be some duplicated code such as

Re: Heron Spouts Code

2019-01-16 Thread Simon Weng
Hi, all: Can it also be one of the options to even have separate repo for each type of spouts? The reasons it is worth considering are: 1. Allow each spout to evolve and release in different pace because each is technically driven by external source software. For example, the community may need

Re: Heron Spouts Code

2019-01-16 Thread Ning Wang
+Siming On Tue, Jan 15, 2019 at 11:35 PM Ning Wang wrote: > Hi, all, > > A few of us (Spencer, Saikat, Siming, Karthik, Josh, Sree) discussed today > in our general slack channel that we should have spouts code somewhere so > that people can reuse them (spouts are highly reusable in general)