Re: Heron Spouts Code

2019-01-17 Thread Ning Wang
It does make sense for each spout has its own version number. We should define a guide about the versions and a good way to track them. For Heron version, it is a good question. Spouts are not that dependent on Heron version so I feel the lib shouldn't need to have this dependency. However we may

Re: Heron Spouts Code

2019-01-17 Thread Saikat Kanjilal
As an analogy I was looking at Hadoop, yarn and spark as a comparison related to get some ideas and it seems that these components work together pretty seamlessly and have independent versioning. I really feel like it’s up to the main engineers of each spout project on how to version things,

Re: Heron Spouts Code

2019-01-17 Thread Simon Weng
This is a good question. Each version spout must maintain a compatibility matrix {Spout Version, external SDK version, Heron API version}. It’s more of a documentation effort so that user haves enough information to determine which one to pick, isn’t it? On Thu, Jan 17, 2019 at 7:48 AM Josh

Re: Heron Spouts Code

2019-01-17 Thread Simon Weng
As long as it does not have to be a single release train for all of the spout, I’m good with a single separate repo hosting all of spout. We just need a bit upfront effort to setup Bazel for the project. However, I expect once one Spout project folder is setup, it can serve as sample for other

Re: Heron Spouts Code

2019-01-16 Thread Ning Wang
This is an option. I have a few concerns about it: - There will be a lot of repos and it will be messy to manage and it might be harder for users to find it. I am expecting at least more than ten (different services times different languages). - There will be some duplicated code such as

Re: Heron Spouts Code

2019-01-16 Thread Simon Weng
Hi, all: Can it also be one of the options to even have separate repo for each type of spouts? The reasons it is worth considering are: 1. Allow each spout to evolve and release in different pace because each is technically driven by external source software. For example, the community may need

Re: Heron Spouts Code

2019-01-16 Thread Ning Wang
+Siming On Tue, Jan 15, 2019 at 11:35 PM Ning Wang wrote: > Hi, all, > > A few of us (Spencer, Saikat, Siming, Karthik, Josh, Sree) discussed today > in our general slack channel that we should have spouts code somewhere so > that people can reuse them (spouts are highly reusable in general)

Heron Spouts Code

2019-01-15 Thread Ning Wang
Hi, all, A few of us (Spencer, Saikat, Siming, Karthik, Josh, Sree) discussed today in our general slack channel that we should have spouts code somewhere so that people can reuse them (spouts are highly reusable in general) and contribute improvements. This is just a recap of the idea and some