1st version of the api server will support the following commands - submit - kill - update - activate - deactivate
We are designing API server to be stateless and it will run as a job in the scheduler (similar to tracker and UI). With this approach, there is no need to worry about availability issues. cheers /karthik On Mon, Jul 24, 2017 at 5:43 PM, Fu Maosong <[email protected]> wrote: > I like the idea of *service mode* for heron. > > But we need to be more cautious about merging tracker into API Server, > since it can easily bring scalability and availability issues. > BTW, storm's nimbus serves both topology management requests as well as > metrics requests, which is kind of "merging tracker into API server". We > can learn the pros&cons of such design from it. > > > 2017-07-24 16:57 GMT-07:00 Karthik Ramasamy <[email protected]>: > > > *Rationale*: > > > > Currently, Heron supports a single mode of deployment called library > mode. > > Library mode requires several steps and client side configuration which > > could be intensive. Hence, we want to support another mode called service > > mode for simplified deployment. > > > > *Library Mode:* > > > > With Heron, the current mode of deployment is called library mode. This > > mode does not require any services running for Heron to deploy which is a > > huge advantage. However, it requires several configuration to be in the > > client side. Because of this administering becomes harder - especially > > maintaining the configuration and distributing them when the > configuration > > is changed. While this is possible for a bigger teams with dedicated > > dev-ops team, it might be overhead for medium and smaller teams. > > Furthermore, this mode of deployment does not have an API to > > submit/kill/activate/deactivate programmatically. > > > > *Service Mode:* > > > > In this mode, an api server will be running as a service. This service > will > > be run as yet another job in the scheduler so that it will be restarted > > during machine and process failures thereby providing fault tolerance. > This > > api server will maintain the configuration and heron cli will be > augmented > > to use the rest API to submit/kill/activate/deactivate the topologies in > > this mode. The advantage of this mode is it simplifies deployment but > > requires running a service. > > > > *Merging Tracker into API Server:* > > > > Current, Heron tracker written in python duplicates the state manager > code > > in python as well. The API server will support the heron tracker api in > > addition to topologies api. Depending on the mode of the deployment, the > > api server can be deployed in one of the modes - library mode (which > > exposes only the tracker API) and services mode (which exposes both the > > tracker + api server). Initially, the tracker and api server will be in > > separate directory until great amount of testing is done. Once it is > > completed, we can think about cutting over to entirely using API server. > > > > This change will not affect any of the existing deployments and it will > be > > backward compatible. > > > > > > -- > With my best Regards > ------------------ > Fu Maosong > Twitter Inc. > Mobile: +001-415-244-7520 >
