Thanks JB for sharing your experience of dealing with tenanted services! Our cluster is relatively small. For apps like spark app, when # of executors is small, the overhead of one Slider AM for each app should be avoided.
Totally agreed with you on accounting. I don't have a good way to split cost of Slider AM. Guess every tenant needs to chip in. :) Thanks, Paul On Mon, Aug 10, 2015 at 1:12 PM, Jean-Baptiste Note <[email protected]> wrote: > Hi Paul, > > While i don't exactly know the specifics of your service, here we use one > slider instance per user/tenant for a given service. For instance we have > several hbase clusters running on separate accounts. We use chef and/or our > automated deployment tools to automate this kind of per-user > instanciations, not slider itself. > > For us it made a lot of sense given we're running on a secure kerberized > cluster where priviledge separation makes the kind of superuser runs (one > service impersonates several tenants) a pain. > It made also a lot of sense because the cost of the application master > relative to the service is negligible; so there's little to no point in > sharing. > It makes sense for scheduler accounting too because each user is billed > precisely for his resource usage (with your model I guess there's only > accounting on the whole service). > Last but not least, it fits the paas model better for us: each user can > interact at slider level with his own instance securely. > > If your service can support this kind of setup (even at the cost of > reorganizing eg communication through zk) it may be worth your while. > > Kind regards, > JB, from mobile >
