Thanks Lei. As of now, we plan to run only one Hbase instance.
On Scale up behaviour, we expect to read the data through Kafka partitions into the application. So would need to make individual container/executor aware about the specific partition it has to read from. We plan to introduce a driver (something like the spark driver) to achieve the same and yet to freeze on whether to run this driver in yarn client or in AM. Also, this driver would need to handle the container failures and running the containers from last known state. Does this answer your question or were you looking for some other detail? Regards, Ashish On 08/07/15 7:19 PM, "Lei Guo" <[email protected]> wrote: >RocketFuel should have multiple HBase instances on Slider in production, >you can find related information from Hadoop Summit or HBaseCon. Do you >plan multiple HBase instances? > >From my view, Slider is more positioned as a wrapper/tool to help >application running on Yarn without code change. For porting, Yarn native >or Twill are the way to go. Take one step back, what's the desired >behavior for your single node application with scaling up feature? This >will help people to give recommendation. > >Lei > >-----Original Message----- >From: Ashish Rawat [mailto:[email protected]] >Sent: Wednesday, July 08, 2015 7:58 AM >To: [email protected] >Subject: Is slider production ready? > >Hi, > >We have been evaluating slider project for multiple purposes: > > 1. Hbase on Yarn > 2. Porting an existing single node application to Yarn for scaling it >up. > >Our main concern is around the stability, maintainability and >extensibility of the project for long term use. Would really help if >someone can help in answering the following questions: > > 1. Have there been any production use of slider, especially with Hbase. > 2. There doesn't seem to be any good documentation around how to write >a new yarn app using Slider. A sample slider app link may help. > 3. When is it recommended to use slider and when should one consider >writing their own Yarn app. > >Regards, >Ashish
