On Mon, Feb 12, 2018 at 2:01 AM Vinod Kancharana <vinod.kanchar...@gmail.com> wrote:
> Is there a recommended cluster server (edge node or master node or cluster > node) for Livy Installation? Right now, I installed Livy on the master node > running Yarn Resource Manager. > > I am not the best person to answer this question, but here is my two cents. Livy is a lightweight process. Even with thousands of sessions running on it, it needs less than 1GB memory, and not much CPU. The only time Livy becomes a bottleneck is, when many sessions/jobs are submitted to it in a short burst (because each new session/job launches a new JVM to submit the job to YARN). > Also, can we enable Load Balancing with two Livy instances running on the > same cluster? > > No. You cannot because the two Livy instances won't coordinate. For example, if a session is created on one Livy instance, it won't be available on the second Livy instance. PS: I have configured Livy to use ZooKeeper for state-store. > > Thanks !! >
