Hi Moon, Thanks again for the answer. Your idea looks great! The purpose of binding notebook with one specific instance is just to save the loading time, right? I’ll do a little bit try with this solution, then probably will bring more questions: such as how to map one ZeppelinServer to different clusters, and so on☺
And Happy new year! Regards, Dafeng From: moon soo Lee [mailto:m...@apache.org] Sent: Thursday, December 31, 2015 9:57 AM To: users@zeppelin.incubator.apache.org Subject: Re: How to support multiple notebooks in Zeppelin Hi Dafeng, Right, all informations are cached in memory once loaded. To serve large amount of notebooks, we'll need to modify Zeppelin a bit to not keep them in the memory. For #2, Multiple ZeppelinServer can be configured to use shared notebook storage. And loadbalancer in front of them can distribute the REST API and websocket traffic. If loadbalancer redirect the traffic based on notebook id in the path (e.g. '2BA6X4HHM' from path 'http://localhost:8080/#/notebook/2BA6X4HHM<http://localhost:9000/#/notebook/2BA6X4HHM>'), and let the same notebook id stick to the same Zeppelin instance, ZeppelinServer will able to scale out for many users. Will this fits for your use case? Let me know if you have different idea. Thanks, moon On Wed, Dec 30, 2015 at 5:37 PM Dafeng Wang <dafe...@microsoft.com<mailto:dafe...@microsoft.com>> wrote: Hi Moon, Thanks for your quick response. 1. For first question, got your answer. that means all meta info, paragraph, and result of one notebook will be cached in memory once loaded, right? 2. For #2, my real question will be how to serve millions of users for Zeppelin. Per my current understanding REST API and websocket connection handling will be coupled together within one server, if it’s true, then my original question will become to “how to scale out the server that servers REST API, Websocket connection”, or not, what’s the differences of scaling them when I wanna support more users? Regards, Dafeng From: moon soo Lee [mailto:m...@apache.org<mailto:m...@apache.org>] Sent: Thursday, December 31, 2015 12:11 AM To: users@zeppelin.incubator.apache.org<mailto:users@zeppelin.incubator.apache.org> Subject: Re: How to support multiple notebooks in Zeppelin Hi Dafeng, Zeppelin at the moment keeps every notebook in memory once it's been loaded. So number of notebooks supported by instance will be limited by the memory on the system. Scale out Zeppelin-Server you mean scale out the server that serves REST API, Websocket connection? Thanks, moon On Wed, Dec 30, 2015 at 1:57 AM Dafeng Wang <dafe...@microsoft.com<mailto:dafe...@microsoft.com>> wrote: Hi All, I had tried with Zeppelin today, it works perfectly in stand-alone mode, my questions now are: 1. Capacity limitation of Zeppelin-Server, in other words, how many notebook instance one server can support 2. If we want to scale out Zepplein-server, is that possible? If so, then how? Let’s say the environment it’s yarn + spark Regards, Dafeng
