BTW, when the daemon is stopped on the host, the notebook just hangs if it was running, without any errors. The only way is to tail the last log in $ZEPPELIN_HOME/logs. So I would say a cron type job is required to scan the log for errors.
Dr Mich Talebzadeh LinkedIn * https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw <https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw>* http://talebzadehmich.wordpress.com On 10 March 2016 at 11:32, ayan guha <guha.a...@gmail.com> wrote: > Thanks guys for reply. Yes, Zeppelin with Spark is pretty compelling > choice, for single user. Any pointers for using Zeppelin for multi user > scenario? In essence, can we either (a) Use Zeppelin to connect to a long > running Spark Application which has some pre-cached Dataframes? (b) Can > Zeppelin user be passed down and use Ranger to implement Hive RBAC? > > I know I am sounding a little vague, but such is the problem state in my > mind :) Any help will be appreciated. > > Best > Ayan > > On Thu, Mar 10, 2016 at 9:51 PM, Mich Talebzadeh < > mich.talebza...@gmail.com> wrote: > >> Zeppelin is pretty a good choice for Spark. It has a UI that allows you >> to run your code. It has Interpreter where you change the connection >> configuration. I made mine run on port 21999 (a deamon process on Linux >> host where your spark master is running). It is pretty easy to set up and >> run. >> >> HTH >> >> Dr Mich Talebzadeh >> >> >> >> LinkedIn * >> https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw >> <https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw>* >> >> >> >> http://talebzadehmich.wordpress.com >> >> >> >> On 10 March 2016 at 10:26, Sabarish Sasidharan < >> sabarish.sasidha...@manthan.com> wrote: >> >>> I believe you need to co-locate your Zeppelin on the same node where >>> Spark is installed. You need to specify the SPARK HOME. The master I used >>> was YARN. >>> >>> Zeppelin exposes a notebook interface. A notebook can have many >>> paragraphs. You run the paragraphs. You can mix multiple contexts in the >>> same notebook. So first paragraph can be scala, second can be sql that uses >>> DF from first paragraph etc. If you use a select query, the output is >>> automatically displayed as a chart. >>> >>> As RDDs are bound to the context that creates them, I don't think >>> Zeppelin can use those RDDs. >>> >>> I don't know if notebooks can be reused within other notebooks. It would >>> be a nice way of doing some common preparatory work (like building these >>> RDDs). >>> >>> Regards >>> Sab >>> >>> On Thu, Mar 10, 2016 at 2:28 PM, ayan guha <guha.a...@gmail.com> wrote: >>> >>>> Hi All >>>> >>>> I am writing this in order to get a fair understanding of how zeppelin >>>> can be integrated with Spark. >>>> >>>> Our use case is to load few tables from a DB to Spark, run some >>>> transformation. Once done, we want to expose data through Zeppelin for >>>> analytics. I have few question around that to sound off any gross >>>> architectural flaws. >>>> >>>> Questions: >>>> >>>> 1. How Zeppelin connects to Spark? Thriftserver? Thrift JDBC? >>>> >>>> 2. What is the scope of Spark application when it is used from >>>> Zeppelin? For example, if I have few subsequent actions in zeppelin like >>>> map,filter,reduceByKey, filter,collect. I assume this will translate to an >>>> application and get submitted to Spark. However, If I want to use reuse >>>> some part of the data (for example) after first map transformation in >>>> earlier application. Can I do it? Or will it be another application and >>>> another spark submit? >>>> >>>> In our use case data will already be loaded in RDDs. So how Zeppelin >>>> can access it? >>>> >>>> 3. How can I control access on specific rdds to specific users in >>>> Zeppelin (assuming we have implemented some way of login mechanism in >>>> Zeppelin and we have a mapping between Zeppelin users and their LDAP >>>> accounts). Is it even possible? >>>> >>>> 4. If Zeppelin is not a good choice, yet, for the use case, what are >>>> the other alternatives? >>>> >>>> appreciate any help/pointers/guidance. >>>> >>>> >>>> -- >>>> Best Regards, >>>> Ayan Guha >>>> >>> >>> >>> >>> -- >>> >>> Architect - Big Data >>> Ph: +91 99805 99458 >>> >>> Manthan Systems | *Company of the year - Analytics (2014 Frost and >>> Sullivan India ICT)* >>> +++ >>> >> >> > > > -- > Best Regards, > Ayan Guha >
