Re: Few issues and questions regarding streaming
Thanks for your response Arpit. As per my understanding, only SparkContext is created. We have to create the StreamingContext. The example that I am following is for Zeppelin where StreamingContext is being created. Now my problem is, I am getting a socket error while running my streaming example. And I have to restart the interpreter every time I need to re-run which is quite annoying. Hence I want to know graceful way so that I can run the example multiple times without need for restarting the interpreter. Even calling in this manner is not helping: ssc.stop(stopSparkContext=false, stopGracefully=true) -Thanks Nikhil On Wed, Jul 5, 2017 at 10:56 AM, arpit tak <arpit@gmail.com> wrote: > Hi Nikhil, > First, Spark context and streaming context already defined in zeppelin > when you write code , you don’t need to initialize it again as we generally > do in writing normal spark code. > Second, you just need to start ssc.start() once and if your next > paragraph is linked to first note then no need to call stop() you can do > that at then end of last paragraph/note. > > Third, in case your note are independent of each other then at a time you > can run only one of them . > > Hope this helps. > > > > On Jul 5, 2017, at 10:49 AM, Nikhil Utane <nikhil.subscri...@gmail.com> > wrote: > > Hi, > > I am trying streaming with Zeppelin using the example given here: zepl.com > <https://www.zepl.com/viewer/notebooks/aHR0cHM6Ly9yYXcuZ2l0aHVidXNlcmNvbnRlbnQuY29tL2hhZml6dXItcmFobWFuL2NjcC1jaGFsbGVuZ2UxL21hc3Rlci96ZXBwZWxpbi9ub3RlYm9vay8yQVdLWFhKTVgvbm90ZS5qc29u> > But not able to get it to work. > I have configured Spark interpreter with a master (spark://dk1:7077). > > a) The first time it ran, the paragraph was in running state but it wasn't > displaying any output (sent using nc command). > > Upon restarting it started giving error: > java.lang.IllegalStateException: Only one StreamingContext may be started > in this JVM. Currently running StreamingContext was started > atorg.apache.spark.streaming.StreamingContext.start( > StreamingContext.scala:569) > I realized that I only have ssc.start() and there is no call to stop(). So > in another paragraph I tried calling ssc.stop() before re-running my main > paragraph but it is still not helping. > > My question is, what is the right way to start/stop streaming app so that > we can run it multiple times back to back? > Is my expectation that I should be seeing output in the same window > correct? > > Pls let me know if any more info is needed. > > -Thanks > Nikhil > > >
Re: Zeppelin [0.7.2]: NullPointerException on executing paragraph from a new Notebook
I use only Spark interpreter. I printed SPARK_HOME in shell interpreter and that didn't have any issue. I can have just these 2 lines and it will throw error. %pyspark from pyspark import SparkContext Once I restart Zeppelin daemon from then on no issues. -Thanks Nikhil On Wed, Jul 5, 2017 at 9:46 AM, arpit tak <arpit@gmail.com> wrote: > Hi Nikhil , > Is it happening only one one interpreter or on all interpreter. > Also share command you writing. > > On Jul 5, 2017, at 9:38 AM, Nikhil Utane <nikhil.subscri...@gmail.com> > wrote: > > > > Hi, > > > > Whenever I create a new notebook and run a paragraph, I get the > NullPointerException error. If I run from an existing notebook then no > problem. > > > > java.lang.NullPointerException > > at org.apache.zeppelin.spark.Utils.invokeMethod(Utils.java:38) > > at org.apache.zeppelin.spark.Utils.invokeMethod(Utils.java:33) > > at org.apache.zeppelin.spark.SparkInterpreter.createSparkContext_2( > SparkInterpreter.java:391) > > at org.apache.zeppelin.spark.SparkInterpreter.createSparkContext( > SparkInterpreter.java:380) > > at org.apache.zeppelin.spark.SparkInterpreter.getSparkContext( > SparkInterpreter.java:146) > > at org.apache.zeppelin.spark.SparkInterpreter.open( > SparkInterpreter.java:828) > > > > I workaround the problem by restarting the Zeppelin daemon. Saw other > instances of this error but resolution given was it is fixed in 0.7.2 > version which is what I am using. Anybody else facing the same problem? > Have I missed something in configuration? > > > > Thanks & Regards > > Nikhil > >
Zeppelin [0.7.2]: NullPointerException on executing paragraph from a new Notebook
Hi, Whenever I create a new notebook and run a paragraph, I get the NullPointerException error. If I run from an existing notebook then no problem. java.lang.NullPointerException at org.apache.zeppelin.spark.Utils.invokeMethod(Utils.java:38) at org.apache.zeppelin.spark.Utils.invokeMethod(Utils.java:33) at org.apache.zeppelin.spark.SparkInterpreter.createSparkContext_2(SparkInterpreter.java:391) at org.apache.zeppelin.spark.SparkInterpreter.createSparkContext(SparkInterpreter.java:380) at org.apache.zeppelin.spark.SparkInterpreter.getSparkContext(SparkInterpreter.java:146) at org.apache.zeppelin.spark.SparkInterpreter.open(SparkInterpreter.java:828) I workaround the problem by restarting the Zeppelin daemon. Saw other instances of this error but resolution given was it is fixed in 0.7.2 version which is what I am using. Anybody else facing the same problem? Have I missed something in configuration? Thanks & Regards Nikhil
Re: Notebook not responding
Doubt that. I am using test data set. After doing clear output I ran all paragraphs and did not see any issue. BTW, when I ran all paragraphs, I expected them to run in sequence. However the later ones ran ahead before the main script finished thereby running into error. Is there any configuration that says run all paragraphs in sequence? -Thanks Nikhil On Thu, Jun 22, 2017 at 11:44 AM, Sachin Janani <sjan...@snappydata.io> wrote: > You might have executed a query which has returned a large result set that > caused this issue. > > Regards, > Sachin Janani > > On Thu, Jun 22, 2017 at 11:42 AM, Nikhil Utane < > nikhil.subscri...@gmail.com> wrote: > >> Phew. I was able to recover by selecting "Clear Output" on the main page. >> After that I was able to open the notebook. >> >> Thanks >> Nikhil >> >> On Thu, Jun 22, 2017 at 11:26 AM, Nikhil Utane < >> nikhil.subscri...@gmail.com> wrote: >> >>> Hi, >>> >>> I am having a serious issue. >>> All of a sudden my notebook has stopped responding. The page doesn't >>> load in any of the browser. I am able to open Tutorial notebooks though. >>> I have restarted zeppelin daemon but it is not helping. >>> In the logs, I only see New Connection and Connection Closed messages. >>> The last changes I made were in a paragraph I added z.show() for two >>> separate tables. Dunno if that has anything to do or not. >>> >>> Please help !! >>> >>> -Regards >>> Nikhil >>> >> >> >
Re: Notebook not responding
Phew. I was able to recover by selecting "Clear Output" on the main page. After that I was able to open the notebook. Thanks Nikhil On Thu, Jun 22, 2017 at 11:26 AM, Nikhil Utane <nikhil.subscri...@gmail.com> wrote: > Hi, > > I am having a serious issue. > All of a sudden my notebook has stopped responding. The page doesn't load > in any of the browser. I am able to open Tutorial notebooks though. > I have restarted zeppelin daemon but it is not helping. > In the logs, I only see New Connection and Connection Closed messages. > The last changes I made were in a paragraph I added z.show() for two > separate tables. Dunno if that has anything to do or not. > > Please help !! > > -Regards > Nikhil >
Notebook not responding
Hi, I am having a serious issue. All of a sudden my notebook has stopped responding. The page doesn't load in any of the browser. I am able to open Tutorial notebooks though. I have restarted zeppelin daemon but it is not helping. In the logs, I only see New Connection and Connection Closed messages. The last changes I made were in a paragraph I added z.show() for two separate tables. Dunno if that has anything to do or not. Please help !! -Regards Nikhil
Re: Printing table with variables in pyspark
Funny thing happened actually.
I had tried same way as above but removed additional content for simplicity.
I made the mistake of naming my variable as str (Duhh). This then overrode
the standard str() function and because of that whatever I tried failed.
After restarting the interpreter it all worked fine.
Thanks for your help.
-Nikhil
On Wed, Jun 21, 2017 at 1:22 PM, Jeff Zhang <zjf...@gmail.com> wrote:
>
> This is not zeppelin issue, it is due to your python code, use following
> code
>
> str="Col1\tCol2"
> print("%table " + str)
>
>
>
> Nikhil Utane <nikhil.subscri...@gmail.com>于2017年6月21日周三 下午12:46写道：
>
>> Hi,
>>
>> The official documentation shows example of printing static values in a
>> table.
>> I have a need to print variable values inside table in pyspark.
>> How to achieve that?
>>
>> %pyspark
>> str="Col1\tCol2"
>> print("%table %s" % str)
>>
>> ValueError: unsupported format character 't' (0x74) at index 1
>>
>> -Thanks & Regards
>> Nikhil
>>
>
