Hive interpreter error - HiveInterpreter.getConnection

2016-02-09 Thread Abhi Basu 
Zeppelin 0.56, running on Yarn on top of CDH 5.5.1, Spark 1.5.0

Getting this error using the hive interpreter, doing  a count(*) or create
of table.

Show tables -- works fine.


java.lang.NullPointerException at
org.apache.zeppelin.hive.HiveInterpreter.getConnection(HiveInterpreter.java:184)
at
org.apache.zeppelin.hive.HiveInterpreter.getStatement(HiveInterpreter.java:204)
at
org.apache.zeppelin.hive.HiveInterpreter.executeSql(HiveInterpreter.java:233)
at
org.apache.zeppelin.hive.HiveInterpreter.interpret(HiveInterpreter.java:328)
at
org.apache.zeppelin.interpreter.ClassloaderInterpreter.interpret(ClassloaderInterpreter.java:57)
at
org.apache.zeppelin.interpreter.LazyOpenInterpreter.interpret(LazyOpenInterpreter.java:93)
at
org.apache.zeppelin.interpreter.remote.RemoteInterpreterServer$InterpretJob.jobRun(RemoteInterpreterServer.java:300)
at org.apache.zeppelin.scheduler.Job.run(Job.java:169) at
org.apache.zeppelin.scheduler.ParallelScheduler$JobRunner.run(ParallelScheduler.java:157)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask.run(FutureTask.java:262) at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)

Thanks,

Abhi

-- 
Abhi Basu

Re: Hive interpreter error - HiveInterpreter.getConnection

2016-02-09 Thread Victor Manuel Garcia 
Hi,  Abhi
This is a bug...you can test in PR #668

Cheers
El 9/2/2016 23:09, "Abhi Basu" <9000r...@gmail.com> escribió:

> Added note: Zeppelin was built from binary and all configs done as stated
> on site.
>
> On Tue, Feb 9, 2016 at 1:33 PM, Abhi Basu <9000r...@gmail.com> wrote:
>
>> Zeppelin 0.56, running on Yarn on top of CDH 5.5.1, Spark 1.5.0
>>
>> Getting this error using the hive interpreter, doing  a count(*) or
>> create of table.
>>
>> Show tables -- works fine.
>>
>>
>> java.lang.NullPointerException at
>> org.apache.zeppelin.hive.HiveInterpreter.getConnection(HiveInterpreter.java:184)
>> at
>> org.apache.zeppelin.hive.HiveInterpreter.getStatement(HiveInterpreter.java:204)
>> at
>> org.apache.zeppelin.hive.HiveInterpreter.executeSql(HiveInterpreter.java:233)
>> at
>> org.apache.zeppelin.hive.HiveInterpreter.interpret(HiveInterpreter.java:328)
>> at
>> org.apache.zeppelin.interpreter.ClassloaderInterpreter.interpret(ClassloaderInterpreter.java:57)
>> at
>> org.apache.zeppelin.interpreter.LazyOpenInterpreter.interpret(LazyOpenInterpreter.java:93)
>> at
>> org.apache.zeppelin.interpreter.remote.RemoteInterpreterServer$InterpretJob.jobRun(RemoteInterpreterServer.java:300)
>> at org.apache.zeppelin.scheduler.Job.run(Job.java:169) at
>> org.apache.zeppelin.scheduler.ParallelScheduler$JobRunner.run(ParallelScheduler.java:157)
>> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
>> at java.util.concurrent.FutureTask.run(FutureTask.java:262) at
>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178)
>> at
>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292)
>> at
>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>> at
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>> at java.lang.Thread.run(Thread.java:745)
>>
>> Thanks,
>>
>> Abhi
>>
>> --
>> Abhi Basu
>>
>
>
>
> --
> Abhi Basu
>

Re: Hive interpreter error - HiveInterpreter.getConnection

2016-02-09 Thread Abhi Basu 
Yes, I had these setup already, still does not work.



On Tue, Feb 9, 2016 at 1:57 PM, Benjamin Kim  wrote:

> Abhi,
>
> Do you have hive-site.xml and yarn-conf symlinked in your spark/conf dir?
>
> ln -s /etc/hive/conf/hive-site.xml /etc/spark/conf/hive-site.xml
> ln -s /etc/spark/conf/yarn-conf /etc/spark/conf/yarn-conf
>
> This made it work for me.
>
> Cheers,
> Ben
>
>
> On Feb 9, 2016, at 1:33 PM, Abhi Basu <9000r...@gmail.com> wrote:
>
> Zeppelin 0.56, running on Yarn on top of CDH 5.5.1, Spark 1.5.0
>
> Getting this error using the hive interpreter, doing  a count(*) or create
> of table.
>
> Show tables -- works fine.
>
>
> java.lang.NullPointerException at
> org.apache.zeppelin.hive.HiveInterpreter.getConnection(HiveInterpreter.java:184)
> at
> org.apache.zeppelin.hive.HiveInterpreter.getStatement(HiveInterpreter.java:204)
> at
> org.apache.zeppelin.hive.HiveInterpreter.executeSql(HiveInterpreter.java:233)
> at
> org.apache.zeppelin.hive.HiveInterpreter.interpret(HiveInterpreter.java:328)
> at
> org.apache.zeppelin.interpreter.ClassloaderInterpreter.interpret(ClassloaderInterpreter.java:57)
> at
> org.apache.zeppelin.interpreter.LazyOpenInterpreter.interpret(LazyOpenInterpreter.java:93)
> at
> org.apache.zeppelin.interpreter.remote.RemoteInterpreterServer$InterpretJob.jobRun(RemoteInterpreterServer.java:300)
> at org.apache.zeppelin.scheduler.Job.run(Job.java:169) at
> org.apache.zeppelin.scheduler.ParallelScheduler$JobRunner.run(ParallelScheduler.java:157)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask.run(FutureTask.java:262) at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
>
> Thanks,
>
> Abhi
>
> --
> Abhi Basu
>
>
>


-- 
Abhi Basu

Re: Hive interpreter error - HiveInterpreter.getConnection

2016-02-09 Thread Abhi Basu 
Added note: Zeppelin was built from binary and all configs done as stated
on site.

On Tue, Feb 9, 2016 at 1:33 PM, Abhi Basu <9000r...@gmail.com> wrote:

> Zeppelin 0.56, running on Yarn on top of CDH 5.5.1, Spark 1.5.0
>
> Getting this error using the hive interpreter, doing  a count(*) or create
> of table.
>
> Show tables -- works fine.
>
>
> java.lang.NullPointerException at
> org.apache.zeppelin.hive.HiveInterpreter.getConnection(HiveInterpreter.java:184)
> at
> org.apache.zeppelin.hive.HiveInterpreter.getStatement(HiveInterpreter.java:204)
> at
> org.apache.zeppelin.hive.HiveInterpreter.executeSql(HiveInterpreter.java:233)
> at
> org.apache.zeppelin.hive.HiveInterpreter.interpret(HiveInterpreter.java:328)
> at
> org.apache.zeppelin.interpreter.ClassloaderInterpreter.interpret(ClassloaderInterpreter.java:57)
> at
> org.apache.zeppelin.interpreter.LazyOpenInterpreter.interpret(LazyOpenInterpreter.java:93)
> at
> org.apache.zeppelin.interpreter.remote.RemoteInterpreterServer$InterpretJob.jobRun(RemoteInterpreterServer.java:300)
> at org.apache.zeppelin.scheduler.Job.run(Job.java:169) at
> org.apache.zeppelin.scheduler.ParallelScheduler$JobRunner.run(ParallelScheduler.java:157)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask.run(FutureTask.java:262) at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
>
> Thanks,
>
> Abhi
>
> --
> Abhi Basu
>



-- 
Abhi Basu

Re: Hive interpreter error - HiveInterpreter.getConnection

2016-02-09 Thread Abhi Basu 
Ok, so this bug is in the current 0.56 binary? Is the binary going to be
regenerated or do I need to build from source?

Thanks,

Abhi

On Tue, Feb 9, 2016 at 2:31 PM, Victor Manuel Garcia <
victor.gar...@beeva.com> wrote:

> Hi,  Abhi
> This is a bug...you can test in PR #668
>
> Cheers
> El 9/2/2016 23:09, "Abhi Basu" <9000r...@gmail.com> escribió:
>
>> Added note: Zeppelin was built from binary and all configs done as stated
>> on site.
>>
>> On Tue, Feb 9, 2016 at 1:33 PM, Abhi Basu <9000r...@gmail.com> wrote:
>>
>>> Zeppelin 0.56, running on Yarn on top of CDH 5.5.1, Spark 1.5.0
>>>
>>> Getting this error using the hive interpreter, doing  a count(*) or
>>> create of table.
>>>
>>> Show tables -- works fine.
>>>
>>>
>>> java.lang.NullPointerException at
>>> org.apache.zeppelin.hive.HiveInterpreter.getConnection(HiveInterpreter.java:184)
>>> at
>>> org.apache.zeppelin.hive.HiveInterpreter.getStatement(HiveInterpreter.java:204)
>>> at
>>> org.apache.zeppelin.hive.HiveInterpreter.executeSql(HiveInterpreter.java:233)
>>> at
>>> org.apache.zeppelin.hive.HiveInterpreter.interpret(HiveInterpreter.java:328)
>>> at
>>> org.apache.zeppelin.interpreter.ClassloaderInterpreter.interpret(ClassloaderInterpreter.java:57)
>>> at
>>> org.apache.zeppelin.interpreter.LazyOpenInterpreter.interpret(LazyOpenInterpreter.java:93)
>>> at
>>> org.apache.zeppelin.interpreter.remote.RemoteInterpreterServer$InterpretJob.jobRun(RemoteInterpreterServer.java:300)
>>> at org.apache.zeppelin.scheduler.Job.run(Job.java:169) at
>>> org.apache.zeppelin.scheduler.ParallelScheduler$JobRunner.run(ParallelScheduler.java:157)
>>> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
>>> at java.util.concurrent.FutureTask.run(FutureTask.java:262) at
>>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178)
>>> at
>>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292)
>>> at
>>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>>> at
>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>>> at java.lang.Thread.run(Thread.java:745)
>>>
>>> Thanks,
>>>
>>> Abhi
>>>
>>> --
>>> Abhi Basu
>>>
>>
>>
>>
>> --
>> Abhi Basu
>>
>


-- 
Abhi Basu

Re: HBase Interpreter

2016-02-09 Thread Benjamin Kim 
It looks like it’s not reaching the zookeeper quorum.

16/02/09 21:52:19 ERROR client.ConnectionManager$HConnectionImplementation: 
Can't get connection to ZooKeeper: KeeperErrorCode = ConnectionLoss for /hbase

And the setting is:

quorum=localhost:2181

The HBase quorum is actually namenode001, namenode002, hbase-master001. Where 
do I set this?

Thanks,
Ben


> On Feb 4, 2016, at 9:15 PM, Felix Cheung  wrote:
> 
> We could probably look into HBase/Pom.xml handling the vendor-repo profile 
> too.
> 
> 
> 
> 
> 
> On Thu, Feb 4, 2016 at 8:08 PM -0800, "Rajat Venkatesh" 
> > wrote:
> 
> Benjamin,
> Can you try compiling Zeppelin by changing the dependencies in hbase/pom.xml 
> to use cloudera jars ? 
> In the long run, one option is to
> 1. run & capture o/p of 'bin/hbase classpath'
> 2. create a classloader
> 3. load all the classes from 1
> 
> Then it will work with any version of HBase theoritically.
>  
> 
> On Fri, Feb 5, 2016 at 8:14 AM Benjamin Kim  > wrote:
> Felix,
> 
> I know that Cloudera practice. We hate that they do that without informing 
> anyone.
> 
> Thanks,
> Ben
> 
> 
> 
>> On Feb 4, 2016, at 9:18 AM, Felix Cheung > > wrote:
>> 
>> CDH is known to cherry pick patches from later releases. Maybe it is because 
>> of that.
>> 
>> Rajat do you have any lead on the release compatibility issue?
>> 
>> 
>> _
>> From: Rajat Venkatesh >
>> Sent: Wednesday, February 3, 2016 10:05 PM
>> Subject: Re: HBase Interpreter
>> To: > >
>> 
>> 
>> Oh. That should work. I've tested with 1.0.0. Hmm
>> 
>> On Thu, Feb 4, 2016 at 10:50 AM Benjamin Kim < bbuil...@gmail.com 
>> > wrote: 
>> Hi Rajat,
>> 
>> The version of HBase that comes with CDH 5.4.8 is 1.0.0. How do I check if 
>> they are compatible?
>> 
>> Thanks,
>> Ben
>> 
>> 
>> On Feb 3, 2016, at 9:16 PM, Rajat Venkatesh < rvenkat...@qubole.com 
>> > wrote:
>> 
>> Can you check the version of HBase ? HBase interpreter has been tested with 
>> HBase 1.0.x and Hadoop 2.6.0. There is a good chance this error is due to 
>> mismatch in versions. 
>> 
>> On Thu, Feb 4, 2016 at 10:20 AM Benjamin Kim < bbuil...@gmail.com 
>> > wrote: 
>> I got this error below trying out the new HBase Interpreter after pulling 
>> and compiling the latest. 
>> 
>> org.jruby.exceptions.RaiseException: (NameError) cannot load Java class 
>> org.apache.hadoop.hbase.quotas.ThrottleType 
>> at 
>> org.jruby.javasupport.JavaUtilities.get_proxy_or_package_under_package(org/jruby/javasupport/JavaUtilities.java:54)
>>  
>> at (Anonymous).method_missing(/builtin/javasupport/java.rb:51) 
>> at 
>> (Anonymous).(root)(/opt/cloudera/parcels/CDH/lib/hbase/lib/ruby/hbase/quotas.rb:23)
>>  
>> at org.jruby.RubyKernel.require(org/jruby/RubyKernel.java:1062) 
>> at 
>> (Anonymous).(root)(/opt/cloudera/parcels/CDH/lib/hbase/lib/ruby/hbase/quotas.rb:24)
>>  
>> at org.jruby.RubyKernel.require(org/jruby/RubyKernel.java:1062) 
>> at 
>> (Anonymous).(root)(/opt/cloudera/parcels/CDH/lib/hbase/lib/ruby/hbase/hbase.rb:90)
>>  
>> at org.jruby.RubyKernel.require(org/jruby/RubyKernel.java:1062) 
>> at 
>> (Anonymous).(root)(/opt/cloudera/parcels/CDH/lib/hbase/lib/ruby/hbase.rb:118)
>>  
>> 
>> Is there something I’m missing. Is it because I’m using CDH 5.4.8? 
>> 
>> Thanks, 
>> Ben
>> 
>> 
>> 
>

Re: Plotting error

2016-02-09 Thread Abhi Basu 
Awesome, that works.

Thanks,

Abhi

On Tue, Feb 9, 2016 at 4:04 PM, moon soo Lee  wrote:

> Hi Abhi,
>
> You'll need to define show(). Such as
>
> %pyspark
> import StringIO
> def show(p):
>   img = StringIO.StringIO()
>   p.savefig(img, format='svg')
>   img.seek(0)
>   print "%html " + img.buf
>
>
> Hope this helps.
>
> Best,
> moon
>
> On Wed, Feb 10, 2016 at 4:34 AM Abhi Basu <9000r...@gmail.com> wrote:
>
>> Using Zeppelin 0.5.6
>>
>> %pyspark
>> import matplotlib.pyplot as plt
>>
>> # Test data
>> x = [1, 2, 3, 4, 5, 6, 7, 8]
>> y = [20, 21, 20.5, 20.81, 21.0, 21.48, 22.0, 21.89]
>>
>> # Plot
>> plt.plot(x, y, linestyle='dashed', marker='o', color='red')
>> show(plt)
>>
>>
>>
>> Traceback (most recent call last): File "/tmp/zeppelin_pyspark.py", line
>> 225, in  eval(compiledCode) File "", line 5, in 
>> NameError: name 'show' is not defined
>>
>>
>> The code executes fine, but the show command gives the error. How to fix
>> this?
>>
>> Thanks,
>>
>> Abhi
>>
>> --
>> Abhi Basu
>>
>


-- 
Abhi Basu

Re: Plotting error

2016-02-09 Thread moon soo Lee 
Hi Abhi,

You'll need to define show(). Such as

%pyspark
import StringIO
def show(p):
  img = StringIO.StringIO()
  p.savefig(img, format='svg')
  img.seek(0)
  print "%html " + img.buf


Hope this helps.

Best,
moon

On Wed, Feb 10, 2016 at 4:34 AM Abhi Basu <9000r...@gmail.com> wrote:

> Using Zeppelin 0.5.6
>
> %pyspark
> import matplotlib.pyplot as plt
>
> # Test data
> x = [1, 2, 3, 4, 5, 6, 7, 8]
> y = [20, 21, 20.5, 20.81, 21.0, 21.48, 22.0, 21.89]
>
> # Plot
> plt.plot(x, y, linestyle='dashed', marker='o', color='red')
> show(plt)
>
>
>
> Traceback (most recent call last): File "/tmp/zeppelin_pyspark.py", line
> 225, in  eval(compiledCode) File "", line 5, in 
> NameError: name 'show' is not defined
>
>
> The code executes fine, but the show command gives the error. How to fix
> this?
>
> Thanks,
>
> Abhi
>
> --
> Abhi Basu
>