Slow Query Log - 'adjust thresholds' settings

[jumbo]

I am using hbase-0.98.9-hadoop2 in pseudo-dist mode.

What are the exact settings required in "hadoop-metrics2-hbase.properties"
to adjust the thresholds for Slow Query logging?

The following entries are not generating the warnings in the region server
log file:
hbase.sink.file4.class=org.apache.hadoop.metrics2.sink.FileSink
hbase.sink.file4.context=rpc
hbase.sink.file4.filename=rpc.metrics
hbase.sink.file4.ipc.warn.response.time=50
hbase.sink.file4.ipc.warn.response.size=100

Thanks.



--
View this message in context: 
http://apache-hbase.679495.n3.nabble.com/Slow-Query-Log-adjust-thresholds-settings-tp4075682.html
Sent from the HBase User mailing list archive at Nabble.com.

RE: delete of cells with visibility expressions

[Anoop Sharma]

hi

which hbase version did you try this on?
We tried on the following 2 hbase versions and see the delete problem.

   Version 1.0.2, r76745a2cbffe08b812be16e0e19e637a23a923c5, Tue Aug 25
15:59:49 PDT 2015
   Version 1.1.2, rcc2b70cf03e3378800661ec5cab11eb43fafe0fc, Wed Aug 26
20:11:27 PDT 2015

Is there a later version that has the fix?

thanks

-Original Message-
From: ramkrishna vasudevan [mailto:ramkrishna.s.vasude...@gmail.com]
Sent: Sunday, November 1, 2015 11:11 PM
To: user@hbase.apache.org
Subject: Re: delete of cells with visibility expressions

Is it still a bug? I reproduced the above steps in latest trunk and I
thought the behaviour was corrected due to a recent bug fix?  Is it not that
case ?

Regards
Ram

On Mon, Nov 2, 2015 at 12:20 PM, Anoop John  wrote:

> I believe it is a bug.. I think I know the reason also..  Can you file
> a jira?  We can discuss under that.  Thanks for the test.
>
> -Anoop-
>
> On Sat, Oct 31, 2015 at 12:45 AM, Anoop Sharma
> 
> wrote:
>
> > Thanks Ram.
> >
> > we are using hbase 1.0.2.
> >
> > anoop
> >
> > -Original Message-
> > From: ramkrishna vasudevan [mailto:ramkrishna.s.vasude...@gmail.com]
> > Sent: Thursday, October 29, 2015 10:22 PM
> > To: user@hbase.apache.org
> > Subject: Re: delete of cells with visibility expressions
> >
> > Hi Anoop
> >
> > Which version of the HBase are you using?  This got solved in the
> > latest version of 0.98 and above. Could you try that?  I just
> > reproduced this
> and
> > this problem no longer occurs.
> >
> > Regards
> > Ram
> >
> > On Fri, Oct 30, 2015 at 3:26 AM, Anoop Sharma
> > 
> > wrote:
> >
> > > hi
> > >
> > >   running into an issue related to visibility expressions and delete.
> > >
> > > Example run from hbase shell is listed below.
> > >
> > > Will appreciate any help on this issue.
> > >
> > > thanks.
> > >
> > >
> > >
> > > In the example below, user running queries has ‘MANAGER’
> > > authorization.
> > >
> > >
> > >
> > > *First example:*
> > >
> > >   add a column with visib expr ‘MANAGER’
> > >
> > >   delete it by passing in visibility of ‘MANAGER’
> > >
> > >   This works and scan doesn’t return anything.
> > >
> > >
> > >
> > > *Second example:*
> > >
> > >   add a column with visib expr ‘MANAGER’
> > >
> > >   delete it by not passing in any visibility.
> > >
> > >   This doesn’t delete the column.
> > >
> > >   Scan doesn’t return the row but RAW scan shows the column
> > >
> > >   marked as deleteColumn.
> > >
> > >
> > >
> > >   Now if delete is done again with visibility of ‘MANAGER’,
> > >
> > >   it still doesn’t delete it and scan returns the original column.
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > > *Example 1:*
> > >
> > > hbase(main):096:0> create 'HBT1', 'cf'
> > >
> > >
> > >
> > > hbase(main):098:0* *put 'HBT1', 'John', 'cf:a', 'CA',
> > > {VISIBILITY=>'MANAGER'}*
> > >
> > >
> > >
> > > hbase(main):099:0> *scan 'HBT1'*
> > >
> > > ROW
> > > COLUMN+CELL
> > >
> > >  John column=cf:a, timestamp=1446154722055,
> > > value=CA
> > >
> > > 1 row(s) in 0.0030 seconds
> > >
> > >
> > >
> > > hbase(main):100:0> *delete 'HBT1', 'John', 'cf:a',
> > > {VISIBILITY=>'MANAGER'}*
> > >
> > > 0 row(s) in 0.0030 seconds
> > >
> > >
> > >
> > > hbase(main):101:0> *scan 'HBT1'*
> > >
> > > ROW
> > > COLUMN+CELL
> > >
> > > 0 row(s) in 0.0030 seconds
> > >
> > >
> > >
> > >
> > >
> > > *Example 2:*
> > >
> > > hbase(main):010:0* *put 'HBT1', 'John', 'cf:a', 'CA',
> > > {VISIBILITY=>'MANAGER'}*
> > >
> > > 0 row(s) in 0.0040 seconds
> > >
> > >
> > >
> > > hbase(main):011:0> *scan 'HBT1'*
> > >
> > > ROW
> > > COLUMN+CELL
> > >
> > >  John column=cf:a, timestamp=1446155346473,
> > > value=CA
> > >
> > > 1 row(s) in 0.0060 seconds
> > >
> > >
> > >
> > > hbase(main):012:0> *delete 'HBT1', 'John', 'cf:a'*
> > >
> > > 0 row(s) in 0.0090 seconds
> > >
> > >
> > >
> > > hbase(main):013:0> *scan 'HBT1'*
> > >
> > > ROW
> > > COLUMN+CELL
> > >
> > >  John column=cf:a, timestamp=1446155346473,
> > > value=CA
> > >
> > > 1 row(s) in 0.0050 seconds
> > >
> > >
> > >
> > > hbase(main):014:0> *scan 'HBT1', {RAW => true}*
> > >
> > > ROW
> > > COLUMN+CELL
> > >
> > >  John column=cf:a, timestamp=1446155346519,
> > > type=DeleteColumn
> > >
> > > 1 row(s) in 0.0060 seconds
> > >
> > >
> > >
> > > hbase(main):015:0> *delete 'HBT1', 'John', 'cf:a',
> > > {VISIBILITY=>'MANAGER'}*
> > >
> > > 0 row(s) in 0.0030 seconds
> > >
> > >
> > >
> > > hbase(main):016:0> *scan 'HBT1'*
> > >
> > > ROW
> > > COLUMN+CELL
> > >
> > >  John column=cf:a, timestamp=1446155346473,
> > > value=CA
> > >
> > > 1 row(s) in 0.0040 seconds
> > >
> > >
> > >
> > > hbase(main):017:0> *scan 'HBT1', {RAW => true}*
> > >
> > > ROW
> > > COLUMN+CELL
> > >
> > >  John column=cf:a, timestamp=1446155346601,
> > > type=DeleteColumn
> > >
> > > 1 row(s) in 0.0060 seconds
> 

Re: HBase error disabled security features are not available

[kumar r]

Yes I have set all this property,

I got user ticket using kinit command and when trying to run *user_permission
or grant *command in hbase shell, getting this exception

Posted the question is stack overflow in the below link

http://stackoverflow.com/questions/33496541/hbase-error-disabled-security-features-are-not-available



On Wed, Nov 4, 2015 at 10:23 AM, Ted Yu  wrote:

> Have you set the following config ?
>
> hbase.master.keytab.file
> hbase.master.kerberos.principal
> hbase.regionserver.keytab.file
> hbase.regionserver.kerberos.principal
>
> Refer to http://hbase.apache.org/book.html for their meaning / sample
> value.
>
> Please show the stack trace of the exception you got.
>
> Cheers
>
> On Tue, Nov 3, 2015 at 8:43 PM, kumar r  wrote:
>
> > Configured secure HBase-1.1.2 with Hadoop-2.7.1 on Windows. When i enable
> > authorization referring Configuring HBase Authorization
> > <
> >
> http://www.cloudera.com/content/www/en-us/documentation/archive/cdh/4-x/4-3-2/CDH4-Security-Guide/cdh4sg_topic_8_3.html
> > >,
> > getting *ERROR: DISABLED: Security features are not available* exception.
> >
> > I have set the authorization configurations as below,
> >
> > 
> >  hbase.security.authorization
> >  true
> > 
> >
> > 
> >  hbase.coprocessor.master.classes
> >
> >  org.apache.hadoop.hbase.security.access.AccessController
> > 
> >
> > 
> >  hbase.coprocessor.region.classes
> >
> >
> org.apache.hadoop.hbase.security.token.TokenProvider,org.apache.hadoop.hbase.security.access.AccessController
> > 
> >
> > But HBase Authorization works fine when i tried with HBase-0.98.13
> > version. Some one help me to enable HBase Authorization in a correct
> > way.
> >
>

Re: delete of cells with visibility expressions

[ramkrishna vasudevan]

Sorry for the confusion. Yes the bug exists. When I tried in the cluster
the Visibilty CP was not on. So it is better we can raise a JIRA and fix
this over there. Thanks Anoop Sharma and sorry for the delay from my side
due to wrong info.

Regards
Ram

On Wed, Nov 4, 2015 at 4:15 AM, Anoop Sharma  wrote:

> hi
>
> which hbase version did you try this on?
> We tried on the following 2 hbase versions and see the delete problem.
>
>Version 1.0.2, r76745a2cbffe08b812be16e0e19e637a23a923c5, Tue Aug 25
> 15:59:49 PDT 2015
>Version 1.1.2, rcc2b70cf03e3378800661ec5cab11eb43fafe0fc, Wed Aug 26
> 20:11:27 PDT 2015
>
> Is there a later version that has the fix?
>
> thanks
>
> -Original Message-
> From: ramkrishna vasudevan [mailto:ramkrishna.s.vasude...@gmail.com]
> Sent: Sunday, November 1, 2015 11:11 PM
> To: user@hbase.apache.org
> Subject: Re: delete of cells with visibility expressions
>
> Is it still a bug? I reproduced the above steps in latest trunk and I
> thought the behaviour was corrected due to a recent bug fix?  Is it not
> that
> case ?
>
> Regards
> Ram
>
> On Mon, Nov 2, 2015 at 12:20 PM, Anoop John  wrote:
>
> > I believe it is a bug.. I think I know the reason also..  Can you file
> > a jira?  We can discuss under that.  Thanks for the test.
> >
> > -Anoop-
> >
> > On Sat, Oct 31, 2015 at 12:45 AM, Anoop Sharma
> > 
> > wrote:
> >
> > > Thanks Ram.
> > >
> > > we are using hbase 1.0.2.
> > >
> > > anoop
> > >
> > > -Original Message-
> > > From: ramkrishna vasudevan [mailto:ramkrishna.s.vasude...@gmail.com]
> > > Sent: Thursday, October 29, 2015 10:22 PM
> > > To: user@hbase.apache.org
> > > Subject: Re: delete of cells with visibility expressions
> > >
> > > Hi Anoop
> > >
> > > Which version of the HBase are you using?  This got solved in the
> > > latest version of 0.98 and above. Could you try that?  I just
> > > reproduced this
> > and
> > > this problem no longer occurs.
> > >
> > > Regards
> > > Ram
> > >
> > > On Fri, Oct 30, 2015 at 3:26 AM, Anoop Sharma
> > > 
> > > wrote:
> > >
> > > > hi
> > > >
> > > >   running into an issue related to visibility expressions and delete.
> > > >
> > > > Example run from hbase shell is listed below.
> > > >
> > > > Will appreciate any help on this issue.
> > > >
> > > > thanks.
> > > >
> > > >
> > > >
> > > > In the example below, user running queries has ‘MANAGER’
> > > > authorization.
> > > >
> > > >
> > > >
> > > > *First example:*
> > > >
> > > >   add a column with visib expr ‘MANAGER’
> > > >
> > > >   delete it by passing in visibility of ‘MANAGER’
> > > >
> > > >   This works and scan doesn’t return anything.
> > > >
> > > >
> > > >
> > > > *Second example:*
> > > >
> > > >   add a column with visib expr ‘MANAGER’
> > > >
> > > >   delete it by not passing in any visibility.
> > > >
> > > >   This doesn’t delete the column.
> > > >
> > > >   Scan doesn’t return the row but RAW scan shows the column
> > > >
> > > >   marked as deleteColumn.
> > > >
> > > >
> > > >
> > > >   Now if delete is done again with visibility of ‘MANAGER’,
> > > >
> > > >   it still doesn’t delete it and scan returns the original column.
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > > *Example 1:*
> > > >
> > > > hbase(main):096:0> create 'HBT1', 'cf'
> > > >
> > > >
> > > >
> > > > hbase(main):098:0* *put 'HBT1', 'John', 'cf:a', 'CA',
> > > > {VISIBILITY=>'MANAGER'}*
> > > >
> > > >
> > > >
> > > > hbase(main):099:0> *scan 'HBT1'*
> > > >
> > > > ROW
> > > > COLUMN+CELL
> > > >
> > > >  John column=cf:a, timestamp=1446154722055,
> > > > value=CA
> > > >
> > > > 1 row(s) in 0.0030 seconds
> > > >
> > > >
> > > >
> > > > hbase(main):100:0> *delete 'HBT1', 'John', 'cf:a',
> > > > {VISIBILITY=>'MANAGER'}*
> > > >
> > > > 0 row(s) in 0.0030 seconds
> > > >
> > > >
> > > >
> > > > hbase(main):101:0> *scan 'HBT1'*
> > > >
> > > > ROW
> > > > COLUMN+CELL
> > > >
> > > > 0 row(s) in 0.0030 seconds
> > > >
> > > >
> > > >
> > > >
> > > >
> > > > *Example 2:*
> > > >
> > > > hbase(main):010:0* *put 'HBT1', 'John', 'cf:a', 'CA',
> > > > {VISIBILITY=>'MANAGER'}*
> > > >
> > > > 0 row(s) in 0.0040 seconds
> > > >
> > > >
> > > >
> > > > hbase(main):011:0> *scan 'HBT1'*
> > > >
> > > > ROW
> > > > COLUMN+CELL
> > > >
> > > >  John column=cf:a, timestamp=1446155346473,
> > > > value=CA
> > > >
> > > > 1 row(s) in 0.0060 seconds
> > > >
> > > >
> > > >
> > > > hbase(main):012:0> *delete 'HBT1', 'John', 'cf:a'*
> > > >
> > > > 0 row(s) in 0.0090 seconds
> > > >
> > > >
> > > >
> > > > hbase(main):013:0> *scan 'HBT1'*
> > > >
> > > > ROW
> > > > COLUMN+CELL
> > > >
> > > >  John column=cf:a, timestamp=1446155346473,
> > > > value=CA
> > > >
> > > > 1 row(s) in 0.0050 seconds
> > > >
> > > >
> > > >
> > > > hbase(main):014:0> *scan 'HBT1', {RAW => true}*
> > > >
> > > > ROW
> > > > COLUMN+CELL
> > > >
> > > > 

HBase error disabled security features are not available

[kumar r]

Configured secure HBase-1.1.2 with Hadoop-2.7.1 on Windows. When i enable
authorization referring Configuring HBase Authorization
,
getting *ERROR: DISABLED: Security features are not available* exception.

I have set the authorization configurations as below,


 hbase.security.authorization
 true



 hbase.coprocessor.master.classes
 org.apache.hadoop.hbase.security.access.AccessController



 hbase.coprocessor.region.classes
 
org.apache.hadoop.hbase.security.token.TokenProvider,org.apache.hadoop.hbase.security.access.AccessController


But HBase Authorization works fine when i tried with HBase-0.98.13
version. Some one help me to enable HBase Authorization in a correct
way.

Re: HBase error disabled security features are not available

[Ted Yu]

Have you set the following config ?

hbase.master.keytab.file
hbase.master.kerberos.principal
hbase.regionserver.keytab.file
hbase.regionserver.kerberos.principal

Refer to http://hbase.apache.org/book.html for their meaning / sample value.

Please show the stack trace of the exception you got.

Cheers

On Tue, Nov 3, 2015 at 8:43 PM, kumar r  wrote:

> Configured secure HBase-1.1.2 with Hadoop-2.7.1 on Windows. When i enable
> authorization referring Configuring HBase Authorization
> <
> http://www.cloudera.com/content/www/en-us/documentation/archive/cdh/4-x/4-3-2/CDH4-Security-Guide/cdh4sg_topic_8_3.html
> >,
> getting *ERROR: DISABLED: Security features are not available* exception.
>
> I have set the authorization configurations as below,
>
> 
>  hbase.security.authorization
>  true
> 
>
> 
>  hbase.coprocessor.master.classes
>
>  org.apache.hadoop.hbase.security.access.AccessController
> 
>
> 
>  hbase.coprocessor.region.classes
>
>  
> org.apache.hadoop.hbase.security.token.TokenProvider,org.apache.hadoop.hbase.security.access.AccessController
> 
>
> But HBase Authorization works fine when i tried with HBase-0.98.13
> version. Some one help me to enable HBase Authorization in a correct
> way.
>

Error while loading bulk data from pig to hbase

[Naresh Reddy]

Hi

I am getting the below error while loading bulk data from pig to hbase
through HBaseStorage.Please help me to resolve this issue.Thanks in advance.


*ERROR:pig script failed to validate: java.lang.RuntimeException: could not
instantiate 'org.apache.pig.backend.hadoop.hbase.HBaseStorage' with
arguments.*

Below is my pig script.







*register '/home/hduser/hbase-1.1.2/lib/zookeeper-3.4.6.jar';register
'/home/hduser/hbase-1.1.2/lib/guava-12.0.1.jar';register
'/home/hduser/hbase-1.1.2/lib/hbase-common-1.1.2.jar';register
'/home/hduser/hbase-1.1.2/lib/hbase-client-1.1.2.jar';register
'/home/hduser/hbase-1.1.2/lib/protobuf-java-2.5.0.jar';raw_data = LOAD
'/user/hduser/input.csv' USING PigStorage(',') AS (id:chararray,
fname:chararray, lname:chararray);STORE raw_data INTO
'hbase://sample_names' USING
org.apache.pig.backend.hadoop.hbase.HBaseStorage ('info:fname info:lname');*


Regards
Naresh

Re: delete of cells with visibility expressions

[ramkrishna vasudevan]

Seeing the code I think we have identified the issue as Anoop John said. We
could fix this probably in the next release.  Let us know if you want us to
file a JIRA for this.

Regards
Ram


On Wed, Nov 4, 2015 at 11:09 AM, ramkrishna vasudevan <
ramkrishna.s.vasude...@gmail.com> wrote:

> Sorry for the confusion. Yes the bug exists. When I tried in the cluster
> the Visibilty CP was not on. So it is better we can raise a JIRA and fix
> this over there. Thanks Anoop Sharma and sorry for the delay from my side
> due to wrong info.
>
> Regards
> Ram
>
> On Wed, Nov 4, 2015 at 4:15 AM, Anoop Sharma 
> wrote:
>
>> hi
>>
>> which hbase version did you try this on?
>> We tried on the following 2 hbase versions and see the delete problem.
>>
>>Version 1.0.2, r76745a2cbffe08b812be16e0e19e637a23a923c5, Tue Aug 25
>> 15:59:49 PDT 2015
>>Version 1.1.2, rcc2b70cf03e3378800661ec5cab11eb43fafe0fc, Wed Aug 26
>> 20:11:27 PDT 2015
>>
>> Is there a later version that has the fix?
>>
>> thanks
>>
>> -Original Message-
>> From: ramkrishna vasudevan [mailto:ramkrishna.s.vasude...@gmail.com]
>> Sent: Sunday, November 1, 2015 11:11 PM
>> To: user@hbase.apache.org
>> Subject: Re: delete of cells with visibility expressions
>>
>> Is it still a bug? I reproduced the above steps in latest trunk and I
>> thought the behaviour was corrected due to a recent bug fix?  Is it not
>> that
>> case ?
>>
>> Regards
>> Ram
>>
>> On Mon, Nov 2, 2015 at 12:20 PM, Anoop John 
>> wrote:
>>
>> > I believe it is a bug.. I think I know the reason also..  Can you file
>> > a jira?  We can discuss under that.  Thanks for the test.
>> >
>> > -Anoop-
>> >
>> > On Sat, Oct 31, 2015 at 12:45 AM, Anoop Sharma
>> > 
>> > wrote:
>> >
>> > > Thanks Ram.
>> > >
>> > > we are using hbase 1.0.2.
>> > >
>> > > anoop
>> > >
>> > > -Original Message-
>> > > From: ramkrishna vasudevan [mailto:ramkrishna.s.vasude...@gmail.com]
>> > > Sent: Thursday, October 29, 2015 10:22 PM
>> > > To: user@hbase.apache.org
>> > > Subject: Re: delete of cells with visibility expressions
>> > >
>> > > Hi Anoop
>> > >
>> > > Which version of the HBase are you using?  This got solved in the
>> > > latest version of 0.98 and above. Could you try that?  I just
>> > > reproduced this
>> > and
>> > > this problem no longer occurs.
>> > >
>> > > Regards
>> > > Ram
>> > >
>> > > On Fri, Oct 30, 2015 at 3:26 AM, Anoop Sharma
>> > > 
>> > > wrote:
>> > >
>> > > > hi
>> > > >
>> > > >   running into an issue related to visibility expressions and
>> delete.
>> > > >
>> > > > Example run from hbase shell is listed below.
>> > > >
>> > > > Will appreciate any help on this issue.
>> > > >
>> > > > thanks.
>> > > >
>> > > >
>> > > >
>> > > > In the example below, user running queries has ‘MANAGER’
>> > > > authorization.
>> > > >
>> > > >
>> > > >
>> > > > *First example:*
>> > > >
>> > > >   add a column with visib expr ‘MANAGER’
>> > > >
>> > > >   delete it by passing in visibility of ‘MANAGER’
>> > > >
>> > > >   This works and scan doesn’t return anything.
>> > > >
>> > > >
>> > > >
>> > > > *Second example:*
>> > > >
>> > > >   add a column with visib expr ‘MANAGER’
>> > > >
>> > > >   delete it by not passing in any visibility.
>> > > >
>> > > >   This doesn’t delete the column.
>> > > >
>> > > >   Scan doesn’t return the row but RAW scan shows the column
>> > > >
>> > > >   marked as deleteColumn.
>> > > >
>> > > >
>> > > >
>> > > >   Now if delete is done again with visibility of ‘MANAGER’,
>> > > >
>> > > >   it still doesn’t delete it and scan returns the original column.
>> > > >
>> > > >
>> > > >
>> > > >
>> > > >
>> > > >
>> > > >
>> > > > *Example 1:*
>> > > >
>> > > > hbase(main):096:0> create 'HBT1', 'cf'
>> > > >
>> > > >
>> > > >
>> > > > hbase(main):098:0* *put 'HBT1', 'John', 'cf:a', 'CA',
>> > > > {VISIBILITY=>'MANAGER'}*
>> > > >
>> > > >
>> > > >
>> > > > hbase(main):099:0> *scan 'HBT1'*
>> > > >
>> > > > ROW
>> > > > COLUMN+CELL
>> > > >
>> > > >  John column=cf:a, timestamp=1446154722055,
>> > > > value=CA
>> > > >
>> > > > 1 row(s) in 0.0030 seconds
>> > > >
>> > > >
>> > > >
>> > > > hbase(main):100:0> *delete 'HBT1', 'John', 'cf:a',
>> > > > {VISIBILITY=>'MANAGER'}*
>> > > >
>> > > > 0 row(s) in 0.0030 seconds
>> > > >
>> > > >
>> > > >
>> > > > hbase(main):101:0> *scan 'HBT1'*
>> > > >
>> > > > ROW
>> > > > COLUMN+CELL
>> > > >
>> > > > 0 row(s) in 0.0030 seconds
>> > > >
>> > > >
>> > > >
>> > > >
>> > > >
>> > > > *Example 2:*
>> > > >
>> > > > hbase(main):010:0* *put 'HBT1', 'John', 'cf:a', 'CA',
>> > > > {VISIBILITY=>'MANAGER'}*
>> > > >
>> > > > 0 row(s) in 0.0040 seconds
>> > > >
>> > > >
>> > > >
>> > > > hbase(main):011:0> *scan 'HBT1'*
>> > > >
>> > > > ROW
>> > > > COLUMN+CELL
>> > > >
>> > > >  John column=cf:a, timestamp=1446155346473,
>> > > > value=CA
>> > > >
>> > > > 1 row(s) in 0.0060 seconds
>> > > >
>> > 

Re: Error while loading bulk data from pig to hbase

[Naresh Reddy]

Hi
Thanks for the reply.PFA for the full error.

Regard
Naresh

On Wed, Nov 4, 2015 at 11:29 AM, Ted Yu  wrote:

> Naresh:
> Can you pastebin the full error ?
> It should be in pig_.log
>
> Cheers
>
> > On Nov 3, 2015, at 9:07 PM, Naresh Reddy <
> naresh.re...@aletheconsulting.com> wrote:
> >
> > Hi
> >
> > I am getting the below error while loading bulk data from pig to hbase
> > through HBaseStorage.Please help me to resolve this issue.Thanks in
> advance.
> >
> >
> > *ERROR:pig script failed to validate: java.lang.RuntimeException: could
> not
> > instantiate 'org.apache.pig.backend.hadoop.hbase.HBaseStorage' with
> > arguments.*
> >
> > Below is my pig script.
> >
> >
> >
> >
> >
> >
> >
> > *register '/home/hduser/hbase-1.1.2/lib/zookeeper-3.4.6.jar';register
> > '/home/hduser/hbase-1.1.2/lib/guava-12.0.1.jar';register
> > '/home/hduser/hbase-1.1.2/lib/hbase-common-1.1.2.jar';register
> > '/home/hduser/hbase-1.1.2/lib/hbase-client-1.1.2.jar';register
> > '/home/hduser/hbase-1.1.2/lib/protobuf-java-2.5.0.jar';raw_data = LOAD
> > '/user/hduser/input.csv' USING PigStorage(',') AS (id:chararray,
> > fname:chararray, lname:chararray);STORE raw_data INTO
> > 'hbase://sample_names' USING
> > org.apache.pig.backend.hadoop.hbase.HBaseStorage ('info:fname
> info:lname');*
> >
> >
> > Regards
> > Naresh
>

Re: Error while loading bulk data from pig to hbase

[Ted Yu]

Naresh:
Can you pastebin the full error ?
It should be in pig_.log

Cheers

> On Nov 3, 2015, at 9:07 PM, Naresh Reddy  
> wrote:
> 
> Hi
> 
> I am getting the below error while loading bulk data from pig to hbase
> through HBaseStorage.Please help me to resolve this issue.Thanks in advance.
> 
> 
> *ERROR:pig script failed to validate: java.lang.RuntimeException: could not
> instantiate 'org.apache.pig.backend.hadoop.hbase.HBaseStorage' with
> arguments.*
> 
> Below is my pig script.
> 
> 
> 
> 
> 
> 
> 
> *register '/home/hduser/hbase-1.1.2/lib/zookeeper-3.4.6.jar';register
> '/home/hduser/hbase-1.1.2/lib/guava-12.0.1.jar';register
> '/home/hduser/hbase-1.1.2/lib/hbase-common-1.1.2.jar';register
> '/home/hduser/hbase-1.1.2/lib/hbase-client-1.1.2.jar';register
> '/home/hduser/hbase-1.1.2/lib/protobuf-java-2.5.0.jar';raw_data = LOAD
> '/user/hduser/input.csv' USING PigStorage(',') AS (id:chararray,
> fname:chararray, lname:chararray);STORE raw_data INTO
> 'hbase://sample_names' USING
> org.apache.pig.backend.hadoop.hbase.HBaseStorage ('info:fname info:lname');*
> 
> 
> Regards
> Naresh

Re: Error while loading bulk data from pig to hbase

[Naresh Reddy]

Hi
Thanks for the reply.Below is the full error log.


















































*Pig Stack Trace---ERROR 1200: Pig script failed to parse:
 pig script failed to validate:
java.lang.RuntimeException: could not instantiate
'org.apache.pig.backend.hadoop.hbase.HBaseStorage' with arguments
'[info:fname info:lname]'Failed to parse: Pig script failed to parse:  pig script failed to validate: java.lang.RuntimeException:
could not instantiate 'org.apache.pig.backend.hadoop.hbase.HBaseStorage'
with arguments '[info:fname info:lname]'at
org.apache.pig.parser.QueryParserDriver.parse(QueryParserDriver.java:199)
at org.apache.pig.PigServer$Graph.validateQuery(PigServer.java:1707)at
org.apache.pig.PigServer$Graph.registerQuery(PigServer.java:1680)at
org.apache.pig.PigServer.registerQuery(PigServer.java:623)at
org.apache.pig.tools.grunt.GruntParser.processPig(GruntParser.java:1063)
at
org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:501)
at
org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:230)
at
org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:205)
at org.apache.pig.tools.grunt.Grunt.run(Grunt.java:66)at
org.apache.pig.Main.run(Main.java:558)at
org.apache.pig.Main.main(Main.java:170)at
sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)at
org.apache.hadoop.util.RunJar.main(RunJar.java:212)Caused by:  pig script failed to validate: java.lang.RuntimeException: could
not instantiate 'org.apache.pig.backend.hadoop.hbase.HBaseStorage' with
arguments '[info:fname info:lname]'at
org.apache.pig.parser.LogicalPlanBuilder.buildStoreOp(LogicalPlanBuilder.java:1009)
at
org.apache.pig.parser.LogicalPlanGenerator.store_clause(LogicalPlanGenerator.java:7775)
at
org.apache.pig.parser.LogicalPlanGenerator.op_clause(LogicalPlanGenerator.java:1669)
at
org.apache.pig.parser.LogicalPlanGenerator.general_statement(LogicalPlanGenerator.java:1102)
at
org.apache.pig.parser.LogicalPlanGenerator.statement(LogicalPlanGenerator.java:560)
at
org.apache.pig.parser.LogicalPlanGenerator.query(LogicalPlanGenerator.java:421)
at
org.apache.pig.parser.QueryParserDriver.parse(QueryParserDriver.java:191)
... 15 moreCaused by: java.lang.RuntimeException: could not instantiate
'org.apache.pig.backend.hadoop.hbase.HBaseStorage' with arguments
'[info:fname info:lname]'at
org.apache.pig.impl.PigContext.instantiateFuncFromSpec(PigContext.java:772)
at
org.apache.pig.parser.LogicalPlanBuilder.buildStoreOp(LogicalPlanBuilder.java:988)
... 21 moreCaused by: java.lang.reflect.InvocationTargetExceptionat
sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)at
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:526)at
org.apache.pig.impl.PigContext.instantiateFuncFromSpec(PigContext.java:740)
... 22 moreCaused by: java.lang.NoSuchMethodError:
org.apache.hadoop.hbase.client.Scan.setCacheBlocks(Z)Vat
org.apache.pig.backend.hadoop.hbase.HBaseStorage.initScan(HBaseStorage.java:405)
at
org.apache.pig.backend.hadoop.hbase.HBaseStorage.(HBaseStorage.java:346)
at
org.apache.pig.backend.hadoop.hbase.HBaseStorage.(HBaseStorage.java:235)
... 27
more*

Regards
Naresh

On Wed, Nov 4, 2015 at 11:46 AM, Naresh Reddy <
naresh.re...@aletheconsulting.com> wrote:

> Hi
> Thanks for the reply.PFA for the full error.
>
> Regard
> Naresh
>
>
> On Wed, Nov 4, 2015 at 11:29 AM, Ted Yu  wrote:
>
>> Naresh:
>> Can you pastebin the full error ?
>> It should be in pig_.log
>>
>> Cheers
>>
>> > On Nov 3, 2015, at 9:07 PM, Naresh Reddy <
>> naresh.re...@aletheconsulting.com> wrote:
>> >
>> > Hi
>> >
>> > I am getting the below error while loading bulk data from pig to hbase
>> > through HBaseStorage.Please help me to resolve this issue.Thanks in
>> advance.
>> >
>> >
>> > *ERROR:pig script failed to validate: java.lang.RuntimeException: could
>> not
>> > instantiate 'org.apache.pig.backend.hadoop.hbase.HBaseStorage' with
>> > arguments.*
>> >
>> > Below is my pig script.
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> > *register '/home/hduser/hbase-1.1.2/lib/zookeeper-3.4.6.jar';register
>> > '/home/hduser/hbase-1.1.2/lib/guava-12.0.1.jar';register
>> > '/home/hduser/hbase-1.1.2/lib/hbase-common-1.1.2.jar';register
>> > '/home/hduser/hbase-1.1.2/lib/hbase-client-1.1.2.jar';register
>> > '/home/hduser/hbase-1.1.2/lib/protobuf-java-2.5.0.jar';raw_data = LOAD
>> > '/user/hduser/input.csv'