Its our pleasure to help you guys. No problem. Do write back with issues if u find any. :-)
Anoop On Tuesday, October 13, 2015, Suresh Subbiah <[email protected]> wrote: > Thank you Anoop, Ram, Ted. > Yes it is likely due to https://issues.apache.org/jira/browse/HBASE-13734 > > Narendra Goyal from our team verified a short while ago that 1.0.2 indeed > does not have this problem. > This is for the Trafodion SQL engine. We have some transactional coprocs > that run on 1.0.0 but are yet to be tested with 1.0.2. > We will do that now. > > Thank you very much for your help. At the Trafodion incubation user dlist > we try to learn from you'll and be as responsive and helpful. Thank you for > being such an excellent role model. > > BTW we were able to use the suggestions from the previous round of email > exchanges and use HFile utility to see the tags associated with a > visibility label. > > Thank you > Suresh > > > > On Mon, Oct 12, 2015 at 10:43 PM, ramkrishna vasudevan < > [email protected]> wrote: > >> I tried it on the latest trunk and this issue is not there. So as Anoop >> said the latest version of 0.98 should be solving this problem. >> @Suresh >> Let us know if you still find the issue in later versions of 0.98 and we >> can work on it to solve the problem. >> >> Regards >> Ram >> >> On Tue, Oct 13, 2015 at 9:09 AM, Anoop John <[email protected]> wrote: >> >> > Yes as such there is not mandatory to use AC along with VC. It can be >> used >> > alone.. >> > I believe u r getting the bug HBASE-13734. This is fixed in 98.13 only. >> > Just change ur version from 98.6 to 98.13 and test once. Let us know >> how >> > is it then. >> > >> > -Anoop- >> > >> > On Tue, Oct 13, 2015 at 9:01 AM, ramkrishna vasudevan < >> > [email protected]> wrote: >> > >> > > I think, even with only configuring VisibilityController there should >> not >> > > be a different behaviour, considering the fact that there are no >> > visibility >> > > labels. With just VisibilityController configured and doing puts and >> > scans >> > > using super user let me check what is happening. >> > > >> > > Regards >> > > Ram >> > > >> > > On Tue, Oct 13, 2015 at 8:47 AM, Anoop John <[email protected]> >> > wrote: >> > > >> > > > Hi Suresh >> > > > You said abt doing test as an HBase super user. You mean even >> when >> > > scan >> > > > is issues as a super user, u are not getting the rows back? >> > > > >> > > > -Anoop- >> > > > >> > > > On Tue, Oct 13, 2015 at 4:06 AM, Ted Yu <[email protected]> wrote: >> > > > >> > > > > Convention is to put AccessController ahead of VisibilityController >> > in >> > > > > hbase-site.xml >> > > > > >> > > > > Took a quick pass over region server log but haven't found much >> yet. >> > > > > >> > > > > FYI >> > > > > >> > > > > On Mon, Oct 12, 2015 at 3:28 PM, Suresh Subbiah < >> > > > > [email protected]> >> > > > > wrote: >> > > > > >> > > > > > Hi Ted, >> > > > > > >> > > > > > Thank you. Yes HDFS cluster has also been kerberized. BTW, this >> is >> > a >> > > > > > "cluster" with only one node. >> > > > > > >> > > > > > Master hbase-site.xml, RS hbase-site.ml and RS log for the time >> > > > interval >> > > > > > test was run is attached >> > > > > > >> > > > > > http://pastebin.com/zuqCC4xG >> > > > > > http://pastebin.com/88Wx0KDf >> > > > > > http://pastebin.com/QZqihN1W >> > > > > > >> > > > > > Will try deploying 1.1.2 next. >> > > > > > >> > > > > > Thanks >> > > > > > Suresh >> > > > > > >> > > > > > >> > > > > > >> > > > > > On Mon, Oct 12, 2015 at 3:46 PM, Ted Yu <[email protected]> >> > wrote: >> > > > > > >> > > > > > > bq. cluster enabled for secure HBase with kerberos >> > > > > > > >> > > > > > > I assume your hdfs cluster has also been kerberized. >> > > > > > > >> > > > > > > Please pastebin the complete hbase-site.xml >> > > > > > > >> > > > > > > Please turn on DEBUG logging and pastebin the region server log >> > > which >> > > > > > hosts >> > > > > > > visibilityTest >> > > > > > > >> > > > > > > BTW if possible, can you deploy 1.1.2 ? >> > > > > > > >> > > > > > > Cheers >> > > > > > > >> > > > > > > On Mon, Oct 12, 2015 at 1:14 PM, Suresh Subbiah < >> > > > > > > [email protected]> >> > > > > > > wrote: >> > > > > > > >> > > > > > > > Hi Ted, >> > > > > > > > >> > > > > > > > I understand that using VisibilityController on an unsercure >> > > > cluster >> > > > > is >> > > > > > > of >> > > > > > > > limited value. I am still in the early stages of my task. I >> am >> > > > logged >> > > > > > in >> > > > > > > as >> > > > > > > > HBase super user and was simply checking if rows could be >> > > accessed. >> > > > > > > > >> > > > > > > > With my colleague's help we did get the cluster enabled for >> > > secure >> > > > > > HBase >> > > > > > > > with kerberos. I repeated the test to get the same result. >> Our >> > > > > cluster >> > > > > > is >> > > > > > > > on 1.0. Do you think I may be doing something incorrectly? >> What >> > > > > > > information >> > > > > > > > can I send to help ensure that I have not made a mistake. >> > > > > > > > >> > > > > > > > Thanks >> > > > > > > > Suresh >> > > > > > > > >> > > > > > > > hbase shell >> > > > > > > > 15/10/12 14:35:09 INFO Configuration.deprecation: >> > > hadoop.native.lib >> > > > > is >> > > > > > > > deprecated. Instead, use io.native.lib.available >> > > > > > > > HBase Shell; enter 'help<RETURN>' for list of supported >> > commands. >> > > > > > > > Type "exit<RETURN>" to leave the HBase Shell >> > > > > > > > Version 1.0.0-cdh5.4.4, rUnknown, Mon Jul 6 16:59:55 PDT >> 2015 >> > > > > > > > >> > > > > > > > hbase(main):001:0> create 'visibilityTest', 'f1' >> > > > > > > > 0 row(s) in 0.7780 seconds >> > > > > > > > >> > > > > > > > => Hbase::Table - visibilityTest >> > > > > > > > hbase(main):002:0> put 'visibilityTest', 'r1', 'f1:c1', >> > 'value1' >> > > > > > > > 0 row(s) in 0.1300 seconds >> > > > > > > > >> > > > > > > > hbase(main):003:0> deleteall 'visibilityTest', 'r1' >> > > > > > > > 0 row(s) in 0.0330 seconds >> > > > > > > > >> > > > > > > > hbase(main):004:0> put 'visibilityTest', 'r1', 'f1:c1', >> > 'value2' >> > > > > > > > 0 row(s) in 0.0150 seconds >> > > > > > > > >> > > > > > > > hbase(main):005:0> scan 'visibilityTest' >> > > > > > > > ROW COLUMN+CELL >> > > > > > > > >> > > > > > > > 0 row(s) in 0.0550 seconds >> > > > > > > > >> > > > > > > > hbase(main):006:0> scan 'visibilityTest', {RAW=>TRUE} >> > > > > > > > ROW COLUMN+CELL >> > > > > > > > >> > > > > > > > r1 column=f1:, timestamp=1444660561138, >> > > > > > > > type=DeleteFamily >> > > > > > > > r1 column=f1:c1, timestamp=1444660576868, >> > > > > > value=value2 >> > > > > > > > >> > > > > > > > 1 row(s) in 0.0370 seconds >> > > > > > > > >> > > > > > > > ----------------------------------------------------- >> > > > > > > > <property> >> > > > > > > > <name>hbase.coprocessor.master.classes</name> >> > > > > > > > >> > > > > > > > >> > > > > > > > >> > > > > > > >> > > > > > >> > > > > >> > > > >> > > >> > >> <value>org.apache.hadoop.hbase.security.visibility.VisibilityController,org.apache.hadoop.hbase.security.access.AccessController</value> >> > > > > > > > </property> >> > > > > > > > >> > > > > > > > <property> >> > > > > > > > <name>hbase.coprocessor.region.classes</name> >> > > > > > > > >> > > > > > > > >> > > > > > > >> > > > > > >> > > > > >> > > > >> > > >> > >> <value>org.apache.hadoop.hbase.security.visibility.VisibilityController,org.apache.hadoop.hbase.security.token.TokenProvider,org.apache.hadoop.hbase.security.access.SecureBulkLoadEndpoint,org.apache.hadoop.hbase.security.access.AccessController</value> >> > > > > > > > </property> >> > > > > > > > >> > > > > > > > -------------------------------------------------------- >> > > > > > > > >> > > > > > > > >> > > > > > > > >> > > > > > > > >> > > > > > > > On Sat, Oct 10, 2015 at 9:51 PM, Ted Yu < [email protected] >> > >> > > > wrote: >> > > > > > > > >> > > > > > > > > To my understanding, VisibilityController is used in a >> secure >> > > > > > cluster. >> > > > > > > > > Without security, how do you enforce that only select >> user(s) >> > > can >> > > > > > > access >> > > > > > > > > certain cells ? >> > > > > > > > > >> > > > > > > > > Please see the following sections in refguide: >> > > > > > > > > >> > > > > > > > > >> http://hbase.apache.org/book.html#hbase.secure.configuration >> > > > > > > > > >> > > > > > > > > >> > > > > > > > >> > > > > > > >> > > > > > >> > > > > >> > > > >> > > >> > >> http://hbase.apache.org/book.html#_server_side_configuration_for_simple_user_access_operation >> > > > > > > > > >> > > > > > > > > On Sat, Oct 10, 2015 at 7:40 PM, Suresh Subbiah < >> > > > > > > > > [email protected]> >> > > > > > > > > wrote: >> > > > > > > > > >> > > > > > > > > > Hi Ted, >> > > > > > > > > > >> > > > > > > > > > Thank you for your response. >> > > > > > > > > > I found a machine with HBase 1.0.0 and tried the script >> > with >> > > > all >> > > > > 6 >> > > > > > > > > coprocs >> > > > > > > > > > you listed (2 in master, and 4 in RS). I still do not see >> > the >> > > > row >> > > > > > > after >> > > > > > > > > the >> > > > > > > > > > second scan. >> > > > > > > > > > >> > > > > > > > > > However my cluster is not secure enabled I think. Is that >> > > > > > necessary? >> > > > > > > I >> > > > > > > > am >> > > > > > > > > > not sure how to do that, though I can ask other members >> of >> > my >> > > > > team >> > > > > > > and >> > > > > > > > > try >> > > > > > > > > > it if that will help. >> > > > > > > > > > >> > > > > > > > > > It will be ideal if we could get this to work on a 1.0 >> > based >> > > > > > version. >> > > > > > > > > > Moving to 1.1 will take more time since we have some >> > > > > dependencies. >> > > > > > > > > > >> > > > > > > > > > Thank you >> > > > > > > > > > Suresh >> > > > > > > > > > >> > > > > > > > > > 15/10/10 19:20:44 INFO Configuration.deprecation: >> > > > > hadoop.native.lib >> > > > > > > is >> > > > > > > > > > deprecated. Instead, use io.native.lib.available >> > > > > > > > > > HBase Shell; enter 'help<RETURN>' for list of supported >> > > > commands. >> > > > > > > > > > Type "exit<RETURN>" to leave the HBase Shell >> > > > > > > > > > Version 1.0.0-cdh5.4.4, rUnknown, Mon Jul 6 16:59:55 PDT >> > > 2015 >> > > > > > > > > > >> > > > > > > > > > *hbase(main):001:0> create 'visibilityTest', 'f1' * >> > > > > > > > > > *0 row(s) in 0.5460 seconds* >> > > > > > > > > > >> > > > > > > > > > *=> Hbase::Table - visibilityTest* >> > > > > > > > > > *hbase(main):002:0> put 'visibilityTest', 'r1', 'f1:c1', >> > > > > 'value1' * >> > > > > > > > > > *0 row(s) in 0.0670 seconds* >> > > > > > > > > > >> > > > > > > > > > *hbase(main):003:0> deleteall 'visibilityTest', 'r1' * >> > > > > > > > > > *0 row(s) in 0.0090 seconds* >> > > > > > > > > > >> > > > > > > > > > *hbase(main):004:0> put 'visibilityTest', 'r1', 'f1:c1', >> > > > > 'value2'* >> > > > > > > > > > *0 row(s) in 0.0040 seconds* >> > > > > > > > > > >> > > > > > > > > > *hbase(main):005:0> scan 'visibilityTest'* >> > > > > > > > > > *ROW COLUMN+CELL >> > > > > > > > > > * >> > > > > > > > > > *0 row(s) in 0.0160 seconds* >> > > > > > > > > > >> > > > > > > > > > *hbase(main):006:0> scan 'visibilityTest', {RAW=>TRUE}* >> > > > > > > > > > *ROW COLUMN+CELL >> > > > > > > > > > * >> > > > > > > > > > * r1 column=f1:, >> timestamp=1444530064056, >> > > > > > > > > > type=DeleteFamily * >> > > > > > > > > > * r1 column=f1:c1, >> > timestamp=1444530064084, >> > > > > > > > > value=value2 >> > > > > > > > > > * >> > > > > > > > > > *1 row(s) in 0.0580 seconds* >> > > > > > > > > > >> > > > > > > > > > *hbase(main):007:0> exit* >> > > > > > > > > > >> > > > > > > > > > >> > > > > > > > > > On Sat, Oct 10, 2015 at 7:26 PM, Ted Yu < >> > [email protected] >> > > > >> > > > > > wrote: >> > > > > > > > > > >> > > > > > > > > > > I tried the sequence of commands from your example on a >> > > > secure >> > > > > > > 1.1.2 >> > > > > > > > > > > cluster with the following config: >> > > > > > > > > > > >> > > > > > > > > > > <property> >> > > > > > > > > > > <name>hbase.coprocessor.master.classes</name> >> > > > > > > > > > > >> > > > > > > > > > > >> > > > > > > > > > >> > > > > > > > > >> > > > > > > > >> > > > > > > >> > > > > > >> > > > > >> > > > >> > > >> > >> <value>org.apache.hadoop.hbase.security.access.AccessController,org.apache.hadoop.hbase.security.visibility.VisibilityController</value> >> > > > > > > > > > > </property> >> > > > > > > > > > > <property> >> > > > > > > > > > > <name>hbase.coprocessor.region.classes</name> >> > > > > > > > > > > >> > > > > > > > > > > >> > > > > > > > > > >> > > > > > > > > >> > > > > > > > >> > > > > > > >> > > > > > >> > > > > >> > > > >> > > >> > >> <value>org.apache.hadoop.hbase.security.token.TokenProvider,org.apache.hadoop.hbase.security.access.SecureBulkLoadEndpoint,org.apache.hadoop.hbase.security.access.AccessController,org.apache.hadoop.hbase.security.visibility.VisibilityController</value> >> > > > > > > > > > > </property> >> > > > > > > > > > > >> > > > > > > > > > > I got: >> > > > > > > > > > > >> > > > > > > > > > > hbase(main):005:0> scan 'visibilityTest' >> > > > > > > > > > > ROW >> COLUMN+CELL >> > > > > > > > > > > r1 >> > column=f1:c1, >> > > > > > > > > > > timestamp=1444522994981, value=value2 >> > > > > > > > > > > 1 row(s) in 0.1020 seconds >> > > > > > > > > > > >> > > > > > > > > > > Can you try again with 0.98.15 release whose vote >> passed >> > > > Friday >> > > > > > to >> > > > > > > > see >> > > > > > > > > if >> > > > > > > > > > > what you observed can be reproduced ? >> > > > > > > > > > > >> > > > > > > > > > > Cheers >> > > > > > > > > > > >> > > > > > > > > > > On Sat, Oct 10, 2015 at 3:58 PM, Suresh Subbiah < >> > > > > > > > > > > [email protected]> >> > > > > > > > > > > wrote: >> > > > > > > > > > > >> > > > > > > > > > > > Hi, >> > > > > > > > > > > > >> > > > > > > > > > > > When I run the following script from hbase shell the >> > last >> > > > > scan >> > > > > > > > > returns >> > > > > > > > > > no >> > > > > > > > > > > > rows >> > > > > > > > > > > > >> > > > > > > > > > > > create 'visibilityTest', 'f1' >> > > > > > > > > > > > put 'visibilityTest', 'r1', 'f1:c1', 'value1' >> > > > > > > > > > > > deleteall 'visibilityTest', 'r1' >> > > > > > > > > > > > put 'visibilityTest', 'r1', 'f1:c1', 'value2' >> > > > > > > > > > > > scan 'visibilityTest' >> > > > > > > > > > > > >> > > > > > > > > > > > *hbase(main):013:0> scan 'visibilityTest'* >> > > > > > > > > > > > *ROW COLUMN+CELL >> > > > > > > > > > > > * >> > > > > > > > > > > > *0 row(s) in 0.0100 seconds* >> > > > > > > > > > > > >> > > > > > > > > > > > >> > > > > > > > > > > > However if I run >> > > > > > > > > > > > scan 'visibilityTest' , {RAW=>TRUE} >> > > > > > > > > > > > >> > > > > > > > > > > > I see that the second row that I put is indeed there >> > and >> > > > has >> > > > > a >> > > > > > > > > > timestamp >> > > > > > > > > > > > value higher that the previous delete >> > > > > > > > > > > > >> > > > > > > > > > > > *hbase(main):014:0> scan 'visibilityTest', >> {RAW=>TRUE}* >> > > > > > > > > > > > *ROW COLUMN+CELL >> > > > > > > > > > > > * >> > > > > > > > > > > > * r1 column=f1:, >> > > timestamp=1444516578296, >> > > > > > > > > > > > type=DeleteFamily * >> > > > > > > > > > > > * r1 column=f1:c1, >> > > > timestamp=1444516647655, >> > > > > > > > > > > value=value2 >> > > > > > > > > > > > * >> > > > > > > > > > > > *1 row(s) in 0.0110 seconds* >> > > > > > > > > > > > >> > > > > > > > > > > > >> > > > > > > > > > > > >> > > > > > > > > > > > This is on hbase 0.98.6. Problem is seen only when >> > > > > > > hbase-site.xml >> > > > > > > > > has >> > > > > > > > > > > > these lines. No other coprocessors were used during >> > this >> > > > > test. >> > > > > > > > > > > > >> > > > > > > > > > > > <property> >> > > > > > > > > > > > <name>hbase.coprocessor.region.classes</name> >> > > > > > > > > > > > >> > > > > > > > > > > > >> > > > > > > > > > > > >> > > > > > > > > > > >> > > > > > > > > > >> > > > > > > > > >> > > > > > > > >> > > > > > > >> > > > > > >> > > > > >> > > > >> > > >> > >> <value>org.apache.hadoop.hbase.security.visibility.VisibilityController</value> >> > > > > > > > > > > > </property> >> > > > > > > > > > > > <property> >> > > > > > > > > > > > <name>hbase.coprocessor.master.classes</name> >> > > > > > > > > > > > >> > > > > > > > > > > > >> > > > > > > > > > > > >> > > > > > > > > > > >> > > > > > > > > > >> > > > > > > > > >> > > > > > > > >> > > > > > > >> > > > > > >> > > > > >> > > > >> > > >> > >> <value>org.apache.hadoop.hbase.security.visibility.VisibilityController</value> >> > > > > > > > > > > > </property> >> > > > > > > > > > > > <property> >> > > > > > > > > > > > <name>hfile.format.version</name> >> > > > > > > > > > > > <value>3</value> >> > > > > > > > > > > > </property> >> > > > > > > > > > > > >> > > > > > > > > > > > Any suggestions of what I may be doing incorrectly? >> Or >> > is >> > > > > this >> > > > > > a >> > > > > > > > bug? >> > > > > > > > > > > > >> > > > > > > > > > > > Thank you >> > > > > > > > > > > > Suresh >> > > > > > > > > > > > >> > > > > > > > > > > >> > > > > > > > > > >> > > > > > > > > >> > > > > > > > >> > > > > > > >> > > > > > >> > > > > >> > > > >> > > >> > >> >
