on blockCache hitRatio

HBASE_HEAP_SIZE=10G, use LruBlockCache with 0.4 of HBASE_HEAP_SIZE after hbase run 15 days , in find in some RS, there are 200M free block cache, but hit ration is 10%, too low i think the hit ration is low may be bacause of small block cache size(4G) is there any suggestions to get a higher

Re: Writing visibility labels with HFileOutputFormat2

Thanks for the updates here. Going through the mails here >> Why is it that a client user without admin/super user privileges can set a visibility expression using Put.setCellVisibility, but if we want to write using HFiles, I get your point now. There is a property

Re: May I run hbase on top of Alluxio/tacyon

I want to test if run on alluxio could improve performance,because alluxio is a distribution filesystem top on memory and under filesystem could be hdfs or s3 or something. 2016-06-16 10:32 GMT+08:00 Ted Yu : > Since you already have hadoop 2.7.1, why is alluxio 1.1.0

Re: May I run hbase on top of Alluxio/tacyon

Since you already have hadoop 2.7.1, why is alluxio 1.1.0 needed ? Can you illustrate your use case ? Thanks On Wed, Jun 15, 2016 at 7:27 PM, kevin wrote: > hi,all: > > I wonder to know If run hbase on Alluxio/tacyon is possible and a good > idea, and can anybody

May I run hbase on top of Alluxio/tacyon

hi,all: I wonder to know If run hbase on Alluxio/tacyon is possible and a good idea, and can anybody share the experience.,thanks. I will try hbase0.98.16 with hadoop2.7.1 on top of alluxio 1.1.0.

Re: HBase regionserver SIGSEGV periodically

This seems to have resolve the issue, no sigsegv seen... yet. Thanks Esteban Harry On Fri, Jun 10, 2016 at 6:08 PM Esteban Gutierrez wrote: > Hi Harry, > > As you mentioned, moving to JDK8 is a good idea. There are many known > issues with G1GC and JDK7 that make using

Re: hbase bulk load with map reduce error

Please let me know what am I missing here. I am using MapR hadoop. Please find the classpath which is showing all the jars with versions, mean while I will get the code snippet also. I am using bulk write approach with mapper and reducer. HBase table created with bulk load enabled as true. +

RE: Writing visibility labels with HFileOutputFormat2

So, I can see that I can correctly get the Lists from the VisibilityExpressionResolver, set them on the Cell, and write them using HFileOutputFormat2, however when I scan using an unprivileged user I can still see the cells. If I write the cells with setCellVisibility the unprivileged user

RE: Writing visibility labels with HFileOutputFormat2

Looking at the source for how DefaultCellLabelServiceImpl checks authorisation I noted it's just that the user just needs to have the 'system' label auth privileges - not admin/super user as I thought you meant Ram. So technically, I could have a client user that is given the system label

RE: Writing visibility labels with HFileOutputFormat2

Thanks Ted - It was just a class cast on line 161 of HFileOutput2.write, because I had previously read that you could give it Puts, but it can actually only take Cells. You can only do Puts if you use configureIncrementalLoad which then sets up the PutSortReducer as I discussed in my other

RE: Writing visibility labels with HFileOutputFormat2

I see now from some other examples I've found that actually this form of using HFileOutputFormat2 to write Puts will use the PutSortReducer if you set the map output class of the job you give it to Put. Looking at the source for PutSourceReducer it seems that it will actually lose the Cell

Re: Writing visibility labels with HFileOutputFormat2

Tom: Can you pastebin the stack trace for the exception ? It would be nice if you can show snippet of your code too. Thanks > On Jun 15, 2016, at 8:24 AM, Ellis, Tom (Financial Markets IT) > wrote: > > So I have a working prototype using just bulk puts

RE: Writing visibility labels with HFileOutputFormat2

So I have a working prototype using just bulk puts on a table and using setCellVisibility as necessary. Now I'm trying to do it using HFile. Sorry Ram, I don't quite follow why the user doing the writing of the HFile has to be an admin/super user? Is that necessary to load HFiles? The use case

Re: HBase acl commands are too slow

Have you looked at http://hbase.apache.org/book.html#security ? I noticed that DEBUG logging was not on in the log you posted earlier. Is it possible to turn on DEBUG logging and repeat the operation ? Thanks On Wed, Jun 15, 2016 at 2:12 AM, kumar r wrote: > Hi Ted, > >

Re: Big Data Interview

Please don't cross post. This seems to be an advertisement. > On Jun 15, 2016, at 4:41 AM, Chaturvedi Chola > wrote: > > Good book on interview preparation for big data > > https://notionpress.com/read/big-data-interview-faqs

Big Data Interview

Good book on interview preparation for big data https://notionpress.com/read/big-data-interview-faqs

Re: Writing visibility labels with HFileOutputFormat2

>>We could I guess create multiple puts for cells in the same row with different labels and use the setCellVisibility on each individual put/cell, but will this create additional overhead? This can be done. If you want different cells in the same row to have different labels then it is better to

HBase number of columns

Hi, As per the official documentation of HBase it is mentioned that HBase typical schema should contain 1 to 3 column families per table (https://hbase.apache.org/book.html#table_schema_rules_of_thumb ) . However there is no mention of how many column qualifiers should a row contain for each

RE: Writing visibility labels with HFileOutputFormat2

Hmm, is there no other way to set labels on individual cells where we don't have to give the client users system perms? For instance, client users can set the cell visibility on the entire put without having this (i.e. put.setCellVisibility("label")) and the VisibilityController will check

Re: Writing visibility labels with HFileOutputFormat2

The visibility expression resolver tries to scan the labels table and the user using the resolver should have the SYSTEM privileges. Since the information that is getting accessed is sensitive information. Suppose in your above case you have the client user added as a an admin then when you scan

RE: Writing visibility labels with HFileOutputFormat2

Yeah, thanks for this Ram. Although in my testing I have found that a client user attempting to use the visibility expression resolver doesn't seem to have the ability to scan the hbase:labels table for the full list of labels and thus can't get the ordinals/tags to add to the cell. Does the

Re: HBase acl commands are too slow

Hi Ted, Thanks for your reply. I cannot find anything in configuration. Can you tell me what might be root cause for this issue? What will be major cause for acl command taking more than 30 seconds to process. I cannot find anything other than this in hbase log. Is there any documentation