Re: How practical is it to add a timestamp oracle on Zookeeper

2013-04-26 Thread Enis Söztutar
Hi, I presume you have read the percolator paper. The design there uses a single ts oracle, and BigTable itself as the transaction manager. In omid, they also have a TS oracle, but I do not know how scalable it is. But using ZK as the TS oracle would not work, since ZK can scale up to 40-50K

Re: HBase Types: Explicit Null Support

2013-04-01 Thread Enis Söztutar
I think having Int32, and NullableInt32 would support minimum overhead, as well as allowing SQL semantics. On Mon, Apr 1, 2013 at 7:26 PM, Nick Dimiduk ndimi...@gmail.com wrote: Furthermore, is is more important to support null values than squeeze all representations into minimum size

Re: Does HBase RegionServer benefit from OS Page Cache

2013-03-25 Thread Enis Söztutar
enjoyed the depth of explanation from both Enis and J-D. I was indeed mistakenly referring to HFile as HLog, fortunately you were still able understand my question. Thanks, Pankaj On Mar 21, 2013, at 1:28 PM, Enis Söztutar enis@gmail.com wrote: I think the page cache

Re: Does HBase RegionServer benefit from OS Page Cache

2013-03-25 Thread Enis Söztutar
From: Enis Söztutar [enis@gmail.com] Sent: Monday, March 25, 2013 11:24 AM To: hbase-user Cc: lars hofhansl Subject: Re: Does HBase RegionServer benefit from OS Page Cache Thanks Liyin for sharing your use cases. Related to those, I was thinking of two improvements: - AFAIK, MySQL

Re: java.lang.OutOfMemoryError: Direct buffer memory

2013-03-25 Thread Enis Söztutar
Hi, From the logs, it seems you are running into the same problem I have reported last week: https://issues.apache.org/jira/browse/HBASE-8143 There are some mitigation strategies outlined in that jira. It would be good if you can confirm: - How many regions in the region server - How many open

Re: Is there a way to only scan data in memstore

2013-03-21 Thread Enis Söztutar
I would caution against going that route for various reasons: - Correctness : You can never be sure to sync the memstore flushes and compactions changing the files under you. - Security: All files from HBase are owned by HBase user. Other users should not be able to read it. -

Re: Evenly splitting the table

2013-03-21 Thread Enis Söztutar
Hi, HBase cannot deduce the row key structure, thus cannot pre split the table unless it knows the basic format for the row keys. shameless_self_plug you can look at the blog post about splits here: http://hortonworks.com/blog/apache-hbase-region-splitting-and-merging//shameless_self_plug Enis

Re: Does HBase RegionServer benefit from OS Page Cache

2013-03-21 Thread Enis Söztutar
I think the page cache is not totally useless, but as long as you can control the GC, you should prefer the block cache. Some of the reasons of the top of my head: - In case of a cache hit, for OS cache, you have to go through the DN layer (an RPC if ssr disabled), and do a kernel jump, and read

Re: Store images in Hbase using Thrift and Php

2013-03-14 Thread Enis Söztutar
If you check out the latest source code, there are some examples under hbase-examples/src/main/php. Enis On Thu, Mar 14, 2013 at 6:06 AM, dimpanagr dimpan...@yahoo.gr wrote: Hi I have set up HBase and trying to use Thrift-Php to upload an image and then display it. Is there any example or

Re: Welcome our newest Committer Anoop

2013-03-11 Thread Enis Söztutar
Congrats and welcome. On Mon, Mar 11, 2013 at 2:21 AM, Nicolas Liochon nkey...@gmail.com wrote: Congrats, Anoop! On Mon, Mar 11, 2013 at 5:35 AM, rajeshbabu chintaguntla rajeshbabu.chintagun...@huawei.com wrote: Contratulations Anoop!

Re: Map Reduce with multiple scans

2013-02-27 Thread Enis Söztutar
There is a MultiTableInputFormat that has been recently added to HBase. You might want to take a look at it. https://github.com/apache/hbase/blob/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/mapreduce/MultiTableInputFormat.java Enis On Wed, Feb 27, 2013 at 8:17 AM, Paul van Hoven

Re: Ganglia Vs Nagios

2013-02-27 Thread Enis Söztutar
Hey, You need to use both. If you setup the cluster through Apache Ambari ( http://incubator.apache.org/ambari/), it will setup everything for you. Enis On Tue, Feb 26, 2013 at 11:43 PM, Leonid Fedotov lfedo...@hortonworks.comwrote: Ganglia is monitoring and trending. Nagios is health

Re: PreSplit the table with Long format

2013-02-21 Thread Enis Söztutar
Hi, Maybe you can implement a SplitAlgorithm, and use the RegionSplitter utility. You can find some information about the usage of SplitAlgorithm here: http://hortonworks.com/blog/apache-hbase-region-splitting-and-merging/ Enis On Tue, Feb 19, 2013 at 11:11 PM, Farrokh Shahriari

Re: HBase without compactions?

2013-02-21 Thread Enis Söztutar
From some of their presentations, I've gathered that they implement B-Tree's instead of LSM's on top of their file system which allows random writes. They also claim that they are converting random mutation requests to the B-Tree leafs to sequential-writes. They are also talking about mini-WALs to

Re: [ANNOUNCE] New Apache HBase Committer - Devaraj Das

2013-02-07 Thread Enis Söztutar
Congrats. Well deserved. Enis On Thu, Feb 7, 2013 at 9:56 AM, Jimmy Xiang jxi...@cloudera.com wrote: Congratulations! On Thu, Feb 7, 2013 at 9:54 AM, Andrew Purtell apurt...@apache.org wrote: Congratulations Devaraj! On Wed, Feb 6, 2013 at 9:19 PM, Ted Yu yuzhih...@gmail.com

Re: Announcing Phoenix: A SQL layer over HBase

2013-01-30 Thread Enis Söztutar
Cool. Will play a bit later on. Was waiting for it to appear. On Wed, Jan 30, 2013 at 1:04 PM, James Taylor jtay...@salesforce.comwrote: We are pleased to announce the immediate availability of a new open source project, Phoenix, a SQL layer over HBase that powers the HBase use cases at

Re: [ANNOUNCE] New Apache HBase Committers - Matteo Bertozzi and Chunhui Shen

2013-01-02 Thread Enis Söztutar
Congrats. Enis On Wed, Jan 2, 2013 at 11:47 AM, Himanshu Vashishtha hvash...@cs.ualberta.ca wrote: Congrats Matteo and Chunhui! :) On Wed, Jan 2, 2013 at 11:40 AM, Jimmy Xiang jxi...@cloudera.com wrote: Congratulations! Matteo and Chunhui!! On Wed, Jan 2, 2013 at 11:37 AM, Jonathan

Re: MemStore and prefix encoding

2012-08-28 Thread Enis Söztutar
I would still caution relying on the sorting order between values of the same cf, qualifier and timestamp. If for example, there is a Delete, it will eclipse subsequent Puts given the same timestamp, even though Put happened after Delete. Enis On Mon, Aug 27, 2012 at 9:20 AM, Tom Brown

Re: Using HBase as ACL store for Spring Security

2012-02-22 Thread Enis Söztutar
Interesting use case. For your product, do you also need to secure hbase as well? Enis On Wed, Feb 22, 2012 at 10:03 AM, Alan Chaney a...@mechnicality.com wrote: Hi We are using Spring Security and HBase in our product. We are adding ACL support through Spring and are looking at

Re: HBase coprocessors blog posted

2012-02-16 Thread Enis Söztutar
Hey Andrew, any update on this. I am dying to see the blog posts : ). I can also offer my relatively unworthy 2 cents, if you guys need any help. Thanks, Enis On Wed, Feb 1, 2012 at 7:23 PM, Andrew Purtell apurt...@apache.org wrote: Gary, Eugene is resurrecting posts on security now. I was

<    1   2