Re: Region balancing query

2015-02-16 Thread Abe Weinograd
balancer said true and it is not disabled. Thanks again for your help. Abe On Mon, Feb 16, 2015 at 11:23 AM, Ted Yu yuzhih...@gmail.com wrote: What was the output from 'balancer' command ? Was is possible that balancer was disabled ? Cheers On Mon, Feb 16, 2015 at 8:04 AM, Abe Weinograd

Re: Region balancing query

2015-02-16 Thread Abe Weinograd
Excellent. If i trigger the balancer, should this start to help or only for future region creation? Thanks, Abe On Mon, Feb 16, 2015 at 9:35 AM, Ted Yu yuzhih...@gmail.com wrote: Yes. This setting should be modified on Master. Cheers On Mon, Feb 16, 2015 at 6:27 AM, Abe Weinograd

Re: Region balancing query

2015-02-16 Thread Ted Yu
You should see effect in the next balancer run. Cheers On Mon, Feb 16, 2015 at 7:52 AM, Abe Weinograd a...@flonet.com wrote: Excellent. If i trigger the balancer, should this start to help or only for future region creation? Thanks, Abe On Mon, Feb 16, 2015 at 9:35 AM, Ted Yu

Re: Region balancing query

2015-02-16 Thread Abe Weinograd
Ok. I forced the balancer run and am not seeing anything after a few minutes. Master logs isn't showing anything. Should I look at the RS ones instead? On Mon, Feb 16, 2015 at 11:03 AM, Ted Yu yuzhih...@gmail.com wrote: You should see effect in the next balancer run. Cheers On Mon, Feb

Re: ColumnSuffixFilter in HBase

2015-02-16 Thread anil gupta
Hi Alok, xyz$ regex is enough to get the desired result. It will perform better than ^.*xyz$. ^.*xyz$ is doing some unnecessary work. Thanks, Anil Gupta On Wed, Feb 11, 2015 at 10:44 AM, Alok Singh aloksi...@gmail.com wrote: A simple String suffix check will be definitely faster, but I doubt

Re: Region balancing query

2015-02-16 Thread Ted Yu
What was the output from 'balancer' command ? Was is possible that balancer was disabled ? Cheers On Mon, Feb 16, 2015 at 8:04 AM, Abe Weinograd a...@flonet.com wrote: Ok. I forced the balancer run and am not seeing anything after a few minutes. Master logs isn't showing anything. Should

Q regarding thrift server to expose RDD via SQL

2015-02-16 Thread Marco
Hi, I've played with the feature to expose RDD via Thrift to enable JDBC access. (Spark 1.2) val eventsView = sqlContext.createSchemaRDD(eventSchemaRdd) eventsView.registerTempTable(Events) HiveThriftServer2.startWithContext(sqlContext) This works all fine. Now, my understanding is you

Re: Help needed on choosing OCR software

2015-02-16 Thread anil gupta
Hi Rams, I don't think HBase mailing list is appropriate to search for an OCR. Please use appropriate mailing list. ~Anil On Mon, Feb 16, 2015 at 5:27 AM, hongbin ma mahong...@apache.org wrote: I used to came across this: https://code.google.com/p/tesseract-ocr/ AFAIK, OCR requires training

Re: Does hbase WAL ensures no data loss?

2015-02-16 Thread Andrew Purtell
What is hbase's philosophy in this? Does it allow some degree of data loss? ​HBase doesn't allow data loss, in the sense that HBase never chooses on its own to be less than fully durable. However, our client API does allow users to submit mutations with different durability guarantees. The

Re: Q regarding thrift server to expose RDD via SQL

2015-02-16 Thread Nick Dimiduk
Using TableInputFormat directly will have better scalability than HS2. Better still, use TableSnapshotInputFormat to work from a snapshot (since RDDs are immutable anyway). -n On Monday, February 16, 2015, Marco marco@gmail.com wrote: Hi, I've played with the feature to expose RDD via

Re: Hbase not taking inserts from Remote Machine

2015-02-16 Thread Serega Sheypak
You need to open region server ports. Client directly sends put to appropriate region server. вторник, 17 февраля 2015 г. пользователь Vineet Mishra написал: -- Forwarded message -- From: Vineet Mishra clearmido...@gmail.com javascript:; Date: Tue, Feb 17, 2015 at 12:32 PM

Re: PerformanceEvaluation: filterScan

2015-02-16 Thread Jerry He
Hi, I was on 0.98 running PerformanceEvaluation. randomRead or sequentialRead all show good read counts on the table regions. filterScan shows exactly the same as you had. Looking at the code. It could be as expected. The filter has no matching returns, so the region server is in

Does hbase WAL ensures no data loss?

2015-02-16 Thread hongbin ma
hi, all It seems WAL.append() in hbase, the javadoc says: * * Append a set of edits to the WAL. The WAL is not flushed/sync'd after this transaction* * * completes BUT on return this edit must have its region edit/sequence id assigned* * * else it messes up our unification of mvcc and

Re: Help needed on choosing OCR software

2015-02-16 Thread hongbin ma
I used to came across this: https://code.google.com/p/tesseract-ocr/ AFAIK, OCR requires training if you want to get a high quality recognition. and it's not easy to have a model that suits all styles of hand writings On Mon, Feb 16, 2015 at 7:33 PM, N. Ramasubramanian

Re: Does hbase WAL ensures no data loss?

2015-02-16 Thread Dave Latham
Hi Hongbin, The WAL class is used internally to the region server. Typically an HBase write operation will first call WAL.append() with the data, then later, after releasing locks, call WAL.sync() to ensure that the data for that write has been synced to be durable before returning to the client

Re: Region balancing query

2015-02-16 Thread Abe Weinograd
Thanks Ted. We are putting this in the hbase-site.xml for the Master? Abe On Fri, Feb 13, 2015 at 5:03 PM, Shahab Yunus shahab.yu...@gmail.com wrote: Thanks, we will try that and report back. Regards, Shahab On Fri, Feb 13, 2015 at 4:56 PM, Ted Yu yuzhih...@gmail.com wrote: You can

Re: Does hbase WAL ensures no data loss?

2015-02-16 Thread Ted Yu
Hongbin: Please go through HRegion#doMiniBatchMutation(). You will see the connection of wal.append() and the WAL sync in step 7. Cheers On Mon, Feb 16, 2015 at 6:05 AM, Dave Latham lat...@davelink.net wrote: Hi Hongbin, The WAL class is used internally to the region server. Typically an

RE: managing HConnection

2015-02-16 Thread Liu, Ming (HPIT-GADSC)
Hi, Thank you Serega for the helpful reply and thanks Jneidi for asking this. I have similar confusion. So Serega, when does your application finally close the HConnection? Or the connection is NEVER closed as long as your application is running? Is it OK to NOT close the HConnection and the

Re: managing HConnection

2015-02-16 Thread Serega Sheypak
Hi, I'm closing it in servlet.destroy. I didn't see any problems here for months. I'm using servlet lifecycle to deal with hconnection. вторник, 17 февраля 2015 г. пользователь Liu, Ming (HPIT-GADSC) написал: Hi, Thank you Serega for the helpful reply and thanks Jneidi for asking this. I

Fwd: Hbase not taking inserts from Remote Machine

2015-02-16 Thread Vineet Mishra
-- Forwarded message -- From: Vineet Mishra clearmido...@gmail.com Date: Tue, Feb 17, 2015 at 12:32 PM Subject: Hbase not taking inserts from Remote Machine To: cdh-u...@cloudera.org Hi All, I am trying to communicate and insert some data to my Hbase(0.98.6-cdh5.3.0) running on

Re: Region balancing query

2015-02-16 Thread Ted Yu
Yes. This setting should be modified on Master. Cheers On Mon, Feb 16, 2015 at 6:27 AM, Abe Weinograd a...@flonet.com wrote: Thanks Ted. We are putting this in the hbase-site.xml for the Master? Abe On Fri, Feb 13, 2015 at 5:03 PM, Shahab Yunus shahab.yu...@gmail.com wrote: Thanks, we

PerformanceEvaluation: filterScan

2015-02-16 Thread Jean-Marc Spaggiari
Any one tries this recently? hbase org.apache.hadoop.hbase.PerformanceEvaluation --nomapred --rows=100 filterScan 3 I let it run for 30 minutes and killed it. 0 requets against the table, 649 requests against the META. Seems to not be working anymore. Did not get a chance to look at the code

Re: Re: managing HConnection

2015-02-16 Thread Serega Sheypak
Newrelic shows 50K RPM each request to servlet == 1-3 put/get to HBase. I have mixed workload. Is it strange :) ? 2015-02-16 10:37 GMT+03:00 David chen c77...@163.com: 5 rpm? I am curious how the result is concluded?

Re:Re: Re: managing HConnection

2015-02-16 Thread David chen
In your scene, per servlet used a HConnecton object, so the response for 5 rpm is so easy. If these servlets should share the only one HConnecton object, whether or not the response latency for 5 rpm would increase?

Re: Streaming data to htable

2015-02-16 Thread hongbin ma
Hi Andrey, thanks for your reply, I found them very inspiring. I'm curious about the scheme you described: It looks to me you've sharded your data into separate regions, are you fixing the number of these regions? If yes, each of the growing region will spawn lots of compactions while writing