Re: Version in HBase

2014-11-12 Thread Anoop John
So you want one version with ts= give ts? Have a look at Scan#setTimeRange(long minStamp, long maxStamp) If you know the exact ts for cells, you can use Scan#setTimeStamp(long timestamp) -Anoop- On Wed, Nov 12, 2014 at 11:17 AM, Krishna Kalyan krishnakaly...@gmail.com wrote: For Example for

Re: Is it possible that HBase update performance is much better than read in YCSB test?

2014-11-12 Thread Andrew Purtell
Try this HBase YCSB client instead: https://github.com/apurtell/ycsb/tree/new_hbase_client The HBase YCSB driver in the master repo holds on to one HTable instance per driver thread. We accumulate writes into a 12MB write buffer before flushing them en masse. This is why the behavior you are

Programmatic HBase version detection/extraction

2014-11-12 Thread Otis Gospodnetic
Hi, Is there a way to detect which version of HBase one is running? Is there an API for that, or a constant with this value, or maybe an MBean or some other way to get to this info? Thanks, Otis -- Monitoring * Alerting * Anomaly Detection * Centralized Log Management Solr Elasticsearch Support

Re: Programmatic HBase version detection/extraction

2014-11-12 Thread Ted Yu
Using hbase shell: hbase(main):002:0 status 'detailed' version 0.98.4.2-hadoop2 Cheers On Wed, Nov 12, 2014 at 1:37 PM, Otis Gospodnetic otis.gospodne...@gmail.com wrote: Hi, Is there a way to detect which version of HBase one is running? Is there an API for that, or a constant with this

Re: Programmatic HBase version detection/extraction

2014-11-12 Thread Gary Helmling
Yes, you can use the org.apache.hadoop.hbase.util.VersionInfo class. From java code, you can use VersionInfo.getVersion(). From shell scripts, you can just run hbase version and parse the output. On Wed, Nov 12, 2014 at 1:37 PM, Otis Gospodnetic otis.gospodne...@gmail.com wrote: Hi, Is there

Re: Programmatic HBase version detection/extraction

2014-11-12 Thread Ted Yu
Otis: You can parse the output from status 'detailed' command - look for the line starting with 'version' I checked the output from /jmx but didn't find such information there. The version would appear in the classpath but that's not easy to parse. One note about hbase version is that it returns

Re: Call for Presentations - HBase User group meeting

2014-11-12 Thread Ryan Rawson
Just popping this back to the top, we are still looking for people to present at the HBase User Group Meetup in 2 weeks: http://www.meetup.com/hbaseusergroup/events/205219992/ As always, food and beverages are being provided. Come and hear about the cool goings on in HBase land, and possibly

Re: Programmatic HBase version detection/extraction

2014-11-12 Thread Ted Yu
Java-wise, you can use this API in HBaseAdmin: ClusterStatus getClusterStatus() throws IOException; ClusterStatus provides: public String getHBaseVersion() { Cheers On Wed, Nov 12, 2014 at 2:06 PM, Ted Yu yuzhih...@gmail.com wrote: Otis: You can parse the output from status 'detailed'

Re: Programmatic HBase version detection/extraction

2014-11-12 Thread Otis Gospodnetic
Hi, Thanks Gary, I think this is exactly what I was after! Btw. might be nice to expose this via JMX, too, for apps who needs this info but are not in process. Otis -- Monitoring * Alerting * Anomaly Detection * Centralized Log Management Solr Elasticsearch Support * http://sematext.com/ On

Trying to connect HBase Java Client I get: Failed to locate the winutils binary in the hadoop binary path

2014-11-12 Thread Néstor Boscán
Hi I'm creating my first HBase application and I'm trying to connect from the Java application in my Java IDE to my HBase server on a Horton Workds 2.1 Virtual Machine. When I run I get: Failed to locate the winutils binary in the hadoop binary path Does this mean that I have to have hadoop

Re: Trying to connect HBase Java Client I get: Failed to locate the winutils binary in the hadoop binary path

2014-11-12 Thread Ted Yu
Cycling bits: http://search-hadoop.com/m/DHED4y3J2B On Wed, Nov 12, 2014 at 3:27 PM, Néstor Boscán nesto...@gmail.com wrote: Hi I'm creating my first HBase application and I'm trying to connect from the Java application in my Java IDE to my HBase server on a Horton Workds 2.1 Virtual

Re: Trying to connect HBase Java Client I get: Failed to locate the winutils binary in the hadoop binary path

2014-11-12 Thread Néstor Boscán
Yes I already applied that. I just wanted to understand that if I have a web application then I'll have to have the hadoop distribution installed to use the hbase client. Regards, Néstor On Wed, Nov 12, 2014 at 7:57 PM, Ted Yu yuzhih...@gmail.com wrote: Cycling bits:

RE: Is it possible that HBase update performance is much better than read in YCSB test?

2014-11-12 Thread Liu, Ming (HPIT-GADSC)
Thank you Andrew, this is an excellent answer, I get it now. I will try your hbase client for a 'fair' test :-) Best Regards, Ming -Original Message- From: Andrew Purtell [mailto:apurt...@apache.org] Sent: Thursday, November 13, 2014 2:08 AM To: user@hbase.apache.org Cc: DeRoo, John

Re: Is it possible that HBase update performance is much better than read in YCSB test?

2014-11-12 Thread ramkrishna vasudevan
Thanks Andrew. This would be a very useful information along with the github link. Regards Ram On Thu, Nov 13, 2014 at 9:00 AM, Liu, Ming (HPIT-GADSC) ming.l...@hp.com wrote: Thank you Andrew, this is an excellent answer, I get it now. I will try your hbase client for a 'fair' test :-)

Storing JSON in HBase value cell, which serialization format is most compact?

2014-11-12 Thread Jianshi Huang
Hi, I'm currently saving JSON in pure String format in the value cell and depends on HBase' block compression to reduce the overhead of JSON. I'm wondering if there's a more space efficient way to store JSON? (there're lots of 0s and 1s, JSON String actually is an OK format) I want to keep the

Re: Storing JSON in HBase value cell, which serialization format is most compact?

2014-11-12 Thread Ted Yu
There is FASTDIFF data block encoding. See also http://bjson.org/ Cheers On Nov 12, 2014, at 9:08 PM, Jianshi Huang jianshi.hu...@gmail.com wrote: Hi, I'm currently saving JSON in pure String format in the value cell and depends on HBase' block compression to reduce the overhead of JSON.

Re: Storing JSON in HBase value cell, which serialization format is most compact?

2014-11-12 Thread Jianshi Huang
I thought FASTDIFF was only for rowkey and columns, great if it also works in value cell. And thanks for the bjson link! Jianshi On Thu, Nov 13, 2014 at 1:18 PM, Ted Yu yuzhih...@gmail.com wrote: There is FASTDIFF data block encoding. See also http://bjson.org/ Cheers On Nov 12, 2014,

Re: Storing JSON in HBase value cell, which serialization format is most compact?

2014-11-12 Thread ramkrishna vasudevan
Hi Since I'm storing historical data (snapshot data) and changes between adjacent value cells are relatively small. If the values are changing even if it is smaller the FASTDIFF will rewrite the value part. Only if there are exact matches then it would skip the value part. JFYI. Regards Ram