Hi St. Ack , Ben I also have a scenario that in my case I have to take periodical backup of Hbase data. For that I have will be using export/import tool. I have decided to take backup based on time range interval. I have read it in some other posts also that it is not good idea for one to use timestamp field of Hbase. Till now my POC works fine at my end but I want to know that if I put same scenario in production will there be any issues to worry ?
Please Suggest. -----Original Message----- From: [email protected] [mailto:[email protected]] On Behalf Of Stack Sent: Thursday, October 20, 2011 3:54 AM To: [email protected]; Ben West Subject: Re: Custom timestamps On Wed, Oct 19, 2011 at 12:18 PM, Ben West <[email protected]> wrote: > We're storing timestamped data in HBase; from lurking on the mailing list it > seems like the recommendation is usually to make the timestamp part of the > row key. I'm curious why this is - is scanning over rows more efficient than > scanning over timestamps within a cell? > I'd be surprised if a noticeable difference. It depends on how you are to access the data. In the tsdb case for instance, it wants to get all metrics within a particular time range. If the timestamp it used were that of the hbase system, then you'd have to do a full table scan each time to find metrics that had been fired during a particular time period -- i.e. you'd check each row and see if any entries on the row for the time period you are interested in -- whereas if the timestamp part of the row key, you instead just have to start scanning at the opening of the time range you are querying about. > The book says: "the version timestamp is internally by HBase for things like > time-to-live calculations. It's usually best to avoid setting this timestamp > yourself. Prefer using a separate timestamp attribute of the row, or have the > timestamp a part of the rowkey, or both." I understand that TTL would be > ruined (or saved, depending on your goal) by custom timestamps, and I also > gather that the way HBase handles concurrency is through MVCC. But we are > using application level locks, and HBase's TTL functionality applying is a > bonus if anything. > The books advice errs on the side of being conservative I'd say. The MVCC that we do internally does not use the cell timestamp but instead a different running sequence number that is associated (internally) with cells (I've not heard of an application atop hbase using the hbase timestamps to do MVCC at the application level). The locks you talk of, are these the locks provided in hbase HTable API? If so, are you aware they are dangerous (see back in this mailing list for explaination)? > So is there any reason why we shouldn't set the timestamps manually? > Generally, hbase works fine with user set timestamps; there can be issues ordering edits if clients have divergent clocks and the version being set is time-based but I'm probably not telling you something you don't already know. St.Ack ::DISCLAIMER:: ----------------------------------------------------------------------------------------------------------------------- The contents of this e-mail and any attachment(s) are confidential and intended for the named recipient(s) only. It shall not attach any liability on the originator or HCL or its affiliates. Any views or opinions presented in this email are solely those of the author and may not necessarily reflect the opinions of HCL or its affiliates. Any form of reproduction, dissemination, copying, disclosure, modification, distribution and / or publication of this message without the prior written consent of the author of this e-mail is strictly prohibited. If you have received this email in error please delete it and notify the sender immediately. Before opening any mail and attachments please check them for viruses and defect. -----------------------------------------------------------------------------------------------------------------------
