Here is an example of what I was thinking... Your row key could be <doc id><position><word> You would want the doc id and position to be fixed length, so choose wisely based on the length (number of words) and number of documents. A single row key might look like:
"0000012345000650lemma" The 0000012345 would be the document ID, the 000650 is the position, and lemma being the word itself. With this set up, you could scan for all the words given a document ID, or a range of positions given a document ID. Scanning a range of positions (i.e. 0000012345000649 to 0000012345000651) would allow you to retrieve the surrounding words of a given document ID, and word position. Just a thought, hope this helps. Travis Hegner http://www.travishegner.com/ -----Original Message----- From: llpind [mailto:[email protected]] Sent: Monday, August 24, 2009 2:37 PM To: [email protected] Subject: RE: HBase data model question Good points. Word combination is what i was trying to say. Say i have a word (lemma), and need words before & after queryable by lemma. lets call this a sentence. So my rowkey will essentially be a sentence (with doc id). But i can have identical rowkeys still within a document. Maybe i'm missing something... hmm... Travis Hegner wrote: > > What about a document-id, word-position, and word combination. With the > proper combo all words in a single document would be located near > each-other. > > Travis Hegner > http://www.travishegner.com/ > > > -----Original Message----- > From: llpind [mailto:[email protected]] > Sent: Monday, August 24, 2009 1:37 PM > To: [email protected] > Subject: HBase data model question > > > Hey, > > I'm trying to move a relational model to HBase, and would like some input. > > Suppose i have constant stream of documents coming in, and I'd like to > parse > these by a single word. > > It makes sense to have this word as my rowkey, but I need a way to handle > duplicate word text. Kind of a dicitionary > > What is the best way to solve this in HBase? timestamp in row key? Since > I > need a way to identify each word uniquely > > > Thanks. > -- > View this message in context: > http://www.nabble.com/HBase-data-model-question-tp25120285p25120285.html > Sent from the HBase User mailing list archive at Nabble.com. > > > The information contained in this communication is confidential and is > intended only for the use of the named recipient. Unauthorized use, > disclosure, or copying is strictly prohibited and may be unlawful. If you > have received this communication in error, you should know that you are > bound to confidentiality, and should please immediately notify the sender > or our IT Department at 866.459.4599. > > -- View this message in context: http://www.nabble.com/HBase-data-model-question-tp25120285p25121358.html Sent from the HBase User mailing list archive at Nabble.com. The information contained in this communication is confidential and is intended only for the use of the named recipient. Unauthorized use, disclosure, or copying is strictly prohibited and may be unlawful. If you have received this communication in error, you should know that you are bound to confidentiality, and should please immediately notify the sender or our IT Department at 866.459.4599.
