You're generally on the right track. In many cases, rather than using
secondary indexes in the relational world, you would have multiple
tables in HBase with different keys.
You may not need a table for each query, but that depends on your
requirements of performance and the specific details of the data
patterns (how sparse or dense certain things will be).
I would start with a User table and a WebPage table, keyed by their ids.
The User table could have a Visited family. The WebPage table could
have a VisitedBy family.
Your queries could be run like this:
1) Get(table=User, row=userid, family=Visited, qualifier=WebPageID)
There are a couple different ways you could model the data here.
You could either put in a new version of the same qualifier for each
visit, or you could make the qualifier a composite key like
WebPageID+VisitStamp, so they would then be grouped together.
2) Get(table=User, row=userid, family=Visited)
All qualifiers would represent all pages visited.
3) Get(table=WebPage, row=pageid, family=VisitedBy)
All qualifiers would represent all users who visited. You could
store multiple visits by the same user in different ways, as above.
As for using hive to run these queries, that is not something I would
recommend. For one, hive integration with hbase is not complete (as far
as I know). Second, hive's emphasis is on batch/offline mapreduce jobs.
Running the above 3 queries can be done with the HBase API directly,
and efficiently. There's no need for SQL or anything like it.
Hope that helps.
JG
Something Something wrote:
Hello,
Trying to figure out what's the recommended way of designing tables under
HBase. Let's say I need a table to gather statistics regarding user's visits
to different web pages.
In the relational database world, we could have a table with following columns:
Primary Key (system generated)
UserId (foreign key)
WebPageId (foreign key)
VisitedDateTime
& so on....
Basically, this table would allow us to answer (amongst many others) the
following questions...
1) How many times a User visited a certain Page?
2) Which web pages did a particular user visit?
3) Which users visited a particular web page? etc etc.
What's the best way to model this in HTable?
Since every HTable is really a distributed hashmap, does that mean I need to
create 3 different HTables (HashMaps) to answer these 3 questions?
1) One table with (UserId + WebPageId) as the compound key? (To answer #1)
2) One table with UserId as the key? (To answer #2)
3) One table with WebPageId as the key? (To answer #3)
Along with HTable should I use Hive to run queries such as #1 above?
Any help in this regard will be greatly appreciated. Thanks.
