On 23/07/11 09:23, Norman Maurer wrote:
> This will influence my design. I read somewhere that the maximum cell
> size in HBase is suggested to be around 10MB. We have two options:
> 1. split the message in more cells
> 2. store a reference to a message and the actual message directly in HDFS.
>
> They both present chalenges, but I think the first option cen be
> implemented faster.
I think 1 makes more sense atm. I guess we could make kind of pluggable
later..
Option 1 is indeed the most adapted (remember the issue on small files
stored in hdfs).
You could even start without worrying about splitting in chunks to begin.
There was a discussion on the hbase mailing list about this a month ago
(or so), and a generic solution could even be used in a short time frame.
If we don't have that generic solution, we can implement ours during the
"optimization" phase.
Sounds like we putting much in that "optimization" phase, but it's how
iterative development works with constant refactoring. You can also
think to "test driven development" and have the needed unit tests that
ensure that your refactoring does not break anything.
--
Eric Charles
http://about.echarles.net
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
