On 12.06.2012 11:30, Eric Charles wrote:
Hi Mihai,
Glad to hear your exams are over (I hope they went fine) :)
Hi Eric,
Thanks, they went very well, I got high marks.
As Ioan said, Avro serialization HBase will be deprecated in favor of
Protobuf (if I understand well...).
I think Avro could be changed rather easily with Protobuf as they're
both doing basically the same thing, only that Avro uses JSON schemas
and can be used with any other language, which is of no of value to the
project.
I also like Avro because it gives you serialization & storage format
in one box, but is this what we want? The key point here is more an
effective access to the persisted data.
If the data is passed through Avro we'll have it serialized and
deserialization is basically handled by Avro, but we'll always have to
interact with the schemas. In Protobuf we have the objects compiled into
our classes, from what i gather it's mostly usefull for RPC, but Avro
also has the protocol in which by using the avro-maven-plugin you can
generate you own classes with which to interact. I can't say I'm an
expert in either but I fancy Avro.
There has been a few tentatives so far to marry HBase and Lucene (see
[1], [2], [3] and [4] for example, see also [5] for a more recent
article).
Thank you for the github links, i will look thouroughly through the
projects. I was already aware of Basene and Solandra(former Lucandra),
they have simillar aproaches.
The questions I am wondering:
1. Will you focus on a 'generic' solution (reusable outside James), or
on a very specific one tuned/optimized only for James mailbox needs?
I was thinking of writing generic code so that maybe it could be used
outside of James but the data format would be specific to James mailbox
needs, so the answer in the end is that it will be tuned for James.
2. What strategy will you take (custom Directory or custom
IndexReader/Writer, usage of Coprocessor or not...)?
I was thinking that a custom Directory was the way to go, but I soon
realized that it's not as simple as it sounds and overriding the higher
level classes of IndexReader and IndexWriter would be more
appropriate.(as in article [5]) So by bypassing the Directory I would
have to make use of Hbase Coprocessors. As far as I can think of it, a
RegionObserver would be employed to gather frequently performed on data
for the Lucene queries and Endpoints.
[1] https://github.com/akkumar/hbasene
[2] https://github.com/thkoch2001/lucehbase
[3] https://github.com/jasonrutherglen/HBASE-SEARCH
[4] https://github.com/jasonrutherglen/LUCENE-FOR-HBASE
[5] http://www.infoq.com/articles/LuceneHbase
---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org
For additional commands, e-mail: server-dev-h...@james.apache.org
