On Sun, May 26, 2013 at 9:21 AM, Nick Wellnhofer <[email protected]> wrote:
> I just pushed a new branch 'separate-clownfish-wip1' which goes a long way
> toward separating Clownfish from Lucy and supporting multiple parcels in
> general. Have a look at the commit log for more details.
Nick++
I think we should prepare to release 0.4.0 once the long-overdue separation of
Clownfish and Lucy into distinct shared objects is finally done. (And in the
future I'll try harder to avoid putting in-progress features onto master.)
There are a couple really nice commits on your branch which I'll reply to
separately.
> The biggest obstacle for now is that the serialization methods of
> Clownfish::Obj use the InStream and OutStream classes which are still part
> of the Lucy parcel.
I've known that this was on the horizon for a while and would block the
Clownfish/Lucy separation, so I've already been tinkering with a few different
approaches. I've come to believe that a stopgap would be best: do whatever it
takes to remove serialization from Clownfish and make it bare-minimum
functional in Lucy. That way we don't have to worry about what the Clownfish
i/o API should look like before the next release.
1. First, remove Serialize() and Deserialize() from Obj. Those methods will
then become novel on a number of Lucy classes: Query, Doc, DocVector,
TermVector, SortSpec, SortRule, MatchDoc, TopDocs, etc.
2. Make sure that Lucy::Util::Freezer's FREEZE and THAW continue to work on
all of those classes. We'll need a stupid chain of class checks like
`if (Obj_Is_A(obj, QUERY))` followed by casts and type-specific
invocations like `Query_Serialize((Query*)obj, outstream)` since we'll no
longer be able to invoke `Obj_Serialize(obj, outstream)`. We'll also need
to implement serialization routines for CharBuf, Hash, VArray, Num, etc.
inside Freezer.c.
3. Change the STORABLE_freeze and STORABLE_thaw routines to wrap
Freezer_freeze and Freezer_thaw.
4. Move the STORABLE_freeze and STORABLE_thaw XS routines from Clownfish::Obj
into Lucy::Util::Freezer, and then use Exporter to install them into all
the relevant Lucy Perl packages.
If I've reasoned this through correctly, we won't need to change any code in
LucyX::Remote -- it can continue to use Storable for object serialization.
Looking forward, I think we should move towards Apache Avro as a serialization
format for both RPC and index metadata -- but switching mechanisms is too
ambitious for the near term.
If there are no objections to this plan, I can work on it this week.
> Another thing I'd like to do is to move the Clownfish runtime and Lucy tests
> to separate parcels. Any suggestions for how to name the test parcels?
* clownfish::test nickname Test -- TestBatch, TestRunner, etc.
* test::clownfish nickname TestCfish -- TestCharBuf, TestHash, etc.
* test::org::apache::lucy nickname TestLucy -- TestAnalyzer, etc.
I expect that the first one, clownfish::test, will be a public API -- the
default test harness for Clownfish-powered projects. The other two are the
test suites for their respective parcels.
Marvin Humphrey
