Grant Ingersoll wrote:
On Aug 10, 2009, at 5:12 PM, Shai Erera wrote:
Maybe we should follow what I seem to read from Earwin and Grant -
come up w/ real use cases, try to implement them w/ the current API,
then if it's impossible, discuss how we can make the current API more
adaptive. If at the end of this we'll get back to the new API, then
we'll at least feel better about it, and more convinced it is the way
to go.
Well, I have real use cases for it, but all of it is still missing the
biggest piece: search side support. It's the 900 lb. elephant in the
room. The 500 lb. elephant is the fact that all these attributes,
AIUI, require you to hook in your own indexing chain, etc. in order to
even be indexed, which is all package private stuff. It's not even
clear to me what happens right now if you were to, say have a Token
Stream that, say, had only one Attribute on it and none of the
existing attributes (term buffer, length, position, etc.) Please
correct me if I am wrong, I still don't have a deep understanding of
it all.
Michael has always been up front that this new API is in preparation for
flexible indexing. It doesn't give us the goodness - he has laid out the
reasons for moving before the goodness comes more than once I think.
From my understanding, Michael looked at what Mike was doing in one of
his flexible indexing patches, wondered how some of the TokenStream
stuff was going to work well with it, and came up with this new API as a
solution. Yes - it gets us nothing now. But its a big move, and there is
no need to do everything at once - in fact it would probably be harder
to do it all at once - the rest has always been on the table. 3.0 has
always been convenient to push it before, as deprecations can than be
removed. Nothing forcing us to make that decision now though.
Honestly, though, it really gives you very little over the current,
well functioning payloads capability other than stronger typing, the
ability to pick only those attributes that you want indexed (in
theory) and a byte (or so) of savings per any token that has a
payload, and we _HAVE_ right now, search support for payloads.
Payloads gives us nothing as developers - you can't use that
functionality without taking it from the users - payloads are for users.
Flexible indexing will lead to all kinds of little cool things - the
likes of which have been discussed a lot in older emails. It will likely
lead to things we cannot predict as well. Everything will be more
flexible. It also could play a part in CSF, and work on allowing custom
files to plug into merging. Plus everything else thats been mentioned
(pfor, etc) I've been sold on the long term benefits. I don't think you
need these API for them, but its my understanding it helps solve part of
the equation.
A bunch of issues have come up. To my knowledge, they have been
addressed with vigor every time. If someone is unhappy with how
something has been addressed, and it needs to be addressed further,
please speak up. Otherwise, I don't think the sky is falling - I think
the new API is being shaken out.
Oh, and now it seems the new QP is dependent on it all.
Dependent how?
--
- Mark
http://www.lucidimagination.com
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: java-dev-h...@lucene.apache.org
