[jira] Closed: (LUCENE-683) Lazy Field Loading has edge case bug causing read past EOF

[ http://issues.apache.org/jira/browse/LUCENE-683?page=all ] Yonik Seeley closed LUCENE-683. --- Fix Version/s: 2.0.1 Resolution: Fixed Assignee: Yonik Seeley fix and modified test case committed. > Lazy Field Loading has edge case

[jira] Commented: (LUCENE-683) Lazy Field Loading has edge case bug causing read past EOF

! Essentially that means that the offset and size were read from a random segment, not necessarily the segment the field value was stored in. > Lazy Field Loading has edge case bug causing read past EOF > -- > >

[jira] Commented: (LUCENE-683) Lazy Field Loading has edge case bug causing read past EOF

even guaranteed to get back the same data you put in! I'll try and look into it further. > Lazy Field Loading has edge case bug causing read past EOF > -- > > Key: LUCENE-683 > URL: http

[jira] Updated: (LUCENE-683) Lazy Field Loading has edge case bug causing read past EOF

[ http://issues.apache.org/jira/browse/LUCENE-683?page=all ] Hoss Man updated LUCENE-683: Attachment: TestLazyBug.java > Lazy Field Loading has edge case bug causing read past

[jira] Created: (LUCENE-683) Lazy Field Loading has edge case bug causing read past EOF

Lazy Field Loading has edge case bug causing read past EOF -- Key: LUCENE-683 URL: http://issues.apache.org/jira/browse/LUCENE-683 Project: Lucene - Java Issue Type: Bug

Re: [jira] Commented: (LUCENE-609) Lazy field loading breaks backward compat

, there was a lot of discussion on how to do this, so that may help inform this discussion (search for subject "Lazy Field Loading") As for getField versus getFieldable, I think getField should be deprecated and a note put on it not to use it if you are using the new FieldSelector me

[jira] Commented: (LUCENE-609) Lazy field loading breaks backward compat

of thing. I don't recall if it did breack backward compatibility, but even so... how many people write their own Searchers/IndexReaders vs how many people call Document.getField()? > Lazy field loading breaks backward compat > - > >

[jira] Commented: (LUCENE-609) Lazy field loading breaks backward compat

abstract class and ParallelWriter is a different implementation. I think it is best to introduce all the abstract classes now for fundamental types that have multiple implementations. Chuck > Lazy field loading breaks backward compat > - &g

RE: [jira] Commented: (LUCENE-609) Lazy field loading breaks backward compat

: I was assuming it would be changed so that LazyField extended Field. in which case we wouldn't really need a LazyDocument class .. Document could be used (and could contain LazyField's directly) I don't know why LazyField doesn't subclass Field ... but i assume there's a good reason (becuaes i

RE: [jira] Commented: (LUCENE-609) Lazy field loading breaks backward compat

, 2006 3:45 PM To: java-dev@lucene.apache.org Cc: [EMAIL PROTECTED] Subject: Re: [jira] Commented: (LUCENE-609) Lazy field loading breaks backward compat On 6/21/06, Chris Hostetter <[EMAIL PROTECTED]> wrote: > LazyField doesn't extend Field. Yeah, that's the crux of the matter. I

RE: [jira] Commented: (LUCENE-609) Lazy field loading breaks backward compat

Hostetter [mailto:[EMAIL PROTECTED] Sent: Wednesday, June 21, 2006 3:32 PM To: java-dev@lucene.apache.org; [EMAIL PROTECTED] Subject: RE: [jira] Commented: (LUCENE-609) Lazy field loading breaks backward compat : Would it not be easier to just create a LazyDocument that extends Document, : and handles

Re: [jira] Commented: (LUCENE-609) Lazy field loading breaks backward compat

On 6/21/06, Chris Hostetter <[EMAIL PROTECTED]> wrote: LazyField doesn't extend Field. Yeah, that's the crux of the matter. I assume this is the case because people need to instantiate Field to add it to a Document at index time, and I guess there is some FIeld baggage that LazyField didn't wa

RE: [jira] Commented: (LUCENE-609) Lazy field loading breaks backward compat

ield doesn't extend Field. : -Original Message- : From: Yonik Seeley (JIRA) [mailto:[EMAIL PROTECTED] : Sent: Wednesday, June 21, 2006 2:29 PM : To: java-dev@lucene.apache.org : Subject: [jira] Commented: (LUCENE-609) Lazy field loading breaks backward : compat : : [ : http://issues

RE: [jira] Commented: (LUCENE-609) Lazy field loading breaks backward compat

al Message- From: Grant Ingersoll [mailto:[EMAIL PROTECTED] Sent: Wednesday, June 21, 2006 2:58 PM To: java-dev@lucene.apache.org Subject: Re: [jira] Commented: (LUCENE-609) Lazy field loading breaks backward compat I am not sure I see why it would require no changes? You would have to have

Re: [jira] Commented: (LUCENE-609) Lazy field loading breaks backward compat

lazy/selective loading is enabled). -Original Message- From: Yonik Seeley (JIRA) [mailto:[EMAIL PROTECTED] Sent: Wednesday, June 21, 2006 2:29 PM To: java-dev@lucene.apache.org Subject: [jira] Commented: (LUCENE-609) Lazy field loading breaks backward compat [ http://issues.apache.org

[jira] Resolved: (LUCENE-609) Lazy field loading breaks backward compat

) Other committers should feel free to change/remove the getFieldable methods... they are still new enough to be considered temporary IMO. > Lazy field loading breaks backward compat > - > > Key: LUCENE-609 > URL: http://is

RE: [jira] Commented: (LUCENE-609) Lazy field loading breaks backward compat

loading is enabled). -Original Message- From: Yonik Seeley (JIRA) [mailto:[EMAIL PROTECTED] Sent: Wednesday, June 21, 2006 2:29 PM To: java-dev@lucene.apache.org Subject: [jira] Commented: (LUCENE-609) Lazy field loading breaks backward compat [ http://issues.apache.org/jira/browse/LUCENE

[jira] Commented: (LUCENE-609) Lazy field loading breaks backward compat

[ http://issues.apache.org/jira/browse/LUCENE-609?page=comments#action_12417166 ] Grant Ingersoll commented on LUCENE-609: I think there is a typo of getFielables on Document.java, other than that looks good. > Lazy field loading breaks backw

[jira] Commented: (LUCENE-609) Lazy field loading breaks backward compat

[ http://issues.apache.org/jira/browse/LUCENE-609?page=comments#action_12417164 ] Yonik Seeley commented on LUCENE-609: - I just verified that Solr now compiles/works correctly again with this patch. Any objections to committing it? > Lazy fi

[jira] Updated: (LUCENE-609) Lazy field loading breaks backward compat

n be considered temporary until we decide if that's really the best way to provide access to the new functionallity. - At least some of the Fieldable instances in all of the unit tests should be reverted back to Field so we get some test coverage (that is what everyone still uses). > La

Re: [jira] Commented: (LUCENE-609) Lazy field loading breaks backward compat

s for Fieldable. Lazy field loading breaks backward compat - Key: LUCENE-609 URL: http://issues.apache.org/jira/browse/LUCENE-609 Project: Lucene - Java Type: Bug Components: Other Versions: 2.0.1 Reporter:

[jira] Commented: (LUCENE-609) Lazy field loading breaks backward compat

[ http://issues.apache.org/jira/browse/LUCENE-609?page=comments#action_12417158 ] Yonik Seeley commented on LUCENE-609: - I'll work up a quick patch to revert those APIs and add new ones for Fieldable. > Lazy field loading breaks backwar

[jira] Created: (LUCENE-609) Lazy field loading breaks backward compat

Lazy field loading breaks backward compat - Key: LUCENE-609 URL: http://issues.apache.org/jira/browse/LUCENE-609 Project: Lucene - Java Type: Bug Components: Other Versions: 2.0.1 Reporter: Yonik Seeley

[jira] Resolved: (LUCENE-545) Field Selection and Lazy Field Loading

[ http://issues.apache.org/jira/browse/LUCENE-545?page=all ] Grant Ingersoll resolved LUCENE-545: Resolution: Fixed Thanks, Chuck, for the assistance! > Field Selection and Lazy Field Load

[jira] Commented: (LUCENE-545) Field Selection and Lazy Field Loading

[ http://issues.apache.org/jira/browse/LUCENE-545?page=comments#action_12415321 ] Grant Ingersoll commented on LUCENE-545: OK, I will take care of it, thanks! > Field Selection and Lazy Field Load

[jira] Commented: (LUCENE-545) Field Selection and Lazy Field Loading

Iterator it = ((Collection)readerToFields.get(reader)).iterator(); Sorry about these issues... FYI, I've been using the capability extensively and it is working well. Thanks for lazy fields! Chuck > Field Selection and Lazy Field Loading > -- &g

Re: [jira] Commented: (LUCENE-545) Field Selection and Lazy Field Loading

ments#action_12415240 ] Grant Ingersoll commented on LUCENE-545: Chuck, I think the patch is missing your ListFieldSelector.java file. Can you please attach it? Thanks, Grant Field Selection and Lazy Field Lo

[jira] Commented: (LUCENE-545) Field Selection and Lazy Field Loading

eld Selection and Lazy Field Loading > -- > > Key: LUCENE-545 > URL: http://issues.apache.org/jira/browse/LUCENE-545 > Project: Lucene - Java > Type: New Feature > Components: Store > Versions: 2.0.0 >

[jira] Assigned: (LUCENE-545) Field Selection and Lazy Field Loading

[ http://issues.apache.org/jira/browse/LUCENE-545?page=all ] Grant Ingersoll reassigned LUCENE-545: -- Assign To: Grant Ingersoll > Field Selection and Lazy Field Loading > -- > > Key

[jira] Updated: (LUCENE-545) Field Selection and Lazy Field Loading

. The version included here is the original version unchanged. > Field Selection and Lazy Field Loading > -- > > Key: LUCENE-545 > URL: http://issues.apache.org/jira/browse/LUCENE-545 > Project: Lucene - Java >

[jira] Updated: (LUCENE-545) Field Selection and Lazy Field Loading

[ http://issues.apache.org/jira/browse/LUCENE-545?page=all ] Grant Ingersoll updated LUCENE-545: --- Attachment: newFiles.tar.gz Forgot the new files. > Field Selection and Lazy Field Loading > -- > >

[jira] Updated: (LUCENE-545) Field Selection and Lazy Field Loading

as well. As always, let me know if I can help with anything. > Field Selection and Lazy Field Loading > -- > > Key: LUCENE-545 > URL: http://issues.apache.org/jira/browse/LUCENE-545 > Project: Lucene - Java >

[jira] Created: (LUCENE-545) Field Selection and Lazy Field Loading

Field Selection and Lazy Field Loading -- Key: LUCENE-545 URL: http://issues.apache.org/jira/browse/LUCENE-545 Project: Lucene - Java Type: New Feature Components: Store Versions: 2.0 Reporter: Grant Ingersoll

Re: Lazy Field Loading

On 4/4/06, Grant Ingersoll <[EMAIL PROTECTED]> wrote: > As I see it, we have a few goals: > 1. Retrieve only the fields someone wants > 2. Retrieve only all fields, but leave some to be lazily loaded > 3. Provide SQL like functionality (as Mark suggested) [a bit harder and > more involved] /*

Re: Lazy Field Loading

Yonik Seeley wrote: On 4/4/06, Grant Ingersoll <[EMAIL PROTECTED]> wrote: I am not sure you need 509 when you have Lazy loading. It would be nice to avoid creating a Field object at all... we have some crazy documents with more than 1000 fields :-) I think the Field object itself ta

Re: Lazy Field Loading

nd z more > >> useful than lazy loading. > >> > > > > Thinking a little more, it would be nice if the field reading API was > > opened up a little more so that multiple things could be done... even > > construct different field/document objects (say a documen

Re: Lazy Field Loading

ple things could be done... even construct different field/document objects (say a document implementation that indexed the fields, etc). That could be used to implement either lazy field loading, or loading of specific fields. The lazy loading alone doesn't really address LUCENE-509 I w

Re: Lazy Field Loading

ice if the field reading API was opened up a little more so that multiple things could be done... even construct different field/document objects (say a document implementation that indexed the fields, etc). That could be used to implement either lazy field loading, or loading of specific fields. The l

Re: Lazy Field Loading

OK, #4 it is. I will most likely submit a patch this weekend. Andrzej Bialecki wrote: Yonik Seeley wrote: On 3/31/06, Erik Hatcher <[EMAIL PROTECTED]> wrote: I prefer option #4 myself. Also note that a similar issue with patches exists within JIRA:

Re: Lazy Field Loading

Yonik Seeley wrote: On 3/31/06, Erik Hatcher <[EMAIL PROTECTED]> wrote: I prefer option #4 myself. Also note that a similar issue with patches exists within JIRA: Yes, I'd personally find a way to retrieve just fields x,

Re: Lazy Field Loading

> I don't think option 3 is baked in at indexing time. Sorry, I misread it. Yes, that is another option. So if options 3 and 4 are about search-time selection (based on size and fieldname respectively) can they be generalized into a more wide-reaching retrieval API? You can imagine a high-level

Re: Lazy Field Loading

On 3/31/06, Erik Hatcher <[EMAIL PROTECTED]> wrote: > I prefer option #4 myself. Also note that a similar issue with > patches exists within JIRA: > > Yes, I'd personally find a way to retrieve just fields x,y, and z more useful than

Re: Lazy Field Loading

mark harwood wrote: Preferably this choice of field loading policy should NOT be "baked in" at index time because index access patterns can vary (ruling out options 1 and 3) I don't think option 3 is baked in at indexing time. I think it would look like: | IndexReader.document(int docN

Re: Lazy Field Loading

I prefer option #4 myself. Also note that a similar issue with patches exists within JIRA: Erik On Mar 31, 2006, at 7:21 AM, Grant Ingersoll wrote: OK, how about a vote on this. I see several ways of implementing the front e

Re: Lazy Field Loading

I'd prefer option 4. Users should expect to provide some form of guidance to the engine about how they are going to access the data if it is expected to be retrieved efficiently. Preferably this choice of field loading policy should NOT be "baked in" at index time because index access patterns ca

Re: Lazy Field Loading

OK, how about a vote on this. I see several ways of implementing the front end to this: 1. Declarative: On construction of a Document, you declare the Field to be Lazy. 2. Implicit: All fields are Lazy 3. Size of Field. Pass into IndexReader.document() the size of the field above which i

Re: Lazy Field Loading

Grant Ingersoll wrote: My main concern is the semantics of the fieldsStream and whether that would be closed behind the back of the LazyField implementation. My understanding is that as long as the IndexReader is open, this stream should also be open. Is that correct? What am I forgetting

Re: Lazy Field Loading

I like the way they handle fields. At query time you specify the fields you want returned from matching documents. Aaron -Original Message- From: Grant Ingersoll [mailto:[EMAIL PROTECTED] Sent: Wednesday, March 29, 2006 9:05 AM To: java-dev@lucene.apache.org Subject: Re: Lazy Field Loa

RE: Lazy Field Loading

06 9:05 AM To: java-dev@lucene.apache.org Subject: Re: Lazy Field Loading Hmmm, I guess I always thought of it as a property of the field that user's would want to explicitly control. I assumed that most fields would not be lazy and a few would be. Now that you have backed me up a bit on it (in

Re: Lazy Field Loading

Erik On Mar 29, 2006, at 8:31 AM, Grant Ingersoll wrote: I have a base implementation of lazy field loading that I am starting to test and wanted to run my approach by everyone to hear their thoughts. I have, as per Doug's suggestion from a while ago, created an interface nam

Re: Lazy Field Loading

rching and reading of an index rather than during indexing itself. Erik On Mar 29, 2006, at 8:31 AM, Grant Ingersoll wrote: I have a base implementation of lazy field loading that I am starting to test and wanted to run my approach by everyone to hear their thoughts. I have, as

Lazy Field Loading

I have a base implementation of lazy field loading that I am starting to test and wanted to run my approach by everyone to hear their thoughts. I have, as per Doug's suggestion from a while ago, created an interface named Fieldable that is implemented by Field and a new, private class,