RE: Save to database...

2006-01-05 Thread Steven Pannell
Look in the old archive mails and you will find a few people have tried this out. There is even some code around. I have tried this, and to be honest it does not make much sense. The real problem is performance it just takes too long to keep getting the index from the database for performing the

RE: Save to database...

2006-01-05 Thread Aditya Liviandi
Just want to play around with it... Is there anyone who could help me better understand the file structure of the indexes though? I read the fileformats.html at the webpage, but am still a little fuzzy about what goes where... Thankyou -Original Message- From: Steven Pannell

RE: Save to database...

2006-01-05 Thread Robert Engels
There are impl in the contrib that do not need to retrieve the entire index from the db in order to query (there store blocks of files in the db, instead of blocks on disk). I also developed an implementation that did not use blocks but rather a custom index persistence mechanism. There can be

RE: Save to database...

2006-01-05 Thread Aditya Liviandi
What I meant is instead of saving the indexes into files, could I save them as tables in a database? I would think there would be a FieldNames table, a TermDictionary table, a TermFrequency table and a TermPosition table (at the least)... Has this been done before? -Original Message-

RE: Save to database...

2006-01-05 Thread Robert Engels
Yes, that is what I did in the custom persistence. There are some not so trivial problems to solve though. Normally you cannot seek with BLOBs, (a lot of JDBC/db impl will read the entire BLOB in all cases) so efficiently reading the postings can be difficult, although you can store the postings

[jira] Resolved: (LUCENE-482) JE Directory Implementation

2006-01-05 Thread Andi Vajda (JIRA)
[ http://issues.apache.org/jira/browse/LUCENE-482?page=all ] Andi Vajda resolved LUCENE-482: --- Fix Version: 1.9 Resolution: Fixed Assign To: Andi Vajda fixed in rev 366041, 'db' contrib area structure was rearranged to accomodate multiple

RE: Save to database...

2006-01-05 Thread Andi Vajda
On Thu, 5 Jan 2006, Aditya Liviandi wrote: Is there anyone who could help me better understand the file structure of the indexes though? The Berkeley DB implementations in the 'db' contrib area write out blocks of index data as fed by the Lucene Directory classes. It has no understanding of

RE: Save to database...

2006-01-05 Thread Andi Vajda
On Thu, 5 Jan 2006, Robert Engels wrote: 4. transactional updates to the index are possible (although index writes are supposedly transactional in std lucene, I have encountered some index corruption with hard failures - I think it is because the files are not synced when flushed/closed).

RE: Save to database...

2006-01-05 Thread Andi Vajda
On Thu, 5 Jan 2006, Robert Engels wrote: Yes, that is what I did in the custom persistence. There are some not so trivial problems to solve though. Normally you cannot seek with BLOBs, (a lot of JDBC/db impl will read the entire BLOB in all cases) so efficiently reading the postings can be

RE: Save to database...

2006-01-05 Thread Robert Engels
Well, I think they are supposed to be - that is the reason the segments file is written last. If the segments file is not updated the only problem becomes orphaned files, but the index SHOULD still be consistent. The Lucene index format is quite ingenious in the simplicity in how it does this.

RE: Save to database...

2006-01-05 Thread Andi Vajda
On Thu, 5 Jan 2006, Robert Engels wrote: Well, I think they are supposed to be - that is the reason the segments file is written last. If the segments file is not updated the only problem becomes orphaned files, but the index SHOULD still be consistent. The Lucene index format is quite

[jira] Commented: (LUCENE-483) QueryParser.getFieldQuery(String,String) doesn't set default slop on MultiPhraseQuery

2006-01-05 Thread Yonik Seeley (JIRA)
[ http://issues.apache.org/jira/browse/LUCENE-483?page=comments#action_12361888 ] Yonik Seeley commented on LUCENE-483: - Hmmm, it seems like the problem is that the Query isn't fully formed after a call to getFieldQuery(String,String).

[jira] Commented: (LUCENE-483) QueryParser.getFieldQuery(String,String) doesn't set default slop on MultiPhraseQuery

2006-01-05 Thread Hoss Man (JIRA)
[ http://issues.apache.org/jira/browse/LUCENE-483?page=comments#action_12361890 ] Hoss Man commented on LUCENE-483: - Really, it seems like the method constructing the query should have access to any specified slop, and should also be able to