Re: Regarding adding one more factor in lucene scoring formula

2006-12-05 Thread Sajid Khan
Hi Grant Thanks for the instant reply. Actually, there are two parts of the problem I'm trying to solve: 1) I want to show the results which contain the exact query phrase on top followed by results containing all the terms in the query but the terms are not in the same sequence as the q

Re: Efficiently expunging deletions of recently added documents

2006-12-05 Thread Ning Li
I'd like to open up the API to mergeSegments() in IndexWriter and am wondering if there are potential problems with this. I'm worried that opening up mergeSegments() could easily break the invariants currently guaranteed by the new merge policy(http://issues.apache.org/jira/browse/LUCENE-672).

Re: Attached proposed modifications to Lucene 2.0 to support Field.Store.Encrypted

2006-12-05 Thread Doug Cutting
robert engels wrote: I would counter that it is not a compelling feature for MOST users of Lucene, but it can still be implemented externally using binary fields for those that require it, and or even easier (and maybe even faster) using a encrypted filesystem with proper security. +1 Some u

Re: Efficiently expunging deletions of recently added documents

2006-12-05 Thread Ning Li
An old issue (http://issues.apache.org/jira/browse/LUCENE-325 new method expungeDeleted() added to IndexWriter) requested a similar functionality as described in the latter half of your email. The patch for that issue breaks the invariants of the new merge policy. An algorithm similar to that of

Re: Efficiently expunging deletions of recently added documents

2006-12-05 Thread Chuck Williams
Thanks Ning. This is all very helpful. I'll make sure to be consistent with the new merge policy and its invariant conditions. Chuck Ning Li wrote on 12/05/2006 08:01 AM: > An old issue (http://issues.apache.org/jira/browse/LUCENE-325 new > method expungeDeleted() added to IndexWriter) request

Re: Attached proposed modifications to Lucene 2.0 to support Field.Store.Encrypted

2006-12-05 Thread negrinv
Chris Hostetter wrote: > > > Compression of stored fields is a feature that the Lucene "core" currently > supports out of the box -- but it does so in a very limited maner that > doesn't allow for much configuration. There is no advantage for users in > using compressed fields over compressing

Re: Attached proposed modifications to Lucene 2.0 to support Field.Store.Encrypted

2006-12-05 Thread Mike Klaas
On 12/5/06, negrinv <[EMAIL PROTECTED]> wrote: Chris Hostetter wrote: > If the code was not already in the core, and someone asked about adding it > I would argue against doing so on the grounds that some helpfull utility > methods (possibly in a contrib) would be just as usefull, and would h

Re: Attached proposed modifications to Lucene 2.0 to support Field.Store.Encrypted

2006-12-05 Thread robert engels
If it is only meant to protect from "prying eyes" a simple field level analyzer that does a simple xor/rotation should suffice. It will be much faster and simpler. Going beyond that, your solution is not very secure as has been pointed out, so you might as well just uses the simplest soluti

Re: Attached proposed modifications to Lucene 2.0 to support Field.Store.Encrypted

2006-12-05 Thread negrinv
Doug Cutting wrote: > > > Some utilities for encrypting and decrypting binary fields might make a > useful contrib module, but I see no compelling reason to add this to the > core at this point. If such a contrib module becomes widely used, and > it becomes clear that it would work better i

Re: Attached proposed modifications to Lucene 2.0 to support Field.Store.Encrypted

2006-12-05 Thread Doug Cutting
Doug Cutting wrote: So, Victor, do you think this functionality could be reasonably packaged as an add-on package to Lucene? Doug, for an answer to most of your questions could you please refer to my answer to Chris Hostetter [ ... ] Let me be more direct. Encryption of Lucene fields may be

Re: Attached proposed modifications to Lucene 2.0 to support Field.Store.Encrypted

2006-12-05 Thread Chuck Williams
Mike Klaas wrote on 12/05/2006 11:38 AM: > On 12/5/06, negrinv <[EMAIL PROTECTED]> wrote: > >> Chris Hostetter wrote: > >> > If the code was not already in the core, and someone asked about >> adding it >> > I would argue against doing so on the grounds that some helpfull >> utility >> > methods

Re: Attached proposed modifications to Lucene 2.0 to support Field.Store.Encrypted

2006-12-05 Thread negrinv
there is absolutely no suggestion to make any changes to the index format. the index format would not change, whether you use encryption or not. Chuck Williams-2 wrote: > > > > Mike Klaas wrote on 12/05/2006 11:38 AM: >> On 12/5/06, negrinv <[EMAIL PROTECTED]> wrote: >> >>> Chris Hostetter wr

Re: Attached proposed modifications to Lucene 2.0 to support Field.Store.Encrypted

2006-12-05 Thread negrinv
there is a third way Doug, and it's for me to stop trying to be polite by answering all the questions that I am being asked, then nobody will get upset by my replies. If the decision is for no encryption at field level, I accept it, but I don't believe it should be externalised. Perhaps someone e

Re: Attached proposed modifications to Lucene 2.0 to support Field.Store.Encrypted

2006-12-05 Thread Michael Busch
negrinv wrote: there is a third way Doug, and it's for me to stop trying to be polite by answering all the questions that I am being asked, then nobody will get upset by my replies. If the decision is for no encryption at field level, I accept it, but I don't believe it should be externalised. P

Re: Attached proposed modifications to Lucene 2.0 to support Field.Store.Encrypted

2006-12-05 Thread Doug Cutting
negrinv wrote: there is a third way Doug, and it's for me to stop trying to be polite by answering all the questions that I am being asked, then nobody will get upset by my replies. If the decision is for no encryption at field level, I accept it, but I don't believe it should be externalised. P