Hi Michael, Thanks a lot for your suggestions.
I was looking at rsync; as per this link (http://samba.anu.edu.au/rsync/features.html), rsync is a file transfer program for UNIX. Is there rsync support for Windows as well? I found few rsync programs that works for Windows, but I am not sure if they will server the purpose. Has anyone using rsync on Windows? Regards, Rajesh --- Michael McCandless <[EMAIL PROTECTED]> wrote: > > Rajesh parab wrote: > > Hi, > > > > We are currently using Lucene 2.0 for full-text > > searches within our enterprise application, which > can > > be deployed in clustered environment. We generate > > Lucene index for data stored inside relational > > database. > > > > As Lucene 2.0 did not have solid NFS support and > as we > > wanted Lucene based searches to work properly in > > Clustered environment, we had decided on following > > approach: > > 1. The index generation happens on a machine > (could be > > one of the cluster nodes or a separate machine) > and > > once the Lucene index is generated, we copy all > the > > index files to the database. > > Note that you can do also incremental replication: > often, the Lucene > index changes in minor ways (eg a single new segment > is flushed and a > new segments_N and segments.gen is written) so you > should only sync > the files that are new (and remove the ones that are > now gone). > Lucene's write-once approach makes this very simple > (you just have to > compare file names, not the contents of each file). > > It's also possible to replicate without using a DB. > EG rsync does a > great job. > > > 2. The index search request on each cluster node > > retrieves the index files from database (during > first > > search or after index update), copies to the file > > system and use it for searches. > > 3. Thus, each cluster node has its own copy of the > > index and it keeps on picking up latest version if > it > > is available inside database. > > > > This has worked fine for us till now, though we > will > > not be able continue with this model in future as > we > > want to support Lucene based searches across our > > application and also want to index large > components > > inside our application like Wiki, forums, etc. As > the > > index will grow, storing and retrieving index > files > > from database will not be an efficient operation. > > > > My questions are: > > - Will we be able to use NFS if we move to Lucene > > 2.3.0? > > Make sure you update to 2.3.1, not 2.3.0. > > > - Will there be any significant performance impact > on > > index generation and searches if we move to NFS? > > - Is Lucene + NFS combination supported for all > > operating systems? (We support Windows, Solaris, > AIX, > > HP-UX, Red Hat Linux) > > NFS *should* work, however: > > * It's not widely used, so, test thoroughly in > your particular setup. > > * Most likely to work is if you use a single > machine writing to > the index, and many readers. > > * Performance is likely not great, especially on > searching, but > you should test in your specific situation. > > > - Is there any other alternative available other > than > > NFS? > > > > It's also possible to replicate without using a DB. > EG rsync does > agreat job. > > You should look at Solr, since it already has all > the infrastructure > toaccept updates, replicate index changes to remote > machines, etc. > > > I will really appreciate your comments/thoughts on > > this topic. > > > > Regards, > > Rajesh > > > > > > > > > ______________________________________________________________________ > > > ______________ > > You rock. That's why Blockbuster's offering you > one month of > > Blockbuster Total Access, No Cost. > > http://tc.deals.yahoo.com/tc/blockbuster/text5.com > > > > > --------------------------------------------------------------------- > > To unsubscribe, e-mail: > [EMAIL PROTECTED] > > For additional commands, e-mail: > [EMAIL PROTECTED] > > > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: > [EMAIL PROTECTED] > For additional commands, e-mail: > [EMAIL PROTECTED] > > ____________________________________________________________________________________ You rock. That's why Blockbuster's offering you one month of Blockbuster Total Access, No Cost. http://tc.deals.yahoo.com/tc/blockbuster/text5.com --------------------------------------------------------------------- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
