Hi, I know there are several options, though none of which I have tried myself.
Rsync win32 port: http://sourceforge.net/projects/rsyncwin32/ rsync python script (should be platform-independent): http://www.vdesmedt.com/~vds2212/rsync.html Jarsync (in java): http://jarsync.sourceforge.net/, which is a Java implementation for rsync And finally cygwin of course. Hope that helps. On Thu, Apr 10, 2008 at 9:10 AM, Rajesh parab <[EMAIL PROTECTED]> wrote: > Hi All, > > Has anyone used rsync or similar utilities on Windows > OS to replicate Lucene index across multiple machines? > > Any pointers on it will be very useful? > > Regards, > Rajesh > > --- Rajesh parab <[EMAIL PROTECTED]> wrote: > > > Hi Michael, > > > > Thanks a lot for your suggestions. > > > > I was looking at rsync; as per this link > > (http://samba.anu.edu.au/rsync/features.html), rsync > > is a file transfer program for UNIX. Is there rsync > > support for Windows as well? I found few rsync > > programs that works for Windows, but I am not sure > > if > > they will server the purpose. Has anyone using rsync > > on Windows? > > > > Regards, > > Rajesh > > > > --- Michael McCandless <[EMAIL PROTECTED]> > > wrote: > > > > > > > > Rajesh parab wrote: > > > > Hi, > > > > > > > > We are currently using Lucene 2.0 for full-text > > > > searches within our enterprise application, > > which > > > can > > > > be deployed in clustered environment. We > > generate > > > > Lucene index for data stored inside relational > > > > database. > > > > > > > > As Lucene 2.0 did not have solid NFS support and > > > as we > > > > wanted Lucene based searches to work properly in > > > > Clustered environment, we had decided on > > following > > > > approach: > > > > 1. The index generation happens on a machine > > > (could be > > > > one of the cluster nodes or a separate machine) > > > and > > > > once the Lucene index is generated, we copy all > > > the > > > > index files to the database. > > > > > > Note that you can do also incremental replication: > > > often, the Lucene > > > index changes in minor ways (eg a single new > > segment > > > is flushed and a > > > new segments_N and segments.gen is written) so you > > > should only sync > > > the files that are new (and remove the ones that > > are > > > now gone). > > > Lucene's write-once approach makes this very > > simple > > > (you just have to > > > compare file names, not the contents of each > > file). > > > > > > It's also possible to replicate without using a > > DB. > > > EG rsync does a > > > great job. > > > > > > > 2. The index search request on each cluster node > > > > retrieves the index files from database (during > > > first > > > > search or after index update), copies to the > > file > > > > system and use it for searches. > > > > 3. Thus, each cluster node has its own copy of > > the > > > > index and it keeps on picking up latest version > > if > > > it > > > > is available inside database. > > > > > > > > This has worked fine for us till now, though we > > > will > > > > not be able continue with this model in future > > as > > > we > > > > want to support Lucene based searches across our > > > > application and also want to index large > > > components > > > > inside our application like Wiki, forums, etc. > > As > > > the > > > > index will grow, storing and retrieving index > > > files > > > > from database will not be an efficient > > operation. > > > > > > > > My questions are: > > > > - Will we be able to use NFS if we move to > > Lucene > > > > 2.3.0? > > > > > > Make sure you update to 2.3.1, not 2.3.0. > > > > > > > - Will there be any significant performance > > impact > > > on > > > > index generation and searches if we move to NFS? > > > > - Is Lucene + NFS combination supported for all > > > > operating systems? (We support Windows, Solaris, > > > AIX, > > > > HP-UX, Red Hat Linux) > > > > > > NFS *should* work, however: > > > > > > * It's not widely used, so, test thoroughly in > > > your particular setup. > > > > > > * Most likely to work is if you use a single > > > machine writing to > > > the index, and many readers. > > > > > > * Performance is likely not great, especially > > on > > > searching, but > > > you should test in your specific situation. > > > > > > > - Is there any other alternative available other > > > than > > > > NFS? > > > > > > > > > > It's also possible to replicate without using a > > DB. > > > EG rsync does > > > agreat job. > > > > > > You should look at Solr, since it already has all > > > the infrastructure > > > toaccept updates, replicate index changes to > > remote > > > machines, etc. > > > > > > > I will really appreciate your comments/thoughts > > on > > > > this topic. > > > > > > > > Regards, > > > > Rajesh > > > > > > > > > > > > > > > > > > > > > > ______________________________________________________________________ > > > > > > > ______________ > > > > You rock. That's why Blockbuster's offering you > > > one month of > > > > Blockbuster Total Access, No Cost. > > > > > > http://tc.deals.yahoo.com/tc/blockbuster/text5.com > > > > > > > > > > > > > > --------------------------------------------------------------------- > > > > To unsubscribe, e-mail: > > > [EMAIL PROTECTED] > > > > For additional commands, e-mail: > > > [EMAIL PROTECTED] > > > > > > > > > > > > > > > > --------------------------------------------------------------------- > > > To unsubscribe, e-mail: > > > [EMAIL PROTECTED] > > > For additional commands, e-mail: > > > [EMAIL PROTECTED] > > > > > > > > > > > > > > > > > > ____________________________________________________________________________________ > > You rock. That's why Blockbuster's offering you one > > month of Blockbuster Total Access, No Cost. > > http://tc.deals.yahoo.com/tc/blockbuster/text5.com > > > > > --------------------------------------------------------------------- > > To unsubscribe, e-mail: > > [EMAIL PROTECTED] > > For additional commands, e-mail: > > [EMAIL PROTECTED] > > > > > > > __________________________________________________ > Do You Yahoo!? > Tired of spam? Yahoo! Mail has the best spam protection around > http://mail.yahoo.com > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [EMAIL PROTECTED] > For additional commands, e-mail: [EMAIL PROTECTED] > > -- Yin Qiu Nanjing University, China -------------------------------------------
