I would recommend that you search the list for some great discussions on NDFS. Doug has a nice writeup of his vision of using a map reduce job to push the indexes to your query servers so they're updates as the webdb is and managed that way.
NDFS just wasn't designed for the I/O of a query. You want to have queries either cached in memory or on fast disk drives or else the latency of reading over a network would slow things down terribly especially if you have concurrent queries. -byron --- Gal Nitzan <[EMAIL PROTECTED]> wrote: > So just use the ndfs command to download the > relevant files from NDFS > and put them on the search server and from there to > follow the sample on > your documentation project? > > Thanks for all the help. > > P.S. Do you have a clear view for the solution to > the "slowness in > search over NDFS"? if so I would be interested in > giving a big hand on > that. Since it is a crucial part for my company... > > Regards, > > Gal > > Stefan Groschupf wrote: > > There will be a solution soon, if I found some > more time, until this > > for smaller installation you need a shell script > that download the > > index and segment to the box that runs the search > server. > > You also can move the index from ndfs to local > instead of copy it. > > check: "bin/nutch ndfs" for documentation. > > > > Am 28.12.2005 um 14:34 schrieb Gal Nitzan: > > > >> Whoa, that was fast... > >> > >> So all in all you would need two sets of the same > data? > >> > >> Did I understand there is an effort to improve > the "poor performance" > >> issue? > >> > >> And if we are at it, would you care to explain > how to download the > >> index to local and what happens if the data is > growing over the > >> boundaries of one machine? do you just add HD to > the machine? > >> > >> Thanks, > >> > >> Gal > >> > >> Stefan Groschupf wrote: > >>> Download index to a local file system. > >>> > >>> Am 28.12.2005 um 14:25 schrieb Gal Nitzan: > >>> > >>>> Hi, > >>>> > >>>> If using search over NDFS is too slow than what > is the alternative > >>>> when all your data is in NDFS? > >>>> > >>>> Thanks, Gal > >>>> > >>>> > >>> > >>> > >>> . > >>> > >> > >> > >> > > > > > > . > > > > >
