Hello, Why do we not want to search using DFS? Why is it not proper for incremental indexing?
Thanks Trey Spiva-3 wrote: > > According to a hadoop tutorial (http://wiki.apache.org/nutch/ > NutchHadoopTutorial) on wiki, > > "you don't want to search using DFS, you want to search using local > filesystems. Once the index has been created on the DFS you can > use the hadoop copyToLocal command to move it to the local file > system as such" ... "Understand that at this point we are not using > the DFS or MapReduce to do the searching, all of it is on a local > machine". > > So my understanding is that hadoop is only good for batch index > building, and is not proper for incremental index building and > search. Is this true? > > The reason I am asking is that when I read the article ACM article by > Mike Cafarella and Doug Cutting, to me it sounded like the concern > was to make the index structures fit in the primary memory, not the > entire crawled database. Did I miss understand the ACM article? > -- View this message in context: http://www.nabble.com/Hadoop-distributed-search.-tp14155234p14265977.html Sent from the Nutch - User mailing list archive at Nabble.com.
