Hi Smitha, We crawled a website which has more than 1m page and 200k pdf file. We only indexed pdf files. It was pretty good. Only bad think about web connector is politeness (bandwidth). Actually its a problem of all web crawlers.
9 Eyl 2015 Çar, 07:25 tarihinde, Smitha S <[email protected]> şunu yazdı: > Hi All, > > > > Has anyone uses ManifoldCF as web crawler in any application which is in > production. > > > > We are using ManifoldCF to crawl window share repository for our > application and its been doing good so far. Now we got some requirement to > crawl some websites. Thinking of using WebCrawler functionality of MCF. > Would like to know the pros and cons of using MCF as web crawler. > > > > Please share your thoughts. > > > > Thanks & Regards, > > Smitha > > **************** CAUTION - Disclaimer ***************** > This e-mail contains PRIVILEGED AND CONFIDENTIAL INFORMATION intended solely > for the use of the addressee(s). If you are not the intended recipient, please > notify the sender by e-mail and delete the original message. Further, you are > not > to copy, disclose, or distribute this e-mail or its contents to any other > person and > any such actions are unlawful. This e-mail may contain viruses. Infosys has > taken > every reasonable precaution to minimize this risk, but is not liable for any > damage > you may sustain as a result of any virus in this e-mail. You should carry out > your > own virus checks before opening the e-mail or attachment. Infosys reserves the > right to monitor and review the content of all messages sent to or from this > e-mail > address. Messages sent to or from this e-mail address may be stored on the > Infosys e-mail system. > ***INFOSYS******** End of Disclaimer ********INFOSYS*** > >
