Yes Thanks .As I am new to search technology did not know about that:) I have anorher question , i have observed if child page has one url link that has been crawled because it was present in parent pages, during crawling child page it mark that link to be crawled in crawldb again . meaning if some url already has been crawled in up hirarchi then it should not crawl it as this has already been crawled, although its not coming twice when you check solr index but in fetch list you see it.
Sent from my iPhone > On Nov 18, 2015, at 7:34 AM, Jorge Luis Betancourt González > <[email protected]> wrote: > > Hi Manish: > > If you're indexing the content into Solr, using a TextField on the url field > could be enough, depending on the analyzers used. For instance, using the > example URL: > > http://www.apple.com/iphone-6s/3d-touch/ > > and a basic field configuration in solr this are the tokens that can finally > be used for search: > > www.apple.com, apple, com, iphone, 6s, 3d, touch > > This results are using a basic StandardTokenizerFactory and a > WordDelimiterFilterFactory. Tweaking WordDelimiterFilterFactory a little and > a couple of stopwords defined can lead to the results that you're expecting. > > Regards, > > ----- Original Message ----- > From: "Manish Verma" <[email protected]> > To: [email protected] > Sent: Friday, November 13, 2015 9:13:56 PM > Subject: Need To Index URL Strings > > Hi, > > For example the URL is http://www.apple.com/iphone-6s/3d-touch/ > <http://www.apple.com/iphone-6s/3d-touch/> I want to pull out URL strings > and index each string like iPhone-6s, 3d-touch. > > Thanks > Manish Verma > AML Search > +1 669 224 9924 > > Noviembre 13-14: Final Caribeña 2015 del Concurso de Programación ACM-ICPC > https://icpc.baylor.edu/regionals/finder/cf-2015
