All, I have a root domain and a couple directories deep I have some files that I want to index. The problem is that they are not referenced on the main page using a hyperlink or anything like that.
http://www.geoglobaldomination.org/kml/temp/ I want to be able to crawl down in to /kml/temp/ without knowing that it's even there. Is there a way to do this in Nutch? echo http://www.geoglobaldomination.org > urls ./nutch crawl urls -threads 10 -depth 10 -topN 20 -solr http://172.16.2.107:8983/solr Nothing and I suspect that it's because there is not a hyperlink on the main page. Thoughts? Adam
