I wanted to send this as a private reply but sent it to the list instead. Sorry for the inconvenience.
On Jan 8, 2008 10:21 AM, Susam Pal <[EMAIL PROTECTED]> wrote: > I have replied this query of yours yesterday in > [EMAIL PROTECTED] If you haven't received the reply, > probably you have not subscribed to the nutch-user mailing list. If > you haven't subscribed, please do so by sending a blank mail to > [EMAIL PROTECTED] > > Nutch 0.9 works fine for us. So it is not a bug in Nutch 0.9 This > looks like a configuration problem at your end. Please discuss this > properly in [EMAIL PROTECTED] instead of submitting it as a > bug in Nutch. > > Regards, > Susam Pal > > > On Jan 8, 2008 7:16 AM, sudarat (JIRA) <[EMAIL PROTECTED]> wrote: > > nutch crawl and index problem > > ----------------------------- > > > > Key: NUTCH-599 > > URL: https://issues.apache.org/jira/browse/NUTCH-599 > > Project: Nutch > > Issue Type: Bug > > Affects Versions: 0.9.0 > > Environment: hadoop-0.12.2, java jdk1.6.0 > > Reporter: sudarat > > Fix For: 0.9.0 > > > > > > first i set > > # skip file:, ftp:, & mailto: urls > > -^(file|ftp|mailto): > > > > # skip image and other suffixes we can't yet parse > > #-\.(png|PNG|ico|ICO|css|sit|eps|wmf|zip|mpg|gz|rpm|tgz|mov|MOV|exe|bmp|BMP)$ > > > > # skip URLs containing certain characters as probable queries, etc. > > [EMAIL PROTECTED] > > > > # skip URLs with slash-delimited segment that repeats 3+ times, to break > > loops > > -.*(/.+?)/.*?\1/.*?\1/ > > > > # skip everything else > > +. > > > > in conf/crawl-urlfilter.txt and use this command "bin/nutch crawl urls > > -dir crawled -depth 3" i can crawl http://guide.kanook.com but i can't > > crawl http://www.kapook.com , some webpage can't crawl all why? and index > > file after crawl don't have segments file for nutch search it have only > > > > -rw-r--r-- 1 nutch users 365 ?.?. 7 16:47 _0.fdt > > -rw-r--r-- 1 nutch users 8 ?.?. 7 16:47 _0.fdx > > -rw-r--r-- 1 nutch users 66 ?.?. 7 16:47 _0.fnm > > -rw-r--r-- 1 nutch users 370 ?.?. 7 16:47 _0.frq > > -rw-r--r-- 1 nutch users 9 ?.?. 7 16:47 _0.nrm > > -rw-r--r-- 1 nutch users 611 ?.?. 7 16:47 _0.prx > > -rw-r--r-- 1 nutch users 135 ?.?. 7 16:47 _0.tii > > -rw-r--r-- 1 nutch users 10553 ?.?. 7 16:47 _0.tis > > -rw-r--r-- 1 nutch users 0 ?.?. 7 16:47 index.done > > -rw-r--r-- 1 nutch users 41 ?.?. 7 16:47 segments_2 > > -rw-r--r-- 1 nutch users 20 ?.?. 7 16:47 segments.gen > > > > how to solve it? > > > > > > -- > > This message is automatically generated by JIRA. > > - > > You can reply to this email to add a comment to the issue online. > > > > >
