[htdig] How to index output of cgi

[Laura Bello]

I have a simple cgi script that lists available standard memos based on their 
section number, i.e. clicking the "Section 01" link will bring up all section 01 
memos, etc. I can't for the life of me get the digging process to follow the 
links and index the memos.

This is the only output I get when I dig and I have heaps of memos to index:

New server: www.my.server, 80
0:0:0:http://www.my.server/path-to-directory/memos_list.html: ---------------- 
size = 2637

Here are some pertinent settings in my conf file:

  start_url: http://www.my.server/path-to-directory/memos_list.html

  limit_urls_to:          http://www.my.server/

  #exclude_urls:      /cgi-bin/ .cgi

I can see the urls in the resulting test.url file I get and they are correct. 
For example:

  http://www.my.server/cgi-bin/memolist.cgi?div=gap&section=01

I am using htidg v3.1.2. I would greatly appreciated any input as I've resorted 
to calling the cgi in a cron job to create static lists of memos for each 
section just so I can index them. What am I forgetting?

I liked Geoff Hutchison's idea about using a separate conf file for the 
individual cgi then merging it with the main database. I had wanted to try it.

thanks ... -laura


------------------------------------
To unsubscribe from the htdig mailing list, send a message to
[EMAIL PROTECTED] containing the single word unsubscribe in
the SUBJECT of the message.