I did some experimenting over the weekend, and I have data to report, though nothing terribly conclusive.
What I'm doing is this: htdig, running on a separate system from the website being indexed, requests a page. That page is a Tcl script which returns a list of links to articles, which htdig follows to index the content.
It takes 8 minutes for htdig to process 500 articles, so you would expect that if one wanted to fill 60 minutes, one would run the dig 8 times. This is what I've been doing, but instead of the 64 minutes one would expect, it takes 114 minutes to complete 8 consecutive runs. I have no explanation for this.
I ran this clump of 8 digs three times for each web server, and here is the result:
3.3+ad13:
initial size 37780K, final size 85188K
4.0.8:
initial size 27360K, final size 76184K
My observation is that both are growing, seemingly without bound, and that they never give any memory back; even if one lets this sit overnight with no activity, the memory footprint stays the same.
I also discovered that I had a long-forgotten cron job which restarted these sites weekly when they were running under 3.3+ad13. I have a vague memory of this having been necessary due to a memory leak in 3.2, which a number of us were having trouble with, and I just never took it out. So this would have prevented us from noticing any similar problems with 3.3.
So, the problem may not be new in 4.x, but there is still a problem and I'm still not sure what to do to track it down. The script that assembles the article for viewing, which is being executed over and over, is pretty simple; it just selects a bunch of data from the database, does an insert to track the fact that the article has been viewed, and then returns the whole shebang to the browser, formatted in the usual OpenACS way with an .adp template. Nothing is being explicitly cached.
This is all going on on the staging site. The live site, which was just restarted two days ago, is up to 987060K. This is a pretty busy site so it's not time to worry yet, but I do wonder if it will ever stop growing.
I have not yet updated Tcl, and still plan to do that. What else can I do to help track this down?
janine
-- AOLserver - http://www.aolserver.com/
To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> with the body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: field of your email blank.
