Author: Gary Briggs Email: [EMAIL PROTECTED] Message: I have a fairly hefty problem with UDM search, I'm not sure what's causing it. OK, let's start with the hardware that I'm using: The host that's indexing stuff is a dual SparcII 450, it has 2G RAM, and LOTS of hard drive [80G or so available if I need it] Solaris 2.6, and a couple of fast ethernet connections into the back of it. A powerful machine. The other day I came into work, having left it indexing overnight, and had to reboot it. According to my post-reboot research, /tmp had run out of space. Fine, it's because I have /tmp in memory. The machine basically ran out of core. Thing is, there's 2G of swap available, and that was run out aswell. 4G is a lot of primary space to be used up. OK, so more snooping found exactly 448 processes called "indexer" had been running at the time we eventually decided to reboot it. Which is not good. If every one of those writes a temp file to /tmp, and has a non-negligible memory footprint... OK, so I've now edited the code so it puts it's temporary files in /var/tmp, which is secondary storage. But I'm sitting here watching it indexing again, and the number of processes is slowly creeping up. I'm afraid to leave this overnight after what happened last time... I've read all of the old posts on this webboard and the mailing list, and I can't see anything except one potential, a guy saying that "sometimes it can take two minutes or more to parse a page". If every document takes 2 mins, and the program forks a parser then moves on, this could be a problem where I've got very high bandwidth going into the machine. I've been trawling through the code and I've found MAX_PARSERS in src/parser.c, which is 256 by default. Which is less than 448, so I feel it probably wasn't working correctly. In the same file, the program also fork()s. I think it's there that my problem lies; I can't find anything that resembles sanity on the fact that it actually forks. Whether it can or not, it does actually for for every document it needs to parse. I'm currently using the in-built database. Could this be the problem? Feel free to tell me I'm being stupid, but I cant see anything I'm doing wrong... Any ideas? Thanks, Gary (-; Reply: <http://search.mnogo.ru/board/message.php?id=549> ______________ If you want to unsubscribe send "unsubscribe udmsearch" to [EMAIL PROTECTED]
