I've had my provider set up a cronjob to clear both of my servers' spray cache directories, and it makes a HUGE difference in terms of spikes during folder access... especially on servers that do not have local storage. We've actually set up the files to clear after they reach a certain age, and it seems to work very well.
-----Original Message----- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of [EMAIL PROTECTED] Sent: Wednesday, July 27, 2005 06:25 To: [email protected] Subject: hlds_linux digest, Vol 1 #4538 - 8 msgs Send hlds_linux mailing list submissions to [email protected] To subscribe or unsubscribe via the World Wide Web, visit http://list.valvesoftware.com/mailman/listinfo/hlds_linux or, via email, send a message with subject or body 'help' to [EMAIL PROTECTED] You can reach the person managing the list at [EMAIL PROTECTED] When replying, please edit your Subject line so it is more specific than "Re: Contents of hlds_linux digest..." Today's Topics: 1. Re: BUG: Server-side sprays and lag spikes (James Tucker) 2. Re: BUG: Server-side sprays and lag spikes (apophis) 3. Re: BUG: Server-side sprays and lag spikes (kama) 4. Re: BUG: Server-side sprays and lag spikes (Stan Bubrouski) 5. RE: BUG: Server-side sprays and lag spikes (Alfred Reynolds) 6. Re: BUG: Server-side sprays and lag spikes (Ronny Schedel) 7. Re: BUG: Server-side sprays and lag spikes (Graham Robinson) 8. RE: BUG: Server-side sprays and lag spikes (Kyle Milnes) --__--__-- Message: 1 Date: Tue, 26 Jul 2005 14:59:39 +0100 From: James Tucker <[EMAIL PROTECTED]> To: [email protected] Subject: Re: [hlds_linux] BUG: Server-side sprays and lag spikes Reply-To: [email protected] With regard to this, Alfred, would it be possible to add an rcon command to allow a mass purge of this cache from within the engine, then we can get support for cronjob clearance of it? On 7/26/05, Graham Robinson <[EMAIL PROTECTED]> wrote: > I forgot to actually mention that I did delete my 344MB of sprays and > it does seem to have made a difference to my server. I might make a > point of deleting them weekly (that was 2.5 months worth as it's a > fairly new server). > > Graham > > On 7/26/05, Kyle Milnes <[EMAIL PROTECTED]> wrote: > > Hi Bart, > > > > If what your research has uncovered is linked to lag-spikes then well played > > on figuring it out. > > _______________________________________________ > To unsubscribe, edit your list preferences, or view the list archives, please visit: > http://list.valvesoftware.com/mailman/listinfo/hlds_linux > --__--__-- Message: 2 Date: Tue, 26 Jul 2005 10:14:27 -0500 From: apophis <[EMAIL PROTECTED]> To: [email protected] Subject: Re: [hlds_linux] BUG: Server-side sprays and lag spikes Reply-To: [email protected] You could make a server plugin for it. On 7/26/05, James Tucker <[EMAIL PROTECTED]> wrote: > With regard to this, Alfred, would it be possible to add an rcon > command to allow a mass purge of this cache from within the engine, > then we can get support for cronjob clearance of it? > > On 7/26/05, Graham Robinson <[EMAIL PROTECTED]> wrote: > > I forgot to actually mention that I did delete my 344MB of sprays and > > it does seem to have made a difference to my server. I might make a > > point of deleting them weekly (that was 2.5 months worth as it's a > > fairly new server). > > > > Graham > > > > On 7/26/05, Kyle Milnes <[EMAIL PROTECTED]> wrote: > > > Hi Bart, > > > > > > If what your research has uncovered is linked to lag-spikes then well played > > > on figuring it out. > > > > _______________________________________________ > > To unsubscribe, edit your list preferences, or view the list archives, please visit: > > http://list.valvesoftware.com/mailman/listinfo/hlds_linux > > > > _______________________________________________ > To unsubscribe, edit your list preferences, or view the list archives, please visit: > http://list.valvesoftware.com/mailman/listinfo/hlds_linux > --__--__-- Message: 3 Date: Tue, 26 Jul 2005 18:14:34 +0200 (CEST) From: kama <[EMAIL PROTECTED]> To: <[email protected]> Subject: Re: [hlds_linux] BUG: Server-side sprays and lag spikes Reply-To: [email protected] Cant you just delete them with rm? Or will that crash the server? /Bjorn On Tue, 26 Jul 2005, apophis wrote: > You could make a server plugin for it. > > On 7/26/05, James Tucker <[EMAIL PROTECTED]> wrote: > > With regard to this, Alfred, would it be possible to add an rcon > > command to allow a mass purge of this cache from within the engine, > > then we can get support for cronjob clearance of it? > > > > On 7/26/05, Graham Robinson <[EMAIL PROTECTED]> wrote: > > > I forgot to actually mention that I did delete my 344MB of sprays and > > > it does seem to have made a difference to my server. I might make a > > > point of deleting them weekly (that was 2.5 months worth as it's a > > > fairly new server). > > > > > > Graham > > > > > > On 7/26/05, Kyle Milnes <[EMAIL PROTECTED]> wrote: > > > > Hi Bart, > > > > > > > > If what your research has uncovered is linked to lag-spikes then well played > > > > on figuring it out. > > > > > > _______________________________________________ > > > To unsubscribe, edit your list preferences, or view the list archives, please visit: > > > http://list.valvesoftware.com/mailman/listinfo/hlds_linux > > > > > > > _______________________________________________ > > To unsubscribe, edit your list preferences, or view the list archives, please visit: > > http://list.valvesoftware.com/mailman/listinfo/hlds_linux > > > > _______________________________________________ > To unsubscribe, edit your list preferences, or view the list archives, please visit: > http://list.valvesoftware.com/mailman/listinfo/hlds_linux > -- You never get a second chance to make a bad first impression. --__--__-- Message: 4 Date: Tue, 26 Jul 2005 13:35:04 -0400 From: Stan Bubrouski <[EMAIL PROTECTED]> To: [email protected] Subject: Re: [hlds_linux] BUG: Server-side sprays and lag spikes Reply-To: [email protected] You can setup a cron job to delete them weekly or daily if you want. AFAIK it will not have any detrimental effect on the server. -sb On 7/26/05, kama <[EMAIL PROTECTED]> wrote: > > Cant you just delete them with rm? Or will that crash the server? > > /Bjorn > > On Tue, 26 Jul 2005, apophis wrote: > > > You could make a server plugin for it. > > > > On 7/26/05, James Tucker <[EMAIL PROTECTED]> wrote: > > > With regard to this, Alfred, would it be possible to add an rcon > > > command to allow a mass purge of this cache from within the engine, > > > then we can get support for cronjob clearance of it? > > > > > > On 7/26/05, Graham Robinson <[EMAIL PROTECTED]> wrote: > > > > I forgot to actually mention that I did delete my 344MB of sprays and > > > > it does seem to have made a difference to my server. I might make a > > > > point of deleting them weekly (that was 2.5 months worth as it's a > > > > fairly new server). > > > > > > > > Graham > > > > > > > > On 7/26/05, Kyle Milnes <[EMAIL PROTECTED]> wrote: > > > > > Hi Bart, > > > > > > > > > > If what your research has uncovered is linked to lag-spikes then well played > > > > > on figuring it out. > > > > > > > > _______________________________________________ > > > > To unsubscribe, edit your list preferences, or view the list archives, please visit: > > > > http://list.valvesoftware.com/mailman/listinfo/hlds_linux > > > > > > > > > > _______________________________________________ > > > To unsubscribe, edit your list preferences, or view the list archives, please visit: > > > http://list.valvesoftware.com/mailman/listinfo/hlds_linux > > > > > > > _______________________________________________ > > To unsubscribe, edit your list preferences, or view the list archives, please visit: > > http://list.valvesoftware.com/mailman/listinfo/hlds_linux > > > > -- > > You never get a second chance to make a bad first impression. > > > _______________________________________________ > To unsubscribe, edit your list preferences, or view the list archives, please visit: > http://list.valvesoftware.com/mailman/listinfo/hlds_linux > --__--__-- Message: 5 Subject: RE: [hlds_linux] BUG: Server-side sprays and lag spikes Date: Tue, 26 Jul 2005 11:29:42 -0700 From: "Alfred Reynolds" <[EMAIL PROTECTED]> To: <[email protected]> Reply-To: [email protected] We will look at doing automatic cleanup of the cache, no need to user intervention. - Alfred ----Original Message---- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of James Tucker Sent: Tuesday, July 26, 2005 7:00 AM To: [email protected] Subject: Re: [hlds_linux] BUG: Server-side sprays and lag spikes > With regard to this, Alfred, would it be possible to add an rcon > command to allow a mass purge of this cache from within the engine, > then we can get support for cronjob clearance of it? > > On 7/26/05, Graham Robinson <[EMAIL PROTECTED]> wrote: > > I forgot to actually mention that I did delete my 344MB of sprays > > and > > it does seem to have made a difference to my server. I might make a > > point of deleting them weekly (that was 2.5 months worth as it's a > > fairly new server). > > > > Graham > > > > On 7/26/05, Kyle Milnes <[EMAIL PROTECTED]> wrote: > > > Hi Bart, > > > > > > If what your research has uncovered is linked to lag-spikes then > > > well played on figuring it out. > > > > _______________________________________________ > > To unsubscribe, edit your list preferences, or view the list > > archives, please visit: > > http://list.valvesoftware.com/mailman/listinfo/hlds_linux > > > > _______________________________________________ > To unsubscribe, edit your list preferences, or view the list > archives, please visit: > http://list.valvesoftware.com/mailman/listinfo/hlds_linux --__--__-- Message: 6 From: "Ronny Schedel" <[EMAIL PROTECTED]> To: <[email protected]> Subject: Re: [hlds_linux] BUG: Server-side sprays and lag spikes Date: Tue, 26 Jul 2005 17:03:54 +0200 Reply-To: [email protected] Old known problem, just delete the sprays autmaticly or manually. Greets Ronny > Hi all, > > Something that has been plaguing our 64 player server for a while is > seemingly random lag spikes, but after some detailed investigation into > the > problem a co-admin and I have managed to find the reason. > > We tried everything to find out why this was happening, including buying > new > hardware and changing bandwidth providers. After all that, we are 100% > convinced it was srcds and I finally have the proof. > > In brief, more detailed report follows: > > - The way srcds calculates the file name for storing uploaded sprays > causes > lag. > > In order to get past the spam in the console[1], we reduced the server to > 32 > players. We then were able to observe lag spikes in the game, and by > coincidence, monitoring the server console we saw this error: > > CreateFragmentsFromFile: 'downloads/ee9becfc.dat' doesn't exist. > [message is repeated several times] > > In the time this message is printed, a lag spike occurs in the game. In > the > past, it has been advised to "ignore" this message - while this is safe to > do so, when you have a popular server (as we do) it happens far too > regularly. > > And the reason why this is bad? We looked into more detail by using > strace > on srcds to see what happens, here is the result just as the > CreateFragmentsFromFile message appears (note: we run Debian Linux): > > open("/cs/css/cstrike/downloads", > O_RDONLY|O_NONBLOCK|O_LARGEFILE|O_DIRECTORY) = 13 > fstat64(13, {st_mode=S_IFDIR|0755, st_size=48, ...}) = 0 > fcntl64(13, F_SETFD, FD_CLOEXEC) = 0 > getdents64(13, /* 2 entries */, 131072) = 48 > getdents64(13, /* 0 entries */, 131072) = 0 > close(13) = 0 > > The downloads directory is where srcds stores uploaded sprays from > clients. > To put this into perspective, we have, on average, 3000 clients connecting > to our server a day, and if the majority of clients have a custom spray, > this will ultimately get uploaded to the srcds server. > > For example: > > banana:/cs/css/cstrike# ls -l downloads | wc -l > 52235 > > 52000 files over a three month period! > > Linux programmers on the list may notice that using getdents(2) on that > many > files, not matter how fast the bus, disk or whatever, *will* take a short > moment - i.e. the exact length of one, random lag spike. > > So to conclude, my questions are: > > - Why doesn't the server clean up the downloads folder? > - How does the server generate the filenames used to store uploaded > sprays? > - Can this be fixed? > > Our solution at the moment is to simply delete all content in the > downloads > folder and/or set sv_allowupload to zero, but you get even more > CreateFragmentsFromFile errors in the console (but no lag!). > > We used a lot of time and effort to get to the bottom of this. I'd > appreciate if Valve would at least acknowledge this. > > [1] - Remove the console message that says "Free sound list is full!" - it > is absolutely meaningless. > > -- > Bart King -- http://www.bart666.com > +44 781 219 5654 -- PGP: 0xC9C3EB8B > > > > _______________________________________________ > To unsubscribe, edit your list preferences, or view the list archives, > please visit: > http://list.valvesoftware.com/mailman/listinfo/hlds_linux > --__--__-- Message: 7 Date: Wed, 27 Jul 2005 08:59:25 +0100 From: Graham Robinson <[EMAIL PROTECTED]> To: [email protected] Subject: Re: [hlds_linux] BUG: Server-side sprays and lag spikes Reply-To: [email protected] With all due repect I wouldn't like Alfred to stop building automatic deletion into the SRCDS just because you knew about the problem. Thanks for that Alfred, it will be one less thing on my list each week. Graham On 7/26/05, Ronny Schedel <[EMAIL PROTECTED]> wrote: > Old known problem, just delete the sprays autmaticly or manually. > > Greets > Ronny --__--__-- Message: 8 From: "Kyle Milnes" <[EMAIL PROTECTED]> To: <[email protected]> Subject: RE: [hlds_linux] BUG: Server-side sprays and lag spikes Date: Wed, 27 Jul 2005 11:34:38 +0100 Reply-To: [email protected] Alfred, What about the dir scanning part, I presume that is potentially the issue causing the tic jumping (lag spikes)? Are you working on a fix for that too? Or just cleaning up the dir? - Kyle -----Original Message----- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Alfred Reynolds Sent: 26 July 2005 19:30 To: [email protected] Subject: RE: [hlds_linux] BUG: Server-side sprays and lag spikes We will look at doing automatic cleanup of the cache, no need to user intervention. - Alfred ----Original Message---- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of James Tucker Sent: Tuesday, July 26, 2005 7:00 AM To: [email protected] Subject: Re: [hlds_linux] BUG: Server-side sprays and lag spikes > With regard to this, Alfred, would it be possible to add an rcon > command to allow a mass purge of this cache from within the engine, > then we can get support for cronjob clearance of it? > > On 7/26/05, Graham Robinson <[EMAIL PROTECTED]> wrote: > > I forgot to actually mention that I did delete my 344MB of sprays > > and it does seem to have made a difference to my server. I might > > make a point of deleting them weekly (that was 2.5 months worth as > > it's a fairly new server). > > > > Graham > > > > On 7/26/05, Kyle Milnes <[EMAIL PROTECTED]> wrote: > > > Hi Bart, > > > > > > If what your research has uncovered is linked to lag-spikes then > > > well played on figuring it out. > > > > _______________________________________________ > > To unsubscribe, edit your list preferences, or view the list > > archives, please visit: > > http://list.valvesoftware.com/mailman/listinfo/hlds_linux > > > > _______________________________________________ > To unsubscribe, edit your list preferences, or view the list archives, > please visit: > http://list.valvesoftware.com/mailman/listinfo/hlds_linux _______________________________________________ To unsubscribe, edit your list preferences, or view the list archives, please visit: http://list.valvesoftware.com/mailman/listinfo/hlds_linux --__--__-- _______________________________________________ To unsubscribe, edit your list preferences, or view the list archives, please visit: http://list.valvesoftware.com/mailman/listinfo/hlds_linux End of hlds_linux Digest _______________________________________________ To unsubscribe, edit your list preferences, or view the list archives, please visit: http://list.valvesoftware.com/mailman/listinfo/hlds_linux
