Re: [hlds] [hlds_linux] TF2 server performance spikes
I hacked a bit ( https://forums.alliedmods.net/showthread.php?p=1953195 ) to get effectively the same no-compression functionality for the other OB games. Here's some new vprof dumps that I took earlier today. http://pastie.org/private/kywlfyn5teu0lxqghgyjg http://pastie.org/private/hei4cdkxvbdsh6llyo1gwa http://pastie.org/private/ya17uvzlnp5gdpldpnuuw http://pastie.org/private/pmvxerm7hy2n3mqcjktqia http://pastie.org/private/qi981phaxbcpto8kczix4w Hope this helps, Kyle. On Wed, May 15, 2013 at 6:19 AM, Aridane Manuel Ojeda Sánchez pcdriv...@gmail.com wrote: Only three active servers, with 7, 12 and 11 players, and use 100% CPU, with 4 cores at 100%. Only one of them has replay enabled, and none have HLTV. 12690 mainip20 0 586m 368m 15m R 80 2.3 20:35.93 srcds_linux 24850 mainip20 0 578m 369m 14m R 76 2.3 3:02.21 srcds_linux 10097 mainip20 0 609m 404m 14m R 73 2.5 165:08.63 srcds_linux I also noticed that consume too much ram. Even I have closed the servers, I made start one after another, and only after starting the first when checking for the dedicated server was updated, and consumed much of the CPU. 2013/5/14 Steven Hartland kill...@multiplay.co.uk Not sure what the current default is now but out of interest does adding -threads 3 to your command line help at all? - Original Message - From: martin v velt...@gmail.com I've noticed that too. HUGE spikes causing waprs/lags to clients. Yellow lerp and ping rising up to 200 (sometimes more) for about 1-2sec. This keeps happening about every 1 minute. Players are fully raged by this. 2013/5/14 Kyle Sanderson kyle.l...@gmail.com http://pastie.org/private/**0nehujv54iw2lwtndcyopw http://pastie.org/private/0nehujv54iw2lwtndcyopw http://pastie.org/private/**hhwl7eougryhiin7n8ma http://pastie.org/private/hhwl7eougryhiin7n8ma http://pastie.org/private/**t8sdtvfxuh9aigylqdc6w http://pastie.org/private/t8sdtvfxuh9aigylqdc6w http://pastie.org/private/**d4q26os0usxa4ofvd04uaw http://pastie.org/private/d4q26os0usxa4ofvd04uaw http://pastie.org/private/**ns6sfnkp59avunqk0lkhpq http://pastie.org/private/ns6sfnkp59avunqk0lkhpq The spikes seem to happen every minute. If you need more vprof spikes I can now provide them. We're running with the parallel convars enabled that are known to crash. Nothing has changed besides updating to Steam|. ==**== This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. In the event of misdirection, illegible or incomplete transmission please telephone +44 845 868 1337 or return the E.mail to postmas...@multiplay.co.uk. __**_ To unsubscribe, edit your list preferences, or view the list archives, please visit: https://list.valvesoftware.**com/cgi-bin/mailman/listinfo/**hlds_linux https://list.valvesoftware.com/cgi-bin/mailman/listinfo/hlds_linux ___ To unsubscribe, edit your list preferences, or view the list archives, please visit: https://list.valvesoftware.com/cgi-bin/mailman/listinfo/hlds_linux ___ To unsubscribe, edit your list preferences, or view the list archives, please visit: https://list.valvesoftware.com/cgi-bin/mailman/listinfo/hlds
Re: [hlds] [hlds_linux] TF2 server performance spikes
This is being actively investigated with high priority right now. -Original Message- From: hlds_linux-boun...@list.valvesoftware.com [mailto:hlds_linux-boun...@list.valvesoftware.com] On Behalf Of martin v Sent: Tuesday, May 14, 2013 10:08 AM To: Half-Life dedicated Linux server mailing list Subject: Re: [hlds_linux] [hlds] TF2 server performance spikes I've noticed that too. HUGE spikes causing waprs/lags to clients. Yellow lerp and ping rising up to 200 (sometimes more) for about 1-2sec. This keeps happening about every 1 minute. Players are fully raged by this. 2013/5/14 Kyle Sanderson kyle.l...@gmail.com http://pastie.org/private/0nehujv54iw2lwtndcyopw http://pastie.org/private/hhwl7eougryhiin7n8ma http://pastie.org/private/t8sdtvfxuh9aigylqdc6w http://pastie.org/private/d4q26os0usxa4ofvd04uaw http://pastie.org/private/ns6sfnkp59avunqk0lkhpq The spikes seem to happen every minute. If you need more vprof spikes I can now provide them. We're running with the parallel convars enabled that are known to crash. Nothing has changed besides updating to Steam|. Thanks, Kyle. On Mon, May 13, 2013 at 7:24 PM, Kyle Sanderson kyle.l...@gmail.com wrote: It's highly possible you're running on a supported game. I had a vprof1.txt in that directory once, and it said no samples. I had thought I created the file, but it's highly possible it does work. If it doesn't work for you, I have a horrible, horrible SM extension available. What it does is capture VProf output text and will write it to a file. What it isn't is portable across platforms, good, or even not a cat killer. But if you're after trying the vprof approach, you're welcome to try it. I can probably fix it up (and make it a MM:S plugin, un-hardcode things) if there's any interest, but I'm hoping Valve fixes VProf before that happens. I probably won't compile/support a Windows version as VTrace / Telemetry is probably a better way forward. The files will be dumped in the directory that Peter Jerde mentioned (vprof/). If the directory doesn't exist, you'll probably have to create it by hand. http://stuff.plaguefest.com/kyle/vprof.tar.xz ( tar xJvf vprof.tar.xz ) There's only TF and CS:S binaries in there. The extension can probably be independent, but this isn't intended as a fix, rather a stopgap. If you've never used a lone extension before, you either need to manually load it, or create a .autoload file alongside the extension (VProfExt.autoload). Hope this helps, Kyle. PS: If your cat does die, it was this. On Mon, May 13, 2013 at 3:54 PM, Peter Jerde peter-h...@jerde.net wrote: On May 13, 2013, at 12:20 PM, Kyle Sanderson kyle.l...@gmail.com wrote: There's no clean way to capture the output from vprof at the time of this writing. Huh? I put in: vprof_on vprof_dump_spikes 20 // dump a vprof report if frame rate drops below 20 ... on one of my servers, and it created /tf/vprof/vprof1.txt the first time it hit a long frame. That seems pretty clean to me. - Peter ___ To unsubscribe, edit your list preferences, or view the list archives, please visit: https://list.valvesoftware.com/cgi-bin/mailman/listinfo/hlds ___ To unsubscribe, edit your list preferences, or view the list archives, please visit: https://list.valvesoftware.com/cgi-bin/mailman/listinfo/hlds_linux ___ To unsubscribe, edit your list preferences, or view the list archives, please visit: https://list.valvesoftware.com/cgi-bin/mailman/listinfo/hlds_linux ___ To unsubscribe, edit your list preferences, or view the list archives, please visit: https://list.valvesoftware.com/cgi-bin/mailman/listinfo/hlds
Re: [hlds] [hlds_linux] TF2 server performance spikes
Has anybody used vprof spike mode? This sort of problem is pretty much exactly what it was designed to investigate. E.g. vprof_on vprof_dump_spikes 20 // dump a vprof report if frame rate drops below 20 If the problem is caused by network traffic, using the -netspike command line argument can also be used to dig deeper into exactly what entity data is being sent. If you have a report and you don't know how to interpret it, you can email it to me or post it here, I'm sure there are some people who can help analyze it. -Original Message- From: hlds_linux-boun...@list.valvesoftware.com [mailto:hlds_linux-boun...@list.valvesoftware.com] On Behalf Of Bjorn Wielens Sent: Monday, May 13, 2013 9:42 AM To: Half-Life dedicated Win32 server mailing list; hlds_li...@list.valvesoftware.com Subject: Re: [hlds_linux] [hlds] TF2 server performance spikes Correct, I was one of the folks discussing this in the last round of messages and we run on Linux. Since my last message about having disabled replays I still get occasional hiccups in a full server, but nowhere near as bad as before, owing to the lack of replay overhead. I'd be happy to run diagnostics if they were available for Linux, but alas that's not the case. From: Alex Kowald abkow...@gmail.com To: Half-Life dedicated Win32 server mailing list hlds@list.valvesoftware.com Sent: Monday, May 13, 2013 1:16:04 PM Subject: Re: [hlds] TF2 server performance spikes You are only accepting traces for windows servers correct? Some of the people that have complained are running linux. I have the issue myself on two independent servers. We get this new game studder after upgrading, leaving everything else the same. On Mon, May 13, 2013 at 12:11 PM, Fletcher Dunn fletch...@valvesoftware.com wrote: I do have a theory about the cause of the CPU spikes. I’ll release a build with the next mandatory update (which will probably be this afternoon) that I think could address the problem. But since we are not reproducing the problem here, this is just a theory, not disciplined engineering. In order to conduct disciplined engineering, we need data from those who are able to reproduce it. Judging by the amount of discussion this problem has generated and the relatively small amount of effort that is required in order to get the data to us, I’m really surprised we haven’t received more traces. I have given out vtrace keys to every person who has asked for one. So far we have received traces from exactly two (2) server operators. One server operator had processor affinities set, the other was running a high number of instances per core, such that the average CPU utilization was high and there was no headroom for more than one process to spike. ___ To unsubscribe, edit your list preferences, or view the list archives, please visit: https://list.valvesoftware.com/cgi-bin/mailman/listinfo/hlds ___ To unsubscribe, edit your list preferences, or view the list archives, please visit: https://list.valvesoftware.com/cgi-bin/mailman/listinfo/hlds ___ To unsubscribe, edit your list preferences, or view the list archives, please visit: https://list.valvesoftware.com/cgi-bin/mailman/listinfo/hlds_linux ___ To unsubscribe, edit your list preferences, or view the list archives, please visit: https://list.valvesoftware.com/cgi-bin/mailman/listinfo/hlds
Re: [hlds] [hlds_linux] TF2 server performance spikes
The problem with vprof is SRCDS has become very verbose with bogus data table warnings and clients flooding server messages by running commands without args. There's con_logfile, but once that starts, it cannot be stopped. I had a 90MB~ log file after a couple hours, which isn't fun on an SSD. vprof needs to write files/dumps automatically to disk in order to become useful for servers. Thanks, Kyle. On 13 May 2013 10:09, Fletcher Dunn fletch...@valvesoftware.com wrote: Has anybody used vprof spike mode? This sort of problem is pretty much exactly what it was designed to investigate. E.g. vprof_on vprof_dump_spikes 20 // dump a vprof report if frame rate drops below 20 If the problem is caused by network traffic, using the -netspike command line argument can also be used to dig deeper into exactly what entity data is being sent. If you have a report and you don't know how to interpret it, you can email it to me or post it here, I'm sure there are some people who can help analyze it. -Original Message- From: hlds_linux-boun...@list.valvesoftware.com [mailto: hlds_linux-boun...@list.valvesoftware.com] On Behalf Of Bjorn Wielens Sent: Monday, May 13, 2013 9:42 AM To: Half-Life dedicated Win32 server mailing list; hlds_li...@list.valvesoftware.com Subject: Re: [hlds_linux] [hlds] TF2 server performance spikes Correct, I was one of the folks discussing this in the last round of messages and we run on Linux. Since my last message about having disabled replays I still get occasional hiccups in a full server, but nowhere near as bad as before, owing to the lack of replay overhead. I'd be happy to run diagnostics if they were available for Linux, but alas that's not the case. From: Alex Kowald abkow...@gmail.com To: Half-Life dedicated Win32 server mailing list hlds@list.valvesoftware.com Sent: Monday, May 13, 2013 1:16:04 PM Subject: Re: [hlds] TF2 server performance spikes You are only accepting traces for windows servers correct? Some of the people that have complained are running linux. I have the issue myself on two independent servers. We get this new game studder after upgrading, leaving everything else the same. On Mon, May 13, 2013 at 12:11 PM, Fletcher Dunn fletch...@valvesoftware.com wrote: I do have a theory about the cause of the CPU spikes. I’ll release a build with the next mandatory update (which will probably be this afternoon) that I think could address the problem. But since we are not reproducing the problem here, this is just a theory, not disciplined engineering. In order to conduct disciplined engineering, we need data from those who are able to reproduce it. Judging by the amount of discussion this problem has generated and the relatively small amount of effort that is required in order to get the data to us, I’m really surprised we haven’t received more traces. I have given out vtrace keys to every person who has asked for one. So far we have received traces from exactly two (2) server operators. One server operator had processor affinities set, the other was running a high number of instances per core, such that the average CPU utilization was high and there was no headroom for more than one process to spike. ___ To unsubscribe, edit your list preferences, or view the list archives, please visit: https://list.valvesoftware.com/cgi-bin/mailman/listinfo/hlds ___ To unsubscribe, edit your list preferences, or view the list archives, please visit: https://list.valvesoftware.com/cgi-bin/mailman/listinfo/hlds ___ To unsubscribe, edit your list preferences, or view the list archives, please visit: https://list.valvesoftware.com/cgi-bin/mailman/listinfo/hlds_linux ___ To unsubscribe, edit your list preferences, or view the list archives, please visit: https://list.valvesoftware.com/cgi-bin/mailman/listinfo/hlds ___ To unsubscribe, edit your list preferences, or view the list archives, please visit: https://list.valvesoftware.com/cgi-bin/mailman/listinfo/hlds
Re: [hlds] [hlds_linux] TF2 server performance spikes
I’m not understanding how vprof is related to the console problems you described. Would you mind clarifying? If you’re reliably able to reproduce a spike, would it not be possible to turn on vprof spike mode, get a report, and then turn vprof off. If the spikes are really huge, then just set the threshold really low. From: hlds-boun...@list.valvesoftware.com [mailto:hlds-boun...@list.valvesoftware.com] On Behalf Of Kyle Sanderson Sent: Monday, May 13, 2013 10:14 AM To: Half-Life dedicated Win32 server mailing list Subject: Re: [hlds] [hlds_linux] TF2 server performance spikes The problem with vprof is SRCDS has become very verbose with bogus data table warnings and clients flooding server messages by running commands without args. There's con_logfile, but once that starts, it cannot be stopped. I had a 90MB~ log file after a couple hours, which isn't fun on an SSD. vprof needs to write files/dumps automatically to disk in order to become useful for servers. Thanks, Kyle. On 13 May 2013 10:09, Fletcher Dunn fletch...@valvesoftware.commailto:fletch...@valvesoftware.com wrote: Has anybody used vprof spike mode? This sort of problem is pretty much exactly what it was designed to investigate. E.g. vprof_on vprof_dump_spikes 20 // dump a vprof report if frame rate drops below 20 If the problem is caused by network traffic, using the -netspike command line argument can also be used to dig deeper into exactly what entity data is being sent. If you have a report and you don't know how to interpret it, you can email it to me or post it here, I'm sure there are some people who can help analyze it. -Original Message- From: hlds_linux-boun...@list.valvesoftware.commailto:hlds_linux-boun...@list.valvesoftware.com [mailto:hlds_linux-boun...@list.valvesoftware.commailto:hlds_linux-boun...@list.valvesoftware.com] On Behalf Of Bjorn Wielens Sent: Monday, May 13, 2013 9:42 AM To: Half-Life dedicated Win32 server mailing list; hlds_li...@list.valvesoftware.commailto:hlds_li...@list.valvesoftware.com Subject: Re: [hlds_linux] [hlds] TF2 server performance spikes Correct, I was one of the folks discussing this in the last round of messages and we run on Linux. Since my last message about having disabled replays I still get occasional hiccups in a full server, but nowhere near as bad as before, owing to the lack of replay overhead. I'd be happy to run diagnostics if they were available for Linux, but alas that's not the case. From: Alex Kowald abkow...@gmail.commailto:abkow...@gmail.com To: Half-Life dedicated Win32 server mailing list hlds@list.valvesoftware.commailto:hlds@list.valvesoftware.com Sent: Monday, May 13, 2013 1:16:04 PM Subject: Re: [hlds] TF2 server performance spikes You are only accepting traces for windows servers correct? Some of the people that have complained are running linux. I have the issue myself on two independent servers. We get this new game studder after upgrading, leaving everything else the same. On Mon, May 13, 2013 at 12:11 PM, Fletcher Dunn fletch...@valvesoftware.commailto:fletch...@valvesoftware.com wrote: I do have a theory about the cause of the CPU spikes. I’ll release a build with the next mandatory update (which will probably be this afternoon) that I think could address the problem. But since we are not reproducing the problem here, this is just a theory, not disciplined engineering. In order to conduct disciplined engineering, we need data from those who are able to reproduce it. Judging by the amount of discussion this problem has generated and the relatively small amount of effort that is required in order to get the data to us, I’m really surprised we haven’t received more traces. I have given out vtrace keys to every person who has asked for one. So far we have received traces from exactly two (2) server operators. One server operator had processor affinities set, the other was running a high number of instances per core, such that the average CPU utilization was high and there was no headroom for more than one process to spike. ___ To unsubscribe, edit your list preferences, or view the list archives, please visit: https://list.valvesoftware.com/cgi-bin/mailman/listinfo/hlds ___ To unsubscribe, edit your list preferences, or view the list archives, please visit: https://list.valvesoftware.com/cgi-bin/mailman/listinfo/hlds ___ To unsubscribe, edit your list preferences, or view the list archives, please visit: https://list.valvesoftware.com/cgi-bin/mailman/listinfo/hlds_linux ___ To unsubscribe, edit your list preferences, or view the list archives, please visit: https://list.valvesoftware.com/cgi-bin/mailman/listinfo/hlds
Re: [hlds] [hlds_linux] TF2 server performance spikes
You can disable con_logfile by doing: con_logfile (Note nothing between the quotes) On Mon, May 13, 2013 at 6:14 PM, Kyle Sanderson kyle.l...@gmail.com wrote: The problem with vprof is SRCDS has become very verbose with bogus data table warnings and clients flooding server messages by running commands without args. There's con_logfile, but once that starts, it cannot be stopped. I had a 90MB~ log file after a couple hours, which isn't fun on an SSD. vprof needs to write files/dumps automatically to disk in order to become useful for servers. Thanks, Kyle. On 13 May 2013 10:09, Fletcher Dunn fletch...@valvesoftware.com wrote: Has anybody used vprof spike mode? This sort of problem is pretty much exactly what it was designed to investigate. E.g. vprof_on vprof_dump_spikes 20 // dump a vprof report if frame rate drops below 20 If the problem is caused by network traffic, using the -netspike command line argument can also be used to dig deeper into exactly what entity data is being sent. If you have a report and you don't know how to interpret it, you can email it to me or post it here, I'm sure there are some people who can help analyze it. -Original Message- From: hlds_linux-boun...@list.valvesoftware.com [mailto: hlds_linux-boun...@list.valvesoftware.com] On Behalf Of Bjorn Wielens Sent: Monday, May 13, 2013 9:42 AM To: Half-Life dedicated Win32 server mailing list; hlds_li...@list.valvesoftware.com Subject: Re: [hlds_linux] [hlds] TF2 server performance spikes Correct, I was one of the folks discussing this in the last round of messages and we run on Linux. Since my last message about having disabled replays I still get occasional hiccups in a full server, but nowhere near as bad as before, owing to the lack of replay overhead. I'd be happy to run diagnostics if they were available for Linux, but alas that's not the case. From: Alex Kowald abkow...@gmail.com To: Half-Life dedicated Win32 server mailing list hlds@list.valvesoftware.com Sent: Monday, May 13, 2013 1:16:04 PM Subject: Re: [hlds] TF2 server performance spikes You are only accepting traces for windows servers correct? Some of the people that have complained are running linux. I have the issue myself on two independent servers. We get this new game studder after upgrading, leaving everything else the same. On Mon, May 13, 2013 at 12:11 PM, Fletcher Dunn fletch...@valvesoftware.com wrote: I do have a theory about the cause of the CPU spikes. I’ll release a build with the next mandatory update (which will probably be this afternoon) that I think could address the problem. But since we are not reproducing the problem here, this is just a theory, not disciplined engineering. In order to conduct disciplined engineering, we need data from those who are able to reproduce it. Judging by the amount of discussion this problem has generated and the relatively small amount of effort that is required in order to get the data to us, I’m really surprised we haven’t received more traces. I have given out vtrace keys to every person who has asked for one. So far we have received traces from exactly two (2) server operators. One server operator had processor affinities set, the other was running a high number of instances per core, such that the average CPU utilization was high and there was no headroom for more than one process to spike. ___ To unsubscribe, edit your list preferences, or view the list archives, please visit: https://list.valvesoftware.com/cgi-bin/mailman/listinfo/hlds ___ To unsubscribe, edit your list preferences, or view the list archives, please visit: https://list.valvesoftware.com/cgi-bin/mailman/listinfo/hlds ___ To unsubscribe, edit your list preferences, or view the list archives, please visit: https://list.valvesoftware.com/cgi-bin/mailman/listinfo/hlds_linux ___ To unsubscribe, edit your list preferences, or view the list archives, please visit: https://list.valvesoftware.com/cgi-bin/mailman/listinfo/hlds ___ To unsubscribe, edit your list preferences, or view the list archives, please visit: https://list.valvesoftware.com/cgi-bin/mailman/listinfo/hlds
Re: [hlds] [hlds_linux] TF2 server performance spikes
There's no clean way to capture the output from vprof at the time of this writing. Messages flyby on a production server console. The only way you're able to capture any sort of output from the vprof_generatereport command (or whatever the generic name is) is to use con_logfile. However, once con_logfile is enabled, the logging cannot be stopped until the server dies. I tried using vprof when CS:S was moved to Steam|, and got burned pretty quickly due to uptime. (I'm on Linux) Thanks, Kyle. On Mon, May 13, 2013 at 10:16 AM, Fletcher Dunn fletch...@valvesoftware.com wrote: I’m not understanding how vprof is related to the console problems you described. Would you mind clarifying? ** ** If you’re reliably able to reproduce a spike, would it not be possible to turn on vprof spike mode, get a report, and then turn vprof off. If the spikes are really huge, then just set the threshold really low. ** ** *From:* hlds-boun...@list.valvesoftware.com [mailto: hlds-boun...@list.valvesoftware.com] *On Behalf Of *Kyle Sanderson *Sent:* Monday, May 13, 2013 10:14 AM *To:* Half-Life dedicated Win32 server mailing list *Subject:* Re: [hlds] [hlds_linux] TF2 server performance spikes ** ** The problem with vprof is SRCDS has become very verbose with bogus data table warnings and clients flooding server messages by running commands without args. There's con_logfile, but once that starts, it cannot be stopped. I had a 90MB~ log file after a couple hours, which isn't fun on an SSD. vprof needs to write files/dumps automatically to disk in order to become useful for servers. Thanks, Kyle. On 13 May 2013 10:09, Fletcher Dunn fletch...@valvesoftware.com wrote: Has anybody used vprof spike mode? This sort of problem is pretty much exactly what it was designed to investigate. E.g. vprof_on vprof_dump_spikes 20 // dump a vprof report if frame rate drops below 20 If the problem is caused by network traffic, using the -netspike command line argument can also be used to dig deeper into exactly what entity data is being sent. If you have a report and you don't know how to interpret it, you can email it to me or post it here, I'm sure there are some people who can help analyze it. -Original Message- From: hlds_linux-boun...@list.valvesoftware.com [mailto: hlds_linux-boun...@list.valvesoftware.com] On Behalf Of Bjorn Wielens Sent: Monday, May 13, 2013 9:42 AM To: Half-Life dedicated Win32 server mailing list; hlds_li...@list.valvesoftware.com Subject: Re: [hlds_linux] [hlds] TF2 server performance spikes Correct, I was one of the folks discussing this in the last round of messages and we run on Linux. Since my last message about having disabled replays I still get occasional hiccups in a full server, but nowhere near as bad as before, owing to the lack of replay overhead. I'd be happy to run diagnostics if they were available for Linux, but alas that's not the case. From: Alex Kowald abkow...@gmail.com To: Half-Life dedicated Win32 server mailing list hlds@list.valvesoftware.com Sent: Monday, May 13, 2013 1:16:04 PM Subject: Re: [hlds] TF2 server performance spikes You are only accepting traces for windows servers correct? Some of the people that have complained are running linux. I have the issue myself on two independent servers. We get this new game studder after upgrading, leaving everything else the same. On Mon, May 13, 2013 at 12:11 PM, Fletcher Dunn fletch...@valvesoftware.com wrote: I do have a theory about the cause of the CPU spikes. I’ll release a build with the next mandatory update (which will probably be this afternoon) that I think could address the problem. But since we are not reproducing the problem here, this is just a theory, not disciplined engineering. In order to conduct disciplined engineering, we need data from those who are able to reproduce it. Judging by the amount of discussion this problem has generated and the relatively small amount of effort that is required in order to get the data to us, I’m really surprised we haven’t received more traces. I have given out vtrace keys to every person who has asked for one. So far we have received traces from exactly two (2) server operators. One server operator had processor affinities set, the other was running a high number of instances per core, such that the average CPU utilization was high and there was no headroom for more than one process to spike. ___ To unsubscribe, edit your list preferences, or view the list archives, please visit: https://list.valvesoftware.com/cgi-bin/mailman/listinfo/hlds ___ To unsubscribe, edit your list preferences, or view the list archives, please visit: https://list.valvesoftware.com/cgi-bin/mailman/listinfo/hlds
Re: [hlds] [hlds_linux] TF2 server performance spikes
I will attempt to get a few vprof dumps and post them here. From: Fletcher Dunn fletch...@valvesoftware.com To: hlds_li...@list.valvesoftware.com hlds_li...@list.valvesoftware.com; Half-Life dedicated Win32 server mailing list hlds@list.valvesoftware.com Sent: Monday, May 13, 2013 2:09:14 PM Subject: Re: [hlds] [hlds_linux] TF2 server performance spikes Has anybody used vprof spike mode? This sort of problem is pretty much exactly what it was designed to investigate. E.g. vprof_on vprof_dump_spikes 20 // dump a vprof report if frame rate drops below 20 If the problem is caused by network traffic, using the -netspike command line argument can also be used to dig deeper into exactly what entity data is being sent. If you have a report and you don't know how to interpret it, you can email it to me or post it here, I'm sure there are some people who can help analyze it. -Original Message- From: hlds_linux-boun...@list.valvesoftware.com [mailto:hlds_linux-boun...@list.valvesoftware.com] On Behalf Of Bjorn Wielens Sent: Monday, May 13, 2013 9:42 AM To: Half-Life dedicated Win32 server mailing list; hlds_li...@list.valvesoftware.com Subject: Re: [hlds_linux] [hlds] TF2 server performance spikes Correct, I was one of the folks discussing this in the last round of messages and we run on Linux. Since my last message about having disabled replays I still get occasional hiccups in a full server, but nowhere near as bad as before, owing to the lack of replay overhead. I'd be happy to run diagnostics if they were available for Linux, but alas that's not the case. From: Alex Kowald abkow...@gmail.com To: Half-Life dedicated Win32 server mailing list hlds@list.valvesoftware.com Sent: Monday, May 13, 2013 1:16:04 PM Subject: Re: [hlds] TF2 server performance spikes You are only accepting traces for windows servers correct? Some of the people that have complained are running linux. I have the issue myself on two independent servers. We get this new game studder after upgrading, leaving everything else the same. On Mon, May 13, 2013 at 12:11 PM, Fletcher Dunn fletch...@valvesoftware.com wrote: I do have a theory about the cause of the CPU spikes. I’ll release a build with the next mandatory update (which will probably be this afternoon) that I think could address the problem. But since we are not reproducing the problem here, this is just a theory, not disciplined engineering. In order to conduct disciplined engineering, we need data from those who are able to reproduce it. Judging by the amount of discussion this problem has generated and the relatively small amount of effort that is required in order to get the data to us, I’m really surprised we haven’t received more traces. I have given out vtrace keys to every person who has asked for one. So far we have received traces from exactly two (2) server operators. One server operator had processor affinities set, the other was running a high number of instances per core, such that the average CPU utilization was high and there was no headroom for more than one process to spike. ___ To unsubscribe, edit your list preferences, or view the list archives, please visit: https://list.valvesoftware.com/cgi-bin/mailman/listinfo/hlds ___ To unsubscribe, edit your list preferences, or view the list archives, please visit: https://list.valvesoftware.com/cgi-bin/mailman/listinfo/hlds ___ To unsubscribe, edit your list preferences, or view the list archives, please visit: https://list.valvesoftware.com/cgi-bin/mailman/listinfo/hlds_linux ___ To unsubscribe, edit your list preferences, or view the list archives, please visit: https://list.valvesoftware.com/cgi-bin/mailman/listinfo/hlds___ To unsubscribe, edit your list preferences, or view the list archives, please visit: https://list.valvesoftware.com/cgi-bin/mailman/listinfo/hlds
Re: [hlds] [hlds_linux] TF2 server performance spikes
Oh my god Valve, the spikes are 10x worse now, what have you done lol. On Mon, May 13, 2013 at 1:38 PM, Bjorn Wielens uniac...@yahoo.ca wrote: I will attempt to get a few vprof dumps and post them here. -- *From:* Fletcher Dunn fletch...@valvesoftware.com *To:* hlds_li...@list.valvesoftware.com hlds_li...@list.valvesoftware.com; Half-Life dedicated Win32 server mailing list hlds@list.valvesoftware.com *Sent:* Monday, May 13, 2013 2:09:14 PM *Subject:* Re: [hlds] [hlds_linux] TF2 server performance spikes Has anybody used vprof spike mode? This sort of problem is pretty much exactly what it was designed to investigate. E.g. vprof_on vprof_dump_spikes 20 // dump a vprof report if frame rate drops below 20 If the problem is caused by network traffic, using the -netspike command line argument can also be used to dig deeper into exactly what entity data is being sent. If you have a report and you don't know how to interpret it, you can email it to me or post it here, I'm sure there are some people who can help analyze it. -Original Message- From: hlds_linux-boun...@list.valvesoftware.com [mailto: hlds_linux-boun...@list.valvesoftware.com] On Behalf Of Bjorn Wielens Sent: Monday, May 13, 2013 9:42 AM To: Half-Life dedicated Win32 server mailing list; hlds_li...@list.valvesoftware.com Subject: Re: [hlds_linux] [hlds] TF2 server performance spikes Correct, I was one of the folks discussing this in the last round of messages and we run on Linux. Since my last message about having disabled replays I still get occasional hiccups in a full server, but nowhere near as bad as before, owing to the lack of replay overhead. I'd be happy to run diagnostics if they were available for Linux, but alas that's not the case. From: Alex Kowald abkow...@gmail.com To: Half-Life dedicated Win32 server mailing list hlds@list.valvesoftware.com Sent: Monday, May 13, 2013 1:16:04 PM Subject: Re: [hlds] TF2 server performance spikes You are only accepting traces for windows servers correct? Some of the people that have complained are running linux. I have the issue myself on two independent servers. We get this new game studder after upgrading, leaving everything else the same. On Mon, May 13, 2013 at 12:11 PM, Fletcher Dunn fletch...@valvesoftware.com wrote: I do have a theory about the cause of the CPU spikes. I’ll release a build with the next mandatory update (which will probably be this afternoon) that I think could address the problem. But since we are not reproducing the problem here, this is just a theory, not disciplined engineering. In order to conduct disciplined engineering, we need data from those who are able to reproduce it. Judging by the amount of discussion this problem has generated and the relatively small amount of effort that is required in order to get the data to us, I’m really surprised we haven’t received more traces. I have given out vtrace keys to every person who has asked for one. So far we have received traces from exactly two (2) server operators. One server operator had processor affinities set, the other was running a high number of instances per core, such that the average CPU utilization was high and there was no headroom for more than one process to spike. ___ To unsubscribe, edit your list preferences, or view the list archives, please visit: https://list.valvesoftware.com/cgi-bin/mailman/listinfo/hlds ___ To unsubscribe, edit your list preferences, or view the list archives, please visit: https://list.valvesoftware.com/cgi-bin/mailman/listinfo/hlds ___ To unsubscribe, edit your list preferences, or view the list archives, please visit: https://list.valvesoftware.com/cgi-bin/mailman/listinfo/hlds_linux ___ To unsubscribe, edit your list preferences, or view the list archives, please visit: https://list.valvesoftware.com/cgi-bin/mailman/listinfo/hlds ___ To unsubscribe, edit your list preferences, or view the list archives, please visit: https://list.valvesoftware.com/cgi-bin/mailman/listinfo/hlds ___ To unsubscribe, edit your list preferences, or view the list archives, please visit: https://list.valvesoftware.com/cgi-bin/mailman/listinfo/hlds
Re: [hlds] [hlds_linux] TF2 server performance spikes
Did you update? for the past hour or so I've been getting complaints about spikes, I'm thinking it's something non-threaded using the Steam API. I noticed Avatars have been working since the change, but I'm just pulling at straws now. vprof hasn't rendered anything useful. Here's a vprof dump after a while (I wrote something to script most of the captures, my goodness vprof takes a long time to print). http://fpaste.org/11930/13684786/ Thanks, Kyle. On Mon, May 13, 2013 at 1:47 PM, Essay Tew Phaun sc2p...@gmail.com wrote: Oh my god Valve, the spikes are 10x worse now, what have you done lol. On Mon, May 13, 2013 at 1:38 PM, Bjorn Wielens uniac...@yahoo.ca wrote: I will attempt to get a few vprof dumps and post them here. -- *From:* Fletcher Dunn fletch...@valvesoftware.com *To:* hlds_li...@list.valvesoftware.com hlds_li...@list.valvesoftware.com; Half-Life dedicated Win32 server mailing list hlds@list.valvesoftware.com *Sent:* Monday, May 13, 2013 2:09:14 PM *Subject:* Re: [hlds] [hlds_linux] TF2 server performance spikes Has anybody used vprof spike mode? This sort of problem is pretty much exactly what it was designed to investigate. E.g. vprof_on vprof_dump_spikes 20 // dump a vprof report if frame rate drops below 20 If the problem is caused by network traffic, using the -netspike command line argument can also be used to dig deeper into exactly what entity data is being sent. If you have a report and you don't know how to interpret it, you can email it to me or post it here, I'm sure there are some people who can help analyze it. -Original Message- From: hlds_linux-boun...@list.valvesoftware.com [mailto: hlds_linux-boun...@list.valvesoftware.com] On Behalf Of Bjorn Wielens Sent: Monday, May 13, 2013 9:42 AM To: Half-Life dedicated Win32 server mailing list; hlds_li...@list.valvesoftware.com Subject: Re: [hlds_linux] [hlds] TF2 server performance spikes Correct, I was one of the folks discussing this in the last round of messages and we run on Linux. Since my last message about having disabled replays I still get occasional hiccups in a full server, but nowhere near as bad as before, owing to the lack of replay overhead. I'd be happy to run diagnostics if they were available for Linux, but alas that's not the case. From: Alex Kowald abkow...@gmail.com To: Half-Life dedicated Win32 server mailing list hlds@list.valvesoftware.com Sent: Monday, May 13, 2013 1:16:04 PM Subject: Re: [hlds] TF2 server performance spikes You are only accepting traces for windows servers correct? Some of the people that have complained are running linux. I have the issue myself on two independent servers. We get this new game studder after upgrading, leaving everything else the same. On Mon, May 13, 2013 at 12:11 PM, Fletcher Dunn fletch...@valvesoftware.com wrote: I do have a theory about the cause of the CPU spikes. I’ll release a build with the next mandatory update (which will probably be this afternoon) that I think could address the problem. But since we are not reproducing the problem here, this is just a theory, not disciplined engineering. In order to conduct disciplined engineering, we need data from those who are able to reproduce it. Judging by the amount of discussion this problem has generated and the relatively small amount of effort that is required in order to get the data to us, I’m really surprised we haven’t received more traces. I have given out vtrace keys to every person who has asked for one. So far we have received traces from exactly two (2) server operators. One server operator had processor affinities set, the other was running a high number of instances per core, such that the average CPU utilization was high and there was no headroom for more than one process to spike. ___ To unsubscribe, edit your list preferences, or view the list archives, please visit: https://list.valvesoftware.com/cgi-bin/mailman/listinfo/hlds ___ To unsubscribe, edit your list preferences, or view the list archives, please visit: https://list.valvesoftware.com/cgi-bin/mailman/listinfo/hlds ___ To unsubscribe, edit your list preferences, or view the list archives, please visit: https://list.valvesoftware.com/cgi-bin/mailman/listinfo/hlds_linux ___ To unsubscribe, edit your list preferences, or view the list archives, please visit: https://list.valvesoftware.com/cgi-bin/mailman/listinfo/hlds ___ To unsubscribe, edit your list preferences, or view the list archives, please visit: https://list.valvesoftware.com/cgi-bin/mailman/listinfo/hlds
Re: [hlds] [hlds_linux] TF2 server performance spikes
Yeah we've updated, it's just a lot worse than it was. It seems to be happening much more frequently now. Attempting to get a vprof dump myself, but we'll see. On Mon, May 13, 2013 at 4:58 PM, Kyle Sanderson kyle.l...@gmail.com wrote: Did you update? for the past hour or so I've been getting complaints about spikes, I'm thinking it's something non-threaded using the Steam API. I noticed Avatars have been working since the change, but I'm just pulling at straws now. vprof hasn't rendered anything useful. Here's a vprof dump after a while (I wrote something to script most of the captures, my goodness vprof takes a long time to print). http://fpaste.org/11930/13684786/ Thanks, Kyle. On Mon, May 13, 2013 at 1:47 PM, Essay Tew Phaun sc2p...@gmail.comwrote: Oh my god Valve, the spikes are 10x worse now, what have you done lol. On Mon, May 13, 2013 at 1:38 PM, Bjorn Wielens uniac...@yahoo.ca wrote: I will attempt to get a few vprof dumps and post them here. -- *From:* Fletcher Dunn fletch...@valvesoftware.com *To:* hlds_li...@list.valvesoftware.com hlds_li...@list.valvesoftware.com; Half-Life dedicated Win32 server mailing list hlds@list.valvesoftware.com *Sent:* Monday, May 13, 2013 2:09:14 PM *Subject:* Re: [hlds] [hlds_linux] TF2 server performance spikes Has anybody used vprof spike mode? This sort of problem is pretty much exactly what it was designed to investigate. E.g. vprof_on vprof_dump_spikes 20 // dump a vprof report if frame rate drops below 20 If the problem is caused by network traffic, using the -netspike command line argument can also be used to dig deeper into exactly what entity data is being sent. If you have a report and you don't know how to interpret it, you can email it to me or post it here, I'm sure there are some people who can help analyze it. -Original Message- From: hlds_linux-boun...@list.valvesoftware.com [mailto: hlds_linux-boun...@list.valvesoftware.com] On Behalf Of Bjorn Wielens Sent: Monday, May 13, 2013 9:42 AM To: Half-Life dedicated Win32 server mailing list; hlds_li...@list.valvesoftware.com Subject: Re: [hlds_linux] [hlds] TF2 server performance spikes Correct, I was one of the folks discussing this in the last round of messages and we run on Linux. Since my last message about having disabled replays I still get occasional hiccups in a full server, but nowhere near as bad as before, owing to the lack of replay overhead. I'd be happy to run diagnostics if they were available for Linux, but alas that's not the case. From: Alex Kowald abkow...@gmail.com To: Half-Life dedicated Win32 server mailing list hlds@list.valvesoftware.com Sent: Monday, May 13, 2013 1:16:04 PM Subject: Re: [hlds] TF2 server performance spikes You are only accepting traces for windows servers correct? Some of the people that have complained are running linux. I have the issue myself on two independent servers. We get this new game studder after upgrading, leaving everything else the same. On Mon, May 13, 2013 at 12:11 PM, Fletcher Dunn fletch...@valvesoftware.com wrote: I do have a theory about the cause of the CPU spikes. I’ll release a build with the next mandatory update (which will probably be this afternoon) that I think could address the problem. But since we are not reproducing the problem here, this is just a theory, not disciplined engineering. In order to conduct disciplined engineering, we need data from those who are able to reproduce it. Judging by the amount of discussion this problem has generated and the relatively small amount of effort that is required in order to get the data to us, I’m really surprised we haven’t received more traces. I have given out vtrace keys to every person who has asked for one. So far we have received traces from exactly two (2) server operators. One server operator had processor affinities set, the other was running a high number of instances per core, such that the average CPU utilization was high and there was no headroom for more than one process to spike. ___ To unsubscribe, edit your list preferences, or view the list archives, please visit: https://list.valvesoftware.com/cgi-bin/mailman/listinfo/hlds ___ To unsubscribe, edit your list preferences, or view the list archives, please visit: https://list.valvesoftware.com/cgi-bin/mailman/listinfo/hlds ___ To unsubscribe, edit your list preferences, or view the list archives, please visit: https://list.valvesoftware.com/cgi-bin/mailman/listinfo/hlds_linux ___ To unsubscribe, edit your list preferences, or view the list archives, please visit: https://list.valvesoftware.com/cgi-bin/mailman/listinfo/hlds
Re: [hlds] [hlds_linux] TF2 server performance spikes
That trace doesn’t appear to be produced as a result of vprof_dump_spikes. That is a profile of average performance over about an hour’s worth of uptime. 314391 frames sampled for 4718.71 seconds Average 66.63 fps, 15.01 ms per frame Peak 3131.67 ms frame 99 pct of time accounted for When spike mode is active, it will clear the profile data each frame, and then dump it if the frame time exceeds some threshold. (You actually specify the reciprocal of the threshold, in hz.) From: hlds-boun...@list.valvesoftware.com [mailto:hlds-boun...@list.valvesoftware.com] On Behalf Of Kyle Sanderson Sent: Monday, May 13, 2013 1:59 PM To: Half-Life dedicated Win32 server mailing list Subject: Re: [hlds] [hlds_linux] TF2 server performance spikes Did you update? for the past hour or so I've been getting complaints about spikes, I'm thinking it's something non-threaded using the Steam API. I noticed Avatars have been working since the change, but I'm just pulling at straws now. vprof hasn't rendered anything useful. Here's a vprof dump after a while (I wrote something to script most of the captures, my goodness vprof takes a long time to print). http://fpaste.org/11930/13684786/ Thanks, Kyle. On Mon, May 13, 2013 at 1:47 PM, Essay Tew Phaun sc2p...@gmail.commailto:sc2p...@gmail.com wrote: Oh my god Valve, the spikes are 10x worse now, what have you done lol. On Mon, May 13, 2013 at 1:38 PM, Bjorn Wielens uniac...@yahoo.camailto:uniac...@yahoo.ca wrote: I will attempt to get a few vprof dumps and post them here. From: Fletcher Dunn fletch...@valvesoftware.commailto:fletch...@valvesoftware.com To: hlds_li...@list.valvesoftware.commailto:hlds_li...@list.valvesoftware.com hlds_li...@list.valvesoftware.commailto:hlds_li...@list.valvesoftware.com; Half-Life dedicated Win32 server mailing list hlds@list.valvesoftware.commailto:hlds@list.valvesoftware.com Sent: Monday, May 13, 2013 2:09:14 PM Subject: Re: [hlds] [hlds_linux] TF2 server performance spikes Has anybody used vprof spike mode? This sort of problem is pretty much exactly what it was designed to investigate. E.g. vprof_on vprof_dump_spikes 20 // dump a vprof report if frame rate drops below 20 If the problem is caused by network traffic, using the -netspike command line argument can also be used to dig deeper into exactly what entity data is being sent. If you have a report and you don't know how to interpret it, you can email it to me or post it here, I'm sure there are some people who can help analyze it. -Original Message- From: hlds_linux-boun...@list.valvesoftware.commailto:hlds_linux-boun...@list.valvesoftware.com [mailto:hlds_linux-boun...@list.valvesoftware.commailto:hlds_linux-boun...@list.valvesoftware.com] On Behalf Of Bjorn Wielens Sent: Monday, May 13, 2013 9:42 AM To: Half-Life dedicated Win32 server mailing list; hlds_li...@list.valvesoftware.commailto:hlds_li...@list.valvesoftware.com Subject: Re: [hlds_linux] [hlds] TF2 server performance spikes Correct, I was one of the folks discussing this in the last round of messages and we run on Linux. Since my last message about having disabled replays I still get occasional hiccups in a full server, but nowhere near as bad as before, owing to the lack of replay overhead. I'd be happy to run diagnostics if they were available for Linux, but alas that's not the case. From: Alex Kowald abkow...@gmail.commailto:abkow...@gmail.com To: Half-Life dedicated Win32 server mailing list hlds@list.valvesoftware.commailto:hlds@list.valvesoftware.com Sent: Monday, May 13, 2013 1:16:04 PM Subject: Re: [hlds] TF2 server performance spikes You are only accepting traces for windows servers correct? Some of the people that have complained are running linux. I have the issue myself on two independent servers. We get this new game studder after upgrading, leaving everything else the same. On Mon, May 13, 2013 at 12:11 PM, Fletcher Dunn fletch...@valvesoftware.commailto:fletch...@valvesoftware.com wrote: I do have a theory about the cause of the CPU spikes. I’ll release a build with the next mandatory update (which will probably be this afternoon) that I think could address the problem. But since we are not reproducing the problem here, this is just a theory, not disciplined engineering. In order to conduct disciplined engineering, we need data from those who are able to reproduce it. Judging by the amount of discussion this problem has generated and the relatively small amount of effort that is required in order to get the data to us, I’m really surprised we haven’t received more traces. I have given out vtrace keys to every person who has asked for one. So far we have received traces from exactly two (2) server operators. One server operator had processor affinities set, the other was running a high number of instances per core, such that the average CPU utilization was high
Re: [hlds] [hlds_linux] TF2 server performance spikes
Can these both be enabled at any point during the server running or must they be specified at startup? I've enabled both and seen 2 servers already dip below and nothing has been generated. On Mon, May 13, 2013 at 5:09 PM, Fletcher Dunn fletch...@valvesoftware.comwrote: That trace doesn’t appear to be produced as a result of vprof_dump_spikes. That is a profile of average performance over about an hour’s worth of uptime. ** ** 314391 frames sampled for 4718.71 seconds Average 66.63 fps, 15.01 ms per frame Peak 3131.67 ms frame 99 pct of time accounted for ** ** When spike mode is active, it will clear the profile data each frame, and then dump it if the frame time exceeds some threshold. (You actually specify the reciprocal of the threshold, in hz.) ** ** ** ** *From:* hlds-boun...@list.valvesoftware.com [mailto: hlds-boun...@list.valvesoftware.com] *On Behalf Of *Kyle Sanderson *Sent:* Monday, May 13, 2013 1:59 PM *To:* Half-Life dedicated Win32 server mailing list *Subject:* Re: [hlds] [hlds_linux] TF2 server performance spikes ** ** Did you update? for the past hour or so I've been getting complaints about spikes, I'm thinking it's something non-threaded using the Steam API. I noticed Avatars have been working since the change, but I'm just pulling at straws now. vprof hasn't rendered anything useful. Here's a vprof dump after a while (I wrote something to script most of the captures, my goodness vprof takes a long time to print). http://fpaste.org/11930/13684786/ Thanks, Kyle. ** ** On Mon, May 13, 2013 at 1:47 PM, Essay Tew Phaun sc2p...@gmail.com wrote: Oh my god Valve, the spikes are 10x worse now, what have you done lol. ** ** On Mon, May 13, 2013 at 1:38 PM, Bjorn Wielens uniac...@yahoo.ca wrote:* *** I will attempt to get a few vprof dumps and post them here. ** ** -- *From:* Fletcher Dunn fletch...@valvesoftware.com *To:* hlds_li...@list.valvesoftware.com hlds_li...@list.valvesoftware.com; Half-Life dedicated Win32 server mailing list hlds@list.valvesoftware.com *Sent:* Monday, May 13, 2013 2:09:14 PM *Subject:* Re: [hlds] [hlds_linux] TF2 server performance spikes Has anybody used vprof spike mode? This sort of problem is pretty much exactly what it was designed to investigate. E.g. vprof_on vprof_dump_spikes 20 // dump a vprof report if frame rate drops below 20 If the problem is caused by network traffic, using the -netspike command line argument can also be used to dig deeper into exactly what entity data is being sent. If you have a report and you don't know how to interpret it, you can email it to me or post it here, I'm sure there are some people who can help analyze it. -Original Message- From: hlds_linux-boun...@list.valvesoftware.com [mailto: hlds_linux-boun...@list.valvesoftware.com] On Behalf Of Bjorn Wielens Sent: Monday, May 13, 2013 9:42 AM To: Half-Life dedicated Win32 server mailing list; hlds_li...@list.valvesoftware.com Subject: Re: [hlds_linux] [hlds] TF2 server performance spikes Correct, I was one of the folks discussing this in the last round of messages and we run on Linux. Since my last message about having disabled replays I still get occasional hiccups in a full server, but nowhere near as bad as before, owing to the lack of replay overhead. I'd be happy to run diagnostics if they were available for Linux, but alas that's not the case. From: Alex Kowald abkow...@gmail.com To: Half-Life dedicated Win32 server mailing list hlds@list.valvesoftware.com Sent: Monday, May 13, 2013 1:16:04 PM Subject: Re: [hlds] TF2 server performance spikes You are only accepting traces for windows servers correct? Some of the people that have complained are running linux. I have the issue myself on two independent servers. We get this new game studder after upgrading, leaving everything else the same. On Mon, May 13, 2013 at 12:11 PM, Fletcher Dunn fletch...@valvesoftware.com wrote: I do have a theory about the cause of the CPU spikes. I’ll release a build with the next mandatory update (which will probably be this afternoon) that I think could address the problem. But since we are not reproducing the problem here, this is just a theory, not disciplined engineering. In order to conduct disciplined engineering, we need data from those who are able to reproduce it. Judging by the amount of discussion this problem has generated and the relatively small amount of effort that is required in order to get the data to us, I’m really surprised we haven’t received more traces. I have given out vtrace keys to every person who has asked for one. So far we have received traces from exactly two (2) server operators. One server operator had processor affinities set, the other was running a high number
Re: [hlds] [hlds_linux] TF2 server performance spikes
You should be able to enable it at any time. You can test if it's working by executing the spike command, which just forces the server frame rate to stall for a second or so. From: hlds-boun...@list.valvesoftware.com [mailto:hlds-boun...@list.valvesoftware.com] On Behalf Of Essay Tew Phaun Sent: Monday, May 13, 2013 2:16 PM To: Half-Life dedicated Win32 server mailing list Subject: Re: [hlds] [hlds_linux] TF2 server performance spikes Can these both be enabled at any point during the server running or must they be specified at startup? I've enabled both and seen 2 servers already dip below and nothing has been generated. On Mon, May 13, 2013 at 5:09 PM, Fletcher Dunn fletch...@valvesoftware.commailto:fletch...@valvesoftware.com wrote: That trace doesn't appear to be produced as a result of vprof_dump_spikes. That is a profile of average performance over about an hour's worth of uptime. 314391 frames sampled for 4718.71 seconds Average 66.63 fps, 15.01 ms per frame Peak 3131.67 ms frame 99 pct of time accounted for When spike mode is active, it will clear the profile data each frame, and then dump it if the frame time exceeds some threshold. (You actually specify the reciprocal of the threshold, in hz.) From: hlds-boun...@list.valvesoftware.commailto:hlds-boun...@list.valvesoftware.com [mailto:hlds-boun...@list.valvesoftware.commailto:hlds-boun...@list.valvesoftware.com] On Behalf Of Kyle Sanderson Sent: Monday, May 13, 2013 1:59 PM To: Half-Life dedicated Win32 server mailing list Subject: Re: [hlds] [hlds_linux] TF2 server performance spikes Did you update? for the past hour or so I've been getting complaints about spikes, I'm thinking it's something non-threaded using the Steam API. I noticed Avatars have been working since the change, but I'm just pulling at straws now. vprof hasn't rendered anything useful. Here's a vprof dump after a while (I wrote something to script most of the captures, my goodness vprof takes a long time to print). http://fpaste.org/11930/13684786/ Thanks, Kyle. On Mon, May 13, 2013 at 1:47 PM, Essay Tew Phaun sc2p...@gmail.commailto:sc2p...@gmail.com wrote: Oh my god Valve, the spikes are 10x worse now, what have you done lol. On Mon, May 13, 2013 at 1:38 PM, Bjorn Wielens uniac...@yahoo.camailto:uniac...@yahoo.ca wrote: I will attempt to get a few vprof dumps and post them here. From: Fletcher Dunn fletch...@valvesoftware.commailto:fletch...@valvesoftware.com To: hlds_li...@list.valvesoftware.commailto:hlds_li...@list.valvesoftware.com hlds_li...@list.valvesoftware.commailto:hlds_li...@list.valvesoftware.com; Half-Life dedicated Win32 server mailing list hlds@list.valvesoftware.commailto:hlds@list.valvesoftware.com Sent: Monday, May 13, 2013 2:09:14 PM Subject: Re: [hlds] [hlds_linux] TF2 server performance spikes Has anybody used vprof spike mode? This sort of problem is pretty much exactly what it was designed to investigate. E.g. vprof_on vprof_dump_spikes 20 // dump a vprof report if frame rate drops below 20 If the problem is caused by network traffic, using the -netspike command line argument can also be used to dig deeper into exactly what entity data is being sent. If you have a report and you don't know how to interpret it, you can email it to me or post it here, I'm sure there are some people who can help analyze it. -Original Message- From: hlds_linux-boun...@list.valvesoftware.commailto:hlds_linux-boun...@list.valvesoftware.com [mailto:hlds_linux-boun...@list.valvesoftware.commailto:hlds_linux-boun...@list.valvesoftware.com] On Behalf Of Bjorn Wielens Sent: Monday, May 13, 2013 9:42 AM To: Half-Life dedicated Win32 server mailing list; hlds_li...@list.valvesoftware.commailto:hlds_li...@list.valvesoftware.com Subject: Re: [hlds_linux] [hlds] TF2 server performance spikes Correct, I was one of the folks discussing this in the last round of messages and we run on Linux. Since my last message about having disabled replays I still get occasional hiccups in a full server, but nowhere near as bad as before, owing to the lack of replay overhead. I'd be happy to run diagnostics if they were available for Linux, but alas that's not the case. From: Alex Kowald abkow...@gmail.commailto:abkow...@gmail.com To: Half-Life dedicated Win32 server mailing list hlds@list.valvesoftware.commailto:hlds@list.valvesoftware.com Sent: Monday, May 13, 2013 1:16:04 PM Subject: Re: [hlds] TF2 server performance spikes You are only accepting traces for windows servers correct? Some of the people that have complained are running linux. I have the issue myself on two independent servers. We get this new game studder after upgrading, leaving everything else the same. On Mon, May 13, 2013 at 12:11 PM, Fletcher Dunn fletch...@valvesoftware.commailto:fletch...@valvesoftware.com wrote: I do have a theory about the cause of the CPU spikes. I'll
Re: [hlds] [hlds_linux] TF2 server performance spikes
cheat protected -_-. I do know that the server dropped down into the red and nothing was generated. Are they stored in the vprof directory or within the root of the game installation? On Mon, May 13, 2013 at 5:19 PM, Fletcher Dunn fletch...@valvesoftware.comwrote: You should be able to enable it at any time. ** ** You can test if it’s working by executing the “spike” command, which just forces the server frame rate to stall for a second or so. ** ** *From:* hlds-boun...@list.valvesoftware.com [mailto: hlds-boun...@list.valvesoftware.com] *On Behalf Of *Essay Tew Phaun *Sent:* Monday, May 13, 2013 2:16 PM *To:* Half-Life dedicated Win32 server mailing list *Subject:* Re: [hlds] [hlds_linux] TF2 server performance spikes ** ** Can these both be enabled at any point during the server running or must they be specified at startup? I've enabled both and seen 2 servers already dip below and nothing has been generated. ** ** On Mon, May 13, 2013 at 5:09 PM, Fletcher Dunn fletch...@valvesoftware.com wrote: That trace doesn’t appear to be produced as a result of vprof_dump_spikes. That is a profile of average performance over about an hour’s worth of uptime. 314391 frames sampled for 4718.71 seconds Average 66.63 fps, 15.01 ms per frame Peak 3131.67 ms frame 99 pct of time accounted for When spike mode is active, it will clear the profile data each frame, and then dump it if the frame time exceeds some threshold. (You actually specify the reciprocal of the threshold, in hz.) *From:* hlds-boun...@list.valvesoftware.com [mailto: hlds-boun...@list.valvesoftware.com] *On Behalf Of *Kyle Sanderson *Sent:* Monday, May 13, 2013 1:59 PM *To:* Half-Life dedicated Win32 server mailing list *Subject:* Re: [hlds] [hlds_linux] TF2 server performance spikes Did you update? for the past hour or so I've been getting complaints about spikes, I'm thinking it's something non-threaded using the Steam API. I noticed Avatars have been working since the change, but I'm just pulling at straws now. vprof hasn't rendered anything useful. Here's a vprof dump after a while (I wrote something to script most of the captures, my goodness vprof takes a long time to print). http://fpaste.org/11930/13684786/ Thanks, Kyle. On Mon, May 13, 2013 at 1:47 PM, Essay Tew Phaun sc2p...@gmail.com wrote: Oh my god Valve, the spikes are 10x worse now, what have you done lol. On Mon, May 13, 2013 at 1:38 PM, Bjorn Wielens uniac...@yahoo.ca wrote:* *** I will attempt to get a few vprof dumps and post them here. -- *From:* Fletcher Dunn fletch...@valvesoftware.com *To:* hlds_li...@list.valvesoftware.com hlds_li...@list.valvesoftware.com; Half-Life dedicated Win32 server mailing list hlds@list.valvesoftware.com *Sent:* Monday, May 13, 2013 2:09:14 PM *Subject:* Re: [hlds] [hlds_linux] TF2 server performance spikes Has anybody used vprof spike mode? This sort of problem is pretty much exactly what it was designed to investigate. E.g. vprof_on vprof_dump_spikes 20 // dump a vprof report if frame rate drops below 20 If the problem is caused by network traffic, using the -netspike command line argument can also be used to dig deeper into exactly what entity data is being sent. If you have a report and you don't know how to interpret it, you can email it to me or post it here, I'm sure there are some people who can help analyze it. -Original Message- From: hlds_linux-boun...@list.valvesoftware.com [mailto: hlds_linux-boun...@list.valvesoftware.com] On Behalf Of Bjorn Wielens Sent: Monday, May 13, 2013 9:42 AM To: Half-Life dedicated Win32 server mailing list; hlds_li...@list.valvesoftware.com Subject: Re: [hlds_linux] [hlds] TF2 server performance spikes Correct, I was one of the folks discussing this in the last round of messages and we run on Linux. Since my last message about having disabled replays I still get occasional hiccups in a full server, but nowhere near as bad as before, owing to the lack of replay overhead. I'd be happy to run diagnostics if they were available for Linux, but alas that's not the case. From: Alex Kowald abkow...@gmail.com To: Half-Life dedicated Win32 server mailing list hlds@list.valvesoftware.com Sent: Monday, May 13, 2013 1:16:04 PM Subject: Re: [hlds] TF2 server performance spikes You are only accepting traces for windows servers correct? Some of the people that have complained are running linux. I have the issue myself on two independent servers. We get this new game studder after upgrading, leaving everything else the same. On Mon, May 13, 2013 at 12:11 PM, Fletcher Dunn fletch...@valvesoftware.com wrote: I do have a theory about the cause of the CPU spikes
Re: [hlds] [hlds_linux] TF2 server performance spikes
You need to watch your console for vprof messages flying by and capture the text some how. I think I saw one once, scrolled up to copy it, and it was gone in a flash. You can try using screen to log the message if you're on *nix, otherwise you can try your hand at using con_logfile, but be aware, it kills. Thanks, Kyle. On Mon, May 13, 2013 at 2:27 PM, Essay Tew Phaun sc2p...@gmail.com wrote: cheat protected -_-. I do know that the server dropped down into the red and nothing was generated. Are they stored in the vprof directory or within the root of the game installation? On Mon, May 13, 2013 at 5:19 PM, Fletcher Dunn fletch...@valvesoftware.com wrote: You should be able to enable it at any time. ** ** You can test if it’s working by executing the “spike” command, which just forces the server frame rate to stall for a second or so. ** ** *From:* hlds-boun...@list.valvesoftware.com [mailto: hlds-boun...@list.valvesoftware.com] *On Behalf Of *Essay Tew Phaun *Sent:* Monday, May 13, 2013 2:16 PM *To:* Half-Life dedicated Win32 server mailing list *Subject:* Re: [hlds] [hlds_linux] TF2 server performance spikes ** ** Can these both be enabled at any point during the server running or must they be specified at startup? I've enabled both and seen 2 servers already dip below and nothing has been generated. ** ** On Mon, May 13, 2013 at 5:09 PM, Fletcher Dunn fletch...@valvesoftware.com wrote: That trace doesn’t appear to be produced as a result of vprof_dump_spikes. That is a profile of average performance over about an hour’s worth of uptime. 314391 frames sampled for 4718.71 seconds Average 66.63 fps, 15.01 ms per frame Peak 3131.67 ms frame 99 pct of time accounted for When spike mode is active, it will clear the profile data each frame, and then dump it if the frame time exceeds some threshold. (You actually specify the reciprocal of the threshold, in hz.) *From:* hlds-boun...@list.valvesoftware.com [mailto: hlds-boun...@list.valvesoftware.com] *On Behalf Of *Kyle Sanderson *Sent:* Monday, May 13, 2013 1:59 PM *To:* Half-Life dedicated Win32 server mailing list *Subject:* Re: [hlds] [hlds_linux] TF2 server performance spikes Did you update? for the past hour or so I've been getting complaints about spikes, I'm thinking it's something non-threaded using the Steam API. I noticed Avatars have been working since the change, but I'm just pulling at straws now. vprof hasn't rendered anything useful. Here's a vprof dump after a while (I wrote something to script most of the captures, my goodness vprof takes a long time to print). http://fpaste.org/11930/13684786/ Thanks, Kyle. On Mon, May 13, 2013 at 1:47 PM, Essay Tew Phaun sc2p...@gmail.com wrote: Oh my god Valve, the spikes are 10x worse now, what have you done lol.*** * On Mon, May 13, 2013 at 1:38 PM, Bjorn Wielens uniac...@yahoo.ca wrote: I will attempt to get a few vprof dumps and post them here. -- *From:* Fletcher Dunn fletch...@valvesoftware.com *To:* hlds_li...@list.valvesoftware.com hlds_li...@list.valvesoftware.com; Half-Life dedicated Win32 server mailing list hlds@list.valvesoftware.com *Sent:* Monday, May 13, 2013 2:09:14 PM *Subject:* Re: [hlds] [hlds_linux] TF2 server performance spikes Has anybody used vprof spike mode? This sort of problem is pretty much exactly what it was designed to investigate. E.g. vprof_on vprof_dump_spikes 20 // dump a vprof report if frame rate drops below 20 If the problem is caused by network traffic, using the -netspike command line argument can also be used to dig deeper into exactly what entity data is being sent. If you have a report and you don't know how to interpret it, you can email it to me or post it here, I'm sure there are some people who can help analyze it. -Original Message- From: hlds_linux-boun...@list.valvesoftware.com [mailto: hlds_linux-boun...@list.valvesoftware.com] On Behalf Of Bjorn Wielens Sent: Monday, May 13, 2013 9:42 AM To: Half-Life dedicated Win32 server mailing list; hlds_li...@list.valvesoftware.com Subject: Re: [hlds_linux] [hlds] TF2 server performance spikes Correct, I was one of the folks discussing this in the last round of messages and we run on Linux. Since my last message about having disabled replays I still get occasional hiccups in a full server, but nowhere near as bad as before, owing to the lack of replay overhead. I'd be happy to run diagnostics if they were available for Linux, but alas that's not the case. From: Alex Kowald abkow...@gmail.com To: Half-Life dedicated Win32 server mailing list hlds@list.valvesoftware.com Sent: Monday, May 13, 2013 1:16:04 PM Subject: Re: [hlds] TF2 server performance
Re: [hlds] [hlds_linux] TF2 server performance spikes
Wonderful... :/ On Mon, May 13, 2013 at 5:32 PM, Kyle Sanderson kyle.l...@gmail.com wrote: You need to watch your console for vprof messages flying by and capture the text some how. I think I saw one once, scrolled up to copy it, and it was gone in a flash. You can try using screen to log the message if you're on *nix, otherwise you can try your hand at using con_logfile, but be aware, it kills. Thanks, Kyle. On Mon, May 13, 2013 at 2:27 PM, Essay Tew Phaun sc2p...@gmail.comwrote: cheat protected -_-. I do know that the server dropped down into the red and nothing was generated. Are they stored in the vprof directory or within the root of the game installation? On Mon, May 13, 2013 at 5:19 PM, Fletcher Dunn fletch...@valvesoftware.com wrote: You should be able to enable it at any time. ** ** You can test if it’s working by executing the “spike” command, which just forces the server frame rate to stall for a second or so. ** ** *From:* hlds-boun...@list.valvesoftware.com [mailto: hlds-boun...@list.valvesoftware.com] *On Behalf Of *Essay Tew Phaun *Sent:* Monday, May 13, 2013 2:16 PM *To:* Half-Life dedicated Win32 server mailing list *Subject:* Re: [hlds] [hlds_linux] TF2 server performance spikes ** ** Can these both be enabled at any point during the server running or must they be specified at startup? I've enabled both and seen 2 servers already dip below and nothing has been generated. ** ** On Mon, May 13, 2013 at 5:09 PM, Fletcher Dunn fletch...@valvesoftware.com wrote: That trace doesn’t appear to be produced as a result of vprof_dump_spikes. That is a profile of average performance over about an hour’s worth of uptime. 314391 frames sampled for 4718.71 seconds Average 66.63 fps, 15.01 ms per frame Peak 3131.67 ms frame 99 pct of time accounted for When spike mode is active, it will clear the profile data each frame, and then dump it if the frame time exceeds some threshold. (You actually specify the reciprocal of the threshold, in hz.) *From:* hlds-boun...@list.valvesoftware.com [mailto: hlds-boun...@list.valvesoftware.com] *On Behalf Of *Kyle Sanderson *Sent:* Monday, May 13, 2013 1:59 PM *To:* Half-Life dedicated Win32 server mailing list *Subject:* Re: [hlds] [hlds_linux] TF2 server performance spikes Did you update? for the past hour or so I've been getting complaints about spikes, I'm thinking it's something non-threaded using the Steam API. I noticed Avatars have been working since the change, but I'm just pulling at straws now. vprof hasn't rendered anything useful. Here's a vprof dump after a while (I wrote something to script most of the captures, my goodness vprof takes a long time to print). http://fpaste.org/11930/13684786/ Thanks, Kyle. On Mon, May 13, 2013 at 1:47 PM, Essay Tew Phaun sc2p...@gmail.com wrote: Oh my god Valve, the spikes are 10x worse now, what have you done lol.** ** On Mon, May 13, 2013 at 1:38 PM, Bjorn Wielens uniac...@yahoo.ca wrote: I will attempt to get a few vprof dumps and post them here. -- *From:* Fletcher Dunn fletch...@valvesoftware.com *To:* hlds_li...@list.valvesoftware.com hlds_li...@list.valvesoftware.com; Half-Life dedicated Win32 server mailing list hlds@list.valvesoftware.com *Sent:* Monday, May 13, 2013 2:09:14 PM *Subject:* Re: [hlds] [hlds_linux] TF2 server performance spikes Has anybody used vprof spike mode? This sort of problem is pretty much exactly what it was designed to investigate. E.g. vprof_on vprof_dump_spikes 20 // dump a vprof report if frame rate drops below 20 If the problem is caused by network traffic, using the -netspike command line argument can also be used to dig deeper into exactly what entity data is being sent. If you have a report and you don't know how to interpret it, you can email it to me or post it here, I'm sure there are some people who can help analyze it. -Original Message- From: hlds_linux-boun...@list.valvesoftware.com [mailto: hlds_linux-boun...@list.valvesoftware.com] On Behalf Of Bjorn Wielens Sent: Monday, May 13, 2013 9:42 AM To: Half-Life dedicated Win32 server mailing list; hlds_li...@list.valvesoftware.com Subject: Re: [hlds_linux] [hlds] TF2 server performance spikes Correct, I was one of the folks discussing this in the last round of messages and we run on Linux. Since my last message about having disabled replays I still get occasional hiccups in a full server, but nowhere near as bad as before, owing to the lack of replay overhead. I'd be happy to run diagnostics if they were available for Linux, but alas that's not the case. From: Alex Kowald abkow...@gmail.com To: Half-Life dedicated Win32 server mailing list hlds
Re: [hlds] [hlds_linux] TF2 server performance spikes
I tried to send a trace twice both times the program froze and I guess it never got sent. On Mon, May 13, 2013 at 5:34 PM, Essay Tew Phaun sc2p...@gmail.com wrote: Wonderful... :/ On Mon, May 13, 2013 at 5:32 PM, Kyle Sanderson kyle.l...@gmail.comwrote: You need to watch your console for vprof messages flying by and capture the text some how. I think I saw one once, scrolled up to copy it, and it was gone in a flash. You can try using screen to log the message if you're on *nix, otherwise you can try your hand at using con_logfile, but be aware, it kills. Thanks, Kyle. On Mon, May 13, 2013 at 2:27 PM, Essay Tew Phaun sc2p...@gmail.comwrote: cheat protected -_-. I do know that the server dropped down into the red and nothing was generated. Are they stored in the vprof directory or within the root of the game installation? On Mon, May 13, 2013 at 5:19 PM, Fletcher Dunn fletch...@valvesoftware.com wrote: You should be able to enable it at any time. ** ** You can test if it’s working by executing the “spike” command, which just forces the server frame rate to stall for a second or so. ** ** *From:* hlds-boun...@list.valvesoftware.com [mailto: hlds-boun...@list.valvesoftware.com] *On Behalf Of *Essay Tew Phaun *Sent:* Monday, May 13, 2013 2:16 PM *To:* Half-Life dedicated Win32 server mailing list *Subject:* Re: [hlds] [hlds_linux] TF2 server performance spikes ** ** Can these both be enabled at any point during the server running or must they be specified at startup? I've enabled both and seen 2 servers already dip below and nothing has been generated. ** ** On Mon, May 13, 2013 at 5:09 PM, Fletcher Dunn fletch...@valvesoftware.com wrote: That trace doesn’t appear to be produced as a result of vprof_dump_spikes. That is a profile of average performance over about an hour’s worth of uptime. 314391 frames sampled for 4718.71 seconds Average 66.63 fps, 15.01 ms per frame Peak 3131.67 ms frame 99 pct of time accounted for When spike mode is active, it will clear the profile data each frame, and then dump it if the frame time exceeds some threshold. (You actually specify the reciprocal of the threshold, in hz.) *From:* hlds-boun...@list.valvesoftware.com [mailto: hlds-boun...@list.valvesoftware.com] *On Behalf Of *Kyle Sanderson *Sent:* Monday, May 13, 2013 1:59 PM *To:* Half-Life dedicated Win32 server mailing list *Subject:* Re: [hlds] [hlds_linux] TF2 server performance spikes Did you update? for the past hour or so I've been getting complaints about spikes, I'm thinking it's something non-threaded using the Steam API. I noticed Avatars have been working since the change, but I'm just pulling at straws now. vprof hasn't rendered anything useful. Here's a vprof dump after a while (I wrote something to script most of the captures, my goodness vprof takes a long time to print). http://fpaste.org/11930/13684786/ Thanks, Kyle. On Mon, May 13, 2013 at 1:47 PM, Essay Tew Phaun sc2p...@gmail.com wrote: Oh my god Valve, the spikes are 10x worse now, what have you done lol.* *** On Mon, May 13, 2013 at 1:38 PM, Bjorn Wielens uniac...@yahoo.ca wrote: I will attempt to get a few vprof dumps and post them here. -- *From:* Fletcher Dunn fletch...@valvesoftware.com *To:* hlds_li...@list.valvesoftware.com hlds_li...@list.valvesoftware.com; Half-Life dedicated Win32 server mailing list hlds@list.valvesoftware.com *Sent:* Monday, May 13, 2013 2:09:14 PM *Subject:* Re: [hlds] [hlds_linux] TF2 server performance spikes Has anybody used vprof spike mode? This sort of problem is pretty much exactly what it was designed to investigate. E.g. vprof_on vprof_dump_spikes 20 // dump a vprof report if frame rate drops below 20 If the problem is caused by network traffic, using the -netspike command line argument can also be used to dig deeper into exactly what entity data is being sent. If you have a report and you don't know how to interpret it, you can email it to me or post it here, I'm sure there are some people who can help analyze it. -Original Message- From: hlds_linux-boun...@list.valvesoftware.com [mailto: hlds_linux-boun...@list.valvesoftware.com] On Behalf Of Bjorn Wielens Sent: Monday, May 13, 2013 9:42 AM To: Half-Life dedicated Win32 server mailing list; hlds_li...@list.valvesoftware.com Subject: Re: [hlds_linux] [hlds] TF2 server performance spikes Correct, I was one of the folks discussing this in the last round of messages and we run on Linux. Since my last message about having disabled replays I still get occasional hiccups in a full server, but nowhere near as bad as before, owing to the lack of replay overhead. I'd be happy to run diagnostics if they were available for Linux
Re: [hlds] [hlds_linux] TF2 server performance spikes
On May 13, 2013, at 12:20 PM, Kyle Sanderson kyle.l...@gmail.com wrote: There's no clean way to capture the output from vprof at the time of this writing. Huh? I put in: vprof_on vprof_dump_spikes 20 // dump a vprof report if frame rate drops below 20 ... on one of my servers, and it created /tf/vprof/vprof1.txt the first time it hit a long frame. That seems pretty clean to me. - Peter ___ To unsubscribe, edit your list preferences, or view the list archives, please visit: https://list.valvesoftware.com/cgi-bin/mailman/listinfo/hlds
Re: [hlds] [hlds_linux] TF2 server performance spikes
It's highly possible you're running on a supported game. I had a vprof1.txt in that directory once, and it said no samples. I had thought I created the file, but it's highly possible it does work. If it doesn't work for you, I have a horrible, horrible SM extension available. What it does is capture VProf output text and will write it to a file. What it isn't is portable across platforms, good, or even not a cat killer. But if you're after trying the vprof approach, you're welcome to try it. I can probably fix it up (and make it a MM:S plugin, un-hardcode things) if there's any interest, but I'm hoping Valve fixes VProf before that happens. I probably won't compile/support a Windows version as VTrace / Telemetry is probably a better way forward. The files will be dumped in the directory that Peter Jerde mentioned (vprof/). If the directory doesn't exist, you'll probably have to create it by hand. http://stuff.plaguefest.com/kyle/vprof.tar.xz ( tar xJvf vprof.tar.xz ) There's only TF and CS:S binaries in there. The extension can probably be independent, but this isn't intended as a fix, rather a stopgap. If you've never used a lone extension before, you either need to manually load it, or create a .autoload file alongside the extension (VProfExt.autoload). Hope this helps, Kyle. PS: If your cat does die, it was this. On Mon, May 13, 2013 at 3:54 PM, Peter Jerde peter-h...@jerde.net wrote: On May 13, 2013, at 12:20 PM, Kyle Sanderson kyle.l...@gmail.com wrote: There's no clean way to capture the output from vprof at the time of this writing. Huh? I put in: vprof_on vprof_dump_spikes 20 // dump a vprof report if frame rate drops below 20 ... on one of my servers, and it created /tf/vprof/vprof1.txt the first time it hit a long frame. That seems pretty clean to me. - Peter ___ To unsubscribe, edit your list preferences, or view the list archives, please visit: https://list.valvesoftware.com/cgi-bin/mailman/listinfo/hlds ___ To unsubscribe, edit your list preferences, or view the list archives, please visit: https://list.valvesoftware.com/cgi-bin/mailman/listinfo/hlds
Re: [hlds] [hlds_linux] TF2 server performance spikes
http://pastie.org/private/0nehujv54iw2lwtndcyopw http://pastie.org/private/hhwl7eougryhiin7n8ma http://pastie.org/private/t8sdtvfxuh9aigylqdc6w http://pastie.org/private/d4q26os0usxa4ofvd04uaw http://pastie.org/private/ns6sfnkp59avunqk0lkhpq The spikes seem to happen every minute. If you need more vprof spikes I can now provide them. We're running with the parallel convars enabled that are known to crash. Nothing has changed besides updating to Steam|. Thanks, Kyle. On Mon, May 13, 2013 at 7:24 PM, Kyle Sanderson kyle.l...@gmail.com wrote: It's highly possible you're running on a supported game. I had a vprof1.txt in that directory once, and it said no samples. I had thought I created the file, but it's highly possible it does work. If it doesn't work for you, I have a horrible, horrible SM extension available. What it does is capture VProf output text and will write it to a file. What it isn't is portable across platforms, good, or even not a cat killer. But if you're after trying the vprof approach, you're welcome to try it. I can probably fix it up (and make it a MM:S plugin, un-hardcode things) if there's any interest, but I'm hoping Valve fixes VProf before that happens. I probably won't compile/support a Windows version as VTrace / Telemetry is probably a better way forward. The files will be dumped in the directory that Peter Jerde mentioned (vprof/). If the directory doesn't exist, you'll probably have to create it by hand. http://stuff.plaguefest.com/kyle/vprof.tar.xz ( tar xJvf vprof.tar.xz ) There's only TF and CS:S binaries in there. The extension can probably be independent, but this isn't intended as a fix, rather a stopgap. If you've never used a lone extension before, you either need to manually load it, or create a .autoload file alongside the extension (VProfExt.autoload). Hope this helps, Kyle. PS: If your cat does die, it was this. On Mon, May 13, 2013 at 3:54 PM, Peter Jerde peter-h...@jerde.net wrote: On May 13, 2013, at 12:20 PM, Kyle Sanderson kyle.l...@gmail.com wrote: There's no clean way to capture the output from vprof at the time of this writing. Huh? I put in: vprof_on vprof_dump_spikes 20 // dump a vprof report if frame rate drops below 20 ... on one of my servers, and it created /tf/vprof/vprof1.txt the first time it hit a long frame. That seems pretty clean to me. - Peter ___ To unsubscribe, edit your list preferences, or view the list archives, please visit: https://list.valvesoftware.com/cgi-bin/mailman/listinfo/hlds ___ To unsubscribe, edit your list preferences, or view the list archives, please visit: https://list.valvesoftware.com/cgi-bin/mailman/listinfo/hlds
