Re: [sniffer] Downloads are slow.
I'm not showing this from my location and the server looks ok. I just downloaded a few rulebases, each in under 3 seconds. Please provide a traceroute -- that should show us where the issue is (if it is still there). Thanks, _M On Tuesday, February 7, 2006, 4:39:35 PM, Chuck wrote: CS Download speeds from your server are running 17 kbps at my location. CS Chuck Schick CS Warp 8, Inc. CS (303)-421-5140 CS www.warp8.com CS This E-Mail came from the Message Sniffer mailing list. For CS information and (un)subscription instructions go to CS http://www.sortmonster.com/MessageSniffer/Help/Help.html This E-Mail came from the Message Sniffer mailing list. For information and (un)subscription instructions go to http://www.sortmonster.com/MessageSniffer/Help/Help.html
RE: [sniffer] Downloads are slow.
Agreed, my last report showed pretty slow times. All today were slower now that I look at them. I normally see up to 1.3M with overall times around 800-900K. John C 0K .. .. .. .. .. 36.79 KB/s 50K .. .. .. .. .. 11.51 KB/s 100K .. .. .. .. .. 19.76 KB/s 150K .. .. .. .. .. 11.98 KB/s 200K .. .. .. .. .. 37.20 KB/s 250K .. .. .. .. .. 10.60 KB/s 300K .. .. .. .. .. 16.00 KB/s 350K .. .. .. .. .. 19.05 KB/s 400K .. .. .. .. .. 22.22 KB/s 450K .. .. .. .. .. 10.32 KB/s 500K .. .. .. .. .. 13.50 KB/s 550K .. .. .. .. ..2.74 KB/s 600K .. .. .. .. ..8.40 KB/s 650K .. .. .. .. ..6.00 KB/s 700K .. .. .. .. ..9.97 KB/s 750K .. .. .. .. ..6.07 KB/s 800K .. .. .. .. ..5.89 KB/s 850K .. .. .. .. ..9.20 KB/s 900K .. .. .. .. ..6.46 KB/s 950K .. .. .. .. ..4.94 KB/s 1000K .. .. .. .. ..7.67 KB/s 1050K .. .. .. .. ..9.97 KB/s 1100K .. .. .. .. .. 13.28 KB/s 1150K .. .. .. .. .. 24.61 KB/s 1200K .. .. .. .. .. 12.36 KB/s 1250K .. .. .. .. .. 31.06 KB/s 1300K .. .. .. .. ..4.87 KB/s 1350K .. .. .. .. .. 34.77 KB/s 1400K .. .. .. .. .. 14.29 KB/s 1450K .. . .. .. .. 16.24 KB/s 1500K .. .. .. .. .. 33.33 KB/s 1550K .. . .. .. .. 21.48 KB/s 1600K .. .. .. .. .. 23.19 KB/s 1650K .. .. .. .. .. 27.34 KB/s 1700K .. .. .. .. .. 14.68 KB/s 1750K .. .. .. .. .. 47.76 KB/s 1800K .. .. .. .. .. 15.17 KB/s 1850K .. .. .. .. .. 16.17 KB/s 1900K .. .. .. .. .. 18.39 KB/s 1950K .. .. .. .. .. 74.40 KB/s 2000K .. .. .. .. .. 14.10 KB/s 2050K .. .. .. .. .. 12.70 KB/s 2100K .. .. .. .. .. 29.36 KB/s 2150K .. .. .. .. .. 16.58 KB/s 2200K .. .. .. .. .. 21.62 KB/s 2250K .. .. .. .. .. 17.49 KB/s 2300K .. .. .. .. .. 11.00 KB/s 2350K .. .. .. .. .. 21.20 KB/s 2400K .. .. .. .. .. 31.69 KB/s 2450K .. .. .. .. .. 20.12 KB/s 2500K .. .. .. .. .. 57.14 KB/s 2550K .. .. .. 13.94 KB/s 15:52:29 (12.45 KB/s) - `.new.gz' saved [2646653] -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Pete McNeil Sent: Tuesday, February 07, 2006 4:46 PM To: Chuck Schick Subject: Re: [sniffer] Downloads are slow. I'm not showing this from my location and the server looks ok. I just downloaded a few rulebases, each in under 3 seconds. Please provide a traceroute -- that should show us where the issue is (if it is still there). Thanks, _M On Tuesday, February 7, 2006, 4:39:35 PM, Chuck wrote: CS Download speeds from your server are running 17 kbps at my location. CS Chuck Schick CS Warp 8, Inc. CS (303)-421-5140 CS www.warp8.com CS This E-Mail came from the Message Sniffer mailing list. For CS information and (un)subscription instructions go to CS http://www.sortmonster.com/MessageSniffer/Help/Help.html This E-Mail came from the Message Sniffer mailing
Re: [sniffer] Downloads are slow...
I agree that something needs to be done about the update scripts that are inadvertently downloading the full rulebase all the time. I didn't even know it but we were doing this until I went through our update script again this morning and found it didn't have the -N option in Wget, so we were downloading the entire rulebase whether we needed it or not. The gzip compression is cool, and I will likely implement it soon, but I think the major problem is everyone that is using scripts that keep downloading the same file over and over again tying up the bandwidth. I would recommend 2 things to help alleviate this problem: 1. Monitor connections to rulebase downloads to see who is downloading the rulebase everytime they connect on a schedule to determine who has their scripts setup wrong, and contact them to correct it. It took me under a minute to add the -N option to wget, it should be a no brainer. 2. Correct the scripts posted on the Sniffer website to include date checking, and possibly gzip compression, I used one of those scripts for our system and assumed it would be setup correctly, but it was not. Jim Matuska Jr. Computer Tech2, CCNA Nez Perce Tribe Information Systems [EMAIL PROTECTED] - Original Message - From: Matt [EMAIL PROTECTED] To: sniffer@SortMonster.com Sent: Monday, December 27, 2004 10:03 PM Subject: Re: [sniffer] Downloads are slow... I agree entirely. If bandwidth has become an issue, it would be resolved with a focus on producing very tight and easily customizable scripts (a variables section in the top of the scripts). I believe that going the VBScript route might be the best way to go, or at least I believe that more of us can hack a more involved VBScript than a batch or CMD file. Enforcing compressed downloads and checking for timestamps prior to downloading should be done in these scripts as well. Right now the script examples assume a familiarity with scripting, and while local participants can mostly handle that stuff, the non-vocal ones are most likely to not even be aware of the issues or how to fix them, and might have scripted timed downloads because it is definitely the easiest way to go. This is probably the majority of the customer base. There is an impression for instance with Declude's user base that +80% use primarily the default config which most of us know is severely lacking in comparison to the potential that exists by tweaking the settings. With better script examples and a careful step-by-step readme promoted in a mailing to your customers, I believe that this issue could go away, or at least theoretically it should. Personally, I have mine tied to the E-mails, I download the zipped versions, I don't bother checking on the status, and have never noticed any issues as a result. It would be a small shame if I was missing downloads due to timeouts, but not that big of a deal if this has never caused a noticeable problem. Matt Andy Schmidt wrote: Pete, With all due respect - I think the download problem is self-inflicted, because your web site is providing unsuitable examples to your customers! Even with moderate bandwidth, your server would be able to handle tens of thousands of hits a day. Checking if an updated file exists should barely be noticeable - as long as it doesn't result in an unnecessary download. You probably suffer TWO problems: A) Most of your customers are downloading rules based on a schedule, even if no rules exists. Potential savings: 100% per download attempt. B) Your customers are not downloading compressed rule files. Potential savings: about 66%, but that's not bad either. One likely explanation is that at least THREE of your sample scripts do an unconditional and uncompressed download! Here the 3 URLs you list on your web site and WGET command they are using: http://www.sortmonster.com/MessageSniffer/Help/UserScripts/david_snifferUpda teMethod.zip wget http://www.sortmonster.net/Sniffer/Updates/.snf -O .new --http-user=username --http-passwd=password http://www.sortmonster.com/MessageSniffer/Help/UserScripts/Hank_SnifferScrip ts.zip wget http://www.sortmonster.net/Sniffer/Updates/.snf -O .new --http-user=sniffer --http-passwd=ki11sp8m http://www.sortmonster.com/MessageSniffer/Help/UserScripts/Michiel_AutoUpdat e.zip wget http://sniffer:[EMAIL PROTECTED]/Sniffer/Updates/12345678.snf -O serial.tst My recommendation: Replace these with examples that implement conditional, compressed downloading. Best Regards Andy Schmidt HM Systems Software, Inc. 600 East Crescent Avenue, Suite 203 Upper Saddle River, NJ 07458-1846 Phone: +1 201 934-3414 x20 (Business) Fax:+1 201 934-9206 http://www.HM-Software.com/ -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Pete McNeil Sent: Monday, December 27, 2004 08:10 AM To: Chuck Schick Subject: Re: [sniffer] Downloads are slow... On Monday, December 27, 2004, 1:17:21 AM
RE: [sniffer] Downloads are slow...
Why do you not use a program alias and only download when you receive notification that a new rule base is available? If everyone used gzip and only downloaded when notified the bandwidth could be controlled by staggering the notifications. Woody Fussell Wilbur Smith Associates -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Jim Matuska Sent: Tuesday, December 28, 2004 12:49 PM To: sniffer@SortMonster.com Subject: Re: [sniffer] Downloads are slow... I agree that something needs to be done about the update scripts that are inadvertently downloading the full rulebase all the time. I didn't even know it but we were doing this until I went through our update script again this morning and found it didn't have the -N option in Wget, so we were downloading the entire rulebase whether we needed it or not. The gzip compression is cool, and I will likely implement it soon, but I think the major problem is everyone that is using scripts that keep downloading the same file over and over again tying up the bandwidth. I would recommend 2 things to help alleviate this problem: 1. Monitor connections to rulebase downloads to see who is downloading the rulebase everytime they connect on a schedule to determine who has their scripts setup wrong, and contact them to correct it. It took me under a minute to add the -N option to wget, it should be a no brainer. 2. Correct the scripts posted on the Sniffer website to include date checking, and possibly gzip compression, I used one of those scripts for our system and assumed it would be setup correctly, but it was not. Jim Matuska Jr. Computer Tech2, CCNA Nez Perce Tribe Information Systems [EMAIL PROTECTED] - Original Message - From: Matt [EMAIL PROTECTED] To: sniffer@SortMonster.com Sent: Monday, December 27, 2004 10:03 PM Subject: Re: [sniffer] Downloads are slow... I agree entirely. If bandwidth has become an issue, it would be resolved with a focus on producing very tight and easily customizable scripts (a variables section in the top of the scripts). I believe that going the VBScript route might be the best way to go, or at least I believe that more of us can hack a more involved VBScript than a batch or CMD file. Enforcing compressed downloads and checking for timestamps prior to downloading should be done in these scripts as well. Right now the script examples assume a familiarity with scripting, and while local participants can mostly handle that stuff, the non-vocal ones are most likely to not even be aware of the issues or how to fix them, and might have scripted timed downloads because it is definitely the easiest way to go. This is probably the majority of the customer base. There is an impression for instance with Declude's user base that +80% use primarily the default config which most of us know is severely lacking in comparison to the potential that exists by tweaking the settings. With better script examples and a careful step-by-step readme promoted in a mailing to your customers, I believe that this issue could go away, or at least theoretically it should. Personally, I have mine tied to the E-mails, I download the zipped versions, I don't bother checking on the status, and have never noticed any issues as a result. It would be a small shame if I was missing downloads due to timeouts, but not that big of a deal if this has never caused a noticeable problem. Matt Andy Schmidt wrote: Pete, With all due respect - I think the download problem is self-inflicted, because your web site is providing unsuitable examples to your customers! Even with moderate bandwidth, your server would be able to handle tens of thousands of hits a day. Checking if an updated file exists should barely be noticeable - as long as it doesn't result in an unnecessary download. You probably suffer TWO problems: A) Most of your customers are downloading rules based on a schedule, even if no rules exists. Potential savings: 100% per download attempt. B) Your customers are not downloading compressed rule files. Potential savings: about 66%, but that's not bad either. One likely explanation is that at least THREE of your sample scripts do an unconditional and uncompressed download! Here the 3 URLs you list on your web site and WGET command they are using: http://www.sortmonster.com/MessageSniffer/Help/UserScripts/david_snifferUp da teMethod.zip wget http://www.sortmonster.net/Sniffer/Updates/.snf -O .new --http-user=username --http-passwd=password http://www.sortmonster.com/MessageSniffer/Help/UserScripts/Hank_SnifferScr ip ts.zip wget http://www.sortmonster.net/Sniffer/Updates/.snf -O .new --http-user=sniffer --http-passwd=ki11sp8m http://www.sortmonster.com/MessageSniffer/Help/UserScripts/Michiel_AutoUpd at e.zip wget http://sniffer:[EMAIL PROTECTED]/Sniffer/Updates/12345678.snf -O serial.tst My recommendation
RE: [sniffer] Downloads are slow...
Does anyone have some simple instructions on how to setup to use a program alias? I'm using Bill Landry's script, have not had the time to look in to using a program alias to download upon receipt of notifications. Thanks, Brian Gregory President Network Innovations Inc. (913) 780-0494 x104 [EMAIL PROTECTED] -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Woody G Fussell Sent: Tuesday, December 28, 2004 12:25 PM To: sniffer@SortMonster.com Subject: RE: [sniffer] Downloads are slow... Why do you not use a program alias and only download when you receive notification that a new rule base is available? If everyone used gzip and only downloaded when notified the bandwidth could be controlled by staggering the notifications. Woody Fussell Wilbur Smith Associates -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Jim Matuska Sent: Tuesday, December 28, 2004 12:49 PM To: sniffer@SortMonster.com Subject: Re: [sniffer] Downloads are slow... I agree that something needs to be done about the update scripts that are inadvertently downloading the full rulebase all the time. I didn't even know it but we were doing this until I went through our update script again this morning and found it didn't have the -N option in Wget, so we were downloading the entire rulebase whether we needed it or not. The gzip compression is cool, and I will likely implement it soon, but I think the major problem is everyone that is using scripts that keep downloading the same file over and over again tying up the bandwidth. I would recommend 2 things to help alleviate this problem: 1. Monitor connections to rulebase downloads to see who is downloading the rulebase everytime they connect on a schedule to determine who has their scripts setup wrong, and contact them to correct it. It took me under a minute to add the -N option to wget, it should be a no brainer. 2. Correct the scripts posted on the Sniffer website to include date checking, and possibly gzip compression, I used one of those scripts for our system and assumed it would be setup correctly, but it was not. Jim Matuska Jr. Computer Tech2, CCNA Nez Perce Tribe Information Systems [EMAIL PROTECTED] - Original Message - From: Matt [EMAIL PROTECTED] To: sniffer@SortMonster.com Sent: Monday, December 27, 2004 10:03 PM Subject: Re: [sniffer] Downloads are slow... I agree entirely. If bandwidth has become an issue, it would be resolved with a focus on producing very tight and easily customizable scripts (a variables section in the top of the scripts). I believe that going the VBScript route might be the best way to go, or at least I believe that more of us can hack a more involved VBScript than a batch or CMD file. Enforcing compressed downloads and checking for timestamps prior to downloading should be done in these scripts as well. Right now the script examples assume a familiarity with scripting, and while local participants can mostly handle that stuff, the non-vocal ones are most likely to not even be aware of the issues or how to fix them, and might have scripted timed downloads because it is definitely the easiest way to go. This is probably the majority of the customer base. There is an impression for instance with Declude's user base that +80% use primarily the default config which most of us know is severely lacking in comparison to the potential that exists by tweaking the settings. With better script examples and a careful step-by-step readme promoted in a mailing to your customers, I believe that this issue could go away, or at least theoretically it should. Personally, I have mine tied to the E-mails, I download the zipped versions, I don't bother checking on the status, and have never noticed any issues as a result. It would be a small shame if I was missing downloads due to timeouts, but not that big of a deal if this has never caused a noticeable problem. Matt Andy Schmidt wrote: Pete, With all due respect - I think the download problem is self-inflicted, because your web site is providing unsuitable examples to your customers! Even with moderate bandwidth, your server would be able to handle tens of thousands of hits a day. Checking if an updated file exists should barely be noticeable - as long as it doesn't result in an unnecessary download. You probably suffer TWO problems: A) Most of your customers are downloading rules based on a schedule, even if no rules exists. Potential savings: 100% per download attempt. B) Your customers are not downloading compressed rule files. Potential savings: about 66%, but that's not bad either. One likely explanation is that at least THREE of your sample scripts do an unconditional and uncompressed download! Here the 3 URLs you list on your web site and WGET command they are using: http://www.sortmonster.com/MessageSniffer/Help/UserScripts/david_snifferUp da
Re: [sniffer] Downloads are slow...
Brian Gregory wrote: Does anyone have some simple instructions on how to setup to use a program alias? I'm using Bill Landry's script, have not had the time to look in to using a program alias to download upon receipt of notifications. I just setup a program alias about 2 weeks ago using this as the script http://www.sortmonster.com/MessageSniffer/Help/UserScripts/david_snifferUpdateMethod.zip that IMail calls, and this as the actual updater http://www.sortmonster.com/MessageSniffer/Help/UserScripts/Bill_Landry_snfupd.zip I did have to do some customizations, and also I had to have all the files in the sniffer directory, but it wasn't too bad. I'd be glad to help you with any specific questions, either on or off list... -Russ --- [This E-mail scanned for viruses by Declude Virus] This E-Mail came from the Message Sniffer mailing list. For information and (un)subscription instructions go to http://www.sortmonster.com/MessageSniffer/Help/Help.html
Re: [sniffer] Downloads are slow...
Quick question if when you have a sucessful download if abcdef.new is renamed than what is wget comparing on the next run of the script? Darrell Jim Matuska writes: So far it seems to be working, at least it doesn't seem to be downloading the rulebase yet, I'll have to see if it does later when there is an updated rulebase. My script uses a copy at the end rather than a move. It's listed below for reference. Do you see any issues? wget -N http://www.sortmonster.net/Sniffer/Updates/fp0o4jye.snf -O abcdefg.new --http-user=* --http-passwd=* if exist abcdefg.new goto Replace goto Done :Replace rename abcdefg.new abcdefg.tst snf2check.exe abcdefg.tst abcdefg if errorlevel 1 goto Done echo New File Tested GOOD! if exist abcdefg.old del abcdefg.old rename abcdefg.snf abcdefg.old rename abcdefg.tst abcdefg.snf copy /V /Y abcdefg.snf C:\sniffer\abcdefg.snf :Done if exist abcdefg.tst del abcdefg.tst Jim Matuska Jr. Computer Tech2, CCNA Nez Perce Tribe Information Systems [EMAIL PROTECTED] - Original Message - From: Pete McNeil [EMAIL PROTECTED] To: Jim Matuska sniffer@SortMonster.com Sent: Tuesday, December 28, 2004 11:12 AM Subject: Re[2]: [sniffer] Downloads are slow... On Tuesday, December 28, 2004, 12:49:21 PM, Jim wrote: JM I agree that something needs to be done about the update scripts that are JM inadvertently downloading the full rulebase all the time. I didn't even JM know it but we were doing this until I went through our update script again JM this morning and found it didn't have the -N option in Wget, so we were Watch out - you may still have not fixed it. One of the tricks with the -N option is that the file downloaded previously must still be in it's place for the comparison. If it has been moved then the -N will not matter. This make things a little bit more complex since you can't download a rulebase file on top of the one that is running. _M This E-Mail came from the Message Sniffer mailing list. For information and (un)subscription instructions go to http://www.sortmonster.com/MessageSniffer/Help/Help.html This E-Mail came from the Message Sniffer mailing list. For information and (un)subscription instructions go to http://www.sortmonster.com/MessageSniffer/Help/Help.html Check out http://www.invariantsystems.com for utilities for Declude And Imail. IMail/Declude Overflow Queue Monitoring, MRTG Integration, and Log Parsers. This E-Mail came from the Message Sniffer mailing list. For information and (un)subscription instructions go to http://www.sortmonster.com/MessageSniffer/Help/Help.html
RE: [sniffer] Downloads are slow...
While the output file is named .new, it IS comparing the file named in the URL, in his case fp0o4jye.snf against a file with the same name in the current directory. The output (-o) option only comes into play IF an download is actually occurring (after the timestamp condition). With CURL things are a bit more flexible - you can specify WHICH file is used for comparison. Best Regards Andy Schmidt HM Systems Software, Inc. 600 East Crescent Avenue, Suite 203 Upper Saddle River, NJ 07458-1846 Phone: +1 201 934-3414 x20 (Business) Fax:+1 201 934-9206 http://www.HM-Software.com/ -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Darrell ([EMAIL PROTECTED]) Sent: Tuesday, December 28, 2004 02:39 PM To: sniffer@SortMonster.com Subject: Re: [sniffer] Downloads are slow... Quick question if when you have a sucessful download if abcdef.new is renamed than what is wget comparing on the next run of the script? Darrell Jim Matuska writes: So far it seems to be working, at least it doesn't seem to be downloading the rulebase yet, I'll have to see if it does later when there is an updated rulebase. My script uses a copy at the end rather than a move. It's listed below for reference. Do you see any issues? wget -N http://www.sortmonster.net/Sniffer/Updates/fp0o4jye.snf -O abcdefg.new --http-user=* --http-passwd=* if exist abcdefg.new goto Replace goto Done :Replace rename abcdefg.new abcdefg.tst snf2check.exe abcdefg.tst abcdefg if errorlevel 1 goto Done echo New File Tested GOOD! if exist abcdefg.old del abcdefg.old rename abcdefg.snf abcdefg.old rename abcdefg.tst abcdefg.snf copy /V /Y abcdefg.snf C:\sniffer\abcdefg.snf :Done if exist abcdefg.tst del abcdefg.tst Jim Matuska Jr. Computer Tech2, CCNA Nez Perce Tribe Information Systems [EMAIL PROTECTED] - Original Message - From: Pete McNeil [EMAIL PROTECTED] To: Jim Matuska sniffer@SortMonster.com Sent: Tuesday, December 28, 2004 11:12 AM Subject: Re[2]: [sniffer] Downloads are slow... On Tuesday, December 28, 2004, 12:49:21 PM, Jim wrote: JM I agree that something needs to be done about the update scripts JM that are JM inadvertently downloading the full rulebase all the time. I JM didn't even JM know it but we were doing this until I went through our update JM script again JM this morning and found it didn't have the -N option in Wget, so JM we were Watch out - you may still have not fixed it. One of the tricks with the -N option is that the file downloaded previously must still be in it's place for the comparison. If it has been moved then the -N will not matter. This make things a little bit more complex since you can't download a rulebase file on top of the one that is running. _M This E-Mail came from the Message Sniffer mailing list. For information and (un)subscription instructions go to http://www.sortmonster.com/MessageSniffer/Help/Help.html This E-Mail came from the Message Sniffer mailing list. For information and (un)subscription instructions go to http://www.sortmonster.com/MessageSniffer/Help/Help.html Check out http://www.invariantsystems.com for utilities for Declude And Imail. IMail/Declude Overflow Queue Monitoring, MRTG Integration, and Log Parsers. This E-Mail came from the Message Sniffer mailing list. For information and (un)subscription instructions go to http://www.sortmonster.com/MessageSniffer/Help/Help.html This E-Mail came from the Message Sniffer mailing list. For information and (un)subscription instructions go to http://www.sortmonster.com/MessageSniffer/Help/Help.html
Re: [sniffer] Downloads are slow...
As far as I understand it wget is comparing the date stamp on the file in the local directory to the date stamp on the file at sortmonster.net, if its not don't download the file, if it is do download it. Jim Matuska Jr. Computer Tech2, CCNA Nez Perce Tribe Information Systems [EMAIL PROTECTED] - Original Message - From: Darrell ([EMAIL PROTECTED]) [EMAIL PROTECTED] To: sniffer@SortMonster.com Sent: Tuesday, December 28, 2004 11:38 AM Subject: Re: [sniffer] Downloads are slow... Quick question if when you have a sucessful download if abcdef.new is renamed than what is wget comparing on the next run of the script? Darrell Jim Matuska writes: So far it seems to be working, at least it doesn't seem to be downloading the rulebase yet, I'll have to see if it does later when there is an updated rulebase. My script uses a copy at the end rather than a move. It's listed below for reference. Do you see any issues? wget -N http://www.sortmonster.net/Sniffer/Updates/fp0o4jye.snf -O abcdefg.new --http-user=* --http-passwd=* if exist abcdefg.new goto Replace goto Done :Replace rename abcdefg.new abcdefg.tst snf2check.exe abcdefg.tst abcdefg if errorlevel 1 goto Done echo New File Tested GOOD! if exist abcdefg.old del abcdefg.old rename abcdefg.snf abcdefg.old rename abcdefg.tst abcdefg.snf copy /V /Y abcdefg.snf C:\sniffer\abcdefg.snf :Done if exist abcdefg.tst del abcdefg.tst Jim Matuska Jr. Computer Tech2, CCNA Nez Perce Tribe Information Systems [EMAIL PROTECTED] - Original Message - From: Pete McNeil [EMAIL PROTECTED] To: Jim Matuska sniffer@SortMonster.com Sent: Tuesday, December 28, 2004 11:12 AM Subject: Re[2]: [sniffer] Downloads are slow... On Tuesday, December 28, 2004, 12:49:21 PM, Jim wrote: JM I agree that something needs to be done about the update scripts that are JM inadvertently downloading the full rulebase all the time. I didn't even JM know it but we were doing this until I went through our update script again JM this morning and found it didn't have the -N option in Wget, so we were Watch out - you may still have not fixed it. One of the tricks with the -N option is that the file downloaded previously must still be in it's place for the comparison. If it has been moved then the -N will not matter. This make things a little bit more complex since you can't download a rulebase file on top of the one that is running. _M This E-Mail came from the Message Sniffer mailing list. For information and (un)subscription instructions go to http://www.sortmonster.com/MessageSniffer/Help/Help.html This E-Mail came from the Message Sniffer mailing list. For information and (un)subscription instructions go to http://www.sortmonster.com/MessageSniffer/Help/Help.html Check out http://www.invariantsystems.com for utilities for Declude And Imail. IMail/Declude Overflow Queue Monitoring, MRTG Integration, and Log Parsers. This E-Mail came from the Message Sniffer mailing list. For information and (un)subscription instructions go to http://www.sortmonster.com/MessageSniffer/Help/Help.html This E-Mail came from the Message Sniffer mailing list. For information and (un)subscription instructions go to http://www.sortmonster.com/MessageSniffer/Help/Help.html
Re: [sniffer] Downloads are slow...
On Monday, December 27, 2004, 1:17:21 AM, Chuck wrote: CS Pete: CS It appears on weekends the sniffer downloads are really slow. I am CS downloading at 14 minutes past the hour and I am about 1/20 th of the normal CS speed. That is an unusual observation - I don't think weekends have anything to do with making things slower. I will look at the logs to see if I can figure out what heppened. You're not manually downloading I hope? _M This E-Mail came from the Message Sniffer mailing list. For information and (un)subscription instructions go to http://www.sortmonster.com/MessageSniffer/Help/Help.html
RE: [sniffer] Downloads are slow...
Pete, With all due respect - I think the download problem is self-inflicted, because your web site is providing unsuitable examples to your customers! Even with moderate bandwidth, your server would be able to handle tens of thousands of hits a day. Checking if an updated file exists should barely be noticeable - as long as it doesn't result in an unnecessary download. You probably suffer TWO problems: A) Most of your customers are downloading rules based on a schedule, even if no rules exists. Potential savings: 100% per download attempt. B) Your customers are not downloading compressed rule files. Potential savings: about 66%, but that's not bad either. One likely explanation is that at least THREE of your sample scripts do an unconditional and uncompressed download! Here the 3 URLs you list on your web site and WGET command they are using: http://www.sortmonster.com/MessageSniffer/Help/UserScripts/david_snifferUpda teMethod.zip wget http://www.sortmonster.net/Sniffer/Updates/.snf -O .new --http-user=username --http-passwd=password http://www.sortmonster.com/MessageSniffer/Help/UserScripts/Hank_SnifferScrip ts.zip wget http://www.sortmonster.net/Sniffer/Updates/.snf -O .new --http-user=sniffer --http-passwd=ki11sp8m http://www.sortmonster.com/MessageSniffer/Help/UserScripts/Michiel_AutoUpdat e.zip wget http://sniffer:[EMAIL PROTECTED]/Sniffer/Updates/12345678.snf -O serial.tst My recommendation: Replace these with examples that implement conditional, compressed downloading. Best Regards Andy Schmidt HM Systems Software, Inc. 600 East Crescent Avenue, Suite 203 Upper Saddle River, NJ 07458-1846 Phone: +1 201 934-3414 x20 (Business) Fax:+1 201 934-9206 http://www.HM-Software.com/ -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Pete McNeil Sent: Monday, December 27, 2004 08:10 AM To: Chuck Schick Subject: Re: [sniffer] Downloads are slow... On Monday, December 27, 2004, 1:17:21 AM, Chuck wrote: CS Pete: CS It appears on weekends the sniffer downloads are really slow. I am CS downloading at 14 minutes past the hour and I am about 1/20 th of CS the normal speed. That is an unusual observation - I don't think weekends have anything to do with making things slower. I will look at the logs to see if I can figure out what heppened. You're not manually downloading I hope? _M This E-Mail came from the Message Sniffer mailing list. For information and (un)subscription instructions go to http://www.sortmonster.com/MessageSniffer/Help/Help.html This E-Mail came from the Message Sniffer mailing list. For information and (un)subscription instructions go to http://www.sortmonster.com/MessageSniffer/Help/Help.html
Re: [sniffer] Downloads are slow...
I agree entirely. If bandwidth has become an issue, it would be resolved with a focus on producing very tight and easily customizable scripts (a variables section in the top of the scripts). I believe that going the VBScript route might be the best way to go, or at least I believe that more of us can hack a more involved VBScript than a batch or CMD file. Enforcing compressed downloads and checking for timestamps prior to downloading should be done in these scripts as well. Right now the script examples assume a familiarity with scripting, and while local participants can mostly handle that stuff, the non-vocal ones are most likely to not even be aware of the issues or how to fix them, and might have scripted timed downloads because it is definitely the easiest way to go. This is probably the majority of the customer base. There is an impression for instance with Declude's user base that +80% use primarily the default config which most of us know is severely lacking in comparison to the potential that exists by tweaking the settings. With better script examples and a careful step-by-step readme promoted in a mailing to your customers, I believe that this issue could go away, or at least theoretically it should. Personally, I have mine tied to the E-mails, I download the zipped versions, I don't bother checking on the status, and have never noticed any issues as a result. It would be a small shame if I was missing downloads due to timeouts, but not that big of a deal if this has never caused a noticeable problem. Matt Andy Schmidt wrote: Pete, With all due respect - I think the download problem is self-inflicted, because your web site is providing unsuitable examples to your customers! Even with moderate bandwidth, your server would be able to handle tens of thousands of hits a day. Checking if an updated file exists should barely be noticeable - as long as it doesn't result in an unnecessary download. You probably suffer TWO problems: A) Most of your customers are downloading rules based on a schedule, even if no rules exists. Potential savings: 100% per download attempt. B) Your customers are not downloading compressed rule files. Potential savings: about 66%, but that's not bad either. One likely explanation is that at least THREE of your sample scripts do an unconditional and uncompressed download! Here the 3 URLs you list on your web site and WGET command they are using: http://www.sortmonster.com/MessageSniffer/Help/UserScripts/david_snifferUpda teMethod.zip wget http://www.sortmonster.net/Sniffer/Updates/.snf -O .new --http-user=username --http-passwd=password http://www.sortmonster.com/MessageSniffer/Help/UserScripts/Hank_SnifferScrip ts.zip wget http://www.sortmonster.net/Sniffer/Updates/.snf -O .new --http-user=sniffer --http-passwd=ki11sp8m http://www.sortmonster.com/MessageSniffer/Help/UserScripts/Michiel_AutoUpdat e.zip wget http://sniffer:[EMAIL PROTECTED]/Sniffer/Updates/12345678.snf -O serial.tst My recommendation: Replace these with examples that implement conditional, compressed downloading. Best Regards Andy Schmidt HM Systems Software, Inc. 600 East Crescent Avenue, Suite 203 Upper Saddle River, NJ 07458-1846 Phone: +1 201 934-3414 x20 (Business) Fax:+1 201 934-9206 http://www.HM-Software.com/ -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Pete McNeil Sent: Monday, December 27, 2004 08:10 AM To: Chuck Schick Subject: Re: [sniffer] Downloads are slow... On Monday, December 27, 2004, 1:17:21 AM, Chuck wrote: CS Pete: CS It appears on weekends the sniffer downloads are really slow. I am CS downloading at 14 minutes past the hour and I am about 1/20 th of CS the normal speed. That is an unusual observation - I don't think weekends have anything to do with making things slower. I will look at the logs to see if I can figure out what heppened. You're not manually downloading I hope? _M This E-Mail came from the Message Sniffer mailing list. For information and (un)subscription instructions go to http://www.sortmonster.com/MessageSniffer/Help/Help.html This E-Mail came from the Message Sniffer mailing list. For information and (un)subscription instructions go to http://www.sortmonster.com/MessageSniffer/Help/Help.html -- = MailPure custom filters for Declude JunkMail Pro. http://www.mailpure.com/software/ = This E-Mail came from the Message Sniffer mailing list. For information and (un)subscription instructions go to http://www.sortmonster.com/MessageSniffer/Help/Help.html
