----- Original Message ----- From: "Ryan Thompson" <[EMAIL PROTECTED]>
| I'm 99% sure wget doesn't support that, or 304 replies (probably since | it has no real concept of the local filesystem :-), but it'd be pretty | trivial to implement with other tools/libraries not nearly as bulky as | rsync. whatever but the most recent changes makes the files look new everytime and downloads them all with a restart of SA http://www.rulesemporium.com/rules/70_sare_bayes_poison_nxm.cf http://www.rulesemporium.com/rules/70_sare_oem.cf http://www.rulesemporium.com/rules/70_sare_random.cf http://www.rulesemporium.com/rules/70_sare_spoof.cf http://www.rulesemporium.com/rules/70_sare_genlsubj0.cf http://www.rulesemporium.com/rules/70_sare_genlsubj1.cf http://www.rulesemporium.com/rules/70_sare_genlsubj2.cf http://www.rulesemporium.com/rules/70_sare_genlsubj3.cf http://www.rulesemporium.com/rules/70_sare_specific.cf http://www.rulesemporium.com/rules/71_sare_redirect_pre3.0.0.cf http://www.rulesemporium.com/rules/72_sare_bml_post25x.cf http://www.rulesemporium.com/rules/evilnumbers.cf http://www.rulesemporium.com/rules/coding_html.cf http://www.rulesemporium.com/rules/header_abuse.cf http://www.rulesemporium.com/rules/99_sare_adult.cf http://www.rulesemporium.com/rules/99_sare_biz_market_learn_post25x.cf http://www.rulesemporium.com/rules/99_sare_fraud_post25x.cf http://www.rulesemporium.com/rules/bogus-virus-warnings.cf http://www.rulesemporium.com/rules/99_FVGT_Tripwire.cf http://www.rulesemporium.com/rules/weeds_2.cf http://www.rulesemporium.com/rules/88_FVGT_Bayes_Poison.cf http://www.rulesemporium.com/rules/88_FVGT_body.cf http://www.rulesemporium.com/rules/88_FVGT_rawbody.cf http://www.rulesemporium.com/rules/88_FVGT_subject.cf http://www.rulesemporium.com/rules/88_FVGT_headers.cf http://www.rulesemporium.com/rules/88_FVGT_uri.cf My point is why go through the complete download if you don't have to and why does SA have to restart every time since (in reality) no changes have occurred. God forbid if you change the download schedule and it is 22 hours and not 24 hours, all your cf files look like: <H1>Rate limiting in effect</H1> Your request could not be processed because you have exceeded the maximum request rate for the requested document. This is a temporary condition; you will be permitted to submit another request in a few hours. <BR><BR>To avoid triggering the rate limiter in future, please make less frequent requests for this document. You should not request the same document more than once every 24 hours. Please also note that continuing to re-request the document while rate limiting is in effect will further increase the amount of time before the file becomes available to you again. What's up with that? What about FTP, that has to be more efficient than http Regards Greg ----- Original Message ----- From: "Ryan Thompson" <[EMAIL PROTECTED]> To: "Cirelle Enterprises" <[EMAIL PROTECTED]> Cc: "Chris Santerre" <[EMAIL PROTECTED]>; <[EMAIL PROTECTED]> Sent: Wednesday, July 14, 2004 12:35 PM Subject: Re: Rules Download question | Cirelle Enterprises wrote to Chris Santerre: | | > Chris, | > | > Why aren't you running rsync for the rules. | > | > That would cut down on bandwidth because if the rule | > didn't change, no download and is much better than the wget | | HTTP would do a good job of this itself, if RDJ took advantage of the | If-Modified-Since request-header field, significant bandwidth could be | saved. | | I'm 99% sure wget doesn't support that, or 304 replies (probably since | it has no real concept of the local filesystem :-), but it'd be pretty | trivial to implement with other tools/libraries not nearly as bulky as | rsync. | | rsync is better for large hierarchies of many files. Using it to update | a dozen or so files would be overkill, IMO. rsync is also quite resource | intensive on server and client side. | | Chris, if you want, I can look at reworking RDJ to take advantage of | If-Modified-Since, as long as it is (relatively) safe to assume that the | modified dates change IFF there are new versions of each ruleset. | | All of the PARSE_NEW_VER_SCRIPTS stuff could still be preserved, I | suppose. | | - Ryan | | -- | Ryan Thompson <[EMAIL PROTECTED]> | | SaskNow Technologies - http://www.sasknow.com | 901-1st Avenue North - Saskatoon, SK - S7K 1Y4 | | Tel: 306-664-3600 Fax: 306-244-7037 Saskatoon | Toll-Free: 877-727-5669 (877-SASKNOW) North America
