files. The files for the month of December, even zipped, came to just under
600 meg (the 30th and 31st account for about 85% of that).
We pay, I think, $5 per Gig over our limit - we ended up with a balloon
payment of nearly $300 for December - really not bad considering the rest of
the year sees basically no traffic and we're only paying $20/month
otherwise.
We're doing standard, out of the box IIS 6 log file options - nothing
special (this is set by the host). Just a lot of traffic. Even with
minimal logging 15 million+ hits in a day is going to create a large file.
(As an aside I'm really surprised that IIS or Apache hasn't standardized on
keeping compressed logs. at the office we keep our logs files on a
compressed volume which works fine. On-the-fly compression isn't as tight
as instance compression, but in this case we went from 12.8 Gig to 580 Meg -
that's an impression compression ratio.)
I'm importing them now. it's taking a little while. ;^) I'm actually very
happy with the performance in fact - I'm importing them via the free (one
site) version of SmaterStats from SmarterTools.com - it's been working
moderately for about 4 hours and is well in to the 30th. I figure it has at
least 6-8 hours left but that's really not that bad.
The "machine" that's doing the import is actually an MS Virtual Server
configuration with 128 meg of dedicated RAM. The host is a Dual 1 Gig PIII
with two other 128 Meg VMs running at the same time.
Considering all of that I think its flying. ;^)
Jim Davis
_____
From: Eric Dawson [mailto:[EMAIL PROTECTED]
Sent: Friday, January 30, 2004 1:55 AM
To: CF-Community
Subject: Re: Anybody ever FTP VERY large files?
how much do you pay for bandwidth - if you don't mind me asking.
are there any command line compression utilities you could run on the server
to do it yourself?
Is the site on a shared server? now I read - $20 a month :)
Big log files. you must be logging everything. Are they IIS logs? Is there a
way to filter what gets logged? ie no images.
----- Original Message -----
From: Jim Davis
To: CF-Community
Sent: Thursday, January 29, 2004 10:42 AM
Subject: RE: Anybody ever FTP VERY large files?
If I had my choice - no doubt.
In fact the host relented and zipped them for me. The argument, at first,
was that zipping such large files was an unfair use of CPU resources (and,
I
think that downloading 8 gig would add a nice bandwidth overage to my
bill).
So I cajoled a little bit and they've zipped them up - now they're a nice,
"small" 500 meg file. ;^)
Thanks for the help.
I'll probably be back once I can finally import these bastards and get
some
useful information. Last I was able to look (before the hosted stats
package went haywire) we were talking 60,000 visits for the day (around
260,000 page views and 12 million hits). All run from a $20/month CFMX
6.1
shared hosting account.
Who says CF can't handle lots of traffic. ;^)
Jim Davis
_____
From: Philip Arnold [mailto:[EMAIL PROTECTED]
Sent: Thursday, January 29, 2004 7:01 AM
To: CF-Community
Subject: RE: Anybody ever FTP VERY large files?
Eric beat me to it - Zip the log first :P
> -----Original Message-----
> From: Jim Davis [mailto:[EMAIL PROTECTED]
> Sent: 29 January 2004 01:49
> To: CF-Community
> Subject: Anybody ever FTP VERY large files?
>
>
> I need to download the log file for www.firstnight.org and
> analyze them myself (they choked the hosting providers stats
> package). I've asked for them to be zipped but I'm not sure
> that will happen (I'm already getting grief for even taking
> up the space).
>
> At any rate the files in question, December 30th and 31st are
> 3.5 and 5.2 gigabytes respectively. Damn big.
>
> I'm a cable modem and can definitely get the data (it'll take
> 12 hours or so, but it the connection stays open I'll get it)
> but I'm having trouble getting an FTP program to actually attempt it.
>
> CuteFTP (my first choice) refuses to try saying "There's not
> enough room on your hard disk for this file" - however the
> disk has 38 gig free (and was recently defragged so it has at
> least twice the needed space contiguous).
>
> Windows command line FTP gave up without trying (besides it
> has no progress status).
>
> Anybody ever do this - what app did you use?
>
> Thanks,
>
> Jim Davis
>
>
>
_____
_____
[Todays Threads] [This Message] [Subscription] [Fast Unsubscribe] [User Settings]
