Re: [analog-help] Directory Hits Seems Inacurate
On Friday, April 07, 2006 11:03 AM [EDT], Wallace, Brian S. <[EMAIL PROTECTED]> wrote: > Yes, the Directory section gives me what I need. I'm using the > following settings: > > DIRECTORY ON > DIRFLOOR 100r > DIRSORTBY ALPHABETICAL > DIRCOLS Rr > DIREXCLUDE /http://,/serveriron/,*/_*/ > SUBDIR /*/*/*/*/*/*/*/*/*/*/ > SUBDIRFLOOR 100r > SUBDIRSORTBY ALPHABETICAL > > Is there a way to specify all SUBDIR's without putting all of the > /*/*/...? I would have to guess as to the actual SUBDIR depth. In practical terms, it's pretty unlikely that you have many subdirectories that arer 10 level deep. If you really think it's likely to be a problem, just specify 15, or 20, but there comes a point where you're wasting your time. Aengus + | TO UNSUBSCRIBE from this list: |http://lists.meer.net/mailman/listinfo/analog-help | | Analog Documentation: http://analog.cx/docs/Readme.html | List archives: http://www.analog.cx/docs/mailing.html#listarchives | Usenet version: news://news.gmane.org/gmane.comp.web.analog.general +
Re: [analog-help] asp code
On Friday, April 07, 2006 12:44 PM [EDT],
[EMAIL PROTECTED] <[EMAIL PROTECTED]> wrote:
> I know this is really outside the realm of what Analog is supposed to
> do - BUT... since it's so customizable, I was just hoping this was
> possible.
>
> I'd like to have my automatically generated stats file (which is
> currently index.html) - automatically include some ASP code that
> would basically DisAllow viewers that are outside of our IP address.
>
> Something like this:
>
> <% dim userIp
> userIp = (Request.ServerVariables("REMOTE_HOST"))
> If Left(userIp, 10) = "208.50.10." Then
> %>
> STATS HERE
> <% Else
> Response.Write("Sorry, this page cannot be viewed from outside the
> network")
> End If %>
>
> so besides changing the .html to .asp I would also need to
> automatically insert the asp code in the correct location.
>
> Is this possible?
http://analog.cx/docs/output.html#HEADERFILE
Put the first ASP snippet in header-asp.txt, and the second snippet in
footer-asp.txt and add
HEADERFILE header-asp.txt
FOOTERFILE footer-asp.txt
to your config file.
Aengus
+
| TO UNSUBSCRIBE from this list:
|http://lists.meer.net/mailman/listinfo/analog-help
|
| Analog Documentation: http://analog.cx/docs/Readme.html
| List archives: http://www.analog.cx/docs/mailing.html#listarchives
| Usenet version: news://news.gmane.org/gmane.comp.web.analog.general
+
Re: [analog-help] Count unique hosts
On Wednesday, April 12, 2006 8:10 PM [EDT], Andrew Ganegama <[EMAIL PROTECTED]> wrote: > I'm running the HOST report on a single file and I'd like to get a > count of the total number of unique hosts that accessed the file. The > HOST report > only gives me the following columns: > > reqs %bytes host > 2 50% 200.200.70.180 > 2 50% 200.200.70.181 > > How can I get a count of the 'host' column in the report? ie.: > > hosts > 2 You can add N to HOSTCOLS (http://analog.cx/docs/othreps.html#othCOLS) or you can look at the "Distinct hosts served" entry in the General Summary. Aengus + | TO UNSUBSCRIBE from this list: |http://lists.meer.net/mailman/listinfo/analog-help | | Analog Documentation: http://analog.cx/docs/Readme.html | List archives: http://www.analog.cx/docs/mailing.html#listarchives | Usenet version: news://news.gmane.org/gmane.comp.web.analog.general +
Re: [analog-help] HEADER / FOOTER FILES - failing to open?
On Tuesday, April 11, 2006 11:08 AM [EDT], [EMAIL PROTECTED] <[EMAIL PROTECTED]> wrote: > Hi... have been trying to include "header" and "footer" files in > Analog reports. > > Are the files to include "html" / "txt" / "inc" files? Have tried > files with all three extensions but when I look in my error file I > get the following; > > "F: Failed to open header file.." and then it lists the link in my > analog.cfg. Header and foot files aren't "links", they're files, like C:\analog\footer.txt, (or whatever is an appropriate specification for your particular operating system). > Does the Header/Footer files have to be in a certain format etc..? If Analog "Failed to open" the file, it obviously can't have disliked the format, as it never saw the format, becuse it couldn't open the file. Aengus + | TO UNSUBSCRIBE from this list: |http://lists.meer.net/mailman/listinfo/analog-help | | Analog Documentation: http://analog.cx/docs/Readme.html | List archives: http://www.analog.cx/docs/mailing.html#listarchives | Usenet version: news://news.gmane.org/gmane.comp.web.analog.general +
Re: [analog-help] Downloads
On Monday, April 10, 2006 6:29 AM [EDT], Jim Mann Taylor <[EMAIL PROTECTED]> wrote: >> I have been puzzling over the figures with Analog for a downloadable >> file. They concern a pdf file of some 68KB. When that is clicked on >> a web >> page a download box appears asking whether one wants to open the file >> or save it. >> When I therefore look in the Analog Request Report for that file and >> find say 200 requests, to get an idea of the number of downloads or >> openings, do I have to divide by 2 (one request to open the download >> box and one request to open or save the file)? No. A "save or open" dilog should only generate one request from your browser - You can verify this by looking at the log file immediately after you request the file yourself. > I appreciate that some >> of the requests could be inflated by download software downloading >> in parts. It's pretty unusal to get an "Open or Save" dialog for a .pdf file. Usually, browsers are configured to open documents that are sent as application/pdf automaticlly in a PDF reader (for example, if you click on http://www.irs.gov/pub/irs-pdf/fw4.pdf). If your server is sending the document type as application/pdf, then your browser is probably not configured the way the vast majority of your visitors browsers are configured. And because of the way Acrobat Reader works, a single "reader" can make multiple requests for the same file (the document can be requested a page at a time). If that's happening on your server, there's no "one size fits all" way to estimate the number of unique document accesses. Aengus + | TO UNSUBSCRIBE from this list: |http://lists.meer.net/mailman/listinfo/analog-help | | Analog Documentation: http://analog.cx/docs/Readme.html | List archives: http://www.analog.cx/docs/mailing.html#listarchives | Usenet version: news://news.gmane.org/gmane.comp.web.analog.general +
Re: [analog-help] help: custom log format not identified by analog
On Monday, April 17, 2006 2:03 PM [EDT],
Mushtaque, Sayed Imtiaz (GE, Corporate,consultant)
<[EMAIL PROTECTED]> wrote:
> Hi
> I am new to analog and have just compiled the same for my
> environment. We run apache web server with a custom logformat string.
> I have tried several iterations of the logformat but no success yet.
> Pls assist me in setting up the logformat string. Below are my
> details.
>
> LogFormat specified in httpd.conf
> LogFormat "\"%{ClientIP}i\" \"%{sm_user}i\" \"%t\" \"%r\" \"%>s\"
> \"%b\" \"%T\" \"%D\" \"%{Referer}i\" \"%{User-Agent}i\" \"%{host}i\""
> combined
If you just add
APACHELOGFORMAT ("\"%{ClientIP}i\" \"%{host}i\"")
to your analog.cfg and run Analog, you'll get the following warning:
analog: Warning C: Bad argument in configuration command: ignoring it:
APACHELOGFORMAT -> LOGFORMAT (""%j" "%j" "[%d/%M/%Y:%h:%n:%j]"
"%j%w%r%wHTTP%j" "%c" "%b" "%t" "%D" "%f" "%B" "%v"")
analog: ...cont..: (reason: one item occurs twice in format)
A quick scan of your logformat shows that you have 2 entries for "Processing
Time", %t (processing time in seconds) and %D (processing time in
microseconds), so you can take the "translated" LOGFORMAT that Analog
displays, and change the %t to %j (Processing Time in seconds is a total
waste of space), and you end up with
LOGFORMAT (""%j" "%j" "[%d/%M/%Y:%h:%n:%j]" "%j%w%r%wHTTP%j" "%c" "%b" "%j"
"%D" "%f" "%B" "%v"")
This still isn't quite right, as you bracketed your original Apache
logformat with ", even though you used " within the logformt itself, so you
need to strip that first and last ". This gives
LOGFORMAT ("%j" "%j" "[%d/%M/%Y:%h:%n:%j]" "%j%w%r%wHTTP%j" "%c" "%b" "%j"
"%D" "%f" "%B" "%v")
which will recognize your logfile.
If you want, you can tighten that up somewhat with this LOGFORMAT string:
LOGFORMAT (%j[%d/%M/%Y:%h:%n:%j]" "%j %r %j" "%c" "%b" "%j" "%D" "%f" "%B"
"%v")
Aengus
>
> Sample log entry
> "3.14.2.203" "500448330" "[17/Apr/2006:12:13:21 -0400]" "GET /
> HTTP/1.1" "302" "-" "0" "2205" "-" "Mozilla/4.0 (compatible; MSIE
> 6.0; Windows NT 5.0; T312461; .NET CLR 1.0.3705; .NET CLR 1.1.4322)"
> "energy.home.ge.com"
>
> Thanks
> Imtiaz
>
> +
>> TO UNSUBSCRIBE from this list:
>>http://lists.meer.net/mailman/listinfo/analog-help
>>
>> Analog Documentation: http://analog.cx/docs/Readme.html
>> List archives: http://www.analog.cx/docs/mailing.html#listarchives
>> Usenet version: news://news.gmane.org/gmane.comp.web.analog.general
> +
+
| TO UNSUBSCRIBE from this list:
|http://lists.meer.net/mailman/listinfo/analog-help
|
| Analog Documentation: http://analog.cx/docs/Readme.html
| List archives: http://www.analog.cx/docs/mailing.html#listarchives
| Usenet version: news://news.gmane.org/gmane.comp.web.analog.general
+
Re: [analog-help] QDNS program and DNSBADHOURS
On Tuesday, April 18, 2006 10:14 AM [EDT], Robert D. <[EMAIL PROTECTED]> wrote: > Hello Analog List: > > I have about 2 weeks of logs and have noticed that if I do a WhoIs on > many , reported as unresolved, numerical IP's, that I get results > whereas the file dnscache.txt still has no resolved entry. For example > > 19085401 4.224.189.246 * > 19062553 4.225.5.23 dialup-4.225.5.23.Dial1.Cincinnati1.Level3.net > > that first entry easily resolves to > dialup-4.224.189.246.Dial1.Cincinnati1.Level3.net > > So I thought I would set DNSBADHOURS or even good-hours, to One [1] > and see IF qdns.exe would check them again. I watch the output window > and see no extra lookups, all zero's, as it goes by until the report > comes up. > > Documentation for QDNS states the proggy reads and uses the analog.cfg > file. QDNS only uses the analog.cfg file if you explicitly tell it to - and in my experience, the only way to get it to resolve previously unresolved addresses is to strip them out (with /s) and rerun qdns against all the logfiles. The author of QDNS ha never answered any questions about it on this list, a far as I know, so any unexplained or buggy behaviour is likely to remain unexplained and buggy. Aengus + | TO UNSUBSCRIBE from this list: |http://lists.meer.net/mailman/listinfo/analog-help | | Analog Documentation: http://analog.cx/docs/Readme.html | List archives: http://www.analog.cx/docs/mailing.html#listarchives | Usenet version: news://news.gmane.org/gmane.comp.web.analog.general +
Re: [analog-help] Getting HOSTNAME/HOSTURL to appear with HEADERFILE
On Friday, April 21, 2006 1:31 PM [EDT], Mike VanHorn <[EMAIL PROTECTED]> wrote: > I've since figured out that it actually *is* putting the line in > there that I thought was missing, but it's putting it before the > HEADERFILE. In other words, this is the order of the file: > > HOSTNAME/HOSTURL line > HEADERFILE > the report > FOOTERFILE > > Since I'm trying to use HEADERFILE to put in the site banner and > navigation menus, this isn't working the way I was wanting it to. So, > what I want to do is to be able to have it to it this way: > > HEADERFILE > HOSTNAME/HOSTURL line > the report > FOOTERFILE > > Failing getting Analog to reorder how it does things, is there a way > to tell Ananlog not to put the very top-most HTML into the output > (i.e. Just output the report part). If that's the case, I can include > the report output in some other file which would contain the code > currently contained in HEADERFILE and FOOTERFILE. Analogs outputs styles are documented here - http://analog.cx/docs/output.html#outstyle If you want to delete the top and bottom of Analogs HTML forms, you'll have to do it yourself. You might be able to use the Timian app that uses Analogs XML output to create custom reports, but I've never used it, so I don't have any direct experience. Aengus + | TO UNSUBSCRIBE from this list: |http://lists.meer.net/mailman/listinfo/analog-help | | Analog Documentation: http://analog.cx/docs/Readme.html | List archives: http://www.analog.cx/docs/mailing.html#listarchives | Usenet version: news://news.gmane.org/gmane.comp.web.analog.general +
Re: [analog-help] help: can analog tell us the source ip address countby time intervals
On Friday, April 21, 2006 2:54 PM [EDT], Mushtaque, Sayed Imtiaz (GE, Corporate,consultant) <[EMAIL PROTECTED]> wrote: > Hi > Can analog report data on hits by source ip address indicating the > total count of hits coming from each ip address Yes. That's what the Host Report does. HOST ON > withing specific time intervals. No. You can easily do seperate report for each time interval, with the number of hits from each IP address in that interval, but you won't get a single report with counts per host within each time interval. http://analog.cx/docs/faq.html#faq128 Aengus + | TO UNSUBSCRIBE from this list: |http://lists.meer.net/mailman/listinfo/analog-help | | Analog Documentation: http://analog.cx/docs/Readme.html | List archives: http://www.analog.cx/docs/mailing.html#listarchives | Usenet version: news://news.gmane.org/gmane.comp.web.analog.general +
Re: [analog-help] Analog won't show requests with query strings
On Saturday, April 22, 2006 11:20 PM [EDT], Michael Bluejay <[EMAIL PROTECTED]> wrote: > Hi, I just signed up for this list to try to get some help with a > problem I'm having. > > I can't get Analog to show requests with a search query string > hierarchically. For example, I'm trying to get it to show something > like this: > > -- > Request Report > > #reqs file > --- -- > 125 /seo/ >16/seo/?adwords > -- > > Instead it just shows the first line and not the second. I think it > used to show the subcategories, so I figured I must have messed up my > analog.conf file somehow by adding something to it, so I tried > running Analog with an *empty* analog.conf file, but I still have the > same problem. I also tried adding ARGSINCLUDE *.* and REQARGSFLOOR > -50r to the analog.conf file but that didn't help either. I checked with one of my own logfiles, and, using the default analog.cfg file, I get hierarchical request reports without any additional commands. > > Here's the sample logfile I'm using: http://WebsiteHelpers.com/ > analog/WebSTAR.log I grabbed your logfile, and didn't get any arguments in the Request Report. I looked at the logfile, and I see that thee entries in question are redirects: "GET /seo/?adwords HTTP/1.1" 301 The is addressed in the FAQ: http://analog.cx/docs/faq.html#faq184 I can't find /script.pl?q=1 in the Request Report. If it causes a redirection, it will be in the Redirection Report not the Request Report. Sure enough, if you add REDIR ON, you'll get reqs file 11 /seo/ 11 /seo/?adwords in the Redirection Report. Aengus + | TO UNSUBSCRIBE from this list: |http://lists.meer.net/mailman/listinfo/analog-help | | Analog Documentation: http://analog.cx/docs/Readme.html | List archives: http://www.analog.cx/docs/mailing.html#listarchives | Usenet version: news://news.gmane.org/gmane.comp.web.analog.general +
Re: [analog-help] Re: Analog won't show requests with query strings
On Sunday, April 23, 2006 12:26 PM [EDT], Michael Bluejay <[EMAIL PROTECTED]> wrote: >> This is addressed in the FAQ: http://analog.cx/docs/faq.html#faq184 >> I can't find /script.pl?q=1 in the Request Report. >> If it causes a redirection, it will be in the Redirection Report >> not the Request Report. >> >> Sure enough, if you add REDIR ON, you'll get >> reqs file >> 11 /seo/ >> 11 /seo/?adwords >> >> in the Redirection Report. > > > Thanks, that explains it. I had searched the FAQ for "query" but > that didn't find anything so I didn't think it was in the FAQ. > Whoops. > > I'd spent hours trying to fix this problem before posting here. But > at least I learned a hell of a lot about Analog in the process that I > didn't know before. Turning on all the different reports (With ALL ON) can sometimes be helpful when you're just finding your way with Analog. The Redirection Report is off by default, so you wouldn't normally come across it. Aengus + | TO UNSUBSCRIBE from this list: |http://lists.meer.net/mailman/listinfo/analog-help | | Analog Documentation: http://analog.cx/docs/Readme.html | List archives: http://www.analog.cx/docs/mailing.html#listarchives | Usenet version: news://news.gmane.org/gmane.comp.web.analog.general +
Re: [analog-help] Using Date Codes for Outfile name in command lineparameters
On Monday, April 24, 2006 9:31 AM [EDT], Mark Edward <[EMAIL PROTECTED]> wrote: > Hi all, > > I'm using a single config file with Analog, for multiple reports, and > passing in different command line options to produce the reports > required. > > This is working fine apart from when I use Date Codes (e.g. %Y, %M for > Year and Month) in the Outfile option. Instead of replacing the code > with the required Year and Month value, the literal string 'M' is > used. > > E.g. the following command in my batch file > > call analog.exe +g"siteConfig.cfg" -G +F-00-0001 +T-00- > +O"C:\Inetpub\wwwroot\Site Stats\monthly-%Y-%M.html" -f -r > > Produces the report filename "monthly-M.html" instead of > "monthly-2006-04.html" > > I've tested this Outfile option in the actual config file, and it > worked fine, just not when passed in through the command line. > > Could it be a Windows reserved character issue? I've tried replacing > the % with %% to no avail. It looks like a problem with "CALL", not with Analog. I can only reproduce your problem when I put a line like yours in a batch file (with %% rather than %, of course). If I call a batchfile instead of analog.exe, and have the batchfile echo the parameters passed, I see the bare "M" test.bat: call analog.bat test.log +O"%%Y%%M.html" analog.bt: echo %* this is the output: C:\Analog>test C:\Analog>call analog.bat test.log +O"%Y%M.html" C:\Analog>echo test.log +O"M.html" test.log +O"M.html" It looks like CALL is converting %Y%M to M. But I'm not sure why you're using CALL - it's for calling batchfiles, and shouldn't be needed for an executable. Aengus + | TO UNSUBSCRIBE from this list: |http://lists.meer.net/mailman/listinfo/analog-help | | Analog Documentation: http://analog.cx/docs/Readme.html | List archives: http://www.analog.cx/docs/mailing.html#listarchives | Usenet version: news://news.gmane.org/gmane.comp.web.analog.general +
Re: [analog-help] SSL report?
On Monday, April 24, 2006 6:33 PM [EDT], Brad Kelley <[EMAIL PROTECTED]> wrote: > Is there a way to get Analog to report on https > requests vs. http requests on a Windows 2000 server? > Counts of each type , percentage, or a pie chart > showing the two service types would be fine. The logs > include port 80 for http and port 443 for https but I > see no way of using that data in Analog 6.0. Create a custom LOGFORMAT with an otherwise unused field such as %u or %v for the port field, and use the User or Virtual Host Report to see the relative proportions of traffic on the 2 ports. This is addressed in the FAQ: http://analog.cx/docs/faq.html#faq182 Aengus + | TO UNSUBSCRIBE from this list: |http://lists.meer.net/mailman/listinfo/analog-help | | Analog Documentation: http://analog.cx/docs/Readme.html | List archives: http://www.analog.cx/docs/mailing.html#listarchives | Usenet version: news://news.gmane.org/gmane.comp.web.analog.general +
Re: [analog-help] Live site only reports current day?
On Wednesday, April 26, 2006 12:37 AM [EDT], Ross Crawford <[EMAIL PROTECTED]> wrote: > Hi, I'm new to analog, so bear with me. > > I downloaded some logs from my site to play around with analog on my > own machine. I got a analog.cfg that I was happy with, and uploaded > everything to my site, but when I run analog with no changes to the > cfg, it only returns results for today, even though logs exist for > much more than that. > > The site is running in IIS, it's using standard IIS log format, and > the filename format is ex%y%M%D.log > > If I download 5 log files to my pooter and run it, I get 5 days > results, but if I run it on the server, I only get today's results, > no matter how many log files are present. > > I tried specifying a +F010101 on the command line, but no difference. > It is not generating errors, the errors.txt file is empty, and the > report generated is correct, I just can't get it to include > historical data. > > Can anyone help? Not based on the information you've given. What's your LOGFILE statement? If you set SETTINGS ON, what does Analog say it's using? Aengus + | TO UNSUBSCRIBE from this list: |http://lists.meer.net/mailman/listinfo/analog-help | | Analog Documentation: http://analog.cx/docs/Readme.html | List archives: http://www.analog.cx/docs/mailing.html#listarchives | Usenet version: news://news.gmane.org/gmane.comp.web.analog.general +
Re: [analog-help] Live site only reports current day?
On Wednesday, April 26, 2006 9:23 PM [EDT], Ross Crawford <[EMAIL PROTECTED]> wrote: > Aengus wrote: >> On Wednesday, April 26, 2006 12:37 AM [EDT], >> Ross Crawford <[EMAIL PROTECTED]> wrote: >> >>> Hi, I'm new to analog, so bear with me. >>> >>> I downloaded some logs from my site to play around with analog on my >>> own machine. I got a analog.cfg that I was happy with, and uploaded >>> everything to my site, but when I run analog with no changes to the >>> cfg, it only returns results for today, even though logs exist for >>> much more than that. >>> >>> The site is running in IIS, it's using standard IIS log format, and >>> the filename format is ex%y%M%D.log >>> >>> If I download 5 log files to my pooter and run it, I get 5 days >>> results, but if I run it on the server, I only get today's results, >>> no matter how many log files are present. >>> >>> I tried specifying a +F010101 on the command line, but no >>> difference. It is not generating errors, the errors.txt file is >>> empty, and the report generated is correct, I just can't get it to >>> include historical data. >>> >>> Can anyone help? >> >> Not based on the information you've given. What's your LOGFILE >> statement? If you set SETTINGS ON, what does Analog say it's using? > > Thanks for your reply. My logfile statement is: > > LOGFILE "\path\to\logs\ex%y%M%D.log" That's LOGFILE "\path\to\logs\ex060426.log". Just todays logfile. > and the relevant part from the SETTINGS output is: > > Reading logfiles: >\path\to\logs\ex060426.log > Logfile format: >\n > > But there are definitely 5 files in that directory: > ex060422.log > ex060423.log > ex060424.log > ex060425.log > ex060426.log > > Do you need anything else from the settings output? The debug output > also just showed it reading the 1 file. You only told it to read one logfile. LOGFILE "\path\to\logs\*.log" will read all the logs in the directory. LOGFILE "\path\to\logs\ex%y%M*.log" will read this months logs. Aengus + | TO UNSUBSCRIBE from this list: |http://lists.meer.net/mailman/listinfo/analog-help | | Analog Documentation: http://analog.cx/docs/Readme.html | List archives: http://www.analog.cx/docs/mailing.html#listarchives | Usenet version: news://news.gmane.org/gmane.comp.web.analog.general +
Re: [analog-help] Different Host one User ID
On Thursday, April 27, 2006 1:49 AM [EDT], [EMAIL PROTECTED] <[EMAIL PROTECTED]> wrote: >> Dear Reader, >> >> I am trying to analyze some Log files in following format >> >> >> >> How could I get a report in which I could see the User ID used by >> which Hosts. >> >> For example I have a userid testuser, I would see which different >> host in my logfiles uses the UserID testuser >> >> Regards Guenter http://analog.cx/docs/faq.html#faq128 Anlog won't crceate a siniigle report that lists all IP addresses used by each user. But you can get a list of all the IP addresses used by any single user by running Analog with USERINCLUDE testuser ALL OFF HOST ON (ALL OFF just turns off all the other reports, assuming you're only concerned about the Host Report in this case). Aengus + | TO UNSUBSCRIBE from this list: |http://lists.meer.net/mailman/listinfo/analog-help | | Analog Documentation: http://analog.cx/docs/Readme.html | List archives: http://www.analog.cx/docs/mailing.html#listarchives | Usenet version: news://news.gmane.org/gmane.comp.web.analog.general +
Re: [analog-help] Re: Excluding directories
On Thursday, April 27, 2006 2:29 AM [EDT], Ross Crawford <[EMAIL PROTECTED]> wrote: > Duh, I worked it out. For the benefit of future users of the list archive, please include the solution when you post "I worked it out". Aengus + | TO UNSUBSCRIBE from this list: |http://lists.meer.net/mailman/listinfo/analog-help | | Analog Documentation: http://analog.cx/docs/Readme.html | List archives: http://www.analog.cx/docs/mailing.html#listarchives | Usenet version: news://news.gmane.org/gmane.comp.web.analog.general +
Re: [analog-help] Different Host one User ID
On Thursday, April 27, 2006 7:40 AM [EDT], [EMAIL PROTECTED] <[EMAIL PROTECTED]> wrote: >> Dear Aengus, >> >> could I set up a report in which I see to each host the used User ID, >> >> for example >> >> HostUserID >> 10.10.10.1 testuser >> 10.10.10.2 testuser2 >> 10.10.10.3 testuser3 >> an so on ? Not in a single report - as the FAQ says, if you want to cross reference two different sorts of items, you have to restrict the analysis of one of the two to a single item at a time - you can get a list of all the IP addresses that testuser2 used, or a list of all the users that came from 10.10.10.3. You would have to manually combine multiple reports if you want to have them all in the same reporter. Aengus + | TO UNSUBSCRIBE from this list: |http://lists.meer.net/mailman/listinfo/analog-help | | Analog Documentation: http://analog.cx/docs/Readme.html | List archives: http://www.analog.cx/docs/mailing.html#listarchives | Usenet version: news://news.gmane.org/gmane.comp.web.analog.general +
Re: [analog-help] How to exclude host before/after a certain date
On Tuesday, May 02, 2006 10:51 AM [EDT], Torres, Pablo <[EMAIL PROTECTED]> wrote: > Using Analog 6.0, is there a way to hostexclude an IP address > before/after a specific date? For example, exclude 1.2.3.4 before Jan > 1st, 2006, but not after? Not directly. You could create cache files from the earlier logfiles, excluding the IP address, and then use that cachefile with the later logfiles to create a report. That would probably work, but it wouldn't be a general solution to the problem, which is a version of FAQ 128 - http://analog.cx/docs/faq.html#faq128 Aengus + | TO UNSUBSCRIBE from this list: |http://lists.meer.net/mailman/listinfo/analog-help | | Analog Documentation: http://analog.cx/docs/Readme.html | List archives: http://www.analog.cx/docs/mailing.html#listarchives | Usenet version: news://news.gmane.org/gmane.comp.web.analog.general +
Re: [analog-help] Count unique hosts
On Wednesday, May 03, 2006 8:39 PM [EDT], Andrew Ganegama <[EMAIL PROTECTED]> wrote: > Thanks for your reply. > > In the case where multiple visitors have the same gateway ip address, > ie. in a corporate network with NAT, is there a way to get the unique > visitors? Define a visitor. If you can come up with a definition of "visitor" that doesn't involve an IP address, then getting a count of unique visitors might be possible. The most obvious solution is to use cookies, and assign a unique permanent cookie to any new "visitor". Then you can use that cookie as a "user" field. The issue is discussed in greater detail in http://analog.cx/docs/webworks.html Aengus + | TO UNSUBSCRIBE from this list: |http://lists.meer.net/mailman/listinfo/analog-help | | Analog Documentation: http://analog.cx/docs/Readme.html | List archives: http://www.analog.cx/docs/mailing.html#listarchives | Usenet version: news://news.gmane.org/gmane.comp.web.analog.general +
Re: [analog-help] Generating a report
On Friday, May 05, 2006 11:32 AM [EDT], [EMAIL PROTECTED] <[EMAIL PROTECTED]> wrote: > I need to set up an Analog report for a domain alias to a main site. > For example, the main site is "www.tasafyns.com" and the domain alias > is "www.tasafyn.com". (We've got plural and singular versions of the > domain pointing to the same server). I presume tasafyn.com is a made up example, as it doesn't resolve for me. Can you confirm that both names resolve to the same IP address, and that if you access that IP address from your browser that you get to your site? > > The site is being hosted on IIS. There is no referrer field in the > web log. I need to generate a report of the requests that come from > "www.tasafyn.com" as opposed to the main domain name, > "www.tasafyns.com". I can see nothing in the logs that I could use > to filter the report; however, when I look at the referring sites > report, "www.tasafyn.com" shows up as a referring site. If there is a Referring Sites report, then Analog is seeing a Referrer field in your logs. Analog will not generate a Referring Site report if it can't find any referrers. > How is analog resolving this and determining that the hit came from > "www.tasafyn.com" for the referring site report? Because that's what's recorded in the Referrer field :-) > I've tried doing > reverse DNS lookups on the client IP addresses with no luck. The > server IP address is the same for every line in the log, so that > doesn't help me. There is not a referrer field in the log, so I > don't see how Analog is arriving at the number for "www.tasafyn.com". > > If I could figure that out, I could use the same technique to create > a separate report with numbers strictly for "www.tasafyn.com". > > Anyone have any idea how to generate a report for a domain alias > based on the above criteria. If you've configured your web server to ignore the difference between both domain names then Analog won't be able to report on the difference, because it's not being recorded. Even a Referring Sites report doesn't really tell you what you want to know, because it only tells you what names are being used after "visitors" arrive at your site, it doesn't tell you what names are used on the links that they follow to get there. IIS 5 and above allow you to log the hostname used in HTTP requests (cs-host). You should turn that field on if you want this kind of information. If you're using ASP pages on your IIS server, you might also consider adding a few lines to your global.asa to log the hostname used and the referring page. > > Any help would be appreciated. > > Thanks. > > Lionel Aengus + | TO UNSUBSCRIBE from this list: |http://lists.meer.net/mailman/listinfo/analog-help | | Analog Documentation: http://analog.cx/docs/Readme.html | List archives: http://www.analog.cx/docs/mailing.html#listarchives | Usenet version: news://news.gmane.org/gmane.comp.web.analog.general +
Re: [analog-help] how do you report on whats after the "?" in the url
On Monday, May 08, 2006 10:40 AM [EDT], [EMAIL PROTECTED] <[EMAIL PROTECTED]> wrote: Hi, I have url's which look thus:- /cgi-bin/caa_rdm?DATE=060420&SC=1:2:4&DATA=FGM:LTEF:LTOF:PEACE:SATT:SC:STEF:STOF:TCAL:WEC&OUTPUT=zip HTTP/1.1" 200 350152474 "-" Is there any way to get analog to report on the contents of the above &DATA=... i.e. can it divide this into stats for each component FGM, LTEF, LTOF etc You can use the Internal Search Engine Reports (http://analog.cx/docs/args.html#SEARCHENGINE) to report on the contents of a particular field that is passed as an argument to a script. With the sample entry you provided, Analog will only recognize a single "word" in the query - Internal Search Query Report This report lists which queries people used on scripts within the site. Listing queries, sorted by the number of requests. reqs: search term : --- 1: fgm:ltef:ltof:peace:satt:sc:stef:stof:tcal:wec Internal Search Word Report --- This report lists which words people used on scripts within the site. Listing query words, sorted by the number of requests. reqs: search term : --- 1: fgm:ltef:ltof:peace:satt:sc:stef:stof:tcal:wec If the argument was DATA=FGM+LTEF+LTOF+PEACE+SATT+SC+STEF+STOF+TCAL+WEC, Analog would report one Query, and 10 words. Internal Search Query Report This report lists which queries people used on scripts within the site. Listing queries, sorted by the number of requests. reqs: search term : --- 1: fgm ltef ltof peace satt sc stef stof tcal wec Internal Search Word Report --- This report lists which words people used on scripts within the site. Listing query words, sorted by the number of requests. reqs: search term : --- 1: stef 1: ltof 1: sc 1: fgm 1: ltef 1: satt 1: wec 1: stof 1: tcal 1: peace -------- Aengus + | TO UNSUBSCRIBE from this list: |http://lists.meer.net/mailman/listinfo/analog-help | | Analog Documentation: http://analog.cx/docs/Readme.html | List archives: http://www.analog.cx/docs/mailing.html#listarchives | Usenet version: news://news.gmane.org/gmane.comp.web.analog.general +
Re: [analog-help] Excluding some search words
On Tuesday, May 09, 2006 11:46 AM [EDT], Mike VanHorn <[EMAIL PROTECTED]> wrote: Is there a way to exclude certain terms from the search words report? The top sixe terms on my search word report are of or and to in a And I'd rather just not have words like that show up in the report. http://analog.cx/docs/include.html#outputexcludes Aengus + | TO UNSUBSCRIBE from this list: |http://lists.meer.net/mailman/listinfo/analog-help | | Analog Documentation: http://analog.cx/docs/Readme.html | List archives: http://www.analog.cx/docs/mailing.html#listarchives | Usenet version: news://news.gmane.org/gmane.comp.web.analog.general +
Re: [analog-help] Excluding some search words
On Tuesday, May 09, 2006 1:59 PM [EDT], Mike VanHorn <[EMAIL PROTECTED]> wrote: On 5/9/06 1:26 PM, "Aengus" <[EMAIL PROTECTED]> wrote: On Tuesday, May 09, 2006 11:46 AM [EDT], Mike VanHorn <[EMAIL PROTECTED]> wrote: Is there a way to exclude certain terms from the search words report? ... http://analog.cx/docs/include.html#outputexcludes Those excludes allow you to keep certain hosts from showing up in the report; they don't keep Analog from making an entry for the word "or" in the report. Does that help to explain my problem? It's not that I want to exclude certain log file lines from being processed, I want to tell Analog not to put articles and conjunctions into the Search Word report. The outputexcludes that I pointed to "don't exclude logfile lines but individual lines from particular reports." The list of Output Excludes includes one which does exactly what you are trying to do - it excludes words from the Search Word report. Aengus + | TO UNSUBSCRIBE from this list: |http://lists.meer.net/mailman/listinfo/analog-help | | Analog Documentation: http://analog.cx/docs/Readme.html | List archives: http://www.analog.cx/docs/mailing.html#listarchives | Usenet version: news://news.gmane.org/gmane.comp.web.analog.general +
Re: [analog-help] Excluding some search words
On Tuesday, May 09, 2006 2:36 PM [EDT], Mike VanHorn <[EMAIL PROTECTED]> wrote: Okay, I got it working now. I was trying that before, and it wasn't making any difference in the report output. Turns out I had the syntax wrong (I was listing multiple words on the same line, rather than one word per SEARCHWORDEXCLUDE line). Thank you for confirming my original suspicion that SEARCHWORDEXCLUDE was what I needed! SEARCHWORDEXCLUDE of,or,and,to,in,a should exclude all of those words from the Search Word Report. Seperate entries for each word will work too, but it should be possible to get them all on a single line. Aengus + | TO UNSUBSCRIBE from this list: |http://lists.meer.net/mailman/listinfo/analog-help | | Analog Documentation: http://analog.cx/docs/Readme.html | List archives: http://www.analog.cx/docs/mailing.html#listarchives | Usenet version: news://news.gmane.org/gmane.comp.web.analog.general +
Re: [analog-help] Semi-newbie Question: Multiple Servers
On Thursday, May 11, 2006 11:41 AM [EDT], Sokolow, Daniel <[EMAIL PROTECTED]> wrote: I do NOT want a single report for all three servers; I want individual reports for each. These are running on our intranet, and I access them all locally. The .cfg file I use on the one server already underway has LOGFILE g:\...etc. as the path to the server. So am I correct that in order to get the separate reports for each server, I will only need to create new *.cfg files with the new path to the new servers? That's correct. Your additional config files will probably also need new LOGFORMAT, OUTFILE, and *CHARTDIR commands too as well as any appropriate customizations. Will I then have to go back into the original analog.cfg and manually redirect to each new *.cfg file for each server I want reports from? Or do I need to rename each *.cfg as analog.cfg each time I want to run the program for that particular server? Is there any way to automate the process so a single run of Analog will create four separate reports from four separate servers? http://analog.cx/docs/faq.html#faq110 analog +gserver1.cfg analog +gserver2.cfg analog +gserver3.cfg analog +gserver4.cfg You need 4 runs, but you can tell Ananlog whic additional configfile to use on the commandline. Analog will still read analog.cfg, so common commands, such as search engines, logtimeoffsets, *IMAGEDIRs, etc can be kept in a single file, and report specific commands are kept in the others. It may help to think in terms of reports, rather than servers. You don't need a seperate .cfg file for each server, you need one for each type of report you want to generate. In your case, you want 4 different reports (which happen to be one for each of 4 servers), so you have 4 .cfg files. But if you wanted 4 different reports from a single servers logs, you might still want 4 different .cfgs. Aengus + | TO UNSUBSCRIBE from this list: |http://lists.meer.net/mailman/listinfo/analog-help | | Analog Documentation: http://analog.cx/docs/Readme.html | List archives: http://www.analog.cx/docs/mailing.html#listarchives | Usenet version: news://news.gmane.org/gmane.comp.web.analog.general +
Re: [analog-help] Semi-newbie Question: Multiple Servers
On Thursday, May 11, 2006 12:28 PM [EDT], Sokolow, Daniel <[EMAIL PROTECTED]> wrote: When actually running the report, do I need to comment out the CFG files I want to skip for that run? I.e., if I want the numbers on server #1, should the G+ lines for the other 3 servers be commented out, and then down the line for each specific report? Commented out of where? The examples that I showed are separate command lines. You can stick all 4 in a batch file, to create all 4 reports, one after the other, or create 4 icons on your desktop with those properties, or you could just type the commands on the command line as needed. It all depends on how you work with Analog - I do all my work at the command line, you may not. Also, If I want to use REQINCLUDE on some reports but not on others, which takes precedence? That is, if the analog.cfg file has REQINCLUDE *.dll, but I don't need that for server #1, and I do a REQEXCLUDE *.dll, which of the two configs will work? http://analog.cx/docs/faq.html#faq123 The later command usually takes precedence. If you use a CONFIGFILE command in the middle of analog.cfg, the contents of the CONFIGFILE will effectively be in the middle of analog.cfg - lines in the analog.cfg that come after the CONFIGFILE will take precendence over anything in the CONFIGFILE. Analog reads analog.cfg before it reads the command line, so the +g argument is like putting CONFIGFILE at the very end of the analog.cfg - it will supercede any conflicting statement in the analog.cfg Though I suppose I could leave all those In & Excludes on the sub CFG files and not even mention it on the analog.cfg. It depends on how often you want to change the default. But with only 4 reports, t probably makes sense to put any non-global changes into the individual .cfg files, rather than into analog.cfg. Aengus + | TO UNSUBSCRIBE from this list: |http://lists.meer.net/mailman/listinfo/analog-help | | Analog Documentation: http://analog.cx/docs/Readme.html | List archives: http://www.analog.cx/docs/mailing.html#listarchives | Usenet version: news://news.gmane.org/gmane.comp.web.analog.general +
Re: [analog-help] REGEXP question
On Thursday, May 11, 2006 8:53 PM [EDT], Moshe <[EMAIL PROTECTED]> wrote: The log file has lines like this GET /Content/ContentUnit.asp CID=145&u=3048&t=0 200 The cfg file has FILEALIAS REGEXP:u=3048 PLAIN:Prospective The errors.txt shows: analog: Warning C: Too many arguments for configuration command: ignoring end of line starting: FILEALIAS REGEXP:u=3048 PLAIN:Prospective The outfile does not have any line with "Prospective" What am I doing wrong? What should the REGEXP be? It works for me: Request Report -- This report lists the files on the site. Listing files, sorted by the number of requests. reqs: %bytes: last time: file : --: ---: 1: : 31/Aug/05 10:41: prospective Aengus + | TO UNSUBSCRIBE from this list: |http://lists.meer.net/mailman/listinfo/analog-help | | Analog Documentation: http://analog.cx/docs/Readme.html | List archives: http://www.analog.cx/docs/mailing.html#listarchives | Usenet version: news://news.gmane.org/gmane.comp.web.analog.general +
Re: [analog-help] executing with a diffrent config file
On Friday, May 12, 2006 7:50 AM [EDT], El Gringo <[EMAIL PROTECTED]> wrote: sorry if this is to easy, but I haven't found (or maybe I have overseen) the possibility to use a diffrent config file in commandline modus. Is it possible to specify a diffrent config file, than the standart? http://analog.cx/docs/syntax.html#CONFIGFILE Aengus + | TO UNSUBSCRIBE from this list: |http://lists.meer.net/mailman/listinfo/analog-help | | Analog Documentation: http://analog.cx/docs/Readme.html | List archives: http://www.analog.cx/docs/mailing.html#listarchives | Usenet version: news://news.gmane.org/gmane.comp.web.analog.general +
Re: [analog-help] Requests and spiders confusion
On Friday, May 12, 2006 2:10 PM [EDT], Moshe <[EMAIL PROTECTED]> wrote: I need your help to clear an issue for me: Let's say that google has crawled my website. The log should have many many entries for that day or week caused by the Google crawler. I am confused about if and how these entries will show in the different reports. General: Successful requests: Included? Not Included? A request is a request is a request. If it's a request, it's included in the Request Reort, unless you explicitly exclude it. Can I exclude crawler requests from this item? How? By HOSTEXCLUDE-ing the IP address of the spider/robot, or BROWEXCLUDE-ing the User Agent string used by the spider/robot. Successful requests for pages: and Distinct files requested: Included? Not Included? Can I exclude crawler requests from this item? How? I assume that answers for General will apply to the Daily, monthly and so on. Referer Report: I assume that crawlers entries in the log do not have referes? Right? That depends on the spider/robot. Browser summary report: What command controls this report. I have on my report browsers such as GoogleBot, msnbot. How can I exclude these from this report and only show "real" browsers? http://analog.cx/docs/include.html#outputexcludes You can use BROWREPEXCLUDE and BROWSUMEXCLUDE to exclude specific User Agents from the Browser Reports. These are "Output Excludes", and only remove the specified lines from those specific reports - the requests would still be counted in other reports. Of course, if you use the "Input Exclude" BROWEXCLUDE, the requests with the specified User agent are ignored before they are processed, so they don't show up in the Browser Reports either. You can also use ROBOTINCLUDE to tell Analog which User Agents are Robots for the Operating System Report. Also, if I excluded crawlers request in the first place (see the first section of my questions) the browser report will then not have lines for these crawlers right? Right. Request Report: How can make sure that crawlers entries in the log will not be counted in this report? See above. Aengus + | TO UNSUBSCRIBE from this list: |http://lists.meer.net/mailman/listinfo/analog-help | | Analog Documentation: http://analog.cx/docs/Readme.html | List archives: http://www.analog.cx/docs/mailing.html#listarchives | Usenet version: news://news.gmane.org/gmane.comp.web.analog.general +
Re: [analog-help] Anyone know where I can get a Solaris 9 compiledversion of Analog 6?
On Monday, May 15, 2006 3:52 PM [EDT], don <[EMAIL PROTECTED]> wrote: Hello Anyone know where I can get a Solaris 9 compiled version of Analog 6? I don't know if anyone running Solaris 9 has made a compiled version available for download, but in the event that you end up having to compile it yourself, this recently published article explains how to compile Aalog on Solaris 9, including some additional steps that you may need to take. http://open.itworld.com/5040/nls_unix_analog_060330/page_1.html http://www.itworld.com/nl/unix_insider/03302006/ (Both URLs point to the same article, but I've included both for the sake of the archive - it's possible that one may have a longer shelf-life than the other). Aengus + | TO UNSUBSCRIBE from this list: |http://lists.meer.net/mailman/listinfo/analog-help | | Analog Documentation: http://analog.cx/docs/Readme.html | List archives: http://www.analog.cx/docs/mailing.html#listarchives | Usenet version: news://news.gmane.org/gmane.comp.web.analog.general +
Re: [analog-help] Monthly Report - Distinct Hosts
On Tuesday, May 16, 2006 11:43 AM [EDT], Shaleel Valsan <[EMAIL PROTECTED]> wrote: I'm using Report Magic/Analog on a web server. Is there a way to determine the number of unique visitors to a site per month. The Monthly Report only shows the number of requests. The Distinct Hosts in the General Summary section shows the number of Unique Visitors to the site during the entire report period. But there seems to be no way to display the number of unique visitors per week or month like in AWStats. I'm thinking it's just a setting in one of the config files to display the Distinct Hosts column in the Monthly Report as well. Please help. To create a count of "unique hosts" Analog has to create a table in memory, and check whether each new host is unique or not. It does this once for the whole period of the logfile, but it would have to create seperate tables for each time period, whch would slow the process down, and eat memory. (It doesn't have to do this for request or Page counts, which are simple cumulative counts). So if you want to get a count of unique visitors in any gven month, run Analog against that months logs - Analog won't calculate the number for different months in a set of logs. (It's also not an obviously useful statistic, as it doesn't tell you whether the unique visitors in one month are the same or different from the unique visitors in another month. A sudden spike or drop in "unique visitors" won't tell you anything useful). Aengus + | TO UNSUBSCRIBE from this list: |http://lists.meer.net/mailman/listinfo/analog-help | | Analog Documentation: http://analog.cx/docs/Readme.html | List archives: http://www.analog.cx/docs/mailing.html#listarchives | Usenet version: news://news.gmane.org/gmane.comp.web.analog.general +
Re: [analog-help] Monthly Report - Distinct Hosts
On Tuesday, May 16, 2006 12:32 PM [EDT], Shaleel Valsan <[EMAIL PROTECTED]> wrote: Aengus, Thanks for the great explanation - you are the first person who gave me a proper answer. Since generating that statistic for each month consumes so much system resources, it would probably be wise for me to just generate it for the month in question,right? Could you advise me on how to run Analog against that month's logs? If your logfiles aren't seperated in such a way that you can easily specify the logs from a particular month (eg analog ex0604*.log), just set the TO and FROM commands - http://analog.cx/docs/include.html#FROMTO. e.g. analog +C"FROM 060401" +C"TO 060430" logfile.log (or put the TO and FROM in the analog.cfg file). Aengus I really appreciate your help with this. Thanks, Shal On 5/16/06, Aengus <[EMAIL PROTECTED]> wrote: On Tuesday, May 16, 2006 11:43 AM [EDT], Shaleel Valsan <[EMAIL PROTECTED]> wrote: I'm using Report Magic/Analog on a web server. Is there a way to determine the number of unique visitors to a site per month. The Monthly Report only shows the number of requests. The Distinct Hosts in the General Summary section shows the number of Unique Visitors to the site during the entire report period. But there seems to be no way to display the number of unique visitors per week or month like in AWStats. I'm thinking it's just a setting in one of the config files to display the Distinct Hosts column in the Monthly Report as well. Please help. To create a count of "unique hosts" Analog has to create a table in memory, and check whether each new host is unique or not. It does this once for the whole period of the logfile, but it would have to create seperate tables for each time period, whch would slow the process down, and eat memory. (It doesn't have to do this for request or Page counts, which are simple cumulative counts). So if you want to get a count of unique visitors in any gven month, run Analog against that months logs - Analog won't calculate the number for different months in a set of logs. (It's also not an obviously useful statistic, as it doesn't tell you whether the unique visitors in one month are the same or different from the unique visitors in another month. A sudden spike or drop in "unique visitors" won't tell you anything useful). Aengus + TO UNSUBSCRIBE from this list: http://lists.meer.net/mailman/listinfo/analog-help Analog Documentation: http://analog.cx/docs/Readme.html List archives: http://www.analog.cx/docs/mailing.html#listarchives Usenet version: news://news.gmane.org/gmane.comp.web.analog.general + + TO UNSUBSCRIBE from this list: http://lists.meer.net/mailman/listinfo/analog-help Analog Documentation: http://analog.cx/docs/Readme.html List archives: http://www.analog.cx/docs/mailing.html#listarchives Usenet version: news://news.gmane.org/gmane.comp.web.analog.general + + | TO UNSUBSCRIBE from this list: |http://lists.meer.net/mailman/listinfo/analog-help | | Analog Documentation: http://analog.cx/docs/Readme.html | List archives: http://www.analog.cx/docs/mailing.html#listarchives | Usenet version: news://news.gmane.org/gmane.comp.web.analog.general +
Re: [analog-help] Monthly Report - Distinct Hosts
On Tuesday, May 16, 2006 2:13 PM [EDT], Shaleel Valsan <[EMAIL PROTECTED]> wrote: Sorry about my total ignorance but i've never worked with Analog/Report Magic - my predecessor configured and set it up on the web server. I'm hunting high and low for the Analog log files for a particular website so that i can insert the To and From commands to it to generate a monthly report. I've no idea where it is... Analog is a cross platform application, so there is no standard installation location - on Windows, you just unzip it into the directory that you want to keep it in. Some people will put it in the "Program Files" folder, with all theother progras, but many other people put it in C:\Analog (it's a command line program, so accessing it from the command line is easier f you don't have to navigate into ProgramFiles). Just search for analog.*. analog.exe and analog.cfg are normally in the same directory. Edit analog.cfg to change settings http://analog.cx/docs/start.html Aengus + | TO UNSUBSCRIBE from this list: |http://lists.meer.net/mailman/listinfo/analog-help | | Analog Documentation: http://analog.cx/docs/Readme.html | List archives: http://www.analog.cx/docs/mailing.html#listarchives | Usenet version: news://news.gmane.org/gmane.comp.web.analog.general +
Re: [analog-help] Monthly Report - Distinct Hosts
On Wednesday, May 17, 2006 2:55 PM [EDT], Shaleel Valsan <[EMAIL PROTECTED]> wrote: Thanks for your reply. It's actually on a Linux server. I found the analog.cfg file. I'm assuming all the sites hosted on this server access the same config file because we have over a 100 sites running on the server and this is the only analog.cfg file i found. Setting the To and From in this file would then generate a monthly report for all the sites,right? Not necessarily. Does that analog.cfg file have LOGFILE commands pointing to the logs for all the sites? Maybe analog is called from a script that creates a seperate report for each site - it all depends on how it's set up on tht server. For instance, this guy wants info for Jan 2006 so if i set it From Jan 1 To Jan 31, it will generate January stats for all the sites? Will this action remove the information from the General Summary or just add a new column for that time period. When you run Analog, it creates a wole new report based on the parameters that you give it, so if you change the parameters, you will overwrite the existing report, unless you tell Analog to create a different OUTFILE. Aengus + | TO UNSUBSCRIBE from this list: |http://lists.meer.net/mailman/listinfo/analog-help | | Analog Documentation: http://analog.cx/docs/Readme.html | List archives: http://www.analog.cx/docs/mailing.html#listarchives | Usenet version: news://news.gmane.org/gmane.comp.web.analog.general +
Re: [analog-help] Browserrep
On Thursday, May 18, 2006 12:20 PM [EDT], Sokolow, Daniel <[EMAIL PROTECTED]> wrote: Clearly this is all but useless - I'm not really interested in what's compatible with what. All I really want to know is how many Firefox, how many IE, how many with WMP 9, WMP 10, etc. Use the Browser Summary - it's the next item after Browser Report in the index. Aengus + | TO UNSUBSCRIBE from this list: |http://lists.meer.net/mailman/listinfo/analog-help | | Analog Documentation: http://analog.cx/docs/Readme.html | List archives: http://www.analog.cx/docs/mailing.html#listarchives | Usenet version: news://news.gmane.org/gmane.comp.web.analog.general +
Re: [analog-help] Show IP# in searchword report
On Thursday, May 18, 2006 12:29 PM [EDT], Nicholas C. Shears <[EMAIL PROTECTED]> wrote: Is it possible to customize the "Search Word Report" so it shows the visitor's IP number for each search word? No. For a start, what would Analog display if 10 people searched on a single word? Aengus + | TO UNSUBSCRIBE from this list: |http://lists.meer.net/mailman/listinfo/analog-help | | Analog Documentation: http://analog.cx/docs/Readme.html | List archives: http://www.analog.cx/docs/mailing.html#listarchives | Usenet version: news://news.gmane.org/gmane.comp.web.analog.general +
Re: [analog-help] Browserrep
On Thursday, May 18, 2006 2:39 PM [EDT], Sokolow, Daniel <[EMAIL PROTECTED]> wrote: Well, duh! Five lashes with a wet noodle for a stupid question. I have another question, however, which I hope is not as dumb. What is the proper syntax for the BROWSUMINCLUDE and EXCLUDE statement so I can get the various versions of IE, Firefox, etc. to appear? I've tried a number of variations on the BROWSUMINCLUDE mozilla/5 and similar syntax, to no avail. I can see on the logfiles mentions of firefox and specific versions of IE, but all that keeps showing up is MSIE. The default settings for Analog 6 will show the various versions of IE, Firefox in the Browser Summary - this is an example from one of the logfiles that I have on hand. (I added BROWSUMFLOOR 1r, to make sure that they al showed up). Browser Summary --- This report lists the vendors of visitors' browsers. Listing browsers, sorted by the number of requests for pages. no.: reqs: pages: browser ---: : -: --- 1: 49:49: MSIE : 41:41: MSIE/6 :8: 8: MSIE/5 2: 27:27: Firefox : 27:27: Firefox/1 3: 24:24: Netscape (compatible) 4:5: 5: Safari :4: 4: Safari/417 :1: 1: Safari/312 5:4: 4: msnbot :4: 4: msnbot/1 6:2: 2: Mozilla :2: 2: Mozilla/1 7:1: 1: Netscape :1: 1: Netscape/4 8:1: 1: Opera :1: 1: Opera/8 9:1: 1: Firebird :1: 1: Firebird/0 Aengus + | TO UNSUBSCRIBE from this list: |http://lists.meer.net/mailman/listinfo/analog-help | | Analog Documentation: http://analog.cx/docs/Readme.html | List archives: http://www.analog.cx/docs/mailing.html#listarchives | Usenet version: news://news.gmane.org/gmane.comp.web.analog.general +
Re: [analog-help] Ageing off old data
On Friday, May 19, 2006 9:54 AM [EDT], Todd Beckett <[EMAIL PROTECTED]> wrote: I have log files from 3 years ago. Every day I run analog with an at command and reanalyze everything. This is valuable because I am able to show our year over year growth, as well as make year to date comparisons. The problem is that these files are beginning to eat up my hard drive space. Some of my log files are in excess of 50megs each. I could create a little batch/activex script that would wipe at the data from a certain date back, but then I would loose the ability to show where came from. Does anyone have a good idea on a way to "roll up" analog reports? First off, have you zipped the logfiles? Analog will report on zipped logfiles almost as fast as unzipped logfiles. Beyond that, you can use cache files to "condense" the information in older logfiles, with certain caveats, allowing you to create your longterm reports with smaller inputs. http://analog.cx/docs/cache.html Aengus + | TO UNSUBSCRIBE from this list: |http://lists.meer.net/mailman/listinfo/analog-help | | Analog Documentation: http://analog.cx/docs/Readme.html | List archives: http://www.analog.cx/docs/mailing.html#listarchives | Usenet version: news://news.gmane.org/gmane.comp.web.analog.general +
Re: Re: [analog-help] Ageing off old data
On Friday, May 19, 2006 10:25 AM [EDT], [EMAIL PROTECTED] <[EMAIL PROTECTED]> wrote: really? you can have your log files in zip format and Analog can still read them? Can I put a bunch of logs in 1 Zip file? Yes. http://analog.cx/docs/logfile.html#UNCOMPRESS Aengus + | TO UNSUBSCRIBE from this list: |http://lists.meer.net/mailman/listinfo/analog-help | | Analog Documentation: http://analog.cx/docs/Readme.html | List archives: http://www.analog.cx/docs/mailing.html#listarchives | Usenet version: news://news.gmane.org/gmane.comp.web.analog.general +
Re: [analog-help] Ageing off old data
On Friday, May 19, 2006 11:33 AM [EDT], Jeremy Wadsack <[EMAIL PROTECTED]> wrote: Actually, Analog can perform faster on zipped/compressed log files than uncompressed files. This is because the slowest part of processing is reading the disk. Memory and processor are so much faster that you can actually get better performance with moderately compressed files even though it has to go through the process of uncompressing them. This is particulary true if your logfiles are stored on a different machine, and are accessed over the network. You'll see a huge performance boost if you zip the logfiles first. But make sure that the zip process runs on the server that the logfiles start on - there's no point in sucking the files across the network to compress them. Aengus + | TO UNSUBSCRIBE from this list: |http://lists.meer.net/mailman/listinfo/analog-help | | Analog Documentation: http://analog.cx/docs/Readme.html | List archives: http://www.analog.cx/docs/mailing.html#listarchives | Usenet version: news://news.gmane.org/gmane.comp.web.analog.general +
Re: [analog-help] Repercussions of using standard log formats?
On Tuesday, May 23, 2006 12:02 PM [EDT], Cody Carey <[EMAIL PROTECTED]> wrote: We host a number of different types of sites. We have plain apache and netscape sites, special application sites, etc. The problem is alot of our sites have custom log formats. This causes us alot of problems in our processing of statistics. My basic question is if we created a single universal log format that included just about every piece of data what could the repercussions be aside from increased log file size and processing time? Log size can more than double if you include referrers and browser strings, if you weren't doing that before. Luckily, this is extremely compressible, because the strings are repeated so often. Adding cookies can also significantly increase the size of logs, depending on how many cookies are set by the applications. But I can't think of any negative repercussions, other than increased log size. Aengus + | TO UNSUBSCRIBE from this list: |http://lists.meer.net/mailman/listinfo/analog-help | | Analog Documentation: http://analog.cx/docs/Readme.html | List archives: http://www.analog.cx/docs/mailing.html#listarchives | Usenet version: news://news.gmane.org/gmane.comp.web.analog.general +
Re: [analog-help] XML pages
On Wednesday, May 24, 2006 4:32 PM [EDT], Jay Collier <[EMAIL PROTECTED]> wrote: In our reports, the percentage of requests that are "pages" is incredibly small, less than 1%. Perhaps this is because our CMS uses .xml as the extension for pages. Am I correct in assuming that the term "pages" in Analog reports refers to .html page types? If so, how can I add add pages with an .xml extension to the pages data? http://analog.cx/docs/indx.html#P Aengus + | TO UNSUBSCRIBE from this list: |http://lists.meer.net/mailman/listinfo/analog-help | | Analog Documentation: http://analog.cx/docs/Readme.html | List archives: http://www.analog.cx/docs/mailing.html#listarchives | Usenet version: news://news.gmane.org/gmane.comp.web.analog.general +
Re: [analog-help] Tracking User Downloads
On Thursday, May 25, 2006 3:38 PM [EDT], Brandon Gabert <[EMAIL PROTECTED]> wrote: Hi, apache22+analog6. I have a site, the main /index.html contains links to subdirs, every subdir has a .htaccess and no index so apache generates its index views. Everyone logs in with mod_auth and provides a username/password. What I'd like is to know which username downloaded which file and at what time. Ie: ASmith /dir/file.pdf - May 25, 2006 10:00 am /dir2/file2.pdf - May 25, 2006 10:01 am BJohnson /dir/file.pdf - May 10, 2006 9:00 am CGeorge /dir5/file5.pdf - May 15, 2006 2:00 pm I've seen this question asked a few times, but no real answer (at least none that I found). Is there way to do this with analog? http://analog.cx/docs/faq.html#faq128 You have to run Analog once for each user if you want to get per user statistics. It's fairly trivial to script, as long as the number of users and the size of the logs aren't hugely unweildy. Aengus + | TO UNSUBSCRIBE from this list: |http://lists.meer.net/mailman/listinfo/analog-help | | Analog Documentation: http://analog.cx/docs/Readme.html | List archives: http://www.analog.cx/docs/mailing.html#listarchives | Usenet version: news://news.gmane.org/gmane.comp.web.analog.general +
Re: [analog-help] Analog on Tomcat
On Friday, May 26, 2006 2:19 PM [EDT], Aruna Jain <[EMAIL PROTECTED]> wrote: Hello I'm trying to install Analog on a Linux Operating System (RedHat 9.0) running Tomcat. Could anybody help me in trying to figure out how to find the access log file on Tomcat so I can configure the analog.cfg file. If you're running Tomcat on it's own, then there probably isn't any access log by default. As far as I can recall, you have to tell Tomcat to create an Access log by adding the right bits to conf/server.xml. See http://tomcat.apache.org/tomcat-5.5-doc/config/host.html If you're running Tomcat as a servlet engine for Apache or some other web server, then use that servers logs. Aengus + | TO UNSUBSCRIBE from this list: |http://lists.meer.net/mailman/listinfo/analog-help | | Analog Documentation: http://analog.cx/docs/Readme.html | List archives: http://www.analog.cx/docs/mailing.html#listarchives | Usenet version: news://news.gmane.org/gmane.comp.web.analog.general +
Re: [analog-help] Log from Windows NT 4
On Wednesday, May 31, 2006 3:50 AM [EDT], Larsson, Lillemor <[EMAIL PROTECTED]> wrote: I have tried to get statistis from logfiles from MS Windows NT 4 server. From logfile: 09:58:29 159.7.77.77 GET /buller/ 302 0 Errormessage: F: Detect that it's in Microsoft's version of extended format C:\analog 6.0\analog.exe: Warning F: Ignoring corrupt format line in logfile C:\analog 6.0\analog.exe: ...cont..: (reason: time without date or vice versa) There are date in the filenamne: ex060529.log How could I defined LOGFORMAT, without date? There should be a utility called convlog on the IIS4 server that will convert the the logfiles into NCSA format, including the date on each line. If not, then Jeremys adddates.pl or my logfix.vbs will do the job. (http://analog.cx/helpers/logfix.vbs) Aengus + | TO UNSUBSCRIBE from this list: |http://lists.meer.net/mailman/listinfo/analog-help | | Analog Documentation: http://analog.cx/docs/Readme.html | List archives: http://www.analog.cx/docs/mailing.html#listarchives | Usenet version: news://news.gmane.org/gmane.comp.web.analog.general +
Re: [analog-help] Analog - generating new stats only
On Wednesday, May 31, 2006 12:58 PM [EDT],
Jeff Swearingen <[EMAIL PROTECTED]> wrote:
I am using Analog with IIS 6.0 log files. What is the best way to
have Analog generate stats for a website on a schedule, but not
process every log file? That is, how can I tell Analog to only
process log entries saved since the last time Analog was run?
The only way to reliably "tell Analog to only process log entries saved
since the last time Analog was run" is to get rid of the old records after
Analog is done with them ("get rid of" can mean moving them to another
location). If you only run Analog on a schedule, and you can easily
calculate when the last run was, then you can try to use that in your
script - fairly easy to do for Daily and Monthly runs, hard to do for weekly
runs.
Note that none of this has anything to do with Analog - any solution will be
part of the script that calls Analog, as Analog doesn't have any capacity
for doing this itself (except for a daily or monthly analysis, using date
codes in the LOGFILE command - http://analog.cx/docs/logfile.html).
Aengus
+
| TO UNSUBSCRIBE from this list:
|http://lists.meer.net/mailman/listinfo/analog-help
|
| Analog Documentation: http://analog.cx/docs/Readme.html
| List archives: http://www.analog.cx/docs/mailing.html#listarchives
| Usenet version: news://news.gmane.org/gmane.comp.web.analog.general
+
Re: [analog-help] Bytes Transferred on Windows Media Server
On Wednesday, May 31, 2006 3:57 PM [EDT],
Sokolow, Daniel <[EMAIL PROTECTED]> wrote:
All,
I've spent the better part of the day trying different
configurations, and getting all kinds of weird outputs, and I'm
still not getting what I want, so I'm asking the smart people.
I'm trying to get a sense of the amount of GB/MB transferred from the
server, and near as I can tell, the logfiles are capturing that
information. Here's the log fields:
#Fields: c-ip date time c-dns cs-uri-stem c-starttime x-duration
c-rate c-status c-playerid c-playerversion c-playerlanguage
cs(User-Agent) cs(Referer) c-hostexe c-hostexever c-os c-osversion
c-cpu filelength filesize avgbandwidth protocol transport audiocodec
videocodec channelURL sc-bytes c-bytes s-pkts-sent c-pkts-received
c-pkts-lost-client c-pkts-lost-net c-pkts-lost-cont-net c-resendreqs
c-pkts-recovered-ECC c-pkts-recovered-resent c-buffercount
c-totalbuffertime c-quality s-ip s-dns s-totalclients s-cpu-util
cs-user-name s-session-id s-content-path cs-url cs-media-name
c-max-bandwidth cs-media-role s-proxied
and a sample line from a log:
IP REDACTED 2006-05-31 00:04:17 - /video/lia.wmv 0 33 3 200
{3300AD50-2C39-46c0-AE0A-F5FC453811FC} 9.0.0.2980 en-US
Mozilla/4.0_(compatible;_MSIE_5.5;_Windows_NT_5.0)_(WMFSDK/9.0.0.2980)_WMPla
yer/9.0.0.3344(WMPDXM) URL REDACTED iexplore.exe 5.51.4807.2300
Windows_2000
5.0.0.2195 Pentium 240 26670830 3373336 rtsp TCP - - - 14147354
14147354 1765 1765 0 0 0 0 0 0 1 2 100 IP REDACTED 1 28 - 900
file://D:\Data\video\lia.wmv URL
REDACTED lia.wmv - - 0
Near as I can tell, the 14147354 is the result of the sc-bytes and
c-bytes fields.
But when I add "%b" to the LOGFORMAT line, I don't get any bytes
returned, and the rest of the output is screwed up. Here's the
LOGFORMAT line that works:
LOGFORMAT (%s %Y-%m-%d %h:%n:%j %S %r %j %j %j %j %j %j %j %B %f %j)
LOGFORMAT (#%j)
Does the %b need to go in a specific spot on the line to return that
information?
By my count, there are 13 fields between the referrer and sc-bytes, so you
need something like:
LOGFORMAT (%S %Y-%m-%d %h:%n:%j %j %r %j %j %j %c %j %j %j %B %f %j %j %j %j
%j %j %j %j %j %j %j %j %j %b %j)
I've tried it at the end, after the %B, in a few other
places, to no luck. Or am I mistaken, and the logfiles are not
actually capturing that data and I need to add that on to the
server's logs?
Add on? You might consider turning off some of the fields that you aren't
using, and don't actually care about.
One other question - is there a way to SORTBY numerically? I'd like
to sort the Domain report by IP address.
HOSTSORTBY ALPHABETICAL will sort numeric IPs numerically in the Host Report
(99 before 100, etc), but the Domain report will treat domain names,
including unresolved IP addresses, as strings, so you get a "pure"
alphabetical sort, with 100 before 99.
+
| TO UNSUBSCRIBE from this list:
|http://lists.meer.net/mailman/listinfo/analog-help
|
| Analog Documentation: http://analog.cx/docs/Readme.html
| List archives: http://www.analog.cx/docs/mailing.html#listarchives
| Usenet version: news://news.gmane.org/gmane.comp.web.analog.general
+
Re: [analog-help] 0 hits on search terms
On Thursday, June 01, 2006 3:29 AM [EDT], HQWEB <[EMAIL PROTECTED]> wrote: We use the search terms report a lot. Can you tell us how to produce statistics on 0 results search terms? ??? What do you mean by "0 results search terms"? If you're talking about an external search on google, etc, then there's obviousy no way to tell what search terms didn't lead to your website. If you're talking about an internal search engine, then the only way that information about how many results your search engine returned would be if your search engine generated that information, either in its own logs, or by redirecting the user to a URL that contained that information. Bear in mind that (external) Search reports rely on Referrer information, and, by definition, only count search results that the user clicked on. Your site might show up on the list of URLs that google produces, but if the user doesn't click on that link, they won't show up in your logs. The Internal Search Reports, on the other hand, report on the actual queries made against your search engine, and don't tell you anything about whether users actually used any of the links generated by by your search engine. If you want to tell how successful interal searches actually are, you really need the search engine to measure that for you, the iformation doesn't reall exist in the access logs that Analog analyzes. Aengus + | TO UNSUBSCRIBE from this list: |http://lists.meer.net/mailman/listinfo/analog-help | | Analog Documentation: http://analog.cx/docs/Readme.html | List archives: http://www.analog.cx/docs/mailing.html#listarchives | Usenet version: news://news.gmane.org/gmane.comp.web.analog.general +
Re: [analog-help] average requests per hour
On Tuesday, June 06, 2006 12:58 PM [EDT], D'Amore, Chris <[EMAIL PROTECTED]> wrote: Is there a report that is similar to the Hourly Summary but with averages per hour, not total (sum) per hour? No. (I'm not sure that it matters - the shape of the graph would be the same, though I suppose if you wanted to compare the numbers from a weekly report with a monthly or annual report, it would be useful to have averages). Aengus + | TO UNSUBSCRIBE from this list: |http://lists.meer.net/mailman/listinfo/analog-help | | Analog Documentation: http://analog.cx/docs/Readme.html | List archives: http://www.analog.cx/docs/mailing.html#listarchives | Usenet version: news://news.gmane.org/gmane.comp.web.analog.general +
Re: [analog-help] ANALOG BEGINNER
On Wednesday, June 07, 2006 5:06 PM [EDT], Mike Murphy <[EMAIL PROTECTED]> wrote: Hello, I am looking for a version for RedHat 3. Does it exist and if so where would I go? Try http://www.iddl.vt.edu/~jackie/analog/ Aengus + | TO UNSUBSCRIBE from this list: |http://lists.meer.net/mailman/listinfo/analog-help | | Analog Documentation: http://analog.cx/docs/Readme.html | List archives: http://www.analog.cx/docs/mailing.html#listarchives | Usenet version: news://news.gmane.org/gmane.comp.web.analog.general +
Re: [analog-help] domain report on more than 1 page/file
On Monday, June 12, 2006 8:15 AM [EDT], Ruiming Chen <[EMAIL PROTECTED]> wrote: Is it possible to report domain request for more than 1 page/file in one analog report (currently, if I define more than one FILEINCLUDE the config file, it only provide the domain report for the entire site, I have to use different config files for individual FILEINCLUDE files)? There will only be one Domain Report created in each run of Analog, so if you have multiple FILEINCLUDEs, the Domain Report will report on all the the domains accessing all of the included pages/files. The ony ay to get separate Domain Reports is to do multiple runs of Analog. http://analog.cx/docs/faq.html#faq128 Aengus + | TO UNSUBSCRIBE from this list: |http://lists.meer.net/mailman/listinfo/analog-help | | Analog Documentation: http://analog.cx/docs/Readme.html | List archives: http://www.analog.cx/docs/mailing.html#listarchives | Usenet version: news://news.gmane.org/gmane.comp.web.analog.general +
Re: [analog-help] Can't auto-detect format
On Monday, June 12, 2006 6:52 PM [EDT],
Pete Mason <[EMAIL PROTECTED]> wrote:
Hi,
I wonder if anyone can see what I'm doing wrong...
I moved host from Demon to 1and1 and for the first time got a "Can't
auto-detect format" in the analog error file, after using analog for
years.
It was odd because the log files are very similar to the apache
combined log files I had before. Typical output is
62.252.32.11 - - [04/Jun/2006:23:59:57 +0200] "GET
/Images/furniture/covers/thumb/51.48.jpg HTTP/1.1" 200 1878
www.socialistparty.org.uk <http://www.socialistparty.org.uk/>
"http://www.socialistparty.org.uk/cgi-bin/bookform7.pl?nextten=31
<http://www.socialistparty.org.uk/cgi-bin/bookform7.pl?nextten=31&ordernumbe
r=2211&cat=0&findword=0&sortby>
&ordernumber=2211&cat=0&findword=0&sortby=" "Mozilla/4.0
(compatible; MSIE 6.0; Windows NT 5.1; SV1; Tesco; .NET CLR
1.1.4322)" "86.29.142.254"
I downloaded analog 6.0, and tested it OK with my old log files.
1and1 told me the custom log:
CustomLog /var/log/httpd/access_log "%{Host}i %h %l %u %t \"%r\" %s
%b %v \"%{Referer}i\" \"%{User-agent}i\"
So in the config file, after trying it exactly as given without
success, I changed it to:
APACHELOGFORMAT (%s %h %l %u %t \"%r\" %s %b %v \"%f\" \"%u\")
LOGFILE C:\logs\splogs\2006\june06\*.*
The whole point of APACHELOGFORMAT is that you just cut and paste the
logformat string from your httpd.conf, so you don't have to "translate" into
Analog. You should use something like:
APACHELOGFORMAT (%{Host}i %h %l %u %t \"%r\" %s %b %v \"%{Referer}i\"
\"%{User-agent}i\")
But your sample string obviously doesn't start with "%{Host}i %h" (it only
has one address at the start, not two), and %{Host}i is the same as %v, so I
took %{Host}i out, and added a %j at the end for that "86.29.142.254" field,
which leaves
APACHELOGFORMAT (%h %l %u %t \"%r\" %s %b %v \"%{Referer}i\"
\"%{User-agent}i\" %j)
That parses the sample line that you provided.
Aengus
+
| TO UNSUBSCRIBE from this list:
|http://lists.meer.net/mailman/listinfo/analog-help
|
| Analog Documentation: http://analog.cx/docs/Readme.html
| List archives: http://www.analog.cx/docs/mailing.html#listarchives
| Usenet version: news://news.gmane.org/gmane.comp.web.analog.general
+
Re: [analog-help] Can hyperlinks in Request Report be altered?
On Friday, June 16, 2006 5:23 PM [EDT], John Bailo <[EMAIL PROTECTED]> wrote: In the Request Report, the file is displayed as a hyperlink. Can the hyperlink references (HREF) be ALIAS'd somehow, so they all point back to a single URL (rather than pointing to the actual file). So, each file would display and be counted individually; but if a user clicked one of them it would take them back to a single destination. Sort of. You could use BASEURL to point to a virtual server that had a 404 page that displayed whatever it is you wanted. So if your server is www.example.com, and a link in the request report would normally link to www.example.com/page1.html for instance, you could set BASEURL http://abcd.example.com The link in the Request Report (and the Failure Report and Redirection Report) will now link to http://abcd.example.com/page1.html. If you set the 404 "Page Not Found" error handler for the server, it should display when a user clicks on any link in your request report. (Though some browsers may not display a custom 404 page, so you may have to send a 200 status back from your 404 page!). http://analog.cx/docs/othreps.html#BASEURL Aengus + | TO UNSUBSCRIBE from this list: |http://lists.meer.net/mailman/listinfo/analog-help | | Analog Documentation: http://analog.cx/docs/Readme.html | List archives: http://www.analog.cx/docs/mailing.html#listarchives | Usenet version: news://news.gmane.org/gmane.comp.web.analog.general +
Re: [analog-help] OUTPUT LATEX and pie chart
On Tuesday, June 20, 2006 11:43 AM [EDT], alcogi <[EMAIL PROTECTED]> wrote: I want to use the Latex output and pdflatex to get a report as a pdf file for mailing. Everything works but the report doesn't include pie charts. The html output does. I also configured LOCALCHARTDIR and CHARTDIR. Is there something wrong with my configuration or doesn't the OUTPUT LATEX command support charts. It doesn't look like it - Analog doesn't seem to produce any charts unless you select OUTPUT XHTML or OUTPUT HTML. Aengus + | TO UNSUBSCRIBE from this list: |http://lists.meer.net/mailman/listinfo/analog-help | | Analog Documentation: http://analog.cx/docs/Readme.html | List archives: http://www.analog.cx/docs/mailing.html#listarchives | Usenet version: news://news.gmane.org/gmane.comp.web.analog.general +
Re: [analog-help] quick question about Analog
On Friday, June 23, 2006 1:47 AM [EDT], toshi <[EMAIL PROTECTED]> wrote: I have one quick question about Analog configuration. What I want to do is to add some columns to the monthly report and so on. Analog can count the total amount of requests and pages, and you can configure which files the request and page should contain. However can you configure Analog to count the request URL which contains a certain keyword? For instance, suppose I manage search engine and this search engine has simple search and detaield search. The request to the simple search is made towards a module called "simples_search.cgi?blahblahblah", and the detailed search is made twowards a module called "detailed_search.cgi?blahblahblah". Now I want to distinguish these two searches and count how many searches were executed on each of them. To do so with Analog, I need to configure Analog to count the request URL which contains a keyword "simple_search.cgi" and "detailed_search.cgi", and output the results in different columns at the monthly report. Can you do something like this with Analog? I could not find configuration how to for this kind of demand. The INTSEARCHENGINE, INTSEARCHQUERY, INTSEARCHWORD commands are designed to report on search URLS in great detail. http://analog.cx/docs/args.html#SEARCHENGINE If you want separate information on 2 report engines, you need to run Analog twice. Aengus + | TO UNSUBSCRIBE from this list: |http://lists.meer.net/mailman/listinfo/analog-help | | Analog Documentation: http://analog.cx/docs/Readme.html | List archives: http://www.analog.cx/docs/mailing.html#listarchives | Usenet version: news://news.gmane.org/gmane.comp.web.analog.general +
Re: [analog-help] Graphical barcharts
On Friday, June 23, 2006 6:45 PM [EDT], Russell Van Tassell <[EMAIL PROTECTED]> wrote: On Fri, Jun 23, 2006 at 02:33:13PM -0700, Mohler, Bruce wrote: Dear analog-help, [...] The problem that I'm getting is that no matter what I do, I get "+" signs in my barcharts rather than the graphical symbols that I was used when I ran an older 5.x version of analog. I'm using a slightly tweaked version of the bigbyrep.cfg example configuration file. Sounds like you need to tweak the IMAGEDIR to point at the BASE URL for your "images" directory that was packaged with Analog. http://www.analog.cx/docs/output.html#IMAGEDIR More secifically, by default the HTML source for Analogs bar charts looks like this: If your browser can't find images/barb16.png, it will display instead. Analog generates the HTML for both, but your browser will only display one. Aengus + | TO UNSUBSCRIBE from this list: |http://lists.meer.net/mailman/listinfo/analog-help | | Analog Documentation: http://analog.cx/docs/Readme.html | List archives: http://www.analog.cx/docs/mailing.html#listarchives | Usenet version: news://news.gmane.org/gmane.comp.web.analog.general +
Re: [analog-help] analog_60w32
On Saturday, June 24, 2006 5:08 PM [EDT], roger jamison <[EMAIL PROTECTED]> wrote: I use Windows XP so tried to download the aforementioned version. I unzipped the file, but it said it needed to be configured first - I'm just lost - what I thought would be a simple download has turned into a nightmare! Can someone help? Appreciated Which part of the documentation are you having a problem with? http://analog.cx/docs/faq.html#startfaqcont Aengus + | TO UNSUBSCRIBE from this list: |http://lists.meer.net/mailman/listinfo/analog-help | | Analog Documentation: http://analog.cx/docs/Readme.html | List archives: http://www.analog.cx/docs/mailing.html#listarchives | Usenet version: news://news.gmane.org/gmane.comp.web.analog.general +
Re: [analog-help] analog_60w32
On Saturday, June 24, 2006 6:06 PM [EDT], roger jamison <[EMAIL PROTECTED]> wrote: I'm puzzled - what happens now? You read the documentation. Follow the instructions for getting started on Windows (http://analog.cx/docs/startpc.html). It starts with this short summary: 1.. Edit analog.cfg 2.. Run analog (a DOS window flashes up). 3.. Read Report.html Usually, you only have to edit one line in analog.cfg to get meaningful results - LOGFILE logfilenameIf you have questions at that point, ask them here.Aengus + | TO UNSUBSCRIBE from this list: |http://lists.meer.net/mailman/listinfo/analog-help | | Analog Documentation: http://analog.cx/docs/Readme.html | List archives: http://www.analog.cx/docs/mailing.html#listarchives | Usenet version: news://news.gmane.org/gmane.comp.web.analog.general +
Re: [analog-help] Analog error message
On Saturday, June 24, 2006 9:52 PM [EDT],
Roland Leong <[EMAIL PROTECTED]> wrote:
I'm getting this error message when attempting to run
Analog:
Processing...
analog: analog version 6.0/Mac
analog: Warning F: Can't auto-detect format of logfile
access_060531.log.txt:
ignoring it
(For help on all errors and warnings, see
docs/errors.html)
Here are a few sample lines from my log file.
68.142.250.179 - - [30/Apr/2006:17:01:18 -0700] "GET
/Reload/ReloadMay02.html HTTP/1.0" 304 - "-"
"Mozilla/5.0 (compatible; Yahoo! Slurp;
http://help.yahoo.com/help/us/ysearch/slurp)"
"www.shotgunreport.com"
That's a very slight modification of the standard Combined logformat - you
appear to have a virtual host name at the end of each line
("www.shotgunreport.com")
Combined logformat is
LOGFORMAT (%S %j %u [%d/%M/%Y:%h:%n:%j] "%j%w%r%wHTTP%j" %c %b "%f" "%B")
so you'll want
LOGFORMAT (%S %j %u [%d/%M/%Y:%h:%n:%j] "%j%w%r%wHTTP%j" %c %b "%f" "%B"
"%v")
http://analog.cx/docs/logfmt.html#fmtexamples
There are some previous postings suggesting a change
in APACHELOGFORMAT. I'm not knowledgeable enough to
create the string necessary to get Analog to work.
APACHELOGFORMAT allows you to take the LogFormat command from Apaches
httpd.conf file, and paste it dorectly into analog.cfg, witout having to
manually translate it from the synta that Apache uses to the syntax that
Analog uses. It's not relevant unless you have access to the Apache
configuration files in the first place.
Aengus
+
| TO UNSUBSCRIBE from this list:
|http://lists.meer.net/mailman/listinfo/analog-help
|
| Analog Documentation: http://analog.cx/docs/Readme.html
| List archives: http://www.analog.cx/docs/mailing.html#listarchives
| Usenet version: news://news.gmane.org/gmane.comp.web.analog.general
+
Re: [analog-help] LOGFORMAT command help
On Thursday, June 29, 2006 12:13 PM [EDT], Jeremy Wadsack <[EMAIL PROTECTED]> wrote: I think the following will work, but you'll have to test it. The first statement needs to be all on one line (the email software has split it across three lines here). LOGFORMAT (Date: %m/%d/%y %h:%n:%j %aM\nMethod: %j\nServer: %j\nUser:%j\nClientIP: %S\nServerIP: %j\nDomain: %v\nContent-Type: %j\nContent-Length: %b\nPath: %f\nOperation: %j\n LOGFORMAT --%j These lines appear to work with the examples given, but frankly, if they were my logs, I would have preprocessed them to put everything on a singe line :-). LOGFORMAT (Date:\t%m/%d/%y %h:%n:%j %aM\n%j\n%j\nUser:\t%S\n%j\n%j\nDomain:\t%v\n%j\n%j\nPath:\t%r\n%j) LOGFORMAT (--%j) (I used \t for the tabs between columns, and junked the names of the fields that aren't required). Aengus See http://analog.cx/docs/logfmt.html for more details on the LOGFORMAT command. Hope that helps, Jeremy Wadsack Seven Simple Machines -Original Message- From: [EMAIL PROTECTED] [mailto:analog-help- [EMAIL PROTECTED] On Behalf Of hemiga Sent: Thursday, June 29, 2006 2:24 AM To: analog-help@lists.meer.net Subject: [analog-help] LOGFORMAT command help Hi all, I do not find out the correct syntax for the LOGFORMAT command to this log file. Problems are the several lines. From this log file I need the following information: DATE: ClientIP: Domain: Path: it would be great, if someone can help me Many Thanks hemiga Date: 6/26/06 12:21:00 AM Method: http Server: ge-muc-nt-pxy User: 172.29.146.12 ClientIP: 172.29.146.12 ServerIP: 255.255.255.255 Domain: www.jstv.co.uk Content-Type: none Content-Length: -1 Path: img/navi_home.gif Operation: GET -- Date: 6/26/06 8:16:42 AM Method: http Server: ge-muc-nt-pxy User: 172.29.16.119 ClientIP: 172.29.16.119 ServerIP: 255.255.255.255 Domain: www.chip.de Content-Type: image/gif Content-Length: 248 Path: Home/Loginbox/Unten_mitte_Kachel.gif Operation: GET -- -- Der GMX SmartSurfer hilft bis zu 70% Ihrer Onlinekosten zu sparen! Ideal für Modem und ISDN: http://www.gmx.net/de/go/smartsurfer + TO UNSUBSCRIBE from this list: http://lists.meer.net/mailman/listinfo/analog-help Analog Documentation: http://analog.cx/docs/Readme.html List archives: http://www.analog.cx/docs/mailing.html#listarchives Usenet version: news://news.gmane.org/gmane.comp.web.analog.general + + TO UNSUBSCRIBE from this list: http://lists.meer.net/mailman/listinfo/analog-help Analog Documentation: http://analog.cx/docs/Readme.html List archives: http://www.analog.cx/docs/mailing.html#listarchives Usenet version: news://news.gmane.org/gmane.comp.web.analog.general + + | TO UNSUBSCRIBE from this list: |http://lists.meer.net/mailman/listinfo/analog-help | | Analog Documentation: http://analog.cx/docs/Readme.html | List archives: http://www.analog.cx/docs/mailing.html#listarchives | Usenet version: news://news.gmane.org/gmane.comp.web.analog.general +
Re: [analog-help] Netscape Logs Won't Process
On Monday, July 03, 2006 10:04 AM [EDT], Cody Carey <[EMAIL PROTECTED]> wrote: /usr/bin/analog: Warning D: LOGFORMAT in configuration file stats.conf with no subsequent LOGFILE ... /usr/bin/analog: Warning D: LOGFORMAT in configuration file stats.conf with no subsequent LOGFILE http://analog.cx/docs/logfmt.html#DEFAULTLOGFORMAT " The times you need to use the DEFAULTLOGFORMAT instead of the LOGFORMAT are if you want to change the format of logfiles which aren't given in a LOGFILE command -- for example, ones specified on the command line, or dragged onto the program icon on a Mac, or compiled in." You're not specifying a logfile for Analog to match the LOGFORMAT command to - Analog can have multiple different LOGFORMATs applying to different logfiles in a single report, so it doesn't make any assumptions about what data a specific LOGFORMAT command is supposed to apply to. To tell Analog what LOGFORMAT to use for a logfile that is piped in, you must use DEFAULTLOGFORMAT. Aengus + | TO UNSUBSCRIBE from this list: |http://lists.meer.net/mailman/listinfo/analog-help | | Analog Documentation: http://analog.cx/docs/Readme.html | List archives: http://www.analog.cx/docs/mailing.html#listarchives | Usenet version: news://news.gmane.org/gmane.comp.web.analog.general +
Re: [analog-help] Grouping hierarchical log entries
On Wednesday, July 05, 2006 5:40 PM [EDT], Holger King <[EMAIL PROTECTED]> wrote: Hi group, I wonder if its possible to use only a part of logfile entries for analysis containing a defined string. Example: the log files containes the following requested resources like .../test/test1/test2/test3... .../servlet/ .../organization1/gb1/... .../organization2/gb2/... I want Analog only to consider all the entries containing ".../gb1/" and nothing else. Which Analog commands should I use for my configuration file? I'm sure you can help me! http://analog.cx/docs/include.html Aengus + | TO UNSUBSCRIBE from this list: |http://lists.meer.net/mailman/listinfo/analog-help | | Analog Documentation: http://analog.cx/docs/Readme.html | List archives: http://www.analog.cx/docs/mailing.html#listarchives | Usenet version: news://news.gmane.org/gmane.comp.web.analog.general +
Re: [analog-help] analog issue
On Friday, July 07, 2006 6:39 PM [EDT], Jeremy Wadsack <[EMAIL PROTECTED]> wrote: Isn't this in the FAQ? It's in the errors documentation: http://analog.cx/docs/errors.html#warnsM Logfile contains lines with no file names (or bytes): page (or byte) counts may be low If a logfile line doesn't contain a file name, analog will assume that the request wasn't for a page. Similarly, if it doesn't give the number of bytes transferred, analog will make the bytes zero. So the number of page requests or bytes credited to the other items on that line will then be too low. Aengus + | TO UNSUBSCRIBE from this list: |http://lists.meer.net/mailman/listinfo/analog-help | | Analog Documentation: http://analog.cx/docs/Readme.html | List archives: http://www.analog.cx/docs/mailing.html#listarchives | Usenet version: news://news.gmane.org/gmane.comp.web.analog.general +
Re: [analog-help] Catch some 302s without perl?
On Wednesday, July 12, 2006 8:59 AM [EDT], analog-help at talent-aid.org <[EMAIL PROTECTED]> wrote: Hi, I have an apache http access log with lines like this: 000.000.000.000 - - [10/Jul/2006:00:20:37 +0200] "GET /?=&locale=en_xx&DB=DB1&parm=bla HTTP/1.1" 302 - "http://referrer.com/page.html"; "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1)" 000.000.000.000 - - [10/Jul/2006:00:20:37 +0200] "GET /?=&locale=fr_xx&DB=DB1&parm=bla HTTP/1.1" 302 - "http://referrer.com/page.html"; "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1)" 000.000.000.000 - - [10/Jul/2006:00:20:37 +0200] "GET /?=&locale=en_xx&DB=DB2&parm=bla HTTP/1.1" 302 - "http://referrer.com/page.html"; "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1)" 000.000.000.000 - - [10/Jul/2006:00:20:37 +0200] "GET /?=&locale=fr_xx&DB=DB2&parm=bla HTTP/1.1" 302 - "http://referrer.com/page.html"; "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1)" I would like to match these and put them in an alias like FILEALIAS REGEXP:\/\?.*en_.*DB1.*302.* DB1_Redirected_EN FILEALIAS REGEXP:\/\?.*fr_.*DB1.*302.* DB2_Redirected_FR FILEALIAS REGEXP:\/\?.*en_.*DB2.*302.* DB1_Redirected_EN FILEALIAS REGEXP:\/\?.*fr_.*DB2.*302.* DB2_Redirected_FR Is that possible? The 302 isn't part of the Filename, so a FILEALIAS isn't going to see it unless you created a custom LOGFORMAT that included the status code. But that would complicate matters (because Analog wouldn't be able to tell that these were redirects any more, and you'd get double counting). Are you using the Redirection report? If the script in question always results in a redirect, then a FILEALIAS or a REQALIAS for the script should give you what you need in the Redirection report. Aengus + | TO UNSUBSCRIBE from this list: |http://lists.meer.net/mailman/listinfo/analog-help | | Analog Documentation: http://analog.cx/docs/Readme.html | List archives: http://www.analog.cx/docs/mailing.html#listarchives | Usenet version: news://news.gmane.org/gmane.comp.web.analog.general +
Re: [analog-help] analog doesn't run
On Wednesday, July 12, 2006 9:19 AM [EDT], Chris Martz <[EMAIL PROTECTED]> wrote: I have run the analog.exe file many times with no troubles but in the last few days, It won't exit after running. It just sits in the dos window saying: C:\Program Files\analog 6.0\analog.exe: analog version 6.0/Win32 C:\Program Files\analog 6.0\analog.exe: Warning E: Redirecting future diagnostic messages to C:\Program Files\analog 6.0\errors.txt It never closes and its failing my tasks to run every evening. Any ideas? What does it say in C:\Program Files\analog 6.0\errors.txt? Aengus + | TO UNSUBSCRIBE from this list: |http://lists.meer.net/mailman/listinfo/analog-help | | Analog Documentation: http://analog.cx/docs/Readme.html | List archives: http://www.analog.cx/docs/mailing.html#listarchives | Usenet version: news://news.gmane.org/gmane.comp.web.analog.general +
Re: [analog-help] Is 301 and 304 counted as a redirect?
On Wednesday, July 12, 2006 9:04 AM [EDT], analog-help at talent-aid.org <[EMAIL PROTECTED]> wrote: I have this in the cfg 304ISSUCCESS OFF #ignore the cached images I have a result of Redirected requests: which is equal to 301+302+304 Is that normal for analog? http://analog.cx/docs/include.html#304 "By default, analog counts code 304 as a successful request, because it assumes that the cached version of the document is then presented to the user. But you can count it as a redirected request with the command 304ISSUCCESS OFF" So yes, it's normal for Analog to treat a 304 as a redirect when you use 304ISSUCCESS OFF. Aengus + | TO UNSUBSCRIBE from this list: |http://lists.meer.net/mailman/listinfo/analog-help | | Analog Documentation: http://analog.cx/docs/Readme.html | List archives: http://www.analog.cx/docs/mailing.html#listarchives | Usenet version: news://news.gmane.org/gmane.comp.web.analog.general +
Re: [analog-help] analog doesn't run
On Wednesday, July 12, 2006 10:08 AM [EDT], Chris Martz <[EMAIL PROTECTED]> wrote: It never writes to the errors.txt file. It just keeps the dos window open and never ends or writes to the errors.txt file. Looking in my errors.txt, the first entry indicates that it reads the logfile first, and then it notes that it successfully opened the .lng file to create the report. So if your error.txt file is empty, it would indcate that there is a problem accessing your logfile - perhaps some sort of file lock problem. Does Analog work if you point at a different log file? Aengus + | TO UNSUBSCRIBE from this list: |http://lists.meer.net/mailman/listinfo/analog-help | | Analog Documentation: http://analog.cx/docs/Readme.html | List archives: http://www.analog.cx/docs/mailing.html#listarchives | Usenet version: news://news.gmane.org/gmane.comp.web.analog.general +
Re: [analog-help] Unique downloads: combination of Unique IP + file
On Wednesday, July 12, 2006 12:53 PM [EDT], Gal Ori Steinitz <[EMAIL PROTECTED]> wrote: Is it possible to get a "unique downloads" number that would be calculated by taking the unique combinations of IP + files ? I am seeing multiple requests for the same file by the same host, because of new podcast clients that download the files in small chunks. (Sometimes as many as 10,000 separate records for the same file). Needless to say this is skewing my reporting and now I can't get a real number anymore representing the number of files people have downloaded. By importing the raw log into a database and doing a select distinct on those two columns I was able to get a real number again. Can Analog do something similar ? No. This is a version of FAQ 128: http://analog.cx/docs/faq.html#faq128 The podcast clients that cause this register a status "206" instead of "200" in the log. I saw on the Analog mailing list archives a reference to a similar situation with a PDF file - however in that case each download was preceded by a single status "200", therefore it was possible to simply exclude the "206" and get a real number. In my case, these podcast clients only register status "206" for all requests, and never "200". Are you logging referrers? Is the referrer for the first part different from the other parts? If your web server can create a "sessionID" that can be logged, that might be a solution too. If you are using Apache, you might be able to create a custom logformat that includes an entry that concatenates the IP address and filename together. Aengus + | TO UNSUBSCRIBE from this list: |http://lists.meer.net/mailman/listinfo/analog-help | | Analog Documentation: http://analog.cx/docs/Readme.html | List archives: http://www.analog.cx/docs/mailing.html#listarchives | Usenet version: news://news.gmane.org/gmane.comp.web.analog.general +
Re: [analog-help] Unique downloads: combination of Unique IP + file
On Wednesday, July 12, 2006 1:49 PM [EDT], Gal Ori Steinitz <[EMAIL PROTECTED]> wrote: Thanks for the response. I'm not using Apache - I'm using IIS, but I can create custom handling for the log. I could create a hashed character string that would be unique for each IP + Filename combination and write it out to the log as a custom field. What option in Analog would I use to report based on that ? Create a custom LOGFORMAT, and use an otherwise unused field for your new string - for example %u if you aren't currently using the User Report. Unique users isn't included in the General Summary, but you can see the total number of unique users in the User Report. if you're not using the Host report, it might be a better choice, because the nmber of Distinct Hosts is listed in the General Summary. If you post one or two lines from your custom logfile, I'll suggest a LOGFORMAT that might work. Aengus + | TO UNSUBSCRIBE from this list: |http://lists.meer.net/mailman/listinfo/analog-help | | Analog Documentation: http://analog.cx/docs/Readme.html | List archives: http://www.analog.cx/docs/mailing.html#listarchives | Usenet version: news://news.gmane.org/gmane.comp.web.analog.general +
Re: [analog-help] Only include index.html
On Friday, July 14, 2006 8:44 AM [EDT], Graham Tinley <[EMAIL PROTECTED]> wrote: Hi folks, I'm trying to configure 6.0 to only report the home page (index.html) and at the same time NOT to report other folder home pages, here's what I've produced. PAGEINCLUDE index.html DIREXCLUDE /21st-century-challenge DIREXCLUDE /articles DIREXCLUDE /newsletter DIREXCLUDE /press-release However it is still reporting the whole of the stats for the whole site, obviously there is another command but I haven't found it FILEINCLUDE /index.html This will make Analog exclude everything except /index.html. DIREXCLUDE is an "output" exclude - it only excludes entries from the Directory report, it doesn't exclude those logfile entries from being included in other reports. Aengus + | TO UNSUBSCRIBE from this list: |http://lists.meer.net/mailman/listinfo/analog-help | | Analog Documentation: http://analog.cx/docs/Readme.html | List archives: http://www.analog.cx/docs/mailing.html#listarchives | Usenet version: news://news.gmane.org/gmane.comp.web.analog.general +
Re: [analog-help] Only include index.html
On Friday, July 14, 2006 9:35 AM [EDT], Graham Tinley <[EMAIL PROTECTED]> wrote: Thanks for that Aengus, Here's my new config: LOGFILE B:\TNE\Web\disc-calif\httpd\logs\access_log.200607* FILEINCLUDE index.html OUTFILE c:\analog\homeonly%Y.%M.html LOCALCHARTDIR "c:\analog\charts\%Y.%M." CHARTDIR "charts/%Y.%M." HOSTNAME "[DISC - HOME ONLY]" ERRFILE c:\analog\errors.txt And here's the result: This report contains overall statistics. Successful requests: 0 Corrupt logfile lines: 3 Unwanted logfile entries: 21,027 This analysis was produced by analog 6.0. Running time: 1 second. As you can see it reports nothing. Thanks for trying. FILEINCLUDE index.html is not the same as FILEINCLUDE /index.html Aengus + | TO UNSUBSCRIBE from this list: |http://lists.meer.net/mailman/listinfo/analog-help | | Analog Documentation: http://analog.cx/docs/Readme.html | List archives: http://www.analog.cx/docs/mailing.html#listarchives | Usenet version: news://news.gmane.org/gmane.comp.web.analog.general +
Re: [analog-help] Only include index.html
On Friday, July 14, 2006 10:37 AM [EDT], Graham Tinley <[EMAIL PROTECTED]> wrote: FILEINCLUDE index.html is not the same as FILEINCLUDE /index.html Aengus Thanks again Aengus, I modified the line, however I still get the same result: Program started at Fri-14-Jul-2006 15:36. General Summary(Go To: Top | General Summary) This report contains overall statistics. Successful requests: 0 Corrupt logfile lines: 3 Unwanted logfile entries: 21,027 This analysis was produced by analog 6.0. Running time: 1 second. (Go To: Top | General Summary) Here is my .cfg content. LOGFILE B:\TNE\Web\disc-calif\httpd\logs\access_log.200607* FILEINCLUDE /index.html OUTFILE c:\analog\homeonly%Y.%M.html LOCALCHARTDIR "c:\analog\charts\%Y.%M." CHARTDIR "charts/%Y.%M." HOSTNAME "[DISC - HOME ONLY]" ERRFILE c:\analog\errors.txt I'm sorry. I forgot about DIRSUFFIX. By default, Analog strips off the directory suffix from any filenames which have it, and Analogs default dirsuffix is index.html, so it treats /index.html as / (and /xyz/index.html as /xyz/, etc). Therefore you need to use FILEINCLUDE / to report on the root home page on its own. Aengus + | TO UNSUBSCRIBE from this list: |http://lists.meer.net/mailman/listinfo/analog-help | | Analog Documentation: http://analog.cx/docs/Readme.html | List archives: http://www.analog.cx/docs/mailing.html#listarchives | Usenet version: news://news.gmane.org/gmane.comp.web.analog.general +
Re: [analog-help] Retrieve title-data from aspx-files in Request Report
On Monday, July 17, 2006 8:20 AM [EDT], Jeroen Feelders <[EMAIL PROTECTED]> wrote: Our site uses Active Server Pages which show up in the Request Report such as: /cmspub.asp?psdn=Page&dbid=76 Although I configured the config-file so the links are click-through, it would be nice if a small explanation (title)could be added such as: /cmspub.asp?psdn=Page&dbid=76 (SomePage) with the first part being click-through. Is there a helper application/other way to add explanations based on the index-number of a page and its title? There is a tool available that will crawl a website, following the links on each page and recording the page titles, and using this information to create a file of REQALIAS commands for use with Analog. http://www.wadsack.com/analog.html#crawler But in your case, if your pages are being created from a database, it should be faster and more efficient to have the database generate a report listing the page names you want, in the format REQALIAS /cmspub.asp?psdn=Page&dbid=76 "/cmspub.asp Bid 76 (widget)" The clickable link will be unaffected - it will only be the displayed text in the Analog report that is changed. Aengus + | TO UNSUBSCRIBE from this list: |http://lists.meer.net/mailman/listinfo/analog-help | | Analog Documentation: http://analog.cx/docs/Readme.html | List archives: http://www.analog.cx/docs/mailing.html#listarchives | Usenet version: news://news.gmane.org/gmane.comp.web.analog.general +
Re: [analog-help] monthly stats
On Monday, July 17, 2006 2:05 PM [EDT], Marian Vicena <[EMAIL PROTECTED]> wrote: I think it does not work as it should. Here is example of daily report: 13/Jul/06 0 0 14/Jul/06 0 0 15/Jul/06 2345468 100559 ++ 16/Jul/06 106052 4689++ It seems to count only the latest log. It counts what you tell it to count. I defined CACHEFILE. Do I have to set also CACHEOUTFILE ? When i did so the log processing could not be completed the next day because the CACHEOUTFILE already existed. CACHEFILE tells Analog where to find the cachefile that you created previously with CACHEOUTFILE. Analog will not overwrite an exsting cachefile. Whan am I doing wrong? The simplest and most effective way to use cachefiles is to create a seperate cachefile for each logfile. Then, when you want to create a weekly or monthly report, run Analog against just the cachefiles. Alternatively, you can also run Analog against yesterdays cachefile and yesterdays logfile to create a new cachefile that you will use with todays logfile, etc. But if you're not careful, you can end up counting the same traffic multiple times. Unless you're dealing with several gigabytes of logs per day, simply zipping your logs may be a more effective way to solve whatever problems you were having in the first place. Aengus + | TO UNSUBSCRIBE from this list: |http://lists.meer.net/mailman/listinfo/analog-help | | Analog Documentation: http://analog.cx/docs/Readme.html | List archives: http://www.analog.cx/docs/mailing.html#listarchives | Usenet version: news://news.gmane.org/gmane.comp.web.analog.general +
Re: [analog-help] GoogleAdwordsSelect
On Tuesday, July 18, 2006 5:17 AM [EDT], Graham Tinley <[EMAIL PROTECTED]> wrote: Sorry folks - an update... I can view fine for the site as a whole, but... when viewing the home page only, the figures for referrer do not show at all. Here's the page for the site: reqs: URL : --- 1833: http://www.disc-calif.com/ 335: http://www.disc-calif.com/?referer=capterra-ad 135: http://www.disc-calif.com/?referer=GoogleAdwordsSelect And for the home page only: reqs: URL : --- 195: [not listed: 71 URLs] Any help will be gratefully received... It would be helpful if you included the relevant lines from your analog.cfg file, showing what commands you changed. But the "[not listed:...]" entries are because the FLOOR or SUBFLOOR is set too low. http://analog.cx/docs/othreps.html#FLOOR http://analog.cx/docs/hierreps.html#SUBFLOOR http://analog.cx/docs/hierreps.html#ARGSFLOOR Aengus + | TO UNSUBSCRIBE from this list: |http://lists.meer.net/mailman/listinfo/analog-help | | Analog Documentation: http://analog.cx/docs/Readme.html | List archives: http://www.analog.cx/docs/mailing.html#listarchives | Usenet version: news://news.gmane.org/gmane.comp.web.analog.general +
Re: Re[2]: [analog-help] monthly stats
On Tuesday, July 18, 2006 3:07 AM [EDT], Marian Vicena <[EMAIL PROTECTED]> wrote: The simplest and most effective way to use cachefiles is to create a seperate cachefile for each logfile. Then, when you want to create a weekly or monthly report, run Analog against just the cachefiles. So everyday I create new cache, e.g. cache1.db, cache2.db, etc. until cache31.db and then when running the monthly analog process I define in the appropriate config all those 31 cache??.db files with CACHEFILE directive? Is this what you are advising me? I'd advise that you don't use cache files at all - they can be confusing, and severely limit the flexibility of Analog (you can't reliably modify your reports if they are based on cache files). Therefore you shouldn't use cache files until you have been using Analog long enough to be entirely satisied with your Analog reports, and that you won't be changing them. The simplest error that you can make with cache files is to include the same information twice in the cache file when making a cumulative cache file. The simplest way to avoid this is to create a cache file for each logfile. http://analog.cx/docs/cache.html#cacheproc Alternatively, you can also run Analog against yesterdays cachefile and yesterdays logfile to create a new cachefile that you will use with todays logfile, etc. But if you're not careful, you can end up counting the same traffic multiple times. Can you show me example on how to do this? Not really - I've never used cache files, so if the description above doesn't make sense, I'd rather not make it any more detailed. Deleting logfiles involves throwing data away, and I don't want to offer advice that might cause problems further down the road. Unless you're dealing with several gigabytes of logs per day, simply zipping your logs may be a more effective way to solve whatever problems you were having in the first place. This will be done only in case I cannot figure out how to do it with analog without the need to keep all the logs. You haven't said why you want to get rid of the logs. How big are they? How long are your reports taking to run? Aengus + | TO UNSUBSCRIBE from this list: |http://lists.meer.net/mailman/listinfo/analog-help | | Analog Documentation: http://analog.cx/docs/Readme.html | List archives: http://www.analog.cx/docs/mailing.html#listarchives | Usenet version: news://news.gmane.org/gmane.comp.web.analog.general +
Re: [analog-help] (newbie) A few question to the final report
On Tuesday, July 18, 2006 11:13 AM [EDT], El Gringo <[EMAIL PROTECTED]> wrote: I am not sure if we mean the same thing. What I mean is, eg. when there is a report line with this entry: /index.html 7783 Does this mean the page was 7783 times requested or requesting the /index.html has caused 7783 hits, including images, scripts etc. Sorry if my question sounds stupid but as I said, I am very new to this theme. It means that there are 7783 lines in your logfile that have "GET /index.html" with a status code that indicates a successful request. There were 7783 requests for /index.html. - Are these counts included in the Request Report? No. So if they are requests, why are they not counted? They're not in the request report because they aren't successful requests. If I ask for page1.html, and get redirected to pageone.html, then as a user, I only "see" one request being fulfilled. So redirected requests and successful requests and failed requests are all counted, but they each get their own report. Aengus + | TO UNSUBSCRIBE from this list: |http://lists.meer.net/mailman/listinfo/analog-help | | Analog Documentation: http://analog.cx/docs/Readme.html | List archives: http://www.analog.cx/docs/mailing.html#listarchives | Usenet version: news://news.gmane.org/gmane.comp.web.analog.general +
Re: [analog-help] GoogleAdwordsSelect
On Tuesday, July 18, 2006 8:57 AM [EDT], Graham Tinley <[EMAIL PROTECTED]> wrote: Hi Aengus, I'm using the same analog.cfg for both the web site and home page results, the only difference is the command in the home page cfg: # only include home page requests FILEINCLUDE http://www.disc-calif.com/ FILEINCLUDE http://www.disc-calif.com/?referer=capterra-ad FILEINCLUDE http://www.disc-calif.com/?referer=GoogleAdwordsSelect FILEINCLUDE http://www.disc-calif.com/?referer=google-adwords FILEINCLUDE http://www.disc-calif.com/?referer=googleadwordsselect As Jeremy pointed out, you don't FILEINCLUDE referrers. You FILEINCLUDE the actual files on your server that you want to report on, so FILEINCLUDE / should report on your home page (assuming you've set DIRSUFFIX, as discussed last week). Aengus + | TO UNSUBSCRIBE from this list: |http://lists.meer.net/mailman/listinfo/analog-help | | Analog Documentation: http://analog.cx/docs/Readme.html | List archives: http://www.analog.cx/docs/mailing.html#listarchives | Usenet version: news://news.gmane.org/gmane.comp.web.analog.general +
Re: [analog-help] Log format
On Tuesday, July 18, 2006 5:19 PM [EDT], John G Doke <[EMAIL PROTECTED]> wrote: Analog is saying that it identifies my log files as: NCSA Combined. I was wondering if it would speed up processing if I told it what format the files were with Logformat? So I tried but cannot find the correct way to state the logformat command. Do I need to do this or just leave it alone. Analog is fast - very fast. As Jeremy pointed out, you won't speed Analog up by bypassing the logformat analysis, which only happens on the first few lines of the logfile - Analog doesn't anayse every single line of the log to figure out the format. The formats that Analog recognizes are described in the documentation: http://analog.cx/docs/logfmt.html#fmtexamples Aengus + | TO UNSUBSCRIBE from this list: |http://lists.meer.net/mailman/listinfo/analog-help | | Analog Documentation: http://analog.cx/docs/Readme.html | List archives: http://www.analog.cx/docs/mailing.html#listarchives | Usenet version: news://news.gmane.org/gmane.comp.web.analog.general +
Re: [analog-help] combination of request and daily / hourly report?
On Tuesday, July 18, 2006 10:12 PM [EDT], Joe Park <[EMAIL PROTECTED]> wrote: Hello, Is there a way to generate a report that's a combination of request report and daily (or hourly) report? Basically, I'm trying to generate a request report that's divided in daily (or hourly) time frame -- or inversely, daily (or hourly) report where it's request number are divided into request for different files. From what I read on user manual, this is not possible with analog? This is one of the FAQs: http://analog.cx/docs/faq.html#faq128 Can I find out which files each referrer pointed to? or Can I find out which files each host has read? or Can I find out which hosts have read each file? or Can I find out the number of hosts visiting on each day? or lots of similar questions. There are lots of questions like this. They all want analog to cross-reference two sorts of item (e.g. files and referrers in the first example above, or hosts and dates in the last). The solution is to use the *INCLUDE commands. For example, to find out which files a particular referrer pointed to, restrict the analysis to just the referrer you're interested in with the REFINCLUDE command. The Request Report will then just list the files reached from that referrer. Similarly, if you're interested in a particular time period, you can focus on just that time by using the FROM and TO commands. You do have to run analog once for each report generated. So for the example of referrers, you would have to run it once for each referrer you're interested in. Of course, it would be useful to be able to cross-refererence all the files and referrers at once. But it is fundamental to analog's speed and minimal memory requirement that it only records statistics for each type of item individually, and doesn't record enough information to cross-reference them afterwards. In summary, you can run Analog multiple times to create hourly reports, and concatenate them, but Analog won't do it automatically. Aengus + | TO UNSUBSCRIBE from this list: |http://lists.meer.net/mailman/listinfo/analog-help | | Analog Documentation: http://analog.cx/docs/Readme.html | List archives: http://www.analog.cx/docs/mailing.html#listarchives | Usenet version: news://news.gmane.org/gmane.comp.web.analog.general +
Re: [analog-help] Trying to find 5.1 readme
On Wednesday, July 19, 2006 3:59 AM [EDT], Wendy Tinley <[EMAIL PROTECTED]> wrote: Followed the link to 'any previous versions' from http://www.analog.cx/ http://www.chiark.greenend.org.uk/~sret1/analog/olddocs/ Clicked on 5.1 Document retrieval failed - --- Sorry, I can't find the document with name = Readme.html version = 5.1 Please try again. Does anyone know of a link where I can find the readme please. http://web.archive.org/web/20021102163217/analog.sourceforge.net/cgi-bin/olddocs.pl?version=5.1&file=Readme.html Aengus + | TO UNSUBSCRIBE from this list: |http://lists.meer.net/mailman/listinfo/analog-help | | Analog Documentation: http://analog.cx/docs/Readme.html | List archives: http://www.analog.cx/docs/mailing.html#listarchives | Usenet version: news://news.gmane.org/gmane.comp.web.analog.general +
Re: [analog-help] Analog not working with webpage
On Wednesday, July 19, 2006 5:12 PM [EDT], R C <[EMAIL PROTECTED]> wrote: The problem I'm having is that when I run the analog script from my webpage with the search options, it either takes so long that it displays "Page not found" with analog.pl being the page its on, or it has a blank page with the page still being the original stats page that I submitted. Any ideas as to what's wrong with my instance of analog? Unless you have very large logfiles, the only thing that normally causes Analog to run slowly is DNS lookups. But DNS lookups are effectively disabled in the form interface, so it's unlikely that that's the problem. You don't mention what web server you are running, but there were a number of reports of difficulties running the form interface on IIS6 last year, and there was ony one report of someone successfully tweaking the .pl script to get consistent output - http://lists.meer.net/pipermail/analog-help/2005-July/018575.html. I don't recall anyone following up on that report. Aengus + | TO UNSUBSCRIBE from this list: |http://lists.meer.net/mailman/listinfo/analog-help | | Analog Documentation: http://analog.cx/docs/Readme.html | List archives: http://www.analog.cx/docs/mailing.html#listarchives | Usenet version: news://news.gmane.org/gmane.comp.web.analog.general +
Re: [analog-help] How many users are logged on at a given time and howlong are they logged on
On Wednesday, July 19, 2006 1:59 PM [EDT], D'Amore, Chris <[EMAIL PROTECTED]> wrote: Is there any way to show how many users are logged in at a given time No. The Time Reports only report on the number of Requests, Page Requests and Bytes transferred for each time period: http://analog.cx/docs/timereps.html This is also a varition of the (infamous) FAQ 128: http://analog.cx/docs/faq.html#faq128 You can generate a User Report or a Host Report (with all the usual caveats) for a specific time period, but you'd have to run 24 individual reports to find out how many "users" were online for each hour in a day, for example. and possibly how long they are logged on for? That depends. If you set a "session cookie", and you use that as the %u field in your LOGFORMAT, you can specify the "Date and time of first access" and "Date and time of last access" columns in the User Report with USERCOLS ED You'd have to use a script to parse these times to get a "session length" if you felt that that information was valuable - it's not something that is built into Analog. Aengus + | TO UNSUBSCRIBE from this list: |http://lists.meer.net/mailman/listinfo/analog-help | | Analog Documentation: http://analog.cx/docs/Readme.html | List archives: http://www.analog.cx/docs/mailing.html#listarchives | Usenet version: news://news.gmane.org/gmane.comp.web.analog.general +
Re: [analog-help] Analog not working with webpage
On Wednesday, July 19, 2006 9:45 PM [EDT], R C <[EMAIL PROTECTED]> wrote: Thanks for your response Aengus. I asked one of my coworkers and he says we are still using IIS 5.0, and I am using Analog 5.21. What is considered a "big" logfile? I can process a gigabyte logfile in under a minute on my desktop. So gigabyte sized logfiles could time out, especially if the server is already moderately busy doing other work. But logfiles upto hundreds of megabytes shouldn't time out on moderate hardware. Anlgform.pl has worked fine for people on IIS5. The obvious questions for troubleshooting anlgform are - do other perl scripts work okay? Does it work with a small logfile? Can you tell from CPU utilization if Analog is even running (Analog will try to use all avaiabe CPU, so if it runs, you should be able to tell by monitoring CPU utilisation). And what do the server logs say (the event og and the IIS web log - what status code does the request return?). Aengus From: "Aengus" <[EMAIL PROTECTED]> Reply-To: Support for analog web log analyzer To: "Support for analog web log analyzer" Subject: Re: [analog-help] Analog not working with webpage Date: Wed, 19 Jul 2006 21:07:41 -0400 On Wednesday, July 19, 2006 5:12 PM [EDT], R C <[EMAIL PROTECTED]> wrote: The problem I'm having is that when I run the analog script from my webpage with the search options, it either takes so long that it displays "Page not found" with analog.pl being the page its on, or it has a blank page with the page still being the original stats page that I submitted. Any ideas as to what's wrong with my instance of analog? Unless you have very large logfiles, the only thing that normally causes Analog to run slowly is DNS lookups. But DNS lookups are effectively disabled in the form interface, so it's unlikely that that's the problem. You don't mention what web server you are running, but there were a number of reports of difficulties running the form interface on IIS6 last year, and there was ony one report of someone successfully tweaking the .pl script to get consistent output - http://lists.meer.net/pipermail/analog-help/2005-July/018575.html. I don't recall anyone following up on that report. Aengus + TO UNSUBSCRIBE from this list: http://lists.meer.net/mailman/listinfo/analog-help Analog Documentation: http://analog.cx/docs/Readme.html List archives: http://www.analog.cx/docs/mailing.html#listarchives Usenet version: news://news.gmane.org/gmane.comp.web.analog.general + + TO UNSUBSCRIBE from this list: http://lists.meer.net/mailman/listinfo/analog-help Analog Documentation: http://analog.cx/docs/Readme.html List archives: http://www.analog.cx/docs/mailing.html#listarchives Usenet version: news://news.gmane.org/gmane.comp.web.analog.general + + | TO UNSUBSCRIBE from this list: |http://lists.meer.net/mailman/listinfo/analog-help | | Analog Documentation: http://analog.cx/docs/Readme.html | List archives: http://www.analog.cx/docs/mailing.html#listarchives | Usenet version: news://news.gmane.org/gmane.comp.web.analog.general +
Re: [analog-help] OS unknown
On Friday, July 21, 2006 7:50 AM [EDT], [EMAIL PROTECTED] <[EMAIL PROTECTED]> wrote: Our group have been using Analog since Apr. 2002 to see how often people access our website. In our reports, the average monthly #reqs under "OS unknown" in "Operating System Report" is as following: Year #reqs 20021600 20032400 20044500 20058300 2006 19000 You can see that the number is increasing each year. It seems not reasonable to count those #reqs as people "actually" using our websites. How can I determine if these OS unknown includes robots or search engines? You can find a fairly comprehensive list of known Robots, nicely confgured as a ROBOTINCLUDE list at: http://www.wadsack.com/robot-list.html How can I decrease the OS unknown numbers? Browser strings usually indicate the OS that the browser is running on. If you do a full Browser Report (BROWSERREP ON) you should be able to pick out browsers that don't specify an OS. You can make this easier by specifying commands like BROWREPEXCLUDE *Windows* etc to exclude the known ones. Once you get a list of Browser Strings that aren't recognized as Robots (with ROBOTINCUDE commands) and that don't include OS strings, you can use BROWALIAS commands to assign an Operating System. Here's an example that changes a Browser from "OS Unknown" to "Windows XP": BROWALIAS Microsoft-WebDAV-MiniRedir/5.1.2600 "Microsoft-WebDAV-MiniRedir/5.1.2600 (Windows NT 5.1)" Aengus + | TO UNSUBSCRIBE from this list: |http://lists.meer.net/mailman/listinfo/analog-help | | Analog Documentation: http://analog.cx/docs/Readme.html | List archives: http://www.analog.cx/docs/mailing.html#listarchives | Usenet version: news://news.gmane.org/gmane.comp.web.analog.general +
Re: [analog-help] OS unknown
On Monday, July 24, 2006 7:48 AM [EDT], Grace Sheu <[EMAIL PROTECTED]> wrote: I trided the first step that you suggested. I uses ROBOTINCLUDE list to include more robots and re-ran one of my log. I thought I should see more "Known robots" numbers and less "OS unknown" number, but suprisedly in opposite; please see the following datas for details. Can you explain this? Not off hand. Unless you had a ROBOTINCLUDE for a local search engine that you didn't include with the new list. It does seem odd that the your original Robot count included so many requests rather than pages - robots don't usually request images, so the pages count for robots is usually much higher than the request count. In the full Browser Report (rather than the Browser Summary) is there one browser that made just 2855 requests? To understand more about what is going on, you may need to use the BROWEXCLUDE command, rather than the BROWREPEXCLUDE command. This will allow you to totally exclude the requests from known browsers from all the reports, so that you can use the Request Report to see what files are being requested by the "OS Unknown" browsers, and you can use the Host report to see where the requests are coming from. Aengus + | TO UNSUBSCRIBE from this list: |http://lists.meer.net/mailman/listinfo/analog-help | | Analog Documentation: http://analog.cx/docs/Readme.html | List archives: http://www.analog.cx/docs/mailing.html#listarchives | Usenet version: news://news.gmane.org/gmane.comp.web.analog.general +
Re: [analog-help] OS unknown
On Monday, July 24, 2006 8:36 AM [EDT], Grace Sheu <[EMAIL PROTECTED]> wrote: Please go to http://www.tunl.duke.edu/nucldata/Website_Report/June2006.html to see the full Browser Report. It's pretty obvious that Mozilla/5.0 (compatible; Yahoo! Slurp; http://help.yahoo.com/help/us/ysearch/slurp) is not being counted as a Robot. You also need to add PAGEINCLUDE *.pdf PAGEINCLUDE *.shtml so that your Page counts make more sense. To get a list of the browsers that are falling into the "OS Unknown" category, search and replace ROBOTINCLUDE to BROWREPEXCLUDE, and add BROWREPEXCLUDE *Windows* BROWREPEXCLUDE *Macintosh* BROWREPEXCLUDE *Linux* BROWREPEXCLUDE *SunOS* BROWREPEXCLUDE *BSD* The browsers that are left are the ones that make up OS Uknown. Aengus Thanks. Grace On Mon, 24 Jul 2006, Aengus wrote: On Monday, July 24, 2006 7:48 AM [EDT], Grace Sheu <[EMAIL PROTECTED]> wrote: I trided the first step that you suggested. I uses ROBOTINCLUDE list to include more robots and re-ran one of my log. I thought I should see more "Known robots" numbers and less "OS unknown" number, but suprisedly in opposite; please see the following datas for details. Can you explain this? Not off hand. Unless you had a ROBOTINCLUDE for a local search engine that you didn't include with the new list. It does seem odd that the your original Robot count included so many requests rather than pages - robots don't usually request images, so the pages count for robots is usually much higher than the request count. In the full Browser Report (rather than the Browser Summary) is there one browser that made just 2855 requests? To understand more about what is going on, you may need to use the BROWEXCLUDE command, rather than the BROWREPEXCLUDE command. This will allow you to totally exclude the requests from known browsers from all the reports, so that you can use the Request Report to see what files are being requested by the "OS Unknown" browsers, and you can use the Host report to see where the requests are coming from. Aengus + TO UNSUBSCRIBE from this list: http://lists.meer.net/mailman/listinfo/analog-help Analog Documentation: http://analog.cx/docs/Readme.html List archives: http://www.analog.cx/docs/mailing.html#listarchives Usenet version: news://news.gmane.org/gmane.comp.web.analog.general + | TO UNSUBSCRIBE from this list: |http://lists.meer.net/mailman/listinfo/analog-help | | Analog Documentation: http://analog.cx/docs/Readme.html | List archives: http://www.analog.cx/docs/mailing.html#listarchives | Usenet version: news://news.gmane.org/gmane.comp.web.analog.general +
Re: [analog-help] OS unknown
Unsubscribe instructions are incuded at the bottom of every singe message posted to this list. + TO UNSUBSCRIBE from this list: http://lists.meer.net/mailman/listinfo/analog-help Analog Documentation: http://analog.cx/docs/Readme.html List archives: http://www.analog.cx/docs/mailing.html#listarchives Usenet version: news://news.gmane.org/gmane.comp.web.analog.general + + | TO UNSUBSCRIBE from this list: |http://lists.meer.net/mailman/listinfo/analog-help | | Analog Documentation: http://analog.cx/docs/Readme.html | List archives: http://www.analog.cx/docs/mailing.html#listarchives | Usenet version: news://news.gmane.org/gmane.comp.web.analog.general +
Re: [analog-help] Re: Which pages did the site refer to?
On Monday, July 24, 2006 12:14 PM [EDT], Sarah Lander <[EMAIL PROTECTED]> wrote: Something very odd is happening with that suggested solution. If I use REFSITEINCLUDE I am getting that there are 139 requests for 139 pages in the referring site report for the domain I specify, so I know there's information there. The request report seems to still list for all referrers in this case though. However, if I modify the word REFSITEINCLUDE to REFINCLUDE as per the suggested solution then I don't get any output at all - in the errors.txt file I get the messages about reports turned off because they are empty. REFSITEINCLUDE includes all logfile entries that have a reference from the specified domain: REFSITEINCLUDE http://my.host.com/ (As the documentation points out, this is not the same as REFSITEINCLUDE http://my.host.com/*) REFINCLUDE will ony include logfile entries that have references that match the specified reference. Therefore REFSITEINCLUDE http://my.host.com/ will only list references that come from the root document on my.host.com. A log entry that was referred from http://my.host.com/page1.html would not be included. So REFSITEINCLUDE http://my.host.com/ will tell you 1) how many visits to your site were referred from my.host.com 2) What were the referring pages on my.host.com (in the Referrer Report) 3) What pages on your site were linked from my.host.com (in the Request Report). Aengus + | TO UNSUBSCRIBE from this list: |http://lists.meer.net/mailman/listinfo/analog-help | | Analog Documentation: http://analog.cx/docs/Readme.html | List archives: http://www.analog.cx/docs/mailing.html#listarchives | Usenet version: news://news.gmane.org/gmane.comp.web.analog.general +
Re: [analog-help] Excluding Robots
On Wednesday, July 26, 2006 5:06 AM [EDT], Martin Ibert <[EMAIL PROTECTED]> wrote: Hi everyone, probably I have overlooked something in the docs, but I just cannot find how to easily exclude all robot-generated traffic from my reports. I know about ROBOTINCLUDE and ROBOTEXCLUDE, but as far as I can see that just affects the operating system report (which shows a large portion of my traffic as "Known robots"). Is there really no better way than to change all ROBOTINCLUDE to BROWEXCLUDE and vice versa? That's what you need to do. If it's something you need to do often, the simple thing to do is to keep your lists of robots in two seperate configfiles, and include the necessary one with a CONFIGFILE command. http://analog.cx/docs/syntax.html#CONFIGFILE Aengus + | TO UNSUBSCRIBE from this list: |http://lists.meer.net/mailman/listinfo/analog-help | | Analog Documentation: http://analog.cx/docs/Readme.html | List archives: http://www.analog.cx/docs/mailing.html#listarchives | Usenet version: news://news.gmane.org/gmane.comp.web.analog.general +
Re: [analog-help] Too many arguments error
On Saturday, July 29, 2006 9:43 AM [EDT], Sean at IMAGINET at home <[EMAIL PROTECTED]> wrote: On Friday 28 July 2006 22:13, Eugene Yang wrote: When I run analog in dos prompt, I get this: C:\analog_60w32\analog 6.0>analog analog: analog version 6.0/Win32 analog: Warning C: Too many arguments for configuration command: ignoring end of line starting: LOGFILE D:\Program Files\Apache ... In my analog conf file: LOGFILE D:\Program Files\Apache Group\Apache2\logs\access_log.* ... However, if I were to move some of the logfiles to anothet location other than Apache Group, the report works fine. Does Analog have a llimitation on how many subdirectories are allowed for the logfiles location? What am I doing wrong? Have you tried putting the whole filepath in quotes (as DOS is likely choking on the spaces within the foldernames)? You're right that the problem is the space in the logfile name, but it's not a DOS problem - D:\Program is not a log file, whether you run Analog on DOS, Unix or a Mac. Analog uses spaces to delimit it's arguments, so if there's a space in the argument, you have to quote the argument. http://analog.cx/docs/syntax.html#commandsyntax "Each command consists of the command name followed by one or two arguments. An argument to a command may optionally be placed in single or double quotes or parentheses, and it must be if the argument contains a hash or a space" To answer Eugenes question, Analog doesn't have any limit on how many directories deep the logfile may be, but there is a limit to the length of an entry in a configfile. But you should run up against that limit (254 characters) under normal circumstances. Aengus + | TO UNSUBSCRIBE from this list: |http://lists.meer.net/mailman/listinfo/analog-help | | Analog Documentation: http://analog.cx/docs/Readme.html | List archives: http://www.analog.cx/docs/mailing.html#listarchives | Usenet version: news://news.gmane.org/gmane.comp.web.analog.general +
Re: [analog-help] Directory report question
On Monday, July 31, 2006 4:52 AM [EDT], Glover, John <[EMAIL PROTECTED]> wrote: Hi, I've been trying to get the directory report to show all sub directories. In debugging I made a sample log file as follows: 127.0.0.1 - - [26/Jul/2006:16:23:08 +0100] "GET /intranet/home HTTP/1.1" 200 25277 127.0.0.1 - - [26/Jul/2006:16:23:13 +0100] "GET /intranet/home/employee-portal HTTP/1.1" 200 35882 127.0.0.1 - - [26/Jul/2006:16:23:15 +0100] "GET /intranet/home/folder_one HTTP/1.1" 200 32053 127.0.0.1 - - [26/Jul/2006:16:23:16 +0100] "GET /intranet/home/departments/mis HTTP/1.1" 200 45119 127.0.0.1 - - [26/Jul/2006:16:23:18 +0100] "GET /intranet/home HTTP/1.1" 200 25277 127.0.0.1 - - [26/Jul/2006:16:25:47 +0100] "GET /intranet/home HTTP/1.1" 200 25277 127.0.0.1 - - [31/Jul/2006:08:21:21 +0100] "GET /intranet/home HTTP/1.1" 200 25277 127.0.0.1 - - [31/Jul/2006:08:22:17 +0100] "GET /foo-bar/test HTTP/1.1" 200 33917 127.0.0.1 - - [31/Jul/2006:08:22:17 +0100] "GET /foo-bar/test/test2 HTTP/1.1" 200 33917 127.0.0.1 - - [31/Jul/2006:08:22:17 +0100] "GET /foo-bar/test/test2 HTTP/1.1" 200 33917 127.0.0.1 - - [31/Jul/2006:08:22:17 +0100] "GET /intranet/home/employee-portal HTTP/1.1" 200 33917 What I see in directory report is: 8 70.91% /intranet/ 4 42.01% /intranet/home/ 1 12.90% /intranet/home/departments/ 3 29.09% /foo-bar/ 2 19.39% /foo-bar/test/ Clearly there are directories not listed for some reason under /intranet and /foo-bar All looks fine to me. Any ideas? /foo-bar/test/test2 is a file called test2 in /foo-bar/test/. Aengus + | TO UNSUBSCRIBE from this list: |http://lists.meer.net/mailman/listinfo/analog-help | | Analog Documentation: http://analog.cx/docs/Readme.html | List archives: http://www.analog.cx/docs/mailing.html#listarchives | Usenet version: news://news.gmane.org/gmane.comp.web.analog.general +
Re: [analog-help] Analog 5.32 will not run
On Tuesday, August 01, 2006 5:21 AM [EDT], Christine Neale <[EMAIL PROTECTED]> wrote: Hi, I have been using Analog 5.32 for years without any problem but had to reinstall windows after hard disk failure and it will not work. I normally go to HYPERLINK "http://www.rix-web.com/analyzer/"www.rix-web.com/analyzer/ to write the analog.cfg file but I now get no output at all. The Output file only has a few lines: Web Server Statistics for HYPERLINK "http://www.lakesnw.co.uk/index.htm"CountryProgram started at Tue-01-Aug-2006 09:48. So you DO get output, and Analog 5.32 IS running. You have FILEINCLUDE log*.* FILEEXCLUDE in your .cfg file. This is telling Analog to ignore all requests in your logfile that don't start with "log" - which in most cases will mean that you have told Analog to ignore just about everything in your logfile. http://analog.cx/docs/include.html Aengus + | TO UNSUBSCRIBE from this list: |http://lists.meer.net/mailman/listinfo/analog-help | | Analog Documentation: http://analog.cx/docs/Readme.html | List archives: http://www.analog.cx/docs/mailing.html#listarchives | Usenet version: news://news.gmane.org/gmane.comp.web.analog.general +
Re: [analog-help] log file missing the "?" before parameters
On Wednesday, August 02, 2006 5:08 PM [EDT], John G Doke <[EMAIL PROTECTED]> wrote: 2006-07-19 20:34:01 131.200.79.9 CORPS GET /software/startpage.html integratedarchitecture=EN 200 Mozilla/4.0+(compatible;+MSIE+6.0;+Windows+NT+5.1;+.NET+CLR+1.1.4322) http://localhost:53401/default.aspx These are lines from my log files. This page is used with parameters. /sooftware/startpage.html?training=EN Without the questionmark Analog does not recognise these as parameters. I need to report on the parameters. Thanks for your help LOGFORMAT (%w%Y-%m-%d %h:%n:%j %S %u %j %r %q %c %B %f) will parse the log examples you provided. (There are 2 leading spaces in you exampe lines - if they aren't actually in your log, just drop the %w in the LOGFORMAT). This produces the following Request Report: 5 19/Jul/06 20:34/software/startpage.html 3 19/Jul/06 20:33 /software/startpage.html?training=EN 1 19/Jul/06 20:34 /software/startpage.html?integratedarchitecture=EN 1 19/Jul/06 20:33 /software/startpage.html?phonesupport=EN If you need to analyse this in greater detail, you might be able to use the Internal Search Engine commands. http://analog.cx/docs/args.html#SEARCHENGINE Aengus + | TO UNSUBSCRIBE from this list: |http://lists.meer.net/mailman/listinfo/analog-help | | Analog Documentation: http://analog.cx/docs/Readme.html | List archives: http://www.analog.cx/docs/mailing.html#listarchives | Usenet version: news://news.gmane.org/gmane.comp.web.analog.general +
Re: [analog-help] Is there a way to see what files have high processingtimes? & when?
On Tuesday, August 08, 2006 3:24 PM [EDT], Jeff Wilkinson <[EMAIL PROTECTED]> wrote: Hello fellow analogers, I'm an experienced analog user and I've been through the help files, but perhaps one of you has a suggestion here: I have a server that is periodically hanging on .ASP files, still serving .htm, etc, but ASP execution hangs. I've been trying to use analog to find out more about the problem since it's not clear from direct looks at the logs what is causing the problem. Anyway, one lead I was following was the processing time report. With the idea that the hung files would show up as long processing times (if they are even logged, which they might not be), I was trying to get analog to give me more detail than the summary it gives by default for the processing time report. Is there any way to add a column for the filename to that report? No. That's one of the FAQ 128 situations http://analog.cx/docs/faq.html#faq128 Can I find out which files each referrer pointed to? or Can I find out which files each host has read? or Can I find out which hosts have read each file? or Can I find out the number of hosts visiting on each day? or lots of similar questions. There are lots of questions like this. They all want analog to cross-reference two sorts of item (e.g. files and referrers in the first example above, or hosts and dates in the last). The solution is to use the *INCLUDE commands. For example, to find out which files a particular referrer pointed to, restrict the analysis to just the referrer you're interested in with the REFINCLUDE command. The Request Report will then just list the files reached from that referrer. Similarly, if you're interested in a particular time period, you can focus on just that time by using the FROM and TO commands. You do have to run analog once for each report generated. So for the example of referrers, you would have to run it once for each referrer you're interested in. Of course, it would be useful to be able to cross-reference all the files and referrers at once. But it is fundamental to analog's speed and minimal memory requirement that it only records statistics for each type of item individually, and doesn't record enough information to cross-reference them afterwards. Unfortunately, Analog does not have a PROCTIMEINCLUDE command to only include log entries that took longer than a specified time. So you'll have to extract log entries with long processing times to do the kind of analysis you want. The PROCTIMECOLS doesn't seem to have an arg for the filename. Presumably the only reports that show individual files do it without being told to... and summary reports don't do it optionally, right? I suspect this is a case for grep or hand-looking through the logfiles (ug), but thought it might be worth a check first. On Unix, I'm sure someone could give you a grep command that would do it quickly. You can do it fairly easily in an Windows command prompt too, but it's a little bit messy. If the processing time field in your logfile is (for example) the 12th field in the log, you can extract all logfile entries where the processing time is greater than 5 seconds (which I think is recorded as 5000 milliseconds in IIS) with this command (all on a single line): FOR /F "tokens=1,2,3,4,5,6,7,8,9,10,11,12*" %a in (ABC.log) do if %l GTR 5000 @echo %a %b %c %d %e %f %g %h %i %j %k %l %m >> LONG.log If it's the 8th field, then you can trim back the list of letters and use "if %h GTR" etc. The * at the end of the list of tokens catches the rest of the line after the field you want to test. And you should probably get the log headers (the lines starting with #) into LONG.log first. It's not lightning fast, but I can crunch about a quarter of a million log entries per minute with that command, so it may be good enough, depending on how big your logfiles are. Aengus + | TO UNSUBSCRIBE from this list: |http://lists.meer.net/mailman/listinfo/analog-help | | Analog Documentation: http://analog.cx/docs/Readme.html | List archives: http://www.analog.cx/docs/mailing.html#listarchives | Usenet version: news://news.gmane.org/gmane.comp.web.analog.general +
Re: [analog-help] Re: Is there a way to see what files have highprocessing times? & when?
On Tuesday, August 08, 2006 3:40 PM [EDT], Jeff Wilkinson <[EMAIL PROTECTED]> wrote: Jeff Wilkinson <[EMAIL PROTECTED]> writes: whoops, sorry to followup my own email, but I forgot to ask this: Is there any way to get more detail on the Status report? or to perhaps do a report where you only include by status? STATUSINCLUDE http://analog.cx/docs/include.html Thinking a requests report that only includes files with status "500 Internal server error" might be real handy... yes, I did see the notes in the docs that mention that almost all the reports only show data on successful hits, and that analog isn't real useful for failure analysis... sigh. You did? That's not in my copy of the documentation! It may be that logfiles aren't the right place to be looking for to find information about a failure, but if the logfiles do contain such information, Analog is probably the best tool available for getting the information out of the data. Analog has a range of Failure reports, but they are off by default. Aengus + | TO UNSUBSCRIBE from this list: |http://lists.meer.net/mailman/listinfo/analog-help | | Analog Documentation: http://analog.cx/docs/Readme.html | List archives: http://www.analog.cx/docs/mailing.html#listarchives | Usenet version: news://news.gmane.org/gmane.comp.web.analog.general +
Re: [analog-help] Re: Is there a way to see what files have highprocessing times? & when?
On Tuesday, August 08, 2006 4:47 PM [EDT], Glenn Keffer <[EMAIL PROTECTED]> wrote: Apologies if this is too obvious, but do you have the Failure Report on? I believe it will show you which files failed, but I don't remember if correlates it to a status code. It does. The Failure Report will show the filenames that were requested that returned a "failure" status code (usually a 404 or a 401, but requests that return a 500 are also listed in the Failure Report). But all failed requests are reported in the Failure Report. To see only the 500's, use STATUSINCLUDE 500- But you would also need FAILURE ON, because the Failure report is not on by default, and if you're only including requests with a 500 status code the default reports, that only report on Successful requests, would all be empty. Aengus + | TO UNSUBSCRIBE from this list: |http://lists.meer.net/mailman/listinfo/analog-help | | Analog Documentation: http://analog.cx/docs/Readme.html | List archives: http://www.analog.cx/docs/mailing.html#listarchives | Usenet version: news://news.gmane.org/gmane.comp.web.analog.general +
Re: [analog-help] Re: Is there a way to see what fileshave highprocessing times? & when?
On Wednesday, August 09, 2006 1:11 AM [EDT], Jeff Wilkinson <[EMAIL PROTECTED]> wrote: Aengus wrote: Jeff Wilkinson <[EMAIL PROTECTED]> wrote: Is there any way to get more detail on the Status report? or to perhaps do a report where you only include by status? STATUSINCLUDE http://analog.cx/docs/include.html ah, missed that. thanks! Thinking a requests report that only includes files with status "500 Internal server error" might be real handy... yes, I did see the notes in the docs that mention that almost all the reports only show data on successful hits, and that analog isn't real useful for failure analysis... sigh. You did? That's not in my copy of the documentation! here are the quotes I was referring to: http://analog.cx/docs/defns.html "Most reports only include successful requests in calculating the number of requests, requests for pages, bytes, and last date: unless, of course, the report is a redirection or failure report." http://analog.cx/docs/faq.html#faq139 # Why doesn't analog agree with grepping the logfile? "Have you understood what analog includes in its counts? In particular, most reports only list "successful" requests (HTTP status codes 200-209 & 304). A naïve grep would count failures too." Analog differentiates between successful and unsuccessful requests, so the number of requests or pages counted in the Request Report or the Host Report etc are successful requests. To see the number of failed attempts to access a particular file you look in the Failure Report, or to see the number of failed request from a particluar host, see the Host Failure report, etc. Differentiating between successful and unsuccessful requests makes Analog better at failure analysis, not worse. Aengus + | TO UNSUBSCRIBE from this list: |http://lists.meer.net/mailman/listinfo/analog-help | | Analog Documentation: http://analog.cx/docs/Readme.html | List archives: http://www.analog.cx/docs/mailing.html#listarchives | Usenet version: news://news.gmane.org/gmane.comp.web.analog.general +
Re: [analog-help] Re: Is there a way to see what fileshave highprocessing times? & when?
On Wednesday, August 09, 2006 1:37 AM [EDT], Jeff Wilkinson <[EMAIL PROTECTED]> wrote: yes, I have failure report on. I was trying to get more info included in it. yes, it has filename/path & reqs, but at least the default isn't showing the status code with each file. The status report shows a summary of how many files have each status code... but neither does both detailed filename AND the status code, which wouldn't seem to be a FAQ 128 situation since that info is definitely in the same line of the logs, not something you'd have to cross-reference. Each file can generate multiple status codes, so it is a FAQ 128 situation. If you want to get a report on the files that generate Status 500, you need to use STATUSINCLUDE 500. Then you would have to run a seperate report with STATUSINCUDE 404 to get those files, etc. FYI, I do get log entries on scripts during the hangs. Here's an example. Note the last field (time-taken) #Fields: date time c-ip cs-username s-ip cs-method cs-uri-stem cs-uri-query sc-status sc-win32-status sc-bytes cs-bytes time-taken 2006-08-01 23:56:26 64.187.48.55 - 216.13.158.101 GET /peren/perenindex.asp |-|ASP_0147|500_Server_Error 302 0 0 515 10956703 2006-08-01 23:56:26 64.187.48.55 - 216.13.158.101 GET /peren/peren.asp -|ASP_0147|500_Server_Error 302 0 0 514 10263640 2006-08-01 23:56:26 69.157.150.101 - 216.13.158.101 GET /roses/roseindex.asp |-|ASP_0147|500_Server_Error 302 0 0 487 10099703 2006-08-01 23:56:26 69.157.150.101 - 216.13.158.101 GET /roses/roseindex.asp |-|ASP_0147|500_Server_Error 302 0 0 487 10096500 2006-08-01 23:56:26 69.157.150.101 - 216.13.158.101 GET /roses/roseindex.asp |-|ASP_0147|500_Server_Error 302 0 0 487 10076875 2006-08-01 23:56:26 69.157.150.101 - 216.13.158.101 GET /roses/sections.asp |-|ASP_0147|500_Server_Error 302 0 0 486 10074718 2006-08-01 23:56:26 69.157.150.101 - 216.13.158.101 GET /roses/rosesearch.asp |-|ASP_0147|500_Server_Error 302 0 0 488 10070828 Unfortunately, since it's on a webhosting account, I don't have direct admin access to much past the logfiles, so I have to pass requests for anything more direct like events logs through support. Fun all around. sigh. It looks like those files are actually returning a 302 status code. The 500 error message is actually in the cs-uri-query field. You might need to look in the Redirection Report rather than the Failure Report. Google ASP_0147 for some more ideas. Aengus + | TO UNSUBSCRIBE from this list: |http://lists.meer.net/mailman/listinfo/analog-help | | Analog Documentation: http://analog.cx/docs/Readme.html | List archives: http://www.analog.cx/docs/mailing.html#listarchives | Usenet version: news://news.gmane.org/gmane.comp.web.analog.general +
