In my webmastering days we used AWStats to analyze our log files.

http://awstats.sourceforge.net/

It has been a while, but I remember it being very configurable and easy
to use.  It might we worth looking it over to see whether it would yield
what you want for your analysis...might save you some headaches.



Eric Lease Morgan wrote:

How would you go about doing some analysis of your website's referrer
data?

I have committed to writing an article for the anniversary issue of
First Monday (as if I don't already have enough to do). Here is the
accepted/proposed title and abstract:

  Ethical issues surrounding freely available information
  found on the Web

  By reverse engineering Google queries and by tracing back
  the referrer values found in Apache log files, the use of
  content made available from infomotions.com is examined and
  ethical questions are asked. While all the content from the
  site is "freely" available under the GNU Public License, the
  content is not always used in the intended manner. This
  raises interesting questions regarding the time spent making
  the content available, the expense of the hardware and
  network connections, and whether or not the application of
  the content is put to good and moral purposes. This essay
  addresses these and other ethical questions in an attempt to
  come to an understanding regarding the place of information
  and knowledge in an "open" environment.

I find it interesting to watch the content of my access_log scroll by
on my console. I am most interested in the referrer information. Most
of my hits originate as searches against Google. It is fun feed these
queries back into Google and see what people searched for, watch what
the searches return, and see what page number my item is located. I
see that a lot of the hits to my site come from MySpace.com where
teenaged and college aged girls have incorporated some of my pictures
into their pages. Another common use is on "bulletin board" systems
where someone used one of my pictures as their avatar. In these
second and third cases should I expect some sort of remuneration or
at least a link back to infomotions.com?

Some hits come from really weird places. For example, the search for
"lease" brings back many hits about equipment rental, but sometimes
my name and/or the Alex Catalogue of Electronic Texts is linked from
the equipment rental site. Sort of strange if you ask me. They are
using my name, sort of. ("Is it 'my' name?")

In any event, I plan to take two months of access_log data, extract
the pages being looked at and the referrer information to more
systematically examine how the content on Infomotions is being
incorporated into other sites. How would you suggest I do this?
Presently I plan to extract the necessary information from my logs
and dump it into a flat database file where I will exploit various
incarnations of SQL SELECT statements. Count this. Group that. Sort
this way. Etc. Mind you, I am most interested in the one-off sort of
hits, not just the overall usage.

How would you go about doing this sort of analysis? All I have to
start with is my Apache "combined" access_log files?

--
Eric Lease Morgan
University Libraries of Notre Dame

Reply via email to