I have this work completed in my workspace and I am almost ready to
check it in, so I'd like to ask for comments/opinions/objections one
more time.
Proposal is here ...
http://rollerweblogger.org/wiki/Wiki.jsp?page=Proposal_IndependentHitCounts
-- Allen
Allen Gilliland wrote:
Anyone have any other comments/opinions about doing independent hit
count tracking?
-- Allen
Allen Gilliland wrote:
Elias Torres wrote:
Allen Gilliland wrote:
Elias Torres wrote:
I'm very much for the motivation behind this effort but I think we
need
to think this about it a little bit more. At first I was imagining
that
*both* referrers and hits would be tracked externally, but this
proposal
only solves half my problem. I'd support this if we did both at the
same
time but we left this for 3.2 unless it was absolutely guaranteed you
could finish it off by the end of the week.
Asynchronous processing of referrers was implemented a while ago.
It is
controlled by these 3 settings in the roller config file ...
referrers.asyncProcessing.enabled=false
referrers.queue.numWorkers=3
referrers.queue.sleepTime=10
So you would have to enable it, and then you can choose how many
threads
run to process the incoming queue and how long each thread should sleep
(in seconds) between runs. Each "run" processes any referrers in the
queue until the queue is empty.
I'll warn you though, AFAIK it works fine and we used it on
blogs.sun.com for a short while with no real problems, except that the
workers really couldn't keep up with the queue. I think I had the
workers up to around 25 before I decided that it wasn't really worth it
anymore and that what I really wanted was to disable the referrer
processing all together.
So, if this proposal goes through then you would be able to control
both
hit counting and referrer tracking independently of each other. i.e.
enable/disable either or both of them, as well as control a bit how
they
work.
good. In a large installation I don't want to do neither referral or hit
count tracking. However, I'm not sure if we can today disable referral
and do it externally because don't our queries assume that the table
contains raw data as opposed to summarized data? I would think we should
rethink our built-in referral think to assume massaged data as in the
tagging case for performance and then we can externalize it. Also, not
sure if your proposal for *just* dailyhits is enough. Maybe it is for
the one sidebar that uses this info today, but I'd think we would like
to keep track of a week's or months number of hits. Am I thinking the
wrong way?
I totally agree. I've been wanting to do something better with the
referrer tracking for a while, but there were more important things to
worry about first.
I would agree that having the summarized data makes more sense for
performance, but I think that part of the way the referrers are used
is like a comment which is gathered via linkback extraction. If it
was only used for tracking hits from external sources then I don't see
the benefit at all, any stats package in the world would give you
better metrics for something like that. So I'm not sure that you
could really do summarized work for the referrers stuff, it's
currently setup to expect a single row per referrer :/
I also agree that just doing daily hits doesn't make a whole ton of
sense and I had tried to think of a few ways to add some more
interesting counts in there, but opted for speed in getting this done
rather than adding new features. We could certainly add weekly and
monthly counts in there, but they would be restricted the same way
that daily is now, eventually it has to be cleared because it's not
meant for archiving. I had also considered the idea of trying to
maintain an average # of hits for a weblog so that you could somehow
identify if that weblog had gotten extra hits on a given day and could
then be identified as a hot weblog that day, even though relatively
speaking it may not get as hit of a hit count as many other blogs in
the system. i.e. basing some of the hot blogs off of % increase in
traffic for a day, rather than purely off of raw hit counts.
In any case, it's an open proposal, so we can design it to do whatever
we think would be useful.
-- Allen
BTW, is the task locking work completed? I have a couple of tasks I
want
to develop and I'm dependent on that feature. Please let me know when
it's ready to be tested. Thanks.
Yes, it should be working. I still want to do some more testing on it,
but I'd be glad to get some help with that :) You should be able to
add
new tasks simply by extending the new RollerTask class and then setting
the right config elements in the config file.
-- Allen
-Elias
Allen Gilliland wrote:
I have a new proposal for separating out the hit count tracking
from the
existing referrer tracking system so that they can each function
independently. This would hopefully happen for 3.1.
http://rollerweblogger.org/wiki/Wiki.jsp?page=Proposal_IndependentHitCounts
Please take a look at the proposal soon and response with any
questions,
comments, objections, etc. I plan to begin working on the
implementation very soon and would likely try and commit before
the end
of the week.
-- Allen
