Allen Gilliland wrote:
Elias Torres wrote:
I'm very much for the motivation behind this effort but I think we need
to think this about it a little bit more. At first I was imagining that
*both* referrers and hits would be tracked externally, but this proposal
only solves half my problem. I'd support this if we did both at the same
time but we left this for 3.2 unless it was absolutely guaranteed you
could finish it off by the end of the week.
Asynchronous processing of referrers was implemented a while ago. It is
controlled by these 3 settings in the roller config file ...
referrers.asyncProcessing.enabled=false
referrers.queue.numWorkers=3
referrers.queue.sleepTime=10
So you would have to enable it, and then you can choose how many threads
run to process the incoming queue and how long each thread should sleep
(in seconds) between runs. Each "run" processes any referrers in the
queue until the queue is empty.
I'll warn you though, AFAIK it works fine and we used it on
blogs.sun.com for a short while with no real problems, except that the
workers really couldn't keep up with the queue. I think I had the
workers up to around 25 before I decided that it wasn't really worth it
anymore and that what I really wanted was to disable the referrer
processing all together.
Sorry, brain fart, I didn't really finish my thought.
What I meant to convey is that, in a small/moderate environment I don't
think the referrer processing is a big deal. It's nice if it happens
asynchronously, but I doubt you get a noticeable performance benefit
given that the installation is smallish. However, if you are
considering doing this for a large installation then you are probably
wasting your time. IMO when you get to be a large installation (meaning
large in terms of traffic) then doing referrer processing in the db just
doesn't make sense, that's what access logs are for.
If you wanted to be really sneaky you could setup a custom task which
would parse out your access logs and dump that data into the db for your
users, but I have no idea what the performance impact would be if the
data set was expanded over many weeks/months. On blogs.sun.com we
already see some slow queries from the referer table even though we only
keep a single days worth of data :/
-- Allen
So, if this proposal goes through then you would be able to control both
hit counting and referrer tracking independently of each other. i.e.
enable/disable either or both of them, as well as control a bit how they
work.
BTW, is the task locking work completed? I have a couple of tasks I want
to develop and I'm dependent on that feature. Please let me know when
it's ready to be tested. Thanks.
Yes, it should be working. I still want to do some more testing on it,
but I'd be glad to get some help with that :) You should be able to add
new tasks simply by extending the new RollerTask class and then setting
the right config elements in the config file.
-- Allen
-Elias
Allen Gilliland wrote:
I have a new proposal for separating out the hit count tracking from the
existing referrer tracking system so that they can each function
independently. This would hopefully happen for 3.1.
http://rollerweblogger.org/wiki/Wiki.jsp?page=Proposal_IndependentHitCounts
Please take a look at the proposal soon and response with any questions,
comments, objections, etc. I plan to begin working on the
implementation very soon and would likely try and commit before the end
of the week.
-- Allen
