I agree with Eric, that it should NOT be enabled by default, until the
appropriate maintenance code to accompany it is also written. The
maintenance code could be a simple quartz job, that truncates the table
after so many (configurable) days. That combined with logging a select
number of events, would probably keep it from growing out of control
with 'all default' settings.
---- Cris J H
Eric Dalquist wrote:
It's too much data as is. This logger is just that ... a logger. All it
does it persist portal events in a nice normalized set of tables. Until
someone writes summarizing and maintenance code we can't enable it
without having to include giant warnings about filling up peoples
databases. As an example we are using this here and track logins,
logouts, channel render time, portlet action time, and page render time.
This really is no different than log rolling issues we have on the file
system. For comparison this is a query from our production stats table:
COUNT(*) 8,651,758
MIN(ACT_DATE) 2008-05-30 10:43:02.85
MAX(ACT_DATE) 2008-06-06 14:08:14.943
So we've had 8.6 million events logged in 7 days and students have been
gone since the 17th.
My hope is that one the code is there and easy to enable (should just be
uncommenting a chunk of xml) my hope would be that someone (like you)
goes and writes that portlet AND a little bit of code to flush old data
from the tables or aggregate it into some more manageable summary view
which can be stored without concern.
As for Jen's post I'd think the most minimal option would be to log page
renders and possibly channel renders/portlet actions. That would allow
some decent performance graphing for people. I'm also going to look into
a additional config option to disable the logging of groups associated
with a stats session to save on table space.
Does that sound reasonable?
-Eric
Andrew Petro wrote:
Eric,
> It will be disabled by default.
How about enabling it by default and in time for 3.1 someone (me?)
goes after adding a JSR-168 portlet that does simple queries and
presents them in Jen-Feedback-Portlet-style lickable graphs, giving
uPortal even more of a marketable ootb suggestion that yes, this is a
suitable platform for building metrics indicators?
I'm envisioning a whole happily DLM-managed Stats tab presented to the
admin user...
Andrew
Eric Dalquist wrote:
Here is a more detailed user-targeted description of the database
stats recorder:
http://www.ja-sig.org/wiki/display/UPC/Database+Event+Logger
I'm planning on committing this to trunk later today if I don't hear
any objections. It will be disabled by default.
-Eric
Eric Dalquist wrote:
I wanted to run this proposal for a database backed portal event
logger by the dev list before going forward with any commits. The
schema design is based closely on what we've been using here at UW
since we switched to uPortal. The PortalEvent persistence is done
using JPA/Hibernate to avoid introducing another JDBC API, in
performance testing using the ORM layer for event logging doesn't
appear to have much excess overhead.
We don't have a reporting interface for this data yet and work will
likely need to be done on a tool to summarize the raw data into
tables that are easier to report against as for large installs the
event log tables can grow much too large for reasonable query
performance.
There are still a few things I need to clean up before I'd be ready
to commit it, the biggest one is to setup a separate JPA persistence
context to allow the stats to be stored to their own database. We
discovered this is necessary in large installs as well since the
portal tables and the stats tables have opposite usage semantics, as
the portal DB is primarily read from and the stats tables are
primarily written to.
I would only plan on including this for 3.1 as there are some
non-trivial changes in the framework to provide additional
information for some of the portal events.
As always I look forward to feedback.
-Eric
Eric Dalquist wrote:
I have the UW database stats recorder mostly ported to uPortal 3.
I've attached a patch to the Jira issue I'm using to track the
work: http://www.ja-sig.org/issues/browse/UP-2075 As noted in the
Jira issue there is more work to be done which should be addressed
in the next few days. Feel free to apply the patch and try the code
if you're so inclined, hopefully I'll have something ready for
inclusion in 3.1 soon.
-Eric
Eric Dalquist wrote:
I'm actually starting on it today, I'll keep everyone posted as I
progress.
-Eric
Matt Peterson wrote:
Cool. I think I would be more of a hindrance than a help if
you’re planning on porting it yourself in the next few weeks. If
it’s likely to be more than a few weeks though, I’ll gladly give
it a go, as I will need it for our next phase (23^rd June).
Cheers,
Matt.
------------------------------------------------------------------------
*From:* [EMAIL PROTECTED]
[mailto:[EMAIL PROTECTED] *On Behalf Of
*Eric Dalquist
*Sent:* Friday, 23 May 2008 12:39 AM
*To:* [EMAIL PROTECTED]
*Subject:* Re: [uportal-user] uPortal 3 stats recorder to database
We have one for the 2.5 stats recorder interfaces that I'll be
porting to 3.0 in the next few weeks If you're interested in
either helping with that work or waiting for us to get to it
we'll be happy (and plan to) share it.
-Eric
Matt Peterson wrote:
Does anyone know of a stats recorder for uPortal3 which records
to db tables using uPortals spring defined datasource? Or any
other suggestion with regards to recording stats to our db?
Cheers,
Matt Peterson
Web Developer
University of New England
Ph. (02) 6773 3550
--
You are currently subscribed to [EMAIL PROTECTED] <mailto:[EMAIL PROTECTED]> as: [EMAIL PROTECTED] <mailto:[EMAIL PROTECTED]>
To unsubscribe, change settings or access archives, see
http://www.ja-sig.org/wiki/display/JSG/uportal-user
--
You are currently subscribed to [EMAIL PROTECTED] as: [EMAIL PROTECTED]
To unsubscribe, change settings or access archives, see
http://www.ja-sig.org/wiki/display/JSG/uportal-user
--
You are currently subscribed to uportal-dev@lists.ja-sig.org as: [EMAIL
PROTECTED]
To unsubscribe, change settings or access archives, see
http://www.ja-sig.org/wiki/display/JSG/uportal-dev
--
You are currently subscribed to uportal-dev@lists.ja-sig.org as: [EMAIL
PROTECTED]
To unsubscribe, change settings or access archives, see
http://www.ja-sig.org/wiki/display/JSG/uportal-dev
