Re: [Mailman-Users] List statistics
At 6:47 PM + 2004-11-15, PeteBell wrote: Namely, the ability to publish a regularly updating set of statistics on my website with figures for: Number of members currently subscribed Average number of emails per day Number of emails today (etc) If you have access to the server, you can run various log processing scripts which can create this kind of information. If you use postfix as your MTA, check out a tool called pflogsumm. Another good tool to look at is lire from logreport.org. Please note that I do NOT have access to the actual mailman program as it is hosted as a server-wide installation shared by all domains on the server, of which I am but one. That's going to be much more difficult. Mailman doesn't do any log processing of this sort, and it doesn't have any statistics gathering/reporting features I am aware of. -- Brad Knowles, [EMAIL PROTECTED] Those who would give up essential Liberty, to purchase a little temporary Safety, deserve neither Liberty nor Safety. -- Benjamin Franklin (1706-1790), reply of the Pennsylvania Assembly to the Governor, November 11, 1755 SAGE member since 1995. See http://www.sage.org/ for more info. -- Mailman-Users mailing list [EMAIL PROTECTED] http://mail.python.org/mailman/listinfo/mailman-users Mailman FAQ: http://www.python.org/cgi-bin/faqw-mm.py Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/
Re: [Mailman-Users] list statistics
Now run the file once to test it: /usr/local/sbin/mm_stats bash: /usr/local/sbin/mm_stats: bad interpreter: No such file or directory You should see no output, and after a few seconds a prompt will come back. What do i wrong? Danny Terweij. You my friend do not have bash! Are you running on Solaris? I'll have to check the script to see if work right on Solaris. I wrote it on Linux. In the meantime, you can try changing the first line of the script to: #! /bin/sh The first line controls which shell runs the script. I love bash, and fortunately for me, most other folks do too, so it's on a huge number of machines. The only boxes I program on that don't have bash on them by default are Solaris boxen. Go figure. Don't worry about experimenting with the script. It only writes to a file in /tmp/.. The rest of the script is a bunch of reads, etc. Let me know what happens! Jon Carnes -- Mailman-Users mailing list [EMAIL PROTECTED] http://mail.python.org/mailman/listinfo/mailman-users Mailman FAQ: http://www.python.org/cgi-bin/faqw-mm.py
Re: [Mailman-Users] list statistics
From: Jon Carnes [EMAIL PROTECTED] bash: /usr/local/sbin/mm_stats: bad interpreter: No such file or directory You my friend do not have bash! Are you running on Solaris? I'll have to check the script to see if work right on Solaris. I wrote it on Linux. Hi, i did a mistake.. i edited the script at Windows notepad. And put it at my homedir though Samba. Later i edit it with pico and it worked.. I'll think it was a formating problem.. It works great now and changed it to my own language. (Only Dutch lists). Danny Terweij -- Mailman-Users mailing list [EMAIL PROTECTED] http://mail.python.org/mailman/listinfo/mailman-users Mailman FAQ: http://www.python.org/cgi-bin/faqw-mm.py
Re: [Mailman-Users] list statistics
Here is the final list stats script. If you use this script as is, then you
should run the script just before rotating your Mailman log files.
If you want to run it right after rotating your log files, and your log files
rotate by adding a .1 to the end of post (so the rotated file would be
post.1) then simply change the line:
POST= ...
and point it to your rotated log file.
Any suggestions or modifications are welcome,
Jon Carnes
#! /bin/bash
# Run monthly stats on Meeting maker logs
# - top 10 users of each list
# - Number of attempted posts (per list)
# - Number of Successful posts (per list)
# - Total bytes sent (per list)
#
POST=~mailman/logs/post
echo Stats from local Mailman lists:
echo -nStarting:
head -1 $POST |cut -f1-3 -d
echo -nEnding:
tail -1 $POST |cut -f1-3 -d
LIST=`/home/mailman/bin/list_lists |awk '{print $1}' |sed -n '2,$p'`
echo
for i in $LIST
do
echo == $i ==
echo -n Total posts to the list:
grep -i post to $i $POST |wc -l
echo -n Total SUCCESSFUL posts to the list:
grep -i post to $i $POST |grep success |wc -l
SIZ=`grep -i post to $i $POST |grep success |cut -f2 -d= |cut -f1 -d,`
k=0; for j in $SIZ; do k=$(( j + k )); done; echo Total bytes = $k
echo
echo Top 10 posters to the list:
grep -i post to $i $POST |cut -f 10 -d |sort |uniq -c \
|sort -bgr |head -10
echo
done
--
Mailman-Users mailing list
[EMAIL PROTECTED]
http://mail.python.org/mailman/listinfo/mailman-users
Mailman FAQ: http://www.python.org/cgi-bin/faqw-mm.py
Re: [Mailman-Users] list statistics - sent to list admins
Alright so I modified it right away. I forgot that originally they wanted to
mail off the stats to each list admin...
This script lists out certain stats for each list and sends those stats to
each list admin (and the mailman user).
=== begin script: mm_stats ===
#! /bin/bash
# Run monthly stats on Mailman logs
# Mail stats to each lists admin
# - Number of attempted posts
# - Number of Successful posts
# - Total bytes sent
# - top 10 users of each list
#
# Mailman's log file to be examined for stats
POST=~mailman/logs/post
# create temp file to collect stats
TMPFILE=`mktemp /tmp/mm_stats.XX` || exit 1
LIST=`/home/mailman/bin/list_lists |awk '{print $1}' |sed -n '2,$p'`
for i in $LIST
do
echo Stats from local Mailman list: $i $TMPFILE
echo $TMPFILE
echo -nStarting:$TMPFILE
head -1 $POST |cut -f1-3 -d $TMPFILE
echo -nEnding: $TMPFILE
tail -1 $POST |cut -f1-3 -d $TMPFILE
echo === $TMPFILE
echo -n Total posts to the list: $TMPFILE
grep -i post to $i $POST |wc -l $TMPFILE
echo -n Total SUCCESSFUL posts to the list: $TMPFILE
grep -i post to $i $POST |grep success |wc -l $TMPFILE
SIZ=`grep -i post to $i $POST |grep success |cut -f2 -d= |cut -f1 -d,`
k=0; for j in $SIZ; do k=$(( j + k )); done
echo Total bytes = $k $TMPFILE
echo $TMPFILE
echo Top 10 posters to the list: $TMPFILE
grep -i post to $i $POST |cut -f 10 -d |sort |uniq -c \
|sort -bgr |head -10 $TMPFILE
echo$TMPFILE
# Mail the collected stats off to the list admin and cc the mailman user
mail -s Mailman Stats for List: $i -c mailman $i-admin $TMPFILE
done
# remove the temp file
rm $TMPFILE
=== end script: mm_stats ===
The script dumps out the names of all the local Mailman lists and then runs
through the POST log for each list. It dumps the stats out to a temp file
and at the end, emails the contents of the temp file to the list admins (and
cc's the mailman user).
Unlike some of my other scribblings to the list, this script is tested and
actually works! Use it as your own risk, etc...
Again, any suggestions and modifications are welcome.
Jon Carnes
--- Original Message: Friday 12 April 2002 03:00 pm ---
Here is the final list stats script. If you use this script as is, then
you should run the script just before rotating your Mailman log files.
If you want to run it right after rotating your log files, and your log
files rotate by adding a .1 to the end of post (so the rotated file would
be post.1) then simply change the line:
POST= ...
and point it to your rotated log file.
Any suggestions or modifications are welcome,
Jon Carnes
#! /bin/bash
# Run monthly stats on Mailman logs
# - top 10 users of each list
# - Number of attempted posts (per list)
# - Number of Successful posts (per list)
# - Total bytes sent (per list)
#
POST=~mailman/logs/post
echo Stats from local Mailman lists:
echo -nStarting:
head -1 $POST |cut -f1-3 -d
echo -nEnding:
tail -1 $POST |cut -f1-3 -d
LIST=`/home/mailman/bin/list_lists |awk '{print $1}' |sed -n '2,$p'`
echo
for i in $LIST
do
echo == $i ==
echo -n Total posts to the list:
grep -i post to $i $POST |wc -l
echo -n Total SUCCESSFUL posts to the list:
grep -i post to $i $POST |grep success |wc -l
SIZ=`grep -i post to $i $POST |grep success |cut -f2 -d= |cut -f1 -d,`
k=0; for j in $SIZ; do k=$(( j + k )); done; echo Total bytes = $k
echo
echo Top 10 posters to the list:
grep -i post to $i $POST |cut -f 10 -d |sort |uniq -c \
|sort -bgr |head -10
echo
done
--
Mailman-Users mailing list
[EMAIL PROTECTED]
http://mail.python.org/mailman/listinfo/mailman-users
Mailman FAQ: http://www.python.org/cgi-bin/faqw-mm.py
--
Mailman-Users mailing list
[EMAIL PROTECTED]
http://mail.python.org/mailman/listinfo/mailman-users
Mailman FAQ: http://www.python.org/cgi-bin/faqw-mm.py
Re: [Mailman-Users] list statistics
Login as root or mailman on your local server Take the latest script and cut and paste it into a file in /usr/local/sbin/.. I called my file mm_stats. The latest script is in an earlier email to the list. Next make the new file executable: chmod a+x /usr/local/sbin/mm_stats Now run the file once to test it: /usr/local/sbin/mm_stats You should see no output, and after a few seconds a prompt will come back. In a few minutes your mailman user and all your list admins should recieve a message indicating the current stats for their lists. If you rotate your Mailman logs, then run this script (via cron) just before you rotate the logs. If you don't rotate your logs, then look at the FAQ and start to do so! otherwise your logs will grow and grow and grow... I've set my logs to rotate on the 1st of every month at 4am. So I would set this script to run at 3:59 am on the 1st of the month: su mailman crontab -e 59 3 1 * * /usr/local/sbin/mm_stats --- Original Message: Wednesday 10 April 2002 05:12 am --- Hi, Can be added that a list-owner gets this via a mail and also the site-admin gets a mail? How to implement the code? just cut and paste the code in /etc/cron.weekly/mailman-stats ? Danny. -- Mailman-Users mailing list [EMAIL PROTECTED] http://mail.python.org/mailman/listinfo/mailman-users Mailman FAQ: http://www.python.org/cgi-bin/faqw-mm.py
Re: [Mailman-Users] list statistics
From: Jon Carnes [EMAIL PROTECTED] Did cut an paste the script. Did modify some paths (Mine was already installed at /var/mailman (RH7.2)) Next make the new file executable: chmod a+x /usr/local/sbin/mm_stats Done. Now run the file once to test it: /usr/local/sbin/mm_stats bash: /usr/local/sbin/mm_stats: bad interpreter: No such file or directory You should see no output, and after a few seconds a prompt will come back. What do i wrong? Danny Terweij. -- Mailman-Users mailing list [EMAIL PROTECTED] http://mail.python.org/mailman/listinfo/mailman-users Mailman FAQ: http://www.python.org/cgi-bin/faqw-mm.py
RE: [Mailman-Users] list statistics
thanks jon, that looks promising!! i will give it a go tonight! talk about quick turnround --- Mark Johnson - Hardware Install Specialist Central Hardware Installation Team (CHIT) B5, Ground Floor, Block 10, Radbroke Hall Barclays Bank PLC - Great Britain E-Mail: [EMAIL PROTECTED] Phone: 01565 612510 --- -Original Message- From: Jon Carnes [mailto:[EMAIL PROTECTED]] Sent: 09 April 2002 20:27 To: Johnson, Mark : Enable; mailman-users Subject: Re: [Mailman-Users] list statistics I'll have to add size to it, but how's this for a start: Weekly === LISTNAM=your list name echo -n Total post to the list $LISTNAM: grep -i post to $LISTNAME ~mailman/logs/post |wc -l echo echo Top 10 posters to the list: grep -i post to $LISTNAM ~mailman/logs/post |cut -f 10 -d | \ sort |uniq -c |sort -r |head -10 === This assumes that there are a weeks worth of logs in ~mailman/logs/post I actually run the script on the file ~mailman/logs/post.1 - after the files have been rotated. You can, however, run it weekly before you rotate your logs. If you want monthly stats then you will need to keep 5 weeks of logs. I rotate my logs weekly and number the logs as such: post - current log for postings post.1 - last weeks log for postings post.2 - postings from 2 weeks ago post.3 - postings from 3 weeks ago post.4 - postings from 4 weeks ago post.5 - postings from 5 weeks ago Monthly (run on 1st of the month) === LISTNAM=your list name MONTH=`date -d last month +%b` echo -n Total post to the list $LISTNAM: grep -i post to $LISTNAME ~mailman/logs/post* |wc -l echo grep -i post to $LISTNAM ~mailman/logs/post* |grep :$MONTH | \ cut -f 10 -d |sort |uniq -c |sort -r |head -10 === Jon Carnes - Original Message - From: [EMAIL PROTECTED] To: [EMAIL PROTECTED] Sent: Tuesday, April 09, 2002 11:20 AM Subject: [Mailman-Users] list statistics all, does anyone know of a plug-in or a hack to get statistics from a list? i am looking for something that will generate a monthly / weekly report with things like: top 10 users, average daily message count, average message size etc? anyone know of a feature to do this? thanks p.s. great software!! ** Internet communications are not secure and therefore the Barclays Group does not accept legal responsibility for the contents of this message. Although the Barclays Group operates anti-virus programmes, it does not accept responsibility for any damage whatsoever that is caused by viruses being passed. Any views or opinions presented are solely those of the author and do not necessarily represent those of the Barclays Group. Replies to this email may be monitored by the Barclays Group for operational or business reasons. -- Mailman-Users mailing list [EMAIL PROTECTED] http://mail.python.org/mailman/listinfo/mailman-users Mailman FAQ: http://www.python.org/cgi-bin/faqw-mm.py Internet communications are not secure and therefore the Barclays Group does not accept legal responsibility for the contents of this message. Although the Barclays Group operates anti-virus programmes, it does not accept responsibility for any damage whatsoever that is caused by viruses being passed. Any views or opinions presented are solely those of the author and do not necessarily represent those of the Barclays Group. Replies to this email may be monitored by the Barclays Group for operational or business reasons. -- Mailman-Users mailing list [EMAIL PROTECTED] http://mail.python.org/mailman/listinfo/mailman-users Mailman FAQ: http://www.python.org/cgi-bin/faqw-mm.py
Re: [Mailman-Users] list statistics
I'll have to add size to it, but how's this for a start: Weekly === LISTNAM=your list name echo -n Total post to the list $LISTNAM: grep -i post to $LISTNAME ~mailman/logs/post |wc -l echo echo Top 10 posters to the list: grep -i post to $LISTNAM ~mailman/logs/post |cut -f 10 -d | \ sort |uniq -c |sort -r |head -10 === This assumes that there are a weeks worth of logs in ~mailman/logs/post I actually run the script on the file ~mailman/logs/post.1 - after the files have been rotated. You can, however, run it weekly before you rotate your logs. If you want monthly stats then you will need to keep 5 weeks of logs. I rotate my logs weekly and number the logs as such: post - current log for postings post.1 - last weeks log for postings post.2 - postings from 2 weeks ago post.3 - postings from 3 weeks ago post.4 - postings from 4 weeks ago post.5 - postings from 5 weeks ago Monthly (run on 1st of the month) === LISTNAM=your list name MONTH=`date -d last month +%b` echo -n Total post to the list $LISTNAM: grep -i post to $LISTNAME ~mailman/logs/post* |wc -l echo grep -i post to $LISTNAM ~mailman/logs/post* |grep :$MONTH | \ cut -f 10 -d |sort |uniq -c |sort -r |head -10 === Jon Carnes - Original Message - From: [EMAIL PROTECTED] To: [EMAIL PROTECTED] Sent: Tuesday, April 09, 2002 11:20 AM Subject: [Mailman-Users] list statistics all, does anyone know of a plug-in or a hack to get statistics from a list? i am looking for something that will generate a monthly / weekly report with things like: top 10 users, average daily message count, average message size etc? anyone know of a feature to do this? thanks p.s. great software!! ** Internet communications are not secure and therefore the Barclays Group does not accept legal responsibility for the contents of this message. Although the Barclays Group operates anti-virus programmes, it does not accept responsibility for any damage whatsoever that is caused by viruses being passed. Any views or opinions presented are solely those of the author and do not necessarily represent those of the Barclays Group. Replies to this email may be monitored by the Barclays Group for operational or business reasons. -- Mailman-Users mailing list [EMAIL PROTECTED] http://mail.python.org/mailman/listinfo/mailman-users Mailman FAQ: http://www.python.org/cgi-bin/faqw-mm.py -- Mailman-Users mailing list [EMAIL PROTECTED] http://mail.python.org/mailman/listinfo/mailman-users Mailman FAQ: http://www.python.org/cgi-bin/faqw-mm.py
Re: [Mailman-Users] list statistics
Okay, here is the addition of size to the scripts. I also filter on success, as some lists reject mail that is too large. SIZ=`grep -i post to $LISTNAM ~mailman/logs/post |grep success | \ cut -f2 -d= |cut -f1 -d,` j=0; for i in $SIZ; do j=$(( i + j )); done; echo Total bytes = $j I'll tweak these and put them in the FAQ - though it's not really a frequently asked question... Jon Carnes - Original Message - From: Jon Carnes [EMAIL PROTECTED] To: [EMAIL PROTECTED]; [EMAIL PROTECTED] Sent: Tuesday, April 09, 2002 3:26 PM Subject: Re: [Mailman-Users] list statistics I'll have to add size to it, but how's this for a start: Weekly === LISTNAM=your list name echo -n Total post to the list $LISTNAM: grep -i post to $LISTNAME ~mailman/logs/post |wc -l echo echo Top 10 posters to the list: grep -i post to $LISTNAM ~mailman/logs/post |cut -f 10 -d | \ sort |uniq -c |sort -r |head -10 === This assumes that there are a weeks worth of logs in ~mailman/logs/post I actually run the script on the file ~mailman/logs/post.1 - after the files have been rotated. You can, however, run it weekly before you rotate your logs. If you want monthly stats then you will need to keep 5 weeks of logs. I rotate my logs weekly and number the logs as such: post - current log for postings post.1 - last weeks log for postings post.2 - postings from 2 weeks ago post.3 - postings from 3 weeks ago post.4 - postings from 4 weeks ago post.5 - postings from 5 weeks ago Monthly (run on 1st of the month) === LISTNAM=your list name MONTH=`date -d last month +%b` echo -n Total post to the list $LISTNAM: grep -i post to $LISTNAME ~mailman/logs/post* |wc -l echo grep -i post to $LISTNAM ~mailman/logs/post* |grep :$MONTH | \ cut -f 10 -d |sort |uniq -c |sort -r |head -10 === Jon Carnes - Original Message - From: [EMAIL PROTECTED] To: [EMAIL PROTECTED] Sent: Tuesday, April 09, 2002 11:20 AM Subject: [Mailman-Users] list statistics all, does anyone know of a plug-in or a hack to get statistics from a list? i am looking for something that will generate a monthly / weekly report with things like: top 10 users, average daily message count, average message size etc? anyone know of a feature to do this? thanks p.s. great software!! ** Internet communications are not secure and therefore the Barclays Group does not accept legal responsibility for the contents of this message. Although the Barclays Group operates anti-virus programmes, it does not accept responsibility for any damage whatsoever that is caused by viruses being passed. Any views or opinions presented are solely those of the author and do not necessarily represent those of the Barclays Group. Replies to this email may be monitored by the Barclays Group for operational or business reasons. -- Mailman-Users mailing list [EMAIL PROTECTED] http://mail.python.org/mailman/listinfo/mailman-users Mailman FAQ: http://www.python.org/cgi-bin/faqw-mm.py -- Mailman-Users mailing list [EMAIL PROTECTED] http://mail.python.org/mailman/listinfo/mailman-users Mailman FAQ: http://www.python.org/cgi-bin/faqw-mm.py -- Mailman-Users mailing list [EMAIL PROTECTED] http://mail.python.org/mailman/listinfo/mailman-users Mailman FAQ: http://www.python.org/cgi-bin/faqw-mm.py
