Author: coar
Date: Tue Mar 9 03:26:27 2010
New Revision: 920643
URL: http://svn.apache.org/viewvc?rev=920643&view=rev
Log:
Start documenting
Added:
labs/pulse/README.txt
Added: labs/pulse/README.txt
URL: http://svn.apache.org/viewvc/labs/pulse/README.txt?rev=920643&view=auto
==============================================================================
--- labs/pulse/README.txt (added)
+++ labs/pulse/README.txt Tue Mar 9 03:26:27 2010
@@ -0,0 +1,62 @@
+The Pulse lab project collects information from the Apache Software
+Foundation's mailing lists, such as the numer of subscribers, who the
+moderators are, and so on. The information is loaded into a MySQL
+database, and is used to generate reports and graphs for the Web site
+(http://Pulse.Apache.Org/). In addition, the information can be
+queried via email requests.
+
+The project is broken into several parts:
+
+o Data collection
+-----------------
+This occurs on the Mail.Apache.Org system, under the username
+'apmail'. The '~apmailbin/collect-listinfo.sh' script collects the
+info and records it in XML, and then sends it to
+'[email protected]' eddress. Qmail forwards it according to
+'~apmail/.qmail-collect-listinfo'. Currently that points to
+<[email protected]>.
+
+o Analysis
+----------
+On Pulse.Apache.Org, the eddress <pulse-load> feeds the
+'pulse/analysis/pulse-demail.pl' script, which strips the mail
+information from the XML data and stores the latter in the
+'pulse/data/queue/' directory.
+
+A cron job running as the 'pulsestat' user runs the
+'pulse/analysis/pulse-load.pl' script to load files from the queue
+into the database. After loading, the files are moved to the
+'pulse/data/processed/' directory.
+
+o Generation
+------------
+'Generation' only applies to the Web-based reports, and is centred on
+the 'pulse/web-reports/' directory. The key here is the 'mlists.php'
+script, which is run each day after all queued data files have been
+processed. Also in this directory are the files that should be copied
+to the Web document tree (such as the CSS files).
+
+o Report upload
+Uploading is performed by the 'mlists.php' script mentioned in the
+previous section. See the cron files for details.
+
+o Email query
+TBS
+
+Files
+=====
+ o SVN Repository:
+ https://svn.apache.org/repos/asf/labs/pulse/
+
+ o Working directory
+ /usr/local/labs/pulse
+ /analysis Scripts to unpack data and load them
+ into the DB
+ /collection Copies of scripts used on Mail.Apache.Org
+ /cron.d Cron jobs
+ /data
+ /queue XML files awaiting processing
+ /processed XML files after loading
+ /email-query Scripts used to answer email queries
+ /test Stuff for testing
+ /web-reports Scripts for generating statistics for Web
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
