Re: Logging insights for Gump3 please
On 26-04-2005 02:10, Adam R. B. Jack [EMAIL PROTECTED] wrote: Change config.py; configure the logging package by hand (I think that's called basicConfig()) instead of using a config file, and make it output everything to the screen. We can fix it later, and there's no test to look for the existence of log files so you're not breaking anything ;) Ok, got it. This LogReporter isn't a reporter to a log it is a reporter of logs. Since my workspace/environment had no Ant script to run (and had done no CVS|SVN updates to get one), I had no properties ending with _log, and (apparently) no exceptions. As such, no reporter output data. D'oh! Only now do I get what you were on about. Sorry 'bout that. I'd half like to see a properties diff tool that runs in between plugins, to see what is changed each time a plugin runs. I could imaging plugins taking credit (say) for a property be updating a table. This is likely overkill for now though, and a dump (at the end) ought teach me what I need to know. This sounds like something to take care of within a Walker. You could perhaps write a custom Walker implementation (subclassing the original one) that pickles the entire model a whole lot and then keeps comparing the pickled stuff. It'd be sluggish as hell, but it probably isn't a whole lot of code :-) The other thing that pops into mind is overriding __setattr__ inside ModelObject to log all calls. Then there's the trace module (if its called that way) which is built precisely for this kind of stuff. Doing things properly probably involves that module. OTOH, debug statements have also worked reasonably for ages. Simplest thing, simplest thing ;) - LSD - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Last Gump runs were using libgcj
and thus failed. The problem must have been some apt-get installation that added a symlink to gij into /usr/bin/java. Obviously Gump still doesn't use $JAVA_HOME. I'll remove the symlinks for now. If you update/upgrade any packages on Brutus, please make sure there is no /usr/bin/java after that. I've been bitten by it at least twice myself. I also had to uninstall Kaffe at least twice (we build it from source, no need for apt-get) with no idea why it got reinstalled in between. Stefan - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
BATCH: All dressed up, with nowhere to go...
Dear Gumpmeisters, The following 9 notifys should have been sent *** G U M P [EMAIL PROTECTED]: Module castor success, but with warnings. [EMAIL PROTECTED]: Module icu4j failed [EMAIL PROTECTED]: Module xdoclet success, but with warnings. [EMAIL PROTECTED]: Module smartfrog success, but with warnings. [EMAIL PROTECTED]: Project nant (in module nant) failed [EMAIL PROTECTED]: Project xml-apis (in module xml-commons) failed [EMAIL PROTECTED]: Project xml-apis-12 (in module xml-apis-12) failed [EMAIL PROTECTED]: Project xml-xerces1 (in module xml-xerces) failed [EMAIL PROTECTED]: Project icu4j (in module icu4j) failed *** G U M P [EMAIL PROTECTED]: Module castor success, but with warnings. To whom it may engage... This is an automated request, but not an unsolicited one. For more information please visit http://gump.apache.org/nagged.html, and/or contact the folk at [EMAIL PROTECTED] Module castor contains errors. The current state of this module is 'Success'. Full details are available at: http://brutus.apache.org/gump/public/castor/index.html That said, some information snippets are provided here. The following annotations (debug/informational/warning/error messages) were provided: -ERROR- *** Failed to update from source control. Stale contents *** The following work was performed: http://brutus.apache.org/gump/public/castor/gump_work/update_castor.html Work Name: update_castor (Type: Update) Work ended in a state of : Failed Elapsed: 1 sec Command Line: cvs -q -z3 -d :pserver:[EMAIL PROTECTED]:/cvs/castor update -P -d -A [Working Directory: /usr/local/gump/public/workspace/cvs/castor] - Unknown host castor.exolab.org. - To subscribe to this information via syndicated feeds: - RSS: http://brutus.apache.org/gump/public/castor/rss.xml - Atom: http://brutus.apache.org/gump/public/castor/atom.xml == Gump Tracking Only === Produced by Gump version 2.2. Gump Run 3026042005, brutus:brutus-public:3026042005 Gump E-mail Identifier (unique within run) #1. *** G U M P [EMAIL PROTECTED]: Module icu4j failed To whom it may engage... This is an automated request, but not an unsolicited one. For more information please visit http://gump.apache.org/nagged.html, and/or contact the folk at [EMAIL PROTECTED] Module icu4j has an issue affecting its community integration, and has been outstanding for 35 runs. The current state of this module is 'Failed', with reason 'Update Failed'. Full details are available at: http://brutus.apache.org/gump/public/icu4j/index.html That said, some information snippets are provided here. The following annotations (debug/informational/warning/error messages) were provided: -INFO- Failed with reason update failed The following work was performed: http://brutus.apache.org/gump/public/icu4j/gump_work/update_icu4j.html Work Name: update_icu4j (Type: Update) Work ended in a state of : Failed Elapsed: 2 secs Command Line: cvs -q -z3 -d :ext:[EMAIL PROTECTED]:/icu checkout -P -d icu4j icu4j [Working Directory: /usr/local/gump/public/workspace/cvs] - IBM CANADA Ltd. CHS Server D25HTTP004 racky2u15 ** WARNING Access Restricted ** Permission denied (publickey,keyboard-interactive). cvs [checkout aborted]: end of file from server (consult above messages if any) - To subscribe to this information via syndicated feeds: - RSS: http://brutus.apache.org/gump/public/icu4j/rss.xml - Atom: http://brutus.apache.org/gump/public/icu4j/atom.xml == Gump Tracking Only === Produced by Gump version 2.2. Gump Run 3026042005, brutus:brutus-public:3026042005 Gump E-mail Identifier (unique within run) #2. *** G U M P [EMAIL PROTECTED]: Module xdoclet success, but with warnings. To whom it may engage... This is an automated request, but not an unsolicited one. For more information please visit http://gump.apache.org/nagged.html, and/or contact the folk at [EMAIL PROTECTED] Module xdoclet contains errors. The current state of this module is 'Success'. Full details are available at: http://brutus.apache.org/gump/public/xdoclet/index.html That said, some information snippets are provided here. The following annotations (debug/informational/warning/error messages) were provided: -ERROR- *** Failed to update from source control. Stale contents *** The following work was performed: http://brutus.apache.org/gump/public/xdoclet/gump_work/update_xdoclet.html Work Name: update_xdoclet (Type: Update) Work ended in a state of : Failed Elapsed: 3 mins 15 secs Command Line: cvs -q
[Gump Wiki] Update of VmgumpConfig by LeoSimons
Dear Wiki user,
You have subscribed to a wiki page or wiki category on Gump Wiki for change
notification.
The following page has been changed by LeoSimons:
http://wiki.apache.org/gump/VmgumpConfig
The comment on the change is:
some changes still needed
--
/workspace}}}
* sync over packages from {{{brutus.apache.org:/usr/local/gump/packages}}}
[shared, not under 'flavour'].
- * create/edit {{{/usr/local/gump/public/gump/local-env-py-vmgump.sh}}}:
+ * create/edit {{{/usr/local/gump/public/gump/cron/local-env-vmgump.sh}}}:
{{{
export JAVA_HOME=/opt/jdk1.4
export CLASSPATH=$JAVA_HOME/lib/tools.jar
@@ -126, +126 @@
* create/edit /home/gump/.bash_profile:
{{{
umask 002
- . /usr/local/gump/public/gump/local-env-py-vmgump.sh
+ . /usr/local/gump/public/gump/cron/local-env-vmgump.sh
}}}
* set up cron for user gump:
{{{
@@ -164, +164 @@
#Clean up after POI...
0 0 * * * /bin/rm -f /tmp/*.xls
}}}
- * copy the file {{{/etc/apache2/sites-available/default}}} into
{{{/etc/apache2/sites-available/[virtual.host]}}}
- * configure {{{/etc/apache2/sites-available/[virtual.host]}}} somewhat like
this:
+ * configure {{{/etc/apache2/sites-available/vmgump.apache.org}}} somewhat
like this:
{{{NameVirtualHost *
VirtualHost *
@@ -219, +218 @@
ProxyPassReverse /gump3/ http://localhost:8080/
/VirtualHost}}}
* {{{mkdir /var/www/vmgump.apache.org chown gump:gump
/var/www/gump.apache.org}}}
- * {{{a2ensite vmgump.apache.org a2enmod proxy}}}
+ * {{{a2ensite vmgump.apache.org a2enmod proxy a2dissite default}}}
* {{{/etc/init.d/apache2 reload}}}
=== gump3 setup ===
-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
Please remove /usr/bin/java
It seems there's a /usr/bin/java on brutus.apache.org right now. That's a problem since gump will use it instead of what it finds in JAVA_HOME/bin. Please get rid of it. Thanks, - some automated crontab-activated script living in /home/gump on brutus. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Last Gump runs were using libgcj
On Tue, 26 Apr 2005, Leo Simons [EMAIL PROTECTED] wrote: That should do it :-D. If this gets messed up again we'll start receiving 4 e-mails per hour. Good idea, thanks. Stefan - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
[RT] Gump run queue
This just popped into my head. Naïve line-based schedule file using simple
commands together with cron could be real nice.
Creating schedules
--
We should have something like
gump schedule-run --profile=public --official
gump schedule-run --profile=public
gump schedule-run --profile=kaffe
Which writes a file $GUMP_HOME/schedule looking like this
gump run --profile=public --official
gump run --profile=public
gump run --profile=kaffe
The code would be along the lines of
schedule_run()
{
local schedulefile=$GUMP_HOME/schedule
echo gump run $* $schedulefile
}
You would also want to schedule runs immediately
gump schedule-run-first --profile=custom --debug
Which would be something like
schedule_run_first()
{
local schedulefile=$GUMP_HOME/schedule
echo gump run $* $schedulefile.new
cat $schedulefile $schedulefile.new
rm $schedulefile
mv $schedulefile.new $schedulefile
}
And of course there's more options like
schedule_run_when_idle()
{
local pidfile=$GUMP_HOME/pygump/pygump.pid
if [[ ! -f $pidfile ]]; then
schedule_run
fi
}
The next bit is to have calls to those schedule-run commands in the
crontab:
# the next run after midnight will be the official one
0 0 * * * . /home/gump/.bash_profile; gump schedule-run-first \
--profile=public --official
0 6 * * * . /home/gump/.bash_profile; gump schedule-run \
--profile=kaffe
0 12 * * * . /home/gump/.bash_profile; gump schedule-run \
--profile=jdk15
0 3,9,15,18 * * * . /home/gump/.bash_profile; gump schedule-run-when-idle \
--profile=public
Keeping schedules
-
And then have something like
gump try-to-invoke-run
Which checks to see if gump is running, and if not, takes a line from
$GUMP_HOME/schedule and executes it. Along the lines of
try_to_invoke_run()
{
local pidfile=$GUMP_HOME/pygump/pygump.pid
if [[ ! -f $pidfile ]]; then
line=remove_first_line_from_schedule
`$line`
fi
}
And we run that real often from the crontab:
60/12 * * * * . /home/gump/.bash_profile; gump try-to-invoke-run
Why
---
This would ensure gump would be running just about continuously, but you
could also easily interrupt using
gump kill
gump schedule-run-first --profile=custom --debug
To manually change the schedule if you wanted to try something.
-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
Re: [RT] Gump run queue
On Tue, 26 Apr 2005, Leo Simons [EMAIL PROTECTED] wrote: This just popped into my head. Naïve line-based schedule file using simple commands together with cron could be real nice. +1 Although we may find that we need overlapping Gump runs at times. Stefan - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Last Gump runs were using libgcj
How about creating a read-only non-executable empty /usr/bin/java file? The warning test would need to be enhanced of course, but it might prevent some problems from occurring. Just a thought. S. On 4/26/05, Stefan Bodewig [EMAIL PROTECTED] wrote: On Tue, 26 Apr 2005, Leo Simons [EMAIL PROTECTED] wrote: That should do it :-D. If this gets messed up again we'll start receiving 4 e-mails per hour. Good idea, thanks. Stefan - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
[Gump Wiki] Update of VmgumpConfig by LeoSimons
Dear Wiki user,
You have subscribed to a wiki page or wiki category on Gump Wiki for change
notification.
The following page has been changed by LeoSimons:
http://wiki.apache.org/gump/VmgumpConfig
The comment on the change is:
mysql instructions
--
chown gump:gump /x1/gump
echo 'general@gump.apache.org' ~gump/.forward
echo '[EMAIL PROTECTED]' ~root/.forward
+ }}}
+
+ === Set up mysql ===
+
+ * Secure the root account
(http://dev.mysql.com/doc/mysql/en/default-privileges.html):
+
+ {{{
+ shell mysql -u root
+ mysql SET PASSWORD FOR 'root'@'localhost' = PASSWORD('newpwd');
+ mysql SET PASSWORD FOR 'root'@'vmgump' = PASSWORD('newpwd');
+ }}}
+
+ * Create a gump database and user
+
+ {{{
+ mysql create database gump_public;
+ mysql GRANT ALL PRIVILEGES ON gump_public.* to 'gump'@'localhost' identified
by 'passwd';
+ Query OK, 0 rows affected (0.00 sec)
+
+ mysql flush privileges;
+ Query OK, 0 rows affected (0.01 sec)
}}}
=== Other prereqs ===
@@ -107, +128 @@
profile href=profile/gump.xml/
- database database=gump_public passwd=password /
+ database database=gump_public user=gump passwd=password /
!-- additional background threads, over main thread --
threads updaters=5 builders=0 /
-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
Re: [RT] Gump3 design idea: workspace/ considered harmful
On Mon, 25 Apr 2005, Leo Simons [EMAIL PROTECTED] wrote:
In the next version of our in development xml model and in the
development of gump3, configuration information is left out and
instead provided on the commandline or using environment
variables.
+1 to separating environment configuration from the
Module/Project/Repository/Whatever stuff making up a Gump definition.
It may too much for env vars, though. I.e. we may end up having too
many environment variables or command line options.
Looking into my old workspace defintion I'd have basedir, pkgdir and
maybe a dir for checked out sources (if I want something other than
{basedir}/cvs). Then there is logdir.
There may be a dir for jars, javdocs or junitreports, if I want to
publish them. Nagging needs a mailserver and a sender address. The
database wants to get configured, I guess I could have a location
independent of logdir for RSS feeds. The current threads
configuration should be part of the environmental setting as well.
IMHO we still better maintain these settings in a file. Doesn't have
to be an XML file and should be separate from project definitions.
Stefan
-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
Has the Kaffe Gump run died?
I can't seem to find any sign of live of it on Brutus (the only active Gump run seems to be the JDK 1.5 one), gump.lock exists and out.txt ends with , | Failed to build project #[(189, 831)] : [asn1-codec], state:Failed | Build Project: #[(190, 831)] : asn1-der : [state:Unset] | Perform Update on #[(125, 243)] : db-commons-sandbox | Build Project: #[(191, 831)] : db-commons-grafolia : [state:Unset] | Build Project: #[(192, 831)] : ant : [state:Unset] | Run Ant on Project: #[(192, 831)] : ant | Perform Update on #[(126, 243)] : jakarta-oro | Perform SVN Update on #[(126, 243)] : jakarta-oro | Build Project: #[(193, 831)] : jakarta-oro : [state:Unset] | Run Ant on Project: #[(193, 831)] : jakarta-oro | Perform Update on #[(127, 243)] : avalon-tools | Perform SVN Update on #[(127, 243)] : avalon-tools | Build Project: #[(194, 831)] : magic : [state:Unset] | Run Ant on Project: #[(194, 831)] : magic | Perform Update on #[(128, 243)] : jakarta-commons | Perform SVN Update on #[(128, 243)] : jakarta-commons | Update(s) received via on #[(128, 243)] : jakarta-commons | Build Project: #[(195, 831)] : commons-net : [state:Unset] | Run Ant on Project: #[(195, 831)] : commons-net | Kill process group (anything launched by PID13325) ` asn1-codec is the last project visible in buildlog.html. out.txt hasn''t been written to since more than two hours now. Could it be that the new Kill prcess group stuff is killing too many processes, taking the main Gump process down with it? Stefan - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Logging insights for Gump3 please
OTOH, debug statements have also worked reasonably for ages. Simplest thing, simplest thing ;) Yup, I'm there for now, however I was thinking about communication between two separate developers of plugins, perhaps the latter not being able to hack debug statements into the former. Still, right now that isn't the case. Still, if plugin developers can log (perhaps at end of the piece of code) the main attributes they publish, that'd be helpful. Perhaps we somehow need to have plugins document themselves (just like you have the list of attributes on the model.) regards, Adam - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Has the Kaffe Gump run died?
Could it be that the new Kill prcess group stuff is killing too many
processes, taking the main Gump process down with it?
I don't think it is the new stuff, per-se, but the fact that some old
slipped in with the new. The old killed the Gump process 'cos it was
running within a standalone process, a copy ('cos M$ doesn't have fork).
Inside the new, I suspect that the fork still sees 'the Gump process' as the
original. Poorly coded, and when mixed with a coincident bug, perhaps fatal.
I would've expected to see a log warning line (that I don't see in your
output) but since this possibility exists, and fits the experience we have,
I'm inclined to blame it. I'll fix it and commit.
regards,
Adam
-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
Re: Has the Kaffe Gump run died?
On Tue, 26 Apr 2005, Adam R. B. Jack [EMAIL PROTECTED] wrote: I don't think it is the new stuff, per-se, but the fact that some old slipped in with the new. I was just throwing out random ideas ... I'll fix it and commit. Great. Please note that I haven't removed the lock file yet. I wanted to prevent a new Gump run from overwriting logs in case you needed more information than I had provided. Cheers - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Has the Kaffe Gump run died?
thanks adam. i was looking into the code myself when i saw ur email :)
will wait for ur patch (both trunk and live...right?)
-- dims
On 4/26/05, Adam R. B. Jack [EMAIL PROTECTED] wrote:
Could it be that the new Kill prcess group stuff is killing too many
processes, taking the main Gump process down with it?
I don't think it is the new stuff, per-se, but the fact that some old
slipped in with the new. The old killed the Gump process 'cos it was
running within a standalone process, a copy ('cos M$ doesn't have fork).
Inside the new, I suspect that the fork still sees 'the Gump process' as the
original. Poorly coded, and when mixed with a coincident bug, perhaps fatal.
I would've expected to see a log warning line (that I don't see in your
output) but since this possibility exists, and fits the experience we have,
I'm inclined to blame it. I'll fix it and commit.
regards,
Adam
-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
--
Davanum Srinivas - http://webservices.apache.org/~dims/
-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
Re: Last Gump runs were using libgcj
The problem must have been some apt-get installation that added a
symlink to gij into /usr/bin/java. Obviously Gump still doesn't use
$JAVA_HOME.
Much as my read of the code would hope to differ, my read of the log say
that Gump (despite efforts) isn't defaulting to $JAVA_HOME/bin/java, but
still using java. See Properties and Annotations here:
http://brutus.apache.org/gump/public/index.html
Ok, much as I hate fixing something I don't understand as broken, I've move
this code to be with the rest (that seems to work).
# Default to $JAVA_HOME/bin/java, can be overridden with
$JAVA_CMD.
if os.environ.has_key('JAVA_HOME'):
self.javaCommand =
os.path.join(os.environ['JAVA_HOME'],'bin',self.javaCommand)
self.addInfo('javaCommand set to $JAVA_HOME/bin/java = ' +
self.javaCommand )
I'll check out the output from the next new run.
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
- - - - - - - - - - - - - - - - - -
Gak, hold one .. commit to trunk, run from live ... I wonder if this was
working, we've just not 'release' recently. Hmm, I wonder if we ought be
running an 'svn info' as part of the start-up of the run, to see what we are
working with (and from where).
regards,
Adam
-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
Re: Has the Kaffe Gump run died?
thanks adam. i was looking into the code myself when i saw ur email :) will wait for ur patch (both trunk and live...right?) Nice to have company in there dims. :-) Let's do a test run, see if things are working (if we can be certain) and then do a release. http://wiki.apache.org/gump/GumpBranches regards, Adam - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Has the Kaffe Gump run died?
oops...i already merged your patch to live :( -- dims On 4/26/05, Adam R. B. Jack [EMAIL PROTECTED] wrote: thanks adam. i was looking into the code myself when i saw ur email :) will wait for ur patch (both trunk and live...right?) Nice to have company in there dims. :-) Let's do a test run, see if things are working (if we can be certain) and then do a release. http://wiki.apache.org/gump/GumpBranches regards, Adam -- Davanum Srinivas - http://webservices.apache.org/~dims/ - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Gump3:Dependency.dependencyInfo[]
I've been wondering whether its a smart idea to change a whole lot of this now. I think the Normalizer code and friends shows we can keep the model the same for now yet develop most of our code the way we see fit. The new model will fall out, and we'll immediately know what the transformation steps are: the Normalizer ends up being the conversion step between the old GOM and the new GOM. Does that make sense to you? It can wait. Of course, I'm now seeing that this is very bad communication-wise, since you'll have to understand all that ugly xml code in order to understand what the code does. Hmm. Maybe we should take it further. I'm thinking here that we should forget about the xml there and build the example model completely in code. Sounds good for a test model we can write unit tests against. regards Adam - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Please remove /usr/bin/java
It seems there's a /usr/bin/java on brutus.apache.org right now. That's a problem since gump will use it instead of what it finds in JAVA_HOME/bin. Please get rid of it. Thanks, - some automated crontab-activated script living in /home/gump on brutus. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: [RT] Gump run queue
Stefan Bodewig wrote: On Tue, 26 Apr 2005, Leo Simons [EMAIL PROTECTED] wrote: This just popped into my head. Naïve line-based schedule file using simple commands together with cron could be real nice. +1 +1 Although we may find that we need overlapping Gump runs at times. Stefan - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] -- Stefano. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: [RT] Gump3 design idea: workspace/ considered harmful
Stefan Bodewig wrote:
On Mon, 25 Apr 2005, Leo Simons [EMAIL PROTECTED] wrote:
In the next version of our in development xml model and in the
development of gump3, configuration information is left out and
instead provided on the commandline or using environment
variables.
+1 to separating environment configuration from the
Module/Project/Repository/Whatever stuff making up a Gump definition.
It may too much for env vars, though. I.e. we may end up having too
many environment variables or command line options.
Looking into my old workspace defintion I'd have basedir, pkgdir and
maybe a dir for checked out sources (if I want something other than
{basedir}/cvs). Then there is logdir.
There may be a dir for jars, javdocs or junitreports, if I want to
publish them. Nagging needs a mailserver and a sender address. The
database wants to get configured, I guess I could have a location
independent of logdir for RSS feeds. The current threads
configuration should be part of the environmental setting as well.
IMHO we still better maintain these settings in a file. Doesn't have
to be an XML file and should be separate from project definitions.
+1
--
Stefano.
-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
Re: Has the Kaffe Gump run died?
no luck :( kaffe run still dies. -- dims On 4/26/05, Adam R. B. Jack [EMAIL PROTECTED] wrote: oops...i already merged your patch to live :( I doubt they are harmful, I just wasn't 100% certain they were sure fixes. regards Adam -- Davanum Srinivas - http://webservices.apache.org/~dims/ - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Has the Kaffe Gump run died?
This looks 'interesting' :
Kill process group (anything launched by PID 21107)
Kill process group 18226 (anything launched by PID 21107) [from 21109]
for:
pgrpID=os.getpgid(pid)
log.warn('Kill process group %s (anything launched by PID %s) [from
%s]' \
% (pgrpID, pid, os.getpid()))
It is unlikely that 21109 is the 'parent' of 21107. (Even Brutus isn't that
busy :-) Hmm, maybe it is that the timer is running in a sub-process (some
Python impl detail?), and the main process was 18226. That might be
possible, seeing as the lock file contains 18231.
Mind you, this leads to ...
Looking at the code, I see no 'setpgid', which is disturbing (to say the
least). The logic of this kill means the forked child needs to place itself
into it's own process group. Without that, the parent (main Gump) is likely
a equal target for termination. I (now) suspect that is the problem. I
certainly makes sense w/ the output we see.
[BTW: Somehow when I ported from the test script, python/misc/pgrp.py I lost
this 'minor' detail. Hopefully with Gump3 we all have better luck w/
repeatably unit testing integration code.]
regards
Adam
- Original Message -
From: Davanum Srinivas [EMAIL PROTECTED]
To: Adam R. B. Jack [EMAIL PROTECTED]
Cc: Gump code and data general@gump.apache.org
Sent: Tuesday, April 26, 2005 1:56 PM
Subject: Re: Has the Kaffe Gump run died?
no luck :( kaffe run still dies.
-- dims
On 4/26/05, Adam R. B. Jack [EMAIL PROTECTED] wrote:
oops...i already merged your patch to live :(
I doubt they are harmful, I just wasn't 100% certain they were sure fixes.
regards
Adam
--
Davanum Srinivas - http://webservices.apache.org/~dims/
-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
Re: Has the Kaffe Gump run died?
On 26-04-2005 22:24, Adam R. B. Jack [EMAIL PROTECTED] wrote:
This looks 'interesting' :
Ooh, then I'd better take a look ;)
Kill process group (anything launched by PID 21107)
Kill process group 18226 (anything launched by PID 21107) [from 21109]
for:
pgrpID=os.getpgid(pid)
log.warn('Kill process group %s (anything launched by PID %s) [from
%s]' \
% (pgrpID, pid, os.getpid()))
It is unlikely that 21109 is the 'parent' of 21107. (Even Brutus isn't that
busy :-) Hmm, maybe it is that the timer is running in a sub-process (some
Python impl detail?),
That could be the case. The threading module builds on the thread module
which is in native code.
and the main process was 18226. That might be
possible, seeing as the lock file contains 18231.
Mind you, this leads to ...
Looking at the code, I see no 'setpgid', which is disturbing (to say the
least).
Well, there is a setpgrp. That's setpgid(0,0), or something.
The logic of this kill means the forked child needs to place itself
into it's own process group.
That can never work properly. A killpg() might lead to the forked child
being killed early on, meaning that the rest of the killpg() could be
aborted and none of the children is ever killed. I.e. The code that does the
actual killing cannot be executed from a process that is in the process
group that is being killed; it needs to remain active for however long it
takes the os to do all the killing.
Without that, the parent (main Gump) is likely
a equal target for termination.
You'll just need to create a process group in another way, making sure it
doesn't contain the child. In the Gump3 code I just create a long-running
process whose sole purpose is to keep a process group around.
An alternative is to put all direct children in a process group belonging to
the parent, and all their children in a process group assocated with the
process. This is what tools like ie bash do, but its way more bookkeeping
than we need to do.
I (now) suspect that is the problem. I
certainly makes sense w/ the output we see.
[BTW: Somehow when I ported from the test script, python/misc/pgrp.py I lost
this 'minor' detail. Hopefully with Gump3 we all have better luck w/
repeatably unit testing integration code.]
The code in Gump3 responsible for all this is much much simpler, because:
1) it uses the subprocess module which simplifies things like error code
handling
2) it uses a single global process group for all of gump instead of one
for each command
3) it doesn't use timeouts or any kind of multithreading but only kills
processes before system exit
4) it doesn't bother writing exec files
5) it does one thing only (we hope it does it well :-D)
It shouldn't be too hard to adapt that code into gump2. Just replace
executeIntoResult:
from gump.util.executor import Popen
def execute(cmd,tmp=dir.tmp):
res = command.CmdResult(cmd)
return executeIntoResult(cmd,res,tmp)
def executeIntoResult(cmd,result,tmp=dir.tmp):
execString = cmd.formatCommandLine()
p = Popen(command,stdout=PIPE,stderr=STDOUT)
output = p.communicate()[0]
result.exit_code = p.exitcode
outputFile = \
os.path.abspath(os.path.join(tmp,gumpSafeName(cmd.name)+'.txt'))
if os.path.exists(outputFile): os.remove(outputFile)
o = open(outputFile,'w')
o.write(output)
o.close()
And additionally reap children before system exit:
# rigorously clean up our child processes
try:
timeout = 300
try:
log.debug(Cleaning up child processes. This may take up to
a little over %s seconds. % (timeout+100))
except:
pass
from gump.util.executor import clean_up_processes
clean_up_processes(timeout)
except:
try:
log.exception(Error cleaning up child processes!)
except:
pass
And you can get rid of everything else in launcher.py, saving a few hundred
lines of complex stuff! I think. It's not tested. I don't fully understand
the code. I'm in the dark here. I'm no expert. It doesn't kill stuff on
windows. Try at your own risk. Don't blame me if it blows up. ***insert your
favorite disclaimer here***
G'night!
- LSD
-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
[Gump Wiki] Update of VmgumpConfig by LeoSimons
Dear Wiki user,
You have subscribed to a wiki page or wiki category on Gump Wiki for change
notification.
The following page has been changed by LeoSimons:
http://wiki.apache.org/gump/VmgumpConfig
The comment on the change is:
you need to create tables...
--
mysql flush privileges;
Query OK, 0 rows affected (0.01 sec)
+ }}}
+
+ * set up tables
+
+ {{{
+ cd /usr/local/gump/public/gump/mysql
+ mysql -u gump -p gump gump.sql
+ # enter password here...
}}}
=== Other prereqs ===
-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
Re: Has the Kaffe Gump run died?
Hey Leo, thanks for taking a peek @ this... Looking at the code, I see no 'setpgid', which is disturbing (to say the least). Well, there is a setpgrp. That's setpgid(0,0), or something. Sorry, typo, I meant : os.setpgrp(). The logic of this kill means the forked child needs to place itself into it's own process group. That can never work properly. No, let's be clear: 1) Main Gump Python process forks 2) Forked child (now) sets it's self as a process group leader [protecting main Gump], and goes off to run children. 3) Main Gump sets a timer (for 1 hour) to do the call to os.killpg(os.getpgid(forkedPid),signal.SIGKILL) IMHO this works. Sorry if I didn't explain it clearly. The code in Gump3 responsible for all this is much much simpler, because: 1) it uses the subprocess module which simplifies things like error code handling 2) it uses a single global process group for all of gump instead of one for each command 3) it doesn't use timeouts or any kind of multithreading but only kills processes before system exit 4) it doesn't bother writing exec files 5) it does one thing only (we hope it does it well :-D) The Gump3 start is nicer than Gump2's, I agree. Unfortunately I don't was to give up on Python 2.3 nor Microsoft (non-Cygwin), and the Process Group stuff is Posix (not even sure if it works on Cygwin). Hence those lines of ugly code are not yet removable. Further, we need to make Gump3 start to kill sub-processes after a timeout (say on a spinning Ant build), or at least (1) not hang on them indefinately (2) stop them burning CPU if spinning. Basically, I'm in a mindset of (1) keep Gump2 working generating build results (even if not pretty) (2) developing Gump3 much more prettily. As such, I hope to (one of these days) take the subprocess stuff and do effectively what Gump2 has per child, in Gump3, but do it in a clean/subprocess/no-extra-fluff way. Feel free to beat me to it, and/or review what I submit. BTW: I merged this patch into LIVE and have a Kaffe run going on Brutus. It isn't happy about bootstrap-ant, so maybe we'll not see a rogue compile/test/build causing a (stray) kill to know if it is working. Hopefully we'll get that soon. regards, Adam - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
