Re: [RT] Gump Architecture
Stefano Mazzocchi wrote: Leo Simons wrote: (did I mention I'm an avalon guy? :D) did I mention that Avalon is dying out of flexibility cancer? nope, and you shouldn't over here, wrong list :D Dude, let's just fix things incrementally. well, "duh" :D. No one said anything about development process. Well, not in this thread, until now :D Lazyness is a virtue and Darwin is your man. Gump has no competition; darwin won't work! (...) Even if you're fixing things incrementally, I think you need to have a goal. You take small steps, but its a good idea to take them in a particular direction. Especially when working in teams. Otherwise everyone walks in different directions and you lose track of each other. In other words, incremental development doesn't mean "stop thinking about architecture". IMO. (...) Did you read what I wrote? Did you actually disagree with the four suggestions I made? Do you think they lead to flexibility cancer? Why? How? Is there no way to circumvent that? What's the alternatives then? Don't jump at the word "architecture" or at avalon but at the lame ideas I'm supposedly ranting on about...I think soundbites don't help us very much here! Let's revisit some of this...tell me what those bad ideas are, dude! > - Replace the batch concept with a command/action/event concept. This came up before as "get rid of the punchcard model and run things persistently". If you run things persistently you do need another abstraction. You can't run things if you have no abstraction. > - Replace the "giant tree transformations" with a "end-user view of > the tree". This came up about 2 hours later after I posted this message when /someone/ (ie, you :D) suggested to start running a dynamic forrest. Isn't that about using an end-user view of the tree instead of a giant transformation? Isn't it? > - Think hard about the best way to represent the graph. How can you disagree with that? Do you think we're doing a real good job at representing our gump graph? Do you think this is not an area where it's desireable to change things? > - Introduce node history as opposed to tree history. Here, I'm /really/ interested in your opinion (as in, even more). You seem to agree that batch transformation of our trees is a bad idea, so you probably agree we need an alternative here as well. Is this the right one? > - A question that might pop up: but who sends the commands? Right now, > this is an algorithm which sorts the nodes in the tree into a flat > list and calls them in order. That's how it should stay for a while, in other words, do it incrementally. -- cheers, - Leo Simons --- Weblog -- http://leosimons.com/ IoC Component Glue -- http://jicarilla.org/ Articles & Opinions -- http://articles.leosimons.com/ --- "We started off trying to set up a small anarchist community, but people wouldn't obey the rules." -- Alan Bennett - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Which gump on the new machines?
On Thu, 25 Mar 2004, Adam Jack <[EMAIL PROTECTED]> wrote: > Which? As in traditioanl verse Python? I think concensus has been > Python for a while. +1 Stefan - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
cvs commit: gump/python/gump/document xdoc.py
ajack 2004/03/25 21:30:57 Modified:python/gump/document xdoc.py Log: Add unlink() to try to clean up & avoid circular links. Revision ChangesPath 1.19 +18 -4 gump/python/gump/document/xdoc.py Index: xdoc.py === RCS file: /home/cvs/gump/python/gump/document/xdoc.py,v retrieving revision 1.18 retrieving revision 1.19 diff -u -r1.18 -r1.19 --- xdoc.py 12 Mar 2004 16:10:39 - 1.18 +++ xdoc.py 26 Mar 2004 05:30:57 - 1.19 @@ -234,6 +234,9 @@ self.middle() self.callEnd() +# Probably ought do this higher up +self.unlink() + def callStart(self,piece=None): if not piece: piece = self if hasattr(piece,'start') and callable(piece.start): @@ -276,6 +279,13 @@ def isEmptyOk(self): return self.emptyOk + +def unlink(self): +# Unlink subpieces... +for subpiece in self.subpieces: +subpiece.unlink() +# Unlink oneself +self.setOwner(None) class XDocSection(XDocPiece): def __init__(self,context,title): @@ -671,7 +681,11 @@ def middle(self): self.context.writeRaw(self.text) - + +def unlink(self): +XDocPiece.unlink(self) +self.text=None + # # Some raw xdocs (for when too lazy to create classes) # - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Gump Thrashing/Spinning...
Unfortunately we are stuck with a recent CVS HEAD, if not latest, due to some features in the skin. We can't go back to a release. FWIIW: The release we have has been working for the last month or so, something just changed and dorked it. regards Adam - Original Message - From: "Michael Davey" <[EMAIL PROTECTED]> To: "Gump code and data" <[EMAIL PROTECTED]> Sent: Thursday, March 25, 2004 4:01 PM Subject: Re: Gump Thrashing/Spinning... > Adam Jack wrote: > > > [snip] > > > >I think it is forrest that is where we are getting stuck right now. > >Now sure why, but it is locking up. So, if we want forest, we have to figure > >out how to get inside that problem. > > > > > Which version are you using? Probably coincidence, but I recently > stopped using > CVS HEAD because cocoon would hang during the Forrest build-site stage. > Version 0.51 of Forrest is quite happy building my site from exactly the > same > xdocs source. > > -- > Michael > > > - > To unsubscribe, e-mail: [EMAIL PROTECTED] > For additional commands, e-mail: [EMAIL PROTECTED] > - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
cvs commit: gump/python/gump/document forrest.py
ajack 2004/03/25 18:27:15 Modified:python/gump/document forrest.py Log: Comment recent additions out, in case they are the cause for forrest spinning... Revision ChangesPath 1.117 +16 -13gump/python/gump/document/forrest.py Index: forrest.py === RCS file: /home/cvs/gump/python/gump/document/forrest.py,v retrieving revision 1.116 retrieving revision 1.117 diff -u -r1.116 -r1.117 --- forrest.py25 Mar 2004 15:28:52 - 1.116 +++ forrest.py26 Mar 2004 02:27:15 - 1.117 @@ -1417,16 +1417,19 @@ # The 'cause' is something upstream. Possibly a project, # possibly a module (so determine paths to module projects). # -if project.cause and not project==project.cause: -if isinstance(project.cause, Project): -for path in project.getDependencyPaths(project.cause): -self.documentDependenciesPath(dependencySection, 'Root Cause Dependency Path', \ -path, 0, 1, project, gumpSet) -elif isinstance(project.cause, Module): -for causeProject in project.cause.getProjects(): -for path in project.getDependencyPaths(causeProject): -self.documentDependenciesPath(dependencySection, 'Root Cause Module Dependency Path',\ -path, 0, 1, project, gumpSet) + +# +#if project.cause and not project==project.cause: +#if isinstance(project.cause, Project): +#for path in project.getDependencyPaths(project.cause): +#self.documentDependenciesPath(dependencySection, 'Root Cause Dependency Path', \ +#path, 0, 1, project, gumpSet) +#elif isinstance(project.cause, Module): +#for causeProject in project.cause.getProjects(): +#for path in project.getDependencyPaths(causeProject): +#self.documentDependenciesPath(dependencySection, 'Root Cause Module Dependency Path', \ +#path, 0, 1, project, gumpSet) +# depens += self.documentDependenciesList(dependencySection, 'Project Dependencies', \ project.getDirectDependencies(), 0, 0, project, gumpSet) - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Gump Thrashing/Spinning...
Adam Jack wrote: [snip] I think it is forrest that is where we are getting stuck right now. Now sure why, but it is locking up. So, if we want forest, we have to figure out how to get inside that problem. Which version are you using? Probably coincidence, but I recently stopped using CVS HEAD because cocoon would hang during the Forrest build-site stage. Version 0.51 of Forrest is quite happy building my site from exactly the same xdocs source. -- Michael - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Generating HTML (was Gump Threashing/Spinning)
> > First, I like the dynamic 'tree of nodes' based approach to writing > > HTML/XML, rather than template > > I like merging the concepts. Once you've built the tree, flatten the > part of it that will make up the page, and feed that to the template > engine. Even if you don't use a template engine, seperate the flattening > and splitting from the transformation step. I am game to explore this. HTML or XML (xdoc) output ought be the same code, just with different templates, so we can have 'pure Python generating HTML' and/or 'forrest on xdocs' without us having to compromise, right? > > This is different from stuff like anakia, where the template walks the > tree directly (bad). I think this is the key. I'd really like to find a good way to go from a tree of objects (not DOM, our models and context, etc.) to variables for templates, or whatever. I don't want this to be kludgy, and feel that cheetah/Python likely have some sort of slick solution. BTW: I can see the need to use includes, and I can see 'if' directives, but maybe we'd still want to use bits of templates glued together to get re-use. I think it depends upon the tree to tempalte choice we make... > > Second, I didn't realize that Python DOM had nice serialization mechanism. > > Maybe I should've used that from the start. > > No idea what you're talking about :D The Python DOM tree will serialize to an XML stream (unlike old DOMs I'm used to). I wrote some stuff I should never have in Python. Hey, we are here to learn, right? > > Now Gump generates it's xdocs using an object tree structure. Watching the > > python memory grow from 20M (after loading all XML) to 136M (during > > generating these pages) it has some sort of leak (actual or effective) > > ouch! Maybe it would pay off to use pipelining (you know, SAX, stuff) > instead of DOM to generate the object tree. I wonder if it is some sort of circular dependency (amonst the objects) so when I destroy a tree (by pointing the variable to a new one) I wonder if it truely gets destroyed. I know the DOM has an unlink() method for some good reason, along these lines. There is so much thrown up into memory, more with translations to try to cope with character sets (and binary junk) and such. I no longer believe that any is being thrown away when I mean it to be... > > Fixing this area is clearly important, and I'd appreciate all insights... > > No insights here, just babbling along ;) Better than my just listening to the babbling in my head. ;-) regards Adam - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
About ForrestCocoon was: Gump Thrashing/Spinning...
Forrest uses cocoon (http://cocoon.apache.org/2.0/) Which is a servlet that does xml pipelines XML -> transform -> transform --> xml (or whatever) out. Along with caching all kinds of other cool stuff. That is what you get when you use the "forrest run" command Because many, many of our pages are never visited this might actually reduce the computational load. So for us that means we run tomcat with a forrest install and then just push our xdocs to the right location. When a users visits a page that the xml underneath has been updated the page will regenerate. R, Nick Adam Jack wrote: 1) use forrest as a dynamic application First, what do you mean by this, please? For those of us who don't know, could somebody elaborate? Second, I think it is forrest that is where we are getting stuck right now. Now sure why, but it is locking up. So, if we want forest, we have to figure out how to get inside that problem. regards Adam - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: [RT] Gump Architecture
Stefano Mazzocchi wrote: Leo Simons wrote: (did I mention I'm an avalon guy? :D) did I mention that Avalon is dying out of flexibility cancer? Yes Stafano ... I've been watching your predications. Stephen. -- || | Magic by Merlin| | Production by Avalon | || | http://avalon.apache.org/merlin| | http://dpml.net/merlin/distributions/latest| || - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Gump Thrashing/Spinning...
> > 1) use forrest as a dynamic application First, what do you mean by this, please? For those of us who don't know, could somebody elaborate? Second, I think it is forrest that is where we are getting stuck right now. Now sure why, but it is locking up. So, if we want forest, we have to figure out how to get inside that problem. regards Adam - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: [RT] Gump Architecture
Leo Simons wrote: (did I mention I'm an avalon guy? :D) did I mention that Avalon is dying out of flexibility cancer? Dude, let's just fix things incrementally. Lazyness is a virtue and Darwin is your man. -- Stefano. smime.p7s Description: S/MIME Cryptographic Signature
Re: Gump Thrashing/Spinning...
Stefano Mazzocchi wrote: Adam Jack wrote: Two possible solutions here: 1) use forrest as a dynamic application 2) have HTML generate by python I would go for 1) since it would keep us the ability to do dynamic stuff like metadata manipulation. I volunteer to setup 1) +1 for dynamic forrest. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Generating HTML (was Gump Threashing/Spinning)
Leo Simons wrote: Now Gump generates it's xdocs using an object tree structure. Watching the python memory grow from 20M (after loading all XML) to 136M (during generating these pages) it has some sort of leak (actual or effective) ouch! Maybe it would pay off to use pipelining (you know, SAX, stuff) instead of DOM to generate the object tree. you are generating a dom on the flight! ahhh please, no matter what you do with forrest, please use a template engine instead of a DOM. there is no need for a dom for gump. -- Stefano, thinking seriously that it's time to take a look at the internals of gumpy smime.p7s Description: S/MIME Cryptographic Signature
Re: [RT] Gump Architecture
Nick Chalko wrote: I had thought of using a event / black board system to support a distributed gump. Using something like javaspaces. [snip workflow] This is a radically change but much more scalable. Wow, what a [RT]! I think this is a fantastic idea. It might be worth taking a look at JXTA. They have a python binding project, but I don't know what kind of state it is in. You could always use JXTA-J as a temporary solution. I guess the main advantage of JXTA would be that all those private Gumps would be helping out the build, unless they opt out (where as right now, private installations have to opt-in by asking to be listed on the Gump pages). Regarding the workflow, I think the controller should also indicate the platform and java version to be used, so that individual peers could filter the build list and choose appropriate work units. I guess that the controller would be responsible for nagging in this scenario. I've got this vision of all these little gump tins around the world, each constantly fetching, building and submitting their work units, just like SETI-at-home! -- Michael - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Gump Thrashing/Spinning...
Adam Jack wrote: I've been trying to run some quick jobs (a check.py not an integrate.py) to try to get insight into this problem. I'm finding (at least on my machine) that forrest is growing to huge size, and getting bogged down in swapping. I can't say this with certainty, but I feel that both Python and Java degrade very poorly when swapping, not just because they do so much dynamic memory management. I suspect any background thread for garbage collection is getting starved. I don't know if this is true, or even if it matters, but we are getting to a point of resoruce shortage, to the point of outage. I wonder if Python (Gumpy) and Java (Forrest ) are fighting with each other for resources. Both are hogs, but I am not sure if being a memory hog is cause or effect. I don't know if growth is occuring 'cos we finally added the last straw to the camel's back, or if we have some wierd loop bug. I suspect forrest is getting stuck on a certain page, but I have no insight into what is causing this -- nor if the cause is somehow Gumpy. Doing a much smaller Gump run, hence smaller xdocs set, works w/o problem. When doing a test run here (without even doign the build parts) I see Gumpy (the python instance) growing to 136M! I have no clue as to why it would do that! I wish I could get inside the Python instance to understand where the memory is, or if it is being leaked. Maybe this is just crowding out forrest. Whatever the cause, I am really starting to get 'done' with forrest. I support it's use, I introduced it & have dealt with the issues and built workarounds from day one, but it is hard work w/ no fun. I feel it (through Gumpy trying to dynamically create xdocs) has been the weak/slow link in Gumpy for a while. I suspect even the forrest developers here might agree that I took this too far. Forrest is great for sites, but the gump outputs are getting huge and not really benefiting from Forrest skins, etc. Maybe I am wrong, maybe I need to approach the forrest developers (mail to their team) to see what they think on this. I wonder if we ought consider replacing Forrest with a pure Python HTML producer. As above, I can't prove that forrest is the problem, but a pure Python solution might just halve the unknowns. Two possible solutions here: 1) use forrest as a dynamic application 2) have HTML generate by python I would go for 1) since it would keep us the ability to do dynamic stuff like metadata manipulation. I volunteer to setup 1) -- Stefano. smime.p7s Description: S/MIME Cryptographic Signature
Re: [RT] Gump Architecture
Leo Simons wrote: As for the buzzwords...let's create an action-based, data-centric, graph-based, versioning-enabled, highly componentized, extensible, continous integration system. I have no idea how to do any of that in python, but it sounds like fun to find out.. I had thought of using a event / black board system to support a distributed gump. Using something like javaspaces. A controller adds projects to a queue With information like * project name * cvs info including timestamp to use * List of dependencies * ant file / target The controller then walks through the project queue to find request that have all dependencies resolved and moves them to the a buildable queue Builders then get projects from the queue, and post the results which are ant output and jar files. When the controller has no more buildable projects it then posts the rest as dependency failures. This is a radically change but much more scalable. R, Nick - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
[RT] Gump Architecture
Remember my jira entry screaming for architecture. Here's some thoughts. This is the big abstract braindump I hope some people will understand. Producing an RT a day...yep, I spend a lot of time thinking about this atm :D == Traditional Gump == Traditional gump has this architecture. === Basic Elements === * an object model (in java) * an object model representation (in documentation and xml) * a way to transform an object model expressed in xml into the java object model (in java, using DOM) * a way to transform that java object model back into a different kind of xml (using DOM) * a way to transform that xml into shell scripts (using XSL) that output html There's some additional glue for doing various things, in the form of shell scripts, web server configuration, etc etc. For example, shell scripts exist for calling parts of the generated shell scripts. === Sample use cases and their implementation === Build all java projects known to gump against each other === Download the gump module, create an XML profile, set some environment variables, install dependencies, copy a whole suite of installed packages from an existing gump installation, create a crontab. On invocation through the crontab, a script invokes a java application which creates a giant xml tree, then invokes a java applications which transforms this xml tree into giant script files, then runs these giant script files. On completion, some other script files are or can be run to do some other things (like send out failure notifications). === Fix a descriptor<->project build mismatch === After receiving a nag e-mail, click on the provided link and read a (usually very familiar-looking) build report and analyze the failure. After identifying the mismatch, locate the appropriate XML file in gump cvs, edit it by hand, run an ant script to verify syntax, commit. Wait for a day to see if things work. On invocation of the ant script, the ant script does the equivalent of what a DTD-aware editor would do, then builds the GOM and walks it to check for circularities. On invocation of the commit message, nothing happens, and the next batch run will hopefully yield better results. (Scripts are available to check things directly provided you have console access to the server, but only a handful of people (approximately 4 or 5 on the planet) know how to use them and have access to them. None of them have the time needed to do this for all the other people on the planet.) == Python Gump == === First iteration === Basically a lot of the same behaviour as with traditional gump, but java+xsl is replaced with python, and all the shell scripts stop working. Proof of concept. === Second iteration === A wxWindows-based GUI keeps a complex and confusing (to mere mortals) but extremely powerful object model in memory. The basic workflow actions and use cases for the java-based gump are ported to this GUI one by one. Most importantly, the functionality for immediate debugging that was once available in scripts is integrated. === Third iteration === The GUI-oriented code is slowly refactored back into a commandline application. Some "serious" plumbing, like logging, is added. An effort is made to move the use of basic system utilities into pure python. Python is put in control of a lot more things, shell scripts are replaced with python scripts piece by piece. Output is XML, which is transformed into HTML, RSS, and other stuff in a distinct step. A lot of the architectural overhaul done in the second iteration is lost in the light of getting the python version of gump feature-complete and backwards-compatible. It's pretty much batch-oriented again. === Fourth iteration? === It's time to introduce some architecture and some commmon idioms, because gump is starting to burst. A first step at responsibility-oriented system decomposition and seperation of concerns (did I mention I'm an avalon guy? :D) Disclaimer: I'm not very familiar with the internals of what I called the third iteration above. I plan to get into them over the next few weeks, but I wanted to write these ideas down before I lose 'em. - get rid of the batch-based system as a core idea. A batch is an ordered sequence of commands. While the challenge at one point was to get this ordering, it's gotten in the way of system growth and clarity know. Replace this batch concept with a command/action/event concept. Commands are sequential (we have an event bus here), need to be queued (real time won't work :D), and have the object tree as inputs and outputs. - Get rid of the "giant tree transformations". It has proven to scale only with difficulty. Gump is a set of loose project definitions which are glued together into an acyclic object graph. The process of serializing and deserializing that graph is complex, and putting it at the core of the architecture makes the architecture complex a
Re: Generating HTML (was Gump Threashing/Spinning)
Adam Jack wrote: We should probably use a template engine. I'm sure there's a python equivalent for something like velocity (or smarty). First, I like the dynamic 'tree of nodes' based approach to writing HTML/XML, rather than template I like merging the concepts. Once you've built the tree, flatten the part of it that will make up the page, and feed that to the template engine. Even if you don't use a template engine, seperate the flattening and splitting from the transformation step. This is different from stuff like anakia, where the template walks the tree directly (bad). -- in the main because of pleasant experiences with the Perl modules for this in the past. That said, even though there are some huge fields here (build output logs) hehehe. Look here: http://www.cheetahtemplate.org/docs/users_guide_html_multipage/language.constructs.html this is what things like #include are for! maybe a template mechanism is enough for the simple structure (results tables, lists, etc.) that we have. We have huge pages, with options missing (or not) based off content. I'm not sure any template could cope with that, but I am open minded to it. The biggest ISP in the Netherlands with the biggest content portal in the Netherlands uses a template engine as a core part of their view layer (but don't tell 'em ;) :D. At least they used to... Second, I didn't realize that Python DOM had nice serialization mechanism. Maybe I should've used that from the start. No idea what you're talking about :D Now Gump generates it's xdocs using an object tree structure. Watching the python memory grow from 20M (after loading all XML) to 136M (during generating these pages) it has some sort of leak (actual or effective) ouch! Maybe it would pay off to use pipelining (you know, SAX, stuff) instead of DOM to generate the object tree. Fixing this area is clearly important, and I'd appreciate all insights... No insights here, just babbling along ;) -- cheers, - Leo Simons --- Weblog -- http://leosimons.com/ IoC Component Glue -- http://jicarilla.org/ Articles & Opinions -- http://articles.leosimons.com/ --- "We started off trying to set up a small anarchist community, but people wouldn't obey the rules." -- Alan Bennett - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Generating HTML (was Gump Threashing/Spinning)
Perhaps we should run cocoon/fforrest live and just update the xdocs underneath? Adam Jack wrote: We should probably use a template engine. I'm sure there's a python equivalent for something like velocity (or smarty). First, I like the dynamic 'tree of nodes' based approach to writing HTML/XML, rather than template -- in the main because of pleasant experiences with the Perl modules for this in the past. That said, even though there are some huge fields here (build output logs) maybe a template mechanism is enough for the simple structure (results tables, lists, etc.) that we have. We have huge pages, with options missing (or not) based off content. I'm not sure any template could cope with that, but I am open minded to it. Second, I didn't realize that Python DOM had nice serialization mechanism. Maybe I should've used that from the start. Now Gump generates it's xdocs using an object tree structure. Watching the python memory grow from 20M (after loading all XML) to 136M (during generating these pages) it has some sort of leak (actual or effective): Fixing this area is clearly important, and I'd appreciate all insights... regards, Adam - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Generating HTML (was Gump Threashing/Spinning)
> We should probably use a template engine. I'm sure there's a python > equivalent for something like velocity (or smarty). First, I like the dynamic 'tree of nodes' based approach to writing HTML/XML, rather than template -- in the main because of pleasant experiences with the Perl modules for this in the past. That said, even though there are some huge fields here (build output logs) maybe a template mechanism is enough for the simple structure (results tables, lists, etc.) that we have. We have huge pages, with options missing (or not) based off content. I'm not sure any template could cope with that, but I am open minded to it. Second, I didn't realize that Python DOM had nice serialization mechanism. Maybe I should've used that from the start. Now Gump generates it's xdocs using an object tree structure. Watching the python memory grow from 20M (after loading all XML) to 136M (during generating these pages) it has some sort of leak (actual or effective): Fixing this area is clearly important, and I'd appreciate all insights... regards, Adam - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Gump Thrashing/Spinning...
I wonder if we ought consider replacing Forrest with a pure Python HTML producer. As above, I can't prove that forrest is the problem, but a pure Python solution might just halve the unknowns. I've been using Python to generate HTML from XML via XSL for a few weeks now using 4suite. I'm fairly new to Python so I wasn't really sure what my options were for XSL. It's been very good so far, speed wise. So, maybe it's an option. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: last night's build on lsd
Adam Jack wrote: I hate to say it, but sometime we really 'pay' (in terms of workarounds, outages, time delays) for the decisions to go with forrest. I don't know if we ought consider it 'yet another Gump stress test' and keep it, but I am starting to think more and more of writing HTML pages directly, via some Python code. I also thought forrest is a good product. But if it takes ages to generate the pages, we should move to something else. As you said, a pure python solution makes sense. Might be a lot of work though. Talking of stress, gump.dotnot is taking almost a day... http://gump.dotnot.org/#Details I'm thinking of giving Gump a shot on moof. Do you think this is a good idea, or just going to damage another server? It cannot damage the server. Go for it ! regards, Adam Antoine - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Gump Thrashing/Spinning...
Leo Simons wrote: We should probably use a template engine. I'm sure there's a python equivalent for something like velocity (or smarty). http://www.cheetahtemplate.org/ I use it in my weblogging software. - Sam Ruby - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Gump Thrashing/Spinning...
Adam Jack wrote: Whatever the cause, I am really starting to get 'done' with forrest. I support it's use, I introduced it & have dealt with the issues and built workarounds from day one, but it is hard work w/ no fun. I remember that feeling. I like the forrest project and I like the forrest devs a lot, but I've been very frustrated with the forrest software a few times. As to why python is a memory hog itself as well -- probably because variables don't go out of scope and because a lot of things (strings probably and object representations in the GOM) get copied around. I wonder if we ought consider replacing Forrest with a pure Python HTML producer. As above, I can't prove that forrest is the problem, but a pure Python solution might just halve the unknowns. Thoughts? It's a good plan, methinks. * I think some people would like gump a lot more if you could run it without needing java at all from a philosophical (free everything) view. * It's a lot simpler (we really don't need forrest's power). * It reduces the number of tools one has to be familiar with. We should probably use a template engine. I'm sure there's a python equivalent for something like velocity (or smarty). -- cheers, - Leo Simons --- Weblog -- http://leosimons.com/ IoC Component Glue -- http://jicarilla.org/ Articles & Opinions -- http://articles.leosimons.com/ --- "We started off trying to set up a small anarchist community, but people wouldn't obey the rules." -- Alan Bennett - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Gump Thrashing/Spinning...
I've been trying to run some quick jobs (a check.py not an integrate.py) to try to get insight into this problem. I'm finding (at least on my machine) that forrest is growing to huge size, and getting bogged down in swapping. I can't say this with certainty, but I feel that both Python and Java degrade very poorly when swapping, not just because they do so much dynamic memory management. I suspect any background thread for garbage collection is getting starved. I don't know if this is true, or even if it matters, but we are getting to a point of resoruce shortage, to the point of outage. I wonder if Python (Gumpy) and Java (Forrest ) are fighting with each other for resources. Both are hogs, but I am not sure if being a memory hog is cause or effect. I don't know if growth is occuring 'cos we finally added the last straw to the camel's back, or if we have some wierd loop bug. I suspect forrest is getting stuck on a certain page, but I have no insight into what is causing this -- nor if the cause is somehow Gumpy. Doing a much smaller Gump run, hence smaller xdocs set, works w/o problem. When doing a test run here (without even doign the build parts) I see Gumpy (the python instance) growing to 136M! I have no clue as to why it would do that! I wish I could get inside the Python instance to understand where the memory is, or if it is being leaked. Maybe this is just crowding out forrest. Whatever the cause, I am really starting to get 'done' with forrest. I support it's use, I introduced it & have dealt with the issues and built workarounds from day one, but it is hard work w/ no fun. I feel it (through Gumpy trying to dynamically create xdocs) has been the weak/slow link in Gumpy for a while. I suspect even the forrest developers here might agree that I took this too far. Forrest is great for sites, but the gump outputs are getting huge and not really benefiting from Forrest skins, etc. Maybe I am wrong, maybe I need to approach the forrest developers (mail to their team) to see what they think on this. I wonder if we ought consider replacing Forrest with a pure Python HTML producer. As above, I can't prove that forrest is the problem, but a pure Python solution might just halve the unknowns. Thoughts? regards, Adama - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Which gump on the new machines?
Adam Jack wrote: OK, it looks like there may be a new ASF hosted machine or two running gump in the next 24 or so hours. Which gump should be run on it? Which? As in traditioanl verse Python? I think concensus has been Python for a while. Just checking... We have a README in /usr/local/gump on moof, ought I post it here for folks? I can't get there this mo, and my eyebrowse search isn't bringing up when I posted it before. It has the steps involved. I see it on moof. I'm not too worried as I generally am a quick learner. ;-) I'm willing to take the lead in getting it up and running, or simply suggest it be turned over to somebody who wishes to volunteer. Once it is up and running, it is my intent that this be a public resource Gump participants. I will be online (from a hotel, after workout/dinner) tonight (PST time) and able to help. I do think we ought use the shared 'gump' account we've been discussing, that'd be good for a common place for the cronjob, etc. My guess is that real work won't start until tomorrow. A shared gump is OK with me. regards, Adam - Sam Ruby - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
cvs commit: gump/project antworks-importer.xml
nickchalko2004/03/25 11:15:04 Modified:project antworks-importer.xml Log: Formated Revision ChangesPath 1.6 +12 -26gump/project/antworks-importer.xml Index: antworks-importer.xml === RCS file: /home/cvs/gump/project/antworks-importer.xml,v retrieving revision 1.5 retrieving revision 1.6 diff -u -r1.5 -r1.6 --- antworks-importer.xml 22 Mar 2004 10:31:29 - 1.5 +++ antworks-importer.xml 25 Mar 2004 19:15:04 - 1.6 @@ -15,30 +15,16 @@ limitations under the License. --> - - http://antworks.sourceforge.xml/importer"/> - -Autodownload and import ant build.xml's. - - - - - - - -org.krysalis.antworks.importer - - - - - - - - - - - - - +http://antworks.sourceforge.xml/importer"/> + Autodownload and import ant build.xml's. + + +org.krysalis.antworks.importer + + + + + + + - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
cvs commit: gump/project antworks-antlets.xml
nickchalko2004/03/25 11:14:42 Modified:project antworks-antlets.xml Log: Formated Revision ChangesPath 1.2 +10 -27gump/project/antworks-antlets.xml Index: antworks-antlets.xml === RCS file: /home/cvs/gump/project/antworks-antlets.xml,v retrieving revision 1.1 retrieving revision 1.2 diff -u -r1.1 -r1.2 --- antworks-antlets.xml 25 Mar 2004 19:13:45 - 1.1 +++ antworks-antlets.xml 25 Mar 2004 19:14:42 - 1.2 @@ -15,31 +15,14 @@ limitations under the License. --> - - http://antworks.sourceforge.xml/antlets"/> - - Generate and publish all the antlets - - - - - - - -org.krysalis.antworks.antlets - - - - - - - - - - - - - - +http://antworks.sourceforge.xml/antlets"/> + Generate and publish all the antlets + + +org.krysalis.antworks.antlets + + + + + - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
cvs commit: gump/project antworks-antlets.xml
nickchalko2004/03/25 11:13:45 Added: project antworks-antlets.xml Log: Added antlets. PR: Obtained from: Submitted by: Reviewed by: CVS: -- CVS: PR: CVS: If this change addresses a PR in the problem report tracking CVS: database, then enter the PR number(s) here. CVS: Obtained from: CVS: If this change has been taken from another system, such as NCSA, CVS: then name the system in this line, otherwise delete it. CVS: Submitted by: CVS: If this code has been contributed to Apache by someone else; i.e., CVS: they sent us a patch or a new module, then include their name/email CVS: address here. If this is your work then delete this line. CVS: Reviewed by: CVS: If we are doing pre-commit code reviews and someone else has CVS: reviewed your changes, include their name(s) here. CVS: If you have not had it reviewed then delete this line. Revision ChangesPath 1.1 gump/project/antworks-antlets.xml Index: antworks-antlets.xml === http://antworks.sourceforge.xml/antlets"/> Generate and publish all the antlets org.krysalis.antworks.antlets - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Which gump on the new machines?
> > We have a README in /usr/local/gump on moof, ought I post it here for folks? > I can't get there this mo, and my eyebrowse search isn't bringing up when I > posted it before. It has the steps involved. > A google search found me this: http://www.mail-archive.com/[EMAIL PROTECTED]/msg00487.html regards Adam - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Which gump on the new machines?
> OK, it looks like there may be a new ASF hosted machine or two running > gump in the next 24 or so hours. Which gump should be run on it? Which? As in traditioanl verse Python? I think concensus has been Python for a while. We have a README in /usr/local/gump on moof, ought I post it here for folks? I can't get there this mo, and my eyebrowse search isn't bringing up when I posted it before. It has the steps involved. > I'm willing to take the lead in getting it up and running, or simply > suggest it be turned over to somebody who wishes to volunteer. Once it > is up and running, it is my intent that this be a public resource Gump > participants. I will be online (from a hotel, after workout/dinner) tonight (PST time) and able to help. I do think we ought use the shared 'gump' account we've been discussing, that'd be good for a common place for the cronjob, etc. regards, Adam - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: nagoya much slower again...
Noel J. Bergman wrote: Who's Gump in whose account? Sam's? Is anybody aware of the outputs, let alone using them? Sam, and ask him. :-) The outputs (in terms of reports indicating what worked and what did not) are directly linked to from gump.apache.org. Click on "Nagoya". In any case, I've commented this entry out of my crontab for now. Hopefully we will be seeing a new installation of Gump on separate hardware shortly. - Sam Ruby - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Which gump on the new machines?
OK, it looks like there may be a new ASF hosted machine or two running gump in the next 24 or so hours. Which gump should be run on it? I'm willing to take the lead in getting it up and running, or simply suggest it be turned over to somebody who wishes to volunteer. Once it is up and running, it is my intent that this be a public resource Gump participants. - Sam Ruby - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: nagoya much slower again...
> Who's Gump in whose account? Sam's? Is anybody aware of the outputs, let > alone using them? Sam, and ask him. :-) --- Noel - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: nagoya much slower again...
Who's Gump in whose account? Sam's? Is anybody aware of the outputs, let alone using them? I don't think folks have had access to the outputs (that they've known about) in forever. http://gump.apache.org/#Where+is+Gump%3F regards Adam - Original Message - From: "Noel J. Bergman" <[EMAIL PROTECTED]> To: "Leo Simons" <[EMAIL PROTECTED]>; "Apache Infrastructure" <[EMAIL PROTECTED]> Sent: Thursday, March 25, 2004 10:12 AM Subject: RE: nagoya much slower again... > > a while ago someone changed something on nagoya (allocating more memory > > to tomcat or something) that made eyebrowse, jira, etc, much more > > snappy. That was nice. The effect seems to have gone away again. Any > > chance of making those apps more responsive? > > There are two problems. One is that GUMP runs for 6-10 hours every day, and > there is a period during that time when GUMP drags nagoya to its knees. The > other problem that I occasionally see is bugzilla's createattachment.cgi > script going into an infinite spin loop, completely chewing up an entire CPU > until I kill it. > > Bugzilla sucks, and I would love to see it die with nagoya, but I don't > think that is likely to happen. :-( When we get the new issues server > running, we will if there is someone willing to install a current version of > bugzilla. I seem to recall that the SpamAssassin folks were willing, and > I'm sure that we can control the access rights to allow that to happen. > > --- Noel > - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
[jira] Assigned: (GUMP-25) document actual gump installations
Message: The following issue has been re-assigned. Assignee: (mailto:) - View the issue: http://issues.apache.org/jira/secure/ViewIssue.jspa?key=GUMP-25 Here is an overview of the issue: - Key: GUMP-25 Summary: document actual gump installations Type: New Feature Status: Unassigned Priority: Minor Project: Gump Components: Python Fix Fors: unspecified Versions: unspecified Assignee: Reporter: Leo Simons Created: Sun, 14 Mar 2004 11:49 AM Updated: Thu, 25 Mar 2004 8:50 AM Description: Add some docs on the wiki about the actual gump setups on various machines, how they were set up, and how they are being maintained. - JIRA INFORMATION: This message is automatically generated by JIRA. If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa If you want more information on JIRA, or have a bug to report see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
[jira] Commented: (GUMP-25) document actual gump installations
The following comment has been added to this issue: Author: Leo Simons Created: Thu, 25 Mar 2004 8:50 AM Body: http://wiki.apache.org/gump/GumpInfrastructure - View this comment: http://issues.apache.org/jira/secure/ViewIssue.jspa?key=GUMP-25&page=comments#action_27809 - View the issue: http://issues.apache.org/jira/secure/ViewIssue.jspa?key=GUMP-25 Here is an overview of the issue: - Key: GUMP-25 Summary: document actual gump installations Type: New Feature Status: Open Priority: Minor Project: Gump Components: Python Fix Fors: unspecified Versions: unspecified Assignee: Leo Simons Reporter: Leo Simons Created: Sun, 14 Mar 2004 11:49 AM Updated: Thu, 25 Mar 2004 8:50 AM Description: Add some docs on the wiki about the actual gump setups on various machines, how they were set up, and how they are being maintained. - JIRA INFORMATION: This message is automatically generated by JIRA. If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa If you want more information on JIRA, or have a bug to report see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
[jira] Work Stopped: (GUMP-25) document actual gump installations
Message: Work on this issue has been stopped by Leo Simons (mailto:[EMAIL PROTECTED]) - View the issue: http://issues.apache.org/jira/secure/ViewIssue.jspa?key=GUMP-25 Here is an overview of the issue: - Key: GUMP-25 Summary: document actual gump installations Type: New Feature Status: Open Priority: Minor Project: Gump Components: Python Fix Fors: unspecified Versions: unspecified Assignee: Leo Simons Reporter: Leo Simons Created: Sun, 14 Mar 2004 11:49 AM Updated: Thu, 25 Mar 2004 8:50 AM Description: Add some docs on the wiki about the actual gump setups on various machines, how they were set up, and how they are being maintained. - JIRA INFORMATION: This message is automatically generated by JIRA. If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa If you want more information on JIRA, or have a bug to report see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
[Gump Wiki] New: GumpInfrastructure
Date: 2004-03-25T08:49:28 Editor: 130.89.169.128 <> Wiki: Gump Wiki Page: GumpInfrastructure URL: http://wiki.apache.org/gump/GumpInfrastructure no comment New Page: == Gump on lsd.student.utwente.nl == [http://lsd.student.utwente.nl/ lsd] is currently the main installation for gump, though that will change real soon now. It's a duron machine running a near-full install of Fedora Core 1 and some optional packages from freshrpms, atrpms, jpackage, sun, nvidia, etc, which are supposed to be kept in the /data/packages directory. About 30GB of disk is allocated to gump. [http://lsd.student.utwente.nl/ Results] are published every night (run starts at 1:00AM GMT, usually finished at about 10AM GMT). Leo Simons is the (only) root on the machine, with several people in a 'gump' group that log in via ssh to help admin the gump installation. Gump (v 2.0) lives in /data3/gump, with results published to /data3/gump/log, and the actual gump install (home of the metadata and the scripts and stuff) in /data3/gump/gump-install. The crontab runs in the user id of ajack, the profile is the lsd.xml file in the gump cvs module, with /data3/gump/gump-install/local-env-py.sh containing local settings (like JAVA_HOME), as is the case for any gump 2 install. If you have a good reason to want shell access to the machine, get in contact with Leo through the gump mailing list. If he knows who you are, chances are you'll be served :D - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
[Gump Wiki] Updated: FrontPage
Date: 2004-03-25T08:41:05 Editor: 130.89.169.128 <> Wiki: Gump Wiki Page: FrontPage URL: http://wiki.apache.org/gump/FrontPage no comment Change Log: -- @@ -26,6 +26,9 @@ '''GumpCommandLineOptions''' Some more notes about those commandline scripts. + '''GumpInfrastructure''' + Some notes on the machines that run gump. Please add your machine here if it runs gump, especially if the results are published publicly. + = Design Topics = * HistoricalResultsDatabase - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
[jira] Closed: (GUMP-24) Make gump build directories accessible through web browser
Message: The following issue has been closed. Resolver: Leo Simons Date: Thu, 25 Mar 2004 8:39 AM after some more talk on the mailing list, it came up that this is distribution of software, often without paying due respect to license requirements for distribution. So that's a no-can do. - View the issue: http://issues.apache.org/jira/secure/ViewIssue.jspa?key=GUMP-24 Here is an overview of the issue: - Key: GUMP-24 Summary: Make gump build directories accessible through web browser Type: Task Status: Closed Priority: Minor Resolution: WON'T FIX Project: Gump Components: Python Fix Fors: unspecified Versions: unspecified Assignee: Leo Simons Reporter: Leo Simons Created: Sun, 14 Mar 2004 11:46 AM Updated: Thu, 25 Mar 2004 8:39 AM Environment: lsd.student.utwente.nl, moof.apache.org Description: Vincent Massol requested we make the full gump outputs accessible through the web browser. Need to think a little about security and then JFDI. It'll help people debug things. (Also update the relevant docs to point to this feature). - JIRA INFORMATION: This message is automatically generated by JIRA. If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa If you want more information on JIRA, or have a bug to report see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
[jira] Work Started: (GUMP-25) document actual gump installations
Message: Work on this issue has been started by Leo Simons (mailto:[EMAIL PROTECTED]) - View the issue: http://issues.apache.org/jira/secure/ViewIssue.jspa?key=GUMP-25 Here is an overview of the issue: - Key: GUMP-25 Summary: document actual gump installations Type: New Feature Status: In Progress Priority: Minor Project: Gump Components: Python Fix Fors: unspecified Versions: unspecified Assignee: Leo Simons Reporter: Leo Simons Created: Sun, 14 Mar 2004 11:49 AM Updated: Thu, 25 Mar 2004 8:40 AM Description: Add some docs on the wiki about the actual gump setups on various machines, how they were set up, and how they are being maintained. - JIRA INFORMATION: This message is automatically generated by JIRA. If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa If you want more information on JIRA, or have a bug to report see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
[jira] Work Stopped: (GUMP-34) Upgrade JDK on lsd to java 1.4.2_04
Message: Work on this issue has been stopped by Leo Simons (mailto:[EMAIL PROTECTED]) - View the issue: http://issues.apache.org/jira/secure/ViewIssue.jspa?key=GUMP-34 Here is an overview of the issue: - Key: GUMP-34 Summary: Upgrade JDK on lsd to java 1.4.2_04 Type: Task Status: Open Priority: Minor Project: Gump Assignee: Leo Simons Reporter: Antoine Levy-Lambert Created: Mon, 15 Mar 2004 11:09 PM Updated: Thu, 25 Mar 2004 8:38 AM Environment: Gump [EMAIL PROTECTED] Description: The build of xml-xerces1 is currently failing due to Unicode escapes present in javadoc comments. I am quoting Michael Glavassevich : http://article.gmane.org/gmane.comp.jakarta.gump/4990 I understand that this was a compiler bug [1] which was recently fixed in JDK 1.4.2_04 [2]. Whomever's responsible for the environment on lsd may want to upgrade the JDK. [1] http://developer.java.sun.com/developer/bugParade/bugs/4863451.html [2] http://java.sun.com/j2se/1.4.2/ReleaseNotes.html I have checked that JDK 1.4.2_04 fixes the problem. So we should go for it. - JIRA INFORMATION: This message is automatically generated by JIRA. If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa If you want more information on JIRA, or have a bug to report see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
[jira] Work Started: (GUMP-34) Upgrade JDK on lsd to java 1.4.2_04
Message: Work on this issue has been started by Leo Simons (mailto:[EMAIL PROTECTED]) - View the issue: http://issues.apache.org/jira/secure/ViewIssue.jspa?key=GUMP-34 Here is an overview of the issue: - Key: GUMP-34 Summary: Upgrade JDK on lsd to java 1.4.2_04 Type: Task Status: In Progress Priority: Minor Project: Gump Assignee: Leo Simons Reporter: Antoine Levy-Lambert Created: Mon, 15 Mar 2004 11:09 PM Updated: Thu, 25 Mar 2004 8:38 AM Environment: Gump [EMAIL PROTECTED] Description: The build of xml-xerces1 is currently failing due to Unicode escapes present in javadoc comments. I am quoting Michael Glavassevich : http://article.gmane.org/gmane.comp.jakarta.gump/4990 I understand that this was a compiler bug [1] which was recently fixed in JDK 1.4.2_04 [2]. Whomever's responsible for the environment on lsd may want to upgrade the JDK. [1] http://developer.java.sun.com/developer/bugParade/bugs/4863451.html [2] http://java.sun.com/j2se/1.4.2/ReleaseNotes.html I have checked that JDK 1.4.2_04 fixes the problem. So we should go for it. - JIRA INFORMATION: This message is automatically generated by JIRA. If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa If you want more information on JIRA, or have a bug to report see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
[jira] Commented: (GUMP-34) Upgrade JDK on lsd to java 1.4.2_04
The following comment has been added to this issue: Author: Leo Simons Created: Thu, 25 Mar 2004 8:37 AM Body: installed. JAVA_HOME changed to /usr/java/j2sdk1.4.2_03 I updated /data3/gump/gump-install/local-env-py.sh but I'm not sure that's the right file to update. I hope so. If so, please close this issue. - View this comment: http://issues.apache.org/jira/secure/ViewIssue.jspa?key=GUMP-34&page=comments#action_27807 - View the issue: http://issues.apache.org/jira/secure/ViewIssue.jspa?key=GUMP-34 Here is an overview of the issue: - Key: GUMP-34 Summary: Upgrade JDK on lsd to java 1.4.2_04 Type: Task Status: Open Priority: Minor Project: Gump Assignee: Leo Simons Reporter: Antoine Levy-Lambert Created: Mon, 15 Mar 2004 11:09 PM Updated: Thu, 25 Mar 2004 8:37 AM Environment: Gump [EMAIL PROTECTED] Description: The build of xml-xerces1 is currently failing due to Unicode escapes present in javadoc comments. I am quoting Michael Glavassevich : http://article.gmane.org/gmane.comp.jakarta.gump/4990 I understand that this was a compiler bug [1] which was recently fixed in JDK 1.4.2_04 [2]. Whomever's responsible for the environment on lsd may want to upgrade the JDK. [1] http://developer.java.sun.com/developer/bugParade/bugs/4863451.html [2] http://java.sun.com/j2se/1.4.2/ReleaseNotes.html I have checked that JDK 1.4.2_04 fixes the problem. So we should go for it. - JIRA INFORMATION: This message is automatically generated by JIRA. If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa If you want more information on JIRA, or have a bug to report see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
[jira] Commented: (GUMP-34) Upgrade JDK on lsd to java 1.4.2_04
The following comment has been added to this issue: Author: Leo Simons Created: Thu, 25 Mar 2004 8:32 AM Body: the release notes page says the following: --- Documentation Java[tm] 2 SDK, Standard Edition Version 1.4.2_03 (Microsoft Windows, Linux, and Solaris Operating Envirnoment) Release Notes JavaTM 2 SDK, Standard Edition Version 1.4.2_04 (Microsoft Windows, Linux, and Solaris Operating Environment) --- weird. Anyway, installing right now :D - View this comment: http://issues.apache.org/jira/secure/ViewIssue.jspa?key=GUMP-34&page=comments#action_27806 - View the issue: http://issues.apache.org/jira/secure/ViewIssue.jspa?key=GUMP-34 Here is an overview of the issue: - Key: GUMP-34 Summary: Upgrade JDK on lsd to java 1.4.2_04 Type: Task Status: Open Priority: Minor Project: Gump Assignee: Leo Simons Reporter: Antoine Levy-Lambert Created: Mon, 15 Mar 2004 11:09 PM Updated: Thu, 25 Mar 2004 8:32 AM Environment: Gump [EMAIL PROTECTED] Description: The build of xml-xerces1 is currently failing due to Unicode escapes present in javadoc comments. I am quoting Michael Glavassevich : http://article.gmane.org/gmane.comp.jakarta.gump/4990 I understand that this was a compiler bug [1] which was recently fixed in JDK 1.4.2_04 [2]. Whomever's responsible for the environment on lsd may want to upgrade the JDK. [1] http://developer.java.sun.com/developer/bugParade/bugs/4863451.html [2] http://java.sun.com/j2se/1.4.2/ReleaseNotes.html I have checked that JDK 1.4.2_04 fixes the problem. So we should go for it. - JIRA INFORMATION: This message is automatically generated by JIRA. If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa If you want more information on JIRA, or have a bug to report see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: last night's build on lsd
I just rebooted (remotely via ssh, took ages). Things seem alright now. Poor machine :D -- cheers, - Leo Simons --- Weblog -- http://leosimons.com/ IoC Component Glue -- http://jicarilla.org/ Articles & Opinions -- http://articles.leosimons.com/ --- "We started off trying to set up a small anarchist community, but people wouldn't obey the rules." -- Alan Bennett - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
cvs commit: gump/template/forrest/src/documentation/content/xdocs site.xml
ajack 2004/03/25 07:28:52
Modified:python/gump/document forrest.py
template/forrest/src/documentation/content/xdocs site.xml
Log:
Remove 'notesLog' page, in case it is causing Forrest to spin/grow.
Revision ChangesPath
1.116 +42 -42gump/python/gump/document/forrest.py
Index: forrest.py
===
RCS file: /home/cvs/gump/python/gump/document/forrest.py,v
retrieving revision 1.115
retrieving revision 1.116
diff -u -r1.115 -r1.116
--- forrest.py24 Mar 2004 15:51:42 - 1.115
+++ forrest.py25 Mar 2004 15:28:52 - 1.116
@@ -479,45 +479,45 @@
#
# notesLog.xml -- Notes log
#
-document=XDocDocument('Annotations', \
-self.resolver.getFile(workspace,'notesLog'))
-self.documentSummary(document, workspace.getProjectSummary())
-
-notesSection=document.createSection('Negative Annotations')
-notesSection.createParagraph(
-"""Entities with errors and warnings.""")
-
-ncount=0
-for module in gumpSet.getModuleSequence():
-if not gumpSet.inModuleSequence(module): continue
-
-moduleSection=document.createSection('Module : ' + module.getName())
-
-# Link to the module
-self.insertLink(module,workspace,moduleSection.createParagraph())
-
-if not module.containsNasties():
-
-# Display the annotations
-self.documentAnnotations(moduleSection,project,1)
-
-for project in module.getProjects():
-if not gumpSet.inProjectSequence(project): continue
-if not project.containsNasties(): continue
-
-projectSection=moduleSection.createSection('Project : ' +
project.getName())
-
-# Link to the project
-self.insertLink(project,workspace,projectSection.createParagraph())
-
-# Display the annotations
-self.documentAnnotations(projectSection,project,1)
-
-ncount+=1
-
-if not ncount: notesTable.createLine('None')
-
-document.serialize()
+#document=XDocDocument('Annotations',\
+#self.resolver.getFile(workspace,'notesLog'))
+#self.documentSummary(document, workspace.getProjectSummary())
+#
+#notesSection=document.createSection('Negative Annotations')
+#notesSection.createParagraph(
+#"""Entities with errors and warnings.""")
+#
+#ncount=0
+#for module in gumpSet.getModuleSequence():
+#if not gumpSet.inModuleSequence(module): continue
+#
+#moduleSection=document.createSection('Module : ' + module.getName())
+#
+## Link to the module
+#self.insertLink(module,workspace,moduleSection.createParagraph())
+#
+#if not module.containsNasties():
+#
+## Display the annotations
+#self.documentAnnotations(moduleSection,project,1)
+#
+#for project in module.getProjects():
+#if not gumpSet.inProjectSequence(project): continue
+#if not project.containsNasties(): continue
+#
+#projectSection=moduleSection.createSection('Project : ' +
project.getName())
+#
+## Link to the project
+#
self.insertLink(project,workspace,projectSection.createParagraph())
+#
+## Display the annotations
+#self.documentAnnotations(projectSection,project,1)
+#
+#ncount+=1
+#
+#if not ncount: notesTable.createLine('None')
+#
+#document.serialize()
#
# --
1.23 +1 -1 gump/template/forrest/src/documentation/content/xdocs/site.xml
Index: site.xml
===
RCS file: /home/cvs/gump/template/forrest/src/documentation/content/xdocs/site.xml,v
retrieving revision 1.22
retrieving revision 1.23
diff -u -r1.22 -r1.23
--- site.xml 24 Mar 2004 15:51:42 - 1.22
+++ site.xml 25 Mar 2004 15:28:52 - 1.23
@@ -11,7 +11,7 @@
-
+
Re: last night's build on lsd
> Yeah, it seems not. I don't know if this is related, but I had woes trying > to run/test Gump on my VM server Gump-box yesterday also. Basically the poor > box swapped itself to death, in part 'cos forrest grew so large. The xdocs > were written, but the forrest site wasn't generated. I wonder if something > similar happened (or is happening) to LSD. [I just tried logging in, and I'm > not getting in -- due to timeouts.] Looks like the same is happening to LSD, huge load, with what I assume is swapping takign up the CPU. I am in, but one command ever five minutes...: 16:24:16 up 22 days, 18:05, 3 users, load average: 110.71, 77.43, 63.57 196 processes: 186 sleeping, 9 running, 1 zombie, 0 stopped CPU states: cpuusernice systemirq softirq iowaitidle total0.9%1.0% 75.1% 0.0% 0.0%0.0% 22.8% Mem: 255584k av, 251740k used,3844k free, 0k shrd,7216k buff 77344k active, 131400k inactive Swap: 1052248k av, 1052248k used, 0k free 10820k cached PID USER PRI NI SIZE RSS SHARE STAT %CPU %MEM TIME CPU COMMAND 5 root 16 0 00 0 SW 83.9 0.0 94:21 0 kswapd regards Adam - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: cvs@gump.apache.org?
> > how do you feel about setting up [EMAIL PROTECTED] for all change > > notifications? +0 regards Adam - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
cvs commit: gump/python/gump integrate.py
ajack 2004/03/25 07:16:07
Modified:python/gump integrate.py
Log:
Try to avoid death by hangup (catch SIGHUP).
Revision ChangesPath
1.25 +12 -0 gump/python/gump/integrate.py
Index: integrate.py
===
RCS file: /home/cvs/gump/python/gump/integrate.py,v
retrieving revision 1.24
retrieving revision 1.25
diff -u -r1.24 -r1.25
--- integrate.py 16 Mar 2004 19:50:14 - 1.24
+++ integrate.py 25 Mar 2004 15:16:07 - 1.25
@@ -43,6 +43,9 @@
# Functions
###
+def ignoreHangup(signum):
+pass
+
def irun():
# Process command line
@@ -79,6 +82,15 @@
# static void main()
if __name__=='__main__':
+
+# Set the signal handler to ignore hangups
+try:
+# Not supported by all OSs
+signal.signal(signal.SIG_HUP, ignoreHangup)
+except:
+pass
+
+
#print 'Profiling'
#import profile
#profile.run('irun()', 'iprof')
-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
cvs commit: gump gumpy.py
ajack 2004/03/25 07:16:07 Modified:.gumpy.py Log: Try to avoid death by hangup (catch SIGHUP). Revision ChangesPath 1.12 +13 -0 gump/gumpy.py Index: gumpy.py === RCS file: /home/cvs/gump/gumpy.py,v retrieving revision 1.11 retrieving revision 1.12 diff -u -r1.11 -r1.12 --- gumpy.py 19 Mar 2004 20:58:00 - 1.11 +++ gumpy.py 25 Mar 2004 15:16:07 - 1.12 @@ -75,12 +75,16 @@ import sys import socket import time +import signal import smtplib import StringIO from xml.dom import minidom LINE=' - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - GUMP' +def ignoreHangup(signum): +pass + def runCommand(command,args='',dir=None,outputFile=None): """ Run a command, and check the result... """ @@ -176,6 +180,15 @@ Please resolve this (waiting or removing the lock file) before retrying. """ % lockFile sys.exit(1) + +# Set the signal handler to ignore hangups +try: +# Not supported by all OSs +signal.signal(signal.SIG_HUP, ignoreHangup) +except: +pass + + lock=open(lockFile,'w') lock.write(`os.getpid()`) lock.close() - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: last night's build on lsd
> It looks like the gump run did not work on lsd last night. I did not > have a look to find out what has gone on. > (the html pages on lsd are of yesterday, the email all dressed up with > nowhere to go did not get sent either) Yeah, it seems not. I don't know if this is related, but I had woes trying to run/test Gump on my VM server Gump-box yesterday also. Basically the poor box swapped itself to death, in part 'cos forrest grew so large. The xdocs were written, but the forrest site wasn't generated. I wonder if something similar happened (or is happening) to LSD. [I just tried logging in, and I'm not getting in -- due to timeouts.] I added a page yesterday, not small, not obscenely large, but maybe that pushed forrest over the edge. I've not done an analysis but it seems that there is either a memory leak as forrest generates the pages (I saw it had started generating, but hung.) I hate to say it, but sometime we really 'pay' (in terms of workarounds, outages, time delays) for the decisions to go with forrest. I don't know if we ought consider it 'yet another Gump stress test' and keep it, but I am starting to think more and more of writing HTML pages directly, via some Python code. Talking of stress, gump.dotnot is taking almost a day... http://gump.dotnot.org/#Details I'm thinking of giving Gump a shot on moof. Do you think this is a good idea, or just going to damage another server? regards, Adam - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Cocoon not picking up framework api
On Thu, 25 Mar 2004, Stephen McConnell <[EMAIL PROTECTED]> wrote: > Seems like the cocoon core build is not picking up the > avalon-framework-api. It does, but the jar file doesn't contain the CascadingThrowable class. Let's see [EMAIL PROTECTED] gump]$ find /javastuff/gump/avalon* -name CascadingThrowable.class /javastuff/gump/avalon/framework/api/target/classes/org/apache/avalon/framework/CascadingThrowable.class /javastuff/gump/avalon/framework/target/classes/org/apache/avalon/framework/CascadingThrowable.class not in impl, but in api and this is the jar that doesn't get picked up. Probably there is some logic (don't have the time to dive in ATM) that says the inside is unneeded since the one on the outside is sufficient. The appended patch should help and hopefully have cocoon building next night. Sorry Stefan Index: gump.xml === RCS file: /home/cvspublic/cocoon-2.1/gump.xml,v retrieving revision 1.132 diff -u -r1.132 gump.xml --- gump.xml25 Mar 2004 03:57:46 - 1.132 +++ gump.xml25 Mar 2004 14:05:24 - @@ -47,7 +47,7 @@ - + - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Cocoon not picking up framework api
Seems like the cocoon core build is not picking up the avalon-framework-api. cannot resolve symbol [javac] symbol : class CascadingThrowable [javac] location: package framework [javac] import org.apache.avalon.framework.CascadingThrowable; http://gump.covalent.net/log/cocoon.html Stephen. -- || | Magic by Merlin| | Production by Avalon | || | http://avalon.apache.org/merlin| | http://dpml.net/merlin/distributions/latest| || - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
cvs commit: gump/profile gump.xml
mcconnell2004/03/25 05:15:59 Modified:profile gump.xml Log: Add the avalon-legacy.xml project. Revision ChangesPath 1.329 +2 -1 gump/profile/gump.xml Index: gump.xml === RCS file: /home/cvs/gump/profile/gump.xml,v retrieving revision 1.328 retrieving revision 1.329 diff -u -r1.328 -r1.329 --- gump.xml 24 Mar 2004 13:44:00 - 1.328 +++ gump.xml 25 Mar 2004 13:15:59 - 1.329 @@ -26,9 +26,10 @@ - + + - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
cvs commit: gump/project avalon.xml
mcconnell2004/03/25 03:26:12 Modified:project avalon.xml Log: Fix avalon meta impl package name. Revision ChangesPath 1.45 +1 -1 gump/project/avalon.xml Index: avalon.xml === RCS file: /home/cvs/gump/project/avalon.xml,v retrieving revision 1.44 retrieving revision 1.45 diff -u -r1.44 -r1.45 --- avalon.xml25 Mar 2004 11:24:49 - 1.44 +++ avalon.xml25 Mar 2004 11:26:12 - 1.45 @@ -562,7 +562,7 @@ -org.apache.avalon.impl +org.apache.avalon.meta.impl - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
cvs commit: gump/project avalon.xml
mcconnell2004/03/25 03:24:49 Modified:project avalon.xml Log: Correct package name for the avalon logging impl package. Revision ChangesPath 1.44 +1 -1 gump/project/avalon.xml Index: avalon.xml === RCS file: /home/cvs/gump/project/avalon.xml,v retrieving revision 1.43 retrieving revision 1.44 diff -u -r1.43 -r1.44 --- avalon.xml24 Mar 2004 08:37:49 - 1.43 +++ avalon.xml25 Mar 2004 11:24:49 - 1.44 @@ -142,7 +142,7 @@ -org.apache.avalon.logginf.impl +org.apache.avalon.logging.impl - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
last night's build on lsd
It looks like the gump run did not work on lsd last night. I did not have a look to find out what has gone on. (the html pages on lsd are of yesterday, the email all dressed up with nowhere to go did not get sent either) Cheers, Antoine - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: cvs commit: gump/project jrefactory.xml
On Wed, 24 Mar 2004, Antoine Lévy-Lambert <[EMAIL PROTECTED]> wrote: > collect.jar in jrefactory is not common-collections from apache Looks like the collection classes for JDK 1.1. Probably unneeded when using a JVM 1.2+ - and thus for us. Cheers Stefan - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: cvs@gump.apache.org?
Leo Simons wrote: Hi gang, how do you feel about setting up [EMAIL PROTECTED] for all change notifications? The list is getting crowded :D +1 -- Nicola Ken Barozzi [EMAIL PROTECTED] - verba volant, scripta manent - (discussions get forgotten, just code remains) - - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Project descriptors (was Re: [RT] href considered harmful)
Stefano Mazzocchi wrote: Leo Simons wrote: I suggest we move to strike and loudly proclaim descriptors not living in gump CVS as harmful. Their use needs to be *strongly* discouraged from now on. Who's with me? I agree with you as a general principle. Too bad that the entire cocoon build system relies on the gump descriptor for its blocks and this is going to be so for a while. If you deprecate href, cocoon will be seriously damaged. Now, I am the one to blame for this, Actually I was the one that started it ;-P History: when I needed a descriptor for Centipede, I looked at the Gump one, and extended it. Then Cocoon needed a similar system, so out of my experience with Centipede I created the code-generating block build system but my rationale was to keep gump and cocoon in synch by making sure that everytime you built cocoon, you were also testing if the gump descriptor was right. ...that was what I thought... Of course, only gump can do the full check, but the cocoon build can help a little bit. ... and here is where it broke. The fact is that the two are now so different in the way they work that the check is not good enough. Or we make the Cocoon build system more akin to Gump, or we make something that keeps the two descriptors lazily in synch. Stefano, I need your help. We've been banging our head on this for months, and I still remember a discussion like this with Sam more than a year ago. We *must* resolve this. Let's see: 1. Gump needs to have all the descriptors at hand. Href is too volatile for Gump. 2. Gumpers need to be able to change that information. 3. Project developers need to be able to change that info too. 4. The two descriptors should be able to stey out of synch. So: From point 1 and 2 we need a repository with all the descriptors. From point 3 and 4 we need to have a replication system between the repository and the project. Let's say that each project has a Gump descriptor in the Gump repo and *can* *also* have an href. A cron job can check the two and report the differrences to the respective communities (ie the one that has the oldest descriptor). In this way we have bi-community peer review of changes. Even better, on every descriptor checkin, a quick build is performed with last night's jars, to see that the descriptor works. But this is part of the -build per checkin- thing that is on the agenda in any case. So, is this an option. Let's get this nailed :-) -- Nicola Ken Barozzi [EMAIL PROTECTED] - verba volant, scripta manent - (discussions get forgotten, just code remains) - - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
