Re: [RT] Moving gump forward
Scott Sanders wrote: On Mar 9, 2004, at 2:27 AM, Stefan Bodewig wrote: BTW: I suspect that gump could implemented by writting the ant script on the fly w/o us having to reinvent the wheel. See the antgump proposal in Alexandria - maybe Scott can chime in here? My latest try was vindico, an ant-based gump. I got far enough to see things building and so on, but not far enough to do the project.xml override/inheritance. I do believe it was a viable model, the biggest issue being that I was still using java products to build java products, and Sam had warned me about these types of things. I only stopped because Adam had started pushing more and more code into Gumpy, and I liked the idea of using non-Java to build Java. That, and he was coding much more than I was :) In short, there are a couple of gotchas with an ant-based gump, but I think it is completly doable. My other ambitions with vindico were more what we are talking about today: doing history on all builds, trying to find who to blame (usually 1 side of a 2-sided interface), moving more toward some type of continuous integration, federation of gumps (so that I am only building 4 projects), etc. I think that gump should *NOT* try to reinvent the wheel and simply make sense of what the projects already do. Keep in mind that gump is currently a top level project and we aim at building *EVERYTHING* the ASF does, including HTTPD. -- Stefano. smime.p7s Description: S/MIME Cryptographic Signature
RE: [RT] Moving gump forward
-Original Message- From: Stefan Bodewig [mailto:[EMAIL PROTECTED] Sent: 11 March 2004 08:37 To: [EMAIL PROTECTED] Subject: Re: [RT] Moving gump forward On Wed, 10 Mar 2004, Vincent Massol [EMAIL PROTECTED] wrote: From: Stefan Bodewig [mailto:[EMAIL PROTECTED] I have no idea whether Maven would support this [the build.compiler property], though. It does as it purely calls Ant's javac task (for example, we're using jikes with Maven on some project at work). Thus setting the build.compiler property is all that is required. Gump could set it on the command line of Maven and it would become a property inside an Ant Project instance associated with the task? Or does it have to be a system property? What do you mean by command line of Maven? Do you mean passing it as a system property like this: maven -Dbuild.property=... [...] ? If so, this is fine. It can also be located in any of the Maven properties files (build.properties, project.properties). Thanks -Vincent - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: [RT] Moving gump forward
On Thu, 11 Mar 2004, Vincent Massol [EMAIL PROTECTED] wrote: What do you mean by command line of Maven? Do you mean passing it as a system property like this: maven -Dbuild.property=... [...] ? Yes. If so, this is fine. Does it also work for other properties? I mean, if we set build.sysclasspath on the command line, would the property be available for the Ant tasks wrapped by Maven? javac for example would adhere to it, no matter what kind of classloader hierarchy Maven has built to load the task. For the most important part (compiling), Gump would be independent of Maven's jar override feature that way. Stefan - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: [RT] Moving gump forward
On Thu, 11 Mar 2004, Vincent Massol [EMAIL PROTECTED] wrote: For the most important part (compiling), Gump would be independent of Maven's jar override feature that way. Yeah but I'm not sure this is the right way with Maven. Probably not. It looks like tweaking a bit too much the product which is not built for that. Just like Sam tweaked Ant a bit too much when he introduced build.sysclasspath three years ago 8-) Thanks Stefan - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: [RT] Moving gump forward
On Mar 9, 2004, at 2:27 AM, Stefan Bodewig wrote: BTW: I suspect that gump could implemented by writting the ant script on the fly w/o us having to reinvent the wheel. See the antgump proposal in Alexandria - maybe Scott can chime in here? My latest try was vindico, an ant-based gump. I got far enough to see things building and so on, but not far enough to do the project.xml override/inheritance. I do believe it was a viable model, the biggest issue being that I was still using java products to build java products, and Sam had warned me about these types of things. I only stopped because Adam had started pushing more and more code into Gumpy, and I liked the idea of using non-Java to build Java. That, and he was coding much more than I was :) In short, there are a couple of gotchas with an ant-based gump, but I think it is completly doable. My other ambitions with vindico were more what we are talking about today: doing history on all builds, trying to find who to blame (usually 1 side of a 2-sided interface), moving more toward some type of continuous integration, federation of gumps (so that I am only building 4 projects), etc. Scott - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: [RT] Moving gump forward
-Original Message- From: Stefan Bodewig [mailto:[EMAIL PROTECTED] Sent: 09 March 2004 11:01 To: [EMAIL PROTECTED] Subject: Re: [RT] Moving gump forward [snip] or, eventually, how can gump execute ant forcing the javac task to be our own? Easy, write an adapter and set the build.compiler property before invoking Ant. I have no idea whether Maven would support this, though. It does as it purely calls Ant's javac task (for example, we're using jikes with Maven on some project at work). Thus setting the build.compiler property is all that is required. -Vincent - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: [RT] Moving gump forward
On Wed, 10 Mar 2004, Vincent Massol [EMAIL PROTECTED] wrote: From: Stefan Bodewig [mailto:[EMAIL PROTECTED] I have no idea whether Maven would support this [the build.compiler property], though. It does as it purely calls Ant's javac task (for example, we're using jikes with Maven on some project at work). Thus setting the build.compiler property is all that is required. Gump could set it on the command line of Maven and it would become a property inside an Ant Project instance associated with the task? Or does it have to be a system property? Stefan - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: [RT] Moving gump forward
On Tue, 09 Mar 2004, Stefano Mazzocchi [EMAIL PROTECTED] wrote: Stefan, any talks in ant-land about using eclipse JTD compiler instead of javac for the javac task? AFAIK eclipse already does that 8-) javac delegates the work to compiler adapters and those are pluggable. Some adapters are part of Ant (Sun's javac, jikes, Microsoft's jvc, gcj, Symantecs sj and kopi), some have been written by others[1] like the one for the early access generics compiler (now obsoleted with the advent of JDK 1.5). There seems to be an adapter for eclipse as well[2] - scroll down to Using the ant javac adapter. or, eventually, how can gump execute ant forcing the javac task to be our own? Easy, write an adapter and set the build.compiler property before invoking Ant. I have no idea whether Maven would support this, though. Stefan Footnotes: [1] http://ant.apache.org/external.html#Compiler%20Implementations [2] http://dev.eclipse.org/viewcvs/index.cgi/~checkout~/org.eclipse.jdt.doc.isv/guide/jdt_api_compile.htm?content-type=text/html - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: [RT] Moving gump forward
On Mon, 8 Mar 2004, Adam R. B. Jack [EMAIL PROTECTED] wrote: I'd like Stefan's input on if we allowed a gump to be like ant|maven -- to have Gump just build/archive. It won't work, at least not without duplicating larger parts of the build file in the gump descriptor. Gump currently doesn't know where the sources are, some project definitions compile from more than one source directory. Many builds contain conditional compilations (like Ant's starteam tasks will never get compiled in Gump since the - non-public - starteam SDK is not available). Many projects produce multiple jars, Gump doesn't know what to put where. If we make all the necessary information available to Gump the descriptor will become as complex as a minimal Ant build file to do the same thing. People simply won't take the effort to create it, and why should they when all the information they have to provide already is available from the build files/project descriptors/whatever. We'd loose the ant regression test suite. Don't restrict it to my (it isn't mine, BTW ;-) Ant regressions suite. The same would apply to Maven or NAnt or GNU make or ... BTW: I suspect that gump could implemented by writting the ant script on the fly w/o us having to reinvent the wheel. See the antgump proposal in Alexandria - maybe Scott can chime in here? Stefan - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: [RT] Moving gump forward
Stefano Mazzocchi wrote: Leo Simons wrote: But since we're going by leaps and bounds right now...we need a wiki-style workflow. Ditch CVS and provide me with an edit this descriptor page. The task of finding the right descriptor to edit can be several minutes of work. That should change to several seconds. Leo: do you realize the potential security implications of this? sure. What's your comfort level? If you run gump from a restricted account on a reasonably secured box (especially a dedicated one with restricted access), and you add some HTTPS digest auth in front of it (for example), I don't see this as a big issue. Its like moving from cvs to svn. A key advantage to the ASF is that not everyone will need shell accounts. Same applies to gump -- you can give out basic access to update project descriptors to lots of people. Changing gump to a wiki-style workflow means changing a whole lot of things anyway. The security concerns are addressable. = Let's go relational = My let's go relational was for the history data, not for the project descriptors. I know :-D Going relational there would buy us only more effort (unless you write a web application to handle those things directly ;-) that was, in fact, the idea in the back of my head. I don't know much about python-based web applications, but I do know there's stuff called Zope, Plone, and the like. It might be worth it :-D Enough brainstorming for tonight. I have more to say, but gotta run right now :-D keep it coming :-) I've been thinking more and more about developer workflow. There's not all that many infrastructure-style tools that have a nice workflow (IMNSHO). The changing of obscure non-DTD-enabled XML files that are hard to track down in some cvs repository or worse, in some undocumented location that you reach over SSH...it is a large part of what makes software development and system administration painful. It doesn't neccessarily have to be a fancy GUI (example of a command-line app that rules: apt), but a lot of things /can/ be hidden. Jira, confluence, sourceforge. That's what I'm thinking of atm. Nothing crystalized yet. -- cheers, - Leo Simons --- Weblog -- http://leosimons.com/ IoC Component Glue -- http://jicarilla.org/ Articles Opinions -- http://articles.leosimons.com/ --- We started off trying to set up a small anarchist community, but people wouldn't obey the rules. -- Alan Bennett - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: [RT] Moving gump forward
In message [EMAIL PROTECTED], Stefano Mazzocchi writes: Now, I think it's possible (even if computationally expensive) to understand exactly what commit broke the build and to nag the exact person and the community and copy all the offended people. ... for those not familiar with exponential growth, it's enough to say that if the build took a minute it would take gump 2,5 billion years to find out what broke the build. Why don't you start with the easy stuff. If a build of project A fails, we know all builds depending on project A will fail. But if project A is dependent on project B, and project B failed, then we keep on walking up the tree to find the first build that failed. Okay, so now that leaves the issue of a build failing because of an API or some other change in a dependent project that built successfully. For projects with a small number of dependencies, use the brute force approach you described. For projects with a large number dependencies, develop some heuristics. You can reason that if project B makes a change that breaks project A and if C also depends on project B, then there's a chance project C might also have broken. Since Gump builds a ton of code bases, a decent number of culprits (i.e., project B) could be identified by analyzing the dependencies shared by projects that failed to build. You can then apply the brute force approach to only those shared dependencies if they number below some manageable figure. Another heuristic approach that would work for Java projects at least, would be to analyze the build failure messages. Usually they'll reference a class or class member/method that is in a dependent code base. Develop an evolving library of patterns to extract the offending methods/classes/etc. and then discover what jar they came from. Anyway, those are some inelegant, inxact, and simplistic--but perhaps useful in some instances--suggestions about how to start adding the functionality you describe that would buy time to figure out how to do it right. That is, assuming it's a hard problem that requires a good while to figure out. My reasoning is that it's okay to nag the wrong projects every once in a while as long as you nag the right projects most of the time. And on first glance, based on your comments, it seems easier to implement that than to figure out the right project to nag all the time. daniel - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: [RT] Moving gump forward
2) we must make sure that people's nagged uncomfort grows with the amount of dependecies they break! Note that giving them a number doesn't work, you have to build up the entire list!!! you have to make them feel really uncomfortable. The more uncomfortable, the more energy they are going to push into the system to keep this from happening!!! or the faster they are going to resolve an issue that emerges!! this increased obnoxiousness of nagging must also result in better web pages, because, after we push them to tackle the problem, at this point, we want to help them figuring out what's wrong and what they can do to fix it. So, scenario of use is: [snip] If Gump knows what dependancies a particular project has, when the project and each of its dependencies last built successfully and it knows how to get the source from CVS, it should be possible to list all the CVS changes that have happened since the last successful build. Of course, if each gump installation also knows of other gump installations including the JRE, it might be possible to further reduce the list of changes by sharing results between gumps. -- Michael - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: [RT] Moving gump forward
Daniel F. Savarese wrote: In message [EMAIL PROTECTED], Daniel F. Savarese writes: Another heuristic approach that would work for Java projects at least, would be to analyze the build failure messages. Usually they'll reference a class or class member/method that is in a dependent code base. Develop an evolving library of patterns to extract the offending methods/classes/etc. and then discover what jar they came from. Right after I sent that I realized that there's a more effective variation of this idea. The compiler often knows exactly why a compile failed. It should be possible to extend one of the several Java compilers out there with some Gump-aware code that will map the package an offending programming construct belongs to back to the project it belongs to. That doesn't help you directly in the case when an API artifact is removed or renamed (although it does help when a method signature changes), but it does help you narrow down the change based on the packages imported by the file. So you can limit the scope of dependencies on which you apply the brute force comparative build approach. In fact, for Java code, instead of going to the trouble of enhancing the compiler, it may be enough just to look at the packages in the file(s) that failed to compile based on the error message spewed by the compiler. Maybe it's better to think about how a human goes through the process of tracking down the source of a failure and see how that can be emulated. Again, that's assuming that 100% accurate solutions are too computationally expensive to be practical. That's it for my RT contribution. I'm on EST. yep, I had the exact same thoughts about modifying a compiler, but as i said previously, I would love to think at language-agnostic ways to do this. this said, it would be just marvellous if gump could spot dependencies transparently by analzying the classloading patterns of the compiler. that would allow gump to suggests improvements to the various gump descriptors automagically. but this modification is not easy, given that we need to change ant for this. Stefan, any talks in ant-land about using eclipse JTD compiler instead of javac for the javac task? or, eventually, how can gump execute ant forcing the javac task to be our own? -- Stefano. smime.p7s Description: S/MIME Cryptographic Signature
