subject:"Re\: Continuous TCK Testing"

Re: Continuous TCK Testing

2008-10-23 Thread Jason Warner

Hi Jason,

Now that I've got my hands on anthill to play with, I'm diving into setting
up things.  I was hoping you'd be able to clarify for me a little bit how
the build-support stuff was integrated with the anthill stuff.
Specifically, I'm working on setting up a project in anthill to build the
geronimo server from the trunk in the repository.  This seems like a good
first step to me.  From what you specified in your explanation, it seems
that every step of this automated testing had an anthill project and a
corresponding groovy-based controller.  So for the geronimo build, I would
have an anthill project devoted to building the server which would help with
shuffling artifacts around, cleaning working directories, and other such
pre/post build things, but the actual build would be handled by the
controller?  So the anthill project would actually be launching the
controller rather than the build?

Thanks!

On Thu, Oct 2, 2008 at 2:34 PM, Jason Dillon <[EMAIL PROTECTED]> wrote:

> On Oct 1, 2008, at 11:20 PM, Jason Warner wrote:
>
>  Is the GBuild stuff in svn the same as the anthill-based code or is that
>> something different?  GBuild seems to have scripts for running tck and that
>> leads me to think they're the same thing, but I see no mention of anthill in
>> the code.
>>
>
> The Anthill stuff is completely different than the GBuild stuff.  I started
> out trying to get the TCK automated using GBuild, but decided that the
> system lacked too many features to perform as I desired, and went ahead with
> Anthill as it did pretty much everything, though had some stability
> problems.
>
> One of the main reasons why I choose Anthill (AHP, Anthill Pro that is) was
> its build agent and code repository systems.  This allowed me to ensure that
> each build used exactly the desired artifacts.  Another was the configurable
> workflow, which allowed me to create a custom chain of events to handle
> running builds on remote agents and control what data gets set to them, what
> it will collect and what logic to execute once all distributed work has been
> completed for a particular build.  And the kicker which help facilitate
> bringing it all together was its concept of a build life.
>
> At the time I could find *no other* build tool which could meet all of
> these needs, and so I went with AHP instead of spending months
> building/testing features in GBuild.
>
> While AHP supports configuring a lot of stuff via its web-interface, I
> found that it was very cumbersome, so I opted to write some glue, which was
> stored in svn here:
>
>
> https://svn.apache.org/viewvc/geronimo/sandbox/build-support/?pathrev=632245
>
> Its been a while, so I have to refresh my memory on how this stuff actually
> worked.  First let me explain about the code repository (what it calls
> codestation) and why it was critical to the TCK testing IMO.  When we use
> Maven normally, it pulls data from a set of external repositories, picks up
> more repositories from the stuff it downloads and quickly we loose control
> where stuff comes from.  After it pulls down all that stuff, it churns
> though a build and spits out the stuff we care about, normally stuffing them
> (via mvn install) into the local repository.
>
> AHP supports by default tasks to publish artifacts (really just a set of
> files controlled by an Ant-like include/exclude path) from a build agent
> into Codestation, as well as tasks to resolve artifacts (ie. download them
> from Codestation to the local working directory on the build agents system).
>  Each top-level build in AHP gets assigned a new (empty) build life.
>  Artifacts are always published to/resolved from a build life, either that
> of the current build, or of a dependency build.
>
> So what I did was I setup builds for Geronimo Server (the normal
> server/trunk stuff), which did the normal mvn install thingy, but I always
> gave it a custom -Dmaven.local.repository which resolved to something inside
> the working directory for the running build.  The build was still online, so
> it pulled down a bunch of stuff into an empty local repository (so it was a
> clean build wrt the repository, as well as the source code, which was always
> fetched for each new build).  Once the build had finished, I used the
> artifact publisher task to push *all* of the stuff in the local repository
> into Codestation, labled as something like "Maven repository artifacts" for
> the current build life.
>
> Then I setup another build for Apache Geronimo CTS Server (the
> porting/branches/* stuff).  This build was dependent upon the "Maven
> repository artifacts" of the Geronimo Server build, and I configured those
> artifacts to get installed on the build agents system in the same directory
> that I configured the CTS Server build to use for its local maven
> repository.  So again the repo started out empty, then got populated with
> all of the outputs from the normal G build, and then the cts-server build
> was started.  The build of t

Re: Continuous TCK Testing

2008-10-18 Thread Jason Dillon

Before when I had those 2 build machines running in my apartment in  
berkeley, I setup one xen domain specifically for running monitoring  
tools, and installed cacti on it, and then setup snmpd on each of the  
other machines configured to allow access from the xen monitoring  
domain.  This provided a very detail easy to grok monitoring console  
for the build agents.


--jason


On Oct 18, 2008, at 5:58 AM, Jay D. McHugh wrote:


Hey Kevan,

Regarding monitoring...

I managed to run into xenmon.py.

It appears to log the system utilization for the whole box as well  
as each

VM to log files in 'your' home directory if you specify the '-n' flag.

Here is the help page for xenmon.py:
[EMAIL PROTECTED]:~$ sudo python /usr/sbin/xenmon.py -h
Usage: xenmon.py [options]

Options:
 -h, --helpshow this help message and exit
 -l, --liveshow the ncurses live monitoring frontend  
(default)

 -n, --notlive write to file instead of live monitoring
 -p PREFIX, --prefix=PREFIX
   prefix to use for output files
 -t DURATION, --time=DURATION
   stop logging to file after this much time has  
elapsed
   (in seconds). set to 0 to keep logging  
indefinitely

 -i INTERVAL, --interval=INTERVAL
   interval for logging (in ms)
 --ms_per_sample=MSPERSAMPLE
   determines how many ms worth of data goes in  
a sample

 --cpu=CPU specifies which cpu to display data for
 --allocated   Display allocated time for each domain
 --noallocated Don't display allocated time for each domain
 --blocked Display blocked time for each domain
 --noblocked   Don't display blocked time for each domain
 --waited  Display waiting time for each domain
 --nowaitedDon't display waiting time for each domain
 --excount Display execution count for each domain
 --noexcount   Don't display execution count for each domain
 --iocount Display I/O count for each domain
 --noiocount   Don't display I/O count for each domain

And here is some sample output:

[EMAIL PROTECTED]:~$ cat log-dom0.log
# passed cpu dom cpu(tot) cpu(%) cpu/ex allocated/ex blocked(tot)  
blocked(%) blocked/io waited(tot) waited(%) waited/ex ex/s io(tot)  
io/ex
0.000 0 0 2.086 0.000 38863.798 3000.000 154.177 0.000 0.000  
0.504 0.000 9383.278 0.000 0.000 0.000
2.750 1 0 2.512 0.000 53804.925 3000.000 153.217 0.000 0.000  
0.316 0.000 6774.813 0.000 0.000 0.000
4.063 2 0 2.625 0.000 59959.942 3000.000 153.886 0.000 0.000  
0.173 0.000 3939.987 0.000 0.000 0.000
5.203 3 0 3.020 0.000 47522.430 3000.000 171.834 0.000 0.000  
0.701 0.000 11031.759 0.000 0.000 0.000
6.403 4 0 2.130 0.000 39256.871 3000.000 171.870 0.000 0.000  
0.617 0.000 11378.014 0.000 0.000 0.000
9.230 6 0 0.836 0.000 53962.875 3000.000 57.287 0.000 0.000  
0.038 0.000 2450.488 0.000 0.000 0.000
10.305 7 0 2.171 0.000 46119.247 3000.000 154.008 0.000 0.000  
0.367 0.000 7804.444 0.000 0.000 0.000
11.518 0 0 15931680.822 1.593 54019.023 3000.000 889706824.191  
88.971 0.000 2630292.436 0.263 8918.446 294.927 0.000 0.000
1009.216 1 0 7687035.544 0.769 53822.548 3000.000 473101345.004  
47.310 0.000 864964.568 0.086 6056.248 142.822 0.000 0.000
1010.199 2 0 20502235.224 2.050 61655.293 3000.000 979188763.754  
97.919 0.000 4279443600.516 427.944 12869345.608 332.530 0.000 0.000
1011.239 3 0 13634865.766 1.363 45934.870 3000.000 985479796.363  
98.548 0.000 1593248.596 0.159 5367.538 296.830 0.000 0.000
1012.312 4 0 18228049.181 1.823 61242.790 3000.000 979822521.396  
97.982 0.000 2593364.560 0.259 8713.213 297.636 0.000 0.000
1013.338 5 0 9891757.872 0.989 65386.046 3000.000 571275802.794  
57.128 0.000 357431.539 0.036 2362.678 151.282 0.000 0.000


We could probably add a cron job to grab a single sample every X  
minutes
and append them together to build up a utilization history (rather  
than

simply running it all of the time).

I just tried to get a single sample and the smallest run I could get  
was

about three seconds with four samples taken.

Or, I also tried xentop in batch mode:

[EMAIL PROTECTED]:~$ sudo xentop -b -i 1
 NAME  STATE   CPU(sec) CPU(%) MEM(k) MEM(%)  MAXMEM(k)  
MAXMEM(%) VCPUS NETS NETTX(k) NETRX(k) VBDS   VBD_OO   VBD_RD
VBD_WR SSID
 Domain-0 -r 4305670.03939328   23.5   no  
limit   n/a 840000 
00 2149631536
tck01 --b--- 7504490.03145728   18.83145728   
18.8 21   483054  18554931   15   655667  8445829  
2149631536
tck02 --b---11012730.03145728   18.83145728   
18.8 21   367792  17734071   83  1131709  9030663  
2149631536
tck03 -r 1445520.03145728   18.83145728   
18.8 21   188115  237006916

Re: Continuous TCK Testing

2008-10-17 Thread Jay D. McHugh

Hey Kevan,

Regarding monitoring...

I managed to run into xenmon.py.

It appears to log the system utilization for the whole box as well as each
VM to log files in 'your' home directory if you specify the '-n' flag.

Here is the help page for xenmon.py:
[EMAIL PROTECTED]:~$ sudo python /usr/sbin/xenmon.py -h
Usage: xenmon.py [options]

Options:
  -h, --helpshow this help message and exit
  -l, --liveshow the ncurses live monitoring frontend (default)
  -n, --notlive write to file instead of live monitoring
  -p PREFIX, --prefix=PREFIX
prefix to use for output files
  -t DURATION, --time=DURATION
stop logging to file after this much time has elapsed
(in seconds). set to 0 to keep logging indefinitely
  -i INTERVAL, --interval=INTERVAL
interval for logging (in ms)
  --ms_per_sample=MSPERSAMPLE
determines how many ms worth of data goes in a sample
  --cpu=CPU specifies which cpu to display data for
  --allocated   Display allocated time for each domain
  --noallocated Don't display allocated time for each domain
  --blocked Display blocked time for each domain
  --noblocked   Don't display blocked time for each domain
  --waited  Display waiting time for each domain
  --nowaitedDon't display waiting time for each domain
  --excount Display execution count for each domain
  --noexcount   Don't display execution count for each domain
  --iocount Display I/O count for each domain
  --noiocount   Don't display I/O count for each domain

And here is some sample output:

[EMAIL PROTECTED]:~$ cat log-dom0.log
# passed cpu dom cpu(tot) cpu(%) cpu/ex allocated/ex blocked(tot) blocked(%) 
blocked/io waited(tot) waited(%) waited/ex ex/s io(tot) io/ex
0.000 0 0 2.086 0.000 38863.798 3000.000 154.177 0.000 0.000 0.504 0.000 
9383.278 0.000 0.000 0.000
2.750 1 0 2.512 0.000 53804.925 3000.000 153.217 0.000 0.000 0.316 0.000 
6774.813 0.000 0.000 0.000
4.063 2 0 2.625 0.000 59959.942 3000.000 153.886 0.000 0.000 0.173 0.000 
3939.987 0.000 0.000 0.000
5.203 3 0 3.020 0.000 47522.430 3000.000 171.834 0.000 0.000 0.701 0.000 
11031.759 0.000 0.000 0.000
6.403 4 0 2.130 0.000 39256.871 3000.000 171.870 0.000 0.000 0.617 0.000 
11378.014 0.000 0.000 0.000
9.230 6 0 0.836 0.000 53962.875 3000.000 57.287 0.000 0.000 0.038 0.000 
2450.488 0.000 0.000 0.000
10.305 7 0 2.171 0.000 46119.247 3000.000 154.008 0.000 0.000 0.367 0.000 
7804.444 0.000 0.000 0.000
11.518 0 0 15931680.822 1.593 54019.023 3000.000 889706824.191 88.971 0.000 
2630292.436 0.263 8918.446 294.927 0.000 0.000
1009.216 1 0 7687035.544 0.769 53822.548 3000.000 473101345.004 47.310 
0.000 864964.568 0.086 6056.248 142.822 0.000 0.000
1010.199 2 0 20502235.224 2.050 61655.293 3000.000 979188763.754 97.919 
0.000 4279443600.516 427.944 12869345.608 332.530 0.000 0.000
1011.239 3 0 13634865.766 1.363 45934.870 3000.000 985479796.363 98.548 
0.000 1593248.596 0.159 5367.538 296.830 0.000 0.000
1012.312 4 0 18228049.181 1.823 61242.790 3000.000 979822521.396 97.982 
0.000 2593364.560 0.259 8713.213 297.636 0.000 0.000
1013.338 5 0 9891757.872 0.989 65386.046 3000.000 571275802.794 57.128 
0.000 357431.539 0.036 2362.678 151.282 0.000 0.000

We could probably add a cron job to grab a single sample every X minutes
and append them together to build up a utilization history (rather than
simply running it all of the time).

I just tried to get a single sample and the smallest run I could get was
about three seconds with four samples taken.

Or, I also tried xentop in batch mode:

[EMAIL PROTECTED]:~$ sudo xentop -b -i 1
  NAME  STATE   CPU(sec) CPU(%) MEM(k) MEM(%)  MAXMEM(k) MAXMEM(%) 
VCPUS NETS NETTX(k) NETRX(k) VBDS   VBD_OO   VBD_RD   VBD_WR SSID
  Domain-0 -r 4305670.03939328   23.5   no limit   n/a 
84000000 2149631536
 tck01 --b--- 7504490.03145728   18.83145728  18.8 
21   483054  18554931   15   655667  8445829 2149631536
 tck02 --b---11012730.03145728   18.83145728  18.8 
21   367792  17734071   83  1131709  9030663 2149631536
 tck03 -r 1445520.03145728   18.83145728  18.8 
21   188115  237006916   370431  1290683 2149631536
 tck04 --b--- 1037420.03145728   18.83145728  18.8 
21   286936  234194117   381523  1484476 2149631536

It looks to me like having a cron job that periodically ran xentop and
build up a history would be the best option (without digging through
a ton of different specialized monitor packages).


Jay

Kevan Miller wrote:
> 
> On Oct 10, 2008, at 11:29 AM, Kevan Miller wrote:
> 
>>
>> On

Re: Continuous TCK Testing

2008-10-17 Thread Kevan Miller

On Oct 10, 2008, at 11:29 AM, Kevan Miller wrote:

On Oct 10, 2008, at 11:25 AM, Kevan Miller wrote:

On Oct 8, 2008, at 11:56 PM, Kevan Miller wrote:

On Oct 8, 2008, at 4:31 PM, Jason Warner wrote:

We had some suggestions earlier for some alternate means of
implementing this (Hudson, Conitnuum, etc...). Now that we've
had Jason Dillon provide an overview of what we had in place
before, does anyone have thoughts on what we should go with? I'm
thinking we should stick with the AHP based solution. It will
need to be updated most likely, but it's been tried and tested
and shown to meet our needs. I'm wondering, though, why we
stopped using it before. Was there a specific issue we're going
to have to deal with again?

IIRC, the overwhelming reason we stopped using it before was
because of hosting issues -- spotty networking, hardware failures,
poor colo support, etc. We shouldn't have any of these problems,
now. If we do run into problems, they should now be fixable. I
have no reason to favor Hudson/Continuum over AHP. So, if we can
get AHP running easily, I'm all for it. There's only one potential
issue, that I'm aware of.

We previously had an Open Source License issued for our use of
Anthill. Here's some of the old discussion -- http://www.nabble.com/Geronimo-build-automation-status-(longish)-tt7649902.html#a7649902

Although the board was aware of our usage of AntHill, since we
weren't running AntHill on ASF hardware, I'm not sure the license
was fully vetted by Infra. I don't see any issues, but I'll want
to run this by Infra.

Jason D, will the existing license cover the version of AntHill
that we'll want to use? I'll run the license by Infra and will
also describe the issue for review by the Board, in our quarterly
report.

Heh. Oops. Just noticed that I sent the following to myself and not
the dev list. I hate when I do that...

One more thing... from emails on [EMAIL PROTECTED] looks
like Infra is cool with us running Anthill on selene and phoebe.

BTW, am planning on installing monitoring software over the weekend
on selene and phoebe. The board is interested in monitoring our
usage...

Also, we now have a new AntHill license for our use. I've placed the
license in ~kevan/License2.txt on phoebe and selene. This license
should only be used for Apache use. So, should not be placed in a
public location (e.g. our public svn tree).

Regarding monitoring software -- I haven't been able to get it to work
yet. vmstat/iostat don't work, unless you run on every virtual
machine. 'xm top' gathers data on all domains, however, doesn't make
the data easy to tuck away in a log file/available to snmp... Advice
welcome...

--kevan

Re: Continuous TCK Testing

2008-10-16 Thread Jason Dillon

Yup, might need to resurrect that stuff if we plan on using it again.

--jason

On Oct 16, 2008, at 10:39 PM, Jason Warner wrote:

Whoops... just realized that this was actually removed and I was
looking at a stickied revision of viewVC. Nevermind.

On Thu, Oct 16, 2008 at 11:15 AM, Jason Warner <[EMAIL PROTECTED]>
wrote:
While we wait to hear back in regards to the license, I'm going to
update the maven used in build-support. The server now requires
2.0.9 and the version currently used by build support is 2.0.5. I
suppose we'll need to update ant, as well. What version of ant
should we be using? 1.7.1?

On Fri, Oct 10, 2008 at 11:25 AM, Kevan Miller
<[EMAIL PROTECTED]> wrote:

On Oct 8, 2008, at 11:56 PM, Kevan Miller wrote:

On Oct 8, 2008, at 4:31 PM, Jason Warner wrote:

We had some suggestions earlier for some alternate means of
implementing this (Hudson, Conitnuum, etc...). Now that we've had
Jason Dillon provide an overview of what we had in place before,
does anyone have thoughts on what we should go with? I'm thinking
we should stick with the AHP based solution. It will need to be
updated most likely, but it's been tried and tested and shown to
meet our needs. I'm wondering, though, why we stopped using it
before. Was there a specific issue we're going to have to deal
with again?

IIRC, the overwhelming reason we stopped using it before was
because of hosting issues -- spotty networking, hardware failures,
poor colo support, etc. We shouldn't have any of these problems,
now. If we do run into problems, they should now be fixable. I have
no reason to favor Hudson/Continuum over AHP. So, if we can get AHP
running easily, I'm all for it. There's only one potential issue,
that I'm aware of.

We previously had an Open Source License issued for our use of
Anthill. Here's some of the old discussion -- http://www.nabble.com/Geronimo-build-automation-status-(longish)-tt7649902.html#a7649902

Jason D, will the existing license cover the version of AntHill
that we'll want to use? I'll run the license by Infra and will also
describe the issue for review by the Board, in our quarterly report.

IMO, I'd proceed with the assumption that we'll be using AHP. Just
don't install it on Apache hardware, yet.

I've requested a new license from Anthill. Will let you know when I
get it.

--kevan

--
~Jason Warner

Re: Continuous TCK Testing

2008-10-16 Thread Jason Warner

Whoops... just realized that this was actually removed and I was looking at
a stickied revision of viewVC.  Nevermind.

On Thu, Oct 16, 2008 at 11:15 AM, Jason Warner <[EMAIL PROTECTED]> wrote:

> While we wait to hear back in regards to the license, I'm going to update
> the maven used in build-support.  The server now requires 2.0.9 and the
> version currently used by build support is 2.0.5.  I suppose we'll need to
> update ant, as well.  What version of ant should we be using?  1.7.1?
>
>
> On Fri, Oct 10, 2008 at 11:25 AM, Kevan Miller <[EMAIL PROTECTED]>wrote:
>
>>
>> On Oct 8, 2008, at 11:56 PM, Kevan Miller wrote:
>>
>>
>> On Oct 8, 2008, at 4:31 PM, Jason Warner wrote:
>>
>> We had some suggestions earlier for some alternate means of implementing
>> this (Hudson, Conitnuum, etc...).  Now that we've had Jason Dillon provide
>> an overview of what we had in place before, does anyone have thoughts on
>> what we should go with?  I'm thinking we should stick with the AHP based
>> solution.  It will need to be updated most likely, but it's been tried and
>> tested and shown to meet our needs.  I'm wondering, though, why we stopped
>> using it before.  Was there a specific issue we're going to have to deal
>> with again?
>>
>>
>> IIRC, the overwhelming reason we stopped using it before was because of
>> hosting issues -- spotty networking, hardware failures, poor colo support,
>> etc. We shouldn't have any of these problems, now. If we do run into
>> problems, they should now be fixable. I have no reason to favor
>> Hudson/Continuum over AHP. So, if we can get AHP running easily, I'm all for
>> it. There's only one potential issue, that I'm aware of.
>>
>> We previously had an Open Source License issued for our use of Anthill.
>> Here's some of the old discussion --
>> http://www.nabble.com/Geronimo-build-automation-status-(longish)-tt7649902.html#a7649902
>>
>> Although the board was aware of our usage of AntHill, since we weren't
>> running AntHill on ASF hardware, I'm not sure the license was fully vetted
>> by Infra. I don't see any issues, but I'll want to run this by Infra.
>>
>> Jason D, will the existing license cover the version of AntHill that we'll
>> want to use? I'll run the license by Infra and will also describe the issue
>> for review by the Board, in our quarterly report.
>>
>> IMO, I'd proceed with the assumption that we'll be using AHP. Just don't
>> install it on Apache hardware, yet.
>>
>>
>> I've requested a new license from Anthill. Will let you know when I get
>> it.
>>
>> --kevan
>>
>>
>
>
> --
> ~Jason Warner
>



-- 
~Jason Warner

Re: Continuous TCK Testing

2008-10-16 Thread Jason Warner

While we wait to hear back in regards to the license, I'm going to update
the maven used in build-support.  The server now requires 2.0.9 and the
version currently used by build support is 2.0.5.  I suppose we'll need to
update ant, as well.  What version of ant should we be using?  1.7.1?

On Fri, Oct 10, 2008 at 11:25 AM, Kevan Miller <[EMAIL PROTECTED]>wrote:

>
> On Oct 8, 2008, at 11:56 PM, Kevan Miller wrote:
>
>
> On Oct 8, 2008, at 4:31 PM, Jason Warner wrote:
>
> We had some suggestions earlier for some alternate means of implementing
> this (Hudson, Conitnuum, etc...).  Now that we've had Jason Dillon provide
> an overview of what we had in place before, does anyone have thoughts on
> what we should go with?  I'm thinking we should stick with the AHP based
> solution.  It will need to be updated most likely, but it's been tried and
> tested and shown to meet our needs.  I'm wondering, though, why we stopped
> using it before.  Was there a specific issue we're going to have to deal
> with again?
>
>
> IIRC, the overwhelming reason we stopped using it before was because of
> hosting issues -- spotty networking, hardware failures, poor colo support,
> etc. We shouldn't have any of these problems, now. If we do run into
> problems, they should now be fixable. I have no reason to favor
> Hudson/Continuum over AHP. So, if we can get AHP running easily, I'm all for
> it. There's only one potential issue, that I'm aware of.
>
> We previously had an Open Source License issued for our use of Anthill.
> Here's some of the old discussion --
> http://www.nabble.com/Geronimo-build-automation-status-(longish)-tt7649902.html#a7649902
>
> Although the board was aware of our usage of AntHill, since we weren't
> running AntHill on ASF hardware, I'm not sure the license was fully vetted
> by Infra. I don't see any issues, but I'll want to run this by Infra.
>
> Jason D, will the existing license cover the version of AntHill that we'll
> want to use? I'll run the license by Infra and will also describe the issue
> for review by the Board, in our quarterly report.
>
> IMO, I'd proceed with the assumption that we'll be using AHP. Just don't
> install it on Apache hardware, yet.
>
>
> I've requested a new license from Anthill. Will let you know when I get it.
>
> --kevan
>
>


-- 
~Jason Warner

Re: Continuous TCK Testing

2008-10-10 Thread Kevan Miller

On Oct 8, 2008, at 11:56 PM, Kevan Miller wrote:

On Oct 8, 2008, at 4:31 PM, Jason Warner wrote:

We had some suggestions earlier for some alternate means of
implementing this (Hudson, Conitnuum, etc...). Now that we've had
Jason Dillon provide an overview of what we had in place before,
does anyone have thoughts on what we should go with? I'm thinking
we should stick with the AHP based solution. It will need to be
updated most likely, but it's been tried and tested and shown to
meet our needs. I'm wondering, though, why we stopped using it
before. Was there a specific issue we're going to have to deal
with again?

IIRC, the overwhelming reason we stopped using it before was because
of hosting issues -- spotty networking, hardware failures, poor colo
support, etc. We shouldn't have any of these problems, now. If we do
run into problems, they should now be fixable. I have no reason to
favor Hudson/Continuum over AHP. So, if we can get AHP running
easily, I'm all for it. There's only one potential issue, that I'm
aware of.

We previously had an Open Source License issued for our use of
Anthill. Here's some of the old discussion -- http://www.nabble.com/Geronimo-build-automation-status-(longish)-tt7649902.html#a7649902

Jason D, will the existing license cover the version of AntHill that
we'll want to use? I'll run the license by Infra and will also
describe the issue for review by the Board, in our quarterly report.

IMO, I'd proceed with the assumption that we'll be using AHP. Just
don't install it on Apache hardware, yet.

I've requested a new license from Anthill. Will let you know when I
get it.

--kevan

Re: Continuous TCK Testing

2008-10-09 Thread Jason Dillon


Yup, it was manually installed on each machine ;-)

--jason


On Oct 9, 2008, at 6:43 PM, Jason Warner wrote:

My apologies.  I didn't phrase my question properly.  Most of the  
software necessary was pulled down via svn, but I saw no such  
behaviour for AHP.  After looking at it some more, I imagine the  
software was just manually installed on the machine.  It was kind of  
a silly question to begin with, I suppose.


On Thu, Oct 9, 2008 at 4:16 AM, Jason Dillon  
<[EMAIL PROTECTED]> wrote:

On Oct 8, 2008, at 11:05 PM, Jason Warner wrote:

Here's a quick question.  Where does AHP come from?


http://www.anthillpro.com

(ever heard of google :-P)

--jason




On Mon, Oct 6, 2008 at 1:18 PM, Jason Dillon  
<[EMAIL PROTECTED]> wrote:

Sure np, took me a while to get around to writing it too ;-)

--jason


On Oct 6, 2008, at 10:24 PM, Jason Warner wrote:

Just got around to reading this.  Thanks for the brain dump,  
Jason.  No questions as of yet, but I'm sure I'll need a few more  
reads before I understand it all.


On Thu, Oct 2, 2008 at 2:34 PM, Jason Dillon  
<[EMAIL PROTECTED]> wrote:

On Oct 1, 2008, at 11:20 PM, Jason Warner wrote:

Is the GBuild stuff in svn the same as the anthill-based code or  
is that something different?  GBuild seems to have scripts for  
running tck and that leads me to think they're the same thing, but  
I see no mention of anthill in the code.


The Anthill stuff is completely different than the GBuild stuff.   
I started out trying to get the TCK automated using GBuild, but  
decided that the system lacked too many features to perform as I  
desired, and went ahead with Anthill as it did pretty much  
everything, though had some stability problems.


One of the main reasons why I choose Anthill (AHP, Anthill Pro  
that is) was its build agent and code repository systems.  This  
allowed me to ensure that each build used exactly the desired  
artifacts.  Another was the configurable workflow, which allowed  
me to create a custom chain of events to handle running builds on  
remote agents and control what data gets set to them, what it will  
collect and what logic to execute once all distributed work has  
been completed for a particular build.  And the kicker which help  
facilitate bringing it all together was its concept of a build life.


At the time I could find *no other* build tool which could meet  
all of these needs, and so I went with AHP instead of spending  
months building/testing features in GBuild.


While AHP supports configuring a lot of stuff via its web- 
interface, I found that it was very cumbersome, so I opted to  
write some glue, which was stored in svn here:


   https://svn.apache.org/viewvc/geronimo/sandbox/build-support/?pathrev=632245

Its been a while, so I have to refresh my memory on how this stuff  
actually worked.  First let me explain about the code repository  
(what it calls codestation) and why it was critical to the TCK  
testing IMO.  When we use Maven normally, it pulls data from a set  
of external repositories, picks up more repositories from the  
stuff it downloads and quickly we loose control where stuff comes  
from.  After it pulls down all that stuff, it churns though a  
build and spits out the stuff we care about, normally stuffing  
them (via mvn install) into the local repository.


AHP supports by default tasks to publish artifacts (really just a  
set of files controlled by an Ant-like include/exclude path) from  
a build agent into Codestation, as well as tasks to resolve  
artifacts (ie. download them from Codestation to the local working  
directory on the build agents system).  Each top-level build in  
AHP gets assigned a new (empty) build life.  Artifacts are always  
published to/resolved from a build life, either that of the  
current build, or of a dependency build.


So what I did was I setup builds for Geronimo Server (the normal  
server/trunk stuff), which did the normal mvn install thingy, but  
I always gave it a custom -Dmaven.local.repository which resolved  
to something inside the working directory for the running build.   
The build was still online, so it pulled down a bunch of stuff  
into an empty local repository (so it was a clean build wrt the  
repository, as well as the source code, which was always fetched  
for each new build).  Once the build had finished, I used the  
artifact publisher task to push *all* of the stuff in the local  
repository into Codestation, labled as something like "Maven  
repository artifacts" for the current build life.


Then I setup another build for Apache Geronimo CTS Server (the  
porting/branches/* stuff).  This build was dependent upon the  
"Maven repository artifacts" of the Geronimo Server build, and I  
configured those artifacts to get installed on the build agents  
system in the same directory that I configured the CTS Server  
build to use for its local maven repository.  So again the repo  
started out empty, then got populated with all of t

Re: Continuous TCK Testing

2008-10-09 Thread Jason Warner

My apologies.  I didn't phrase my question properly.  Most of the software
necessary was pulled down via svn, but I saw no such behaviour for AHP.
After looking at it some more, I imagine the software was just manually
installed on the machine.  It was kind of a silly question to begin with, I
suppose.

On Thu, Oct 9, 2008 at 4:16 AM, Jason Dillon <[EMAIL PROTECTED]> wrote:

> On Oct 8, 2008, at 11:05 PM, Jason Warner wrote:
>
> Here's a quick question.  Where does AHP come from?
>
>
> http://www.anthillpro.com
>
> (ever heard of google :-P)
>
> --jason
>
>
>
> On Mon, Oct 6, 2008 at 1:18 PM, Jason Dillon <[EMAIL PROTECTED]>wrote:
>
>> Sure np, took me a while to get around to writing it too ;-)
>> --jason
>>
>>
>> On Oct 6, 2008, at 10:24 PM, Jason Warner wrote:
>>
>> Just got around to reading this.  Thanks for the brain dump, Jason.  No
>> questions as of yet, but I'm sure I'll need a few more reads before I
>> understand it all.
>>
>> On Thu, Oct 2, 2008 at 2:34 PM, Jason Dillon <[EMAIL PROTECTED]>wrote:
>>
>>> On Oct 1, 2008, at 11:20 PM, Jason Warner wrote:
>>>
>>>  Is the GBuild stuff in svn the same as the anthill-based code or is that
 something different?  GBuild seems to have scripts for running tck and that
 leads me to think they're the same thing, but I see no mention of anthill 
 in
 the code.

>>>
>>> The Anthill stuff is completely different than the GBuild stuff.  I
>>> started out trying to get the TCK automated using GBuild, but decided that
>>> the system lacked too many features to perform as I desired, and went ahead
>>> with Anthill as it did pretty much everything, though had some stability
>>> problems.
>>>
>>> One of the main reasons why I choose Anthill (AHP, Anthill Pro that is)
>>> was its build agent and code repository systems.  This allowed me to ensure
>>> that each build used exactly the desired artifacts.  Another was the
>>> configurable workflow, which allowed me to create a custom chain of events
>>> to handle running builds on remote agents and control what data gets set to
>>> them, what it will collect and what logic to execute once all distributed
>>> work has been completed for a particular build.  And the kicker which help
>>> facilitate bringing it all together was its concept of a build life.
>>>
>>> At the time I could find *no other* build tool which could meet all of
>>> these needs, and so I went with AHP instead of spending months
>>> building/testing features in GBuild.
>>>
>>> While AHP supports configuring a lot of stuff via its web-interface, I
>>> found that it was very cumbersome, so I opted to write some glue, which was
>>> stored in svn here:
>>>
>>>
>>> https://svn.apache.org/viewvc/geronimo/sandbox/build-support/?pathrev=632245
>>>
>>> Its been a while, so I have to refresh my memory on how this stuff
>>> actually worked.  First let me explain about the code repository (what it
>>> calls codestation) and why it was critical to the TCK testing IMO.  When we
>>> use Maven normally, it pulls data from a set of external repositories, picks
>>> up more repositories from the stuff it downloads and quickly we loose
>>> control where stuff comes from.  After it pulls down all that stuff, it
>>> churns though a build and spits out the stuff we care about, normally
>>> stuffing them (via mvn install) into the local repository.
>>>
>>> AHP supports by default tasks to publish artifacts (really just a set of
>>> files controlled by an Ant-like include/exclude path) from a build agent
>>> into Codestation, as well as tasks to resolve artifacts (ie. download them
>>> from Codestation to the local working directory on the build agents system).
>>>  Each top-level build in AHP gets assigned a new (empty) build life.
>>>  Artifacts are always published to/resolved from a build life, either that
>>> of the current build, or of a dependency build.
>>>
>>> So what I did was I setup builds for Geronimo Server (the normal
>>> server/trunk stuff), which did the normal mvn install thingy, but I always
>>> gave it a custom -Dmaven.local.repository which resolved to something inside
>>> the working directory for the running build.  The build was still online, so
>>> it pulled down a bunch of stuff into an empty local repository (so it was a
>>> clean build wrt the repository, as well as the source code, which was always
>>> fetched for each new build).  Once the build had finished, I used the
>>> artifact publisher task to push *all* of the stuff in the local repository
>>> into Codestation, labled as something like "Maven repository artifacts" for
>>> the current build life.
>>>
>>> Then I setup another build for Apache Geronimo CTS Server (the
>>> porting/branches/* stuff).  This build was dependent upon the "Maven
>>> repository artifacts" of the Geronimo Server build, and I configured those
>>> artifacts to get installed on the build agents system in the same directory
>>> that I configured the CTS Server build to use for its local maven
>>> repos

Re: Continuous TCK Testing

2008-10-09 Thread Jason Dillon

I'd imagine we need to ask the AHP folks for a new license.

--jason

On Oct 9, 2008, at 10:56 AM, Kevan Miller wrote:

On Oct 8, 2008, at 4:31 PM, Jason Warner wrote:

We had some suggestions earlier for some alternate means of
implementing this (Hudson, Conitnuum, etc...). Now that we've had
Jason Dillon provide an overview of what we had in place before,
does anyone have thoughts on what we should go with? I'm thinking
we should stick with the AHP based solution. It will need to be
updated most likely, but it's been tried and tested and shown to
meet our needs. I'm wondering, though, why we stopped using it
before. Was there a specific issue we're going to have to deal
with again?

IIRC, the overwhelming reason we stopped using it before was because
of hosting issues -- spotty networking, hardware failures, poor colo
support, etc. We shouldn't have any of these problems, now. If we do
run into problems, they should now be fixable. I have no reason to
favor Hudson/Continuum over AHP. So, if we can get AHP running
easily, I'm all for it. There's only one potential issue, that I'm
aware of.

We previously had an Open Source License issued for our use of
Anthill. Here's some of the old discussion -- http://www.nabble.com/Geronimo-build-automation-status-(longish)-tt7649902.html#a7649902

Jason D, will the existing license cover the version of AntHill that
we'll want to use? I'll run the license by Infra and will also
describe the issue for review by the Board, in our quarterly report.

IMO, I'd proceed with the assumption that we'll be using AHP. Just
don't install it on Apache hardware, yet.

--kevan

Re: Continuous TCK Testing

2008-10-09 Thread Jason Dillon


On Oct 8, 2008, at 11:05 PM, Jason Warner wrote:

Here's a quick question.  Where does AHP come from?


http://www.anthillpro.com

(ever heard of google :-P)

--jason




On Mon, Oct 6, 2008 at 1:18 PM, Jason Dillon  
<[EMAIL PROTECTED]> wrote:

Sure np, took me a while to get around to writing it too ;-)

--jason


On Oct 6, 2008, at 10:24 PM, Jason Warner wrote:

Just got around to reading this.  Thanks for the brain dump,  
Jason.  No questions as of yet, but I'm sure I'll need a few more  
reads before I understand it all.


On Thu, Oct 2, 2008 at 2:34 PM, Jason Dillon  
<[EMAIL PROTECTED]> wrote:

On Oct 1, 2008, at 11:20 PM, Jason Warner wrote:

Is the GBuild stuff in svn the same as the anthill-based code or is  
that something different?  GBuild seems to have scripts for running  
tck and that leads me to think they're the same thing, but I see no  
mention of anthill in the code.


The Anthill stuff is completely different than the GBuild stuff.  I  
started out trying to get the TCK automated using GBuild, but  
decided that the system lacked too many features to perform as I  
desired, and went ahead with Anthill as it did pretty much  
everything, though had some stability problems.


One of the main reasons why I choose Anthill (AHP, Anthill Pro that  
is) was its build agent and code repository systems.  This allowed  
me to ensure that each build used exactly the desired artifacts.   
Another was the configurable workflow, which allowed me to create a  
custom chain of events to handle running builds on remote agents  
and control what data gets set to them, what it will collect and  
what logic to execute once all distributed work has been completed  
for a particular build.  And the kicker which help facilitate  
bringing it all together was its concept of a build life.


At the time I could find *no other* build tool which could meet all  
of these needs, and so I went with AHP instead of spending months  
building/testing features in GBuild.


While AHP supports configuring a lot of stuff via its web- 
interface, I found that it was very cumbersome, so I opted to write  
some glue, which was stored in svn here:


   https://svn.apache.org/viewvc/geronimo/sandbox/build-support/?pathrev=632245

Its been a while, so I have to refresh my memory on how this stuff  
actually worked.  First let me explain about the code repository  
(what it calls codestation) and why it was critical to the TCK  
testing IMO.  When we use Maven normally, it pulls data from a set  
of external repositories, picks up more repositories from the stuff  
it downloads and quickly we loose control where stuff comes from.   
After it pulls down all that stuff, it churns though a build and  
spits out the stuff we care about, normally stuffing them (via mvn  
install) into the local repository.


AHP supports by default tasks to publish artifacts (really just a  
set of files controlled by an Ant-like include/exclude path) from a  
build agent into Codestation, as well as tasks to resolve artifacts  
(ie. download them from Codestation to the local working directory  
on the build agents system).  Each top-level build in AHP gets  
assigned a new (empty) build life.  Artifacts are always published  
to/resolved from a build life, either that of the current build, or  
of a dependency build.


So what I did was I setup builds for Geronimo Server (the normal  
server/trunk stuff), which did the normal mvn install thingy, but I  
always gave it a custom -Dmaven.local.repository which resolved to  
something inside the working directory for the running build.  The  
build was still online, so it pulled down a bunch of stuff into an  
empty local repository (so it was a clean build wrt the repository,  
as well as the source code, which was always fetched for each new  
build).  Once the build had finished, I used the artifact publisher  
task to push *all* of the stuff in the local repository into  
Codestation, labled as something like "Maven repository artifacts"  
for the current build life.


Then I setup another build for Apache Geronimo CTS Server (the  
porting/branches/* stuff).  This build was dependent upon the  
"Maven repository artifacts" of the Geronimo Server build, and I  
configured those artifacts to get installed on the build agents  
system in the same directory that I configured the CTS Server build  
to use for its local maven repository.  So again the repo started  
out empty, then got populated with all of the outputs from the  
normal G build, and then the cts-server build was started.  The  
build of the components and assemblies is normally fairly quick and  
aside from some stuff in the private tck repo won't download muck  
more stuff, because it already had most of its dependencies  
installed via the Codestation dependency resolution.   Once the  
build finished, I published to cts-server assembly artifacts back  
to Codestation under like "CTS Server Assemblies" or something.


Up until this

Re: Continuous TCK Testing

2008-10-08 Thread Kevan Miller

On Oct 8, 2008, at 4:31 PM, Jason Warner wrote:

We had some suggestions earlier for some alternate means of
implementing this (Hudson, Conitnuum, etc...). Now that we've had
Jason Dillon provide an overview of what we had in place before,
does anyone have thoughts on what we should go with? I'm thinking
we should stick with the AHP based solution. It will need to be
updated most likely, but it's been tried and tested and shown to
meet our needs. I'm wondering, though, why we stopped using it
before. Was there a specific issue we're going to have to deal with
again?

IIRC, the overwhelming reason we stopped using it before was because
of hosting issues -- spotty networking, hardware failures, poor colo
support, etc. We shouldn't have any of these problems, now. If we do
run into problems, they should now be fixable. I have no reason to
favor Hudson/Continuum over AHP. So, if we can get AHP running easily,
I'm all for it. There's only one potential issue, that I'm aware of.

We previously had an Open Source License issued for our use of
Anthill. Here's some of the old discussion -- http://www.nabble.com/Geronimo-build-automation-status-(longish)-tt7649902.html#a7649902

Although the board was aware of our usage of AntHill, since we weren't
running AntHill on ASF hardware, I'm not sure the license was fully
vetted by Infra. I don't see any issues, but I'll want to run this by
Infra.

Jason D, will the existing license cover the version of AntHill that
we'll want to use? I'll run the license by Infra and will also
describe the issue for review by the Board, in our quarterly report.

IMO, I'd proceed with the assumption that we'll be using AHP. Just
don't install it on Apache hardware, yet.

--kevan

Re: Continuous TCK Testing

2008-10-08 Thread Jason Warner

We had some suggestions earlier for some alternate means of implementing
this (Hudson, Conitnuum, etc...).  Now that we've had Jason Dillon provide
an overview of what we had in place before, does anyone have thoughts on
what we should go with?  I'm thinking we should stick with the AHP based
solution.  It will need to be updated most likely, but it's been tried and
tested and shown to meet our needs.  I'm wondering, though, why we stopped
using it before.  Was there a specific issue we're going to have to deal
with again?

Thanks,

On Wed, Oct 8, 2008 at 12:05 PM, Jason Warner <[EMAIL PROTECTED]> wrote:

> Here's a quick question.  Where does AHP come from?
>
> On Mon, Oct 6, 2008 at 1:18 PM, Jason Dillon <[EMAIL PROTECTED]>wrote:
>
>> Sure np, took me a while to get around to writing it too ;-)
>> --jason
>>
>>
>> On Oct 6, 2008, at 10:24 PM, Jason Warner wrote:
>>
>> Just got around to reading this.  Thanks for the brain dump, Jason.  No
>> questions as of yet, but I'm sure I'll need a few more reads before I
>> understand it all.
>>
>> On Thu, Oct 2, 2008 at 2:34 PM, Jason Dillon <[EMAIL PROTECTED]>wrote:
>>
>>> On Oct 1, 2008, at 11:20 PM, Jason Warner wrote:
>>>
>>>  Is the GBuild stuff in svn the same as the anthill-based code or is that
 something different?  GBuild seems to have scripts for running tck and that
 leads me to think they're the same thing, but I see no mention of anthill 
 in
 the code.

>>>
>>> The Anthill stuff is completely different than the GBuild stuff.  I
>>> started out trying to get the TCK automated using GBuild, but decided that
>>> the system lacked too many features to perform as I desired, and went ahead
>>> with Anthill as it did pretty much everything, though had some stability
>>> problems.
>>>
>>> One of the main reasons why I choose Anthill (AHP, Anthill Pro that is)
>>> was its build agent and code repository systems.  This allowed me to ensure
>>> that each build used exactly the desired artifacts.  Another was the
>>> configurable workflow, which allowed me to create a custom chain of events
>>> to handle running builds on remote agents and control what data gets set to
>>> them, what it will collect and what logic to execute once all distributed
>>> work has been completed for a particular build.  And the kicker which help
>>> facilitate bringing it all together was its concept of a build life.
>>>
>>> At the time I could find *no other* build tool which could meet all of
>>> these needs, and so I went with AHP instead of spending months
>>> building/testing features in GBuild.
>>>
>>> While AHP supports configuring a lot of stuff via its web-interface, I
>>> found that it was very cumbersome, so I opted to write some glue, which was
>>> stored in svn here:
>>>
>>>
>>> https://svn.apache.org/viewvc/geronimo/sandbox/build-support/?pathrev=632245
>>>
>>> Its been a while, so I have to refresh my memory on how this stuff
>>> actually worked.  First let me explain about the code repository (what it
>>> calls codestation) and why it was critical to the TCK testing IMO.  When we
>>> use Maven normally, it pulls data from a set of external repositories, picks
>>> up more repositories from the stuff it downloads and quickly we loose
>>> control where stuff comes from.  After it pulls down all that stuff, it
>>> churns though a build and spits out the stuff we care about, normally
>>> stuffing them (via mvn install) into the local repository.
>>>
>>> AHP supports by default tasks to publish artifacts (really just a set of
>>> files controlled by an Ant-like include/exclude path) from a build agent
>>> into Codestation, as well as tasks to resolve artifacts (ie. download them
>>> from Codestation to the local working directory on the build agents system).
>>>  Each top-level build in AHP gets assigned a new (empty) build life.
>>>  Artifacts are always published to/resolved from a build life, either that
>>> of the current build, or of a dependency build.
>>>
>>> So what I did was I setup builds for Geronimo Server (the normal
>>> server/trunk stuff), which did the normal mvn install thingy, but I always
>>> gave it a custom -Dmaven.local.repository which resolved to something inside
>>> the working directory for the running build.  The build was still online, so
>>> it pulled down a bunch of stuff into an empty local repository (so it was a
>>> clean build wrt the repository, as well as the source code, which was always
>>> fetched for each new build).  Once the build had finished, I used the
>>> artifact publisher task to push *all* of the stuff in the local repository
>>> into Codestation, labled as something like "Maven repository artifacts" for
>>> the current build life.
>>>
>>> Then I setup another build for Apache Geronimo CTS Server (the
>>> porting/branches/* stuff).  This build was dependent upon the "Maven
>>> repository artifacts" of the Geronimo Server build, and I configured those
>>> artifacts to get installed on the build agents syste

Re: Continuous TCK Testing

2008-10-08 Thread Jason Warner

Here's a quick question.  Where does AHP come from?

On Mon, Oct 6, 2008 at 1:18 PM, Jason Dillon <[EMAIL PROTECTED]> wrote:

> Sure np, took me a while to get around to writing it too ;-)
> --jason
>
>
> On Oct 6, 2008, at 10:24 PM, Jason Warner wrote:
>
> Just got around to reading this.  Thanks for the brain dump, Jason.  No
> questions as of yet, but I'm sure I'll need a few more reads before I
> understand it all.
>
> On Thu, Oct 2, 2008 at 2:34 PM, Jason Dillon <[EMAIL PROTECTED]>wrote:
>
>> On Oct 1, 2008, at 11:20 PM, Jason Warner wrote:
>>
>>  Is the GBuild stuff in svn the same as the anthill-based code or is that
>>> something different?  GBuild seems to have scripts for running tck and that
>>> leads me to think they're the same thing, but I see no mention of anthill in
>>> the code.
>>>
>>
>> The Anthill stuff is completely different than the GBuild stuff.  I
>> started out trying to get the TCK automated using GBuild, but decided that
>> the system lacked too many features to perform as I desired, and went ahead
>> with Anthill as it did pretty much everything, though had some stability
>> problems.
>>
>> One of the main reasons why I choose Anthill (AHP, Anthill Pro that is)
>> was its build agent and code repository systems.  This allowed me to ensure
>> that each build used exactly the desired artifacts.  Another was the
>> configurable workflow, which allowed me to create a custom chain of events
>> to handle running builds on remote agents and control what data gets set to
>> them, what it will collect and what logic to execute once all distributed
>> work has been completed for a particular build.  And the kicker which help
>> facilitate bringing it all together was its concept of a build life.
>>
>> At the time I could find *no other* build tool which could meet all of
>> these needs, and so I went with AHP instead of spending months
>> building/testing features in GBuild.
>>
>> While AHP supports configuring a lot of stuff via its web-interface, I
>> found that it was very cumbersome, so I opted to write some glue, which was
>> stored in svn here:
>>
>>
>> https://svn.apache.org/viewvc/geronimo/sandbox/build-support/?pathrev=632245
>>
>> Its been a while, so I have to refresh my memory on how this stuff
>> actually worked.  First let me explain about the code repository (what it
>> calls codestation) and why it was critical to the TCK testing IMO.  When we
>> use Maven normally, it pulls data from a set of external repositories, picks
>> up more repositories from the stuff it downloads and quickly we loose
>> control where stuff comes from.  After it pulls down all that stuff, it
>> churns though a build and spits out the stuff we care about, normally
>> stuffing them (via mvn install) into the local repository.
>>
>> AHP supports by default tasks to publish artifacts (really just a set of
>> files controlled by an Ant-like include/exclude path) from a build agent
>> into Codestation, as well as tasks to resolve artifacts (ie. download them
>> from Codestation to the local working directory on the build agents system).
>>  Each top-level build in AHP gets assigned a new (empty) build life.
>>  Artifacts are always published to/resolved from a build life, either that
>> of the current build, or of a dependency build.
>>
>> So what I did was I setup builds for Geronimo Server (the normal
>> server/trunk stuff), which did the normal mvn install thingy, but I always
>> gave it a custom -Dmaven.local.repository which resolved to something inside
>> the working directory for the running build.  The build was still online, so
>> it pulled down a bunch of stuff into an empty local repository (so it was a
>> clean build wrt the repository, as well as the source code, which was always
>> fetched for each new build).  Once the build had finished, I used the
>> artifact publisher task to push *all* of the stuff in the local repository
>> into Codestation, labled as something like "Maven repository artifacts" for
>> the current build life.
>>
>> Then I setup another build for Apache Geronimo CTS Server (the
>> porting/branches/* stuff).  This build was dependent upon the "Maven
>> repository artifacts" of the Geronimo Server build, and I configured those
>> artifacts to get installed on the build agents system in the same directory
>> that I configured the CTS Server build to use for its local maven
>> repository.  So again the repo started out empty, then got populated with
>> all of the outputs from the normal G build, and then the cts-server build
>> was started.  The build of the components and assemblies is normally fairly
>> quick and aside from some stuff in the private tck repo won't download muck
>> more stuff, because it already had most of its dependencies installed via
>> the Codestation dependency resolution.   Once the build finished, I
>> published to cts-server assembly artifacts back to Codestation under like
>> "CTS Server Assemblies" or something.
>>
>> Up until this point its n

Re: Continuous TCK Testing

2008-10-06 Thread Jason Dillon


Sure np, took me a while to get around to writing it too ;-)

--jason


On Oct 6, 2008, at 10:24 PM, Jason Warner wrote:

Just got around to reading this.  Thanks for the brain dump, Jason.   
No questions as of yet, but I'm sure I'll need a few more reads  
before I understand it all.


On Thu, Oct 2, 2008 at 2:34 PM, Jason Dillon  
<[EMAIL PROTECTED]> wrote:

On Oct 1, 2008, at 11:20 PM, Jason Warner wrote:

Is the GBuild stuff in svn the same as the anthill-based code or is  
that something different?  GBuild seems to have scripts for running  
tck and that leads me to think they're the same thing, but I see no  
mention of anthill in the code.


The Anthill stuff is completely different than the GBuild stuff.  I  
started out trying to get the TCK automated using GBuild, but  
decided that the system lacked too many features to perform as I  
desired, and went ahead with Anthill as it did pretty much  
everything, though had some stability problems.


One of the main reasons why I choose Anthill (AHP, Anthill Pro that  
is) was its build agent and code repository systems.  This allowed  
me to ensure that each build used exactly the desired artifacts.   
Another was the configurable workflow, which allowed me to create a  
custom chain of events to handle running builds on remote agents and  
control what data gets set to them, what it will collect and what  
logic to execute once all distributed work has been completed for a  
particular build.  And the kicker which help facilitate bringing it  
all together was its concept of a build life.


At the time I could find *no other* build tool which could meet all  
of these needs, and so I went with AHP instead of spending months  
building/testing features in GBuild.


While AHP supports configuring a lot of stuff via its web-interface,  
I found that it was very cumbersome, so I opted to write some glue,  
which was stored in svn here:


   https://svn.apache.org/viewvc/geronimo/sandbox/build-support/?pathrev=632245

Its been a while, so I have to refresh my memory on how this stuff  
actually worked.  First let me explain about the code repository  
(what it calls codestation) and why it was critical to the TCK  
testing IMO.  When we use Maven normally, it pulls data from a set  
of external repositories, picks up more repositories from the stuff  
it downloads and quickly we loose control where stuff comes from.   
After it pulls down all that stuff, it churns though a build and  
spits out the stuff we care about, normally stuffing them (via mvn  
install) into the local repository.


AHP supports by default tasks to publish artifacts (really just a  
set of files controlled by an Ant-like include/exclude path) from a  
build agent into Codestation, as well as tasks to resolve artifacts  
(ie. download them from Codestation to the local working directory  
on the build agents system).  Each top-level build in AHP gets  
assigned a new (empty) build life.  Artifacts are always published  
to/resolved from a build life, either that of the current build, or  
of a dependency build.


So what I did was I setup builds for Geronimo Server (the normal  
server/trunk stuff), which did the normal mvn install thingy, but I  
always gave it a custom -Dmaven.local.repository which resolved to  
something inside the working directory for the running build.  The  
build was still online, so it pulled down a bunch of stuff into an  
empty local repository (so it was a clean build wrt the repository,  
as well as the source code, which was always fetched for each new  
build).  Once the build had finished, I used the artifact publisher  
task to push *all* of the stuff in the local repository into  
Codestation, labled as something like "Maven repository artifacts"  
for the current build life.


Then I setup another build for Apache Geronimo CTS Server (the  
porting/branches/* stuff).  This build was dependent upon the "Maven  
repository artifacts" of the Geronimo Server build, and I configured  
those artifacts to get installed on the build agents system in the  
same directory that I configured the CTS Server build to use for its  
local maven repository.  So again the repo started out empty, then  
got populated with all of the outputs from the normal G build, and  
then the cts-server build was started.  The build of the components  
and assemblies is normally fairly quick and aside from some stuff in  
the private tck repo won't download muck more stuff, because it  
already had most of its dependencies installed via the Codestation  
dependency resolution.   Once the build finished, I published to cts- 
server assembly artifacts back to Codestation under like "CTS Server  
Assemblies" or something.


Up until this point its normal builds, but now we have built the G  
server, then built the CTS server (using the *exact* artifacts from  
the G server build, even though each might have happened on a  
different build agent).  And now we need to go and run a

Re: Continuous TCK Testing

2008-10-06 Thread Jason Warner

Just got around to reading this.  Thanks for the brain dump, Jason.  No
questions as of yet, but I'm sure I'll need a few more reads before I
understand it all.

On Thu, Oct 2, 2008 at 2:34 PM, Jason Dillon <[EMAIL PROTECTED]> wrote:

> On Oct 1, 2008, at 11:20 PM, Jason Warner wrote:
>
>  Is the GBuild stuff in svn the same as the anthill-based code or is that
>> something different?  GBuild seems to have scripts for running tck and that
>> leads me to think they're the same thing, but I see no mention of anthill in
>> the code.
>>
>
> The Anthill stuff is completely different than the GBuild stuff.  I started
> out trying to get the TCK automated using GBuild, but decided that the
> system lacked too many features to perform as I desired, and went ahead with
> Anthill as it did pretty much everything, though had some stability
> problems.
>
> One of the main reasons why I choose Anthill (AHP, Anthill Pro that is) was
> its build agent and code repository systems.  This allowed me to ensure that
> each build used exactly the desired artifacts.  Another was the configurable
> workflow, which allowed me to create a custom chain of events to handle
> running builds on remote agents and control what data gets set to them, what
> it will collect and what logic to execute once all distributed work has been
> completed for a particular build.  And the kicker which help facilitate
> bringing it all together was its concept of a build life.
>
> At the time I could find *no other* build tool which could meet all of
> these needs, and so I went with AHP instead of spending months
> building/testing features in GBuild.
>
> While AHP supports configuring a lot of stuff via its web-interface, I
> found that it was very cumbersome, so I opted to write some glue, which was
> stored in svn here:
>
>
> https://svn.apache.org/viewvc/geronimo/sandbox/build-support/?pathrev=632245
>
> Its been a while, so I have to refresh my memory on how this stuff actually
> worked.  First let me explain about the code repository (what it calls
> codestation) and why it was critical to the TCK testing IMO.  When we use
> Maven normally, it pulls data from a set of external repositories, picks up
> more repositories from the stuff it downloads and quickly we loose control
> where stuff comes from.  After it pulls down all that stuff, it churns
> though a build and spits out the stuff we care about, normally stuffing them
> (via mvn install) into the local repository.
>
> AHP supports by default tasks to publish artifacts (really just a set of
> files controlled by an Ant-like include/exclude path) from a build agent
> into Codestation, as well as tasks to resolve artifacts (ie. download them
> from Codestation to the local working directory on the build agents system).
>  Each top-level build in AHP gets assigned a new (empty) build life.
>  Artifacts are always published to/resolved from a build life, either that
> of the current build, or of a dependency build.
>
> So what I did was I setup builds for Geronimo Server (the normal
> server/trunk stuff), which did the normal mvn install thingy, but I always
> gave it a custom -Dmaven.local.repository which resolved to something inside
> the working directory for the running build.  The build was still online, so
> it pulled down a bunch of stuff into an empty local repository (so it was a
> clean build wrt the repository, as well as the source code, which was always
> fetched for each new build).  Once the build had finished, I used the
> artifact publisher task to push *all* of the stuff in the local repository
> into Codestation, labled as something like "Maven repository artifacts" for
> the current build life.
>
> Then I setup another build for Apache Geronimo CTS Server (the
> porting/branches/* stuff).  This build was dependent upon the "Maven
> repository artifacts" of the Geronimo Server build, and I configured those
> artifacts to get installed on the build agents system in the same directory
> that I configured the CTS Server build to use for its local maven
> repository.  So again the repo started out empty, then got populated with
> all of the outputs from the normal G build, and then the cts-server build
> was started.  The build of the components and assemblies is normally fairly
> quick and aside from some stuff in the private tck repo won't download muck
> more stuff, because it already had most of its dependencies installed via
> the Codestation dependency resolution.   Once the build finished, I
> published to cts-server assembly artifacts back to Codestation under like
> "CTS Server Assemblies" or something.
>
> Up until this point its normal builds, but now we have built the G server,
> then built the CTS server (using the *exact* artifacts from the G server
> build, even though each might have happened on a different build agent).
>  And now we need to go and run a bunch of tests, using the *exact* CTS
> server assemblies, produce some output, collect it, and once all of

Re: Continuous TCK Testing

2008-10-02 Thread Jason Dillon


On Oct 1, 2008, at 11:20 PM, Jason Warner wrote:

Is the GBuild stuff in svn the same as the anthill-based code or is  
that something different?  GBuild seems to have scripts for running  
tck and that leads me to think they're the same thing, but I see no  
mention of anthill in the code.


The Anthill stuff is completely different than the GBuild stuff.  I  
started out trying to get the TCK automated using GBuild, but decided  
that the system lacked too many features to perform as I desired, and  
went ahead with Anthill as it did pretty much everything, though had  
some stability problems.


One of the main reasons why I choose Anthill (AHP, Anthill Pro that  
is) was its build agent and code repository systems.  This allowed me  
to ensure that each build used exactly the desired artifacts.  Another  
was the configurable workflow, which allowed me to create a custom  
chain of events to handle running builds on remote agents and control  
what data gets set to them, what it will collect and what logic to  
execute once all distributed work has been completed for a particular  
build.  And the kicker which help facilitate bringing it all together  
was its concept of a build life.


At the time I could find *no other* build tool which could meet all of  
these needs, and so I went with AHP instead of spending months  
building/testing features in GBuild.


While AHP supports configuring a lot of stuff via its web-interface, I  
found that it was very cumbersome, so I opted to write some glue,  
which was stored in svn here:


https://svn.apache.org/viewvc/geronimo/sandbox/build-support/?pathrev=632245

Its been a while, so I have to refresh my memory on how this stuff  
actually worked.  First let me explain about the code repository (what  
it calls codestation) and why it was critical to the TCK testing IMO.   
When we use Maven normally, it pulls data from a set of external  
repositories, picks up more repositories from the stuff it downloads  
and quickly we loose control where stuff comes from.  After it pulls  
down all that stuff, it churns though a build and spits out the stuff  
we care about, normally stuffing them (via mvn install) into the local  
repository.


AHP supports by default tasks to publish artifacts (really just a set  
of files controlled by an Ant-like include/exclude path) from a build  
agent into Codestation, as well as tasks to resolve artifacts (ie.  
download them from Codestation to the local working directory on the  
build agents system).  Each top-level build in AHP gets assigned a new  
(empty) build life.  Artifacts are always published to/resolved from a  
build life, either that of the current build, or of a dependency build.


So what I did was I setup builds for Geronimo Server (the normal  
server/trunk stuff), which did the normal mvn install thingy, but I  
always gave it a custom -Dmaven.local.repository which resolved to  
something inside the working directory for the running build.  The  
build was still online, so it pulled down a bunch of stuff into an  
empty local repository (so it was a clean build wrt the repository, as  
well as the source code, which was always fetched for each new  
build).  Once the build had finished, I used the artifact publisher  
task to push *all* of the stuff in the local repository into  
Codestation, labled as something like "Maven repository artifacts" for  
the current build life.


Then I setup another build for Apache Geronimo CTS Server (the porting/ 
branches/* stuff).  This build was dependent upon the "Maven  
repository artifacts" of the Geronimo Server build, and I configured  
those artifacts to get installed on the build agents system in the  
same directory that I configured the CTS Server build to use for its  
local maven repository.  So again the repo started out empty, then got  
populated with all of the outputs from the normal G build, and then  
the cts-server build was started.  The build of the components and  
assemblies is normally fairly quick and aside from some stuff in the  
private tck repo won't download muck more stuff, because it already  
had most of its dependencies installed via the Codestation dependency  
resolution.   Once the build finished, I published to cts-server  
assembly artifacts back to Codestation under like "CTS Server  
Assemblies" or something.


Up until this point its normal builds, but now we have built the G  
server, then built the CTS server (using the *exact* artifacts from  
the G server build, even though each might have happened on a  
different build agent).  And now we need to go and run a bunch of  
tests, using the *exact* CTS server assemblies, produce some output,  
collect it, and once all of the tests are done render some nice  
reports, etc.


AHP supports setting up builds which contain "parallel" tasks, each of  
those tasks is then performed by a build agent, they have fancy build  
agent selection stuff, but for my needs I had basically

Re: Continuous TCK Testing

2008-10-01 Thread Jason Dillon

sorry been busy, I will write up some details tomorrow about this...  
but its late now and the sleep fairy is calling me.


--jason


On Oct 1, 2008, at 8:56 PM, Kevan Miller wrote:



Not seeing too much progress here.  Has anyone dug up the Anthill- 
based code? I'll have a look.


--kevan

Re: Continuous TCK Testing

2008-10-01 Thread Jason Warner

Is the GBuild stuff in svn the same as the anthill-based code or is that
something different?  GBuild seems to have scripts for running tck and that
leads me to think they're the same thing, but I see no mention of anthill in
the code.

On Wed, Oct 1, 2008 at 9:56 AM, Kevan Miller <[EMAIL PROTECTED]> wrote:

>
> Not seeing too much progress here.  Has anyone dug up the Anthill-based
> code? I'll have a look.
>
> --kevan
>

-- 
~Jason Warner

Re: Continuous TCK Testing

2008-10-01 Thread Kevan Miller



Not seeing too much progress here.  Has anyone dug up the Anthill- 
based code? I'll have a look.


--kevan

Re: Continuous TCK Testing

2008-09-19 Thread Kevan Miller



On Sep 19, 2008, at 12:05 PM, David Blevins wrote:



On Sep 18, 2008, at 3:40 PM, Jay D. McHugh wrote:


I would like to be involved too.

But, I don't have any experience with either AntHill or Hudson.

Has anyone used Continuum?  Would that be harder/easier to  
configure and

use than the other two?


The first version of GBuild ran a mashup of Continuum/ActiveMQ/ 
Maven.  No GUI aside from the TCK reports though, it just used the  
backend part of continuum.  It ran fine till the machines got hacked  
(not related to the testing software).


Essentially the TCK was divided up into several individual build  
definitions and each of them were put on a JMS queue.  Each of the  
build agents would pull a build definition from the queue, run it  
via the continuum backend, then send the results onto a JMS topic.   
There was another agent listening on the topic who would grab the  
results, add them to the rest of the results then rerun the report  
tool.


It wouldn't be hard for us to set that up again, we'd just need VMs  
(the OS kind, not the java kind) on our two boxes so we can get more  
parallel processing without port conflicts.


Anyone know if there are these kind of VMs setup already?


Yes. Each box has 4 VM's :-) phoebe (tck01-04) and selene (tck05-08).

--kevan

Re: Continuous TCK Testing

2008-09-19 Thread Kevan Miller



On Sep 19, 2008, at 12:26 PM, Donald Woods wrote:

Can we really setup "continuous" TCK testing?  How long does it take  
2 VMs to complete a TCK run?


I didn't mean to imply continuous integration-style testing. I think  
the rate of change on an branch/trunk under active development is  
likely going to exceed our ability to test every change.




Shouldn't running the TCK prereq a Continuous Build first, by moving  
the automated builds that Jarek runs over to these machines and only  
run a build when there are svn changes in that branch (instead of a  
time based approach)?  That would only generate builds on active  
branches (like 2.1 and trunk) while increasing the overall usage of  
these 2 machines to show that we're really using them.


It could, but it's not necessary. I'd be happy to see once a day  
testing. This isn't about generating load on the boxes -- it's about  
generating value to the community.




Also, which releases of Geronimo would we setup for automated TCK  
runs?


Certainly 2.2 and 2.1 (IMO). We could throw in 2.0 if we have capacity  
and someone wants to set it up.


We've also had requests from other projects to get some time (e.g.  
CXF). I'd be happy to see this happening, also.


--kevan

Re: Continuous TCK Testing

2008-09-19 Thread Donald Woods

Can we really setup "continuous" TCK testing?  How long does it take 2 
VMs to complete a TCK run?


Shouldn't running the TCK prereq a Continuous Build first, by moving the 
automated builds that Jarek runs over to these machines and only run a 
build when there are svn changes in that branch (instead of a time based 
approach)?  That would only generate builds on active branches (like 2.1 
and trunk) while increasing the overall usage of these 2 machines to 
show that we're really using them.


Also, which releases of Geronimo would we setup for automated TCK runs?


-Donald

Kevan Miller wrote:
As many of you know, we have two apache.org machines for TCK testing of 
Geronimo (phoebe and selene). We used the machines for certification of 
our 2.1.3 release. However, this testing was run manually. It's time to 
get continuous, automatic TCK testing running on these machines.


We had the basic setup running on GBuild machines. IIRC, this was built 
around AntHill -- Jason Dillon knows it best. There are other testing 
systems available (e.g. Hudson) which could be used for this task.


What do others think? What underlying technology should we use? Who 
wants to get involved?


I think this discussion should be on our dev@ mailing list. TCK should 
be for test specific discussions.


--kevan

Re: Continuous TCK Testing

2008-09-19 Thread David Blevins



On Sep 18, 2008, at 3:40 PM, Jay D. McHugh wrote:


I would like to be involved too.

But, I don't have any experience with either AntHill or Hudson.

Has anyone used Continuum?  Would that be harder/easier to configure  
and

use than the other two?


The first version of GBuild ran a mashup of Continuum/ActiveMQ/Maven.   
No GUI aside from the TCK reports though, it just used the backend  
part of continuum.  It ran fine till the machines got hacked (not  
related to the testing software).


Essentially the TCK was divided up into several individual build  
definitions and each of them were put on a JMS queue.  Each of the  
build agents would pull a build definition from the queue, run it via  
the continuum backend, then send the results onto a JMS topic.  There  
was another agent listening on the topic who would grab the results,  
add them to the rest of the results then rerun the report tool.


It wouldn't be hard for us to set that up again, we'd just need VMs  
(the OS kind, not the java kind) on our two boxes so we can get more  
parallel processing without port conflicts.


Anyone know if there are these kind of VMs setup already?

-David

Re: Continuous TCK Testing

2008-09-18 Thread Jay D. McHugh

I would like to be involved too.

But, I don't have any experience with either AntHill or Hudson.

Has anyone used Continuum?  Would that be harder/easier to configure and
use than the other two?

Jay

Kevan Miller wrote:
> As many of you know, we have two apache.org machines for TCK testing of
> Geronimo (phoebe and selene). We used the machines for certification of
> our 2.1.3 release. However, this testing was run manually. It's time to
> get continuous, automatic TCK testing running on these machines.
> 
> We had the basic setup running on GBuild machines. IIRC, this was built
> around AntHill -- Jason Dillon knows it best. There are other testing
> systems available (e.g. Hudson) which could be used for this task.
> 
> What do others think? What underlying technology should we use? Who
> wants to get involved?
> 
> I think this discussion should be on our dev@ mailing list. TCK should
> be for test specific discussions.
> 
> --kevan
>

Re: Continuous TCK Testing

2008-09-18 Thread Jason Warner

I am all for this idea.  I don't really know about anthill but I did look
into hudson a little bit a while ago.  It seems like it would meet our
requirements but I haven't actually used the technology so I'm not sure how
well it works.  If we already have a solution, I'm not sure why we wouldn't
go with that, though.  Is there any reason we wouldn't want to use a setup
similar to what we had before using anthill?

I'd be happy to get involved, especially if it makes it easier to verify tck
when release time comes around.

I'm kind of curious about how this would work, though.  I assume someone
would commit some code and this would trigger a tck test with the new code.
How would errors be reported?  Would we have to have all committers sign the
NDA so we can report details of failures to whoever committed the changes?

On Thu, Sep 18, 2008 at 1:50 PM, Kevan Miller <[EMAIL PROTECTED]>wrote:

> As many of you know, we have two apache.org machines for TCK testing of
> Geronimo (phoebe and selene). We used the machines for certification of our
> 2.1.3 release. However, this testing was run manually. It's time to get
> continuous, automatic TCK testing running on these machines.
>
> We had the basic setup running on GBuild machines. IIRC, this was built
> around AntHill -- Jason Dillon knows it best. There are other testing
> systems available (e.g. Hudson) which could be used for this task.
>
> What do others think? What underlying technology should we use? Who wants
> to get involved?
>
> I think this discussion should be on our dev@ mailing list. TCK should be
> for test specific discussions.
>
> --kevan
>
>

-- 
~Jason Warner

Re: Continuous TCK Testing

Re: Continuous TCK Testing

Re: Continuous TCK Testing

Re: Continuous TCK Testing

Re: Continuous TCK Testing

Re: Continuous TCK Testing

Re: Continuous TCK Testing

Re: Continuous TCK Testing

Re: Continuous TCK Testing

Re: Continuous TCK Testing

Re: Continuous TCK Testing

Re: Continuous TCK Testing

Re: Continuous TCK Testing

Re: Continuous TCK Testing

Re: Continuous TCK Testing

Re: Continuous TCK Testing

Re: Continuous TCK Testing

Re: Continuous TCK Testing

Re: Continuous TCK Testing

Re: Continuous TCK Testing

Re: Continuous TCK Testing

Re: Continuous TCK Testing

Re: Continuous TCK Testing

Re: Continuous TCK Testing

Re: Continuous TCK Testing

Re: Continuous TCK Testing

Re: Continuous TCK Testing

27 matches

Site Navigation

Mail list logo

Footer information