2008/10/22 Emmanuel Venisse <[EMAIL PROTECTED]>:
> Before to start on distributed builds (I want it in the next version), I
> think we must work on some things before.
> - the build system must be totally independant of the DB
> - the build system must sent events for each steps so Continuum will know
> the status and if the DB should be updated
> - We must refactor the build system with a plugin mechanism like maven
> (Olivier has started to work on a proposal)
My bad only on a paper :-)
>
> Then we'll can play with distributed builds.
> I added some comment below.
>
> On Wed, Oct 22, 2008 at 4:07 AM, Marica Tan <[EMAIL PROTECTED]> wrote:
>
>> Hi,
>>
>> Our team in exist came up with a requirements for distributed builds.
>>
>> *A. Distribution of work*
>>
>> Master and Slave Agent will communicate with each other through
>> xml-rpc.
>
> I'm not sure xmlrpc is the best choice for the communication.
> 1- the communication between the master and slaves must be secured
> 2- when Continuum master is updated (new version), I think it can be good to
> update all slaves too
>
> A solution would can be to store the agent binaries in the master, so if you
> update the master, agent binaries will be updated too. For the
> communication, the master would can open a ssh connection to the slave
> machine and start the agent remotely, the agent can be installed by the
> master automatically.
> With that, the communication is totally secured, configured in the master
> and the agent is always up-to-date with the master version.
>
>
>>
>> 1. Master Agent - controls build via xml-rpc
>> 1. slaves need to be added to the master server
>> - attach the slave to a build environment as a type of
>> "installation". Each slave reports a number of available
>> installations.
>> 2. check availability of slaves
>> - slave is available if it's up and has no current build.
>> - if slave is available then dispatch build to the slave.
>> - if slave is unavailable then enqueue build on the master
>
> I'm not totally agree. With a build/slaves mechanism, you can choose to
> delegate the build to an available slave, but if it's the first time a build
> is assigned for a specific project to this node, a full checkout will be
> required on the node even if the checkout was already done on an other node.
> The disk usage is potentially the size of all projects X nb nodes.
> Some users will want to use always the same node for all builds of a project
> or attach a specific node to a project group.
>
> What about parallel build?
>
>>
>> 3. distribute work
>> - select slave based on build environment (master)
>> - build
>> - pass current configuration
>> - expect result (master)/return result (slave)
>> - attach each result returned by the slave to each project in
>> the group (add to database)
>> 2. Slave Agent - is a dumb build agent that will run on another
>> build server. It will only build the project and return the result. It
>> also
>> has it's own installation.
>> - queried for available installations
>> - start build, stop build
>> - returns build result
>>
>
> I think we'll need an "internal" remote repository used by all nodes. If the
> build definition run only "mvn clean install", the artifact will be stored
> on the local repository of the node and other nodes will can't use the new
> artifact if we don't use our remote repository.
>
>>
>> *B. UI/Configuration*
>>
>> 1. Aggregated view of distributed projects
>
>
> Can you add more informations? Will it modify the aspect of the actual
> pages?
>
>>
>> 2. Slave server list (new page)
>> 3. Build Environment
>> - add slave selection
>> - modify installations based on slave selection
>> 4. distributed build configuration stored and editable in XML
>> configuration file
>>
>> Limitations
>>
>> - credentials (s.a. svn credentials) are passed along if specified, but
>> if server cache is used it will need to be done individually on the
>> slaves
>> - slaves can only be configured at the project group level, by a system
>> administrator
>> - project dependencies are not considered in distribution -
>> interdependent projects should target the same slave server
>>
>> Future Enhancements
>>
>> 1. Policy-based distribution
>> - next available
>> - load balanced
>> - target environment matching
>>
>>
>> Comments, suggestions and/or violent reactions are welcome :)
>
>
> Can you add the proposal in the wiki?
> Thanks
>
> Emmanuel
>
>>
>>
>>
>> Thanks,
>> --
>> Marica
>>
>
