On 31 May 2013 16:25, Neil Williams <codeh...@debian.org> wrote:
> On Fri, 31 May 2013 15:50:54 +0100
> James Tunnicliffe <james.tunnicli...@linaro.org> wrote:
>
>> On 31 May 2013 14:22, Paul Sokolovsky <paul.sokolov...@linaro.org> wrote:
>> > On Wed, 29 May 2013 17:11:11 +0100
>> > James Tunnicliffe <james.tunnicli...@linaro.org> wrote:
>> >
>> >> Issues raised in
>> >> http://bazaar.launchpad.net/~linaro-automation/linaro-android-build-tools/trunk/view/head:/utils/new-publish/README
>
> This is very similar to what the Multi-Node parent <-> child
> communication may need.
>
>> >> Getting a token:
>> >> I see this as the service that starts the job has some secret allowing
>> >> it to request the token, which is then passed onto the job.
>
> Precisely as the Multi-Node setup could use. I'm thinking the token
> could be added as part of the jobdata in lava-dispatcher, commands.py:
> job = LavaTestJob(jobdata, oob_file, config, self.args.output_dir)
>
> My expectation for this Multi-Node support would be that the token
> would be written to the test environment and the test would use that to
> get the list of other clients for this multi-node job. Once each client
> has booted and "called home" to the parent using the token, clients
> could get the IP address and role of other clients. This then allows
> Multi-Node tests to make calls between clients to test their own
> protocols and other requirements.

Cool. Looks like if one of us finds or invents a library to do this
then we should share. I did use a REST library with Django before that
did some token based auth, but it was a bit lacking in features where
I needed them.

Linaro-license-protection (which runs snapshots and releases) uses a
SQLite database to store some data so if I wrote something (and
frankly not writing something right now just as an experiment seems
impossible) I would likely store a user: key mapping in there and have
a nice interface to request a new key and delete old ones. Essentially
it sounds like a persistent dictionary though. (I know that this
doesn't cover group based stuff, but that doesn't matter for a proof
of concept hack and would be simple enough to add).

>> I already have a protocol designed to do the file update thing (server
>> has old file, client has new one but not old one) for a pet project
>> that I was going to open source, but it is just a bit of fun and
>> hasn't been tested, so even having got that far I would still tell
>> other people to use rsync.
>>
>> >> publish --token=<token> --type=<build_type> --strip=<strip> <build_id>
>> >> <glob_pattern>...
>> >>
>> >> This seems like a reasonable starting point. Lets make sure that it
>> >> uses a configuration file to specify what to do with those build types
>> >> etc. Preferably one that it can update from a public location so we
>> >> don't have to re-spin the tool to add a new build type (though I guess
>> >> we normally check it out of VCS as we go, so that works too).
>> >
>> > Well, on client side, it's ideally just a single file which just handle
>> > obvious filtering options (like <glob_pattern> or --strip=) locally and
>> > passes the rest to API/service. Server-side can handle the options in
>> > any way it wants, note that options above don't require much
>> > "configuration", for example --type= just maps to top-level download
>> > dir.
>
> Just wondering how much overlap there could be between publishing for
> CI and publishing IP addresses between clients running a Multi-Node job.
>
> I've only done the briefest look at the Multi-Node aspects so far,
> there's a PDF I've shared with some rough flow ideas for
> Parent-Child-Communication.

It sounds like a similar problem. Writing a Django app that receives a
GET /<job ID> and returns a JSON blob of metadata is simple to
implement and I would be happy to hack something together to get you
started or pair program with you if that would help. Adding in a layer
of security and the ability for the clients to put updates to the
server would be easy as well.

--
James Tunnicliffe

_______________________________________________
linaro-validation mailing list
linaro-validation@lists.linaro.org
http://lists.linaro.org/mailman/listinfo/linaro-validation

Reply via email to