Re: [yocto] [[PATCH][yocto-autobuilder]] bin/forcebuild.py: Add a script to support force a build

[Lock, Joshua G]

On Wed, 2017-02-08 at 09:10 -0600, Aníbal Limón wrote:
> 
> On 02/08/2017 08:12 AM, Lock, Joshua G wrote:
> > On Wed, 2017-02-01 at 14:14 -0600, Aníbal Limón wrote:
> > > I know we have the remote_kick.py [1] script but i don't know if
> > > is
> > > currently on use also this new script support list the options in
> > > builders and specify custom ones via cmdline.
> > 
> > I don't use remote_kick.py, I'm loathe to carry two scripts with a
> > similar purpose. Any idea how they differ?
> 
> The remote_kick script has a fixed set of parameters like builder
> (nigthly), buildbot url (localhost:8010) and forces the same build.
> 
> The new forcebuild script you can specify buildbot server url, user,
> password, builder and options (WEB inputs).

Thank you. I've queued up this patch for inclusion with a follow-on
patch that removes remote_kick with the above as reasoning.

Joshua
-
Intel Corporation (UK) Limited
Registered No. 1134945 (England)
Registered Office: Pipers Way, Swindon SN3 1RJ
VAT No: 860 2173 47

This e-mail and any attachments may contain confidential material for
the sole use of the intended recipient(s). Any review or distribution
by others is strictly prohibited. If you are not the intended
recipient, please contact the sender and delete all copies.
-- 
___
yocto mailing list
yocto@yoctoproject.org
https://lists.yoctoproject.org/listinfo/yocto

Re: [yocto] [[PATCH][yocto-autobuilder]] bin/forcebuild.py: Add a script to support force a build

[Aníbal Limón]

On 02/08/2017 08:12 AM, Lock, Joshua G wrote:
> On Wed, 2017-02-01 at 14:14 -0600, Aníbal Limón wrote:
>> I know we have the remote_kick.py [1] script but i don't know if is
>> currently on use also this new script support list the options in
>> builders and specify custom ones via cmdline.
> 
> I don't use remote_kick.py, I'm loathe to carry two scripts with a
> similar purpose. Any idea how they differ?

The remote_kick script has a fixed set of parameters like builder
(nigthly), buildbot url (localhost:8010) and forces the same build.

The new forcebuild script you can specify buildbot server url, user,
password, builder and options (WEB inputs).

Cheers,
alimon

> 
> Thanks,
> 
> Joshua
> 
>> Cheers,
>>  alimon
>>
>> [1]
>> http://git.yoctoproject.org/cgit/cgit.cgi/yocto-autobuilder/tree/bin/
>> remote_kick.py
>>
>> On 02/01/2017 02:12 PM, Aníbal Limón wrote:
>>> The script supports to list avaialable builders and his options,
>>> also can force a build if the builder is IDLE or not.
>>>
>>> Also supports specify options via cmdline using python dictionaries
>>> as string.
>>>
>>> Signed-off-by: Aníbal Limón 
>>> ---
>>>  bin/forcebuild.py | 183
>>> ++
>>>  1 file changed, 183 insertions(+)
>>>  create mode 100755 bin/forcebuild.py
>>>
>>> diff --git a/bin/forcebuild.py b/bin/forcebuild.py
>>> new file mode 100755
>>> index 000..5fe61a0
>>> --- /dev/null
>>> +++ b/bin/forcebuild.py
>>> @@ -0,0 +1,183 @@
>>> +#!/usr/bin/env python
>>> +
>>> +# YoctoAutobuilder force build script
>>> +#
>>> +# Copyright (C) 2017 Intel Corporation
>>> +#
>>> +# This program is free software; you can redistribute it and/or
>>> modify
>>> +# it under the terms of the GNU General Public License version 2
>>> as
>>> +# published by the Free Software Foundation.
>>> +#
>>> +# This program is distributed in the hope that it will be useful,
>>> +# but WITHOUT ANY WARRANTY; without even the implied warranty of
>>> +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
>>> +# GNU General Public License for more details.
>>> +#
>>> +# You should have received a copy of the GNU General Public
>>> License along
>>> +# with this program; if not, write to the Free Software
>>> Foundation, Inc.,
>>> +# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
>>> +
>>> +import urllib2
>>> +import urllib
>>> +import cookielib
>>> +import uuid
>>> +import sys
>>> +import argparse
>>> +import json
>>> +from bs4 import BeautifulSoup
>>> +
>>> +class YoctoAutobuilderAPI(object):
>>> +def __init__(self, server):
>>> +self.server = server
>>> +self.opener = urllib2.build_opener(
>>> +urllib2.HTTPCookieProcessor(cookielib.CookieJar())
>>> )
>>> +
>>> +def login(self, user, passwd):
>>> +data = urllib.urlencode(dict(username=user,
>>> + passwd=passwd))
>>> +url = self.server + "/login"
>>> +request = urllib2.Request(url, data)
>>> +result = self.opener.open(request).read()
>>> +if result.find("The username or password you entered were
>>> not correct") > 0:
>>> +print("Invalid username or password")
>>> +return 1
>>> +return 0
>>> +
>>> +def _get_builders(self):
>>> +url = "%s/json/builders/?as_text=1" % (self.server)
>>> +request = urllib2.Request(url)
>>> +json_raw_data = self.opener.open(request)
>>> +
>>> +return json.load(json_raw_data)
>>> +
>>> +def list_builders(self):
>>> +builders = self._get_builders()
>>> +
>>> +if builders:
>>> +print('Available builders:\n')
>>> +for builder in builders:
>>> +print(builder)
>>> +return 0
>>> +else:
>>> +print('No available builders in this server.')
>>> +return 1
>>> +
>>> +def _get_options_by_builder(self, builder):
>>> +options = {}
>>> +
>>> +url = "%s/builders/%s" % (self.server, builder)
>>> +request = urllib2.Request(url)
>>> +html = self.opener.open(request).read()
>>> +
>>> +inputs = BeautifulSoup(html, 'lxml').find_all('input')
>>> +for input in inputs:
>>> +type = input.attrs['type']
>>> +if  type == 'submit':
>>> +continue
>>> +
>>> +options[input.attrs['name']] = input.attrs['value']
>>> +
>>> +selects = BeautifulSoup(html, 'lxml').find_all('select')
>>> +for select in selects:
>>> +value = select.find_all('option',
>>> selected=True)[0].getText()
>>> +options[select.attrs['name']] = value
>>> +
>>> +return options
>>> +
>>> +def list_options(self, builder):
>>> +builders = self._get_builders()
>>> +if not builder in builders:
>>> +print('Builder %s specified isn\'t available' %
>>> builder)
>>> +

Re: [yocto] [[PATCH][yocto-autobuilder]] bin/forcebuild.py: Add a script to support force a build

[Lock, Joshua G]

On Wed, 2017-02-01 at 14:14 -0600, Aníbal Limón wrote:
> I know we have the remote_kick.py [1] script but i don't know if is
> currently on use also this new script support list the options in
> builders and specify custom ones via cmdline.

I don't use remote_kick.py, I'm loathe to carry two scripts with a
similar purpose. Any idea how they differ?

Thanks,

Joshua

> Cheers,
>   alimon
> 
> [1]
> http://git.yoctoproject.org/cgit/cgit.cgi/yocto-autobuilder/tree/bin/
> remote_kick.py
> 
> On 02/01/2017 02:12 PM, Aníbal Limón wrote:
> > The script supports to list avaialable builders and his options,
> > also can force a build if the builder is IDLE or not.
> > 
> > Also supports specify options via cmdline using python dictionaries
> > as string.
> > 
> > Signed-off-by: Aníbal Limón 
> > ---
> >  bin/forcebuild.py | 183
> > ++
> >  1 file changed, 183 insertions(+)
> >  create mode 100755 bin/forcebuild.py
> > 
> > diff --git a/bin/forcebuild.py b/bin/forcebuild.py
> > new file mode 100755
> > index 000..5fe61a0
> > --- /dev/null
> > +++ b/bin/forcebuild.py
> > @@ -0,0 +1,183 @@
> > +#!/usr/bin/env python
> > +
> > +# YoctoAutobuilder force build script
> > +#
> > +# Copyright (C) 2017 Intel Corporation
> > +#
> > +# This program is free software; you can redistribute it and/or
> > modify
> > +# it under the terms of the GNU General Public License version 2
> > as
> > +# published by the Free Software Foundation.
> > +#
> > +# This program is distributed in the hope that it will be useful,
> > +# but WITHOUT ANY WARRANTY; without even the implied warranty of
> > +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> > +# GNU General Public License for more details.
> > +#
> > +# You should have received a copy of the GNU General Public
> > License along
> > +# with this program; if not, write to the Free Software
> > Foundation, Inc.,
> > +# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
> > +
> > +import urllib2
> > +import urllib
> > +import cookielib
> > +import uuid
> > +import sys
> > +import argparse
> > +import json
> > +from bs4 import BeautifulSoup
> > +
> > +class YoctoAutobuilderAPI(object):
> > +def __init__(self, server):
> > +self.server = server
> > +self.opener = urllib2.build_opener(
> > +urllib2.HTTPCookieProcessor(cookielib.CookieJar())
> > )
> > +
> > +def login(self, user, passwd):
> > +data = urllib.urlencode(dict(username=user,
> > + passwd=passwd))
> > +url = self.server + "/login"
> > +request = urllib2.Request(url, data)
> > +result = self.opener.open(request).read()
> > +if result.find("The username or password you entered were
> > not correct") > 0:
> > +print("Invalid username or password")
> > +return 1
> > +return 0
> > +
> > +def _get_builders(self):
> > +url = "%s/json/builders/?as_text=1" % (self.server)
> > +request = urllib2.Request(url)
> > +json_raw_data = self.opener.open(request)
> > +
> > +return json.load(json_raw_data)
> > +
> > +def list_builders(self):
> > +builders = self._get_builders()
> > +
> > +if builders:
> > +print('Available builders:\n')
> > +for builder in builders:
> > +print(builder)
> > +return 0
> > +else:
> > +print('No available builders in this server.')
> > +return 1
> > +
> > +def _get_options_by_builder(self, builder):
> > +options = {}
> > +
> > +url = "%s/builders/%s" % (self.server, builder)
> > +request = urllib2.Request(url)
> > +html = self.opener.open(request).read()
> > +
> > +inputs = BeautifulSoup(html, 'lxml').find_all('input')
> > +for input in inputs:
> > +type = input.attrs['type']
> > +if  type == 'submit':
> > +continue
> > +
> > +options[input.attrs['name']] = input.attrs['value']
> > +
> > +selects = BeautifulSoup(html, 'lxml').find_all('select')
> > +for select in selects:
> > +value = select.find_all('option',
> > selected=True)[0].getText()
> > +options[select.attrs['name']] = value
> > +
> > +return options
> > +
> > +def list_options(self, builder):
> > +builders = self._get_builders()
> > +if not builder in builders:
> > +print('Builder %s specified isn\'t available' %
> > builder)
> > +return 1
> > +
> > +opts = self._get_options_by_builder(builder)
> > +print('Available options in builder %s:\n' % (builder))
> > +for name in opts:
> > +value = opts[name]
> > +print('Name: %s, Default value: %s' % (name, value))
> > +
> > +def force_build(self, builder, opts, idle_check=False):

Re: [yocto] [[PATCH][yocto-autobuilder]] bin/forcebuild.py: Add a script to support force a build

[Aníbal Limón]

I know we have the remote_kick.py [1] script but i don't know if is
currently on use also this new script support list the options in
builders and specify custom ones via cmdline.

Cheers,
alimon

[1]
http://git.yoctoproject.org/cgit/cgit.cgi/yocto-autobuilder/tree/bin/remote_kick.py

On 02/01/2017 02:12 PM, Aníbal Limón wrote:
> The script supports to list avaialable builders and his options,
> also can force a build if the builder is IDLE or not.
> 
> Also supports specify options via cmdline using python dictionaries
> as string.
> 
> Signed-off-by: Aníbal Limón 
> ---
>  bin/forcebuild.py | 183 
> ++
>  1 file changed, 183 insertions(+)
>  create mode 100755 bin/forcebuild.py
> 
> diff --git a/bin/forcebuild.py b/bin/forcebuild.py
> new file mode 100755
> index 000..5fe61a0
> --- /dev/null
> +++ b/bin/forcebuild.py
> @@ -0,0 +1,183 @@
> +#!/usr/bin/env python
> +
> +# YoctoAutobuilder force build script
> +#
> +# Copyright (C) 2017 Intel Corporation
> +#
> +# This program is free software; you can redistribute it and/or modify
> +# it under the terms of the GNU General Public License version 2 as
> +# published by the Free Software Foundation.
> +#
> +# This program is distributed in the hope that it will be useful,
> +# but WITHOUT ANY WARRANTY; without even the implied warranty of
> +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> +# GNU General Public License for more details.
> +#
> +# You should have received a copy of the GNU General Public License along
> +# with this program; if not, write to the Free Software Foundation, Inc.,
> +# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
> +
> +import urllib2
> +import urllib
> +import cookielib
> +import uuid
> +import sys
> +import argparse
> +import json
> +from bs4 import BeautifulSoup
> +
> +class YoctoAutobuilderAPI(object):
> +def __init__(self, server):
> +self.server = server
> +self.opener = urllib2.build_opener(
> +urllib2.HTTPCookieProcessor(cookielib.CookieJar()))
> +
> +def login(self, user, passwd):
> +data = urllib.urlencode(dict(username=user,
> + passwd=passwd))
> +url = self.server + "/login"
> +request = urllib2.Request(url, data)
> +result = self.opener.open(request).read()
> +if result.find("The username or password you entered were not 
> correct") > 0:
> +print("Invalid username or password")
> +return 1
> +return 0
> +
> +def _get_builders(self):
> +url = "%s/json/builders/?as_text=1" % (self.server)
> +request = urllib2.Request(url)
> +json_raw_data = self.opener.open(request)
> +
> +return json.load(json_raw_data)
> +
> +def list_builders(self):
> +builders = self._get_builders()
> +
> +if builders:
> +print('Available builders:\n')
> +for builder in builders:
> +print(builder)
> +return 0
> +else:
> +print('No available builders in this server.')
> +return 1
> +
> +def _get_options_by_builder(self, builder):
> +options = {}
> +
> +url = "%s/builders/%s" % (self.server, builder)
> +request = urllib2.Request(url)
> +html = self.opener.open(request).read()
> +
> +inputs = BeautifulSoup(html, 'lxml').find_all('input')
> +for input in inputs:
> +type = input.attrs['type']
> +if  type == 'submit':
> +continue
> +
> +options[input.attrs['name']] = input.attrs['value']
> +
> +selects = BeautifulSoup(html, 'lxml').find_all('select')
> +for select in selects:
> +value = select.find_all('option', selected=True)[0].getText()
> +options[select.attrs['name']] = value
> +
> +return options
> +
> +def list_options(self, builder):
> +builders = self._get_builders()
> +if not builder in builders:
> +print('Builder %s specified isn\'t available' % builder)
> +return 1
> +
> +opts = self._get_options_by_builder(builder)
> +print('Available options in builder %s:\n' % (builder))
> +for name in opts:
> +value = opts[name]
> +print('Name: %s, Default value: %s' % (name, value))
> +
> +def force_build(self, builder, opts, idle_check=False):
> +builders = self._get_builders()
> +if not builder in builders:
> +print('Builder specified isn\'t available')
> +return 1
> +
> +state = builders[builder]['state']
> +if idle_check and not state == 'idle':
> +print('Builder %s specified isn\'t IDLE, current state %s' \
> +% (builder, state))
> +return 1
> +
> +opts = eval(opts) # FIXME: transform string