Re: [yocto] Buildbot / Autobuilder / custom?

2017-05-31 Thread Joshua Lock 
On Mon, 2017-05-29 at 17:01 -0400, Alain Achkar wrote:
> Hello Beth and/or Joshua,
> I would really appreciate some pointers on how to get the Variscite
> builds to work with Autobuilder.
> 
> Since my last email, I was able to setup the Yocto Autobuilder, add
> the sample hello.conf and run it.
> I was also able to add another file, nightly-variscite-x11.conf and I
> can see it in the Waterfall view. Initially, I started by filling it
> with content similar to hello.conf and this worked. Now, I need to be
> able to add the Variscite build.
> 
> Since my last email, Variscite have moved to the morty branch, since
> Freescale/NXP have moved to the morty branch. So, the wiki
> instructions here http://variwiki.com/index.php?title=Yocto_Build_Rel
> ease=RELEASE_MORTY_BETA_DART-6UL are now simpler:
> 
> 1) Checkout the source code
> $ mkdir ~/var-fslc-yocto
> $ cd ~/var-fslc-yocto
> $ repo init -u https://github.com/varigit/variscite-bsp-platform.git
> -b morty
> $ repo sync -j4
> 
> 2) Run the setup script (which calls oe-init-environment)
> $ cd ~/var-fslc-yocto
> $ MACHINE=imx6ul-var-dart DISTRO=fslc-x11 . setup-environment
> build_x11
> 
> 3) Build X11 demo images
> # Without Qt content:
> $ bitbake fsl-image-gui
> 
> # Or with Qt content:
> $ bitbake fsl-image-qt5
> 
> The 'repo' command reads the file default.xml which has the following
> contents:
> 
> 
> 
> 
> 
>   
> 
> 
>   https://git.yoctoproject.org/git; name="yocto"/>
>   https://github.com/Freescale;     name="freescale"/>
>   https://github.com/openembedded;  name="oe"/>
>   https://github.com/varigit;       name="variscite"/>
>   https://github.com/OSSystems;     name="OSSystems"/>
>   
> 
> 
>    revision="6a1f33cc40bfac33cf030fe41e1a8efd1e5fad6f" upstream="morty"
>  name="poky" path="sources/poky"/>
>    revision="56a1460777d1e24b1dfb426cad2b103740936a83" upstream="morty"
>  name="meta-freescale" path="sources/meta-freescale" />
> 
> 
>    revision="fe5c83312de11e80b85680ef237f8acb04b4b26e" upstream="morty"
> name="meta-openembedded" path="sources/meta-openembedded" />
> 
> 
>   
>   
>   
>   
> 
> 
>    revision="4be3db25959e741ac9164a32c82b8c09b59213ad" upstream="morty"
> name="meta-freescale-3rdparty" path="sources/meta-freescale-
> 3rdparty"/>
>    revision="dd482aeb7913c099c69f82362776bd92b584553c" upstream="morty"
> name="meta-freescale-distro" path="sources/meta-freescale-distro"/>
>    revision="3601fd2c5306ac6d5d0d536e0be8cbb90da9b4c1" upstream="morty"
> name="meta-qt5" path="sources/meta-qt5"/>
>    revision="7d37ae2adec5cc5ea27367ce125bc7bce2310ca1" upstream="morty"
> name="meta-browser" path="sources/meta-browser"/>
>    revision="7a214e7579869be810aa8992c59627ba064db076" upstream="morty"
> name="meta-variscite-fslc" path="sources/meta-variscite-fslc"  />
> 
> 
> 
> 
> 
> 
> So, AFAIK, I need to either (a) translate these into the 'repos'
> property in my buildset, or write a custom buildstep that can use the
> 'repo' tool to clone the needed repositories.

You are correct.

The other step you might have issues with is the setup environment
script, yocto-autobuilder currently expects to source oe-init-build-env 
and doesn't support other init scripts presently.

I have a WIP patch to support alternative environment scripts, but it
doesn't fully resolve support for your BSP as it seems from the example
above that MACHINE and DISTRO need to be set in the environment when
the script is sourced.
Will those values be picked up from auto.conf if not set in the
environment?

Joshua-- 
___
yocto mailing list
yocto@yoctoproject.org
https://lists.yoctoproject.org/listinfo/yocto

Re: [yocto] Buildbot / Autobuilder / custom?

2017-05-29 Thread Alain Achkar 
Hello Beth and/or Joshua,

I would really appreciate some pointers on how to get the Variscite builds
to work with Autobuilder.

Since my last email, I was able to setup the Yocto Autobuilder, add the
sample hello.conf and run it.
I was also able to add another file, nightly-variscite-x11.conf and I can
see it in the Waterfall view. Initially, I started by filling it with
content similar to hello.conf and this worked. Now, I need to be able to
add the Variscite build.

Since my last email, Variscite have moved to the morty branch, since
Freescale/NXP have moved to the morty branch. So, the wiki instructions
here
http://variwiki.com/index.php?title=Yocto_Build_Release=RELEASE_MORTY_BETA_DART-6UL
are now simpler:

1) Checkout the source code

$ mkdir ~/var-fslc-yocto
$ cd ~/var-fslc-yocto
$ repo init -u https://github.com/varigit/variscite-bsp-platform.git -b morty
$ repo sync -j4


2) Run the setup script (which calls oe-init-environment)

$ cd ~/var-fslc-yocto
$ MACHINE=imx6ul-var-dart DISTRO=fslc-x11 . setup-environment build_x11


3) Build X11 demo images

# Without Qt content:
$ bitbake fsl-image-gui

# Or with Qt content:
$ bitbake fsl-image-qt5


The 'repo' command reads the file default.xml which has the following
contents:




  

  https://git.yoctoproject.org/git; name="yocto"/>
  https://github.com/Freescale; name="freescale"/>
  https://github.com/openembedded;  name="oe"/>
  https://github.com/varigit;   name="variscite"/>
  https://github.com/OSSystems; name="OSSystems"/>
  

  
  

  

  


  

  
  
  
  
  




So, AFAIK, I need to either (a) translate these into the 'repos' property
in my buildset, or write a custom buildstep that can use the 'repo' tool to
clone the needed repositories.

Any help would be greatly appreciated.
Thanks!


On Fri, Apr 21, 2017 at 11:39 AM, Alain Achkar 
wrote:

> Kindly requesting help again, please.
> I understand that Variscite might not be following the "Yocto way" 100%
> but surely we can run custom commands using Autobuilder?
>
> Please see detailed description in my last post.
> Thanks!
>
> On Mon, Apr 3, 2017 at 9:43 PM, Alain Achkar 
> wrote:
>
>> Hi Chris!
>>
>> I had to put this task on hold and I am re-visiting it now. I followed
>> the instructions here:  http://git.yoctoproject
>> .org/cgit.cgi/yocto-autobuilder/tree/README-QUICKSTART and managed to
>> force one of the builds that comes with Yocto. I got errors but I'm not
>> interested in the Yocto-provided nightly builds anyway. (The errors are
>> related to buildhistory - it seems that some builds set Buildhistory=True
>> but I haven't set up a git repo for build history)
>>
>> Now, I am still confused how to build the Variscite image for the
>> i.MX6UL. We are still at Jethro but don't mind upgrading to Krogoth.
>>
>> I would appreciate some general guidelines on how to take the
>> instructions in here:
>> http://variwiki.com/index.php?title=DART-6UL_Yocto_Jethro_R1_build
>> and transform them to the config format that Autobuilder understands.
>>
>> Mainly, I understand that I have to:
>>
>> 1) checkout the source code
>>
>> $ mkdir ~/var-mx6ul-mx7-yocto-jethro
>> $ cd ~/var-mx6ul-mx7-yocto-jethro
>> $ repo init -u git://git.freescale.com/imx/fsl-arm-yocto-bsp.git -b 
>> imx-4.1.15-1.0.0_ga
>> $ repo sync
>>
>> $ cd ~/var-mx6ul-mx7-yocto-jethro/sources
>> $ git clone https://github.com/varigit/meta-variscite-mx6ul-mx7.git -b 
>> imx_4.1.15_ga-var01
>>
>>
>> 2) copy and patch
>>
>> $ cp meta-variscite-mx6ul-mx7/scripts/var-setup-release.sh ../
>> $ patch -p1 < 
>> meta-variscite-mx6ul-mx7/patch/Fix-FSL-multi-patch-append-bugs.patch
>>
>>
>> 3) setup a download directory
>>
>> $ sudo mkdir /opt/yocto_downloads
>> $ sudo chmod 777 /opt/yocto_downloads/
>> $ sed -i 's/DL_DIR ?= "${BSPDIR}\/downloads/DL_DIR = 
>> "\/opt\/yocto_downloads/g' conf/local.conf
>>
>>
>> 4) setup the environment
>>
>> $ cd ~/var-mx6ul-mx7-yocto-jethro
>> $ MACHINE=imx6ul-var-dart DISTRO=fsl-imx-x11 source var-setup-release.sh -b 
>> build_x11
>>
>>
>> 5) build
>>
>> $ bitbake fsl-image-gui
>>
>>
>> I looked at some nightly-*.conf files, for example nightly-x86-64.conf to
>> try to guess what to do.
>>
>> I know I have to create a file like nightly-var-mx6ul.conf but given that
>> Variscite is not using the 'repo' command that Freescale is using (they do
>> a git clone) and given step (2) of copying their own setup script and
>> patching the whole freescale meta- layers, I am not sure about what to put
>> instead of the line:
>>
>> {'CheckOutLayers': {}},
>>
>>
>> I also can't figure out how the info in Variscite's build-x11/local.conf
>> can be translated to the Autobuilder format, i.e. where to specify all
>> these config parameters.
>>
>> Any help would be greatly appreciated.
>>
>>
>> On Mon, Nov 14, 2016 at 12:35 PM, Alain Achkar 
>> wrote:
>>
>>> Thanks Chris!
>>>
>>> This is the answer I was looking

Re: [yocto] Buildbot / Autobuilder / custom?

2017-04-21 Thread Alain Achkar 
Kindly requesting help again, please.
I understand that Variscite might not be following the "Yocto way" 100% but
surely we can run custom commands using Autobuilder?

Please see detailed description in my last post.
Thanks!

On Mon, Apr 3, 2017 at 9:43 PM, Alain Achkar 
wrote:

> Hi Chris!
>
> I had to put this task on hold and I am re-visiting it now. I followed the
> instructions here:  http://git.yoctoproject.org/cgit.cgi/yocto-
> autobuilder/tree/README-QUICKSTART and managed to force one of the builds
> that comes with Yocto. I got errors but I'm not interested in the
> Yocto-provided nightly builds anyway. (The errors are related to
> buildhistory - it seems that some builds set Buildhistory=True but I
> haven't set up a git repo for build history)
>
> Now, I am still confused how to build the Variscite image for the i.MX6UL.
> We are still at Jethro but don't mind upgrading to Krogoth.
>
> I would appreciate some general guidelines on how to take the instructions
> in here:
> http://variwiki.com/index.php?title=DART-6UL_Yocto_Jethro_R1_build
> and transform them to the config format that Autobuilder understands.
>
> Mainly, I understand that I have to:
>
> 1) checkout the source code
>
> $ mkdir ~/var-mx6ul-mx7-yocto-jethro
> $ cd ~/var-mx6ul-mx7-yocto-jethro
> $ repo init -u git://git.freescale.com/imx/fsl-arm-yocto-bsp.git -b 
> imx-4.1.15-1.0.0_ga
> $ repo sync
>
> $ cd ~/var-mx6ul-mx7-yocto-jethro/sources
> $ git clone https://github.com/varigit/meta-variscite-mx6ul-mx7.git -b 
> imx_4.1.15_ga-var01
>
>
> 2) copy and patch
>
> $ cp meta-variscite-mx6ul-mx7/scripts/var-setup-release.sh ../
> $ patch -p1 < 
> meta-variscite-mx6ul-mx7/patch/Fix-FSL-multi-patch-append-bugs.patch
>
>
> 3) setup a download directory
>
> $ sudo mkdir /opt/yocto_downloads
> $ sudo chmod 777 /opt/yocto_downloads/
> $ sed -i 's/DL_DIR ?= "${BSPDIR}\/downloads/DL_DIR = 
> "\/opt\/yocto_downloads/g' conf/local.conf
>
>
> 4) setup the environment
>
> $ cd ~/var-mx6ul-mx7-yocto-jethro
> $ MACHINE=imx6ul-var-dart DISTRO=fsl-imx-x11 source var-setup-release.sh -b 
> build_x11
>
>
> 5) build
>
> $ bitbake fsl-image-gui
>
>
> I looked at some nightly-*.conf files, for example nightly-x86-64.conf to
> try to guess what to do.
>
> I know I have to create a file like nightly-var-mx6ul.conf but given that
> Variscite is not using the 'repo' command that Freescale is using (they do
> a git clone) and given step (2) of copying their own setup script and
> patching the whole freescale meta- layers, I am not sure about what to put
> instead of the line:
>
> {'CheckOutLayers': {}},
>
>
> I also can't figure out how the info in Variscite's build-x11/local.conf
> can be translated to the Autobuilder format, i.e. where to specify all
> these config parameters.
>
> Any help would be greatly appreciated.
>
>
> On Mon, Nov 14, 2016 at 12:35 PM, Alain Achkar 
> wrote:
>
>> Thanks Chris!
>>
>> This is the answer I was looking for.
>>
>> On Mon, Nov 14, 2016 at 11:58 AM, Chris Whittenburg <
>> whittenb...@gmail.com> wrote:
>>
>>>
>>> Hi Alain,
>>>
>>> We use the same module-- Variscite DART MX6, and I was able to hack the
>>> yocto autobuilder to build our image nightly.  I host it all on a single
>>> machine.
>>>
>>> -chris
>>>
>>>
>>> On Mon, Nov 14, 2016 at 10:13 AM, Alain Achkar >> > wrote:
>>>
 Thanks for your answers! From reading these links, it is still not
 clear to me if this might be overkill for my requirements. AB Cluster Setup
 talks about "*the worker requires 2+ TB to hold all the build temp
 files and git repos. If build artifacts and a local sstate mirror are
 included, additional worker space is required.*"

 Currently, my build only takes 35GB, so I think what these links are
 talking about is how to replicate what the Autobuilder project
 https://autobuilder.yoctoproject.org/ already does.

 To clarify, I am not interested in running builds and tests for
 everything that Yocto already builds and tests (i.e. all the processor
 architectures, all the machine types, etc.).  I am interested in running
 one build for one machine type (the Variscite DART-6UL i.MX6UL arm-based
 processor, for which NXP/Freescale and Variscite have provided recipes and
 layers for).

 I know that autobuilder includes BuildBot (this is why I specified it
 in parentheses) but my question remains: do I only install BuildBot and try
 to build my machine type with it, or do I install Autobuilder?

 On Mon, Nov 14, 2016 at 10:10 AM, Bill Randle 
 wrote:

> Also, be sure to check the Yocto Project wiki pages:
> https://wiki.yoctoproject.org/wiki/The_Yocto_Autobuilder
> in particular, the AB cluster setup and AB maintenance links. Even
> though the one link refers to setting up an entire cluster, I've used
> that procedure to setup

Re: [yocto] Buildbot / Autobuilder / custom?

2017-04-03 Thread Alain Achkar 
Hi Chris!

I had to put this task on hold and I am re-visiting it now. I followed the
instructions here:  http://git.yoctoproject.org/cgit.cgi/
yocto-autobuilder/tree/README-QUICKSTART and managed to force one of the
builds that comes with Yocto. I got errors but I'm not interested in the
Yocto-provided nightly builds anyway. (The errors are related to
buildhistory - it seems that some builds set Buildhistory=True but I
haven't set up a git repo for build history)

Now, I am still confused how to build the Variscite image for the i.MX6UL.
We are still at Jethro but don't mind upgrading to Krogoth.

I would appreciate some general guidelines on how to take the instructions
in here:
http://variwiki.com/index.php?title=DART-6UL_Yocto_Jethro_R1_build
and transform them to the config format that Autobuilder understands.

Mainly, I understand that I have to:

1) checkout the source code

$ mkdir ~/var-mx6ul-mx7-yocto-jethro
$ cd ~/var-mx6ul-mx7-yocto-jethro
$ repo init -u git://git.freescale.com/imx/fsl-arm-yocto-bsp.git -b
imx-4.1.15-1.0.0_ga
$ repo sync

$ cd ~/var-mx6ul-mx7-yocto-jethro/sources
$ git clone https://github.com/varigit/meta-variscite-mx6ul-mx7.git -b
imx_4.1.15_ga-var01


2) copy and patch

$ cp meta-variscite-mx6ul-mx7/scripts/var-setup-release.sh ../
$ patch -p1 < 
meta-variscite-mx6ul-mx7/patch/Fix-FSL-multi-patch-append-bugs.patch


3) setup a download directory

$ sudo mkdir /opt/yocto_downloads
$ sudo chmod 777 /opt/yocto_downloads/
$ sed -i 's/DL_DIR ?= "${BSPDIR}\/downloads/DL_DIR =
"\/opt\/yocto_downloads/g' conf/local.conf


4) setup the environment

$ cd ~/var-mx6ul-mx7-yocto-jethro
$ MACHINE=imx6ul-var-dart DISTRO=fsl-imx-x11 source
var-setup-release.sh -b build_x11


5) build

$ bitbake fsl-image-gui


I looked at some nightly-*.conf files, for example nightly-x86-64.conf to
try to guess what to do.

I know I have to create a file like nightly-var-mx6ul.conf but given that
Variscite is not using the 'repo' command that Freescale is using (they do
a git clone) and given step (2) of copying their own setup script and
patching the whole freescale meta- layers, I am not sure about what to put
instead of the line:

{'CheckOutLayers': {}},


I also can't figure out how the info in Variscite's build-x11/local.conf
can be translated to the Autobuilder format, i.e. where to specify all
these config parameters.

Any help would be greatly appreciated.


On Mon, Nov 14, 2016 at 12:35 PM, Alain Achkar 
wrote:

> Thanks Chris!
>
> This is the answer I was looking for.
>
> On Mon, Nov 14, 2016 at 11:58 AM, Chris Whittenburg  > wrote:
>
>>
>> Hi Alain,
>>
>> We use the same module-- Variscite DART MX6, and I was able to hack the
>> yocto autobuilder to build our image nightly.  I host it all on a single
>> machine.
>>
>> -chris
>>
>>
>> On Mon, Nov 14, 2016 at 10:13 AM, Alain Achkar 
>> wrote:
>>
>>> Thanks for your answers! From reading these links, it is still not clear
>>> to me if this might be overkill for my requirements. AB Cluster Setup talks
>>> about "*the worker requires 2+ TB to hold all the build temp files and
>>> git repos. If build artifacts and a local sstate mirror are included,
>>> additional worker space is required.*"
>>>
>>> Currently, my build only takes 35GB, so I think what these links are
>>> talking about is how to replicate what the Autobuilder project
>>> https://autobuilder.yoctoproject.org/ already does.
>>>
>>> To clarify, I am not interested in running builds and tests for
>>> everything that Yocto already builds and tests (i.e. all the processor
>>> architectures, all the machine types, etc.).  I am interested in running
>>> one build for one machine type (the Variscite DART-6UL i.MX6UL arm-based
>>> processor, for which NXP/Freescale and Variscite have provided recipes and
>>> layers for).
>>>
>>> I know that autobuilder includes BuildBot (this is why I specified it in
>>> parentheses) but my question remains: do I only install BuildBot and try to
>>> build my machine type with it, or do I install Autobuilder?
>>>
>>> On Mon, Nov 14, 2016 at 10:10 AM, Bill Randle 
>>> wrote:
>>>
 Also, be sure to check the Yocto Project wiki pages:
 https://wiki.yoctoproject.org/wiki/The_Yocto_Autobuilder
 in particular, the AB cluster setup and AB maintenance links. Even
 though the one link refers to setting up an entire cluster, I've used
 that procedure to setup a single autobuilder, as well.

 -Bill

 On Fri, Nov 11, 2016 at 10:37 AM, Beth 'pidge' Flanagan
  wrote:
 > On Fri, 2016-11-11 at 12:35 -0500, Alain Achkar wrote:
 >> Hello Joshua!
 >>
 >> We are using Yocto to build for the Variscite DART-6UL SoM which is
 >> based on Freescale/NXP i.MX6UL (http://variwiki.com/index.php
 ?title=D
 >> ART-6UL_Yocto_Jethro_R1_build).
 >>
 >> Currently, developers run builds

Re: [yocto] Buildbot / Autobuilder / custom?

2017-03-24 Thread Alain Achkar 
Thanks Joshua! This fix worked.

On Fri, Mar 24, 2017 at 6:44 AM, Joshua Lock 
wrote:

> Hi Alain,
>
> On Thu, 2017-03-23 at 17:40 -0400, Alain Achkar wrote:
>
> I followed http://git.yoctoproject.org/cgit.cgi/
> yocto-autobuilder/tree/README-QUICKSTART
> and got:
>
>
> 
>
> checking controller.cfg
> LOADING CONFIG FILE
> *error while parsing config file:*
> Traceback (most recent call last):
>   File "/media/data/yocto-autobuilder/lib/python2.7/
> site-packages/Twisted-12.2.0-py2.7-linux-x86_64.egg/twisted/internet/defer.py",
> line 1187, in unwindGenerator
> return _inlineCallbacks(None, gen, Deferred())
>   File "/media/data/yocto-autobuilder/lib/python2.7/
> site-packages/Twisted-12.2.0-py2.7-linux-x86_64.egg/twisted/internet/defer.py",
> line 1045, in _inlineCallbacks
> result = g.send(result)
>   File "/media/data/yocto-autobuilder/lib/python2.7/
> site-packages/buildbot-0.8.8-py2.7.egg/buildbot/scripts/upgrade_master.py",
> line 169, in upgradeMaster
> master_cfg = loadConfig(config, configFile)
>   File "/media/data/yocto-autobuilder/lib/python2.7/
> site-packages/buildbot-0.8.8-py2.7.egg/buildbot/scripts/upgrade_master.py",
> line 53, in loadConfig
> config['basedir'], configFileName)
> ---  ---
>   File "/media/data/yocto-autobuilder/lib/python2.7/
> site-packages/buildbot-0.8.8-py2.7.egg/buildbot/config.py", line 149, in
> loadConfig
> exec f in localDict
>   File "/media/data/yocto-autobuilder/yocto-controller/controller.cfg",
> line 94, in 
> yocto_buildsets.createBuildsets()
>   File "/media/data/yocto-autobuilder/lib/python2.7/
> site-packages/autobuilder/Autobuilder.py", line 65, in createBuildsets
> self.parseBuildSet(buildset)
>   File "/media/data/yocto-autobuilder/lib/python2.7/
> site-packages/autobuilder/Autobuilder.py", line 102, in parseBuildSet
> self.parseRepos(buildset)
>   File "/media/data/yocto-autobuilder/lib/python2.7/
> site-packages/autobuilder/Autobuilder.py", line 127, in parseRepos
> if layer.iterkeys().next() not in self.repos:
> *exceptions.AttributeError: 'list' object has no attribute 'iterkeys'*
>
>
> I appear to have left a stray comma in buildset-config.controller/nightly.conf
> which causes the parser to choke. I've fixed this in the master branch of
> yocto-autobuilder: http://git.yoctoproject.org/clean/cgit.cgi/yocto-
> autobuilder/commit/?id=239eeaa1e2cf3a985e75f8b75dedfe704a7d6991
>
>
> You'll need to edit buildset-config/nightly.conf on your local instance
> similarly to remove the comma at the end of the repos list, around line 38.
>
> Thanks for reporting this error!
>
> Joshua
>
> Errors loading configuration:
>   error while parsing config file: 'list' object has no attribute
> 'iterkeys' (traceback in logfile)
>  To start the autobuilder:
>  ./yocto-start-autobuilder 
>
>  To stop the autobuilder:
>  ./yocto-stop-autobuilder 
>
> >0< alain@esxi-ub1 Thu Mar 23 05:34 PM [master !?]
> /media/data/yocto-autobuilder >
>
> Any ideas on how to fix this?
>
> On Mon, Nov 14, 2016 at 1:40 PM, Bill Randle 
> wrote:
>
> Yes, the 2 TB is for everything you might do with an AutoBuilder. The
> actual space used will be closer to your 35 GB, since you're building
> only a single image / work product. People have used BuilBot directly
> for automated builds, but using the AutoBuilder code (which runs
> BuildBot underneath) gives you added flexibility and capability
> without much additional overhead.
>
> -Bill
>
> On Mon, Nov 14, 2016 at 8:13 AM, Alain Achkar 
> wrote:
> > Thanks for your answers! From reading these links, it is still not clear
> to
> > me if this might be overkill for my requirements. AB Cluster Setup talks
> > about "the worker requires 2+ TB to hold all the build temp files and git
> > repos. If build artifacts and a local sstate mirror are included,
> additional
> > worker space is required."
> >
> > Currently, my build only takes 35GB, so I think what these links are
> talking
> > about is how to replicate what the Autobuilder project
> > https://autobuilder.yoctoproject.org/ already does.
> >
> > To clarify, I am not interested in running builds and tests for
> everything
> > that Yocto already builds and tests (i.e. all the processor
> architectures,
> > all the machine types, etc.).  I am interested in running one build for
> one
> > machine type (the Variscite DART-6UL i.MX6UL arm-based processor, for
> which
> > NXP/Freescale and Variscite have provided recipes and layers for).
> >
> > I know that autobuilder includes BuildBot (this is why I specified it in
> > parentheses) but my question remains: do I only install BuildBot and try
> to
> > build my machine type with it, or do I install Autobuilder?
> >
> > On Mon, Nov 14, 2016 at 10:10 AM, Bill Randle 
> wrote:
> >>
> >> Also, be sure to check the Yocto Project wiki pages:
> >>

Re: [yocto] Buildbot / Autobuilder / custom?

2017-03-24 Thread Joshua Lock 
Hi Alain,
On Thu, 2017-03-23 at 17:40 -0400, Alain Achkar wrote:
> I followed http://git.yoctoproject.org/cgit.cgi/yocto-autobuilder/tre
> e/README-QUICKSTARTand got:


> checking controller.cfg
> LOADING CONFIG FILE
> error while parsing config file:
> Traceback (most recent call last):
>   File "/media/data/yocto-autobuilder/lib/python2.7/site-
> packages/Twisted-12.2.0-py2.7-linux-
> x86_64.egg/twisted/internet/defer.py", line 1187, in unwindGenerator
>     return _inlineCallbacks(None, gen, Deferred())
>   File "/media/data/yocto-autobuilder/lib/python2.7/site-
> packages/Twisted-12.2.0-py2.7-linux-
> x86_64.egg/twisted/internet/defer.py", line 1045, in _inlineCallbacks
>     result = g.send(result)
>   File "/media/data/yocto-autobuilder/lib/python2.7/site-
> packages/buildbot-0.8.8-
> py2.7.egg/buildbot/scripts/upgrade_master.py", line 169, in
> upgradeMaster
>     master_cfg = loadConfig(config, configFile)
>   File "/media/data/yocto-autobuilder/lib/python2.7/site-
> packages/buildbot-0.8.8-
> py2.7.egg/buildbot/scripts/upgrade_master.py", line 53, in loadConfig
>     config['basedir'], configFileName)
> ---  ---
>   File "/media/data/yocto-autobuilder/lib/python2.7/site-
> packages/buildbot-0.8.8-py2.7.egg/buildbot/config.py", line 149, in
> loadConfig
>     exec f in localDict
>   File "/media/data/yocto-autobuilder/yocto-
> controller/controller.cfg", line 94, in 
>     yocto_buildsets.createBuildsets()
>   File "/media/data/yocto-autobuilder/lib/python2.7/site-
> packages/autobuilder/Autobuilder.py", line 65, in createBuildsets
>     self.parseBuildSet(buildset)
>   File "/media/data/yocto-autobuilder/lib/python2.7/site-
> packages/autobuilder/Autobuilder.py", line 102, in parseBuildSet
>     self.parseRepos(buildset)
>   File "/media/data/yocto-autobuilder/lib/python2.7/site-
> packages/autobuilder/Autobuilder.py", line 127, in parseRepos
>     if layer.iterkeys().next() not in self.repos:
> exceptions.AttributeError: 'list' object has no attribute 'iterkeys'

I appear to have left a stray comma in buildset-
config.controller/nightly.conf which causes the parser to choke. I've
fixed this in the master branch of yocto-autobuilder: http://git.yoctop
roject.org/clean/cgit.cgi/yocto-
autobuilder/commit/?id=239eeaa1e2cf3a985e75f8b75dedfe704a7d6991

You'll need to edit buildset-config/nightly.conf on your local instance
similarly to remove the comma at the end of the repos list, around line
38.
Thanks for reporting this error!
Joshua
> Errors loading configuration:
>   error while parsing config file: 'list' object has no attribute
> 'iterkeys' (traceback in logfile)
>  To start the autobuilder:
>  ./yocto-start-autobuilder 
> 
> 
>  To stop the autobuilder:
>  ./yocto-stop-autobuilder 
> 
> 
> >0< alain@esxi-ub1 Thu Mar 23 05:34 PM [master !?] /media/data/yocto-
> autobuilder >
> 
> 
> Any ideas on how to fix this?
> On Mon, Nov 14, 2016 at 1:40 PM, Bill Randle 
> wrote:
> > Yes, the 2 TB is for everything you might do with an AutoBuilder.
> > The
> > 
> > actual space used will be closer to your 35 GB, since you're
> > building
> > 
> > only a single image / work product. People have used BuilBot
> > directly
> > 
> > for automated builds, but using the AutoBuilder code (which runs
> > 
> > BuildBot underneath) gives you added flexibility and capability
> > 
> > without much additional overhead.
> > 
> > 
> > 
> >     -Bill
> > 
> > 
> > 
> > On Mon, Nov 14, 2016 at 8:13 AM, Alain Achkar  > .com> wrote:
> > 
> > > Thanks for your answers! From reading these links, it is still
> > not clear to
> > 
> > > me if this might be overkill for my requirements. AB Cluster
> > Setup talks
> > 
> > > about "the worker requires 2+ TB to hold all the build temp files
> > and git
> > 
> > > repos. If build artifacts and a local sstate mirror are included,
> > additional
> > 
> > > worker space is required."
> > 
> > >
> > 
> > > Currently, my build only takes 35GB, so I think what these links
> > are talking
> > 
> > > about is how to replicate what the Autobuilder project
> > 
> > > https://autobuilder.yoctoproject.org/ already does.
> > 
> > >
> > 
> > > To clarify, I am not interested in running builds and tests for
> > everything
> > 
> > > that Yocto already builds and tests (i.e. all the processor
> > architectures,
> > 
> > > all the machine types, etc.).  I am interested in running one
> > build for one
> > 
> > > machine type (the Variscite DART-6UL i.MX6UL arm-based processor,
> > for which
> > 
> > > NXP/Freescale and Variscite have provided recipes and layers
> > for).
> > 
> > >
> > 
> > > I know that autobuilder includes BuildBot (this is why I
> > specified it in
> > 
> > > parentheses) but my question remains: do I only install BuildBot
> > and try to
> > 
> > > build my machine type with it, or do I install Autobuilder?
> > 
> > >
> > 
> > > On Mon, Nov 14, 2016 at 10:10 AM, Bill Randle

Re: [yocto] Buildbot / Autobuilder / custom?

2017-03-23 Thread Alain Achkar 
I followed
http://git.yoctoproject.org/cgit.cgi/yocto-autobuilder/tree/README-QUICKSTART
and got:

>0< alain@esxi-ub1 Thu Mar 23 05:34 PM [master]
/media/data/yocto-autobuilder >  *. ./yocto-autobuilder-setup*

Creating yocto-controller/buildbot.tac from an example buildbot.tac


Creating yocto-worker/buildbot.tac from an example buildbot.tac


Creating yocto-controller/controller.cfg from an example controller.cfg

#
 Setting envvars.
 In the future though please add the following to your shell environment:
 
PYTHONPATH=/media/data/yocto-autobuilder/lib/python2.7/site-packages/:/media/data/yocto-autobuilder/lib/python2.7/site-packages/autobuilder/:/media/data/yocto-autobuilder/lib/python2.7/site-packages/autobuilder/buildsteps:$PYTHONPATH
 
PATH=/media/data/yocto-autobuilder/bin:/media/data/yocto-autobuilder/yocto-autobuilder/scripts:$PATH
 YOCTO_AB_CONFIG=/media/data/yocto-autobuilder/buildset-config/yoctoAB.conf

 If you don't, you should source this script everytime you want start the
 autobuilder.

Creating config/autobuilder.conf from an example autobuilder.conf


Creating buildset-config from buildset-config.controller


 You've not setup the autobuilder before. Generating a server/client
password
 combo for you.
 Client/Server Password = (deleted, on purpose)

 Modifying the following files with this password:

 /media/data/yocto-autobuilder/yocto-controller/controller.cfg
 /media/data/yocto-autobuilder/yocto-controller/buildbot.tac
 /media/data/yocto-autobuilder/yocto-worker/buildbot.tac

 Updating worker-init script used for google cloud building.

 If you wish to use your own generated username and password please
 modify the above files as needed. Please see the README for more
 information.

 Generating an .htpasswd file using your current username and (deleted, on
purpose) at /media/data/yocto-autobuilder/.htpasswd
#

 Please modify /media/data/yocto-autobuilder/config/autobuilder.conf if you
wish to specify
 a different location in which to publish build artifacts, etc.

#

 Ready to start the yocto autobuilder.

 The yocto-autobuilder runs buildbot 0.8.8 with some modifications and
 a different git fetcher (yoctogit.py)

#
checking basedir
checking for running master
checking controller.cfg
LOADING CONFIG FILE
*error while parsing config file:*
Traceback (most recent call last):
  File
"/media/data/yocto-autobuilder/lib/python2.7/site-packages/Twisted-12.2.0-py2.7-linux-x86_64.egg/twisted/internet/defer.py",
line 1187, in unwindGenerator
return _inlineCallbacks(None, gen, Deferred())
  File
"/media/data/yocto-autobuilder/lib/python2.7/site-packages/Twisted-12.2.0-py2.7-linux-x86_64.egg/twisted/internet/defer.py",
line 1045, in _inlineCallbacks
result = g.send(result)
  File
"/media/data/yocto-autobuilder/lib/python2.7/site-packages/buildbot-0.8.8-py2.7.egg/buildbot/scripts/upgrade_master.py",
line 169, in upgradeMaster
master_cfg = loadConfig(config, configFile)
  File
"/media/data/yocto-autobuilder/lib/python2.7/site-packages/buildbot-0.8.8-py2.7.egg/buildbot/scripts/upgrade_master.py",
line 53, in loadConfig
config['basedir'], configFileName)
---  ---
  File
"/media/data/yocto-autobuilder/lib/python2.7/site-packages/buildbot-0.8.8-py2.7.egg/buildbot/config.py",
line 149, in loadConfig
exec f in localDict
  File "/media/data/yocto-autobuilder/yocto-controller/controller.cfg",
line 94, in 
yocto_buildsets.createBuildsets()
  File
"/media/data/yocto-autobuilder/lib/python2.7/site-packages/autobuilder/Autobuilder.py",
line 65, in createBuildsets
self.parseBuildSet(buildset)
  File
"/media/data/yocto-autobuilder/lib/python2.7/site-packages/autobuilder/Autobuilder.py",
line 102, in parseBuildSet
self.parseRepos(buildset)
  File
"/media/data/yocto-autobuilder/lib/python2.7/site-packages/autobuilder/Autobuilder.py",
line 127, in parseRepos
if layer.iterkeys().next() not in self.repos:
*exceptions.AttributeError: 'list' object has no attribute 'iterkeys'*
Errors loading configuration:
  error while parsing config file: 'list' object has no attribute
'iterkeys' (traceback in logfile)
 To start the autobuilder:
 ./yocto-start-autobuilder 

 To stop the autobuilder:
 ./yocto-stop-autobuilder 

>0< alain@esxi-ub1 Thu Mar 23 05:34 PM [master !?]
/media/data/yocto-autobuilder >

Any ideas on how to fix this?

On Mon, Nov 14, 2016 at 1:40 PM, Bill Randle  wrote:

> Yes, the 2 TB is for everything you might do with an AutoBuilder. The
> actual space used will be closer to your 35 GB, since you're building
> only a single image / work product. People have used BuilBot directly
> for automated builds, but using the

Re: [yocto] Buildbot / Autobuilder / custom?

2016-11-14 Thread Bill Randle 
Yes, the 2 TB is for everything you might do with an AutoBuilder. The
actual space used will be closer to your 35 GB, since you're building
only a single image / work product. People have used BuilBot directly
for automated builds, but using the AutoBuilder code (which runs
BuildBot underneath) gives you added flexibility and capability
without much additional overhead.

-Bill

On Mon, Nov 14, 2016 at 8:13 AM, Alain Achkar  wrote:
> Thanks for your answers! From reading these links, it is still not clear to
> me if this might be overkill for my requirements. AB Cluster Setup talks
> about "the worker requires 2+ TB to hold all the build temp files and git
> repos. If build artifacts and a local sstate mirror are included, additional
> worker space is required."
>
> Currently, my build only takes 35GB, so I think what these links are talking
> about is how to replicate what the Autobuilder project
> https://autobuilder.yoctoproject.org/ already does.
>
> To clarify, I am not interested in running builds and tests for everything
> that Yocto already builds and tests (i.e. all the processor architectures,
> all the machine types, etc.).  I am interested in running one build for one
> machine type (the Variscite DART-6UL i.MX6UL arm-based processor, for which
> NXP/Freescale and Variscite have provided recipes and layers for).
>
> I know that autobuilder includes BuildBot (this is why I specified it in
> parentheses) but my question remains: do I only install BuildBot and try to
> build my machine type with it, or do I install Autobuilder?
>
> On Mon, Nov 14, 2016 at 10:10 AM, Bill Randle  wrote:
>>
>> Also, be sure to check the Yocto Project wiki pages:
>> https://wiki.yoctoproject.org/wiki/The_Yocto_Autobuilder
>> in particular, the AB cluster setup and AB maintenance links. Even
>> though the one link refers to setting up an entire cluster, I've used
>> that procedure to setup a single autobuilder, as well.
>>
>> -Bill
>>
>> On Fri, Nov 11, 2016 at 10:37 AM, Beth 'pidge' Flanagan
>>  wrote:
>> > On Fri, 2016-11-11 at 12:35 -0500, Alain Achkar wrote:
>> >> Hello Joshua!
>> >>
>> >> We are using Yocto to build for the Variscite DART-6UL SoM which is
>> >> based on Freescale/NXP i.MX6UL (http://variwiki.com/index.php?title=D
>> >> ART-6UL_Yocto_Jethro_R1_build).
>> >>
>> >> Currently, developers run builds manually on their desktops. We are
>> >> considering using a continuous integration server. I found out that
>> >> The Yocto Project uses Autobuilder which in turn uses BuildBot. I
>> >> read most of the introductory documentation.
>> >>
>> >> Would you recommend that we install only BuildBot and configure it to
>> >> build what we need?
>> >>
>> >> OR
>> >>
>> >> Would you recommend that we install Autobuilder (and BuildBot) and
>> >> configure it to build what we need?
>> >
>> > Actually, the yocto-autobuilder already contains buildbot so no need
>> > for both.
>> >
>> > If you read:
>> >
>> > http://git.yoctoproject.org/cgit.cgi/yocto-autobuilder/tree/README-QUIC
>> > KSTART
>> >
>> > and probably some of the other READMEs and the documentation in the
>> > docs directory of the yocto-autobuilder directory, it should get you
>> > started in the right direction.
>> >
>> > Feel free to ask questions if you need help.
>> >
>> > -b
>> >
>> > Elizabeth 'pidge' Flanagan
>> > Founder/CTO Togán Labs
>> > www.toganlabs.com
>> >
>> >>
>> >> What is the faster path to get up and running?
>> >>
>> >> Thanks!
>> >> Alain.
>> >>
>> >> --
>> >> Alain Achkar, M.Eng. | Software Engineer
>> >> Triacta Power Solutions
>> >> (O) 613.256.2868 (C) 613.406.6520
>> >> www.triacta.com
>> >
>> >
>> >
>> > --
>> > ___
>> > yocto mailing list
>> > yocto@yoctoproject.org
>> > https://lists.yoctoproject.org/listinfo/yocto
>
>
>
>
> --
>
> Alain Achkar, M.Eng. | Software Engineer
>
> Triacta Power Solutions
>
> (O) 613.256.2868 (C) 613.406.6520
>
> www.triacta.com
-- 
___
yocto mailing list
yocto@yoctoproject.org
https://lists.yoctoproject.org/listinfo/yocto

Re: [yocto] Buildbot / Autobuilder / custom?

2016-11-14 Thread Alain Achkar 
Thanks for your answers! From reading these links, it is still not clear to
me if this might be overkill for my requirements. AB Cluster Setup talks
about "*the worker requires 2+ TB to hold all the build temp files and git
repos. If build artifacts and a local sstate mirror are included,
additional worker space is required.*"

Currently, my build only takes 35GB, so I think what these links are
talking about is how to replicate what the Autobuilder project
https://autobuilder.yoctoproject.org/ already does.

To clarify, I am not interested in running builds and tests for everything
that Yocto already builds and tests (i.e. all the processor architectures,
all the machine types, etc.).  I am interested in running one build for one
machine type (the Variscite DART-6UL i.MX6UL arm-based processor, for which
NXP/Freescale and Variscite have provided recipes and layers for).

I know that autobuilder includes BuildBot (this is why I specified it in
parentheses) but my question remains: do I only install BuildBot and try to
build my machine type with it, or do I install Autobuilder?

On Mon, Nov 14, 2016 at 10:10 AM, Bill Randle  wrote:

> Also, be sure to check the Yocto Project wiki pages:
> https://wiki.yoctoproject.org/wiki/The_Yocto_Autobuilder
> in particular, the AB cluster setup and AB maintenance links. Even
> though the one link refers to setting up an entire cluster, I've used
> that procedure to setup a single autobuilder, as well.
>
> -Bill
>
> On Fri, Nov 11, 2016 at 10:37 AM, Beth 'pidge' Flanagan
>  wrote:
> > On Fri, 2016-11-11 at 12:35 -0500, Alain Achkar wrote:
> >> Hello Joshua!
> >>
> >> We are using Yocto to build for the Variscite DART-6UL SoM which is
> >> based on Freescale/NXP i.MX6UL (http://variwiki.com/index.php?title=D
> >> ART-6UL_Yocto_Jethro_R1_build).
> >>
> >> Currently, developers run builds manually on their desktops. We are
> >> considering using a continuous integration server. I found out that
> >> The Yocto Project uses Autobuilder which in turn uses BuildBot. I
> >> read most of the introductory documentation.
> >>
> >> Would you recommend that we install only BuildBot and configure it to
> >> build what we need?
> >>
> >> OR
> >>
> >> Would you recommend that we install Autobuilder (and BuildBot) and
> >> configure it to build what we need?
> >
> > Actually, the yocto-autobuilder already contains buildbot so no need
> > for both.
> >
> > If you read:
> >
> > http://git.yoctoproject.org/cgit.cgi/yocto-autobuilder/tree/README-QUIC
> > KSTART
> >
> > and probably some of the other READMEs and the documentation in the
> > docs directory of the yocto-autobuilder directory, it should get you
> > started in the right direction.
> >
> > Feel free to ask questions if you need help.
> >
> > -b
> >
> > Elizabeth 'pidge' Flanagan
> > Founder/CTO Togán Labs
> > www.toganlabs.com
> >
> >>
> >> What is the faster path to get up and running?
> >>
> >> Thanks!
> >> Alain.
> >>
> >> --
> >> Alain Achkar, M.Eng. | Software Engineer
> >> Triacta Power Solutions
> >> (O) 613.256.2868 (C) 613.406.6520
> >> www.triacta.com
> >
> >
> >
> > --
> > ___
> > yocto mailing list
> > yocto@yoctoproject.org
> > https://lists.yoctoproject.org/listinfo/yocto
>



-- 

Alain Achkar, M.Eng. | Software Engineer

*Triacta Power Solutions*

(O) 613.256.2868 (C) 613.406.6520

*www.triacta.com  *
-- 
___
yocto mailing list
yocto@yoctoproject.org
https://lists.yoctoproject.org/listinfo/yocto

Re: [yocto] Buildbot / Autobuilder / custom?

2016-11-14 Thread Bill Randle 
Also, be sure to check the Yocto Project wiki pages:
https://wiki.yoctoproject.org/wiki/The_Yocto_Autobuilder
in particular, the AB cluster setup and AB maintenance links. Even
though the one link refers to setting up an entire cluster, I've used
that procedure to setup a single autobuilder, as well.

-Bill

On Fri, Nov 11, 2016 at 10:37 AM, Beth 'pidge' Flanagan
 wrote:
> On Fri, 2016-11-11 at 12:35 -0500, Alain Achkar wrote:
>> Hello Joshua!
>>
>> We are using Yocto to build for the Variscite DART-6UL SoM which is
>> based on Freescale/NXP i.MX6UL (http://variwiki.com/index.php?title=D
>> ART-6UL_Yocto_Jethro_R1_build).
>>
>> Currently, developers run builds manually on their desktops. We are
>> considering using a continuous integration server. I found out that
>> The Yocto Project uses Autobuilder which in turn uses BuildBot. I
>> read most of the introductory documentation.
>>
>> Would you recommend that we install only BuildBot and configure it to
>> build what we need?
>>
>> OR
>>
>> Would you recommend that we install Autobuilder (and BuildBot) and
>> configure it to build what we need?
>
> Actually, the yocto-autobuilder already contains buildbot so no need
> for both.
>
> If you read:
>
> http://git.yoctoproject.org/cgit.cgi/yocto-autobuilder/tree/README-QUIC
> KSTART
>
> and probably some of the other READMEs and the documentation in the
> docs directory of the yocto-autobuilder directory, it should get you
> started in the right direction.
>
> Feel free to ask questions if you need help.
>
> -b
>
> Elizabeth 'pidge' Flanagan
> Founder/CTO Togán Labs
> www.toganlabs.com
>
>>
>> What is the faster path to get up and running?
>>
>> Thanks!
>> Alain.
>>
>> --
>> Alain Achkar, M.Eng. | Software Engineer
>> Triacta Power Solutions
>> (O) 613.256.2868 (C) 613.406.6520
>> www.triacta.com
>
>
>
> --
> ___
> yocto mailing list
> yocto@yoctoproject.org
> https://lists.yoctoproject.org/listinfo/yocto
-- 
___
yocto mailing list
yocto@yoctoproject.org
https://lists.yoctoproject.org/listinfo/yocto

Re: [yocto] Buildbot / Autobuilder / custom?

2016-11-11 Thread Beth 'pidge' Flanagan 
On Fri, 2016-11-11 at 12:35 -0500, Alain Achkar wrote:
> Hello Joshua!
> 
> We are using Yocto to build for the Variscite DART-6UL SoM which is
> based on Freescale/NXP i.MX6UL (http://variwiki.com/index.php?title=D
> ART-6UL_Yocto_Jethro_R1_build).
> 
> Currently, developers run builds manually on their desktops. We are
> considering using a continuous integration server. I found out that
> The Yocto Project uses Autobuilder which in turn uses BuildBot. I
> read most of the introductory documentation.
> 
> Would you recommend that we install only BuildBot and configure it to
> build what we need?
> 
> OR
> 
> Would you recommend that we install Autobuilder (and BuildBot) and
> configure it to build what we need?

Actually, the yocto-autobuilder already contains buildbot so no need
for both.

If you read:

http://git.yoctoproject.org/cgit.cgi/yocto-autobuilder/tree/README-QUIC
KSTART

and probably some of the other READMEs and the documentation in the
docs directory of the yocto-autobuilder directory, it should get you
started in the right direction.

Feel free to ask questions if you need help.

-b

Elizabeth 'pidge' Flanagan
Founder/CTO Togán Labs
www.toganlabs.com

> 
> What is the faster path to get up and running?
> 
> Thanks!
> Alain.
> 
> -- 
> Alain Achkar, M.Eng. | Software Engineer
> Triacta Power Solutions
> (O) 613.256.2868 (C) 613.406.6520
> www.triacta.com  



-- 
___
yocto mailing list
yocto@yoctoproject.org
https://lists.yoctoproject.org/listinfo/yocto