Since your clean koji has SSD I can set dmcache for you for the EBS
volume to speed up workload, but I am wrapping up for today, so
tomorrow.
The key is to turn off repo regen after each build, not sure how to do
this. I know it can be done somewhere in configs...
LZ
On Fri, Jun 9, 2017 at 11:43 PM, Lukas Zapletal <l...@redhat.com> wrote:
> Basically you go to foreman-packaging rpm/develop and then:
>
> git annex get (few won't download so delete them for now and skip them)
>
> for each package directory (you can do in loop)
> tito release koji-foreman
> tito release koji-foreman-plugins
>
> First build all packages without "foreman" in it, then regen
> buildroots, then all with "foreman" in it and then solve all failed
> builds. The order matters.
>
> But I suggest to turn off repo regeneration after each build, I don't
> remember how this can be done (anyone?) Then you regenerate at the end
> of the build with:
>
> koji list-targets --quiet | awk '{print $2}' | sort -u | egrep
> '(foreman|katello|pulp)' | grep nightly | xargs -n 1 koji regen-repo
>
> Also tune your kojibuilder1, I set maxjobs to 7 and capacity to 999 so
> it is under nice load of 5-7 concurrent jobs.
>
> On Fri, Jun 9, 2017 at 11:32 PM, Bryan Kearney <bryan.kear...@gmail.com>
> wrote:
>> also.. which rhel7 build is used? scl or nonscl?
>>
>> -- bk
>>
>>
>> On 06/09/2017 05:26 PM, Bryan Kearney wrote:
>>>
>>> Is there a script to rebuild for nightlies? The tags and external repos
>>> are up and running for nightlies at
>>>
>>> http://34.226.218.207/koji/
>>>
>>> (cleankoji)
>>>
>>> I am planning to reverse-engineer from the composes since I have all the
>>> builds loaded already. But, if there is an easy rebuild that would be great.
>>>
>>> -- bk
>>>
>>> On 06/09/2017 05:15 PM, Lukas Zapletal wrote:
>>>>
>>>> Quick update. Bryan is working on clean koji installation creating
>>>> tags from scratch.
>>>>
>>>> I have koji running from backup, last version built is 1.12. My plan is:
>>>>
>>>> - use our release engineering script to create new 1.13 tags
>>>> - regen repos for 1.13 and rebuild all 1.13 packages using tito rebuild
>>>> - make a compose and compare that against what we have on
>>>> yum.theforeman.org
>>>>
>>>> If this is success, I will do the same for 1.15.
>>>>
>>>> New koji recovered from backup is at
>>>> http://ec2-54-163-236-166.compute-1.amazonaws.com/koji/ and we
>>>> requested DNS change already, hopefully it's ready on Monday at least
>>>> for nightly builds.
>>>>
>>>> On Fri, Jun 9, 2017 at 2:46 PM, Lukas Zapletal <l...@redhat.com> wrote:
>>>>>
>>>>> Hmmm if I'd have to create new koji I'd rather start from scratch
>>>>> reimporting our repos back in.
>>>>>
>>>>> Current problems we had:
>>>>>
>>>>> - small vm with not enough cores/RAM to run builds
>>>>> - outdated system (RHEL6) and koji version
>>>>> - no use of LVM
>>>>> - data on ephemeral volumes
>>>>> - lots of symlinks which were added as we grew
>>>>>
>>>>> If I'd need recreate Koji from scratch, I'd try to avoid all of these.
>>>>> Local volumes should be only used as cache, swap and temp, external
>>>>> repos should be on external NFS, everything else on dedicated block
>>>>> device with regular backups.
>>>>>
>>>>> On Fri, Jun 9, 2017 at 2:33 PM, Bryan Kearney <bryan.kear...@gmail.com>
>>>>> wrote:
>>>>>>
>>>>>> On 06/09/2017 08:27 AM, Lukas Zapletal wrote:
>>>>>>>
>>>>>>>
>>>>>>> I have some bad news to share. I started on moving from our 2nd koji
>>>>>>> to 3rd koji created as m1.xxlarge instance so we would avoid NFS and
>>>>>>> could run on a single VM. I prepared everything and when I was
>>>>>>> shutting down all the builders and test VMs I accidentaly stopped the
>>>>>>> 2nd koji. Unfortunately the original koji instance was stopped
>>>>>>> yesterday as well. We lost all data from epheremal storage volumes,
>>>>>>> these were /mnt/tmp holding most importantly Postgres database for
>>>>>>> kojihub and not sure what else. Our deployment was dozen of symlinks
>>>>>>> across two local volumes and EBS.
>>>>>>>
>>>>>>> I found some backups of pgsql on EBS volume, but unfortunately these
>>>>>>> stopped being dumped summer 2016
>>>>>>> (koji.katello.org-kojidb-20160830-230505.dump is the newest one). Here
>>>>>>> are the options we have:
>>>>>>>
>>>>>>> 1) Restore the DB and attempt to start services back - but this will
>>>>>>> give us instance back with huge gap of almost one year. I am not sure
>>>>>>> how much useful that would be - we will not be able to bu ild into
>>>>>>> 1.15/1.14 repos, missing tags, builds, logs.
>>>>>>>
>>>>>>> 2) Move to https://cbs.centos.org CentOS CBS, they run their own Koji
>>>>>>> and start building there
>>>>>>>
>>>>>>> 3) Speed up Copr transition
>>>>>>>
>>>>>>> 4) Build whole new Koji from scratch
>>>>>>
>>>>>>
>>>>>>
>>>>>> A variation here is to use a new DB on the instance we have. So, same
>>>>>> certs
>>>>>> just a blank DB.
>>>>>>
>>>>>> -- bk
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Later,
>>>>> Lukas @lzap Zapletal
>>>>
>>>>
>>>>
>>>>
>>>
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "foreman-dev" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to foreman-dev+unsubscr...@googlegroups.com.
>> For more options, visit https://groups.google.com/d/optout.
>
>
>
> --
> Later,
> Lukas @lzap Zapletal
--
Later,
Lukas @lzap Zapletal
--
You received this message because you are subscribed to the Google Groups
"foreman-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to foreman-dev+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
