Hmmm if I'd have to create new koji I'd rather start from scratch
reimporting our repos back in.

Current problems we had:

- small vm with not enough cores/RAM to run builds
- outdated system (RHEL6) and koji version
- no use of LVM
- data on ephemeral volumes
- lots of symlinks which were added as we grew

If I'd need recreate Koji from scratch, I'd try to avoid all of these.
Local volumes should be only used as cache, swap and temp, external
repos should be on external NFS, everything else on dedicated block
device with regular backups.

On Fri, Jun 9, 2017 at 2:33 PM, Bryan Kearney <bryan.kear...@gmail.com> wrote:
> On 06/09/2017 08:27 AM, Lukas Zapletal wrote:
>>
>> I have some bad news to share. I started on moving from our 2nd koji
>> to 3rd koji created as m1.xxlarge instance so we would avoid NFS and
>> could run on a single VM. I prepared everything and when I was
>> shutting down all the builders and test VMs I accidentaly stopped the
>> 2nd koji. Unfortunately the original koji instance was stopped
>> yesterday as well. We lost all data from epheremal storage volumes,
>> these were /mnt/tmp holding most importantly Postgres database for
>> kojihub and not sure what else. Our deployment was dozen of symlinks
>> across two local volumes and EBS.
>>
>> I found some backups of pgsql on EBS volume, but unfortunately these
>> stopped being dumped summer 2016
>> (koji.katello.org-kojidb-20160830-230505.dump is the newest one). Here
>> are the options we have:
>>
>> 1) Restore the DB and attempt to start services back - but this will
>> give us instance back with huge gap of almost one year. I am not sure
>> how much useful that would be - we will not be able to bu ild into
>> 1.15/1.14 repos, missing tags, builds, logs.
>>
>> 2) Move to https://cbs.centos.org CentOS CBS, they run their own Koji
>> and start building there
>>
>> 3) Speed up Copr transition
>>
>> 4) Build whole new Koji from scratch
>
>
> A variation here is to use a new DB on the instance we have. So, same certs
> just a blank DB.
>
> -- bk



-- 
Later,
  Lukas @lzap Zapletal

-- 
You received this message because you are subscribed to the Google Groups 
"foreman-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to foreman-dev+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Reply via email to