Well, the benefit is that we don't have engineers and managers spending cycles looking at the log and wondering if there was a timeout due to network latency! (though it would be interesting to see how effective the caching on the proxy is at mitigating this, as well - maybe it IS a non issue.)
I understand about always testing with the latest images, but I think we can live with the small delay introduced by the mirroring. (should be measured in minutes, but if it's measured in hours+, that means our users are probably hurting too! and it would be nice to be able to measure that separately.) Agree on the need for a promotion job, but that seems like a larger task since you'd have to verify the image on a wide variety of hardware, no? Regards, Mike On Thu, Jul 9, 2015 at 5:15 PM, Blake Rouse <[email protected]> wrote: > Also a proxy is already used so those files should be cached in the test > environment if they haven't changed. > On Jul 9, 2015 8:14 PM, "Blake Rouse" <[email protected]> wrote: > >> I really don't see a benefit, it would only make the import faster. We >> never had this issue before, something in the code has affected it. >> >> Also the CI tests that the images work on deployed nodes. If a mirror is >> used then that test would be delayed until the mirror is updated as well. >> Keeping the CI in sync with maas.ubuntu.com will make sure not only that >> the mirror is always working but also the images in the mirror actually >> work and deploy. >> >> Now I do think there needs to be a CI job that takes new images from the >> daily stream test them and then promote them to release stream. >> On Jul 9, 2015 8:09 PM, "Mike Pontillo" <[email protected]> >> wrote: >> >>> I seriously think it would be better if we created a separate job for >>> mirroring the images, and the CI could just use the downloaded mirror. >>> >>> Mike >>> >>> On Thu, Jul 9, 2015 at 5:07 PM, Blake Rouse <[email protected]> >>> wrote: >>> >>>> Haha. We don't need another, we got this one. :-) >>>> On Jul 9, 2015 8:02 PM, "Mike Pontillo" <[email protected]> >>>> wrote: >>>> >>>>> Then we need another CI to test that! ;-) >>>>> >>>>> On Thu, Jul 9, 2015 at 4:53 PM, Blake Rouse <[email protected] >>>>> > wrote: >>>>> >>>>>> I disagree as using the maas.ubuntu.com images makes sure that >>>>>> service is working as everyone else relies on it. >>>>>> On Jul 9, 2015 7:37 PM, "Mike Pontillo" <[email protected]> >>>>>> wrote: >>>>>> >>>>>>> Maybe we should change the CI to use a local mirror of the images, >>>>>>> to make the CI faster and eliminate side effects from network latency. >>>>>>> >>>>>>> Mike >>>>>>> >>>>>>> On Tue, Jul 7, 2015 at 7:38 AM, Andres Rodriguez < >>>>>>> [email protected]> wrote: >>>>>>> >>>>>>>> Yeah, so then the only issues we need to figure out are: >>>>>>>> >>>>>>>> 1. Why is the region trying to access clusterd.conf >>>>>>>> 2. Why the tests failing to pass? Is it not importing for another >>>>>>>> issue or is it just taking longer than expected, since judging by the >>>>>>>> logs, >>>>>>>> both the cluster and region are up. >>>>>>>> >>>>>>>> On Tue, Jul 7, 2015 at 9:33 AM, Gavin Panella < >>>>>>>> [email protected]> wrote: >>>>>>>> >>>>>>>>> On 7 July 2015 at 13:47, Andres Rodriguez >>>>>>>>> <[email protected]> wrote: >>>>>>>>> > Hi Gavin, >>>>>>>>> >> >>>>>>>>> >> >>>>>>>>> >> > http://162.213.35.104:8080/job/maas-utopic-trunk/576/console >>>>>>>>> >> > >>>>>>>>> >> > >>>>>>>>> http://162.213.35.104:8080/job/maas-utopic-trunk/576/artifact/results/artifacts/maas-logs/var/log/maas/regiond.log >>>>>>>>> >> > >>>>>>>>> >> > >>>>>>>>> http://162.213.35.104:8080/job/maas-utopic-trunk/576/artifact/results/artifacts/maas-logs/var/log/maas/clusterd.log >>>>>>>>> >> > >>>>>>>>> >> > So, the last issue seems liek the test does not wait long >>>>>>>>> enough for >>>>>>>>> >> > the images to be present, however, that's caused for other >>>>>>>>> things (see >>>>>>>>> >> > regiond.log): >>>>>>>>> >> > >>>>>>>>> >> > 1. OSError: [Errno 13] Permission denied: >>>>>>>>> '/etc/maas/regiond.conf' >>>>>>>>> >> > 2. django.db.utils.OperationalError: fe_sendauth: no password >>>>>>>>> supplied >>>>>>>>> >> > 3. exceptions.OSError: [Errno 13] Permission denied: >>>>>>>>> >> > '/etc/maas/clusterd.conf' >>>>>>>>> >> > >>>>>>>>> >> > So, by the looks of it, it seems that after the installation >>>>>>>>> goes >>>>>>>>> >> > through the regiond.conf is not with the correct permissions. >>>>>>>>> >> > Packaging itself does not configure nor changes the >>>>>>>>> permissions. So >>>>>>>>> >> > questions are: >>>>>>>>> >> > 1. Why are the daemons not creating the configs with correct >>>>>>>>> >> > permissions? >>>>>>>>> >> >>>>>>>>> >> I'm investigating this first. This was working, for the region. >>>>>>>>> At the >>>>>>>>> >> time I had to make several changes in trunk and in packaging to >>>>>>>>> get this >>>>>>>>> >> working. I haven't yet figured out what has made it regress. >>>>>>>>> >> >>>>>>>>> > >>>>>>>>> > After some investigation, the permissions issue is only seen for >>>>>>>>> a >>>>>>>>> > while, but it gets to the point that it is correct and the >>>>>>>>> daemons can >>>>>>>>> > read it just fine. >>>>>>>>> >>>>>>>>> I've figured out a fix for this. >>>>>>>>> >>>>>>>>> > >>>>>>>>> >> >>>>>>>>> >> > 2. What causes the permissions to be changed correctly later >>>>>>>>> on? >>>>>>>>> >> > >>>>>>>>> >> > Then, after the daemon successfully loads regiond.conf, no >>>>>>>>> password is >>>>>>>>> >> > supplied to the DB: >>>>>>>>> >> > 3. So, why does the daemon seem not to have the DB password >>>>>>>>> if it was >>>>>>>>> >> > supposed to be set? >>>>>>>>> >> >>>>>>>>> >> This is probably a symptom of #1. >>>>>>>>> >>>>>>>>> The database password is configured by >>>>>>>>> maas-region-controller.postinst, >>>>>>>>> but is needed by maas-region-controller-min which is started >>>>>>>>> before the >>>>>>>>> database has been configured. The errors do eventually go away. >>>>>>>>> >>>>>>>>> >> >>>>>>>>> >> > >>>>>>>>> >> > Last, but not least, the regiond is trying to access >>>>>>>>> cluster.conf: >>>>>>>>> >> > 4. Why is the region daemon trying to access cluster.conf ? >>>>>>>>> This >>>>>>>>> >> > should not be happening at all. >>>>>>>>> >> >>>>>>>>> >> This looks like a bug, but I'm not 100% sure about that. >>>>>>>>> > >>>>>>>>> > >>>>>>>>> > yup. IIRC, i mentioned this before. For whatever reason the >>>>>>>>> region is >>>>>>>>> > trying to access this. >>>>>>>>> >> >>>>>>>>> >> >>>>>>>>> >> > >>>>>>>>> >> > Gavin, can you please look at the above and fix asap? >>>>>>>>> >> > >>>>>>>>> >> > Thanks. >>>>>>>>> > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> > -- >>>>>>>>> > Andres Rodriguez >>>>>>>>> > Engineering Manager, MAAS & OIL DevOps >>>>>>>>> > Canonical USA, Inc. >>>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> -- >>>>>>>> Andres Rodriguez >>>>>>>> Engineering Manager, MAAS & OIL DevOps >>>>>>>> Canonical USA, Inc. >>>>>>>> >>>>>>>> -- >>>>>>>> Mailing list: https://launchpad.net/~maas-builds >>>>>>>> Post to : [email protected] >>>>>>>> Unsubscribe : https://launchpad.net/~maas-builds >>>>>>>> More help : https://help.launchpad.net/ListHelp >>>>>>>> >>>>>>>> >>>>>>> >>>>>>> -- >>>>>>> Mailing list: https://launchpad.net/~maas-builds >>>>>>> Post to : [email protected] >>>>>>> Unsubscribe : https://launchpad.net/~maas-builds >>>>>>> More help : https://help.launchpad.net/ListHelp >>>>>>> >>>>>>> >>>>> >>>
-- Mailing list: https://launchpad.net/~maas-builds Post to : [email protected] Unsubscribe : https://launchpad.net/~maas-builds More help : https://help.launchpad.net/ListHelp
