Hi Pierre,
As always, thanks for steering me in the right direction.
First off before we get to the interesting stuff, a couple nits in the
documentation...
Here
(http://docs.buildbot.net/latest/manual/cfg-global.html#mq-specification)
I see:
c['mq'] = {
'type' : 'wamp',
'router_url': 'ws://url/to/crossbar'
'realm': 'buildbot'
'debug' : False,
'debug_websockets' : False,
'debug_lowlevel' : False,
} It would be very, very helpful to those doing a copy/paste on that to
include commas after the crossbar URL and the realm. It would also be
useful to include that unicode indicator before 'buildbot'. It might
also be helpful to add host:port to the url, just so the user doesn't
forget to put it in there.
Now on to the good stuff.
On 9/6/2016 9:18 AM, Pierre Tardy wrote:
You need to configure all the force-schedulers on your UI masters.
forceschedulers defined in the non-UI masters will not be used, as
they have to be in the UI.
This is tracked on http://trac.buildbot.net/ticket/2673
This is good to know, and not hard to do. Most of my users want to be
able to force builds.
the masters are talking to each other via crossbar. an easy way to
make sure this work is to start a build on a non-ui master, and see it
auto-update in the ui-master.
While I was not using symmetric masters, I'm certain I got this working,
even though I was doing the force in the UI master.
If you get autoupdate then the mq is correctly working. If you have to
reload the UI (F5) to see build progress, then it means the ui master
can only get updated data via DB, and not via message queue.
You might also have a look at tcpdump on the crossbar port to see the
activity to crossbar.
I think it's sufficient to run crossbar in the foreground using a
loglevel of debug. Since each master has a different signature, you can
see them talk. Probably a bit easier (at least for experimentation) than
using tcpdump.
I would recommend to first start with symetric configuration where all
the masters have UI, and all masters have all builders/schedulers
I can see where it might be easier to test things this way, but the
whole point for us to use multi-master is to NOT have all
builders/schedulers active on the same masters. For our purposes, it'll
be bad enough that the database and crossbar are single points of failure.
I still see what I interpret as 'lost deferred' problems often, and I
think we're leery of not being able tor recover from them easily in a
way that doesn't inconvenience people by having to take down a master
controlling a build that they're waiting for. Every time a deferred is
lost and the master effectively stops talking to the worker, it's
unpleasant. And it happens pretty often for us.
Anyway, I have asymmetric multi-master working in a small test setup
just using mostly the defaults and a postgres database (as I'm not
familiar with MySQL). I may even be able to create that tutorial you'd
like (but no promises).
Now I just have to get it working with our master.cfg. I think the plan
is to use the same master.cfg for every master, but have it process
differently for each master. Part of the idea I have is that if someone
needed to, they could run their own master fairly easily, tied in to the
overall system. We definitely plan to have the UI as separate as
possible, as well as a couple other things that people tend to complain
about when they aren't running.
Neil Gilmore
grammatech.com
_______________________________________________
users mailing list
[email protected]
https://lists.buildbot.net/mailman/listinfo/users
