> On Oct 17, 2016, at 3:19 AM, Pierre Tardy <tar...@gmail.com> wrote:
> On buildbot eight, the database only store information of the build requests
> before the build is started.
> Switching to a new database you will only lose the buildrequests and not the
> builds which are stored in pickle files.
So, if I wait until all scheduled builds have been completed and the builders
are all idle, I could then stop the system, switch the config to postgresql,
start the system, and not have lost any information about prior builds? If so,
that would be fine.
> There is no tool to migrate data from db to db except what you may find on
> the internet with pure SQL dumps. That should work as the database model is
> exactly the same.
> For your scaling problems i fear there are few hopes that switching db will
> help a lot as the db is not used that much in eight. Multimaster is the usual
> solution to scaling. buildbot nine helps a lot to simplify the setup for
> multimaster config.
I haven't looked into multimaster, but the benefits listed here don't seem to
apply in our situation:
We only have a single worker handling each possible build on each OS version.
We use buildbot to build binaries of ports for MacPorts. When a new version of
macOS is released and we set up a new buildbot worker and builder for it, we
need to build all 20,000 ports on it.
With our old buildbot setup, we would force a single build that would build all
20,000 ports. This build would take several weeks to complete. If a problem was
encountered that caused the build to abort, we had to start over. And when it
did complete, the huge log that was generated caused an out of memory condition
in buildbot, but not before making the web interface unresponsive for a few
With our new buildbot setup, the single build that gets forced in turn triggers
individual builds for each port on another builder. We haven't tried doing this
for all 20,000 ports yet, but I have tried doing around 3,000 ports at once,
and the process of triggering the individual builds took several minutes,
during which the web interface would become unresponsive. Also, looking at a
builder page for a builder that has hundreds of scheduled builds can take a
long time to display.
users mailing list