Pierre, 9.3 seems to have done the trick. Weird I was the only one having such a huge perf degradation.
Francesco On Fri, Feb 3, 2017 at 4:55 PM, Pierre Tardy <[email protected]> wrote: > Buildbot <0.9.3 did compress the whole log at once inside a transaction. I > can imagine this taking very long with 100% CPU. > > > Le mer. 1 févr. 2017 18:12, Francesco Di Mizio <[email protected]> > a écrit : > >> Hey Pierre, >> >> what I noticed is that the query does not take long at all when I run it >> myself. >> When it is issued through buildbot the state though, it is in a 'idle in >> transaction' state, which seems to be something bad according to many blogs >> on the web. As if bbot is waiting for postgres to do something but postgres >> isnt actually doing anything. >> >> Will give 9.3 but at this point I doubt it'll help. >> >> Thanks! >> >> >> On Wed, Feb 1, 2017 at 4:56 PM, Pierre Tardy <[email protected]> wrote: >> >> >> >> Le lun. 30 janv. 2017 à 17:49, Francesco Di Mizio < >> [email protected]> a écrit : >> >> Digging this up again. Havent moved to a physical server yet. I will >> first be splitting the bbot server and postgres on different hosts. >> >> What I've noticed the CPU spiking is always (and I rea) due to queries >> like: >> SELECT logchunks.first_line, logchunks.last_line, logchunks.content, >> logchunks.compressed FROM logchunks WHERE logchunks.logid = 56343 ORDER BY >> logchunks.first_line >> >> If I am correct that call can only come from CompressLog >> <https://github.com/buildbot/buildbot/blob/cfe60e31206748bcf01232eb13c0d6177cd04d5d/master/buildbot/db/logs.py#L252> >> . >> They do take a very long time (even 1 minute). While running all the system >> is stuck. I wonder if I should use some custom postgres config, I am simply >> running the official postgres 9.5 container. >> >> Hi Francesco, >> If your logs are not anormally large, like you said already this should >> not take one minute. >> >> That said, I have released in 0.9.3 an optimization that removes that >> particular request so that the compression happens on a stream of smaller >> requests. >> >> You might consider an upgrade in order to see if the situation improves. >> >> Pierre >> >> >>
_______________________________________________ users mailing list [email protected] https://lists.buildbot.net/mailman/listinfo/users
