Hi Chris, Christopher Baines <m...@cbaines.net> skribis:
> This is what I've currently tried to implement. The patch I'll send adds > two new tables to the Cuirass database, one to store events relating to > builds (like it being scheduled, or succeeding), and another to store > the ids of events which haven't yet been sent out. The code relating to > builds is then adjusted to populate these tables, and a new binary is > added to query for unsent events, and then send them out to some URL. So every event only has two states (sent/unsent), which means we assume there’s a single subscriber, right? (Not a limitation because we could use a dedicated hub on top of that like you write, but I want to make sure I understand correctly.) > In the short term, the destination would be the Guix Data Service. In > the longer term, I think it would be better to send events to a WebSub > style hub, which then would distribute the events to one or more > subscribers. That sounds great. > Now that I've actually dug in to the Cuirass database to write this, I'm > more aware of the data model it uses, and the limitations this places on > what information it can provide. I'd assumed for a while that Cuirass > not showing complete information for each evaluation was a UI thing, but > as I understand it, the database only contains a record of what > derivations each evaluation has that no other evaluation in the > currently in the database has. > > Maybe that's something to improve on within Cuirass, but at least by > getting the build information in to the Guix Data Service, it will be > feasible to look at the status of derivations for a revision of Guix (as > the Guix Data Service knows all the derivations associated with a > specific revision). Yeah, though perhaps we should fix that in Cuirass itself, too… > --- > Makefile.am | 8 ++- > bin/cuirass-send-events.in | 90 ++++++++++++++++++++++++ > src/cuirass/base.scm | 6 +- > src/cuirass/database.scm | 135 +++++++++++++++++++++++++++++++++--- > src/cuirass/http.scm | 16 +++++ > src/cuirass/send-events.scm | 39 +++++++++++ > src/schema.sql | 13 ++++ > src/sql/upgrade-5.sql | 16 +++++ > 8 files changed, 310 insertions(+), 13 deletions(-) > create mode 100644 bin/cuirass-send-events.in > create mode 100644 src/cuirass/send-events.scm > create mode 100644 src/sql/upgrade-5.sql [...] > +(define (show-help) > + (format #t "Usage: ~a [OPTIONS]~%" (%program-name)) > + (display "Run build jobs from internal database. Not correct. :-) > +(define (changes-count db) > + (vector-ref (car (sqlite-exec db "SELECT changes();")) > + 0)) Not sure what that does, could you add a docstring? > +(define* (send-build-events target-url > + #:key (batch-limit 100)) Docstring! :-) > +CREATE TABLE BuildEvents ( > + id INTEGER PRIMARY KEY, > + derivation TEXT NOT NULL, > + timestamp INTEGER NOT NULL, > + event TEXT NOT NULL, > + FOREIGN KEY (derivation) REFERENCES Builds (derivation) > +); This assumes build events are necessarily related to a derivation, though one could imagine events having to do with evaluations, jobsets, etc. Should ‘BuildEvents’ be more generic and have ‘event’ be an sexp or JSON string that could describe any kind of event? If we did that, we could keep ‘derivation’ but remove “NOT NULL” so that non-derivation events can exist but we can still query derivation-related events quickly. Does that make sense? > +CREATE TABLE BuildEventsOutbox ( > + build_event_id INTEGER NOT NULL, > + FOREIGN KEY (build_event_id) REFERENCES BuildEvents (id) > +); These are events that have not yet been sent, right? Thanks! Ludo’.
