Interesting contemplation Thomi :), and thanks a lot for the link on CQRS.
On a side note, ever since I started getting involved in agile processes, the use of the word 'hypothesis' has been diminishing in my daily work :)
But surely this will help when we have such a requirement. Thanks On 09/05/15 21:24, Thomi Richards wrote:
Hi, Thanks for the trip report, there looks like some really interesting things there. I've been thinking about event-sourced services and the CQRS pattern for a while now. It seems like a reasonably good strategy for a service where data consistency is important. I don't think we have built such a service yet. However, imagine a hypothetical example..... We want to build a UI showing the current state of the proposed-migration system for developers. It should show what packages are being processed ("in-flight"), where in the system they are, as well as the results of any that have completed, in chronological order. The issue we hit straight away is that you can't easily inspect the contents of a rabbit queue without de-queueing and re-queueing all the messages. Let's not do that. We could solve this with a service (package-tracker) that listened on HTTP. All the other services would send it messages ("I got a new request for package 'linux'", or "I'm starting to test package 'autopilot'", or "I'm publishing results for package 'unity'"). The package-tracker could then store those in a database, and provide a nice API & UI on top of the database. That would work, but every time we need to re-deploy the service we'd need to either: - Copy the old database file to the new host (if it's using something on-disk like sqlite, json flat file or whatever). This makes our deployments more icky than we'd want, I think - Point the new instance at the separate postgres server. This means we now have to keep a sql database schema compatible, or deal with schema upgrades. noooooo! - Use event-sourced service (see below) With an event sourced model, we would: 1. Use a tool to store those events (new package, start testing, publish results etc) in a queue. We could use rabbit, but there are better systems out there. 2. Build a service that connects to that tool, and when it starts, builds an in-memory model of everything that's happened by replaying *all* events, since the dawn of time. This is surprisingly efficient: - You can scan millions of events in seconds. - If you ever get too many events, you can safely discard the oldest ones. - Since we're only showing a UI on the last N days (a week? a month?) you can discard packages from your in-memory model as soon as you get a "published results" message with a timestamp outside your time-window. There are some disadvantages as well though: - You need to maintain compatibility in the service for old event formats. Then again, if you do break compatibility, you can just blow away the event queue and your UI will be empty until it gets repopulated. - It requires a separate event queue system, maintenance etc. etc. Fowler does a better job of explaining CQRS than I can: http://martinfowler.com/bliki/CQRS.html If we ever build such a system, I think we should consider something like this.... Cheers, On Sun, May 10, 2015 at 2:27 AM, psivaa <[email protected]> wrote:Hi, Here comes the trip report as promised, There were 3 talks in this meet up, containing a lot very interesting content. They also had some debatable content. Because it was about the subject of uServices, the authors had their own take on the key part of what micro services are for and advocated the communication methods based on that. **This** should not be new to us :P. The link to the whole meet-up is: https://skillsmatter.com/meetups/7005-inter-service-communication The report on it is in: https://docs.google.com/document/d/1ejlPMGwWrpjtmdlfqUGFDSsJ8YKRqSXTltkgZowT84M/edit# Thanks -- Mailing list: https://launchpad.net/~canonical-ci-engineering Post to : [email protected] Unsubscribe : https://launchpad.net/~canonical-ci-engineering More help : https://help.launchpad.net/ListHelp
-- Mailing list: https://launchpad.net/~canonical-ci-engineering Post to : [email protected] Unsubscribe : https://launchpad.net/~canonical-ci-engineering More help : https://help.launchpad.net/ListHelp
