Hi Gavin also has a proposal to come, but this is the basis of what I discussed with Robert earlier.
The main aim is hyperscale. That is, we want to support hundreds of thousands of nodes. Cobbler simply doesn't allow us to do that. In addition we are currently tied into driving it synchronously so that we can be sure it synced its own database, which causes bugs like: https://bugs.launchpad.net/maas/+bug/989355 The other main issue surrounds DHCP/TFTP and PXE booting. When we write out config files for these services we need to make sure there are no conflicts with multiple processes/threads doing the same thing. Finally, we need to separate appserver requests from provisioning tasks because the latter can be long running (as per the above bug). To this end we discussed: * Use Celery as a task queue (can use Rabbit or Django's DB as a broker) → Django has a Celery plugin which makes this very easy * Have one queue per pserv process * Each pserv is responsible for servicing the tasks on that queue * It scales by having multiple queues each with another pserv, or just multiple pservs pulling from one queue. * The pserv can read the database by using Django's ORM, and write to it by sending API requests back to Django's appserver threads (again, scalable). We also talked about batching requests on the Rabbit task queues because we might want to accumulate several requests that change the configs for DHCP etc. For now though we can process serially and optimise later. We'd need to think about how to implement locking though if there are multiple writers to the same files. J -- Mailing list: https://launchpad.net/~maas-devel Post to : [email protected] Unsubscribe : https://launchpad.net/~maas-devel More help : https://help.launchpad.net/ListHelp
