The Cyborg agent in a compute node collects information about
devices from the Cyborg drivers on that node. It then needs to push that
information to the Cyborg conductor in the controller, which then needs
to persist it in the Cyborg db and update Placement. Further, the agent
needs to collect and update this information periodically (or possibly
in response to notifications) to handle hot add/delete of devices,
reprogramming (for FPGAs), health failure of devices, etc.
In this morning's call, we discussed how to do this periodic update .
In particular, we talked about how to compute the difference between the
previous device configuration in a compute node and the current one,
whether the agent do should do that diff or the controller, etc. Since
there are many fields per device, and they are tree-structured, the
complexity of doing the diff seemed large.
On taking a closer look, however, the amount of computation needed to do
the update is not huge. Say, for discussion's sake, that the controller
has a snapshot of the entire device config for a specific compute node,
i.e. an array of device structures NewConfig. It reads the current
list of devices for that node from the db, CurrentConfig. Then the
controller's logic is like this:
* Determine the list of devices in NewConfig but not in
CurrentConfig (this is a set difference in Python ): they are
the newly added ones. For each newly added device, do a single
transaction to add all the fields to the db together.
* Determine the list of devices in CurrentConfig but not in
NewConfig: they are the deleted devices.For each such device, do a
single transaction to delete that entry.
* For each modified device, compute what has changed, and update that
alone. This is the per-field diff.
Say each field in the device structure is a string of 100 characters,
and it takes 1 nanosecond to add, delete or modify a character. So, each
field takes 100 ns to update (add/delete/modify). Say 20 fields per
device: so 2 us to add, delete or modify a device. Say 10 devices per
compute node: so 20 us per node. 500 nodes will take 10 milliseconds.
So, if each node sends a refresh every second, the controller will spend
a very small fraction of that time in updating the db, even including
transaction costs, set difference computation, etc.
This back-of-the-envelope calculation shows that we need not try to
optimize too early: the agent should send the entire device config over
to the controller, and let it update the db per-device and per-field.
OpenStack Development Mailing List (not for usage questions)