Excerpts from Don Zickus's message of 2017-04-20 15:21 -04:00: > Hi Dan, > > Shawn is working with Jon and myself to add system firmware/bios info to the > beaker database to be displayed on the system details page. > > The idea was to make it easy to find how old a bios is on our lab machines > and see which ones need an update. > > Shawn made all the front end changes in the inventory job, but the backend > work is becoming a challenge. > > It seems like we are going to have to add a table entry to the database in > server/model/inventory.py? Which would probably cause a database migration > event. > > For now, we are using numa_nodes as our template to copy the output of lshw > (from the inventory script) into the database and onto the webpage. > > > Are we going in the right direction?
Yes, it's important to settle on a good db structure for this now because it is quite costly to put it in place and to change it in future. The NUMA stuff is not a great example necessarily because it was always intended that we would flesh it out to store more info about NUMA topology, but that never happened. Right now it is just a single table 1-to-1 with "system" with a single column. It could have just been a new column on the system table. It also depends exactly what info you want to store about the firmware. Just an opaque version string? Or also a release date? Assuming the latter, the options I can see would be: * Add a new table "firmware" which is 1-to-1 with system, rows are created for a system when the value is first populated. This is how Cpu, Numa, Power, and others work right now. * Add new columns to the "system" table for firmware version and firmware release date. NULLable with default NULL, and populated by the inventory script if possible. * Represent the system firmware as a device in the existing devices table (in fact it might already be in there from lshw?). You could add new columns to the "device" table for firmware version and firmware release date. Indeed this might make more sense, if for example we want to record details about *other* firmware inside the system besides the actual BIOS (system firmware). Think hard disk firmware, or attached USB devices, or similar. It might help to discuss examples. You said lshw is already extracting the info you want (which is good). Can you paste a piece of the XML showing it? Which XML element does lshw put it into? I have been wanting to gradually move Beaker's device model to more closely match lshw's tree of devices so this might be a good opportunity to bring them into closer alignment. > Is this more work than we thought? Probably. :-) The good news is that you don't need to be *too* scared of adding new db structures. You would have found the existing setup we have with Alembic migrations, and there is an extensive test suite that goes with them. The only downside we have is that they require an outage for deployment. > Then as a follow-on, we are assuming the database has to be updated to > add > this entry, would it make sense to convert the whole database to a generic > key/value table to dynamically support new fields without having to migrate > the whole database all the time? And then to prevent random junk from being > added by the inventory script (or other script), have a whitelist filter > that only allows certain keys to be added/updated. Maintaining the > whitelist would be easier on the database then adding table entries. > > This is just us trying to understand the architecture a little more and > trying to see where we can add some value to make it easier to maintain our > tests. As Nick mentioned, Beaker *does* already have a key-value store for inventory, based on the original design from RHTS, but we are trying to get away from it. Its biggest deficiency is types. Keys can be either strings or numbers, but that is not really structured enough. For example how do you represent multiple disks, each of which has various properties like size and model etc. The existing key-value system has no good way to store and query that kind of info. And the key names tended to be just invented by Beaker admins randomly and then not used consistently. So we prefer instead to build proper db structures to represent the hardware info. (As we did for disks, for example.) The other problem with the existing key-value system is performance in MySQL's query planner, this proved to be quite a nightmarish bug: https://bugzilla.redhat.com/show_bug.cgi?id=590723 For pie-in-the-sky kind of future goals, there are indeed some modern alternatives like JSON columns in Postgres like Nick suggested. But that is pretty far off. We are stuck on MySQL 5.1 for the foreseeable future and migrating MySQL->Postgres is an even bigger task. -- Dan Callaghan <dcall...@redhat.com> Senior Software Engineer, Products & Technologies Operations Red Hat
signature.asc
Description: PGP signature
_______________________________________________ Beaker-devel mailing list -- beaker-devel@lists.fedorahosted.org To unsubscribe send an email to beaker-devel-le...@lists.fedorahosted.org
