Hello all.. apologies if zombie thread is inappropriate but I'd like to
revisit this conversation regarding firmware information being included.
We have started a patch to display firmware and date and are hoping to get
some input on progressing. What we were last discussing was to update the
device table. I'm also curious about a migration script. Here is what we
have so far.
Thanks for looking. Shawn.
diff --git a/Server/bkr/server/controllers.py
b/Server/bkr/server/controllers.py
index cece8db..32dc0d4 100644
--- a/Server/bkr/server/controllers.py
+++ b/Server/bkr/server/controllers.py
@@ -123,9 +123,11 @@ def default(self, *args, **kw):
widgets.PaginateDataGrid.Column(name='driver',
getter=lambda x: x.driver, title='Driver', options=dict(sortable=True)),
widgets.PaginateDataGrid.Column(name='vendor_id',
getter=lambda x: x.vendor_id, title='Vendor ID',
options=dict(sortable=True)),
widgets.PaginateDataGrid.Column(name='device_id',
getter=lambda x: x.device_id, title='Device ID',
options=dict(sortable=True)),
-
widgets.PaginateDataGrid.Column(name='subsys_vendor_id', getter=lambda x:
x.subsys_vendor_id, title='Subsys Vendor ID', options=dict(sortable=True)),
+ widgets.PaginateDataGrid.Column(name='subsys_vendor_id',
getter=lambda x: x.subsys_vendor_id, title='Subsys Vendor ID',
options=dict(sortable=True)),
widgets.PaginateDataGrid.Column(name='subsys_device_id', getter=lambda x:
x.subsys_device_id, title='Subsys Device ID', options=dict(sortable=True)),
- ])
+
widgets.PaginateDataGrid.Column(name='subsys_device_id', getter=lambda x:
x.fw_version, title='Firmware Version', options=dict(sortable=True)),
++
widgets.PaginateDataGrid.Column(name='subsys_device_id', getter=lambda x:
x.fw_date, title='Firmware Date', options=dict(sortable=True)),
+ ])
return dict(title="Devices",
grid = devices_grid,
search_bar=None,
diff --git a/Server/bkr/server/model/inventory.py
b/Server/bkr/server/model/inventory.py
index ecbc073..88301ae 100644
--- a/Server/bkr/server/model/inventory.py
+++ b/Server/bkr/server/model/inventory.py
@@ -297,6 +297,8 @@ class System(DeclarativeMappedObject, ActivityMixin):
kernel_type_id = Column(Integer, ForeignKey('kernel_type.id'),
default=select([KernelType.id],
limit=1).where(KernelType.kernel_type==u'default').correlate(None),
nullable=False)
+ fw_version = Column(String(32))
+ fw_date = Column(DateTime, default=None)
kernel_type = relationship('KernelType')
devices = relationship('Device', secondary=system_device_map,
back_populates='systems')
@@ -1246,6 +1248,13 @@ def update(self, inventory):
self.date_modified = datetime.utcnow()
return 0
+ def updateSystemFirmware(self, firmware):
+ # TODO finish this method
+ if firmware
+
+ self.fw_version = firmware['version']
+ self.fw_date = firmware['date']
+
def updateHypervisor(self, hypervisor):
if hypervisor:
hvisor = Hypervisor.by_name(hypervisor)
@@ -2362,6 +2371,8 @@ class Device(DeclarativeMappedObject):
device_class = relationship(DeviceClass)
date_added = Column(DateTime, default=datetime.utcnow, nullable=False)
systems = relationship(System, secondary=system_device_map,
back_populates='devices')
+ fw_version = Column(String(32))
+ fw_date = Column(DateTime, default=None)
Index('ix_device_pciid', Device.vendor_id, Device.device_id)
diff --git a/Server/bkr/server/templates/system_details.kid
b/Server/bkr/server/templates/system_details.kid
index 19ba196..fd6f9d1 100644
--- a/Server/bkr/server/templates/system_details.kid
+++ b/Server/bkr/server/templates/system_details.kid
@@ -156,6 +156,12 @@
<td>
${device.subsys_device_id}
</td>
+ <td>
+ ${device.fw_revision}
+ </td>
+ <td>
+ ${device.fw_date}
+ </td>
</tr>
</tbody>
</table>
diff --git a/systemscan/main.py b/systemscan/main.py
index 64b9312..41895f2 100755
--- a/systemscan/main.py
+++ b/systemscan/main.py
@@ -286,6 +286,14 @@ def read_inventory(inventory, arch = None,
proc_cpuinfo='/proc/cpuinfo'):
#Break the xml into the relevant sets of data
cpuinfo = inventory.xpath(".//node[@class='processor']")[0]
+
+ #system firmware info
+ sysfwinfo = inventory.xpath('.//node[@id="firmware"]')[0]
+ #create dictionary with key-values for version and date
+ SystemFirmware = dict(version = sysfwinfo.findtext('version'), date =
sysfwinfo.findtext('date'))
+ #add to data
+ data['SystemFirmware'] = SystemFirmware
+
memory_elems = inventory.xpath(".//node[@class='memory']")
memoryinfo = None
for m in memory_elems:
On Thu, Apr 27, 2017 at 3:13 AM, Dan Callaghan <dcall...@redhat.com> wrote:
> Excerpts from Don Zickus's message of 2017-04-20 15:21 -04:00:
> > Hi Dan,
> >
> > Shawn is working with Jon and myself to add system firmware/bios info to
> the
> > beaker database to be displayed on the system details page.
> >
> > The idea was to make it easy to find how old a bios is on our lab
> machines
> > and see which ones need an update.
> >
> > Shawn made all the front end changes in the inventory job, but the
> backend
> > work is becoming a challenge.
> >
> > It seems like we are going to have to add a table entry to the database
> in
> > server/model/inventory.py? Which would probably cause a database
> migration
> > event.
> >
> > For now, we are using numa_nodes as our template to copy the output of
> lshw
> > (from the inventory script) into the database and onto the webpage.
> >
> >
> > Are we going in the right direction?
>
> Yes, it's important to settle on a good db structure for this now
> because it is quite costly to put it in place and to change it in
> future.
>
> The NUMA stuff is not a great example necessarily because it was always
> intended that we would flesh it out to store more info about NUMA
> topology, but that never happened. Right now it is just a single table
> 1-to-1 with "system" with a single column. It could have just been a new
> column on the system table.
>
> It also depends exactly what info you want to store about the firmware.
> Just an opaque version string? Or also a release date? Assuming the
> latter, the options I can see would be:
>
> * Add a new table "firmware" which is 1-to-1 with system, rows are
> created for a system when the value is first populated. This is how
> Cpu, Numa, Power, and others work right now.
>
> * Add new columns to the "system" table for firmware version and
> firmware release date. NULLable with default NULL, and populated by
> the inventory script if possible.
>
> * Represent the system firmware as a device in the existing devices
> table (in fact it might already be in there from lshw?). You could add
> new columns to the "device" table for firmware version and firmware
> release date. Indeed this might make more sense, if for example we
> want to record details about *other* firmware inside the system
> besides the actual BIOS (system firmware). Think hard disk firmware,
> or attached USB devices, or similar.
>
> It might help to discuss examples. You said lshw is already extracting
> the info you want (which is good). Can you paste a piece of the XML
> showing it? Which XML element does lshw put it into?
>
> I have been wanting to gradually move Beaker's device model to more
> closely match lshw's tree of devices so this might be a good opportunity
> to bring them into closer alignment.
>
> > Is this more work than we thought?
>
> Probably. :-)
>
> The good news is that you don't need to be *too* scared of adding new db
> structures. You would have found the existing setup we have with Alembic
> migrations, and there is an extensive test suite that goes with them.
> The only downside we have is that they require an outage for deployment.
>
> > Then as a follow-on, we are assuming the database has to be updated to
> > add
> > this entry, would it make sense to convert the whole database to a
> generic
> > key/value table to dynamically support new fields without having to
> migrate
> > the whole database all the time? And then to prevent random junk from
> being
> > added by the inventory script (or other script), have a whitelist filter
> > that only allows certain keys to be added/updated. Maintaining the
> > whitelist would be easier on the database then adding table entries.
> >
> > This is just us trying to understand the architecture a little more and
> > trying to see where we can add some value to make it easier to maintain
> our
> > tests.
>
> As Nick mentioned, Beaker *does* already have a key-value store for
> inventory, based on the original design from RHTS, but we are trying to
> get away from it.
>
> Its biggest deficiency is types. Keys can be either strings or numbers,
> but that is not really structured enough. For example how do you
> represent multiple disks, each of which has various properties like size
> and model etc. The existing key-value system has no good way to store
> and query that kind of info. And the key names tended to be just
> invented by Beaker admins randomly and then not used consistently.
>
> So we prefer instead to build proper db structures to represent the
> hardware info. (As we did for disks, for example.)
>
> The other problem with the existing key-value system is performance in
> MySQL's query planner, this proved to be quite a nightmarish bug:
>
> https://bugzilla.redhat.com/show_bug.cgi?id=590723
>
> For pie-in-the-sky kind of future goals, there are indeed some modern
> alternatives like JSON columns in Postgres like Nick suggested. But that
> is pretty far off. We are stuck on MySQL 5.1 for the foreseeable future
> and migrating MySQL->Postgres is an even bigger task.
>
> --
> Dan Callaghan <dcall...@redhat.com>
> Senior Software Engineer, Products & Technologies Operations
> Red Hat
>
--
Shawn Doherty
Software Engineer, Kernel-HW
Red Hat
<https://www.redhat.com>
314 Littleton Rd
Westford, MA 01886
sdohe...@redhat.com T: 19785891080 INTERNAL:-8131080 IM: sdoherty
<https://red.ht/sig>
_______________________________________________
Beaker-devel mailing list -- beaker-devel@lists.fedorahosted.org
To unsubscribe send an email to beaker-devel-le...@lists.fedorahosted.org
