Re: [Qemu-devel] E820 (Re: [v4 PATCH 00/12] SMBIOS: build full tables in QEMU)

[Gabriel L. Somlo]

On Mon, Apr 07, 2014 at 02:57:08PM -0400, Kevin O'Connor wrote:
> On Mon, Apr 07, 2014 at 02:05:21PM -0400, Gabriel L. Somlo wrote:
> > On Mon, Apr 07, 2014 at 11:23:44AM -0400, Kevin O'Connor wrote:
> > > So, I'm suggesting QEMU produce two new fw_cfg files: an anchor file
> > > with the valid anchor table (the address pointer can be just set to
> > > zero), and an smbios table file with the complete set of smbios tables
> > > formatted according to the smbios spec.  SeaBIOS can then use the
> > > existence of one of these new files to determine if it should deploy
> > > (and optionally modify) them or if it should use the old smbios
> > > generation code.
> > 
> > Oh, OK. Right now we have (in qemu):
> > 
> > #define SMBIOS_FIELD_ENTRY 0
> > #define SMBIOS_TABLE_ENTRY 1
> > 
> > I will be adding (actually, migrating to):
> > 
> > #define SMBIOS_ANCHOR_ENTRY 2 /* for the smbios entry point table */
> > #define SMBIOS_FULLTABLE_ENTRY 3 /* for the blob containing all types */
> 
> No - don't do that.  Lets leave the existing smbios fw_cfg entry
> (0x8001) unchanged.  Instead, introduce two new fw_cfg files using the
> fw_cfg_add_file() interface (eg, "etc/smbios/smbios-anchor" and
> "etc/smbios/smbios-tables").

OK.

> > I can add such a structure for the anchor/entrypoint table and for the
> > full blob-of-tables payload, in which I can tell you how big type 0 is,
> > so the BIOS (SeaBIOS/TianoCore) side surgery can be made that much
> > easier...
> 
> It's trivial for the firmware to calculate this on its own, so I
> recommend just putting the anchor table and main tables unmodified in
> their respective fw_cfg files.

Not sure why it never occurred to me before (lack of caffeine, or various
day-job related distractions :) ) but if the QEMU default dummy type 0
table is just that -- a dummy table -- there's *nothing* preventing me
from leaving all (three) of its strings *empty* :)

Then we'll know *exactly* what the size of type 0 is, when it's time
to surgically transplant a new one within the BIOS...

I remember neither Windows, Linux (F20 live) or OS X objecting to a
type 0 smbios table with all-undefined strings, during some previous
tests.

I'll try to send out an updated patch set later in the week.

Thanks again,
--Gabriel

Re: [Qemu-devel] E820 (Re: [v4 PATCH 00/12] SMBIOS: build full tables in QEMU)

[Kevin O'Connor]

On Mon, Apr 07, 2014 at 02:05:21PM -0400, Gabriel L. Somlo wrote:
> On Mon, Apr 07, 2014 at 11:23:44AM -0400, Kevin O'Connor wrote:
> > So, I'm suggesting QEMU produce two new fw_cfg files: an anchor file
> > with the valid anchor table (the address pointer can be just set to
> > zero), and an smbios table file with the complete set of smbios tables
> > formatted according to the smbios spec.  SeaBIOS can then use the
> > existence of one of these new files to determine if it should deploy
> > (and optionally modify) them or if it should use the old smbios
> > generation code.
> 
> Oh, OK. Right now we have (in qemu):
> 
> #define SMBIOS_FIELD_ENTRY 0
> #define SMBIOS_TABLE_ENTRY 1
> 
> I will be adding (actually, migrating to):
> 
> #define SMBIOS_ANCHOR_ENTRY 2 /* for the smbios entry point table */
> #define SMBIOS_FULLTABLE_ENTRY 3 /* for the blob containing all types */

No - don't do that.  Lets leave the existing smbios fw_cfg entry
(0x8001) unchanged.  Instead, introduce two new fw_cfg files using the
fw_cfg_add_file() interface (eg, "etc/smbios/smbios-anchor" and
"etc/smbios/smbios-tables").

[...]
> I can add such a structure for the anchor/entrypoint table and for the
> full blob-of-tables payload, in which I can tell you how big type 0 is,
> so the BIOS (SeaBIOS/TianoCore) side surgery can be made that much
> easier...

It's trivial for the firmware to calculate this on its own, so I
recommend just putting the anchor table and main tables unmodified in
their respective fw_cfg files.

-Kevin

Re: [Qemu-devel] E820 (Re: [v4 PATCH 00/12] SMBIOS: build full tables in QEMU)

[Gabriel L. Somlo]

On Mon, Apr 07, 2014 at 11:23:44AM -0400, Kevin O'Connor wrote:
> So, I'm suggesting QEMU produce two new fw_cfg files: an anchor file
> with the valid anchor table (the address pointer can be just set to
> zero), and an smbios table file with the complete set of smbios tables
> formatted according to the smbios spec.  SeaBIOS can then use the
> existence of one of these new files to determine if it should deploy
> (and optionally modify) them or if it should use the old smbios
> generation code.

Oh, OK. Right now we have (in qemu):

#define SMBIOS_FIELD_ENTRY 0
#define SMBIOS_TABLE_ENTRY 1

I will be adding (actually, migrating to):

#define SMBIOS_ANCHOR_ENTRY 2 /* for the smbios entry point table */
#define SMBIOS_FULLTABLE_ENTRY 3 /* for the blob containing all types */

The cool thing here is that, along with the payload for each type, I
can create a wrapper structure, like there already exists for fields
and individual table types:

struct smbios_field {
struct smbios_header header;
uint8_t type;
uint16_t offset;
uint8_t data[];
} QEMU_PACKED;

struct smbios_table {
struct smbios_header header;
uint8_t data[];
} QEMU_PACKED;

I can add such a structure for the anchor/entrypoint table and for the
full blob-of-tables payload, in which I can tell you how big type 0 is,
so the BIOS (SeaBIOS/TianoCore) side surgery can be made that much
easier...

Thanks,
--Gabriel

Re: [Qemu-devel] E820 (Re: [v4 PATCH 00/12] SMBIOS: build full tables in QEMU)

[Kevin O'Connor]

On Mon, Apr 07, 2014 at 10:49:54AM -0400, Gabriel L. Somlo wrote:
> On Mon, Apr 07, 2014 at 10:14:36AM -0400, Kevin O'Connor wrote:
> > How about having QEMU produce the smbios table with a dummy type0
> > table and then both seabios and ovmf can replace the type0 table if
> > desired.  After all, if OVMF is splitting the blob into tables, it can
> > just as easily replace type0 as append it.
> > 
> > This way, the QEMU output is technically complete.  And if someone
> > wishes to code up SeaBIOS to do the type0 replace (I'm not convinced
> > it's even necessary) then at least that SeaBIOS code could be used on
> > coreboot as well.
> 
> OK, so as far as I'm concerned, it's down to two alternatives:
> 
> 1. I create a full blob, starting with the anchor/entrypoint, and
> followed by a (dummy) type 0, type 1, etc, etc. all the way to
> the type 127 end marker. Pass that in via fw_cfg, and each BIOS
> is then responsible for editing type 0, overwriting it with
> appropriate values.
> 
> I like this very much :) except for a serious discomfort with the idea
> of imposing an additional "convention" on the size (and choice of)
> strings included with the type0 table, beyond the smbios spec.

I agree that was too ugly.  What I'm proposing now is that QEMU
produce a valid type0 table (with strings populated, but no special
padding) along with all the other tables (up to and including type
127).

Then SeaBIOS and OVMF can either pass this on exactly as is, or they
can modify the table to replace type0.

This is a few more lines of code for SeaBIOS (to replace type0 instead
of just patching it), but it has the advantage that QEMU developers
know they must produce a valid smbios table according to the specs and
the firmware developers know they will get a valid smbios table
(according to the specs) and know they must ultimately produce a valid
smbios table.

So, I'm suggesting QEMU produce two new fw_cfg files: an anchor file
with the valid anchor table (the address pointer can be just set to
zero), and an smbios table file with the complete set of smbios tables
formatted according to the smbios spec.  SeaBIOS can then use the
existence of one of these new files to determine if it should deploy
(and optionally modify) them or if it should use the old smbios
generation code.

-Kevin

Re: [Qemu-devel] E820 (Re: [v4 PATCH 00/12] SMBIOS: build full tables in QEMU)

[Gabriel L. Somlo]

On Mon, Apr 07, 2014 at 10:14:36AM -0400, Kevin O'Connor wrote:
> How about having QEMU produce the smbios table with a dummy type0
> table and then both seabios and ovmf can replace the type0 table if
> desired.  After all, if OVMF is splitting the blob into tables, it can
> just as easily replace type0 as append it.
> 
> This way, the QEMU output is technically complete.  And if someone
> wishes to code up SeaBIOS to do the type0 replace (I'm not convinced
> it's even necessary) then at least that SeaBIOS code could be used on
> coreboot as well.

OK, so as far as I'm concerned, it's down to two alternatives:

1. I create a full blob, starting with the anchor/entrypoint, and
followed by a (dummy) type 0, type 1, etc, etc. all the way to
the type 127 end marker. Pass that in via fw_cfg, and each BIOS
is then responsible for editing type 0, overwriting it with
appropriate values.

I like this very much :) except for a serious discomfort with the idea
of imposing an additional "convention" on the size (and choice of)
strings included with the type0 table, beyond the smbios spec.


2. I create a third fw_cfg smbios "type" for the entry point
structure.

The BIOS (e.g. smbios_setup() in SeaBIOS) checks for the presence of
this blob *first*. If present, it simply grabs all table blobs from
fw_cfg and assembles them (e.g. via get_external()). Create its own
type 0 if not already included in fw_cfg, and recompute the checksum for
the entry point.

If the entry point blob is not found, smbios_setup() proceeds with its
current logic (looking for table or field blobs in fw_cfg, and
creating its own entry point structure).


Now, 2 would be uglier, hands down, but has the advantage of not
adding arbitrary restrictions on what the type0 structure can look
like.



If we go with 1 (all I need is you all to say "go for it, we're OK
with arbitrary restrictions on type0" :) ), I'll add prominent
comments in the qemu smbios source about what the restrictions are.

We have vendor, version, and date, currently. Date can be "-MM-DD"
or somesuch (for a total strlen of 10, not including \0).

Any idea what the longest "vendor" and "version" string might look
like ?

What do we do if we have *shorter* strings than that, fill with
spaces to keep the hardcoded strlen intact across the qemu/bios
demarc ?


Thanks,
--Gabriel

Re: [Qemu-devel] E820 (Re: [v4 PATCH 00/12] SMBIOS: build full tables in QEMU)

[Laszlo Ersek]

On 04/07/14 16:14, Kevin O'Connor wrote:
> On Mon, Apr 07, 2014 at 09:09:56AM +0200, Gerd Hoffmann wrote:
 The only fly in this ointment may be that type 0 doesn't have a fixed
 length that could be edited in place, if you consider the various
 strings that get tacked on to the end of it. So you'd still have to
 slide the rest of the smbios payload left or right to shrink or
 enlarge the type 0 blob, depending on how you modify the various
 strings it contains...
>>>
>>> The dummy type 0 subtable that QEMU generates can have dummy space
>>> padded strings that the firmware can overwrite.  Until recently, the
>>> max size smbios string was 64 bytes, so that size could be used.  (As
>>> above, I admit that this is ugly, but the alternatives also seem
>>> ugly.)  Another option would be to just leave the strings at a QEMU
>>> default as that's no different from what SeaBIOS does today.
>>
>> I don't think we need to make it that complicated.  smbios tables don't
>> have any references, right?  I mean any references which would need a
>> fixup (such as table pointers in RSDP in acpi) and therefore would need
>> the romfile_loader.  The string references within a table are relative
>> don't need special care.
> 
> The smbios anchor table needs to have the address of the main smbios
> table.  It would be preferable to get the anchor table from qemu as
> the anchor table has the smbios version info.
> 
> But, anchor table aside, you are correct.
> 
>> Gabriel has code to generate all tables needed in qemu meanwhile, so I
>> think we can simply have a blob in fw_cfg with all tables (except
>> type0).  firmware generates type0 table like it does today, then simply
>> appends the fw_cfg blob as-is, then appends a end-of-tables marker.
>> Done.
>>
>> OVMF probably would have to parse the blob, split it into tables, then
>> install them one by one.  But I suspect that will be less code than
>> dealing with the complex smbios fw_cfg interface we have today ...
> 
> How about having QEMU produce the smbios table with a dummy type0
> table and then both seabios and ovmf can replace the type0 table if
> desired.  After all, if OVMF is splitting the blob into tables, it can
> just as easily replace type0 as append it.
> 
> This way, the QEMU output is technically complete.  And if someone
> wishes to code up SeaBIOS to do the type0 replace (I'm not convinced
> it's even necessary) then at least that SeaBIOS code could be used on
> coreboot as well.

Works for me.

Thanks
Laszlo

Re: [Qemu-devel] E820 (Re: [v4 PATCH 00/12] SMBIOS: build full tables in QEMU)

[Kevin O'Connor]

On Mon, Apr 07, 2014 at 09:09:56AM +0200, Gerd Hoffmann wrote:
> > > The only fly in this ointment may be that type 0 doesn't have a fixed
> > > length that could be edited in place, if you consider the various
> > > strings that get tacked on to the end of it. So you'd still have to
> > > slide the rest of the smbios payload left or right to shrink or
> > > enlarge the type 0 blob, depending on how you modify the various
> > > strings it contains...
> > 
> > The dummy type 0 subtable that QEMU generates can have dummy space
> > padded strings that the firmware can overwrite.  Until recently, the
> > max size smbios string was 64 bytes, so that size could be used.  (As
> > above, I admit that this is ugly, but the alternatives also seem
> > ugly.)  Another option would be to just leave the strings at a QEMU
> > default as that's no different from what SeaBIOS does today.
> 
> I don't think we need to make it that complicated.  smbios tables don't
> have any references, right?  I mean any references which would need a
> fixup (such as table pointers in RSDP in acpi) and therefore would need
> the romfile_loader.  The string references within a table are relative
> don't need special care.

The smbios anchor table needs to have the address of the main smbios
table.  It would be preferable to get the anchor table from qemu as
the anchor table has the smbios version info.

But, anchor table aside, you are correct.

> Gabriel has code to generate all tables needed in qemu meanwhile, so I
> think we can simply have a blob in fw_cfg with all tables (except
> type0).  firmware generates type0 table like it does today, then simply
> appends the fw_cfg blob as-is, then appends a end-of-tables marker.
> Done.
> 
> OVMF probably would have to parse the blob, split it into tables, then
> install them one by one.  But I suspect that will be less code than
> dealing with the complex smbios fw_cfg interface we have today ...

How about having QEMU produce the smbios table with a dummy type0
table and then both seabios and ovmf can replace the type0 table if
desired.  After all, if OVMF is splitting the blob into tables, it can
just as easily replace type0 as append it.

This way, the QEMU output is technically complete.  And if someone
wishes to code up SeaBIOS to do the type0 replace (I'm not convinced
it's even necessary) then at least that SeaBIOS code could be used on
coreboot as well.

-Kevin

Re: [Qemu-devel] E820 (Re: [v4 PATCH 00/12] SMBIOS: build full tables in QEMU)

[Gerd Hoffmann]

  Hi,

> > The only fly in this ointment may be that type 0 doesn't have a fixed
> > length that could be edited in place, if you consider the various
> > strings that get tacked on to the end of it. So you'd still have to
> > slide the rest of the smbios payload left or right to shrink or
> > enlarge the type 0 blob, depending on how you modify the various
> > strings it contains...
> 
> The dummy type 0 subtable that QEMU generates can have dummy space
> padded strings that the firmware can overwrite.  Until recently, the
> max size smbios string was 64 bytes, so that size could be used.  (As
> above, I admit that this is ugly, but the alternatives also seem
> ugly.)  Another option would be to just leave the strings at a QEMU
> default as that's no different from what SeaBIOS does today.

I don't think we need to make it that complicated.  smbios tables don't
have any references, right?  I mean any references which would need a
fixup (such as table pointers in RSDP in acpi) and therefore would need
the romfile_loader.  The string references within a table are relative
don't need special care.

Gabriel has code to generate all tables needed in qemu meanwhile, so I
think we can simply have a blob in fw_cfg with all tables (except
type0).  firmware generates type0 table like it does today, then simply
appends the fw_cfg blob as-is, then appends a end-of-tables marker.
Done.

OVMF probably would have to parse the blob, split it into tables, then
install them one by one.  But I suspect that will be less code than
dealing with the complex smbios fw_cfg interface we have today ...

cheers,
  Gerd

Re: [Qemu-devel] E820 (Re: [v4 PATCH 00/12] SMBIOS: build full tables in QEMU)

[Gerd Hoffmann]

On Do, 2014-04-03 at 09:32 -0400, Gabriel L. Somlo wrote:
> For that, ideally, I'd like to clone a git repo (and pull once
> every few days to stay on top of things). Looks like the top-level
> upstream one doesn't have your smbios code, so would there be a
> "mid-stream" (for the lack of a better term) git repo you have
> that I can clone and hack against for the smbios stuff ?

http://www.kraxel.org/cgit/jenkins/edk2/tree/

spec file and patches.  specfile is patched by the jenkins build
scripts, so a manual rpm build will need some fiddeling, but I guess you
are more interested in the patches ;)

cheers,
  Gerd

Re: [Qemu-devel] E820 (Re: [v4 PATCH 00/12] SMBIOS: build full tables in QEMU)

[Gerd Hoffmann]

On Do, 2014-04-03 at 11:42 +0200, Laszlo Ersek wrote:
> (a) Gerd's packages:
> 
> http://www.kraxel.org/repos/

> Under (a) you find some short instructions, and a set of RPMs that is
> automatically rebuilt twice a day (IIRC).

It polls the git repos once per hour and kicks a build on new commits.
So usually it should not take more than two hours from commit to updated
packages.

cheers,
  Gerd

Re: [Qemu-devel] E820 (Re: [v4 PATCH 00/12] SMBIOS: build full tables in QEMU)

[Kevin O'Connor]

On Wed, Apr 02, 2014 at 12:35:26AM +0200, Laszlo Ersek wrote:
> On 04/02/14 00:00, Kevin O'Connor wrote:
> > On Tue, Apr 01, 2014 at 11:44:12PM +0200, Laszlo Ersek wrote:
> >> Right now, OVMF can accept individual fields, or table-at-a-time blobs,
> >> via fw_cfg.
> >>
> >> The internal interface (EFI_SMBIOS_PROTOCOL) expects one table at a time
> >> (for which table-at-a-time blobs are a perfect match).
> > 
> > I wasn't aware of this.  The SMBIOS spec calls for all the sub-tables
> > to be concatenanted into a single linear area of memory.  Is there
> > something in EFI or OVMF that is dictating otherwise?  Can you provide
> > a link so I can further understand?  (I briefly checked through the
> > UEFI v2.3.1 spec and nothing popped out at me.)
> 
> The "UEFI Specification" is not relevant here, the "UEFI Platform
> Initialization (PI) Specification" is.
> 
> You can download the PI spec at . The
> relevant section is (I have version 1.2.1):

Oh, those crazy UEFI guys!

The internal edk2 code appears to use these individual calls to
ultimately build the concatenated smbios table from them.  So, nothing
special here - just standard UEFI over-design.

> >> I think that concatenating table-at-a-time blobs in SeaBIOS is easier
> >> than parsing & splitting a complete dump into tables in OVMF.
> > 
> > I don't think it's very difficult either way.  It would be nice,
> > though, if there was just one owner for the smbios.  The current setup
> > where some data comes from QEMU and some from the firmware, along with
> > mechanisms for providing defaults and overrides is way too complex in
> > my opinion.
> 
> I certainly agree with the direction. I'm OK with the current
> table-at-a-time blobs (which should leave only the SMBIOS entry point to
> the firmware). I'm also OK with any new, comprehensive format that
> allows me, with reasonable parsing, to identify the individual tables in
> the big concat (and to throw away the passed down entry point).
> 
> I already wrote display_uuid() [src/fw/smbios.c] for SeaBIOS, so I guess
> I could repeat the exercise if it's unavoidable... :)

Makes sense.

Thanks.
-Kevin

Re: [Qemu-devel] E820 (Re: [v4 PATCH 00/12] SMBIOS: build full tables in QEMU)

[Kevin O'Connor]

On Fri, Apr 04, 2014 at 09:15:14PM -0400, Gabriel L. Somlo wrote:
> On Fri, Apr 04, 2014 at 08:34:11PM -0400, Kevin O'Connor wrote:
> > > 
> > > IMO 'dmidecode -t0' should show what firmware you are running
> > > (seabios/ovmf/coreboot/whatever), not something made up by qemu.
> > 
> > Ultimately my preference would be to make a clean break from the
> > existing smbios fw_cfg system as it is too complex, too confusing, and
> > too inflexible.  We could implement the above as you suggest, but I
> > fear it would require keeping much of the complexity of the current
> > fw_cfg interface.  (It's also a new feature as SeaBIOS doesn't
> > currently put any useful info in type 0.)
> 
> I like this idea too: older/current versions of qemu just do what they
> do, and work with SeaBIOS the way they always have. Future versions
> just send a completely new fw_cfg message type, which, when
> encountered by SeaBIOS, causes it to only add in (or edit) type 0, and
> leave everything else as is.

Right.

> > I don't see anything in the spec that would prohibit it, but I don't
> > think it's done in practice.  Given how many different OS parsers
> > there are and their dubious quality, I think we'd want to be as simple
> > as possible and continue to put table 0 at the start.
> >...
> > The already existing romfile_loader interface seems to provide a
> > nearly complete solution to replace the smbios fw_cfg system.  If
> > there is really demand for more firmware info in the type 0 table, why
> > don't we use romfile_loader, have qemu put together a dummy type 0
> > table, and then have seabios update it in place?  Sure, that's ugly,
> > but I'm pretty sure it would be less ugly then keeping the existing
> > smbios fw_cfg system around.
> 
> The only fly in this ointment may be that type 0 doesn't have a fixed
> length that could be edited in place, if you consider the various
> strings that get tacked on to the end of it. So you'd still have to
> slide the rest of the smbios payload left or right to shrink or
> enlarge the type 0 blob, depending on how you modify the various
> strings it contains...

The dummy type 0 subtable that QEMU generates can have dummy space
padded strings that the firmware can overwrite.  Until recently, the
max size smbios string was 64 bytes, so that size could be used.  (As
above, I admit that this is ugly, but the alternatives also seem
ugly.)  Another option would be to just leave the strings at a QEMU
default as that's no different from what SeaBIOS does today.

-Kevin

Re: [Qemu-devel] E820 (Re: [v4 PATCH 00/12] SMBIOS: build full tables in QEMU)

[Gabriel L. Somlo]

On Fri, Apr 04, 2014 at 08:34:11PM -0400, Kevin O'Connor wrote:
> > 
> > IMO 'dmidecode -t0' should show what firmware you are running
> > (seabios/ovmf/coreboot/whatever), not something made up by qemu.
> 
> Ultimately my preference would be to make a clean break from the
> existing smbios fw_cfg system as it is too complex, too confusing, and
> too inflexible.  We could implement the above as you suggest, but I
> fear it would require keeping much of the complexity of the current
> fw_cfg interface.  (It's also a new feature as SeaBIOS doesn't
> currently put any useful info in type 0.)

I like this idea too: older/current versions of qemu just do what they
do, and work with SeaBIOS the way they always have. Future versions
just send a completely new fw_cfg message type, which, when
encountered by SeaBIOS, causes it to only add in (or edit) type 0, and
leave everything else as is.

> I don't see anything in the spec that would prohibit it, but I don't
> think it's done in practice.  Given how many different OS parsers
> there are and their dubious quality, I think we'd want to be as simple
> as possible and continue to put table 0 at the start.
>...
> The already existing romfile_loader interface seems to provide a
> nearly complete solution to replace the smbios fw_cfg system.  If
> there is really demand for more firmware info in the type 0 table, why
> don't we use romfile_loader, have qemu put together a dummy type 0
> table, and then have seabios update it in place?  Sure, that's ugly,
> but I'm pretty sure it would be less ugly then keeping the existing
> smbios fw_cfg system around.

The only fly in this ointment may be that type 0 doesn't have a fixed
length that could be edited in place, if you consider the various
strings that get tacked on to the end of it. So you'd still have to
slide the rest of the smbios payload left or right to shrink or
enlarge the type 0 blob, depending on how you modify the various
strings it contains...

--Gabriel

Re: [Qemu-devel] E820 (Re: [v4 PATCH 00/12] SMBIOS: build full tables in QEMU)

[Kevin O'Connor]

On Wed, Apr 02, 2014 at 05:04:57PM +0200, Gerd Hoffmann wrote:
> > >   - therefore, the maximum granularity of QEMU-generated
> > > elements should be full tables of a given type, and
> > > not the full SMBIOS blob at once (other mechanisms to
> > > allow the BIOS to insert its own type 0 welcome, but
> > > so far this seems the most straightforward one).
> 
> Just an idea:  Is the table ordering important?  I don't think so.  If
> qemu supplies a blob with all tables except type0, can the firmware
> simply append a type0 record to the blob supplied by qemu?

I don't see anything in the spec that would prohibit it, but I don't
think it's done in practice.  Given how many different OS parsers
there are and their dubious quality, I think we'd want to be as simple
as possible and continue to put table 0 at the start.

> > I don't agree - I think ultimately we want QEMU to generate the full
> > SMBIOS table and pass it to the firmware via the romfile_loader
> > mechanism.
> 
> I still think the firmware (and *only* the firmware) should supply the
> type0 table.  I also think seabios should fill in something more
> sensible in there, such as "Vendor: SeaBIOS" and "Version:
> rel-1.7.4-0-g96917a8-...".
> 
> > The only thing that has been raised as an issue with this
> > is one bit in the smbios table (UEFI support).
> 
> IMO 'dmidecode -t0' should show what firmware you are running
> (seabios/ovmf/coreboot/whatever), not something made up by qemu.

Ultimately my preference would be to make a clean break from the
existing smbios fw_cfg system as it is too complex, too confusing, and
too inflexible.  We could implement the above as you suggest, but I
fear it would require keeping much of the complexity of the current
fw_cfg interface.  (It's also a new feature as SeaBIOS doesn't
currently put any useful info in type 0.)

The already existing romfile_loader interface seems to provide a
nearly complete solution to replace the smbios fw_cfg system.  If
there is really demand for more firmware info in the type 0 table, why
don't we use romfile_loader, have qemu put together a dummy type 0
table, and then have seabios update it in place?  Sure, that's ugly,
but I'm pretty sure it would be less ugly then keeping the existing
smbios fw_cfg system around.

-Kevin

Re: [Qemu-devel] E820 (Re: [v4 PATCH 00/12] SMBIOS: build full tables in QEMU)

[Laszlo Ersek]

On 04/03/14 15:32, Gabriel L. Somlo wrote:
> On Thu, Apr 03, 2014 at 11:42:31AM +0200, Laszlo Ersek wrote:

>> You don't see SMBIOS tables in the guest because you've built upstream
>> OVMF. As I said before, upstream OvmfPkg doesn't include my SMBIOS
>> patches. Both (a) and (b) do however.
> 
> Oh, OK. Basically, I'm interested in looking at the sources
> (specifically, the SMBIOS code :) ) to try and develop my own
> sense of what might be the most agreeable way forward for my
> QEMU smbios patches... That way, I'd actually have a (hopefully
> somewhat intelligent) position of my own to support :) :)
> 
> For that, ideally, I'd like to clone a git repo (and pull once
> every few days to stay on top of things). Looks like the top-level
> upstream one doesn't have your smbios code, so would there be a
> "mid-stream" (for the lack of a better term) git repo you have
> that I can clone and hack against for the smbios stuff ?

https://github.com/lersek/edk2/commits/smbios

(I don't push to this repo normally, so you should continue fetching
from tianocore/edk2.)

I've thrown in two convenience patches for more detailed debug messages.
(See also the debug console hints in the README file.)

> I ask because yesterday was the first time I started really paying
> attention to edk2 and ovmf, and I don't yet have a sense of how fast
> the "state of the universe" would run away from me while my back is
> turned, i.e. how fast a point-in-time snapshot of e.g. Gerd or Paolo's
> RPMs would become stale... Extracting and patching source code from
> RPMs is OK once or twice, but I imagine will be much less fun than
> "git pull" once it gets repetitive :D

Taking the patches from the SRPM and applying them manually would bring
you to the same spot as fetching the patches from the above link (except
the debug patches). Afterwards you should continue fetching form
tianocore/edk2, either merging those commits into your (now new) branch,
or rebasing your (now new) branch.

(In general I don't push my WIP publicly.)

>> One further note (also mentioned in OvmfPkg/README): don't use OVMF.fd
>> with -bios; use it with -pflash (you need a Linux 3.7+ host for this).
>> This will give your guest real runtime variable services -- non-volatile
>> variable data will be written back to the OVMF.fd file.
> 
> Good to know (-bios == read-only, -pflash == writable) :)
> 
> I'm really only in it for testing smbios interactions with qemu though,
> at least at this stage... :)
> 
> Thanks a ton,

No; thank you for doing this.

Laszlo

Re: [Qemu-devel] E820 (Re: [v4 PATCH 00/12] SMBIOS: build full tables in QEMU)

[Gabriel L. Somlo]

On Thu, Apr 03, 2014 at 11:42:31AM +0200, Laszlo Ersek wrote:
> Unless you want to do OVMF development yourself (ie. as long as you'd
> like to test only), you're best off with
> 
> (a) Gerd's packages:
> 
> http://www.kraxel.org/repos/
> 
> (b) If you use a Fedora host, you can also try a (recently refreshed)
> Copr build, thanks to Paolo:
> 
> http://copr-be.cloud.fedoraproject.org/results/bonzini/ovmf/
> 
> Under (a) you find some short instructions, and a set of RPMs that is
> automatically rebuilt twice a day (IIRC).
> 
> Both (a) and (b) include the downstream-only SMBIOS patches.
>
> [...]
>
> You don't see SMBIOS tables in the guest because you've built upstream
> OVMF. As I said before, upstream OvmfPkg doesn't include my SMBIOS
> patches. Both (a) and (b) do however.

Oh, OK. Basically, I'm interested in looking at the sources
(specifically, the SMBIOS code :) ) to try and develop my own
sense of what might be the most agreeable way forward for my
QEMU smbios patches... That way, I'd actually have a (hopefully
somewhat intelligent) position of my own to support :) :)

For that, ideally, I'd like to clone a git repo (and pull once
every few days to stay on top of things). Looks like the top-level
upstream one doesn't have your smbios code, so would there be a
"mid-stream" (for the lack of a better term) git repo you have
that I can clone and hack against for the smbios stuff ?

I ask because yesterday was the first time I started really paying
attention to edk2 and ovmf, and I don't yet have a sense of how fast
the "state of the universe" would run away from me while my back is
turned, i.e. how fast a point-in-time snapshot of e.g. Gerd or Paolo's
RPMs would become stale... Extracting and patching source code from
RPMs is OK once or twice, but I imagine will be much less fun than
"git pull" once it gets repetitive :D

> One further note (also mentioned in OvmfPkg/README): don't use OVMF.fd
> with -bios; use it with -pflash (you need a Linux 3.7+ host for this).
> This will give your guest real runtime variable services -- non-volatile
> variable data will be written back to the OVMF.fd file.

Good to know (-bios == read-only, -pflash == writable) :)

I'm really only in it for testing smbios interactions with qemu though,
at least at this stage... :)

Thanks a ton,
--Gabriel

Re: [Qemu-devel] E820 (Re: [v4 PATCH 00/12] SMBIOS: build full tables in QEMU)

[Laszlo Ersek]

On 04/03/14 03:57, Gabriel L. Somlo wrote:
> On Wed, Apr 02, 2014 at 01:01:28PM -0400, Gabriel L. Somlo wrote:
>> Speaking of, I *thought* I had a vague idea of how all this stuff fits
>> together, but it turns out I don't... There's
>>
>>  - OVMF
>>http://sourceforge.net/apps/mediawiki/tianocore/index.php?title=OVMF
>>
>>  - TianoCore
>>http://www.coreboot.org/TianoCore
>>
>>  - coreboot
>>http://www.coreboot.org/Download_coreboot
>>
>> Apparently, TianoCore is a "coreboot payload", which in my mind is
>> somewhat analogous to bootloader "stages" chaining off each other,
>> but then what's OVMF (the only thing I actually tried, which only
>> works on piix) ? Is it a packaged bundle of coreboot+tianocore ?
>> or something else entirely ?
>>
>> What if I want to send a patch against this whole "thing" to
>> facilitate integration with the new smbios table generator in qemu ?
>>
>> Which git repos do I need to have around, and how to stitch them
>> together to obtain "the thing you use as an argument to -bios in lieu
>> of SeaBIOS", when it comes time to test ? :)

Unless you want to do OVMF development yourself (ie. as long as you'd
like to test only), you're best off with

(a) Gerd's packages:

http://www.kraxel.org/repos/

(b) If you use a Fedora host, you can also try a (recently refreshed)
Copr build, thanks to Paolo:

http://copr-be.cloud.fedoraproject.org/results/bonzini/ovmf/

Under (a) you find some short instructions, and a set of RPMs that is
automatically rebuilt twice a day (IIRC).

Both (a) and (b) include the downstream-only SMBIOS patches.

>> I'm guessing this is a FAQ, so if there's one place that explains it
>> all, please point me at it. Otherwise I'd be happy to write it up once
>> I get my head wrapped around it :)
> 
> Nevermind, it seems it's all under git://git.code.sf.net/p/tianocore/edk2 :)
> 
> Although the nomenclature is still a bit fuzzy to me, the "thing to build"
> within edk2 appears to be OvmfPkg (ACTIVE_PLATFORM = OvmfPkg/OvmfPkgX64.dsc,
> with TOOL_CHAIN_TAG = GCC48, in Conf/target.txt, at least on F20).
> 
> I now have the latest and greatest "upstream" OVMF.fd, and I can use it
> (piix only) to boot Fedora 20 live x86_64. Guess I'm on my way :)

You can most certainly build OVMF yourself, yes; the OvmfPkg/build.sh
script is a convenience wrapper. See also OvmfPkg/README.

> I get "missing smbios entry point" when I do a dmidecode, BTW. QEMU is
> sending type 1, 3, 4, etc. blobs in fw_cfg, not sure yet what I need
> to do to get OVMF to add the entry point... Maybe I should try without
> my smbios-patched qemu ?

You don't see SMBIOS tables in the guest because you've built upstream
OVMF. As I said before, upstream OvmfPkg doesn't include my SMBIOS
patches. Both (a) and (b) do however.

One further note (also mentioned in OvmfPkg/README): don't use OVMF.fd
with -bios; use it with -pflash (you need a Linux 3.7+ host for this).
This will give your guest real runtime variable services -- non-volatile
variable data will be written back to the OVMF.fd file.

Thanks
Laszlo

Re: [Qemu-devel] E820 (Re: [v4 PATCH 00/12] SMBIOS: build full tables in QEMU)

[Gabriel L. Somlo]

On Wed, Apr 02, 2014 at 01:01:28PM -0400, Gabriel L. Somlo wrote:
> Speaking of, I *thought* I had a vague idea of how all this stuff fits
> together, but it turns out I don't... There's
> 
>   - OVMF
> http://sourceforge.net/apps/mediawiki/tianocore/index.php?title=OVMF
> 
>   - TianoCore
> http://www.coreboot.org/TianoCore
> 
>   - coreboot
> http://www.coreboot.org/Download_coreboot
> 
> Apparently, TianoCore is a "coreboot payload", which in my mind is
> somewhat analogous to bootloader "stages" chaining off each other,
> but then what's OVMF (the only thing I actually tried, which only
> works on piix) ? Is it a packaged bundle of coreboot+tianocore ?
> or something else entirely ?
> 
> What if I want to send a patch against this whole "thing" to
> facilitate integration with the new smbios table generator in qemu ?
> 
> Which git repos do I need to have around, and how to stitch them
> together to obtain "the thing you use as an argument to -bios in lieu
> of SeaBIOS", when it comes time to test ? :)
> 
> I'm guessing this is a FAQ, so if there's one place that explains it
> all, please point me at it. Otherwise I'd be happy to write it up once
> I get my head wrapped around it :)

Nevermind, it seems it's all under git://git.code.sf.net/p/tianocore/edk2 :)

Although the nomenclature is still a bit fuzzy to me, the "thing to build"
within edk2 appears to be OvmfPkg (ACTIVE_PLATFORM = OvmfPkg/OvmfPkgX64.dsc,
with TOOL_CHAIN_TAG = GCC48, in Conf/target.txt, at least on F20).

I now have the latest and greatest "upstream" OVMF.fd, and I can use it
(piix only) to boot Fedora 20 live x86_64. Guess I'm on my way :)

I get "missing smbios entry point" when I do a dmidecode, BTW. QEMU is
sending type 1, 3, 4, etc. blobs in fw_cfg, not sure yet what I need
to do to get OVMF to add the entry point... Maybe I should try without
my smbios-patched qemu ?

Thanks,
--Gabriel

Re: [Qemu-devel] E820 (Re: [v4 PATCH 00/12] SMBIOS: build full tables in QEMU)

[Gabriel L. Somlo]

On Wed, Apr 02, 2014 at 05:07:05PM +0200, Gerd Hoffmann wrote:
>   Hi,
> 
> > If qemu gives OVMF a complete, concatenated dump of all tables, I'll
> > have to split that up into individual tables, and install those one by one.
> 
> I feel like I should have a look at how coreboot handles this for an
> additional data point ...

Sounds like maybe I also need to take a step back and try this stuff
out (at least with Linux and Windows), for some first hand experience
with how smbios tables are handled outside SeaBIOS... :)

Speaking of, I *thought* I had a vague idea of how all this stuff fits
together, but it turns out I don't... There's

- OVMF
  http://sourceforge.net/apps/mediawiki/tianocore/index.php?title=OVMF

- TianoCore
  http://www.coreboot.org/TianoCore

- coreboot
  http://www.coreboot.org/Download_coreboot

Apparently, TianoCore is a "coreboot payload", which in my mind is
somewhat analogous to bootloader "stages" chaining off each other,
but then what's OVMF (the only thing I actually tried, which only
works on piix) ? Is it a packaged bundle of coreboot+tianocore ?
or something else entirely ?

What if I want to send a patch against this whole "thing" to
facilitate integration with the new smbios table generator in qemu ?

Which git repos do I need to have around, and how to stitch them
together to obtain "the thing you use as an argument to -bios in lieu
of SeaBIOS", when it comes time to test ? :)

I'm guessing this is a FAQ, so if there's one place that explains it
all, please point me at it. Otherwise I'd be happy to write it up once
I get my head wrapped around it :)

Thanks much,
--Gabriel

PS. Coreboot says it supports q35 as well
(http://www.coreboot.org/Supported_Motherboards)
so if my guess that "coreboot + tianocore == ovmf" is true, then
it must be tianocore that doesn't yet support q35 ?

Re: [Qemu-devel] E820 (Re: [v4 PATCH 00/12] SMBIOS: build full tables in QEMU)

[Gerd Hoffmann]

  Hi,

> If qemu gives OVMF a complete, concatenated dump of all tables, I'll
> have to split that up into individual tables, and install those one by one.

I feel like I should have a look at how coreboot handles this for an
additional data point ...

cheers,
  Gerd

Re: [Qemu-devel] E820 (Re: [v4 PATCH 00/12] SMBIOS: build full tables in QEMU)

[Gerd Hoffmann]

  Hi,

> > From the conversation so far, it seems to me that:
> > 
> > - type 0 is best left to the BIOS (user overrides via
> >   command line at their own risk)

I think it was a bad idea to allow overriding type0 fields in the first
place.  It also isn't used in practice.  I don't think it is a big
problem to loose that capability.

> > - therefore, the maximum granularity of QEMU-generated
> >   elements should be full tables of a given type, and
> >   not the full SMBIOS blob at once (other mechanisms to
> >   allow the BIOS to insert its own type 0 welcome, but
> >   so far this seems the most straightforward one).

Just an idea:  Is the table ordering important?  I don't think so.  If
qemu supplies a blob with all tables except type0, can the firmware
simply append a type0 record to the blob supplied by qemu?

> I don't agree - I think ultimately we want QEMU to generate the full
> SMBIOS table and pass it to the firmware via the romfile_loader
> mechanism.

I still think the firmware (and *only* the firmware) should supply the
type0 table.  I also think seabios should fill in something more
sensible in there, such as "Vendor: SeaBIOS" and "Version:
rel-1.7.4-0-g96917a8-...".

> The only thing that has been raised as an issue with this
> is one bit in the smbios table (UEFI support).

IMO 'dmidecode -t0' should show what firmware you are running
(seabios/ovmf/coreboot/whatever), not something made up by qemu.

cheers,
  Gerd

Re: [Qemu-devel] E820 (Re: [v4 PATCH 00/12] SMBIOS: build full tables in QEMU)

[Laszlo Ersek]

On 04/02/14 14:38, Gabriel L. Somlo wrote:
> On Wed, Apr 02, 2014 at 12:35:26AM +0200, Laszlo Ersek wrote:
>> On 04/02/14 00:00, Kevin O'Connor wrote:
>>> On Tue, Apr 01, 2014 at 11:44:12PM +0200, Laszlo Ersek wrote:
 Right now, OVMF can accept individual fields, or table-at-a-time blobs,
 via fw_cfg.

 The internal interface (EFI_SMBIOS_PROTOCOL) expects one table at a time
 (for which table-at-a-time blobs are a perfect match).
>>>
>>> I wasn't aware of this.  The SMBIOS spec calls for all the sub-tables
>>> to be concatenanted into a single linear area of memory.  Is there
>>> something in EFI or OVMF that is dictating otherwise?  Can you provide
>>> a link so I can further understand?  (I briefly checked through the
>>> UEFI v2.3.1 spec and nothing popped out at me.)
>>
>> The "UEFI Specification" is not relevant here, the "UEFI Platform
>> Initialization (PI) Specification" is.
>>
>> You can download the PI spec at . The
>> relevant section is (I have version 1.2.1):
>>
>> VOLUME 5: Platform Initialization Specification
>> Standards
>> 6 SMBIOS Protocol
>>
>> The function to call is EFI_SMBIOS_PROTOCOL.Add().
>>
 I think that concatenating table-at-a-time blobs in SeaBIOS is easier
 than parsing & splitting a complete dump into tables in OVMF.
>>>
>>> I don't think it's very difficult either way.  It would be nice,
>>> though, if there was just one owner for the smbios.  The current setup
>>> where some data comes from QEMU and some from the firmware, along with
>>> mechanisms for providing defaults and overrides is way too complex in
>>> my opinion.
>>
>> I certainly agree with the direction. I'm OK with the current
>> table-at-a-time blobs (which should leave only the SMBIOS entry point to
>> the firmware). I'm also OK with any new, comprehensive format that
>> allows me, with reasonable parsing, to identify the individual tables in
>> the big concat (and to throw away the passed down entry point).
>>
>> I already wrote display_uuid() [src/fw/smbios.c] for SeaBIOS, so I guess
>> I could repeat the exercise if it's unavoidable... :)
> 
> Kevin, Laszlo,
> 
> What if I found a way to send an entry point structure via fw_cfg, as
> a signal to ${BIOS} to simply assemble all the table-at-a-time blobs,
> but without generating any of its own ?
> 
> I'm still working my way through whether *I* like the idea or not, but
> figured I'd throw it out there as a potential compromise ? :)

If you send the entry point structure in a new fw_cfg file, then (I
believe) I could ignore it easily, and just go ahead with the
table-at-a-time blobs. OVMF has defaults (fallbacks) only for Type 0 and
Type 1 tables now, which you are (almost) guaranteed to send down
anyway, so the above approach might work for OVMF without even changing
the OVMF code. (Regarding individual fields, you'd simply not send such,
if I understand correctly.)

Thanks
Laszlo

Re: [Qemu-devel] E820 (Re: [v4 PATCH 00/12] SMBIOS: build full tables in QEMU)

[Gabriel L. Somlo]

On Wed, Apr 02, 2014 at 12:35:26AM +0200, Laszlo Ersek wrote:
> On 04/02/14 00:00, Kevin O'Connor wrote:
> > On Tue, Apr 01, 2014 at 11:44:12PM +0200, Laszlo Ersek wrote:
> >> Right now, OVMF can accept individual fields, or table-at-a-time blobs,
> >> via fw_cfg.
> >>
> >> The internal interface (EFI_SMBIOS_PROTOCOL) expects one table at a time
> >> (for which table-at-a-time blobs are a perfect match).
> > 
> > I wasn't aware of this.  The SMBIOS spec calls for all the sub-tables
> > to be concatenanted into a single linear area of memory.  Is there
> > something in EFI or OVMF that is dictating otherwise?  Can you provide
> > a link so I can further understand?  (I briefly checked through the
> > UEFI v2.3.1 spec and nothing popped out at me.)
> 
> The "UEFI Specification" is not relevant here, the "UEFI Platform
> Initialization (PI) Specification" is.
> 
> You can download the PI spec at . The
> relevant section is (I have version 1.2.1):
> 
> VOLUME 5: Platform Initialization Specification
> Standards
> 6 SMBIOS Protocol
> 
> The function to call is EFI_SMBIOS_PROTOCOL.Add().
> 
> >> I think that concatenating table-at-a-time blobs in SeaBIOS is easier
> >> than parsing & splitting a complete dump into tables in OVMF.
> > 
> > I don't think it's very difficult either way.  It would be nice,
> > though, if there was just one owner for the smbios.  The current setup
> > where some data comes from QEMU and some from the firmware, along with
> > mechanisms for providing defaults and overrides is way too complex in
> > my opinion.
> 
> I certainly agree with the direction. I'm OK with the current
> table-at-a-time blobs (which should leave only the SMBIOS entry point to
> the firmware). I'm also OK with any new, comprehensive format that
> allows me, with reasonable parsing, to identify the individual tables in
> the big concat (and to throw away the passed down entry point).
> 
> I already wrote display_uuid() [src/fw/smbios.c] for SeaBIOS, so I guess
> I could repeat the exercise if it's unavoidable... :)

Kevin, Laszlo,

What if I found a way to send an entry point structure via fw_cfg, as
a signal to ${BIOS} to simply assemble all the table-at-a-time blobs,
but without generating any of its own ?

I'm still working my way through whether *I* like the idea or not, but
figured I'd throw it out there as a potential compromise ? :)

Thx,
--Gabriel

Re: [Qemu-devel] E820 (Re: [v4 PATCH 00/12] SMBIOS: build full tables in QEMU)

[Laszlo Ersek]

On 04/02/14 00:00, Kevin O'Connor wrote:
> On Tue, Apr 01, 2014 at 11:44:12PM +0200, Laszlo Ersek wrote:
>> Right now, OVMF can accept individual fields, or table-at-a-time blobs,
>> via fw_cfg.
>>
>> The internal interface (EFI_SMBIOS_PROTOCOL) expects one table at a time
>> (for which table-at-a-time blobs are a perfect match).
> 
> I wasn't aware of this.  The SMBIOS spec calls for all the sub-tables
> to be concatenanted into a single linear area of memory.  Is there
> something in EFI or OVMF that is dictating otherwise?  Can you provide
> a link so I can further understand?  (I briefly checked through the
> UEFI v2.3.1 spec and nothing popped out at me.)

The "UEFI Specification" is not relevant here, the "UEFI Platform
Initialization (PI) Specification" is.

You can download the PI spec at . The
relevant section is (I have version 1.2.1):

VOLUME 5: Platform Initialization Specification
Standards
6 SMBIOS Protocol

The function to call is EFI_SMBIOS_PROTOCOL.Add().

>> I think that concatenating table-at-a-time blobs in SeaBIOS is easier
>> than parsing & splitting a complete dump into tables in OVMF.
> 
> I don't think it's very difficult either way.  It would be nice,
> though, if there was just one owner for the smbios.  The current setup
> where some data comes from QEMU and some from the firmware, along with
> mechanisms for providing defaults and overrides is way too complex in
> my opinion.

I certainly agree with the direction. I'm OK with the current
table-at-a-time blobs (which should leave only the SMBIOS entry point to
the firmware). I'm also OK with any new, comprehensive format that
allows me, with reasonable parsing, to identify the individual tables in
the big concat (and to throw away the passed down entry point).

I already wrote display_uuid() [src/fw/smbios.c] for SeaBIOS, so I guess
I could repeat the exercise if it's unavoidable... :)

Thanks
Laszlo

Re: [Qemu-devel] E820 (Re: [v4 PATCH 00/12] SMBIOS: build full tables in QEMU)

[Kevin O'Connor]

On Tue, Apr 01, 2014 at 11:44:12PM +0200, Laszlo Ersek wrote:
> Right now, OVMF can accept individual fields, or table-at-a-time blobs,
> via fw_cfg.
> 
> The internal interface (EFI_SMBIOS_PROTOCOL) expects one table at a time
> (for which table-at-a-time blobs are a perfect match).

I wasn't aware of this.  The SMBIOS spec calls for all the sub-tables
to be concatenanted into a single linear area of memory.  Is there
something in EFI or OVMF that is dictating otherwise?  Can you provide
a link so I can further understand?  (I briefly checked through the
UEFI v2.3.1 spec and nothing popped out at me.)

> I think that concatenating table-at-a-time blobs in SeaBIOS is easier
> than parsing & splitting a complete dump into tables in OVMF.

I don't think it's very difficult either way.  It would be nice,
though, if there was just one owner for the smbios.  The current setup
where some data comes from QEMU and some from the firmware, along with
mechanisms for providing defaults and overrides is way too complex in
my opinion.

Thanks,
-Kevin

Re: [Qemu-devel] E820 (Re: [v4 PATCH 00/12] SMBIOS: build full tables in QEMU)

[Kevin O'Connor]

On Tue, Apr 01, 2014 at 05:28:10PM -0400, Gabriel L. Somlo wrote:
> Assuming all relevant QEMU maintainers are OK with the idea of
> creating a full SMBIOS blob (with e.g. type 0 defaulting to the
> relevant SeaBIOS values, override-able to fit some different bios,
> e.g. OVMF), would you take a patch to check for this blob in
> smbios_setup() (in SeaBIOS src/fw/smbios.c) ? Right now, it's either
> individual fields or table-at-a-time blobs only, AFAICT.

Yes.

-Kevin

Re: [Qemu-devel] E820 (Re: [v4 PATCH 00/12] SMBIOS: build full tables in QEMU)

[Laszlo Ersek]

On 04/01/14 23:28, Gabriel L. Somlo wrote:
> On Tue, Apr 01, 2014 at 04:28:32PM -0400, Kevin O'Connor wrote:
>>> From the conversation so far, it seems to me that:
>>>
>>> - type 0 is best left to the BIOS (user overrides via
>>>   command line at their own risk)
>>>
>>> - therefore, the maximum granularity of QEMU-generated
>>>   elements should be full tables of a given type, and
>>>   not the full SMBIOS blob at once (other mechanisms to
>>>   allow the BIOS to insert its own type 0 welcome, but
>>>   so far this seems the most straightforward one).
>>
>> I don't agree - I think ultimately we want QEMU to generate the full
>> SMBIOS table and pass it to the firmware via the romfile_loader
>> mechanism.  The only thing that has been raised as an issue with this
>> is one bit in the smbios table (UEFI support).  For this one bit, I
>> think QEMU can just put together a sane default and the firmware can
>> patch up the one bit (either manually or via a new romfile_loader
>> command).
>>
>>>
>>> - this means the smbios structure header has to be left
>>>   up to the BIOS
>>>
>>> - the BIOS is then responsible for setting the smbios
>>>   spec version (2.4 for SeaBIOS, 2.7.1 for OVMF).
>>>
>>> On that last point, at least Linux seems to be OK with individual
>>> type tables having a higher version than the structure header; i.e.,
>>> dmidecode works fine when e.g. the structure header says 2.4 but
>>> the type 4 cpu record is 2.6. I'll test on Windows and OS X as well,
>>> and post my results here.
>>>
>>> My one remaining question is: how do we get the BIOS to *not* generate
>>> a certain table type that's being left out on purpose by QEMU ?
>>>
>>> I'm talking here of type 20, which is no longer required as of spec
>>> v2.5, and which would unnecessarily complicate things if/when more
>>> than two E820_RAM memory areas are present...
>>
>> The above are good examples why I think QEMU should be the sole owner
>> of the SMBIOS.
> 
> Assuming all relevant QEMU maintainers are OK with the idea of
> creating a full SMBIOS blob (with e.g. type 0 defaulting to the
> relevant SeaBIOS values, override-able to fit some different bios,
> e.g. OVMF), would you take a patch to check for this blob in
> smbios_setup() (in SeaBIOS src/fw/smbios.c) ? Right now, it's either
> individual fields or table-at-a-time blobs only, AFAICT.
> 
> Assuming "yes", would OVMF accept a similar patch (unless it's already
> set up to receive such a blob, I forget whether that came up earlier
> in the thread) ?

Right now, OVMF can accept individual fields, or table-at-a-time blobs,
via fw_cfg.

The internal interface (EFI_SMBIOS_PROTOCOL) expects one table at a time
(for which table-at-a-time blobs are a perfect match).

If qemu gives OVMF a complete, concatenated dump of all tables, I'll
have to split that up into individual tables, and install those one by one.

qemu --[fw_cfg]--> OVMF platform code --[EFI_SMBIOS_PROTOCOL]--> edk2
 "some" format:   strictly per-table
 - field patch
 - per-table blob
 - complete dump?

I think that concatenating table-at-a-time blobs in SeaBIOS is easier
than parsing & splitting a complete dump into tables in OVMF.

Kevin might disagree of course :)

Thanks
Laszlo

Re: [Qemu-devel] E820 (Re: [v4 PATCH 00/12] SMBIOS: build full tables in QEMU)

[Gabriel L. Somlo]

On Tue, Apr 01, 2014 at 04:28:32PM -0400, Kevin O'Connor wrote:
> > From the conversation so far, it seems to me that:
> > 
> > - type 0 is best left to the BIOS (user overrides via
> >   command line at their own risk)
> > 
> > - therefore, the maximum granularity of QEMU-generated
> >   elements should be full tables of a given type, and
> >   not the full SMBIOS blob at once (other mechanisms to
> >   allow the BIOS to insert its own type 0 welcome, but
> >   so far this seems the most straightforward one).
> 
> I don't agree - I think ultimately we want QEMU to generate the full
> SMBIOS table and pass it to the firmware via the romfile_loader
> mechanism.  The only thing that has been raised as an issue with this
> is one bit in the smbios table (UEFI support).  For this one bit, I
> think QEMU can just put together a sane default and the firmware can
> patch up the one bit (either manually or via a new romfile_loader
> command).
> 
> > 
> > - this means the smbios structure header has to be left
> >   up to the BIOS
> > 
> > - the BIOS is then responsible for setting the smbios
> >   spec version (2.4 for SeaBIOS, 2.7.1 for OVMF).
> > 
> > On that last point, at least Linux seems to be OK with individual
> > type tables having a higher version than the structure header; i.e.,
> > dmidecode works fine when e.g. the structure header says 2.4 but
> > the type 4 cpu record is 2.6. I'll test on Windows and OS X as well,
> > and post my results here.
> > 
> > My one remaining question is: how do we get the BIOS to *not* generate
> > a certain table type that's being left out on purpose by QEMU ?
> > 
> > I'm talking here of type 20, which is no longer required as of spec
> > v2.5, and which would unnecessarily complicate things if/when more
> > than two E820_RAM memory areas are present...
> 
> The above are good examples why I think QEMU should be the sole owner
> of the SMBIOS.

Assuming all relevant QEMU maintainers are OK with the idea of
creating a full SMBIOS blob (with e.g. type 0 defaulting to the
relevant SeaBIOS values, override-able to fit some different bios,
e.g. OVMF), would you take a patch to check for this blob in
smbios_setup() (in SeaBIOS src/fw/smbios.c) ? Right now, it's either
individual fields or table-at-a-time blobs only, AFAICT.

Assuming "yes", would OVMF accept a similar patch (unless it's already
set up to receive such a blob, I forget whether that came up earlier
in the thread) ?

Thanks,
  Gabriel

--
In Soviet Russia, problem divide-and-conquer YOU ;)

Re: [Qemu-devel] E820 (Re: [v4 PATCH 00/12] SMBIOS: build full tables in QEMU)

[Kevin O'Connor]

On Tue, Apr 01, 2014 at 02:47:27PM -0400, Gabriel L. Somlo wrote:
> On Tue, Apr 01, 2014 at 05:47:09PM +0200, Laszlo Ersek wrote:
> > bit 2 of the BIOS Characteristics Extension Byte 2 (7.1.2.2) is set, for
> > "Enable Targeted Content Distribution".
> > 
> > In OVMF, the same byte has the following bits set:
> > 
> > Bit 3 -- UEFI Specification is supported.
> > Bit 4 -- The SMBIOS table describes a virtual machine. (If this bit is
> >  not set, no inference can be made about the virtuality of the
> >  system.)
> > 
> > I have nothing against bit 2 (I didn't include it because I had no clue
> > what it meant, but we can certainly set that bit down the road). Bit 3
> > would be wrong for SeaBIOS, and it would be wrong to leave clear for
> > OVMF. Bit 4 would be wrong for SeaBIOS (as a static default), but is
> > correct (and very nice, although not necessary) for OVMF.
> 
> I can add an extra command line option for type 0 defaults (e.g.
> "char_ext" but we can pick a better name):
> 
> "-smbios type=0,vendor='foo',version='x.y',char_ext=4"
> 
> ... and make the user responsible for supplying the correct value
> if/when they wish to override the defaults. I'll do that in the
> v5 patch set I'm working on right now :)
> 
> As an aside, I think in the end it doesn't matter much if we supply
> individual field defaults or full tables for *optional* types such
> as type 0. I just like to generate full tables across the board to
> keep reusing the same code, rather than leave the individual-field
> logic in just for this one table type...
> 
> From the conversation so far, it seems to me that:
> 
>   - type 0 is best left to the BIOS (user overrides via
> command line at their own risk)
> 
>   - therefore, the maximum granularity of QEMU-generated
> elements should be full tables of a given type, and
> not the full SMBIOS blob at once (other mechanisms to
> allow the BIOS to insert its own type 0 welcome, but
> so far this seems the most straightforward one).

I don't agree - I think ultimately we want QEMU to generate the full
SMBIOS table and pass it to the firmware via the romfile_loader
mechanism.  The only thing that has been raised as an issue with this
is one bit in the smbios table (UEFI support).  For this one bit, I
think QEMU can just put together a sane default and the firmware can
patch up the one bit (either manually or via a new romfile_loader
command).

> 
>   - this means the smbios structure header has to be left
> up to the BIOS
> 
>   - the BIOS is then responsible for setting the smbios
> spec version (2.4 for SeaBIOS, 2.7.1 for OVMF).
> 
> On that last point, at least Linux seems to be OK with individual
> type tables having a higher version than the structure header; i.e.,
> dmidecode works fine when e.g. the structure header says 2.4 but
> the type 4 cpu record is 2.6. I'll test on Windows and OS X as well,
> and post my results here.
> 
> My one remaining question is: how do we get the BIOS to *not* generate
> a certain table type that's being left out on purpose by QEMU ?
> 
> I'm talking here of type 20, which is no longer required as of spec
> v2.5, and which would unnecessarily complicate things if/when more
> than two E820_RAM memory areas are present...

The above are good examples why I think QEMU should be the sole owner
of the SMBIOS.

-Kevin

Re: [Qemu-devel] E820 (Re: [v4 PATCH 00/12] SMBIOS: build full tables in QEMU)

[Gabriel L. Somlo]

On Tue, Apr 01, 2014 at 05:47:09PM +0200, Laszlo Ersek wrote:
> On 04/01/14 16:39, Kevin O'Connor wrote:
> > On Tue, Apr 01, 2014 at 10:40:00AM +0200, Laszlo Ersek wrote:
> >> On 03/31/14 22:18, Gabriel L. Somlo wrote:
> >>> The only sticking point remaining would be who gets to generate the
> >>> Type 0 (BIOS Information) table and when, which is something QEMU
> >>> should arguably NOT be doing on behalf of SeaBIOS (or OVMF, or ...).
> >>>
> >>
> >> Sorry I can follow this discussion only intermittently.
> >>
> >> - I think OVMF would be fine with the Type 0 table as-is (it has a
> >> default table and adheres to field-level patches). Full tables for other
> >> types would be welcome.
> > 
> > When SeaBIOS generates the smbios table, it creates a hardcoded type 0
> > sub-table.  (See SeaBIOS code src/fw/smbios.c:smbios_init_type_0.)  If
> > OVMF is okay with the same hardcodes, then I'd suggest QEMU create the
> > table the same way SeaBIOS currently does.

OK, smbios_init_type_0() is called by the add_struct() macro in
smbios_setup(), but not if fw_cfg already holds a QEMU-supplied
type 0 table.

When it is called, smbios_init_type_0() will look for individual
fields supplied via fw_cfg, and only use its own defaults otherwise.

By default, QEMU will not set any type 0 fields, nor generate
a type 0 table, which means that each specific BIOS gets to
supply its own default type 0 table.

My patch set supplies various full tables via fw_cfg, but by default
it still doesn't built a type 0. That only happens if either a binary
table or default fields for type 0 are provided on the command line
(same as the current behavior, except I'm always sending a full table
rather than field-by-field).

> bit 2 of the BIOS Characteristics Extension Byte 2 (7.1.2.2) is set, for
> "Enable Targeted Content Distribution".
> 
> In OVMF, the same byte has the following bits set:
> 
> Bit 3 -- UEFI Specification is supported.
> Bit 4 -- The SMBIOS table describes a virtual machine. (If this bit is
>  not set, no inference can be made about the virtuality of the
>  system.)
> 
> I have nothing against bit 2 (I didn't include it because I had no clue
> what it meant, but we can certainly set that bit down the road). Bit 3
> would be wrong for SeaBIOS, and it would be wrong to leave clear for
> OVMF. Bit 4 would be wrong for SeaBIOS (as a static default), but is
> correct (and very nice, although not necessary) for OVMF.

I can add an extra command line option for type 0 defaults (e.g.
"char_ext" but we can pick a better name):

"-smbios type=0,vendor='foo',version='x.y',char_ext=4"

... and make the user responsible for supplying the correct value
if/when they wish to override the defaults. I'll do that in the
v5 patch set I'm working on right now :)

As an aside, I think in the end it doesn't matter much if we supply
individual field defaults or full tables for *optional* types such
as type 0. I just like to generate full tables across the board to
keep reusing the same code, rather than leave the individual-field
logic in just for this one table type...

>From the conversation so far, it seems to me that:

- type 0 is best left to the BIOS (user overrides via
  command line at their own risk)

- therefore, the maximum granularity of QEMU-generated
  elements should be full tables of a given type, and
  not the full SMBIOS blob at once (other mechanisms to
  allow the BIOS to insert its own type 0 welcome, but
  so far this seems the most straightforward one).

- this means the smbios structure header has to be left
  up to the BIOS

- the BIOS is then responsible for setting the smbios
  spec version (2.4 for SeaBIOS, 2.7.1 for OVMF).

On that last point, at least Linux seems to be OK with individual
type tables having a higher version than the structure header; i.e.,
dmidecode works fine when e.g. the structure header says 2.4 but
the type 4 cpu record is 2.6. I'll test on Windows and OS X as well,
and post my results here.

My one remaining question is: how do we get the BIOS to *not* generate
a certain table type that's being left out on purpose by QEMU ?

I'm talking here of type 20, which is no longer required as of spec
v2.5, and which would unnecessarily complicate things if/when more
than two E820_RAM memory areas are present...

Thanks much,
--Gabriel 

Re: [Qemu-devel] E820 (Re: [v4 PATCH 00/12] SMBIOS: build full tables in QEMU)

[Laszlo Ersek]

On 04/01/14 16:39, Kevin O'Connor wrote:
> On Tue, Apr 01, 2014 at 10:40:00AM +0200, Laszlo Ersek wrote:
>> On 03/31/14 22:18, Gabriel L. Somlo wrote:
>>> The only sticking point remaining would be who gets to generate the
>>> Type 0 (BIOS Information) table and when, which is something QEMU
>>> should arguably NOT be doing on behalf of SeaBIOS (or OVMF, or ...).
>>>
>>> I guess everyone's busy with QEMU 2.0, so I'll keep playing with this
>>> stuff on my own and bring it up again afterwards.
>>>
>>> Obviously, more comments and feedback are welcome at any time, should
>>> there be any interest :)
>>
>> Sorry I can follow this discussion only intermittently.
>>
>> - I think OVMF would be fine with the Type 0 table as-is (it has a
>> default table and adheres to field-level patches). Full tables for other
>> types would be welcome.
> 
> When SeaBIOS generates the smbios table, it creates a hardcoded type 0
> sub-table.  (See SeaBIOS code src/fw/smbios.c:smbios_init_type_0.)  If
> OVMF is okay with the same hardcodes, then I'd suggest QEMU create the
> table the same way SeaBIOS currently does.

In SeaBIOS,

if (!get_field(0, offsetof(struct smbios_type_0,
   bios_characteristics_extension_bytes),
   &p->bios_characteristics_extension_bytes)) {
p->bios_characteristics_extension_bytes[0] = 0;
/* Enable targeted content distribution. Needed for SVVP */
p->bios_characteristics_extension_bytes[1] = 4;
}

bit 2 of the BIOS Characteristics Extension Byte 2 (7.1.2.2) is set, for
"Enable Targeted Content Distribution".

In OVMF, the same byte has the following bits set:

Bit 3 -- UEFI Specification is supported.
Bit 4 -- The SMBIOS table describes a virtual machine. (If this bit is
 not set, no inference can be made about the virtuality of the
 system.)

I have nothing against bit 2 (I didn't include it because I had no clue
what it meant, but we can certainly set that bit down the road). Bit 3
would be wrong for SeaBIOS, and it would be wrong to leave clear for
OVMF. Bit 4 would be wrong for SeaBIOS (as a static default), but is
correct (and very nice, although not necessary) for OVMF.

Gerd posted a textual comparison before (with dmidecode):

  http://thread.gmane.org/gmane.comp.emulators.qemu/260804/focus=261248

But I'm including a textual diff too (RHEL-7 host, RHEL-7 guests):

> --- seabios-rhel7.txt 2014-04-01 17:39:49.148601405 +0200
> +++ ovmf-rhel7.txt2014-04-01 17:40:01.075658204 +0200
> @@ -1,128 +1,34 @@
>  # dmidecode 2.12
> -SMBIOS 2.4 present.
> -11 structures occupying 369 bytes.
> -Table at 0x000FDD80.
> +# SMBIOS entry point at 0x3fed8000
> +SMBIOS 2.7 present.
> +3 structures occupying 185 bytes.
> +Table at 0x3FED7000.
>
>  Handle 0x, DMI type 0, 24 bytes
>  BIOS Information
> - Vendor: Bochs
> - Version: Bochs
> - Release Date: 01/01/2011
> + Vendor: EFI Development Kit II / OVMF
> + Version: 0.1
> + Release Date: 06/03/2013
>   Address: 0xE8000
>   Runtime Size: 96 kB
>   ROM Size: 64 kB
>   Characteristics:
>   BIOS characteristics not supported
> - Targeted content distribution is supported
> - BIOS Revision: 1.0
> + UEFI is supported
> + System is a virtual machine
> + BIOS Revision: 0.1

Of course, this is not to say that SeaBIOS and OVMF should continue
patching the Type 0 table field-wise. If we can control the Table 0
fields on this level of detail on the qemu command line, and the parent
of qemu (libvirtd or the user's shell) takes care to set them in
accordance with the guest firmware, then the full blob suffices for
Table 0 too.

(I'm retaining the rest of the diff below.)

Thanks
Laszlo

>
> -Handle 0x0100, DMI type 1, 27 bytes
> +Handle 0x0001, DMI type 1, 27 bytes
>  System Information
>   Manufacturer: Red Hat
>   Product Name: KVM
>   Version: RHEL 7.0.0 PC (i440FX + PIIX, 1996)
> - Serial Number: Not Specified
> - UUID: C0F384E5-6AEA-46E9-8340-E5AF2F0DCC9B
> + Serial Number: n/a
> + UUID: 8A2A6D47-99A6-486D-AFAE-A53741464C37
>   Wake-up Type: Power Switch
> - SKU Number: Not Specified
> + SKU Number: n/a
>   Family: Red Hat Enterprise Linux
>
> -Handle 0x0300, DMI type 3, 20 bytes
> -Chassis Information
> - Manufacturer: Bochs
> - Type: Other
> - Lock: Not Present
> - Version: Not Specified
> - Serial Number: Not Specified
> - Asset Tag: Not Specified
> - Boot-up State: Safe
> - Power Supply State: Safe
> - Thermal State: Safe
> - Security Status: Unknown
> - OEM Information: 0x
> - Height: Unspecified
> - Number Of Power Cords: Unspecified
> -
> -Handle 0x0401, DMI type 4, 32 bytes
> -Processor Information
> - Socket Designation: CPU 1
> - Type: Central Processor
> - Family: Other
> - Manufacturer: Bochs
> - ID: 63 06 00 00 FD FB 8B 07
> - Version: Not Speci

Re: [Qemu-devel] E820 (Re: [v4 PATCH 00/12] SMBIOS: build full tables in QEMU)

[Kevin O'Connor]

On Tue, Apr 01, 2014 at 10:40:00AM +0200, Laszlo Ersek wrote:
> On 03/31/14 22:18, Gabriel L. Somlo wrote:
> > The only sticking point remaining would be who gets to generate the
> > Type 0 (BIOS Information) table and when, which is something QEMU
> > should arguably NOT be doing on behalf of SeaBIOS (or OVMF, or ...).
> > 
> > I guess everyone's busy with QEMU 2.0, so I'll keep playing with this
> > stuff on my own and bring it up again afterwards.
> > 
> > Obviously, more comments and feedback are welcome at any time, should
> > there be any interest :)
> 
> Sorry I can follow this discussion only intermittently.
> 
> - I think OVMF would be fine with the Type 0 table as-is (it has a
> default table and adheres to field-level patches). Full tables for other
> types would be welcome.

When SeaBIOS generates the smbios table, it creates a hardcoded type 0
sub-table.  (See SeaBIOS code src/fw/smbios.c:smbios_init_type_0.)  If
OVMF is okay with the same hardcodes, then I'd suggest QEMU create the
table the same way SeaBIOS currently does.

-Kevin

Re: [Qemu-devel] E820 (Re: [v4 PATCH 00/12] SMBIOS: build full tables in QEMU)

[Laszlo Ersek]

On 03/31/14 22:18, Gabriel L. Somlo wrote:
> On Wed, Mar 26, 2014 at 06:36:10PM -0400, Kevin O'Connor wrote:
>> On Wed, Mar 26, 2014 at 03:58:50PM -0400, Gabriel L. Somlo wrote:
>>> - SeaBIOS is still in charge of providing the smbios_entry_point
>>>   structure, and it's unlikely we can reasonably expect it to
>>>   bump the version to 2.5 (not that it seems to matter, if my
>>>   tests are to be believed)
>>
>> This is why ultimately we want to use the romfile_loader mechanism for
>> smbios - that interface will allow qemu to generate both the smbios
>> table and the smbios entry point.
> 
> Ah, so romfile_loader is the "whole-blob" fw_cfg transfer method (so
> far in smbios.c we've been dealing with individual fields, and
> individual table types).
> 
> The only sticking point remaining would be who gets to generate the
> Type 0 (BIOS Information) table and when, which is something QEMU
> should arguably NOT be doing on behalf of SeaBIOS (or OVMF, or ...).
> 
> I guess everyone's busy with QEMU 2.0, so I'll keep playing with this
> stuff on my own and bring it up again afterwards.
> 
> Obviously, more comments and feedback are welcome at any time, should
> there be any interest :)

Sorry I can follow this discussion only intermittently.

- I think OVMF would be fine with the Type 0 table as-is (it has a
default table and adheres to field-level patches). Full tables for other
types would be welcome.

- In edk2 (and on platforms that conform to the platform init (PI)
spec), platform drivers install their smbios tables through
EFI_SMBIOS_PROTOCOL. (Documented in Vol5 of the PI spec.)

In edk2, the "central" driver that produces this protocol (== provides
the service) is "MdeModulePkg/Universal/SmbiosDxe".

The OVMF platform driver that installs the smbios tables through the
protocol (== consumes the service) is "OvmfPkg/SmbiosPlatformDxe".

The set and the contents of the smbios tables are the jurisdiction of
the platform driver (the service consumer) -- the driver in OvmfPkg.

The smbios entry point (including the smbios std version) is the
jurisdiction of the service producer -- the driver in MdeModulePkg.

The platform driver can query the smbios std version from
EFI_SMBIOS_PROTOCOL. The platform driver should also filter the tables
it intends to install through the protocol so that the table types
conform to the smbios std version supported by the service provider.

This is to say, OVMF can fetch tables via fw_cfg, and install them via
the EFI_SMBIOS_PROTOCOL instance that "MdeModulePkg/Universal/SmbiosDxe"
provides. However, OVMF can't change the SMBIOS version or other aspects
of the smbios entry point. At best OVMF could filter out tables (grabbed
from fw_cfg) that are above the std version that EFI_SMBIOS_PROTOCOL
supports/advertises.

Currently OVMF does no such filtering (it would require hard-coding
which table type is defined in which smbios version). It does require
support for version 2.3, or it gives up.

In any case, the SMBIOS version currently supported by
"MdeModulePkg/Universal/SmbiosDxe" is 2.7.1 (SVN r11690), which is quite
permissive.

Summary:
- type 0 should be OK as-is, other types are most welcome as full blobs,
- OvmfPkg can't influence the smbios version advertised, it belongs to
qanother (central) driver in edk2 that is built into OVMF.fd,
- OvmfPkg currently doesn't filter fw_cfg table types against the smbios
version (determined by that other, central driver in edk2),
- however that version is reasonably high (2.7.1).

(Note: most of "OvmfPkg/SmbiosPlatformDxe" that's relevant for the above
is not in upstream edk2. You can find the patches in
 and
.)

Thanks
Laszlo

Re: [Qemu-devel] E820 (Re: [v4 PATCH 00/12] SMBIOS: build full tables in QEMU)

[Gabriel L. Somlo]

On Wed, Mar 26, 2014 at 06:36:10PM -0400, Kevin O'Connor wrote:
> On Wed, Mar 26, 2014 at 03:58:50PM -0400, Gabriel L. Somlo wrote:
> > - SeaBIOS is still in charge of providing the smbios_entry_point
> >   structure, and it's unlikely we can reasonably expect it to
> >   bump the version to 2.5 (not that it seems to matter, if my
> >   tests are to be believed)
> 
> This is why ultimately we want to use the romfile_loader mechanism for
> smbios - that interface will allow qemu to generate both the smbios
> table and the smbios entry point.

Ah, so romfile_loader is the "whole-blob" fw_cfg transfer method (so
far in smbios.c we've been dealing with individual fields, and
individual table types).

The only sticking point remaining would be who gets to generate the
Type 0 (BIOS Information) table and when, which is something QEMU
should arguably NOT be doing on behalf of SeaBIOS (or OVMF, or ...).

I guess everyone's busy with QEMU 2.0, so I'll keep playing with this
stuff on my own and bring it up again afterwards.

Obviously, more comments and feedback are welcome at any time, should
there be any interest :)

Thanks,
--Gabriel

Re: [Qemu-devel] E820 (Re: [v4 PATCH 00/12] SMBIOS: build full tables in QEMU)

[Gabriel L. Somlo]

On Wed, Mar 26, 2014 at 03:58:50PM -0400, Gabriel L. Somlo wrote:
> On Tue, Mar 18, 2014 at 07:23:17PM -0400, Gabriel L. Somlo wrote:
> > At this point, can anyone with access to a real, physical, NUMA
> > system dump the smbios tables with dmidecode and post them here?
> > I think that would be very informative.
> 
> So I thrashed around a bit trying to find a real NUMA box,
> and found a Dell R410 whose BIOS claims to support NUMA by
> disabling the "Node Interleaving" option

So, flipping that option on and off does indeed switch the system
between NUMA and UMA modes. Running "numactl -H" in UMA mode gets us
this output:

available: 1 nodes (0)
node 0 cpus: 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21
22 23
node 0 size: 131059 MB
node 0 free: 127898 MB
node distances:
node   0 
  0:  10 

In NUMA mode, we get this:

available: 2 nodes (0-1)
node 0 cpus: 0 2 4 6 8 10 12 14 16 18 20 22
node 0 size: 65536 MB
node 0 free: 64047 MB
node 1 cpus: 1 3 5 7 9 11 13 15 17 19 21 23
node 1 size: 65523 MB
node 1 free: 63841 MB
node distances:
node   0   1 
  0:  10  20 
  1:  20  10 

The "dmidecode" output stays unchanged between UMA and NUMA
modes (showing only memory tables, types 16-19, and there is NO
type 20):

Handle 0x1000, DMI type 16, 15 bytes
Physical Memory Array
Location: System Board Or Motherboard
Use: System Memory
Error Correction Type: Multi-bit ECC
Maximum Capacity: 128 GB
Error Information Handle: Not Provided
Number Of Devices: 8

Handle 0x1100, DMI type 17, 28 bytes
Memory Device
Array Handle: 0x1000
Error Information Handle: Not Provided
Total Width: 72 bits
Data Width: 64 bits
Size: 16384 MB
Form Factor: DIMM
Set: 1
Locator: DIMM_A1 
Bank Locator: Not Specified
Type: DDR3
Type Detail: Synchronous Registered (Buffered)
Speed: 1066 MHz
Manufacturer: 00CE00B380CE
Serial Number: 44056D5E
Asset Tag: 01105061
Part Number: M393B2K70CM0-YF8  
Rank: 4

Handle 0x1101, DMI type 17, 28 bytes
Handle 0x1102, DMI type 17, 28 bytes
Handle 0x1103, DMI type 17, 28 bytes
Handle 0x1109, DMI type 17, 28 bytes
Handle 0x110A, DMI type 17, 28 bytes
Handle 0x110B, DMI type 17, 28 bytes
Handle 0x110C, DMI type 17, 28 bytes
/* all of them similar to 0x1100 above [GLS] */

Handle 0x1300, DMI type 19, 15 bytes
Memory Array Mapped Address
Starting Address: 0x000
Ending Address: 0x000BFFF
Range Size: 3 GB
Physical Array Handle: 0x1000
Partition Width: 2

Handle 0x1301, DMI type 19, 15 bytes
Memory Array Mapped Address
Starting Address: 0x001
Ending Address: 0x0203FFF
Range Size: 125 GB
Physical Array Handle: 0x1000
Partition Width: 2

The output of "dmesg | grep -i e820" remains unchanged between UMA and
NUMA modes:

e820: BIOS-provided physical RAM map:
BIOS-e820: [mem 0x-0x0009] usable
BIOS-e820: [mem 0x0010-0xbf378fff] usable
BIOS-e820: [mem 0xbf379000-0xbf38efff] reserved
BIOS-e820: [mem 0xbf38f000-0xbf3cdfff] ACPI data
BIOS-e820: [mem 0xbf3ce000-0xbfff] reserved
BIOS-e820: [mem 0xe000-0xefff] reserved
BIOS-e820: [mem 0xfe00-0x] reserved
BIOS-e820: [mem 0x0001-0x00203fff] usable
e820: update [mem 0x-0x] usable ==> reserved
e820: remove [mem 0x000a-0x000f] usable
e820: last_pfn = 0x204 max_arch_pfn = 0x4
e820: update [mem 0xc000-0x] usable ==> reserved
e820: last_pfn = 0xbf379 max_arch_pfn = 0x4
e820: [mem 0xc000-0xdfff] available for PCI devices
PCI: MMCONFIG at [mem 0xe000-0xefff] reserved in E820
e820: reserve RAM buffer [mem 0xbf379000-0xbfff]

So, even in NUMA mode, there still appear to be only two contiguous
E820_RAM type entries in the "sanitized" e820 table (the regions
marked "usable" appear to be adjacent). And the E820_RAM contiguous
regions are not per-node or anything like that. Not that this is to
be taken as the "One True Way", but still, it's a data point.

Anyhow, my original question/worry ("Oonce you create Type17 DIMMs
from all the RAM, and Type19 regions for each E820_RAM type entry,
how do you tie them together with Type20s ?") has been answered
("You just don't" :) )

Just figured I'd share this, maybe it can be useful to someone else
besides me...

Cheers,
--Gabriel

Re: [Qemu-devel] E820 (Re: [v4 PATCH 00/12] SMBIOS: build full tables in QEMU)

[Kevin O'Connor]

On Wed, Mar 26, 2014 at 03:58:50PM -0400, Gabriel L. Somlo wrote:
> - SeaBIOS is still in charge of providing the smbios_entry_point
>   structure, and it's unlikely we can reasonably expect it to
>   bump the version to 2.5 (not that it seems to matter, if my
>   tests are to be believed)

This is why ultimately we want to use the romfile_loader mechanism for
smbios - that interface will allow qemu to generate both the smbios
table and the smbios entry point.

-Kevin

[Qemu-devel] E820 (Re: [v4 PATCH 00/12] SMBIOS: build full tables in QEMU)

[Gabriel L. Somlo]

On Tue, Mar 18, 2014 at 07:23:17PM -0400, Gabriel L. Somlo wrote:
> At this point, can anyone with access to a real, physical, NUMA
> system dump the smbios tables with dmidecode and post them here?
> I think that would be very informative.

So I thrashed around a bit trying to find a real NUMA box,
and found a Dell R410 whose BIOS claims to support NUMA by
disabling the "Node Interleaving" option (haven't actually
configured it to run NUMA, but based on what else I found,
I no longer think I need to -- keep reading :)

So, to my surprise, I noticed this machine did NOT have any
Type 20 tables in SMBIOS at all !!!

Then, after a more careful reading of the SMBIOS manual, I
noticed that Type 20 was made OPTIONAL as v2.5 of the spec,
cca. 2006 !!!

In conclusion, we *could* simply scan e820 for E820_RAM type
regions and generate Type 19 tables for each one we find,
and we no longer have to care at all about generating Type 20
nodes to link 19s to 17s, which basically makes the problem
go away, as far as I am concerned.

I tested this (omitting Type 20) on OS X (10.6 through 10.9),
XPsp3 and Win7, successfully, without either of them noticing
any changes from previous executions where smbios tables
were "different" (i.e., when they used to include T20).


At this point, though, I'd like some feedback before I shoot
out another version (v5) of this patch set:


- Should I pretend we're generating v2.5 tables ?

- this means Type 4 now has extra fields (i.e. the
  number of cores, etc), which should be relatively
  easy to add, so I'd be OK doing that, if we agree
  on everything else. Heck, how about going all out
  and implementing v2.8 ? That would get us past the
  2T limit in Types 16, 17, and 19 !

- SeaBIOS is still in charge of providing the smbios_entry_point
  structure, and it's unlikely we can reasonably expect it to
  bump the version to 2.5 (not that it seems to matter, if my
  tests are to be believed)

- on that note, SeaBIOS will still cheerfully generate
  Type 20 tables if nothing is supplied to it via fw_cfg
  from QEMU, and, again, I'm not sure how likely it is
  we can get it to stop doing that :)

- Does anyone still care ? ;)


Let me know what you all think!

Thanks,
--Gabriel