Re: [Qemu-devel] [PATCH v6 1/7] linker-loader: Add new 'write pointer' command

[Eric Blake]

[meta-comment]

On 02/16/2017 05:10 AM, Igor Mammedov wrote:
> On Wed, 15 Feb 2017 11:19:44 -0800
> Ben Warren  wrote:
> 
>>> On Feb 15, 2017, at 11:14 AM, Ben Warren  wrote:
...
> This is similar to the existing 'add pointer' functionality, but 
> instead

13 levels of quoting...

 +char pad[120];  
>>> wr_pointer is 121 (56 + 56 + 32 + 32 + 1), so 124 still makes sense, 
>>> doesn’t it? (also, 124 + 4 from command) % 8 == 0, so it’s nicely aligned.  
>> I mean (56 + 56 + 4 + 4 + 1), of course :)
> I' was wrong, as Laszlo pointed out pad is a member of union so it
> as the biggest member defines total size of union/struct
> as far as new command doesn't exceed 124 bytes size.

...and several pages of scrolling before I find the 3-line real content
of the message...

> #endif  

...buried by yet more scrolling through excessive quoting.  It's not
only okay, but ENCOURAGED, to trim your replies down to the relevant
portions (as I've done here), to quickly call attention to the content
you are adding to the thread.  Publicly archived lists have the benefit
that you don't have to full-quote the entire thread, because readers
interested in trimmed content can refer to the archives.

-- 
Eric Blake   eblake redhat com+1-919-301-3266
Libvirt virtualization library http://libvirt.org



signature.asc
Description: OpenPGP digital signature

Re: [Qemu-devel] [PATCH v6 1/7] linker-loader: Add new 'write pointer' command

[Igor Mammedov]

On Wed, 15 Feb 2017 11:19:44 -0800
Ben Warren  wrote:

> > On Feb 15, 2017, at 11:14 AM, Ben Warren  wrote:
> >   
> >> 
> >> On Feb 15, 2017, at 10:24 AM, Igor Mammedov  >> > wrote:
> >> 
> >> On Wed, 15 Feb 2017 20:04:40 +0200
> >> "Michael S. Tsirkin" > wrote:
> >>   
> >>> On Wed, Feb 15, 2017 at 06:43:09PM +0100, Igor Mammedov wrote:  
>  On Wed, 15 Feb 2017 18:39:06 +0200
>  "Michael S. Tsirkin" > wrote:
>    
> > On Wed, Feb 15, 2017 at 04:56:02PM +0100, Igor Mammedov wrote:
> >> On Wed, 15 Feb 2017 17:30:00 +0200
> >> "Michael S. Tsirkin" > wrote:
> >>   
> >>> On Wed, Feb 15, 2017 at 04:22:25PM +0100, Igor Mammedov wrote:  
>  On Wed, 15 Feb 2017 15:13:20 +0100
>  Laszlo Ersek > wrote:
>    
> > Commenting under Igor's reply for simplicity
> > 
> > On 02/15/17 11:57, Igor Mammedov wrote:
> >> On Tue, 14 Feb 2017 22:15:43 -0800
> >> b...@skyportsystems.com  wrote:
> >>   
> >>> From: Ben Warren  >>> >
> >>> 
> >>> This is similar to the existing 'add pointer' functionality, but 
> >>> instead
> >>> of instructing the guest (BIOS or UEFI) to patch memory, it 
> >>> instructs
> >>> the guest to write the pointer back to QEMU via a writeable 
> >>> fw_cfg file.
> >>> 
> >>> Signed-off-by: Ben Warren  >>> >
> >>> ---
> >>> hw/acpi/bios-linker-loader.c | 58 
> >>> ++--
> >>> include/hw/acpi/bios-linker-loader.h |  6 
> >>> 2 files changed, 61 insertions(+), 3 deletions(-)
> >>> 
> >>> diff --git a/hw/acpi/bios-linker-loader.c 
> >>> b/hw/acpi/bios-linker-loader.c
> >>> index d963ebe..5030cf1 100644
> >>> --- a/hw/acpi/bios-linker-loader.c
> >>> +++ b/hw/acpi/bios-linker-loader.c
> >>> @@ -78,6 +78,19 @@ struct BiosLinkerLoaderEntry {
> >>> uint32_t length;
> >>> } cksum;
> >>> 
> >>> +/*
> >>> + * COMMAND_WRITE_POINTER - write the fw_cfg file 
> >>> (originating from
> >>> + * @dest_file) at @wr_pointer.offset, by adding a 
> >>> pointer to the table
> >>> + * originating from @src_file. 1,2,4 or 8 byte unsigned
> >>> + * addition is used depending on @wr_pointer.size.
> >>> + */  
> > 
> > The words "adding" and "addition" are causing confusion here.
> > 
> > In all of the previous discussion, *addition* was out of scope from
> > WRITE_POINTER. Again, the firmware is specifically not required to
> > *read* any part of the fw_cfg blob identified by "dest_file".
> > 
> > WRITE_POINTER instructs the firmware to return the allocation 
> > address of
> > the downloaded "src_file" to QEMU. Any necessary runtime 
> > subscripting
> > within "src_file" is to be handled by QEMU code dynamically.
> > 
> > For example, consider that "src_file" has *several* fields that QEMU
> > wants to massage; in that case, indexing within QEMU code with field
> > offsets is simply unavoidable.
>  what I don't like here is that this indexing would be rather fragile
>  and has to be done in different parts of QEMU /device, AML/.
>  
>  I'd prefer this helper function to have the same @src_offset
>  behavior as ADD_POINTER where patched address could point to
>  any part of src_file i.e. not just beginning.
> >>> 
> >>> 
> >>> 
> >>>/*
> >>> * COMMAND_ADD_POINTER - patch the table (originating from
> >>> * @dest_file) at @pointer.offset, by adding a pointer to the 
> >>> table
> >>> * originating from @src_file. 1,2,4 or 8 byte unsigned
> >>> * addition is used depending on @pointer.size.
> >>> */
> >>> 
> >>> so the way ADD works is
> >>>   read at offset
> >>>   add table address
> >>>   write result at offset
> >>> 
> >>> in other words it is always beginning of table that is added.  
> >> more exactly it's, read at 
> >>  src_offset = *(dst_blob_ptr+dst_offset)
> >>  *(dst_blob+dst_offset) = src_blob_ptr + src_offset
> >>   
> >>> Would the following be acceptable?
> 

Re: [Qemu-devel] [PATCH v6 1/7] linker-loader: Add new 'write pointer' command

[Laszlo Ersek]

On 02/16/17 09:25, Igor Mammedov wrote:
> On Wed, 15 Feb 2017 21:34:45 +0200
> "Michael S. Tsirkin"  wrote:
> 
>> On Wed, Feb 15, 2017 at 07:24:36PM +0100, Igor Mammedov wrote:
 As long as all users pass in 0 though there's a real possibility guests
 will implement this incorrectly.  
>>> We are here to ensure that at least Seabios (I'll review it)
>>> and OVMF (Laszlo would take care of it I suppose) do it right,
>>> and if there are other firmwares, they should do it correctly
>>> as described fix their own bugs later wrt randomly written
>>> implementation.  
>>
>> As long as it's untested, it can always do the wrong thing
>> even if it looks right, or it can bitrot.
>>
 I guess we can put in the offset just
 behind the zero-filled padding we have there.  
>>> I've assumed padding was there to make commands fixed size and give
>>> a room for future extensions  
>>
>> you mean
>> char pad[124];
>> ?
>>
>> Right. So you can easily add fields, but if you do firmware
>> will just assume it does not know how to handle these
>> commands and skip them.
>>
>>> so hunk changing BiosLinkerLoaderEntry
>>> would look like:
>>>
>>> diff --git a/hw/acpi/bios-linker-loader.c b/hw/acpi/bios-linker-loader.c
>>> index d963ebe..6983713 100644
>>> --- a/hw/acpi/bios-linker-loader.c
>>> +++ b/hw/acpi/bios-linker-loader.c
>>> @@ -49,6 +49,7 @@ struct BiosLinkerLoaderEntry {
>>>  char file[BIOS_LINKER_LOADER_FILESZ];
>>>  uint32_t align;
>>>  uint8_t zone;
>>> +uint32_t padding;
>>>  } alloc;
>>>  
>>>  /*
>>> @@ -62,6 +63,7 @@ struct BiosLinkerLoaderEntry {
>>>  char src_file[BIOS_LINKER_LOADER_FILESZ];
>>>  uint32_t offset;
>>>  uint8_t size;
>>> +uint32_t padding;
>>>  } pointer;
>>>  
>>>  /*
>>> @@ -76,10 +78,25 @@ struct BiosLinkerLoaderEntry {
>>>  uint32_t offset;
>>>  uint32_t start;
>>>  uint32_t length;
>>> +uint32_t padding;
>>>  } cksum;  
>>
>> why is this necessary?
>>
>>> +/*
>>> + * COMMAND_WRITE_POINTER - write the fw_cfg file (originating from
>>> + * @dest_file) at @wr_pointer.offset, by writing a pointer to 
>>> @src_offset
>>> + * within the table originating from @src_file. 1,2,4 or 8 byte 
>>> unsigned
>>> + * addition is used depending on @wr_pointer.size.
>>> + */
>>> + struct {
>>> + char dest_file[BIOS_LINKER_LOADER_FILESZ];
>>> + char src_file[BIOS_LINKER_LOADER_FILESZ];
>>> + uint32_t dst_offset;
>>> + uint32_t src_offset;
>>> + uint8_t size;
>>> +} wr_pointer;
>>> +
>>>  /* padding */
>>> -char pad[124];
>>> +char pad[120];  
>>
>> and this shrinks entry size by 4 bytes. Why?
> so total size won't change, due to +4 to account for src_offest in wr_pointer

To be clear, I still disagree with the introduction of "src_offset"
(please let's discuss that in the other sub-thread). Still I can comment
on this question in isolation:

The individual command structures, and the "pad" field (which is 124
bytes), are all members of a *union*. They all start at offset zero in
the union, and the union's size is determined by the largest member,
plus any additional padding at the end that the compiler wishes to
append. (In this case, becasue we use QEMU_PACKED, the compiler doesn't
do that.)

Therefore, as long as you don't exceed the size of

  char pad[124];

you can extend any of the command structures freely -- "pad" will remain
the largest union member, and thereby dictate the size of the union.

In short, "pad" does not *follow* any of the command structures; they
are all laid over each other, from offset 0 in the union.

(If you are curious about the magic constant 124, that comes from 128
minus the size of the "command" field in the outermost
BiosLinkerLoaderEntry structure. The guiding principle here is that the
outermost BiosLinkerLoaderEntry structure be 128 bytes in size.)

--*--

Again: I disagree with the introduction of "src_offset", and I'd like to
hear your opinion about my counter-argument in the other subthread.

Thanks
Laszlo

> 
>>
>>>  };
>>>  } QEMU_PACKED;
>>>  typedef struct BiosLinkerLoaderEntry BiosLinkerLoaderEntry;
>>>
>>>
 I'm mostly concerned we are adding new features to something
 that has been through 25 revisions already.  
>>> It's ABI so it's worth extra effort,  
>>
>> There's always yet another possible enhancement you can add.
>> I see it as a bit of a feature creep frankly.
>>
>>> it looks like only one more revision is left and there is
>>> about a week left to post and merge it.  
>>
>> If we can do it quickly, fine, but I think we should merge ASAP.
>>
> 

Re: [Qemu-devel] [PATCH v6 1/7] linker-loader: Add new 'write pointer' command

[Ben Warren]

> On Feb 15, 2017, at 11:14 AM, Ben Warren  wrote:
> 
>> 
>> On Feb 15, 2017, at 10:24 AM, Igor Mammedov > > wrote:
>> 
>> On Wed, 15 Feb 2017 20:04:40 +0200
>> "Michael S. Tsirkin" > wrote:
>> 
>>> On Wed, Feb 15, 2017 at 06:43:09PM +0100, Igor Mammedov wrote:
 On Wed, 15 Feb 2017 18:39:06 +0200
 "Michael S. Tsirkin" > wrote:
 
> On Wed, Feb 15, 2017 at 04:56:02PM +0100, Igor Mammedov wrote:  
>> On Wed, 15 Feb 2017 17:30:00 +0200
>> "Michael S. Tsirkin" > wrote:
>> 
>>> On Wed, Feb 15, 2017 at 04:22:25PM +0100, Igor Mammedov wrote:
 On Wed, 15 Feb 2017 15:13:20 +0100
 Laszlo Ersek > wrote:
 
> Commenting under Igor's reply for simplicity
> 
> On 02/15/17 11:57, Igor Mammedov wrote:  
>> On Tue, 14 Feb 2017 22:15:43 -0800
>> b...@skyportsystems.com  wrote:
>> 
>>> From: Ben Warren >> >
>>> 
>>> This is similar to the existing 'add pointer' functionality, but 
>>> instead
>>> of instructing the guest (BIOS or UEFI) to patch memory, it 
>>> instructs
>>> the guest to write the pointer back to QEMU via a writeable fw_cfg 
>>> file.
>>> 
>>> Signed-off-by: Ben Warren >> >
>>> ---
>>> hw/acpi/bios-linker-loader.c | 58 
>>> ++--
>>> include/hw/acpi/bios-linker-loader.h |  6 
>>> 2 files changed, 61 insertions(+), 3 deletions(-)
>>> 
>>> diff --git a/hw/acpi/bios-linker-loader.c 
>>> b/hw/acpi/bios-linker-loader.c
>>> index d963ebe..5030cf1 100644
>>> --- a/hw/acpi/bios-linker-loader.c
>>> +++ b/hw/acpi/bios-linker-loader.c
>>> @@ -78,6 +78,19 @@ struct BiosLinkerLoaderEntry {
>>> uint32_t length;
>>> } cksum;
>>> 
>>> +/*
>>> + * COMMAND_WRITE_POINTER - write the fw_cfg file 
>>> (originating from
>>> + * @dest_file) at @wr_pointer.offset, by adding a pointer 
>>> to the table
>>> + * originating from @src_file. 1,2,4 or 8 byte unsigned
>>> + * addition is used depending on @wr_pointer.size.
>>> + */
> 
> The words "adding" and "addition" are causing confusion here.
> 
> In all of the previous discussion, *addition* was out of scope from
> WRITE_POINTER. Again, the firmware is specifically not required to
> *read* any part of the fw_cfg blob identified by "dest_file".
> 
> WRITE_POINTER instructs the firmware to return the allocation address 
> of
> the downloaded "src_file" to QEMU. Any necessary runtime subscripting
> within "src_file" is to be handled by QEMU code dynamically.
> 
> For example, consider that "src_file" has *several* fields that QEMU
> wants to massage; in that case, indexing within QEMU code with field
> offsets is simply unavoidable.  
 what I don't like here is that this indexing would be rather fragile
 and has to be done in different parts of QEMU /device, AML/.
 
 I'd prefer this helper function to have the same @src_offset
 behavior as ADD_POINTER where patched address could point to
 any part of src_file i.e. not just beginning.  
>>> 
>>> 
>>> 
>>>/*
>>> * COMMAND_ADD_POINTER - patch the table (originating from
>>> * @dest_file) at @pointer.offset, by adding a pointer to the 
>>> table
>>> * originating from @src_file. 1,2,4 or 8 byte unsigned
>>> * addition is used depending on @pointer.size.
>>> */
>>> 
>>> so the way ADD works is
>>> read at offset
>>> add table address
>>> write result at offset
>>> 
>>> in other words it is always beginning of table that is added.
>> more exactly it's, read at 
>>  src_offset = *(dst_blob_ptr+dst_offset)
>>  *(dst_blob+dst_offset) = src_blob_ptr + src_offset
>> 
>>> Would the following be acceptable?
>>> 
>>> 
>>> * COMMAND_WRITE_POINTER - update the fw_cfg file (originating 
>>> from
>>> * @dest_file) at @wr_pointer.offset, by writing a pointer to 
>>> the table
>>> * originating from @src_file. 1,2,4 or 8 byte unsigned value
>>> * is written depending on 

Re: [Qemu-devel] [PATCH v6 1/7] linker-loader: Add new 'write pointer' command

[Igor Mammedov]

On Wed, 15 Feb 2017 10:14:55 -0800
Ben Warren  wrote:

> > On Feb 15, 2017, at 10:06 AM, Michael S. Tsirkin  wrote:
> > 
> > On Wed, Feb 15, 2017 at 09:54:08AM -0800, Ben Warren wrote:  
> >> 
> >>On Feb 15, 2017, at 9:43 AM, Igor Mammedov  wrote:
> >> 
> >>On Wed, 15 Feb 2017 18:39:06 +0200
> >>"Michael S. Tsirkin"  wrote:
> >> 
> >> 
> >>On Wed, Feb 15, 2017 at 04:56:02PM +0100, Igor Mammedov wrote:
> >> 
> >>On Wed, 15 Feb 2017 17:30:00 +0200
> >>"Michael S. Tsirkin"  wrote:
> >> 
> >> 
> >>On Wed, Feb 15, 2017 at 04:22:25PM +0100, Igor Mammedov 
> >> wrote:
> >> 
> >> 
> >>On Wed, 15 Feb 2017 15:13:20 +0100
> >>Laszlo Ersek  wrote:
> >> 
> >> 
> >>Commenting under Igor's reply for simplicity
> >> 
> >>On 02/15/17 11:57, Igor Mammedov wrote:
> >> 
> >>On Tue, 14 Feb 2017 22:15:43 -0800
> >>b...@skyportsystems.com wrote:
> >> 
> >> 
> >>From: Ben Warren 
> >> 
> >>This is similar to the existing 'add 
> >> pointer'
> >>functionality, but instead
> >>of instructing the guest (BIOS or UEFI) to
> >>patch memory, it instructs
> >>the guest to write the pointer back to QEMU 
> >> via
> >>a writeable fw_cfg file.
> >> 
> >>Signed-off-by: Ben Warren <  
> >>b...@skyportsystems.com>  
> >>---
> >>hw/acpi/bios-linker-loader.c | 58
> >>++--
> >>include/hw/acpi/bios-linker-loader.h |  6 
> >> 
> >>2 files changed, 61 insertions(+), 3 
> >> deletions
> >>(-)
> >> 
> >>diff --git a/hw/acpi/bios-linker-loader.c 
> >> b/hw/
> >>acpi/bios-linker-loader.c
> >>index d963ebe..5030cf1 100644
> >>--- a/hw/acpi/bios-linker-loader.c
> >>+++ b/hw/acpi/bios-linker-loader.c
> >>@@ -78,6 +78,19 @@ struct 
> >> BiosLinkerLoaderEntry
> >>{
> >>uint32_t length;
> >>} cksum;
> >> 
> >>+/*
> >>+ * COMMAND_WRITE_POINTER - write 
> >> the
> >>fw_cfg file (originating from
> >>+ * @dest_file) at 
> >> @wr_pointer.offset,
> >>by adding a pointer to the table
> >>+ * originating from @src_file. 
> >> 1,2,4
> >>or 8 byte unsigned
> >>+ * addition is used depending on
> >>@wr_pointer.size.
> >>+ */  
> >> 
> >> 
> >>The words "adding" and "addition" are causing 
> >> confusion
> >>here.
> >> 
> >>In all of the previous discussion, *addition* was 
> >> out
> >>of scope from
> >>WRITE_POINTER. Again, the firmware is specifically 
> >> not
> >>required to
> >>*read* any part of the fw_cfg blob identified by
> >>"dest_file".
> >> 
> >>WRITE_POINTER instructs the firmware to return the
> >>allocation address of
> >>the downloaded "src_file" to QEMU. Any necessary
> >>runtime subscripting
> >>within "src_file" is to be handled by QEMU code
> >>dynamically.
> >> 
> >>For example, consider that "src_file" has *several*
> >>fields that QEMU
> >>wants to massage; in that case, indexing within QEMU
> >>code with field
> >>offsets is simply unavoidable.
> >> 
> >>what I don't like here is that this indexing would be
> >>rather fragile
> >>and has to be done in different parts of QEMU /device, 
> >> AML
> >>   

Re: [Qemu-devel] [PATCH v6 1/7] linker-loader: Add new 'write pointer' command

[Michael S. Tsirkin]

On Wed, Feb 15, 2017 at 06:43:09PM +0100, Igor Mammedov wrote:
> On Wed, 15 Feb 2017 18:39:06 +0200
> "Michael S. Tsirkin"  wrote:
> 
> > On Wed, Feb 15, 2017 at 04:56:02PM +0100, Igor Mammedov wrote:
> > > On Wed, 15 Feb 2017 17:30:00 +0200
> > > "Michael S. Tsirkin"  wrote:
> > >   
> > > > On Wed, Feb 15, 2017 at 04:22:25PM +0100, Igor Mammedov wrote:  
> > > > > On Wed, 15 Feb 2017 15:13:20 +0100
> > > > > Laszlo Ersek  wrote:
> > > > > 
> > > > > > Commenting under Igor's reply for simplicity
> > > > > > 
> > > > > > On 02/15/17 11:57, Igor Mammedov wrote:
> > > > > > > On Tue, 14 Feb 2017 22:15:43 -0800
> > > > > > > b...@skyportsystems.com wrote:
> > > > > > >   
> > > > > > >> From: Ben Warren 
> > > > > > >>
> > > > > > >> This is similar to the existing 'add pointer' functionality, but 
> > > > > > >> instead
> > > > > > >> of instructing the guest (BIOS or UEFI) to patch memory, it 
> > > > > > >> instructs
> > > > > > >> the guest to write the pointer back to QEMU via a writeable 
> > > > > > >> fw_cfg file.
> > > > > > >>
> > > > > > >> Signed-off-by: Ben Warren 
> > > > > > >> ---
> > > > > > >>  hw/acpi/bios-linker-loader.c | 58 
> > > > > > >> ++--
> > > > > > >>  include/hw/acpi/bios-linker-loader.h |  6 
> > > > > > >>  2 files changed, 61 insertions(+), 3 deletions(-)
> > > > > > >>
> > > > > > >> diff --git a/hw/acpi/bios-linker-loader.c 
> > > > > > >> b/hw/acpi/bios-linker-loader.c
> > > > > > >> index d963ebe..5030cf1 100644
> > > > > > >> --- a/hw/acpi/bios-linker-loader.c
> > > > > > >> +++ b/hw/acpi/bios-linker-loader.c
> > > > > > >> @@ -78,6 +78,19 @@ struct BiosLinkerLoaderEntry {
> > > > > > >>  uint32_t length;
> > > > > > >>  } cksum;
> > > > > > >>  
> > > > > > >> +/*
> > > > > > >> + * COMMAND_WRITE_POINTER - write the fw_cfg file 
> > > > > > >> (originating from
> > > > > > >> + * @dest_file) at @wr_pointer.offset, by adding a 
> > > > > > >> pointer to the table
> > > > > > >> + * originating from @src_file. 1,2,4 or 8 byte unsigned
> > > > > > >> + * addition is used depending on @wr_pointer.size.
> > > > > > >> + */  
> > > > > > 
> > > > > > The words "adding" and "addition" are causing confusion here.
> > > > > > 
> > > > > > In all of the previous discussion, *addition* was out of scope from
> > > > > > WRITE_POINTER. Again, the firmware is specifically not required to
> > > > > > *read* any part of the fw_cfg blob identified by "dest_file".
> > > > > > 
> > > > > > WRITE_POINTER instructs the firmware to return the allocation 
> > > > > > address of
> > > > > > the downloaded "src_file" to QEMU. Any necessary runtime 
> > > > > > subscripting
> > > > > > within "src_file" is to be handled by QEMU code dynamically.
> > > > > > 
> > > > > > For example, consider that "src_file" has *several* fields that QEMU
> > > > > > wants to massage; in that case, indexing within QEMU code with field
> > > > > > offsets is simply unavoidable.
> > > > > what I don't like here is that this indexing would be rather fragile
> > > > > and has to be done in different parts of QEMU /device, AML/.
> > > > > 
> > > > > I'd prefer this helper function to have the same @src_offset
> > > > > behavior as ADD_POINTER where patched address could point to
> > > > > any part of src_file i.e. not just beginning.
> > > > 
> > > > 
> > > > 
> > > > /*
> > > >  * COMMAND_ADD_POINTER - patch the table (originating from
> > > >  * @dest_file) at @pointer.offset, by adding a pointer to the 
> > > > table
> > > >  * originating from @src_file. 1,2,4 or 8 byte unsigned
> > > >  * addition is used depending on @pointer.size.
> > > >  */
> > > >  
> > > > so the way ADD works is
> > > > read at offset
> > > > add table address
> > > > write result at offset
> > > > 
> > > > in other words it is always beginning of table that is added.  
> > > more exactly it's, read at 
> > >   src_offset = *(dst_blob_ptr+dst_offset)
> > >   *(dst_blob+dst_offset) = src_blob_ptr + src_offset
> > >   
> > > > Would the following be acceptable?
> > > > 
> > > > 
> > > >  * COMMAND_WRITE_POINTER - update the fw_cfg file (originating 
> > > > from
> > > >  * @dest_file) at @wr_pointer.offset, by writing a pointer to 
> > > > the table
> > > >  * originating from @src_file. 1,2,4 or 8 byte unsigned value
> > > >  * is written depending on @wr_pointer.size.  
> > > it looses 'adding' part of ADD_POINTER command which handles src_offset,
> > > however implementing adding part looks a bit complicated
> > > as patched blob (dst) is not in guest memory but in QEMU and
> > > on reset *(dst_blob+dst_offset) should be reset to src_offset.
> > > Considering dst file could be device specific memory 

Re: [Qemu-devel] [PATCH v6 1/7] linker-loader: Add new 'write pointer' command

[Ben Warren]

> On Feb 15, 2017, at 9:43 AM, Igor Mammedov  wrote:
> 
> On Wed, 15 Feb 2017 18:39:06 +0200
> "Michael S. Tsirkin" > wrote:
> 
>> On Wed, Feb 15, 2017 at 04:56:02PM +0100, Igor Mammedov wrote:
>>> On Wed, 15 Feb 2017 17:30:00 +0200
>>> "Michael S. Tsirkin"  wrote:
>>> 
 On Wed, Feb 15, 2017 at 04:22:25PM +0100, Igor Mammedov wrote:  
> On Wed, 15 Feb 2017 15:13:20 +0100
> Laszlo Ersek  wrote:
> 
>> Commenting under Igor's reply for simplicity
>> 
>> On 02/15/17 11:57, Igor Mammedov wrote:
>>> On Tue, 14 Feb 2017 22:15:43 -0800
>>> b...@skyportsystems.com wrote:
>>> 
 From: Ben Warren 
 
 This is similar to the existing 'add pointer' functionality, but 
 instead
 of instructing the guest (BIOS or UEFI) to patch memory, it instructs
 the guest to write the pointer back to QEMU via a writeable fw_cfg 
 file.
 
 Signed-off-by: Ben Warren 
 ---
 hw/acpi/bios-linker-loader.c | 58 
 ++--
 include/hw/acpi/bios-linker-loader.h |  6 
 2 files changed, 61 insertions(+), 3 deletions(-)
 
 diff --git a/hw/acpi/bios-linker-loader.c 
 b/hw/acpi/bios-linker-loader.c
 index d963ebe..5030cf1 100644
 --- a/hw/acpi/bios-linker-loader.c
 +++ b/hw/acpi/bios-linker-loader.c
 @@ -78,6 +78,19 @@ struct BiosLinkerLoaderEntry {
 uint32_t length;
 } cksum;
 
 +/*
 + * COMMAND_WRITE_POINTER - write the fw_cfg file (originating 
 from
 + * @dest_file) at @wr_pointer.offset, by adding a pointer to 
 the table
 + * originating from @src_file. 1,2,4 or 8 byte unsigned
 + * addition is used depending on @wr_pointer.size.
 + */  
>> 
>> The words "adding" and "addition" are causing confusion here.
>> 
>> In all of the previous discussion, *addition* was out of scope from
>> WRITE_POINTER. Again, the firmware is specifically not required to
>> *read* any part of the fw_cfg blob identified by "dest_file".
>> 
>> WRITE_POINTER instructs the firmware to return the allocation address of
>> the downloaded "src_file" to QEMU. Any necessary runtime subscripting
>> within "src_file" is to be handled by QEMU code dynamically.
>> 
>> For example, consider that "src_file" has *several* fields that QEMU
>> wants to massage; in that case, indexing within QEMU code with field
>> offsets is simply unavoidable.
> what I don't like here is that this indexing would be rather fragile
> and has to be done in different parts of QEMU /device, AML/.
> 
> I'd prefer this helper function to have the same @src_offset
> behavior as ADD_POINTER where patched address could point to
> any part of src_file i.e. not just beginning.
 
 
 
/*
 * COMMAND_ADD_POINTER - patch the table (originating from
 * @dest_file) at @pointer.offset, by adding a pointer to the table
 * originating from @src_file. 1,2,4 or 8 byte unsigned
 * addition is used depending on @pointer.size.
 */
 
 so the way ADD works is
read at offset
add table address
write result at offset
 
 in other words it is always beginning of table that is added.  
>>> more exactly it's, read at 
>>>  src_offset = *(dst_blob_ptr+dst_offset)
>>>  *(dst_blob+dst_offset) = src_blob_ptr + src_offset
>>> 
 Would the following be acceptable?
 
 
 * COMMAND_WRITE_POINTER - update the fw_cfg file (originating from
 * @dest_file) at @wr_pointer.offset, by writing a pointer to the 
 table
 * originating from @src_file. 1,2,4 or 8 byte unsigned value
 * is written depending on @wr_pointer.size.  
>>> it looses 'adding' part of ADD_POINTER command which handles src_offset,
>>> however implementing adding part looks a bit complicated
>>> as patched blob (dst) is not in guest memory but in QEMU and
>>> on reset *(dst_blob+dst_offset) should be reset to src_offset.
>>> Considering dst file could be device specific memory (field/blob/whatever)
>>> it could be hard to track/notice proper reset behavior.
>>> 
>>> So now I'm not sure if src_offset is worth adding.  
>> 
>> Right. Let's just do this math in QEMU if we have to.
> Math complicates QEMU code though and not only QMEMU but AML code as well.
> Considering that we are adding a new command and don't have to keep
> any sort of compatibility we can pass src_offset as part
> of command instead of hiding it inside of dst_file.
> Something like this:
> 
> 

Re: [Qemu-devel] [PATCH v6 1/7] linker-loader: Add new 'write pointer' command

[Laszlo Ersek]

On 02/15/17 17:39, Michael S. Tsirkin wrote:
> On Wed, Feb 15, 2017 at 04:56:02PM +0100, Igor Mammedov wrote:
>> On Wed, 15 Feb 2017 17:30:00 +0200
>> "Michael S. Tsirkin"  wrote:
>>
>>> On Wed, Feb 15, 2017 at 04:22:25PM +0100, Igor Mammedov wrote:
 On Wed, 15 Feb 2017 15:13:20 +0100
 Laszlo Ersek  wrote:
   
> Commenting under Igor's reply for simplicity
>
> On 02/15/17 11:57, Igor Mammedov wrote:  
>> On Tue, 14 Feb 2017 22:15:43 -0800
>> b...@skyportsystems.com wrote:
>> 
>>> From: Ben Warren 
>>>
>>> This is similar to the existing 'add pointer' functionality, but instead
>>> of instructing the guest (BIOS or UEFI) to patch memory, it instructs
>>> the guest to write the pointer back to QEMU via a writeable fw_cfg file.
>>>
>>> Signed-off-by: Ben Warren 
>>> ---
>>>  hw/acpi/bios-linker-loader.c | 58 
>>> ++--
>>>  include/hw/acpi/bios-linker-loader.h |  6 
>>>  2 files changed, 61 insertions(+), 3 deletions(-)
>>>
>>> diff --git a/hw/acpi/bios-linker-loader.c b/hw/acpi/bios-linker-loader.c
>>> index d963ebe..5030cf1 100644
>>> --- a/hw/acpi/bios-linker-loader.c
>>> +++ b/hw/acpi/bios-linker-loader.c
>>> @@ -78,6 +78,19 @@ struct BiosLinkerLoaderEntry {
>>>  uint32_t length;
>>>  } cksum;
>>>  
>>> +/*
>>> + * COMMAND_WRITE_POINTER - write the fw_cfg file (originating 
>>> from
>>> + * @dest_file) at @wr_pointer.offset, by adding a pointer to 
>>> the table
>>> + * originating from @src_file. 1,2,4 or 8 byte unsigned
>>> + * addition is used depending on @wr_pointer.size.
>>> + */
>
> The words "adding" and "addition" are causing confusion here.
>
> In all of the previous discussion, *addition* was out of scope from
> WRITE_POINTER. Again, the firmware is specifically not required to
> *read* any part of the fw_cfg blob identified by "dest_file".
>
> WRITE_POINTER instructs the firmware to return the allocation address of
> the downloaded "src_file" to QEMU. Any necessary runtime subscripting
> within "src_file" is to be handled by QEMU code dynamically.
>
> For example, consider that "src_file" has *several* fields that QEMU
> wants to massage; in that case, indexing within QEMU code with field
> offsets is simply unavoidable.  
 what I don't like here is that this indexing would be rather fragile
 and has to be done in different parts of QEMU /device, AML/.

 I'd prefer this helper function to have the same @src_offset
 behavior as ADD_POINTER where patched address could point to
 any part of src_file i.e. not just beginning.  
>>>
>>>
>>>
>>> /*
>>>  * COMMAND_ADD_POINTER - patch the table (originating from
>>>  * @dest_file) at @pointer.offset, by adding a pointer to the table
>>>  * originating from @src_file. 1,2,4 or 8 byte unsigned
>>>  * addition is used depending on @pointer.size.
>>>  */
>>>  
>>> so the way ADD works is
>>> read at offset
>>> add table address
>>> write result at offset
>>>
>>> in other words it is always beginning of table that is added.
>> more exactly it's, read at 
>>   src_offset = *(dst_blob_ptr+dst_offset)
>>   *(dst_blob+dst_offset) = src_blob_ptr + src_offset
>>
>>> Would the following be acceptable?
>>>
>>>
>>>  * COMMAND_WRITE_POINTER - update the fw_cfg file (originating from
>>>  * @dest_file) at @wr_pointer.offset, by writing a pointer to the 
>>> table
>>>  * originating from @src_file. 1,2,4 or 8 byte unsigned value
>>>  * is written depending on @wr_pointer.size.
>> it looses 'adding' part of ADD_POINTER command which handles src_offset,
>> however implementing adding part looks a bit complicated
>> as patched blob (dst) is not in guest memory but in QEMU and
>> on reset *(dst_blob+dst_offset) should be reset to src_offset.
>> Considering dst file could be device specific memory (field/blob/whatever)
>> it could be hard to track/notice proper reset behavior.
>>
>> So now I'm not sure if src_offset is worth adding.
> 
> Right. Let's just do this math in QEMU if we have to.

Deal. :)

Thanks
Laszlo

Re: [Qemu-devel] [PATCH v6 1/7] linker-loader: Add new 'write pointer' command

[Michael S. Tsirkin]

On Wed, Feb 15, 2017 at 04:56:02PM +0100, Igor Mammedov wrote:
> On Wed, 15 Feb 2017 17:30:00 +0200
> "Michael S. Tsirkin"  wrote:
> 
> > On Wed, Feb 15, 2017 at 04:22:25PM +0100, Igor Mammedov wrote:
> > > On Wed, 15 Feb 2017 15:13:20 +0100
> > > Laszlo Ersek  wrote:
> > >   
> > > > Commenting under Igor's reply for simplicity
> > > > 
> > > > On 02/15/17 11:57, Igor Mammedov wrote:  
> > > > > On Tue, 14 Feb 2017 22:15:43 -0800
> > > > > b...@skyportsystems.com wrote:
> > > > > 
> > > > >> From: Ben Warren 
> > > > >>
> > > > >> This is similar to the existing 'add pointer' functionality, but 
> > > > >> instead
> > > > >> of instructing the guest (BIOS or UEFI) to patch memory, it instructs
> > > > >> the guest to write the pointer back to QEMU via a writeable fw_cfg 
> > > > >> file.
> > > > >>
> > > > >> Signed-off-by: Ben Warren 
> > > > >> ---
> > > > >>  hw/acpi/bios-linker-loader.c | 58 
> > > > >> ++--
> > > > >>  include/hw/acpi/bios-linker-loader.h |  6 
> > > > >>  2 files changed, 61 insertions(+), 3 deletions(-)
> > > > >>
> > > > >> diff --git a/hw/acpi/bios-linker-loader.c 
> > > > >> b/hw/acpi/bios-linker-loader.c
> > > > >> index d963ebe..5030cf1 100644
> > > > >> --- a/hw/acpi/bios-linker-loader.c
> > > > >> +++ b/hw/acpi/bios-linker-loader.c
> > > > >> @@ -78,6 +78,19 @@ struct BiosLinkerLoaderEntry {
> > > > >>  uint32_t length;
> > > > >>  } cksum;
> > > > >>  
> > > > >> +/*
> > > > >> + * COMMAND_WRITE_POINTER - write the fw_cfg file 
> > > > >> (originating from
> > > > >> + * @dest_file) at @wr_pointer.offset, by adding a pointer 
> > > > >> to the table
> > > > >> + * originating from @src_file. 1,2,4 or 8 byte unsigned
> > > > >> + * addition is used depending on @wr_pointer.size.
> > > > >> + */
> > > > 
> > > > The words "adding" and "addition" are causing confusion here.
> > > > 
> > > > In all of the previous discussion, *addition* was out of scope from
> > > > WRITE_POINTER. Again, the firmware is specifically not required to
> > > > *read* any part of the fw_cfg blob identified by "dest_file".
> > > > 
> > > > WRITE_POINTER instructs the firmware to return the allocation address of
> > > > the downloaded "src_file" to QEMU. Any necessary runtime subscripting
> > > > within "src_file" is to be handled by QEMU code dynamically.
> > > > 
> > > > For example, consider that "src_file" has *several* fields that QEMU
> > > > wants to massage; in that case, indexing within QEMU code with field
> > > > offsets is simply unavoidable.  
> > > what I don't like here is that this indexing would be rather fragile
> > > and has to be done in different parts of QEMU /device, AML/.
> > > 
> > > I'd prefer this helper function to have the same @src_offset
> > > behavior as ADD_POINTER where patched address could point to
> > > any part of src_file i.e. not just beginning.  
> > 
> > 
> > 
> > /*
> >  * COMMAND_ADD_POINTER - patch the table (originating from
> >  * @dest_file) at @pointer.offset, by adding a pointer to the table
> >  * originating from @src_file. 1,2,4 or 8 byte unsigned
> >  * addition is used depending on @pointer.size.
> >  */
> >  
> > so the way ADD works is
> > read at offset
> > add table address
> > write result at offset
> > 
> > in other words it is always beginning of table that is added.
> more exactly it's, read at 
>   src_offset = *(dst_blob_ptr+dst_offset)
>   *(dst_blob+dst_offset) = src_blob_ptr + src_offset
> 
> > Would the following be acceptable?
> > 
> > 
> >  * COMMAND_WRITE_POINTER - update the fw_cfg file (originating from
> >  * @dest_file) at @wr_pointer.offset, by writing a pointer to the 
> > table
> >  * originating from @src_file. 1,2,4 or 8 byte unsigned value
> >  * is written depending on @wr_pointer.size.
> it looses 'adding' part of ADD_POINTER command which handles src_offset,
> however implementing adding part looks a bit complicated
> as patched blob (dst) is not in guest memory but in QEMU and
> on reset *(dst_blob+dst_offset) should be reset to src_offset.
> Considering dst file could be device specific memory (field/blob/whatever)
> it could be hard to track/notice proper reset behavior.
> 
> So now I'm not sure if src_offset is worth adding.

Right. Let's just do this math in QEMU if we have to.

> > 
> > 
> > > 
> > >   
> > > > (1) So, the above looks correct, but please replace "adding" with
> > > > "storing", and "unsigned addition" with "store".
> > > > 
> > > > Side point: the case for ADD_POINTER is different; there we patch
> > > > several individual ACPI objects. The fact that I requested explicit
> > > > addition within the ADDR method, as opposed to pre-setting VGIA to a
> > > > nonzero offset, is an *incidental* limitation 

Re: [Qemu-devel] [PATCH v6 1/7] linker-loader: Add new 'write pointer' command

[Laszlo Ersek]

Commenting under Igor's reply for simplicity

On 02/15/17 11:57, Igor Mammedov wrote:
> On Tue, 14 Feb 2017 22:15:43 -0800
> b...@skyportsystems.com wrote:
> 
>> From: Ben Warren 
>>
>> This is similar to the existing 'add pointer' functionality, but instead
>> of instructing the guest (BIOS or UEFI) to patch memory, it instructs
>> the guest to write the pointer back to QEMU via a writeable fw_cfg file.
>>
>> Signed-off-by: Ben Warren 
>> ---
>>  hw/acpi/bios-linker-loader.c | 58 
>> ++--
>>  include/hw/acpi/bios-linker-loader.h |  6 
>>  2 files changed, 61 insertions(+), 3 deletions(-)
>>
>> diff --git a/hw/acpi/bios-linker-loader.c b/hw/acpi/bios-linker-loader.c
>> index d963ebe..5030cf1 100644
>> --- a/hw/acpi/bios-linker-loader.c
>> +++ b/hw/acpi/bios-linker-loader.c
>> @@ -78,6 +78,19 @@ struct BiosLinkerLoaderEntry {
>>  uint32_t length;
>>  } cksum;
>>  
>> +/*
>> + * COMMAND_WRITE_POINTER - write the fw_cfg file (originating from
>> + * @dest_file) at @wr_pointer.offset, by adding a pointer to the 
>> table
>> + * originating from @src_file. 1,2,4 or 8 byte unsigned
>> + * addition is used depending on @wr_pointer.size.
>> + */

The words "adding" and "addition" are causing confusion here.

In all of the previous discussion, *addition* was out of scope from
WRITE_POINTER. Again, the firmware is specifically not required to
*read* any part of the fw_cfg blob identified by "dest_file".

WRITE_POINTER instructs the firmware to return the allocation address of
the downloaded "src_file" to QEMU. Any necessary runtime subscripting
within "src_file" is to be handled by QEMU code dynamically.

For example, consider that "src_file" has *several* fields that QEMU
wants to massage; in that case, indexing within QEMU code with field
offsets is simply unavoidable.

(1) So, the above looks correct, but please replace "adding" with
"storing", and "unsigned addition" with "store".

Side point: the case for ADD_POINTER is different; there we patch
several individual ACPI objects. The fact that I requested explicit
addition within the ADDR method, as opposed to pre-setting VGIA to a
nonzero offset, is an *incidental* limitation (coming from the OVMF ACPI
SDT header probe suppressor), and we'll likely fix that up later, with
ALLOCATE command hints or something like that. However, in
WRITE_POINTER, asking for the exact allocation address of "src_file" is
an *inherent* characteristic.

For reference, this is the command's description from the (not as yet
posted) OVMF series:

// QemuLoaderCmdWritePointer: the bytes at
// [PointerOffset..PointerOffset+PointerSize) in the writeable fw_cfg
// file PointerFile are to receive the absolute address of PointeeFile,
// as allocated and downloaded by the firmware. Store the base address
// of where PointeeFile's contents have been placed (when
// QemuLoaderCmdAllocate has been executed for PointeeFile) to this
// portion of PointerFile.
//
// This command is similar to QemuLoaderCmdAddPointer; the difference is
// that the "pointer to patch" does not exist in guest-physical address
// space, only in "fw_cfg file space". In addition, the "pointer to
// patch" is not initialized by QEMU with a possibly nonzero offset
// value: the base address of the memory allocated for downloading
// PointeeFile shall not increment the pointer, but overwrite it.

In the last SeaBIOS patch series, namely

[SeaBIOS] [PATCH v3 2/2] QEMU fw_cfg: Add command to write back address
 of file

function romfile_loader_write_pointer() implemented just that plain
store (not an addition), and that was exactly right.

Continuing:

>> +struct {
>> +char dest_file[BIOS_LINKER_LOADER_FILESZ];
>> +char src_file[BIOS_LINKER_LOADER_FILESZ];
>> +uint32_t offset;
>> +uint8_t size;
>> +} wr_pointer;
>> +
>>  /* padding */
>>  char pad[124];
>>  };
>> @@ -85,9 +98,10 @@ struct BiosLinkerLoaderEntry {
>>  typedef struct BiosLinkerLoaderEntry BiosLinkerLoaderEntry;
>>  
>>  enum {
>> -BIOS_LINKER_LOADER_COMMAND_ALLOCATE = 0x1,
>> -BIOS_LINKER_LOADER_COMMAND_ADD_POINTER  = 0x2,
>> -BIOS_LINKER_LOADER_COMMAND_ADD_CHECKSUM = 0x3,
>> +BIOS_LINKER_LOADER_COMMAND_ALLOCATE  = 0x1,
>> +BIOS_LINKER_LOADER_COMMAND_ADD_POINTER   = 0x2,
>> +BIOS_LINKER_LOADER_COMMAND_ADD_CHECKSUM  = 0x3,
>> +BIOS_LINKER_LOADER_COMMAND_WRITE_POINTER = 0x4,
>>  };
>>  
>>  enum {
>> @@ -278,3 +292,41 @@ void bios_linker_loader_add_pointer(BIOSLinker *linker,
>>  
>>  g_array_append_vals(linker->cmd_blob, , sizeof entry);
>>  }
>> +
>> +/*
>> + * bios_linker_loader_write_pointer: ask guest to write a pointer to the
>> + * source file into the destination file, and write it back to QEMU via
>> + * fw_cfg DMA.
>> + *
>> + *