Re: [edk2-devel] static data in dxe_runtime modules

[Roman Kagan]

On Wed, Aug 14, 2019 at 09:26:47AM -0700, Andrew Fish via Groups.Io wrote:
> Every runtime driver I've every seen usually works like this:
> 1) Loads as an EFI driver and uses EFI Boot Services in its constructor (gBS, 
> gDS, AllocatePool(), etc.)
> 2) You use the EFI Boot Service to register the ExitBootServices Event. 
> 3) SetVirtualAddressMap event fires and converts all the pointers 
> 4) After all the ExitBootServices events have been processed the DXE Runtime 
> driver re-relocates the Runtime Driver
> 5) The next code that runs is a call from the kernel virtual mapping, and the 
> system is at runtime 
> 
> It is important to remember that when gRT->SetVirtualAddressMap() is
> called by the OS Loader (or early kernel) that gBS->ExitBootServices()
> has already been called. So by the time you get 3) almost all of EFI
> is gone. The only services that remain are gRT. Note you find the
> location of gRT by using the gST passed into the entry point of the
> driver. So here is lies the problem the entry point is passed a Handle
> (EFI Boot Services concept) and a pointer to gST (another boot
> services table). So you can't really reload a module and expect it to
> work. 

Indeed.  My hope was that the particular VariableRuntimeDxe module was
just a bunch of self-contained code that accesses all the data it needs
through the arguments it's passed.  Having now looked at the code, this
assumption is certainly wrong: the module depends on a lot of private
data that is initialized at boot services time.

Thanks and sorry for the noise,
Roman.

-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.

View/Reply Online (#45831): https://edk2.groups.io/g/devel/message/45831
Mute This Topic: https://groups.io/mt/32686575/21656
Group Owner: devel+ow...@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [edk2-devel] static data in dxe_runtime modules

[Laszlo Ersek]

On 08/14/19 18:26, Andrew Fish wrote:
> 
> 
>> On Aug 14, 2019, at 8:16 AM, Laszlo Ersek  wrote:
>>
>> On 08/13/19 13:23, Roman Kagan wrote:
>>> On Tue, Aug 13, 2019 at 11:10:27AM +0200, Laszlo Ersek wrote:
 On 08/12/19 20:43, Roman Kagan wrote:
> On Fri, Aug 09, 2019 at 04:07:00PM +, Roman Kagan via Groups.Io wrote:
>> On Thu, Aug 08, 2019 at 07:39:14PM +0200, Laszlo Ersek wrote:
>>> On 08/07/19 19:41, Andrew Fish wrote:
> On Aug 7, 2019, at 10:29 AM, Laszlo Ersek  wrote:
> On 08/05/19 12:18, Roman Kagan wrote:
>> On Sat, Aug 03, 2019 at 04:03:04AM +0200, Laszlo Ersek via Groups.Io 
>> wrote:
>>> On 08/01/19 21:16, Roman Kagan wrote:
> I'm convinced that OpenSSL needs to expose a new API for this 
> particular
> problem.
>>
>> Since, as you point out below, the problem only affects the essentially
>> broken configuration (SECURE_BOOT_ENABLE && !SMM_REQUIRE), I'm fine with
>> saving time and effort and sticking to the hack-ish approach proposed in
>> the bugzilla issue, which is to iterate over "thread-local" pointers and
>> EfiConvertPointer() on each.  (As long as it fixes the problem of
>> course; I'll test and report back.)
>
> It doesn't :(  It just gets slightly further and hits another static
> pointer variable which is not part of the thread-local array:
>
> ...
>  Pkcs7Verify
>EVP_add_digest
>  OBJ_NAME_add
>
> this one uses a few static pointer variables that are also initialized
> on demand and become stale upon SetVirtualAddressMap().

 So it looks like the issue can't be solved without making OpenSSL aware
 of this use case.
>>>
>>> Is reloading the module from scratch ruled out completely?
>>
>> Not my place to say authoritatively, but:
>> - it would be a first, as much as I can say,
>> - it would duplicate (in purpose) an existing facility.
>>
>> Personally I'd expect it to be rejected, but it's not up to me. If
>> you're willing to "build one to (possibly) throw away", that could be
>> the most direct way to get authoritative (= maintainer) feedback.
>>
> 
> Laszlo,
> 
> I thunk it is more likely to get rejected as it would not work. 
> 
> Every runtime driver I've every seen usually works like this:
> 1) Loads as an EFI driver and uses EFI Boot Services in its constructor (gBS, 
> gDS, AllocatePool(), etc.)
> 2) You use the EFI Boot Service to register the ExitBootServices Event. 
> 3) SetVirtualAddressMap event fires and converts all the pointers 
> 4) After all the ExitBootServices events have been processed the DXE Runtime 
> driver re-relocates the Runtime Driver
> 5) The next code that runs is a call from the kernel virtual mapping, and the 
> system is at runtime 
> 
> It is important to remember that when gRT->SetVirtualAddressMap() is called 
> by the OS Loader (or early kernel) that gBS->ExitBootServices() has already 
> been called. So by the time you get 3) almost all of EFI is gone. The only 
> services that remain are gRT. Note you find the location of gRT by using the 
> gST passed into the entry point of the driver. So here is lies the problem 
> the entry point is passed a Handle (EFI Boot Services concept) and a pointer 
> to gST (another boot services table). So you can't really reload a module and 
> expect it to work. 
> 
> In EFI the transition from Boot Service time to Runtime already requires a 
> lot of coding discipline to not call services at runtime that will go away 
> after ExitBootServices. Having C code that can be called in Physical mode, 
> and then with a virtual mapping is a very unnatural and what EFI does today 
> is the minimum required to make things work. 
> 
> Also remember dumping more into EFI runtime is stealing memory and usually 
> kernel virtual address space from the OS. 

Thank you for the detailed explanation!
Laszlo

-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.

View/Reply Online (#45820): https://edk2.groups.io/g/devel/message/45820
Mute This Topic: https://groups.io/mt/32686575/21656
Group Owner: devel+ow...@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [edk2-devel] static data in dxe_runtime modules

[Andrew Fish via Groups.Io]

> On Aug 14, 2019, at 8:16 AM, Laszlo Ersek  wrote:
> 
> On 08/13/19 13:23, Roman Kagan wrote:
>> On Tue, Aug 13, 2019 at 11:10:27AM +0200, Laszlo Ersek wrote:
>>> On 08/12/19 20:43, Roman Kagan wrote:
 On Fri, Aug 09, 2019 at 04:07:00PM +, Roman Kagan via Groups.Io wrote:
> On Thu, Aug 08, 2019 at 07:39:14PM +0200, Laszlo Ersek wrote:
>> On 08/07/19 19:41, Andrew Fish wrote:
 On Aug 7, 2019, at 10:29 AM, Laszlo Ersek  wrote:
 On 08/05/19 12:18, Roman Kagan wrote:
> On Sat, Aug 03, 2019 at 04:03:04AM +0200, Laszlo Ersek via Groups.Io 
> wrote:
>> On 08/01/19 21:16, Roman Kagan wrote:
 I'm convinced that OpenSSL needs to expose a new API for this 
 particular
 problem.
> 
> Since, as you point out below, the problem only affects the essentially
> broken configuration (SECURE_BOOT_ENABLE && !SMM_REQUIRE), I'm fine with
> saving time and effort and sticking to the hack-ish approach proposed in
> the bugzilla issue, which is to iterate over "thread-local" pointers and
> EfiConvertPointer() on each.  (As long as it fixes the problem of
> course; I'll test and report back.)
 
 It doesn't :(  It just gets slightly further and hits another static
 pointer variable which is not part of the thread-local array:
 
 ...
  Pkcs7Verify
EVP_add_digest
  OBJ_NAME_add
 
 this one uses a few static pointer variables that are also initialized
 on demand and become stale upon SetVirtualAddressMap().
>>> 
>>> So it looks like the issue can't be solved without making OpenSSL aware
>>> of this use case.
>> 
>> Is reloading the module from scratch ruled out completely?
> 
> Not my place to say authoritatively, but:
> - it would be a first, as much as I can say,
> - it would duplicate (in purpose) an existing facility.
> 
> Personally I'd expect it to be rejected, but it's not up to me. If
> you're willing to "build one to (possibly) throw away", that could be
> the most direct way to get authoritative (= maintainer) feedback.
> 

Laszlo,

I thunk it is more likely to get rejected as it would not work. 

Every runtime driver I've every seen usually works like this:
1) Loads as an EFI driver and uses EFI Boot Services in its constructor (gBS, 
gDS, AllocatePool(), etc.)
2) You use the EFI Boot Service to register the ExitBootServices Event. 
3) SetVirtualAddressMap event fires and converts all the pointers 
4) After all the ExitBootServices events have been processed the DXE Runtime 
driver re-relocates the Runtime Driver
5) The next code that runs is a call from the kernel virtual mapping, and the 
system is at runtime 

It is important to remember that when gRT->SetVirtualAddressMap() is called by 
the OS Loader (or early kernel) that gBS->ExitBootServices() has already been 
called. So by the time you get 3) almost all of EFI is gone. The only services 
that remain are gRT. Note you find the location of gRT by using the gST passed 
into the entry point of the driver. So here is lies the problem the entry point 
is passed a Handle (EFI Boot Services concept) and a pointer to gST (another 
boot services table). So you can't really reload a module and expect it to 
work. 

In EFI the transition from Boot Service time to Runtime already requires a lot 
of coding discipline to not call services at runtime that will go away after 
ExitBootServices. Having C code that can be called in Physical mode, and then 
with a virtual mapping is a very unnatural and what EFI does today is the 
minimum required to make things work. 

Also remember dumping more into EFI runtime is stealing memory and usually 
kernel virtual address space from the OS. 

Thanks,

Andrew Fish


> Thanks
> Laszlo
> 
>> I'd try to cook up a patch for that unless there's a strong no-go.
>> 
>> Thanks,
>> Roman.


-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.

View/Reply Online (#45623): https://edk2.groups.io/g/devel/message/45623
Mute This Topic: https://groups.io/mt/32686575/21656
Group Owner: devel+ow...@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [edk2-devel] static data in dxe_runtime modules

[Laszlo Ersek]

On 08/13/19 13:23, Roman Kagan wrote:
> On Tue, Aug 13, 2019 at 11:10:27AM +0200, Laszlo Ersek wrote:
>> On 08/12/19 20:43, Roman Kagan wrote:
>>> On Fri, Aug 09, 2019 at 04:07:00PM +, Roman Kagan via Groups.Io wrote:
 On Thu, Aug 08, 2019 at 07:39:14PM +0200, Laszlo Ersek wrote:
> On 08/07/19 19:41, Andrew Fish wrote:
>>> On Aug 7, 2019, at 10:29 AM, Laszlo Ersek  wrote:
>>> On 08/05/19 12:18, Roman Kagan wrote:
 On Sat, Aug 03, 2019 at 04:03:04AM +0200, Laszlo Ersek via Groups.Io 
 wrote:
> On 08/01/19 21:16, Roman Kagan wrote:
>>> I'm convinced that OpenSSL needs to expose a new API for this particular
>>> problem.

 Since, as you point out below, the problem only affects the essentially
 broken configuration (SECURE_BOOT_ENABLE && !SMM_REQUIRE), I'm fine with
 saving time and effort and sticking to the hack-ish approach proposed in
 the bugzilla issue, which is to iterate over "thread-local" pointers and
 EfiConvertPointer() on each.  (As long as it fixes the problem of
 course; I'll test and report back.)
>>>
>>> It doesn't :(  It just gets slightly further and hits another static
>>> pointer variable which is not part of the thread-local array:
>>>
>>> ...
>>>   Pkcs7Verify
>>> EVP_add_digest
>>>   OBJ_NAME_add
>>>
>>> this one uses a few static pointer variables that are also initialized
>>> on demand and become stale upon SetVirtualAddressMap().
>>
>> So it looks like the issue can't be solved without making OpenSSL aware
>> of this use case.
> 
> Is reloading the module from scratch ruled out completely?

Not my place to say authoritatively, but:
- it would be a first, as much as I can say,
- it would duplicate (in purpose) an existing facility.

Personally I'd expect it to be rejected, but it's not up to me. If
you're willing to "build one to (possibly) throw away", that could be
the most direct way to get authoritative (= maintainer) feedback.

Thanks
Laszlo

> I'd try to cook up a patch for that unless there's a strong no-go.
> 
> Thanks,
> Roman.
> 


-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.

View/Reply Online (#45615): https://edk2.groups.io/g/devel/message/45615
Mute This Topic: https://groups.io/mt/32686575/21656
Group Owner: devel+ow...@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [edk2-devel] static data in dxe_runtime modules

[Roman Kagan]

On Tue, Aug 13, 2019 at 11:10:27AM +0200, Laszlo Ersek wrote:
> On 08/12/19 20:43, Roman Kagan wrote:
> > On Fri, Aug 09, 2019 at 04:07:00PM +, Roman Kagan via Groups.Io wrote:
> >> On Thu, Aug 08, 2019 at 07:39:14PM +0200, Laszlo Ersek wrote:
> >>> On 08/07/19 19:41, Andrew Fish wrote:
> > On Aug 7, 2019, at 10:29 AM, Laszlo Ersek  wrote:
> > On 08/05/19 12:18, Roman Kagan wrote:
> >> On Sat, Aug 03, 2019 at 04:03:04AM +0200, Laszlo Ersek via Groups.Io 
> >> wrote:
> >>> On 08/01/19 21:16, Roman Kagan wrote:
> > I'm convinced that OpenSSL needs to expose a new API for this particular
> > problem.
> >>
> >> Since, as you point out below, the problem only affects the essentially
> >> broken configuration (SECURE_BOOT_ENABLE && !SMM_REQUIRE), I'm fine with
> >> saving time and effort and sticking to the hack-ish approach proposed in
> >> the bugzilla issue, which is to iterate over "thread-local" pointers and
> >> EfiConvertPointer() on each.  (As long as it fixes the problem of
> >> course; I'll test and report back.)
> > 
> > It doesn't :(  It just gets slightly further and hits another static
> > pointer variable which is not part of the thread-local array:
> > 
> > ...
> >   Pkcs7Verify
> > EVP_add_digest
> >   OBJ_NAME_add
> > 
> > this one uses a few static pointer variables that are also initialized
> > on demand and become stale upon SetVirtualAddressMap().
> 
> So it looks like the issue can't be solved without making OpenSSL aware
> of this use case.

Is reloading the module from scratch ruled out completely?

I'd try to cook up a patch for that unless there's a strong no-go.

Thanks,
Roman.

-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.

View/Reply Online (#45525): https://edk2.groups.io/g/devel/message/45525
Mute This Topic: https://groups.io/mt/32686575/21656
Group Owner: devel+ow...@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [edk2-devel] static data in dxe_runtime modules

[Laszlo Ersek]

On 08/12/19 20:43, Roman Kagan wrote:
> On Fri, Aug 09, 2019 at 04:07:00PM +, Roman Kagan via Groups.Io wrote:
>> On Thu, Aug 08, 2019 at 07:39:14PM +0200, Laszlo Ersek wrote:
>>> On 08/07/19 19:41, Andrew Fish wrote:
> On Aug 7, 2019, at 10:29 AM, Laszlo Ersek  wrote:
> On 08/05/19 12:18, Roman Kagan wrote:
>> On Sat, Aug 03, 2019 at 04:03:04AM +0200, Laszlo Ersek via Groups.Io 
>> wrote:
>>> On 08/01/19 21:16, Roman Kagan wrote:
> I'm convinced that OpenSSL needs to expose a new API for this particular
> problem.
>>
>> Since, as you point out below, the problem only affects the essentially
>> broken configuration (SECURE_BOOT_ENABLE && !SMM_REQUIRE), I'm fine with
>> saving time and effort and sticking to the hack-ish approach proposed in
>> the bugzilla issue, which is to iterate over "thread-local" pointers and
>> EfiConvertPointer() on each.  (As long as it fixes the problem of
>> course; I'll test and report back.)
> 
> It doesn't :(  It just gets slightly further and hits another static
> pointer variable which is not part of the thread-local array:
> 
> ...
>   Pkcs7Verify
> EVP_add_digest
>   OBJ_NAME_add
> 
> this one uses a few static pointer variables that are also initialized
> on demand and become stale upon SetVirtualAddressMap().

So it looks like the issue can't be solved without making OpenSSL aware
of this use case.

>> So we should be good without a new API from OpenSSL.
> 
>>> In other words, the problem doesn't exist when OpenSSL (with the rest of
>>> the variable driver stack) is protected with SMM, as pointers into SMRAM
>>> remain valid "forever", after the initial SMM driver dispatch.
>>
>> Makes perfect sense.  We happen to build this broken configuration for
>> some historical reasons, I'm failing to recall exactly which.  Will try
>> to get rid of it.
> 
> We appear to have some i440fx-based VMs with SecureBoot in the field
> (dunno about their origin) and those don't allow SMM.

Indeed i440fx has no support for SMM/SMRAM -- but that only means that
Secure Boot cannot be made actually secure on i440fx.

Red Hat used to offer an OVMF binary that ran on i440fx too, and
included the Secure Boot feature without SMM. But the package containing
that binary was always marked as Tech Preview. When we exited Tech
Preview with OVMF, we removed said binary.

Thanks
Laszlo

-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.

View/Reply Online (#45512): https://edk2.groups.io/g/devel/message/45512
Mute This Topic: https://groups.io/mt/32686575/21656
Group Owner: devel+ow...@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [edk2-devel] static data in dxe_runtime modules

[Roman Kagan]

On Fri, Aug 09, 2019 at 04:07:00PM +, Roman Kagan via Groups.Io wrote:
> On Thu, Aug 08, 2019 at 07:39:14PM +0200, Laszlo Ersek wrote:
> > On 08/07/19 19:41, Andrew Fish wrote:
> > >> On Aug 7, 2019, at 10:29 AM, Laszlo Ersek  wrote:
> > >> On 08/05/19 12:18, Roman Kagan wrote:
> > >>> On Sat, Aug 03, 2019 at 04:03:04AM +0200, Laszlo Ersek via Groups.Io 
> > >>> wrote:
> >  On 08/01/19 21:16, Roman Kagan wrote:
> > >> I'm convinced that OpenSSL needs to expose a new API for this particular
> > >> problem.
> 
> Since, as you point out below, the problem only affects the essentially
> broken configuration (SECURE_BOOT_ENABLE && !SMM_REQUIRE), I'm fine with
> saving time and effort and sticking to the hack-ish approach proposed in
> the bugzilla issue, which is to iterate over "thread-local" pointers and
> EfiConvertPointer() on each.  (As long as it fixes the problem of
> course; I'll test and report back.)

It doesn't :(  It just gets slightly further and hits another static
pointer variable which is not part of the thread-local array:

...
  Pkcs7Verify
EVP_add_digest
  OBJ_NAME_add

this one uses a few static pointer variables that are also initialized
on demand and become stale upon SetVirtualAddressMap().

> So we should be good without a new API from OpenSSL.

> > In other words, the problem doesn't exist when OpenSSL (with the rest of
> > the variable driver stack) is protected with SMM, as pointers into SMRAM
> > remain valid "forever", after the initial SMM driver dispatch.
> 
> Makes perfect sense.  We happen to build this broken configuration for
> some historical reasons, I'm failing to recall exactly which.  Will try
> to get rid of it.

We appear to have some i440fx-based VMs with SecureBoot in the field
(dunno about their origin) and those don't allow SMM.

Thanks,
Roman.

-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.

View/Reply Online (#45466): https://edk2.groups.io/g/devel/message/45466
Mute This Topic: https://groups.io/mt/32686575/21656
Group Owner: devel+ow...@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [edk2-devel] static data in dxe_runtime modules

[Roman Kagan]

On Thu, Aug 08, 2019 at 07:39:14PM +0200, Laszlo Ersek wrote:
> On 08/07/19 19:41, Andrew Fish wrote:
> >> On Aug 7, 2019, at 10:29 AM, Laszlo Ersek  wrote:
> >> ummm... not sure why, but I never got this email in my inbox. I only see
> >> it in my list folder. I see myself addressed on it as:
> >>
> >>  Laszlo Ersek via Groups.Io  >> >

I've adjusted my mail config to honor the 'reply-to:' header, should be
ok now.

> >> On 08/05/19 12:18, Roman Kagan wrote:
> >>> On Sat, Aug 03, 2019 at 04:03:04AM +0200, Laszlo Ersek via Groups.Io 
> >>> wrote:
>  On 08/01/19 21:16, Roman Kagan wrote:
> >> I'm convinced that OpenSSL needs to expose a new API for this particular
> >> problem.

Since, as you point out below, the problem only affects the essentially
broken configuration (SECURE_BOOT_ENABLE && !SMM_REQUIRE), I'm fine with
saving time and effort and sticking to the hack-ish approach proposed in
the bugzilla issue, which is to iterate over "thread-local" pointers and
EfiConvertPointer() on each.  (As long as it fixes the problem of
course; I'll test and report back.)  So we should be good without a new
API from OpenSSL.

> In other words, the problem doesn't exist when OpenSSL (with the rest of
> the variable driver stack) is protected with SMM, as pointers into SMRAM
> remain valid "forever", after the initial SMM driver dispatch.

Makes perfect sense.  We happen to build this broken configuration for
some historical reasons, I'm failing to recall exactly which.  Will try
to get rid of it.

Thanks,
Roman.

-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.

View/Reply Online (#45287): https://edk2.groups.io/g/devel/message/45287
Mute This Topic: https://groups.io/mt/32686575/21656
Group Owner: devel+ow...@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [edk2-devel] static data in dxe_runtime modules

[Laszlo Ersek]

On 08/07/19 19:41, Andrew Fish wrote:
> 
> 
>> On Aug 7, 2019, at 10:29 AM, Laszlo Ersek  wrote:
>>
>> ummm... not sure why, but I never got this email in my inbox. I only see
>> it in my list folder. I see myself addressed on it as:
>>
>>  Laszlo Ersek via Groups.Io > >
>>
>> which could be the reason.
>>
>> Anyway:
>>
>> On 08/05/19 12:18, Roman Kagan wrote:
>>> On Sat, Aug 03, 2019 at 04:03:04AM +0200, Laszlo Ersek via Groups.Io wrote:
 On 08/01/19 21:16, Roman Kagan wrote:
 This is a serious bug. Thank you for reporting and analyzing it. Can you
 file it in the TianoCore Bugzilla too, please?
>>>
>>> https://bugzilla.tianocore.org/show_bug.cgi?id=2053
>>>
 I wonder how far in OpenSSL history this issue goes back. Is this a
 regression from the latest OpenSSL rebase in edk2?
>>>
>>> This is certainly not a recent regression.  We've initially caught it
>>> with Virtuozzo OVMF based on the one from RHEL-7.6, based in turn on
>>> EDKII as of May 2018.  However, the infrastructure that causes the
>>> problem is there for much longer.
>>
>> OK. Thank you for confirming!
>>
> What would be the best way to fix it?
>
> One option is to audit OpenSSL and make sure it either doesn't put
> pointers into static storage or properly cleans them up (and call the
> cleanup function in RuntimeCryptLibAddressChangeEvent); then assume the
> rest of EDKII code is safe in this regard.
>
> Another is to assume that no static data in dxe_runtime modules should
> survive SetVirtualAddressMap, and thus make
> PeCoffLoaderRelocateImageForRuntime reinitialize the modules from
> scratch instead of re-applying the relocations only.
>
> I must admit I don't yet quite understand the full consequences of
> either option.  Perhaps there are better ones.
> Any suggestion is appreciated.

 If the runtime driver remembers pointer *values* from before
 SetVirtualAddressMap() -- i.e. it saves pointer values into some
 storage, for de-referencing at OS runtime --, those stored values have
 to be converted in the virtual address change event notification function.
>>> [...]
 The "thread_local_storage" array from
 "CryptoPkg/Library/OpensslLib/openssl/crypto/threads_none.c" has to be
 exposed to RuntimeCryptLibAddressChangeEvent() somehow.

 Perhaps OpenSSL should allow edk2 to bring its own CRYPTO_THREAD_* APIs.
>>>
>>> I think this would be too awkward, as edk2 has no reason to have any
>>> visibility into it.
>>
>> Edk2 already implements various sets of APIs that "plug" into OpenSSL.
>>
>> Not saying that it's optimal, but there is precedence.
>>
>>> I'd rather make use of the observation that in reality there's no data
>>> in OpenSSL that needs to survive SetVirtualAddressMap().  At first I
>>> started to cook up a fix that involved calling OPENSSL_cleanup() from
>>> RuntimeCryptLibAddressChangeEvent(), but it soon turned out it didn't
>>> clean things up to the pristine state so it didn't address the problem.
>>>
>>> Moreover I think it's overoptimistic to expect from OpenSSL developers
>>> the mindset that their code should work seamlessly across relocations at
>>> runtime.
>>
>> Well, they do have a UEFI system ID in "Configurations/10-main.conf".
>>
>> And I do think OpenSSL developers are interested in robustness over a
>> number of use cases. After all, the thread-specific key abstraction
>> exists for this kind of portability in the first place.
>>
>>> I don't see what would stop them from introducing another
>>> pointer variable with global storage further down the road, and nothing
>>> would allow to even timely spot this new problem.
>>
>> Edk2 could deal with this kind of problem a lot better if we timed our
>> OpenSSL submodule updates to the *start* of every edk2 development
>> cycle, not to the *end* of it.
>>
>>> That's why I think the most reliable solution would be to just reload
>>> the module completely.  If this can't be done for all runtime modules
>>> then I'd do it for the one(s) linking to OpenSSL.
>>
>> I don't think we should special-case how
>> RuntimeDriverSetVirtualAddressMap()
>> [MdeModulePkg/Core/RuntimeDxe/Runtime.c] works. UEFI (the specification)
>> already specifies a general facility for this problem; we should use it.
>>
>> I'm convinced that OpenSSL needs to expose a new API for this particular
>> problem.
>>
> 
> This feels right to me too. The reality is any pointer assigned in code needs 
> to get "fixed up" for the new virtual mapping. The other restriction is non 
> of the EFI Boot Services are available so you can't allocate memory etc.  The 
> code that runs to fixup pointers runs in the physical mapping, but has access 
> to an API that can convert a physical address to its future virtual mapping. 
> 
> If there is an API EFI can call to re-init passing in the old buffers that 
> may be the easiest thing to maintain. The EFI driver would 

Re: [edk2-devel] static data in dxe_runtime modules

[Andrew Fish via Groups.Io]

> On Aug 7, 2019, at 10:29 AM, Laszlo Ersek  wrote:
> 
> ummm... not sure why, but I never got this email in my inbox. I only see
> it in my list folder. I see myself addressed on it as:
> 
>  Laszlo Ersek via Groups.Io  >
> 
> which could be the reason.
> 
> Anyway:
> 
> On 08/05/19 12:18, Roman Kagan wrote:
>> On Sat, Aug 03, 2019 at 04:03:04AM +0200, Laszlo Ersek via Groups.Io wrote:
>>> On 08/01/19 21:16, Roman Kagan wrote:
>>> This is a serious bug. Thank you for reporting and analyzing it. Can you
>>> file it in the TianoCore Bugzilla too, please?
>> 
>> https://bugzilla.tianocore.org/show_bug.cgi?id=2053
>> 
>>> I wonder how far in OpenSSL history this issue goes back. Is this a
>>> regression from the latest OpenSSL rebase in edk2?
>> 
>> This is certainly not a recent regression.  We've initially caught it
>> with Virtuozzo OVMF based on the one from RHEL-7.6, based in turn on
>> EDKII as of May 2018.  However, the infrastructure that causes the
>> problem is there for much longer.
> 
> OK. Thank you for confirming!
> 
 What would be the best way to fix it?
 
 One option is to audit OpenSSL and make sure it either doesn't put
 pointers into static storage or properly cleans them up (and call the
 cleanup function in RuntimeCryptLibAddressChangeEvent); then assume the
 rest of EDKII code is safe in this regard.
 
 Another is to assume that no static data in dxe_runtime modules should
 survive SetVirtualAddressMap, and thus make
 PeCoffLoaderRelocateImageForRuntime reinitialize the modules from
 scratch instead of re-applying the relocations only.
 
 I must admit I don't yet quite understand the full consequences of
 either option.  Perhaps there are better ones.
 Any suggestion is appreciated.
>>> 
>>> If the runtime driver remembers pointer *values* from before
>>> SetVirtualAddressMap() -- i.e. it saves pointer values into some
>>> storage, for de-referencing at OS runtime --, those stored values have
>>> to be converted in the virtual address change event notification function.
>> [...]
>>> The "thread_local_storage" array from
>>> "CryptoPkg/Library/OpensslLib/openssl/crypto/threads_none.c" has to be
>>> exposed to RuntimeCryptLibAddressChangeEvent() somehow.
>>> 
>>> Perhaps OpenSSL should allow edk2 to bring its own CRYPTO_THREAD_* APIs.
>> 
>> I think this would be too awkward, as edk2 has no reason to have any
>> visibility into it.
> 
> Edk2 already implements various sets of APIs that "plug" into OpenSSL.
> 
> Not saying that it's optimal, but there is precedence.
> 
>> I'd rather make use of the observation that in reality there's no data
>> in OpenSSL that needs to survive SetVirtualAddressMap().  At first I
>> started to cook up a fix that involved calling OPENSSL_cleanup() from
>> RuntimeCryptLibAddressChangeEvent(), but it soon turned out it didn't
>> clean things up to the pristine state so it didn't address the problem.
>> 
>> Moreover I think it's overoptimistic to expect from OpenSSL developers
>> the mindset that their code should work seamlessly across relocations at
>> runtime.
> 
> Well, they do have a UEFI system ID in "Configurations/10-main.conf".
> 
> And I do think OpenSSL developers are interested in robustness over a
> number of use cases. After all, the thread-specific key abstraction
> exists for this kind of portability in the first place.
> 
>> I don't see what would stop them from introducing another
>> pointer variable with global storage further down the road, and nothing
>> would allow to even timely spot this new problem.
> 
> Edk2 could deal with this kind of problem a lot better if we timed our
> OpenSSL submodule updates to the *start* of every edk2 development
> cycle, not to the *end* of it.
> 
>> That's why I think the most reliable solution would be to just reload
>> the module completely.  If this can't be done for all runtime modules
>> then I'd do it for the one(s) linking to OpenSSL.
> 
> I don't think we should special-case how
> RuntimeDriverSetVirtualAddressMap()
> [MdeModulePkg/Core/RuntimeDxe/Runtime.c] works. UEFI (the specification)
> already specifies a general facility for this problem; we should use it.
> 
> I'm convinced that OpenSSL needs to expose a new API for this particular
> problem.
> 

This feels right to me too. The reality is any pointer assigned in code needs 
to get "fixed up" for the new virtual mapping. The other restriction is non of 
the EFI Boot Services are available so you can't allocate memory etc.  The code 
that runs to fixup pointers runs in the physical mapping, but has access to an 
API that can convert a physical address to its future virtual mapping. 

If there is an API EFI can call to re-init passing in the old buffers that may 
be the easiest thing to maintain. The EFI driver would then have a boolean flag 
and need to call the OpenSSL re-init on the 1st virtual call into the driver. 
This would let the OpenSSL 

Re: [edk2-devel] static data in dxe_runtime modules

[Laszlo Ersek]

ummm... not sure why, but I never got this email in my inbox. I only see
it in my list folder. I see myself addressed on it as:

  Laszlo Ersek via Groups.Io 

which could be the reason.

Anyway:

On 08/05/19 12:18, Roman Kagan wrote:
> On Sat, Aug 03, 2019 at 04:03:04AM +0200, Laszlo Ersek via Groups.Io wrote:
>> On 08/01/19 21:16, Roman Kagan wrote:
>> This is a serious bug. Thank you for reporting and analyzing it. Can you
>> file it in the TianoCore Bugzilla too, please?
> 
> https://bugzilla.tianocore.org/show_bug.cgi?id=2053
> 
>> I wonder how far in OpenSSL history this issue goes back. Is this a
>> regression from the latest OpenSSL rebase in edk2?
> 
> This is certainly not a recent regression.  We've initially caught it
> with Virtuozzo OVMF based on the one from RHEL-7.6, based in turn on
> EDKII as of May 2018.  However, the infrastructure that causes the
> problem is there for much longer.

OK. Thank you for confirming!

>>> What would be the best way to fix it?
>>>
>>> One option is to audit OpenSSL and make sure it either doesn't put
>>> pointers into static storage or properly cleans them up (and call the
>>> cleanup function in RuntimeCryptLibAddressChangeEvent); then assume the
>>> rest of EDKII code is safe in this regard.
>>>
>>> Another is to assume that no static data in dxe_runtime modules should
>>> survive SetVirtualAddressMap, and thus make
>>> PeCoffLoaderRelocateImageForRuntime reinitialize the modules from
>>> scratch instead of re-applying the relocations only.
>>>
>>> I must admit I don't yet quite understand the full consequences of
>>> either option.  Perhaps there are better ones.
>>> Any suggestion is appreciated.
>>
>> If the runtime driver remembers pointer *values* from before
>> SetVirtualAddressMap() -- i.e. it saves pointer values into some
>> storage, for de-referencing at OS runtime --, those stored values have
>> to be converted in the virtual address change event notification function.
> [...]
>> The "thread_local_storage" array from
>> "CryptoPkg/Library/OpensslLib/openssl/crypto/threads_none.c" has to be
>> exposed to RuntimeCryptLibAddressChangeEvent() somehow.
>>
>> Perhaps OpenSSL should allow edk2 to bring its own CRYPTO_THREAD_* APIs.
> 
> I think this would be too awkward, as edk2 has no reason to have any
> visibility into it.

Edk2 already implements various sets of APIs that "plug" into OpenSSL.

Not saying that it's optimal, but there is precedence.

> I'd rather make use of the observation that in reality there's no data
> in OpenSSL that needs to survive SetVirtualAddressMap().  At first I
> started to cook up a fix that involved calling OPENSSL_cleanup() from
> RuntimeCryptLibAddressChangeEvent(), but it soon turned out it didn't
> clean things up to the pristine state so it didn't address the problem.
> 
> Moreover I think it's overoptimistic to expect from OpenSSL developers
> the mindset that their code should work seamlessly across relocations at
> runtime.

Well, they do have a UEFI system ID in "Configurations/10-main.conf".

And I do think OpenSSL developers are interested in robustness over a
number of use cases. After all, the thread-specific key abstraction
exists for this kind of portability in the first place.

> I don't see what would stop them from introducing another
> pointer variable with global storage further down the road, and nothing
> would allow to even timely spot this new problem.

Edk2 could deal with this kind of problem a lot better if we timed our
OpenSSL submodule updates to the *start* of every edk2 development
cycle, not to the *end* of it.

> That's why I think the most reliable solution would be to just reload
> the module completely.  If this can't be done for all runtime modules
> then I'd do it for the one(s) linking to OpenSSL.

I don't think we should special-case how
RuntimeDriverSetVirtualAddressMap()
[MdeModulePkg/Core/RuntimeDxe/Runtime.c] works. UEFI (the specification)
already specifies a general facility for this problem; we should use it.

I'm convinced that OpenSSL needs to expose a new API for this particular
problem.

David, can you please offer some thoughts on this?

Thanks!
Laszlo

> 
> Roman.
> 
> 
> 


-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.

View/Reply Online (#45029): https://edk2.groups.io/g/devel/message/45029
Mute This Topic: https://groups.io/mt/32686575/21656
Group Owner: devel+ow...@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [edk2-devel] static data in dxe_runtime modules

[Roman Kagan]

On Sat, Aug 03, 2019 at 04:03:04AM +0200, Laszlo Ersek via Groups.Io wrote:
> On 08/01/19 21:16, Roman Kagan wrote:
> This is a serious bug. Thank you for reporting and analyzing it. Can you
> file it in the TianoCore Bugzilla too, please?

https://bugzilla.tianocore.org/show_bug.cgi?id=2053

> I wonder how far in OpenSSL history this issue goes back. Is this a
> regression from the latest OpenSSL rebase in edk2?

This is certainly not a recent regression.  We've initially caught it
with Virtuozzo OVMF based on the one from RHEL-7.6, based in turn on
EDKII as of May 2018.  However, the infrastructure that causes the
problem is there for much longer.

> > What would be the best way to fix it?
> > 
> > One option is to audit OpenSSL and make sure it either doesn't put
> > pointers into static storage or properly cleans them up (and call the
> > cleanup function in RuntimeCryptLibAddressChangeEvent); then assume the
> > rest of EDKII code is safe in this regard.
> > 
> > Another is to assume that no static data in dxe_runtime modules should
> > survive SetVirtualAddressMap, and thus make
> > PeCoffLoaderRelocateImageForRuntime reinitialize the modules from
> > scratch instead of re-applying the relocations only.
> > 
> > I must admit I don't yet quite understand the full consequences of
> > either option.  Perhaps there are better ones.
> > Any suggestion is appreciated.
> 
> If the runtime driver remembers pointer *values* from before
> SetVirtualAddressMap() -- i.e. it saves pointer values into some
> storage, for de-referencing at OS runtime --, those stored values have
> to be converted in the virtual address change event notification function.
[...]
> The "thread_local_storage" array from
> "CryptoPkg/Library/OpensslLib/openssl/crypto/threads_none.c" has to be
> exposed to RuntimeCryptLibAddressChangeEvent() somehow.
> 
> Perhaps OpenSSL should allow edk2 to bring its own CRYPTO_THREAD_* APIs.

I think this would be too awkward, as edk2 has no reason to have any
visibility into it.

I'd rather make use of the observation that in reality there's no data
in OpenSSL that needs to survive SetVirtualAddressMap().  At first I
started to cook up a fix that involved calling OPENSSL_cleanup() from
RuntimeCryptLibAddressChangeEvent(), but it soon turned out it didn't
clean things up to the pristine state so it didn't address the problem.

Moreover I think it's overoptimistic to expect from OpenSSL developers
the mindset that their code should work seamlessly across relocations at
runtime.  I don't see what would stop them from introducing another
pointer variable with global storage further down the road, and nothing
would allow to even timely spot this new problem.

That's why I think the most reliable solution would be to just reload
the module completely.  If this can't be done for all runtime modules
then I'd do it for the one(s) linking to OpenSSL.

Roman.

-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.

View/Reply Online (#44910): https://edk2.groups.io/g/devel/message/44910
Mute This Topic: https://groups.io/mt/32686575/21656
Group Owner: devel+ow...@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [edk2-devel] static data in dxe_runtime modules

[Laszlo Ersek]

On 08/01/19 21:16, Roman Kagan wrote:
> I'm trying to come up with a solution to the problem that OpenSSL
> internally uses static variables ("per-thread" in no-threads config) to
> store pointers, which remain unadjusted upon SetVirtualAddressMap and
> cause the guest OS crash.
> 
> More specifically, trying to set a signed variable leads to the
> following call chain:
> 
> VariableServiceSetVariable
>   ProcessVarWithPk
> VerifyTimeBasedPayloadAndUpdate
>   Pkcs7GetSigners
>   ASN1_item_d2i
> asn1_item_embed_d2i
>   asn1_template_ex_d2i
> asn1_template_noexp_d2i
>   asn1_item_embed_d2i
> asn1_template_ex_d2i
>   asn1_template_noexp_d2i
> asn1_item_embed_d2i
>   asn1_template_ex_d2i
> asn1_template_noexp_d2i
>   asn1_item_embed_d2i
> pubkey_cb
>   ERR_set_mark
> ERR_get_state
> 
> The latter performs one-time initialization if needed, and then returns
> a pointer maintained in a static array using CRYPTO_THREAD_get_local().
> If a signed variable is set before SetVirtualAddressMap and (another
> one) after it, the pointer is initialized while in the old mapping and
> dereferenced while in the new one, crashing the OS.
> 
> The real-world reproducer: install Windows Server 2016 in a VM with
> OVMF, shut it down, replace the varstore with a fresh template copy,
> boot Windows, try to update, say, "dbx" from within Windows => BSOD.
> The reason is that the Windows bootloader stores a secure variable
> "CurrentPolicy" if it isn't there, triggering the above callchain while
> still in identity mapping.
> (This problem isn't widely noticed because during the installation the
> bootloader stores CurrentPolicy, but the OS is soon rebooted, before it
> attempts to update dbx; upon that CurrentPolicy remains in the varstore
> and on further boots the bootloader skips setting it.)

This is a serious bug. Thank you for reporting and analyzing it. Can you
file it in the TianoCore Bugzilla too, please?

I think it's of general interest; virtualization is one way to reproduce
it, but it's not exclusive. Loss or wipign of UEFI variables can occur
on physical hardware too.

I wonder how far in OpenSSL history this issue goes back. Is this a
regression from the latest OpenSSL rebase in edk2?

> What would be the best way to fix it?
> 
> One option is to audit OpenSSL and make sure it either doesn't put
> pointers into static storage or properly cleans them up (and call the
> cleanup function in RuntimeCryptLibAddressChangeEvent); then assume the
> rest of EDKII code is safe in this regard.
> 
> Another is to assume that no static data in dxe_runtime modules should
> survive SetVirtualAddressMap, and thus make
> PeCoffLoaderRelocateImageForRuntime reinitialize the modules from
> scratch instead of re-applying the relocations only.
> 
> I must admit I don't yet quite understand the full consequences of
> either option.  Perhaps there are better ones.
> Any suggestion is appreciated.

If the runtime driver remembers pointer *values* from before
SetVirtualAddressMap() -- i.e. it saves pointer values into some
storage, for de-referencing at OS runtime --, those stored values have
to be converted in the virtual address change event notification function.

If your runtime driver says

  Ptr = 
  Ptr->Field = 1;

at runtime, you don't have to do anything. If your driver says

  mPtr = 

at boot time, and then at OS runtime it does

  mPtr->Field = 1;

then "mPtr" has to be converted, with EfiConvertPointer() in
"MdePkg/Library/UefiRuntimeLib/RuntimeLib.c".

PE/COFF massaging is unneeded.

The "thread_local_storage" array from
"CryptoPkg/Library/OpensslLib/openssl/crypto/threads_none.c" has to be
exposed to RuntimeCryptLibAddressChangeEvent() somehow.

Perhaps OpenSSL should allow edk2 to bring its own CRYPTO_THREAD_* APIs.

Thanks
Laszlo

-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.

View/Reply Online (#44878): https://edk2.groups.io/g/devel/message/44878
Mute This Topic: https://groups.io/mt/32686575/21656
Group Owner: devel+ow...@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-