Re: [edk2] StdLib usage for drivers?

[Michael Zimmermann]

> At that point, you will have all of the StdLIb functionality except for
command-line, console I/O and file system support.

all these could be added in later patches though.
stdin/stdout/stderr are really just redirecting writes to fd's 0,1,2 to the
default console device.

The Shell Protocol actually already allows passing arguments "raw" inside
the EFI arguments, the same could be done for drivers, so you can specify
some arguments when starting the binary via your boot manager.

file-system support isn't that complicated either if you just iterate all
block IO's and map them to drives C-Z which would be a windows-like
behavior. If unix style paths are required you could map them to sth. like
"/mnt/[GUID]/...".

Thanks
Michael

On Thu, Jul 7, 2016 at 7:58 AM, Daryl McDaniel <edk2-li...@mc2research.org>
wrote:

> Breaking the Shell file-system dependency is very easy, if you don’t need
> file system support from StdLib.  All you have to do is make sure that you
> don’t list DevMedia in your driver or application’s .inf file.  That way no
> file system support is included.
>
>
>
> A similar thing can be done to remove the interactive console support:
> remove DevConsole from the list of libraries in LibC.inf.
>
>
>
> StdLib/LibC/Main/Main.c could then be modified to remove the command-line
> argument support and standard I/O initialization (stdin, stdout, stderr).
> Once that is all done, you can change from ShellCEntryLib style entry point
> to a standard pure UEFI entry point.
>
>
>
> Then, go through all of the .inf files and remove ShellCEntryLib from the
> LibraryClass lists and I believe that you will have removed all Shell
> dependencies.
>
>
>
> At that point, you will have all of the StdLIb functionality except for
> command-line, console I/O and file system support.
>
>
>
> Those modifications will get a version of StdLIb that should be safe for
> use in drivers.  There will still be “excessive” run-time memory usage in
> some cases because of the MainData structure.
>
>
>
> Because of the (often unnecessary) interdependencies between functional
> groupings and sub-libraries within StdLib, you can end up with a lot more
> code built in than you might want.  This is sort of OK for applications,
> but not so good for drivers where you could possibly have several drivers
> using StdLib which would mean that you would have to fit several copies of
> the StdLib code within your available flash space.  (one copy of StdLib for
> each driver that uses it)
>
>
>
> Daryl McDaniel
>
>
>
> *From:* Michael Zimmermann [mailto:sigmaepsilo...@gmail.com]
> *Sent:* Wednesday, July 06, 2016 2:04 PM
> *To:* Tim Lewis <tim.le...@insyde.com>
> *Cc:* Carsey, Jaben <jaben.car...@intel.com>; af...@apple.com;
> edk2-devel@lists.01.org; Daryl McDaniel (edk2-li...@mc2research.org) <
> edk2-li...@mc2research.org>
>
> *Subject:* Re: [edk2] StdLib usage for drivers?
>
>
>
> The daShell problem is easy to solve since it's just a StdLib internal
> "driver" that's registered among others like stdout,stderr,stdin.
>
> We can just stop it from registering to StdLib if we don't have the Shell
> available.
>
>
>
> If we disable PcdShellLibAutoInitialize, we'd have to be very careful to
> find all usages of the ShellProtocol and add NULL pointer checks so StdLib
> won't crash.
>
>
>
> A good way to find them all is to make UefiShellLib.c an empty file and
> disable linker-GC.
>
>
>
> Thanks
>
> Michael
>
>
>
> On Wed, Jul 6, 2016 at 10:57 PM, Tim Lewis <tim.le...@insyde.com> wrote:
>
> Jaben --
>
> I can certainly recall instances where drivers that publish HII setup
> pages load things from the disk (such as when checking for specific file
> names).
>
> I also seem to recall that the original EDK shell library was designed to
> that volume names were simply ignored if the shell protocol was not
> present. This is essentially what the Shell APIs do, right? They just
> convert mapping names into device path snippets.
>
> Tim
>
>
> -Original Message-----
> From: edk2-devel [mailto:edk2-devel-boun...@lists.01.org] On Behalf Of
> Carsey, Jaben
> Sent: Wednesday, July 06, 2016 1:53 PM
> To: af...@apple.com
> Cc: Carsey, Jaben <jaben.car...@intel.com>; edk2-devel@lists.01.org;
> Michael Zimmermann <sigmaepsilo...@gmail.com>; Daryl McDaniel (
> edk2-li...@mc2research.org) <edk2-li...@mc2research.org>
> Subject: Re: [edk2] StdLib usage for drivers?
>
> The thing that I am trying to figure out is what LibC APIs is really
> needed by the driver?
>
> UEFI Drivers are not supposed to do things like FILE I/O, so w

Re: [edk2] StdLib usage for drivers?

[Daryl McDaniel]

Breaking the Shell file-system dependency is very easy, if you don’t need file 
system support from StdLib.  All you have to do is make sure that you don’t 
list DevMedia in your driver or application’s .inf file.  That way no file 
system support is included.

 

A similar thing can be done to remove the interactive console support: remove 
DevConsole from the list of libraries in LibC.inf.

 

StdLib/LibC/Main/Main.c could then be modified to remove the command-line 
argument support and standard I/O initialization (stdin, stdout, stderr).  Once 
that is all done, you can change from ShellCEntryLib style entry point to a 
standard pure UEFI entry point.

 

Then, go through all of the .inf files and remove ShellCEntryLib from the 
LibraryClass lists and I believe that you will have removed all Shell 
dependencies.

 

At that point, you will have all of the StdLIb functionality except for 
command-line, console I/O and file system support.

 

Those modifications will get a version of StdLIb that should be safe for use in 
drivers.  There will still be “excessive” run-time memory usage in some cases 
because of the MainData structure.

 

Because of the (often unnecessary) interdependencies between functional 
groupings and sub-libraries within StdLib, you can end up with a lot more code 
built in than you might want.  This is sort of OK for applications, but not so 
good for drivers where you could possibly have several drivers using StdLib 
which would mean that you would have to fit several copies of the StdLib code 
within your available flash space.  (one copy of StdLib for each driver that 
uses it)

 

Daryl McDaniel

 

From: Michael Zimmermann [mailto:sigmaepsilo...@gmail.com] 
Sent: Wednesday, July 06, 2016 2:04 PM
To: Tim Lewis <tim.le...@insyde.com>
Cc: Carsey, Jaben <jaben.car...@intel.com>; af...@apple.com; 
edk2-devel@lists.01.org; Daryl McDaniel (edk2-li...@mc2research.org) 
<edk2-li...@mc2research.org>
Subject: Re: [edk2] StdLib usage for drivers?

 

The daShell problem is easy to solve since it's just a StdLib internal "driver" 
that's registered among others like stdout,stderr,stdin.

We can just stop it from registering to StdLib if we don't have the Shell 
available.

 

If we disable PcdShellLibAutoInitialize, we'd have to be very careful to find 
all usages of the ShellProtocol and add NULL pointer checks so StdLib won't 
crash.

 

A good way to find them all is to make UefiShellLib.c an empty file and disable 
linker-GC.

 

Thanks

Michael

 

On Wed, Jul 6, 2016 at 10:57 PM, Tim Lewis <tim.le...@insyde.com 
<mailto:tim.le...@insyde.com> > wrote:

Jaben --

I can certainly recall instances where drivers that publish HII setup pages 
load things from the disk (such as when checking for specific file names).

I also seem to recall that the original EDK shell library was designed to that 
volume names were simply ignored if the shell protocol was not present. This is 
essentially what the Shell APIs do, right? They just convert mapping names into 
device path snippets.

Tim


-Original Message-
From: edk2-devel [mailto:edk2-devel-boun...@lists.01.org 
<mailto:edk2-devel-boun...@lists.01.org> ] On Behalf Of Carsey, Jaben
Sent: Wednesday, July 06, 2016 1:53 PM
To: af...@apple.com <mailto:af...@apple.com> 
Cc: Carsey, Jaben <jaben.car...@intel.com <mailto:jaben.car...@intel.com> >; 
edk2-devel@lists.01.org <mailto:edk2-devel@lists.01.org> ; Michael Zimmermann 
<sigmaepsilo...@gmail.com <mailto:sigmaepsilo...@gmail.com> >; Daryl McDaniel 
(edk2-li...@mc2research.org <mailto:edk2-li...@mc2research.org> ) 
<edk2-li...@mc2research.org <mailto:edk2-li...@mc2research.org> >
Subject: Re: [edk2] StdLib usage for drivers?

The thing that I am trying to figure out is what LibC APIs is really needed by 
the driver?

UEFI Drivers are not supposed to do things like FILE I/O, so who cares if they 
would fail if they were called…

Sidenote: I think that daShell.c is only used when you ask LibC for things that 
come from the shell…

This snippet makes standard APIs point to shell specific ones…
LibC/Uefi/Devices/UefiShell/daShell.c:814:  Stream->Abstraction.fo_close= 
_ShellClose;
LibC/Uefi/Devices/UefiShell/daShell.c:815:  Stream->Abstraction.fo_read = 
_ShellRead;
LibC/Uefi/Devices/UefiShell/daShell.c:816:  Stream->Abstraction.fo_write= 
_ShellWrite;
LibC/Uefi/Devices/UefiShell/daShell.c:818:  Stream->Abstraction.fo_poll = 
_ShellPoll;
LibC/Uefi/Devices/UefiShell/daShell.c:820:  Stream->Abstraction.fo_stat = 
_ShellStat;
LibC/Uefi/Devices/UefiShell/daShell.c:821:  Stream->Abstraction.fo_ioctl= 
_ShellIoctl;
LibC/Uefi/Devices/UefiShell/daShell.c:822:  Stream->Abstraction.fo_delete   = 
_ShellDelete;
LibC/Uefi/Devices/UefiShell/daShell.c:823:  Stream->Abstraction.fo_rmdir= 
_ShellRmdir;
LibC/Uefi/Devices/UefiShell/daShell.c:824

Re: [edk2] StdLib usage for drivers?

[Carsey, Jaben]

Sounds good.

So how about you contribute that coolness? 
:)


> -Original Message-
> From: edk2-devel [mailto:edk2-devel-boun...@lists.01.org] On Behalf Of
> Tim Lewis
> Sent: Wednesday, July 06, 2016 2:21 PM
> To: Carsey, Jaben <jaben.car...@intel.com>; af...@apple.com
> Cc: edk2-devel@lists.01.org; Michael Zimmermann
> <sigmaepsilo...@gmail.com>; Daryl McDaniel (edk2-li...@mc2research.org)
> <edk2-li...@mc2research.org>
> Subject: Re: [edk2] StdLib usage for drivers?
> Importance: High
> 
> Jaben, thanks.
> 
> I think we're on the same page as far as the pre-Shell limitations for paths.
> 
> The downside of using this strategy is the code size overhead for UEFI
> drivers.
> 
> BTW, we've extended the ShellLib to contain wrapper functions for nearly all
> Shell Protocol member functions. It makes smaller and cleaner shell
> applications.
> 
> Tim
> 
> -Original Message-
> From: Carsey, Jaben [mailto:jaben.car...@intel.com]
> Sent: Wednesday, July 06, 2016 2:12 PM
> To: Tim Lewis <tim.le...@insyde.com>; af...@apple.com
> Cc: edk2-devel@lists.01.org; Michael Zimmermann
> <sigmaepsilo...@gmail.com>; Daryl McDaniel (edk2-li...@mc2research.org)
> <edk2-li...@mc2research.org>; Carsey, Jaben <jaben.car...@intel.com>
> Subject: RE: [edk2] StdLib usage for drivers?
> 
> You are correct.
> 
> Device paths always work fine.  DevicePaths are what are in Boot and
> Driver variables.
> 
> The missing piece is the conversion from FSX to the starting node.
> 
> > -Original Message-
> > From: edk2-devel [mailto:edk2-devel-boun...@lists.01.org] On Behalf Of
> > Tim Lewis
> > Sent: Wednesday, July 06, 2016 2:09 PM
> > To: Carsey, Jaben <jaben.car...@intel.com>; af...@apple.com
> > Cc: edk2-devel@lists.01.org; Michael Zimmermann
> > <sigmaepsilo...@gmail.com>; Daryl McDaniel
> > (edk2-li...@mc2research.org) <edk2-li...@mc2research.org>
> > Subject: Re: [edk2] StdLib usage for drivers?
> > Importance: High
> >
> > Sure, the FSx: names are created by the shell. So before the shell
> > loads, there is no Shell protocol, so no mapping names. That means
> > that the StdLib implementation statically linked with a UEFI driver
> > will not find the Shell protocol and will not be able to match any
> > volume names (FSx or consistent mapping or any user-created mappings),
> > but the rest of the device path rules should apply. So, for example
> > \PciRoot(0)\Pci(0,31)\Usb(0,0)\Mbr()\BootX64.efi could work, right?
> > Those are the types of device paths that a UEFI driver might find in
> > the Boot variables and want to use.
> >
> > Tim
> >
> > -Original Message-
> > From: Carsey, Jaben [mailto:jaben.car...@intel.com]
> > Sent: Wednesday, July 06, 2016 2:02 PM
> > To: Tim Lewis <tim.le...@insyde.com>; af...@apple.com
> > Cc: edk2-devel@lists.01.org; Michael Zimmermann
> > <sigmaepsilo...@gmail.com>; Daryl McDaniel
> > (edk2-li...@mc2research.org) <edk2-li...@mc2research.org>; Carsey,
> > Jaben <jaben.car...@intel.com>
> > Subject: RE: [edk2] StdLib usage for drivers?
> >
> > Tim,
> >
> > Good catch.  I did not consider HII stuff.
> >
> > If you use a UEFI Shell Spec defined "consistent map name", it works
> > without the shell since that is a 1:1 conversion with DevicePath.
> >
> > If you use a Shell map name, then it can try, but there is no
> > guarantee that enumerations are right.  Using FS0: based mapping in a
> > driver is very scary to me.
> >
> > -Jaben
> >
> > > -Original Message-
> > > From: Tim Lewis [mailto:tim.le...@insyde.com]
> > > Sent: Wednesday, July 06, 2016 1:58 PM
> > > To: Carsey, Jaben <jaben.car...@intel.com>; af...@apple.com
> > > Cc: edk2-devel@lists.01.org; Michael Zimmermann
> > > <sigmaepsilo...@gmail.com>; Daryl McDaniel
> > > (edk2-li...@mc2research.org) <edk2-li...@mc2research.org>
> > > Subject: RE: [edk2] StdLib usage for drivers?
> > > Importance: High
> > >
> > > Jaben --
> > >
> > > I can certainly recall instances where drivers that publish HII
> > > setup pages load things from the disk (such as when checking for
> > > specific file
> > names).
> > >
> > > I also seem to recall that the original EDK shell library was
> > > designed to that volume names were simply ignored if the shell
> > > protocol was not present. This is essentially what the Shell APIs
> > &

Re: [edk2] StdLib usage for drivers?

[Tim Lewis]

Jaben, thanks. 

I think we're on the same page as far as the pre-Shell limitations for paths.

The downside of using this strategy is the code size overhead for UEFI drivers.

BTW, we've extended the ShellLib to contain wrapper functions for nearly all  
Shell Protocol member functions. It makes smaller and cleaner shell 
applications.

Tim

-Original Message-
From: Carsey, Jaben [mailto:jaben.car...@intel.com] 
Sent: Wednesday, July 06, 2016 2:12 PM
To: Tim Lewis <tim.le...@insyde.com>; af...@apple.com
Cc: edk2-devel@lists.01.org; Michael Zimmermann <sigmaepsilo...@gmail.com>; 
Daryl McDaniel (edk2-li...@mc2research.org) <edk2-li...@mc2research.org>; 
Carsey, Jaben <jaben.car...@intel.com>
Subject: RE: [edk2] StdLib usage for drivers?

You are correct.

Device paths always work fine.  DevicePaths are what are in Boot and 
Driver variables.

The missing piece is the conversion from FSX to the starting node.

> -Original Message-
> From: edk2-devel [mailto:edk2-devel-boun...@lists.01.org] On Behalf Of 
> Tim Lewis
> Sent: Wednesday, July 06, 2016 2:09 PM
> To: Carsey, Jaben <jaben.car...@intel.com>; af...@apple.com
> Cc: edk2-devel@lists.01.org; Michael Zimmermann 
> <sigmaepsilo...@gmail.com>; Daryl McDaniel 
> (edk2-li...@mc2research.org) <edk2-li...@mc2research.org>
> Subject: Re: [edk2] StdLib usage for drivers?
> Importance: High
> 
> Sure, the FSx: names are created by the shell. So before the shell 
> loads, there is no Shell protocol, so no mapping names. That means 
> that the StdLib implementation statically linked with a UEFI driver 
> will not find the Shell protocol and will not be able to match any 
> volume names (FSx or consistent mapping or any user-created mappings), 
> but the rest of the device path rules should apply. So, for example 
> \PciRoot(0)\Pci(0,31)\Usb(0,0)\Mbr()\BootX64.efi could work, right? 
> Those are the types of device paths that a UEFI driver might find in 
> the Boot variables and want to use.
> 
> Tim
> 
> -Original Message-
> From: Carsey, Jaben [mailto:jaben.car...@intel.com]
> Sent: Wednesday, July 06, 2016 2:02 PM
> To: Tim Lewis <tim.le...@insyde.com>; af...@apple.com
> Cc: edk2-devel@lists.01.org; Michael Zimmermann 
> <sigmaepsilo...@gmail.com>; Daryl McDaniel 
> (edk2-li...@mc2research.org) <edk2-li...@mc2research.org>; Carsey, 
> Jaben <jaben.car...@intel.com>
> Subject: RE: [edk2] StdLib usage for drivers?
> 
> Tim,
> 
> Good catch.  I did not consider HII stuff.
> 
> If you use a UEFI Shell Spec defined "consistent map name", it works 
> without the shell since that is a 1:1 conversion with DevicePath.
> 
> If you use a Shell map name, then it can try, but there is no 
> guarantee that enumerations are right.  Using FS0: based mapping in a 
> driver is very scary to me.
> 
> -Jaben
> 
> > -Original Message-
> > From: Tim Lewis [mailto:tim.le...@insyde.com]
> > Sent: Wednesday, July 06, 2016 1:58 PM
> > To: Carsey, Jaben <jaben.car...@intel.com>; af...@apple.com
> > Cc: edk2-devel@lists.01.org; Michael Zimmermann 
> > <sigmaepsilo...@gmail.com>; Daryl McDaniel
> > (edk2-li...@mc2research.org) <edk2-li...@mc2research.org>
> > Subject: RE: [edk2] StdLib usage for drivers?
> > Importance: High
> >
> > Jaben --
> >
> > I can certainly recall instances where drivers that publish HII 
> > setup pages load things from the disk (such as when checking for 
> > specific file
> names).
> >
> > I also seem to recall that the original EDK shell library was 
> > designed to that volume names were simply ignored if the shell 
> > protocol was not present. This is essentially what the Shell APIs 
> > do, right? They just convert mapping names into device path snippets.
> >
> > Tim
> >
> > -Original Message-
> > From: edk2-devel [mailto:edk2-devel-boun...@lists.01.org] On Behalf 
> > Of Carsey, Jaben
> > Sent: Wednesday, July 06, 2016 1:53 PM
> > To: af...@apple.com
> > Cc: Carsey, Jaben <jaben.car...@intel.com>; edk2-devel@lists.01.org; 
> > Michael Zimmermann <sigmaepsilo...@gmail.com>; Daryl McDaniel
> (edk2-
> > li...@mc2research.org) <edk2-li...@mc2research.org>
> > Subject: Re: [edk2] StdLib usage for drivers?
> >
> > The thing that I am trying to figure out is what LibC APIs is really 
> > needed by the driver?
> >
> > UEFI Drivers are not supposed to do things like FILE I/O, so who 
> > cares if they would fail if they were called…
> >
> > Sidenote: I think that daShell.c is only used when you ask LibC for 
> > things t

Re: [edk2] StdLib usage for drivers?

[Carsey, Jaben]

You are correct.

Device paths always work fine.  DevicePaths are what are in Boot and 
Driver variables.

The missing piece is the conversion from FSX to the starting node.

> -Original Message-
> From: edk2-devel [mailto:edk2-devel-boun...@lists.01.org] On Behalf Of
> Tim Lewis
> Sent: Wednesday, July 06, 2016 2:09 PM
> To: Carsey, Jaben <jaben.car...@intel.com>; af...@apple.com
> Cc: edk2-devel@lists.01.org; Michael Zimmermann
> <sigmaepsilo...@gmail.com>; Daryl McDaniel (edk2-li...@mc2research.org)
> <edk2-li...@mc2research.org>
> Subject: Re: [edk2] StdLib usage for drivers?
> Importance: High
> 
> Sure, the FSx: names are created by the shell. So before the shell loads,
> there is no Shell protocol, so no mapping names. That means that the StdLib
> implementation statically linked with a UEFI driver will not find the Shell
> protocol and will not be able to match any volume names (FSx or consistent
> mapping or any user-created mappings), but the rest of the device path rules
> should apply. So, for example
> \PciRoot(0)\Pci(0,31)\Usb(0,0)\Mbr()\BootX64.efi could work, right? Those
> are the types of device paths that a UEFI driver might find in the Boot
> variables and want to use.
> 
> Tim
> 
> -Original Message-
> From: Carsey, Jaben [mailto:jaben.car...@intel.com]
> Sent: Wednesday, July 06, 2016 2:02 PM
> To: Tim Lewis <tim.le...@insyde.com>; af...@apple.com
> Cc: edk2-devel@lists.01.org; Michael Zimmermann
> <sigmaepsilo...@gmail.com>; Daryl McDaniel (edk2-li...@mc2research.org)
> <edk2-li...@mc2research.org>; Carsey, Jaben <jaben.car...@intel.com>
> Subject: RE: [edk2] StdLib usage for drivers?
> 
> Tim,
> 
> Good catch.  I did not consider HII stuff.
> 
> If you use a UEFI Shell Spec defined "consistent map name", it works without
> the shell since that is a 1:1 conversion with DevicePath.
> 
> If you use a Shell map name, then it can try, but there is no guarantee that
> enumerations are right.  Using FS0: based mapping in a driver is very scary to
> me.
> 
> -Jaben
> 
> > -Original Message-
> > From: Tim Lewis [mailto:tim.le...@insyde.com]
> > Sent: Wednesday, July 06, 2016 1:58 PM
> > To: Carsey, Jaben <jaben.car...@intel.com>; af...@apple.com
> > Cc: edk2-devel@lists.01.org; Michael Zimmermann
> > <sigmaepsilo...@gmail.com>; Daryl McDaniel
> > (edk2-li...@mc2research.org) <edk2-li...@mc2research.org>
> > Subject: RE: [edk2] StdLib usage for drivers?
> > Importance: High
> >
> > Jaben --
> >
> > I can certainly recall instances where drivers that publish HII setup
> > pages load things from the disk (such as when checking for specific file
> names).
> >
> > I also seem to recall that the original EDK shell library was designed
> > to that volume names were simply ignored if the shell protocol was not
> > present. This is essentially what the Shell APIs do, right? They just
> > convert mapping names into device path snippets.
> >
> > Tim
> >
> > -Original Message-
> > From: edk2-devel [mailto:edk2-devel-boun...@lists.01.org] On Behalf Of
> > Carsey, Jaben
> > Sent: Wednesday, July 06, 2016 1:53 PM
> > To: af...@apple.com
> > Cc: Carsey, Jaben <jaben.car...@intel.com>; edk2-devel@lists.01.org;
> > Michael Zimmermann <sigmaepsilo...@gmail.com>; Daryl McDaniel
> (edk2-
> > li...@mc2research.org) <edk2-li...@mc2research.org>
> > Subject: Re: [edk2] StdLib usage for drivers?
> >
> > The thing that I am trying to figure out is what LibC APIs is really
> > needed by the driver?
> >
> > UEFI Drivers are not supposed to do things like FILE I/O, so who cares
> > if they would fail if they were called…
> >
> > Sidenote: I think that daShell.c is only used when you ask LibC for
> > things that come from the shell…
> >
> > This snippet makes standard APIs point to shell specific ones…
> > LibC/Uefi/Devices/UefiShell/daShell.c:814:  Stream->Abstraction.fo_close
> =
> > _ShellClose;
> > LibC/Uefi/Devices/UefiShell/daShell.c:815:  Stream->Abstraction.fo_read
> =
> > _ShellRead;
> > LibC/Uefi/Devices/UefiShell/daShell.c:816:  Stream->Abstraction.fo_write
> =
> > _ShellWrite;
> > LibC/Uefi/Devices/UefiShell/daShell.c:818:  Stream->Abstraction.fo_poll
> =
> > _ShellPoll;
> > LibC/Uefi/Devices/UefiShell/daShell.c:820:  Stream->Abstraction.fo_stat
> =
> > _ShellStat;
> > LibC/Uefi/Devices/UefiShell/daShell.c:821:  Stream->Abstraction.fo_ioctl
> =
> > _ShellIoctl;
> > LibC/Uefi/Device

Re: [edk2] StdLib usage for drivers?

[Tim Lewis]

Sure, the FSx: names are created by the shell. So before the shell loads, there 
is no Shell protocol, so no mapping names. That means that the StdLib 
implementation statically linked with a UEFI driver will not find the Shell 
protocol and will not be able to match any volume names (FSx or consistent 
mapping or any user-created mappings), but the rest of the device path rules 
should apply. So, for example \PciRoot(0)\Pci(0,31)\Usb(0,0)\Mbr()\BootX64.efi 
could work, right? Those are the types of device paths that a UEFI driver might 
find in the Boot variables and want to use.

Tim

-Original Message-
From: Carsey, Jaben [mailto:jaben.car...@intel.com] 
Sent: Wednesday, July 06, 2016 2:02 PM
To: Tim Lewis <tim.le...@insyde.com>; af...@apple.com
Cc: edk2-devel@lists.01.org; Michael Zimmermann <sigmaepsilo...@gmail.com>; 
Daryl McDaniel (edk2-li...@mc2research.org) <edk2-li...@mc2research.org>; 
Carsey, Jaben <jaben.car...@intel.com>
Subject: RE: [edk2] StdLib usage for drivers?

Tim,

Good catch.  I did not consider HII stuff.

If you use a UEFI Shell Spec defined "consistent map name", it works without 
the shell since that is a 1:1 conversion with DevicePath.  

If you use a Shell map name, then it can try, but there is no guarantee that 
enumerations are right.  Using FS0: based mapping in a driver is very scary to 
me.

-Jaben

> -Original Message-
> From: Tim Lewis [mailto:tim.le...@insyde.com]
> Sent: Wednesday, July 06, 2016 1:58 PM
> To: Carsey, Jaben <jaben.car...@intel.com>; af...@apple.com
> Cc: edk2-devel@lists.01.org; Michael Zimmermann 
> <sigmaepsilo...@gmail.com>; Daryl McDaniel 
> (edk2-li...@mc2research.org) <edk2-li...@mc2research.org>
> Subject: RE: [edk2] StdLib usage for drivers?
> Importance: High
> 
> Jaben --
> 
> I can certainly recall instances where drivers that publish HII setup 
> pages load things from the disk (such as when checking for specific file 
> names).
> 
> I also seem to recall that the original EDK shell library was designed 
> to that volume names were simply ignored if the shell protocol was not 
> present. This is essentially what the Shell APIs do, right? They just 
> convert mapping names into device path snippets.
> 
> Tim
> 
> -Original Message-
> From: edk2-devel [mailto:edk2-devel-boun...@lists.01.org] On Behalf Of 
> Carsey, Jaben
> Sent: Wednesday, July 06, 2016 1:53 PM
> To: af...@apple.com
> Cc: Carsey, Jaben <jaben.car...@intel.com>; edk2-devel@lists.01.org; 
> Michael Zimmermann <sigmaepsilo...@gmail.com>; Daryl McDaniel (edk2-
> li...@mc2research.org) <edk2-li...@mc2research.org>
> Subject: Re: [edk2] StdLib usage for drivers?
> 
> The thing that I am trying to figure out is what LibC APIs is really 
> needed by the driver?
> 
> UEFI Drivers are not supposed to do things like FILE I/O, so who cares 
> if they would fail if they were called…
> 
> Sidenote: I think that daShell.c is only used when you ask LibC for 
> things that come from the shell…
> 
> This snippet makes standard APIs point to shell specific ones…
> LibC/Uefi/Devices/UefiShell/daShell.c:814:  Stream->Abstraction.fo_close=
> _ShellClose;
> LibC/Uefi/Devices/UefiShell/daShell.c:815:  Stream->Abstraction.fo_read =
> _ShellRead;
> LibC/Uefi/Devices/UefiShell/daShell.c:816:  Stream->Abstraction.fo_write=
> _ShellWrite;
> LibC/Uefi/Devices/UefiShell/daShell.c:818:  Stream->Abstraction.fo_poll =
> _ShellPoll;
> LibC/Uefi/Devices/UefiShell/daShell.c:820:  Stream->Abstraction.fo_stat =
> _ShellStat;
> LibC/Uefi/Devices/UefiShell/daShell.c:821:  Stream->Abstraction.fo_ioctl=
> _ShellIoctl;
> LibC/Uefi/Devices/UefiShell/daShell.c:822:  
> Stream->Abstraction.fo_delete = _ShellDelete;
> LibC/Uefi/Devices/UefiShell/daShell.c:823:  
> Stream->Abstraction.fo_rmdir = _ShellRmdir;
> LibC/Uefi/Devices/UefiShell/daShell.c:824:  
> Stream->Abstraction.fo_mkdir = _ShellMkdir;
> LibC/Uefi/Devices/UefiShell/daShell.c:825:  
> Stream->Abstraction.fo_rename = _ShellRename;
> LibC/Uefi/Devices/UefiShell/daShell.c:826:  Stream->Abstraction.fo_lseek=
> _ShellSeek;
> 
> 
> From: af...@apple.com [mailto:af...@apple.com]
> Sent: Wednesday, July 06, 2016 1:48 PM
> To: Carsey, Jaben <jaben.car...@intel.com>
> Cc: Michael Zimmermann <sigmaepsilo...@gmail.com>; Marvin Häuser 
> <marvin.haeu...@outlook.com>; edk2-devel@lists.01.org; edk2- 
> li...@mc2research.org
> Subject: Re: [edk2] StdLib usage for drivers?
> Importance: High
> 
> 
> On Jul 6, 2016, at 1:39 PM, Carsey, Jaben 
> <jaben.car...@intel.com<mailto:jaben.car...@intel.com>> wrote:
> 
> It works for libs.  If you 

Re: [edk2] StdLib usage for drivers?

[Michael Zimmermann]

The daShell problem is easy to solve since it's just a StdLib internal
"driver" that's registered among others like stdout,stderr,stdin.
We can just stop it from registering to StdLib if we don't have the Shell
available.

If we disable PcdShellLibAutoInitialize, we'd have to be very careful to
find all usages of the ShellProtocol and add NULL pointer checks so StdLib
won't crash.

A good way to find them all is to make UefiShellLib.c an empty file and
disable linker-GC.

Thanks
Michael

On Wed, Jul 6, 2016 at 10:57 PM, Tim Lewis <tim.le...@insyde.com> wrote:

> Jaben --
>
> I can certainly recall instances where drivers that publish HII setup
> pages load things from the disk (such as when checking for specific file
> names).
>
> I also seem to recall that the original EDK shell library was designed to
> that volume names were simply ignored if the shell protocol was not
> present. This is essentially what the Shell APIs do, right? They just
> convert mapping names into device path snippets.
>
> Tim
>
> -Original Message-
> From: edk2-devel [mailto:edk2-devel-boun...@lists.01.org] On Behalf Of
> Carsey, Jaben
> Sent: Wednesday, July 06, 2016 1:53 PM
> To: af...@apple.com
> Cc: Carsey, Jaben <jaben.car...@intel.com>; edk2-devel@lists.01.org;
> Michael Zimmermann <sigmaepsilo...@gmail.com>; Daryl McDaniel (
> edk2-li...@mc2research.org) <edk2-li...@mc2research.org>
> Subject: Re: [edk2] StdLib usage for drivers?
>
> The thing that I am trying to figure out is what LibC APIs is really
> needed by the driver?
>
> UEFI Drivers are not supposed to do things like FILE I/O, so who cares if
> they would fail if they were called…
>
> Sidenote: I think that daShell.c is only used when you ask LibC for things
> that come from the shell…
>
> This snippet makes standard APIs point to shell specific ones…
> LibC/Uefi/Devices/UefiShell/daShell.c:814:  Stream->Abstraction.fo_close
>   = _ShellClose;
> LibC/Uefi/Devices/UefiShell/daShell.c:815:  Stream->Abstraction.fo_read
>  = _ShellRead;
> LibC/Uefi/Devices/UefiShell/daShell.c:816:  Stream->Abstraction.fo_write
>   = _ShellWrite;
> LibC/Uefi/Devices/UefiShell/daShell.c:818:  Stream->Abstraction.fo_poll
>  = _ShellPoll;
> LibC/Uefi/Devices/UefiShell/daShell.c:820:  Stream->Abstraction.fo_stat
>  = _ShellStat;
> LibC/Uefi/Devices/UefiShell/daShell.c:821:  Stream->Abstraction.fo_ioctl
>   = _ShellIoctl;
> LibC/Uefi/Devices/UefiShell/daShell.c:822:  Stream->Abstraction.fo_delete
>  = _ShellDelete;
> LibC/Uefi/Devices/UefiShell/daShell.c:823:  Stream->Abstraction.fo_rmdir
>   = _ShellRmdir;
> LibC/Uefi/Devices/UefiShell/daShell.c:824:  Stream->Abstraction.fo_mkdir
>   = _ShellMkdir;
> LibC/Uefi/Devices/UefiShell/daShell.c:825:  Stream->Abstraction.fo_rename
>  = _ShellRename;
> LibC/Uefi/Devices/UefiShell/daShell.c:826:  Stream->Abstraction.fo_lseek
>   = _ShellSeek;
>
>
> From: af...@apple.com [mailto:af...@apple.com]
> Sent: Wednesday, July 06, 2016 1:48 PM
> To: Carsey, Jaben <jaben.car...@intel.com>
> Cc: Michael Zimmermann <sigmaepsilo...@gmail.com>; Marvin Häuser <
> marvin.haeu...@outlook.com>; edk2-devel@lists.01.org;
> edk2-li...@mc2research.org
> Subject: Re: [edk2] StdLib usage for drivers?
> Importance: High
>
>
> On Jul 6, 2016, at 1:39 PM, Carsey, Jaben <jaben.car...@intel.com jaben.car...@intel.com>> wrote:
>
> It works for libs.  If you specify a specific LibraryClass implementation,
> it will not add a second of that library class.
>
> Side note: what if you turn off AutoInitialize PCD for the shell lib?
> That would get rid of your main issue (the constructor ASSERT)…  then we
> can see what other issues you have.
>
>
> This seems like the set of library APIs that would be needed (and one
> usage of  gEfiShellProtocol->SetCurDir()).
>
> ~/work/src/edk2/StdLib(master)>git grep Shell -- *.c
> LibC/Main/Main.c:20:#include  
> LibC/Main/Main.c:130:ShellAppMain ( LibC/StdLib/Environs.c:23:#include
> 
> LibC/StdLib/Environs.c:155:  OpStat = ShellExecute( ,
> gMD->UString, FALSE, NULL, );
> LibC/StdLib/Environs.c:167:are determined by the underlying UEFI Shell
> implementation.
> LibC/StdLib/Environs.c:181:  EfiEnv =
> ShellGetEnvironmentVariable(gMD->UString);
> LibC/StdLib/Environs.c:248:HostName = ShellGetEnvironmentVariable (
> UName );
> LibC/StdLib/Environs.c:254:  Status = ShellSetEnvironmentVariable (
> UName, UValue, TRUE );
> LibC/StdLib/Environs.c:265:Status = ShellSetEnvironmentVariable (
> UName, UValue, FALSE );
> LibC/Uefi/Devices/UefiShell/daShell.c:2:  Abstract device driver for the
> UEFI Shell-hosted environment.
> Li

Re: [edk2] StdLib usage for drivers?

[Carsey, Jaben]

Tim,

Good catch.  I did not consider HII stuff.

If you use a UEFI Shell Spec defined "consistent map name", it works without 
the shell since that is a 1:1 conversion with DevicePath.  

If you use a Shell map name, then it can try, but there is no guarantee that 
enumerations are right.  Using FS0: based mapping in a driver is very scary to 
me.

-Jaben

> -Original Message-
> From: Tim Lewis [mailto:tim.le...@insyde.com]
> Sent: Wednesday, July 06, 2016 1:58 PM
> To: Carsey, Jaben <jaben.car...@intel.com>; af...@apple.com
> Cc: edk2-devel@lists.01.org; Michael Zimmermann
> <sigmaepsilo...@gmail.com>; Daryl McDaniel (edk2-li...@mc2research.org)
> <edk2-li...@mc2research.org>
> Subject: RE: [edk2] StdLib usage for drivers?
> Importance: High
> 
> Jaben --
> 
> I can certainly recall instances where drivers that publish HII setup pages 
> load
> things from the disk (such as when checking for specific file names).
> 
> I also seem to recall that the original EDK shell library was designed to that
> volume names were simply ignored if the shell protocol was not present. This
> is essentially what the Shell APIs do, right? They just convert mapping names
> into device path snippets.
> 
> Tim
> 
> -Original Message-
> From: edk2-devel [mailto:edk2-devel-boun...@lists.01.org] On Behalf Of
> Carsey, Jaben
> Sent: Wednesday, July 06, 2016 1:53 PM
> To: af...@apple.com
> Cc: Carsey, Jaben <jaben.car...@intel.com>; edk2-devel@lists.01.org;
> Michael Zimmermann <sigmaepsilo...@gmail.com>; Daryl McDaniel (edk2-
> li...@mc2research.org) <edk2-li...@mc2research.org>
> Subject: Re: [edk2] StdLib usage for drivers?
> 
> The thing that I am trying to figure out is what LibC APIs is really needed by
> the driver?
> 
> UEFI Drivers are not supposed to do things like FILE I/O, so who cares if they
> would fail if they were called…
> 
> Sidenote: I think that daShell.c is only used when you ask LibC for things 
> that
> come from the shell…
> 
> This snippet makes standard APIs point to shell specific ones…
> LibC/Uefi/Devices/UefiShell/daShell.c:814:  Stream->Abstraction.fo_close=
> _ShellClose;
> LibC/Uefi/Devices/UefiShell/daShell.c:815:  Stream->Abstraction.fo_read =
> _ShellRead;
> LibC/Uefi/Devices/UefiShell/daShell.c:816:  Stream->Abstraction.fo_write=
> _ShellWrite;
> LibC/Uefi/Devices/UefiShell/daShell.c:818:  Stream->Abstraction.fo_poll =
> _ShellPoll;
> LibC/Uefi/Devices/UefiShell/daShell.c:820:  Stream->Abstraction.fo_stat =
> _ShellStat;
> LibC/Uefi/Devices/UefiShell/daShell.c:821:  Stream->Abstraction.fo_ioctl=
> _ShellIoctl;
> LibC/Uefi/Devices/UefiShell/daShell.c:822:  Stream->Abstraction.fo_delete
> = _ShellDelete;
> LibC/Uefi/Devices/UefiShell/daShell.c:823:  Stream->Abstraction.fo_rmdir
> = _ShellRmdir;
> LibC/Uefi/Devices/UefiShell/daShell.c:824:  Stream->Abstraction.fo_mkdir
> = _ShellMkdir;
> LibC/Uefi/Devices/UefiShell/daShell.c:825:  Stream->Abstraction.fo_rename
> = _ShellRename;
> LibC/Uefi/Devices/UefiShell/daShell.c:826:  Stream->Abstraction.fo_lseek=
> _ShellSeek;
> 
> 
> From: af...@apple.com [mailto:af...@apple.com]
> Sent: Wednesday, July 06, 2016 1:48 PM
> To: Carsey, Jaben <jaben.car...@intel.com>
> Cc: Michael Zimmermann <sigmaepsilo...@gmail.com>; Marvin Häuser
> <marvin.haeu...@outlook.com>; edk2-devel@lists.01.org; edk2-
> li...@mc2research.org
> Subject: Re: [edk2] StdLib usage for drivers?
> Importance: High
> 
> 
> On Jul 6, 2016, at 1:39 PM, Carsey, Jaben
> <jaben.car...@intel.com<mailto:jaben.car...@intel.com>> wrote:
> 
> It works for libs.  If you specify a specific LibraryClass implementation, it 
> will
> not add a second of that library class.
> 
> Side note: what if you turn off AutoInitialize PCD for the shell lib?  That 
> would
> get rid of your main issue (the constructor ASSERT)…  then we can see what
> other issues you have.
> 
> 
> This seems like the set of library APIs that would be needed (and one usage
> of  gEfiShellProtocol->SetCurDir()).
> 
> ~/work/src/edk2/StdLib(master)>git grep Shell -- *.c
> LibC/Main/Main.c:20:#include  
> LibC/Main/Main.c:130:ShellAppMain ( LibC/StdLib/Environs.c:23:#include
> 
> LibC/StdLib/Environs.c:155:  OpStat = ShellExecute( , gMD-
> >UString, FALSE, NULL, );
> LibC/StdLib/Environs.c:167:are determined by the underlying UEFI Shell
> implementation.
> LibC/StdLib/Environs.c:181:  EfiEnv = ShellGetEnvironmentVariable(gMD-
> >UString);
> LibC/StdLib/Environs.c:248:HostName = ShellGetEnvironmentVariable (
> UName );
> LibC/StdLib/Environs.c:25

Re: [edk2] StdLib usage for drivers?

[Tim Lewis]

Jaben --

I can certainly recall instances where drivers that publish HII setup pages 
load things from the disk (such as when checking for specific file names).

I also seem to recall that the original EDK shell library was designed to that 
volume names were simply ignored if the shell protocol was not present. This is 
essentially what the Shell APIs do, right? They just convert mapping names into 
device path snippets. 

Tim

-Original Message-
From: edk2-devel [mailto:edk2-devel-boun...@lists.01.org] On Behalf Of Carsey, 
Jaben
Sent: Wednesday, July 06, 2016 1:53 PM
To: af...@apple.com
Cc: Carsey, Jaben <jaben.car...@intel.com>; edk2-devel@lists.01.org; Michael 
Zimmermann <sigmaepsilo...@gmail.com>; Daryl McDaniel 
(edk2-li...@mc2research.org) <edk2-li...@mc2research.org>
Subject: Re: [edk2] StdLib usage for drivers?

The thing that I am trying to figure out is what LibC APIs is really needed by 
the driver?

UEFI Drivers are not supposed to do things like FILE I/O, so who cares if they 
would fail if they were called…

Sidenote: I think that daShell.c is only used when you ask LibC for things that 
come from the shell…

This snippet makes standard APIs point to shell specific ones…
LibC/Uefi/Devices/UefiShell/daShell.c:814:  Stream->Abstraction.fo_close= 
_ShellClose;
LibC/Uefi/Devices/UefiShell/daShell.c:815:  Stream->Abstraction.fo_read = 
_ShellRead;
LibC/Uefi/Devices/UefiShell/daShell.c:816:  Stream->Abstraction.fo_write= 
_ShellWrite;
LibC/Uefi/Devices/UefiShell/daShell.c:818:  Stream->Abstraction.fo_poll = 
_ShellPoll;
LibC/Uefi/Devices/UefiShell/daShell.c:820:  Stream->Abstraction.fo_stat = 
_ShellStat;
LibC/Uefi/Devices/UefiShell/daShell.c:821:  Stream->Abstraction.fo_ioctl= 
_ShellIoctl;
LibC/Uefi/Devices/UefiShell/daShell.c:822:  Stream->Abstraction.fo_delete   = 
_ShellDelete;
LibC/Uefi/Devices/UefiShell/daShell.c:823:  Stream->Abstraction.fo_rmdir= 
_ShellRmdir;
LibC/Uefi/Devices/UefiShell/daShell.c:824:  Stream->Abstraction.fo_mkdir= 
_ShellMkdir;
LibC/Uefi/Devices/UefiShell/daShell.c:825:  Stream->Abstraction.fo_rename   = 
_ShellRename;
LibC/Uefi/Devices/UefiShell/daShell.c:826:  Stream->Abstraction.fo_lseek= 
_ShellSeek;


From: af...@apple.com [mailto:af...@apple.com]
Sent: Wednesday, July 06, 2016 1:48 PM
To: Carsey, Jaben <jaben.car...@intel.com>
Cc: Michael Zimmermann <sigmaepsilo...@gmail.com>; Marvin Häuser 
<marvin.haeu...@outlook.com>; edk2-devel@lists.01.org; 
edk2-li...@mc2research.org
Subject: Re: [edk2] StdLib usage for drivers?
Importance: High


On Jul 6, 2016, at 1:39 PM, Carsey, Jaben 
<jaben.car...@intel.com<mailto:jaben.car...@intel.com>> wrote:

It works for libs.  If you specify a specific LibraryClass implementation, it 
will not add a second of that library class.

Side note: what if you turn off AutoInitialize PCD for the shell lib?  That 
would get rid of your main issue (the constructor ASSERT)…  then we can see 
what other issues you have.


This seems like the set of library APIs that would be needed (and one usage of  
gEfiShellProtocol->SetCurDir()).

~/work/src/edk2/StdLib(master)>git grep Shell -- *.c 
LibC/Main/Main.c:20:#include   
LibC/Main/Main.c:130:ShellAppMain ( LibC/StdLib/Environs.c:23:#include  

LibC/StdLib/Environs.c:155:  OpStat = ShellExecute( , gMD->UString, 
FALSE, NULL, );
LibC/StdLib/Environs.c:167:are determined by the underlying UEFI Shell 
implementation.
LibC/StdLib/Environs.c:181:  EfiEnv = ShellGetEnvironmentVariable(gMD->UString);
LibC/StdLib/Environs.c:248:HostName = ShellGetEnvironmentVariable ( UName );
LibC/StdLib/Environs.c:254:  Status = ShellSetEnvironmentVariable ( UName, 
UValue, TRUE );
LibC/StdLib/Environs.c:265:Status = ShellSetEnvironmentVariable ( 
UName, UValue, FALSE );
LibC/Uefi/Devices/UefiShell/daShell.c:2:  Abstract device driver for the UEFI 
Shell-hosted environment.
LibC/Uefi/Devices/UefiShell/daShell.c:4:  In a Shell-hosted environment, this 
is the driver that is called LibC/Uefi/Devices/UefiShell/daShell.c:21:#include  

LibC/Uefi/Devices/UefiShell/daShell.c:40:/** EFI Shell specific operations for 
close().
LibC/Uefi/Devices/UefiShell/daShell.c:50:da_ShellClose(
LibC/Uefi/Devices/UefiShell/daShell.c:54:  EFIerrno = ShellCloseFile( 
(SHELL_FILE_HANDLE *)>devdata);
LibC/Uefi/Devices/UefiShell/daShell.c:61:/** EFI Shell specific operations for 
deleting a file or directory.
LibC/Uefi/Devices/UefiShell/daShell.c:71:da_ShellDelete(
LibC/Uefi/Devices/UefiShell/daShell.c:77:  Status = ShellDeleteFile( 
(SHELL_FILE_HANDLE *)>devdata);
LibC/Uefi/Devices/UefiShell/daShell.c:86:/** EFI Shell specific operations for 
setting the position within a file.
LibC/Uefi/Devices/UefiShell/daShell.c:97:da_ShellSeek(
LibC/Uefi/Devices/UefiShell/daShell.c:113:  Status = ShellSetFilePosition( 
FileHandle, 0xULL);
LibC/Uefi/Devices/UefiShell/daShe

Re: [edk2] StdLib usage for drivers?

[Carsey, Jaben]

The thing that I am trying to figure out is what LibC APIs is really needed by 
the driver?

UEFI Drivers are not supposed to do things like FILE I/O, so who cares if they 
would fail if they were called…

Sidenote: I think that daShell.c is only used when you ask LibC for things that 
come from the shell…

This snippet makes standard APIs point to shell specific ones…
LibC/Uefi/Devices/UefiShell/daShell.c:814:  Stream->Abstraction.fo_close= 
_ShellClose;
LibC/Uefi/Devices/UefiShell/daShell.c:815:  Stream->Abstraction.fo_read = 
_ShellRead;
LibC/Uefi/Devices/UefiShell/daShell.c:816:  Stream->Abstraction.fo_write= 
_ShellWrite;
LibC/Uefi/Devices/UefiShell/daShell.c:818:  Stream->Abstraction.fo_poll = 
_ShellPoll;
LibC/Uefi/Devices/UefiShell/daShell.c:820:  Stream->Abstraction.fo_stat = 
_ShellStat;
LibC/Uefi/Devices/UefiShell/daShell.c:821:  Stream->Abstraction.fo_ioctl= 
_ShellIoctl;
LibC/Uefi/Devices/UefiShell/daShell.c:822:  Stream->Abstraction.fo_delete   = 
_ShellDelete;
LibC/Uefi/Devices/UefiShell/daShell.c:823:  Stream->Abstraction.fo_rmdir= 
_ShellRmdir;
LibC/Uefi/Devices/UefiShell/daShell.c:824:  Stream->Abstraction.fo_mkdir= 
_ShellMkdir;
LibC/Uefi/Devices/UefiShell/daShell.c:825:  Stream->Abstraction.fo_rename   = 
_ShellRename;
LibC/Uefi/Devices/UefiShell/daShell.c:826:  Stream->Abstraction.fo_lseek= 
_ShellSeek;


From: af...@apple.com [mailto:af...@apple.com]
Sent: Wednesday, July 06, 2016 1:48 PM
To: Carsey, Jaben <jaben.car...@intel.com>
Cc: Michael Zimmermann <sigmaepsilo...@gmail.com>; Marvin Häuser 
<marvin.haeu...@outlook.com>; edk2-devel@lists.01.org; 
edk2-li...@mc2research.org
Subject: Re: [edk2] StdLib usage for drivers?
Importance: High


On Jul 6, 2016, at 1:39 PM, Carsey, Jaben 
<jaben.car...@intel.com<mailto:jaben.car...@intel.com>> wrote:

It works for libs.  If you specify a specific LibraryClass implementation, it 
will not add a second of that library class.

Side note: what if you turn off AutoInitialize PCD for the shell lib?  That 
would get rid of your main issue (the constructor ASSERT)…  then we can see 
what other issues you have.


This seems like the set of library APIs that would be needed (and one usage of  
gEfiShellProtocol->SetCurDir()).

~/work/src/edk2/StdLib(master)>git grep Shell -- *.c
LibC/Main/Main.c:20:#include  
LibC/Main/Main.c:130:ShellAppMain (
LibC/StdLib/Environs.c:23:#include  
LibC/StdLib/Environs.c:155:  OpStat = ShellExecute( , gMD->UString, 
FALSE, NULL, );
LibC/StdLib/Environs.c:167:are determined by the underlying UEFI Shell 
implementation.
LibC/StdLib/Environs.c:181:  EfiEnv = ShellGetEnvironmentVariable(gMD->UString);
LibC/StdLib/Environs.c:248:HostName = ShellGetEnvironmentVariable ( UName );
LibC/StdLib/Environs.c:254:  Status = ShellSetEnvironmentVariable ( UName, 
UValue, TRUE );
LibC/StdLib/Environs.c:265:Status = ShellSetEnvironmentVariable ( 
UName, UValue, FALSE );
LibC/Uefi/Devices/UefiShell/daShell.c:2:  Abstract device driver for the UEFI 
Shell-hosted environment.
LibC/Uefi/Devices/UefiShell/daShell.c:4:  In a Shell-hosted environment, this 
is the driver that is called
LibC/Uefi/Devices/UefiShell/daShell.c:21:#include  
LibC/Uefi/Devices/UefiShell/daShell.c:40:/** EFI Shell specific operations for 
close().
LibC/Uefi/Devices/UefiShell/daShell.c:50:da_ShellClose(
LibC/Uefi/Devices/UefiShell/daShell.c:54:  EFIerrno = ShellCloseFile( 
(SHELL_FILE_HANDLE *)>devdata);
LibC/Uefi/Devices/UefiShell/daShell.c:61:/** EFI Shell specific operations for 
deleting a file or directory.
LibC/Uefi/Devices/UefiShell/daShell.c:71:da_ShellDelete(
LibC/Uefi/Devices/UefiShell/daShell.c:77:  Status = ShellDeleteFile( 
(SHELL_FILE_HANDLE *)>devdata);
LibC/Uefi/Devices/UefiShell/daShell.c:86:/** EFI Shell specific operations for 
setting the position within a file.
LibC/Uefi/Devices/UefiShell/daShell.c:97:da_ShellSeek(
LibC/Uefi/Devices/UefiShell/daShell.c:113:  Status = ShellSetFilePosition( 
FileHandle, 0xULL);
LibC/Uefi/Devices/UefiShell/daShell.c:117:  Status = ShellGetFilePosition( 
FileHandle, (UINT64 *));
LibC/Uefi/Devices/UefiShell/daShell.c:128:Status = ShellSetFilePosition( 
FileHandle, (UINT64)(CurPos + offset));
LibC/Uefi/Devices/UefiShell/daShell.c:131:  Status = ShellGetFilePosition( 
FileHandle, (UINT64 *));
LibC/Uefi/Devices/UefiShell/daShell.c:161:da_ShellMkdir(
LibC/Uefi/Devices/UefiShell/daShell.c:177:Status = ShellCreateDirectory( 
NewPath, );
LibC/Uefi/Devices/UefiShell/daShell.c:179:  FileInfo = ShellGetFileInfo( 
FileHandle);
LibC/Uefi/Devices/UefiShell/daShell.c:180:  Status = RETURN_ABORTED;  // In 
case ShellGetFileInfo() failed
LibC/Uefi/Devices/UefiShell/daShell.c:184:Status = ShellSetFileInfo( 
FileHandle, FileInfo);
LibC/Uefi/Devices/UefiShell/daShell.c:187:  
(void)ShellCloseFile();
LibC/Uefi/Devices/UefiShell/daShell.c:

Re: [edk2] StdLib usage for drivers?

[Carsey, Jaben]

It works for libs.  If you specify a specific LibraryClass implementation, it 
will not add a second of that library class.

Side note: what if you turn off AutoInitialize PCD for the shell lib?  That 
would get rid of your main issue (the constructor ASSERT)…  then we can see 
what other issues you have.

From: Michael Zimmermann [mailto:sigmaepsilo...@gmail.com]
Sent: Wednesday, July 06, 2016 1:32 PM
To: Carsey, Jaben <jaben.car...@intel.com>
Cc: Andrew Fish <af...@apple.com>; Marvin Häuser <marvin.haeu...@outlook.com>; 
edk2-devel@lists.01.org; edk2-li...@mc2research.org
Subject: Re: [edk2] StdLib usage for drivers?
Importance: High

> You can have different libs and different flags per driver.  You use the {} 
> syntax on the INF file in the components section.  This is already done in 
> the Shell.DSC to control PCDs separating the ShellLib when used for the shell 
> compared to ShellLib used for shell applications.

Thanks for clearing that up. Looks like I totally misunderstood how the {} 
syntax works.
Does this only work for Applications/Drivers or for Libraries too so you don't 
have to do this for every driver?

I prefer fixing the behavior at build time over implementing a fallback because 
it would be weird if the application behaves totally different depending on if 
you start it via the BDS or from the Shell.

Thanks
Michael

On Wed, Jul 6, 2016 at 10:20 PM, Carsey, Jaben 
<jaben.car...@intel.com<mailto:jaben.car...@intel.com>> wrote:
Andrew,

Note: The ShellLib is not part of the shell spec.  The ShellProtocol and 
ShellParametersProtocol are.  The current ShellLib is designed to facilitate 
porting and writing of apps for the shell.  The porting will become obsolete 
over time as we encounter fewer and fewer EDKShell apps.

That does’t mean that your conclusion is wrong.  We could make a different 
ShellLib that is not actually dependent on the shell.


Michael,

You can have different libs and different flags per driver.  You use the {} 
syntax on the INF file in the components section.  This is already done in the 
Shell.DSC to control PCDs separating the ShellLib when used for the shell 
compared to ShellLib used for shell applications.


-Jaben



From: Michael Zimmermann 
[mailto:sigmaepsilo...@gmail.com<mailto:sigmaepsilo...@gmail.com>]
Sent: Wednesday, July 06, 2016 1:13 PM
To: Andrew Fish <af...@apple.com<mailto:af...@apple.com>>
Cc: Marvin Häuser 
<marvin.haeu...@outlook.com<mailto:marvin.haeu...@outlook.com>>; 
edk2-devel@lists.01.org<mailto:edk2-devel@lists.01.org>; Carsey, Jaben 
<jaben.car...@intel.com<mailto:jaben.car...@intel.com>>; 
edk2-li...@mc2research.org<mailto:edk2-li...@mc2research.org>
Subject: Re: [edk2] StdLib usage for drivers?
Importance: High

Andrew,

the problem with multiple library instances is that this does only work 
globally and it gets in your way if you need different versions of a dependent 
library.

In our case, Applications/Drivers only depend on LibC, an LibC then depends on 
ShellLib which means we'd have to create another LibC instance which depends on 
another version of ShellLib.

In other words: you can't use the same Library with different build options in 
different drivers built by the same DSC. You'd have to create a renamed .inf 
with different build options which you can't do because your driver doesn't 
depend directly on that lib.

Thanks
Michael

On Wed, Jul 6, 2016 at 9:55 PM, Andrew Fish 
<af...@apple.com<mailto:af...@apple.com>> wrote:

On Jul 6, 2016, at 12:44 PM, Michael Zimmermann 
<sigmaepsilo...@gmail.com<mailto:sigmaepsilo...@gmail.com>> wrote:

Andrew,

sry for not reasing the Shell Spec but I think you can answer this faster.
Does the Spec prevent implementing such a 'pure-EFI' fallback by default so we 
can use the same ShellLib globally?


Michael,

If I remember correctly the Shell Spec specifies how the shell functions and 
what protocols it produces that other code can depend on.

and follow-up question: Libraries are not compiled multiple times right? so If 
I would specify additional CFLAGs these would only be used to build your 
package and not for the libraríes you are using right?


The edk2 has the concept of library instances. The DSC file for the platform 
picks the instance of the library for the Module Type. Basically ShellLib.h 
would be the public API and ShellLib is the library class. The library class is 
the public API and it is names in the [Defines] section of the libraries INF 
file via LIBRARY_CLASS =.  The build system is smart enough to only build the 
libraries that are actually being used.

For example this would be the line in the DSC:
ShellLib|ShellPkg/Library/UefiShellLib/UefiShellLib.inf

And you could change it to:
ShellLib | AltPathToUefiShellLib/AltUefiShellLib.inf

The upside to all this is the same reason we introduced the concept of library 
classes in the 1st place

Re: [edk2] StdLib usage for drivers?

[Michael Zimmermann]

> You can have different libs and different flags per driver.  You use the
{} syntax on the INF file in the components section.  This is already done
in the Shell.DSC to control PCDs separating the ShellLib when used for the
shell compared to ShellLib used for shell applications.

Thanks for clearing that up. Looks like I totally misunderstood how the {}
syntax works.
Does this only work for Applications/Drivers or for Libraries too so you
don't have to do this for every driver?

I prefer fixing the behavior at build time over implementing a fallback
because it would be weird if the application behaves totally different
depending on if you start it via the BDS or from the Shell.

Thanks
Michael

On Wed, Jul 6, 2016 at 10:20 PM, Carsey, Jaben <jaben.car...@intel.com>
wrote:

> Andrew,
>
>
>
> Note: The ShellLib is not part of the shell spec.  The ShellProtocol and
> ShellParametersProtocol are.  The current ShellLib is designed to
> facilitate porting and writing of apps for the shell.  The porting will
> become obsolete over time as we encounter fewer and fewer EDKShell apps.
>
>
>
> That does’t mean that your conclusion is wrong.  We could make a different
> ShellLib that is not actually dependent on the shell.
>
>
>
>
>
> Michael,
>
>
>
> You can have different libs and different flags per driver.  You use the
> {} syntax on the INF file in the components section.  This is already done
> in the Shell.DSC to control PCDs separating the ShellLib when used for the
> shell compared to ShellLib used for shell applications.
>
>
>
>
>
> -Jaben
>
>
>
>
>
>
>
> *From:* Michael Zimmermann [mailto:sigmaepsilo...@gmail.com]
> *Sent:* Wednesday, July 06, 2016 1:13 PM
> *To:* Andrew Fish <af...@apple.com>
> *Cc:* Marvin Häuser <marvin.haeu...@outlook.com>; edk2-devel@lists.01.org;
> Carsey, Jaben <jaben.car...@intel.com>; edk2-li...@mc2research.org
> *Subject:* Re: [edk2] StdLib usage for drivers?
> *Importance:* High
>
>
>
> Andrew,
>
>
>
> the problem with multiple library instances is that this does only work
> globally and it gets in your way if you need different versions of a
> dependent library.
>
>
>
> In our case, Applications/Drivers only depend on LibC, an LibC then
> depends on ShellLib which means we'd have to create another LibC instance
> which depends on another version of ShellLib.
>
>
>
> In other words: you can't use the same Library with different build
> options in different drivers built by the same DSC. You'd have to create a
> renamed .inf with different build options which you can't do because your
> driver doesn't depend directly on that lib.
>
>
>
> Thanks
>
> Michael
>
>
>
> On Wed, Jul 6, 2016 at 9:55 PM, Andrew Fish <af...@apple.com> wrote:
>
>
>
> On Jul 6, 2016, at 12:44 PM, Michael Zimmermann <sigmaepsilo...@gmail.com>
> wrote:
>
>
>
> Andrew,
>
>
>
> sry for not reasing the Shell Spec but I think you can answer this faster.
>
> Does the Spec prevent implementing such a 'pure-EFI' fallback by default
> so we can use the same ShellLib globally?
>
>
>
>
>
> Michael,
>
>
>
> If I remember correctly the Shell Spec specifies how the shell functions
> and what protocols it produces that other code can depend on.
>
>
>
> and follow-up question: Libraries are not compiled multiple times right?
> so If I would specify additional CFLAGs these would only be used to build
> your package and not for the libraríes you are using right?
>
>
>
>
>
> The edk2 has the concept of library instances. The DSC file for the
> platform picks the instance of the library for the Module Type. Basically
> ShellLib.h would be the public API and ShellLib is the library class. The
> library class is the public API and it is names in the [Defines] section of
> the libraries INF file via LIBRARY_CLASS =.  The build system is smart
> enough to only build the libraries that are actually being used.
>
>
>
> For example this would be the line in the DSC:
>
> ShellLib|ShellPkg/Library/UefiShellLib/UefiShellLib.inf
>
>
>
> And you could change it to:
>
> ShellLib | AltPathToUefiShellLib/AltUefiShellLib.inf
>
>
>
> The upside to all this is the same reason we introduced the concept of
> library classes in the 1st place, you can change the behavior of the StdLib
> without modifying any of the code in the standard lib. This would be a good
> way to experiment and if you get it working I guess it could get added to
> the StdLib at some point as an alternate way to build it.
>
>
>
> Thanks,
>
>
>
> Andrew Fish
>
>
>
> If th

Re: [edk2] StdLib usage for drivers?

[Andrew Fish]

ior using cflags in your DSC because it would prevent building both
>> shell and efi users of ShellLib with one DSC file.
>> 
>> Thanks,
>> Michael
>> 
>> On Wed, Jul 6, 2016 at 9:25 PM, Andrew Fish <af...@apple.com> wrote:
>> 
>>> 
>>>> On Jul 4, 2016, at 11:17 AM, Marvin Häuser <marvin.haeu...@outlook.com>
>>> wrote:
>>>> 
>>>> Daryl, Jaben,
>>>> 
>>>> As you are the package maintainers of StdLib, could you please comment
>>> on the situation?
>>>> If modifications to have StdLib working for drivers are welcome, I
>>> would offer my help in cleaning up the existing stuff and/or write my own
>>> patches, though of course there is little point if there is no reaction
>>> from the reviewers.
>>>> If it is of any interest, I want to write a shim library for Capstone
>>> and would prefer not to have three copies of Std functions in my tree
>>> (StdLib, CryptoPkg (SSL) and Capstone), but rather an improved StdLib that
>>> works for all.
>>>> 
>>>> Thanks to those who have offered alternative solutions, but I prefer to
>>> keep it ‚clean‘ and have the libraries shipping with EDK2 work. :)
>>>> 
>>> 
>>> Marvin,
>>> 
>>> It looks like the StdLib has a dependency on the ShellLib and
>>> ShellCEntryLib. One option would be to implement a new instance of the
>>> ShellLib and the ShellCEntryLib that don't depend on the Shell. You can get
>>> a template of how to do file operations from here:
>>> https://github.com/tianocore/edk2/blob/master/EmbeddedPkg/Library/EfiFileLib/EfiFileLib.c
>>> 
>>> You would have a couple of options. You good go with the EfiFileLib
>>> volume synatax, you could just do pure EFI (unclear how to specify a volume
>>> (driver letter)). You could also just add code to fallback if you can't
>>> find the shell protocols to due more pure EFI stuff. That way you don't
>>> need to modify the StdLib just how you build the StdLib from the DSC.
>>> 
>>> For example you could implement the functions in the ShellLib that the
>>> StdLib requires and use something like the EfiFileLib to make it easier. I
>>> think ShellCEntryLib would just be a copy of
>>> https://github.com/tianocore/edk2/tree/master/MdePkg/Library/UefiApplicationEntryPoint
>>> 
>>> Thanks,
>>> 
>>> Andrew Fish
>>> 
>>>> Regards,
>>>> Marvin.
>>>> 
>>>> From: Michael Zimmermann [mailto:sigmaepsilo...@gmail.com]
>>>> Sent: Thursday, June 23, 2016 5:32 AM
>>>> To: Marvin Häuser <marvin.haeu...@outlook.com>
>>>> Cc: edk2-devel@lists.01.org; af...@apple.com
>>>> Subject: Re: [edk2] StdLib usage for drivers?
>>>> 
>>>> well for the patch to go upstream it would have to get improved a lot.
>>>> I've tried to implement this in a way that doesn't need a different dsc
>>> but the problem is that ShellLib's constructor ASSERT's if it can't find
>>> the shell protocol and removing that would probably be against the spec's.
>>>> 
>>>> Also it appears that the StdLib maintainers are kinda busy because most
>>> of the StdLib patches I've sent to this mailing list didn't get reviewed.
>>>> 
>>>> Thanks
>>>> Michael
>>>> 
>>>> On Wed, Jun 22, 2016 at 10:37 PM, Marvin Häuser <
>>> marvin.haeu...@outlook.com<mailto:marvin.haeu...@outlook.com>> wrote:
>>>> Hey Michael,
>>>> 
>>>> Thank you for your input! This looks interesting.
>>>> Maybe it would be a good idea to provide the libraries that depend on
>>> Shell (in master) with functions that call ASSERT (FALSE); for drivers? I
>>> do not need file I/O, so I think your modifications might work out well for
>>> me. Would be very nice of course if such changes found their way upstream.
>>> :)
>>>> 
>>>> And thank you very much for your comment as well, Andrew! It makes
>>> sense that StdLib is primarily targeted at porting console applications to
>>> UEFI Shell.
>>>> 
>>>> Regards,
>>>> Marvin.
>>>> 
>>>> From: Michael Zimmermann [mailto:sigmaepsilo...@gmail.com>> sigmaepsilo...@gmail.com>]
>>>> Sent: Wednesday, June 22, 2016 10:28 PM
>>>> To: Marvin H?user <marvin.haeu...@outlook.com>> marvin.haeu...@outlook.com>>
>>>&

Re: [edk2] StdLib usage for drivers?

[Carsey, Jaben]

Andrew,

Note: The ShellLib is not part of the shell spec.  The ShellProtocol and 
ShellParametersProtocol are.  The current ShellLib is designed to facilitate 
porting and writing of apps for the shell.  The porting will become obsolete 
over time as we encounter fewer and fewer EDKShell apps.

That does’t mean that your conclusion is wrong.  We could make a different 
ShellLib that is not actually dependent on the shell.


Michael,

You can have different libs and different flags per driver.  You use the {} 
syntax on the INF file in the components section.  This is already done in the 
Shell.DSC to control PCDs separating the ShellLib when used for the shell 
compared to ShellLib used for shell applications.


-Jaben



From: Michael Zimmermann [mailto:sigmaepsilo...@gmail.com]
Sent: Wednesday, July 06, 2016 1:13 PM
To: Andrew Fish <af...@apple.com>
Cc: Marvin Häuser <marvin.haeu...@outlook.com>; edk2-devel@lists.01.org; 
Carsey, Jaben <jaben.car...@intel.com>; edk2-li...@mc2research.org
Subject: Re: [edk2] StdLib usage for drivers?
Importance: High

Andrew,

the problem with multiple library instances is that this does only work 
globally and it gets in your way if you need different versions of a dependent 
library.

In our case, Applications/Drivers only depend on LibC, an LibC then depends on 
ShellLib which means we'd have to create another LibC instance which depends on 
another version of ShellLib.

In other words: you can't use the same Library with different build options in 
different drivers built by the same DSC. You'd have to create a renamed .inf 
with different build options which you can't do because your driver doesn't 
depend directly on that lib.

Thanks
Michael

On Wed, Jul 6, 2016 at 9:55 PM, Andrew Fish 
<af...@apple.com<mailto:af...@apple.com>> wrote:

On Jul 6, 2016, at 12:44 PM, Michael Zimmermann 
<sigmaepsilo...@gmail.com<mailto:sigmaepsilo...@gmail.com>> wrote:

Andrew,

sry for not reasing the Shell Spec but I think you can answer this faster.
Does the Spec prevent implementing such a 'pure-EFI' fallback by default so we 
can use the same ShellLib globally?


Michael,

If I remember correctly the Shell Spec specifies how the shell functions and 
what protocols it produces that other code can depend on.


and follow-up question: Libraries are not compiled multiple times right? so If 
I would specify additional CFLAGs these would only be used to build your 
package and not for the libraríes you are using right?


The edk2 has the concept of library instances. The DSC file for the platform 
picks the instance of the library for the Module Type. Basically ShellLib.h 
would be the public API and ShellLib is the library class. The library class is 
the public API and it is names in the [Defines] section of the libraries INF 
file via LIBRARY_CLASS =.  The build system is smart enough to only build the 
libraries that are actually being used.

For example this would be the line in the DSC:
ShellLib|ShellPkg/Library/UefiShellLib/UefiShellLib.inf

And you could change it to:
ShellLib | AltPathToUefiShellLib/AltUefiShellLib.inf

The upside to all this is the same reason we introduced the concept of library 
classes in the 1st place, you can change the behavior of the StdLib without 
modifying any of the code in the standard lib. This would be a good way to 
experiment and if you get it working I guess it could get added to the StdLib 
at some point as an alternate way to build it.

Thanks,

Andrew Fish


If that's correct then we should stay away from changing ShellLib's behavior 
using cflags in your DSC because it would prevent building both shell and efi 
users of ShellLib with one DSC file.

Thanks,
Michael

On Wed, Jul 6, 2016 at 9:25 PM, Andrew Fish 
<af...@apple.com<mailto:af...@apple.com>> wrote:

> On Jul 4, 2016, at 11:17 AM, Marvin Häuser 
> <marvin.haeu...@outlook.com<mailto:marvin.haeu...@outlook.com>> wrote:
>
> Daryl, Jaben,
>
> As you are the package maintainers of StdLib, could you please comment on the 
> situation?
> If modifications to have StdLib working for drivers are welcome, I would 
> offer my help in cleaning up the existing stuff and/or write my own patches, 
> though of course there is little point if there is no reaction from the 
> reviewers.
> If it is of any interest, I want to write a shim library for Capstone and 
> would prefer not to have three copies of Std functions in my tree (StdLib, 
> CryptoPkg (SSL) and Capstone), but rather an improved StdLib that works for 
> all.
>
> Thanks to those who have offered alternative solutions, but I prefer to keep 
> it ‚clean‘ and have the libraries shipping with EDK2 work. :)
>

Marvin,

It looks like the StdLib has a dependency on the ShellLib and ShellCEntryLib. 
One option would be to implement a new instance of the ShellLib and the 
ShellCEntryLib that don't depend on the Shell.

Re: [edk2] StdLib usage for drivers?

[Michael Zimmermann]

Andrew,

the problem with multiple library instances is that this does only work
globally and it gets in your way if you need different versions of a
dependent library.

In our case, Applications/Drivers only depend on LibC, an LibC then depends
on ShellLib which means we'd have to create another LibC instance which
depends on another version of ShellLib.

In other words: you can't use the same Library with different build options
in different drivers built by the same DSC. You'd have to create a renamed
.inf with different build options which you can't do because your driver
doesn't depend directly on that lib.

Thanks
Michael

On Wed, Jul 6, 2016 at 9:55 PM, Andrew Fish <af...@apple.com> wrote:

>
> On Jul 6, 2016, at 12:44 PM, Michael Zimmermann <sigmaepsilo...@gmail.com>
> wrote:
>
> Andrew,
>
> sry for not reasing the Shell Spec but I think you can answer this faster.
> Does the Spec prevent implementing such a 'pure-EFI' fallback by default
> so we can use the same ShellLib globally?
>
>
> Michael,
>
> If I remember correctly the Shell Spec specifies how the shell functions
> and what protocols it produces that other code can depend on.
>
> and follow-up question: Libraries are not compiled multiple times right?
> so If I would specify additional CFLAGs these would only be used to build
> your package and not for the libraríes you are using right?
>
>
> The edk2 has the concept of library instances. The DSC file for the
> platform picks the instance of the library for the Module Type. Basically
> ShellLib.h would be the public API and ShellLib is the library class. The
> library class is the public API and it is names in the [Defines] section of
> the libraries INF file via LIBRARY_CLASS =.  The build system is smart
> enough to only build the libraries that are actually being used.
>
> For example this would be the line in the DSC:
> ShellLib|ShellPkg/Library/UefiShellLib/UefiShellLib.inf
>
> And you could change it to:
> ShellLib | AltPathToUefiShellLib/AltUefiShellLib.inf
>
> The upside to all this is the same reason we introduced the concept of
> library classes in the 1st place, you can change the behavior of the StdLib
> without modifying any of the code in the standard lib. This would be a good
> way to experiment and if you get it working I guess it could get added to
> the StdLib at some point as an alternate way to build it.
>
> Thanks,
>
> Andrew Fish
>
> If that's correct then we should stay away from changing ShellLib's
> behavior using cflags in your DSC because it would prevent building both
> shell and efi users of ShellLib with one DSC file.
>
> Thanks,
> Michael
>
> On Wed, Jul 6, 2016 at 9:25 PM, Andrew Fish <af...@apple.com> wrote:
>
>>
>> > On Jul 4, 2016, at 11:17 AM, Marvin Häuser <marvin.haeu...@outlook.com>
>> wrote:
>> >
>> > Daryl, Jaben,
>> >
>> > As you are the package maintainers of StdLib, could you please comment
>> on the situation?
>> > If modifications to have StdLib working for drivers are welcome, I
>> would offer my help in cleaning up the existing stuff and/or write my own
>> patches, though of course there is little point if there is no reaction
>> from the reviewers.
>> > If it is of any interest, I want to write a shim library for Capstone
>> and would prefer not to have three copies of Std functions in my tree
>> (StdLib, CryptoPkg (SSL) and Capstone), but rather an improved StdLib that
>> works for all.
>> >
>> > Thanks to those who have offered alternative solutions, but I prefer to
>> keep it ‚clean‘ and have the libraries shipping with EDK2 work. :)
>> >
>>
>> Marvin,
>>
>> It looks like the StdLib has a dependency on the ShellLib and
>> ShellCEntryLib. One option would be to implement a new instance of the
>> ShellLib and the ShellCEntryLib that don't depend on the Shell. You can get
>> a template of how to do file operations from here:
>> https://github.com/tianocore/edk2/blob/master/EmbeddedPkg/Library/EfiFileLib/EfiFileLib.c
>>
>> You would have a couple of options. You good go with the EfiFileLib
>> volume synatax, you could just do pure EFI (unclear how to specify a volume
>> (driver letter)). You could also just add code to fallback if you can't
>> find the shell protocols to due more pure EFI stuff. That way you don't
>> need to modify the StdLib just how you build the StdLib from the DSC.
>>
>> For example you could implement the functions in the ShellLib that the
>> StdLib requires and use something like the EfiFileLib to make it easier. I
>> think ShellCEntryLib would just be a copy of
>> https://gi

Re: [edk2] StdLib usage for drivers?

[Andrew Fish]

> On Jul 6, 2016, at 12:44 PM, Michael Zimmermann <sigmaepsilo...@gmail.com> 
> wrote:
> 
> Andrew,
> 
> sry for not reasing the Shell Spec but I think you can answer this faster.
> Does the Spec prevent implementing such a 'pure-EFI' fallback by default so 
> we can use the same ShellLib globally?
> 

Michael,

If I remember correctly the Shell Spec specifies how the shell functions and 
what protocols it produces that other code can depend on. 

> and follow-up question: Libraries are not compiled multiple times right? so 
> If I would specify additional CFLAGs these would only be used to build your 
> package and not for the libraríes you are using right?
> 

The edk2 has the concept of library instances. The DSC file for the platform 
picks the instance of the library for the Module Type. Basically ShellLib.h 
would be the public API and ShellLib is the library class. The library class is 
the public API and it is names in the [Defines] section of the libraries INF 
file via LIBRARY_CLASS =.  The build system is smart enough to only build the 
libraries that are actually being used. 

For example this would be the line in the DSC:
ShellLib|ShellPkg/Library/UefiShellLib/UefiShellLib.inf

And you could change it to:
ShellLib | AltPathToUefiShellLib/AltUefiShellLib.inf

The upside to all this is the same reason we introduced the concept of library 
classes in the 1st place, you can change the behavior of the StdLib without 
modifying any of the code in the standard lib. This would be a good way to 
experiment and if you get it working I guess it could get added to the StdLib 
at some point as an alternate way to build it. 

Thanks,

Andrew Fish

> If that's correct then we should stay away from changing ShellLib's behavior 
> using cflags in your DSC because it would prevent building both shell and efi 
> users of ShellLib with one DSC file.
> 
> Thanks,
> Michael 
> 
> On Wed, Jul 6, 2016 at 9:25 PM, Andrew Fish <af...@apple.com 
> <mailto:af...@apple.com>> wrote:
> 
> > On Jul 4, 2016, at 11:17 AM, Marvin Häuser <marvin.haeu...@outlook.com 
> > <mailto:marvin.haeu...@outlook.com>> wrote:
> >
> > Daryl, Jaben,
> >
> > As you are the package maintainers of StdLib, could you please comment on 
> > the situation?
> > If modifications to have StdLib working for drivers are welcome, I would 
> > offer my help in cleaning up the existing stuff and/or write my own 
> > patches, though of course there is little point if there is no reaction 
> > from the reviewers.
> > If it is of any interest, I want to write a shim library for Capstone and 
> > would prefer not to have three copies of Std functions in my tree (StdLib, 
> > CryptoPkg (SSL) and Capstone), but rather an improved StdLib that works for 
> > all.
> >
> > Thanks to those who have offered alternative solutions, but I prefer to 
> > keep it ‚clean‘ and have the libraries shipping with EDK2 work. :)
> >
> 
> Marvin,
> 
> It looks like the StdLib has a dependency on the ShellLib and ShellCEntryLib. 
> One option would be to implement a new instance of the ShellLib and the 
> ShellCEntryLib that don't depend on the Shell. You can get a template of how 
> to do file operations from here: 
> https://github.com/tianocore/edk2/blob/master/EmbeddedPkg/Library/EfiFileLib/EfiFileLib.c
>  
> <https://github.com/tianocore/edk2/blob/master/EmbeddedPkg/Library/EfiFileLib/EfiFileLib.c>
> 
> You would have a couple of options. You good go with the EfiFileLib volume 
> synatax, you could just do pure EFI (unclear how to specify a volume (driver 
> letter)). You could also just add code to fallback if you can't find the 
> shell protocols to due more pure EFI stuff. That way you don't need to modify 
> the StdLib just how you build the StdLib from the DSC.
> 
> For example you could implement the functions in the ShellLib that the StdLib 
> requires and use something like the EfiFileLib to make it easier. I think 
> ShellCEntryLib would just be a copy of 
> https://github.com/tianocore/edk2/tree/master/MdePkg/Library/UefiApplicationEntryPoint
>  
> <https://github.com/tianocore/edk2/tree/master/MdePkg/Library/UefiApplicationEntryPoint>
> 
> Thanks,
> 
> Andrew Fish
> 
> > Regards,
> > Marvin.
> >
> > From: Michael Zimmermann [mailto:sigmaepsilo...@gmail.com 
> > <mailto:sigmaepsilo...@gmail.com>]
> > Sent: Thursday, June 23, 2016 5:32 AM
> > To: Marvin Häuser <marvin.haeu...@outlook.com 
> > <mailto:marvin.haeu...@outlook.com>>
> > Cc: edk2-devel@lists.01.org <mailto:edk2-devel@lists.01.org>; 
> > af...@apple.com <mailto:af...@apple.com>
> > Subject: Re: [edk2] StdLib u

Re: [edk2] StdLib usage for drivers?

[Michael Zimmermann]

Andrew,

sry for not reasing the Shell Spec but I think you can answer this faster.
Does the Spec prevent implementing such a 'pure-EFI' fallback by default so
we can use the same ShellLib globally?

and follow-up question: Libraries are not compiled multiple times right? so
If I would specify additional CFLAGs these would only be used to build your
package and not for the libraríes you are using right?

If that's correct then we should stay away from changing ShellLib's
behavior using cflags in your DSC because it would prevent building both
shell and efi users of ShellLib with one DSC file.

Thanks,
Michael

On Wed, Jul 6, 2016 at 9:25 PM, Andrew Fish <af...@apple.com> wrote:

>
> > On Jul 4, 2016, at 11:17 AM, Marvin Häuser <marvin.haeu...@outlook.com>
> wrote:
> >
> > Daryl, Jaben,
> >
> > As you are the package maintainers of StdLib, could you please comment
> on the situation?
> > If modifications to have StdLib working for drivers are welcome, I would
> offer my help in cleaning up the existing stuff and/or write my own
> patches, though of course there is little point if there is no reaction
> from the reviewers.
> > If it is of any interest, I want to write a shim library for Capstone
> and would prefer not to have three copies of Std functions in my tree
> (StdLib, CryptoPkg (SSL) and Capstone), but rather an improved StdLib that
> works for all.
> >
> > Thanks to those who have offered alternative solutions, but I prefer to
> keep it ‚clean‘ and have the libraries shipping with EDK2 work. :)
> >
>
> Marvin,
>
> It looks like the StdLib has a dependency on the ShellLib and
> ShellCEntryLib. One option would be to implement a new instance of the
> ShellLib and the ShellCEntryLib that don't depend on the Shell. You can get
> a template of how to do file operations from here:
> https://github.com/tianocore/edk2/blob/master/EmbeddedPkg/Library/EfiFileLib/EfiFileLib.c
>
> You would have a couple of options. You good go with the EfiFileLib volume
> synatax, you could just do pure EFI (unclear how to specify a volume
> (driver letter)). You could also just add code to fallback if you can't
> find the shell protocols to due more pure EFI stuff. That way you don't
> need to modify the StdLib just how you build the StdLib from the DSC.
>
> For example you could implement the functions in the ShellLib that the
> StdLib requires and use something like the EfiFileLib to make it easier. I
> think ShellCEntryLib would just be a copy of
> https://github.com/tianocore/edk2/tree/master/MdePkg/Library/UefiApplicationEntryPoint
>
> Thanks,
>
> Andrew Fish
>
> > Regards,
> > Marvin.
> >
> > From: Michael Zimmermann [mailto:sigmaepsilo...@gmail.com]
> > Sent: Thursday, June 23, 2016 5:32 AM
> > To: Marvin Häuser <marvin.haeu...@outlook.com>
> > Cc: edk2-devel@lists.01.org; af...@apple.com
> > Subject: Re: [edk2] StdLib usage for drivers?
> >
> > well for the patch to go upstream it would have to get improved a lot.
> > I've tried to implement this in a way that doesn't need a different dsc
> but the problem is that ShellLib's constructor ASSERT's if it can't find
> the shell protocol and removing that would probably be against the spec's.
> >
> > Also it appears that the StdLib maintainers are kinda busy because most
> of the StdLib patches I've sent to this mailing list didn't get reviewed.
> >
> > Thanks
> > Michael
> >
> > On Wed, Jun 22, 2016 at 10:37 PM, Marvin Häuser <
> marvin.haeu...@outlook.com<mailto:marvin.haeu...@outlook.com>> wrote:
> > Hey Michael,
> >
> > Thank you for your input! This looks interesting.
> > Maybe it would be a good idea to provide the libraries that depend on
> Shell (in master) with functions that call ASSERT (FALSE); for drivers? I
> do not need file I/O, so I think your modifications might work out well for
> me. Would be very nice of course if such changes found their way upstream.
> :)
> >
> > And thank you very much for your comment as well, Andrew! It makes sense
> that StdLib is primarily targeted at porting console applications to UEFI
> Shell.
> >
> > Regards,
> > Marvin.
> >
> > From: Michael Zimmermann [mailto:sigmaepsilo...@gmail.com sigmaepsilo...@gmail.com>]
> > Sent: Wednesday, June 22, 2016 10:28 PM
> > To: Marvin H?user <marvin.haeu...@outlook.com marvin.haeu...@outlook.com>>
> > Cc: edk2-devel@lists.01.org<mailto:edk2-devel@lists.01.org>
> > Subject: Re: [edk2] StdLib usage for drivers?
> >
> > In my fork of edk2 I've added support for using StdLib without the Shell.
> > It's quite hacky(setenv/g

Re: [edk2] StdLib usage for drivers?

[Andrew Fish]

> On Jul 4, 2016, at 11:17 AM, Marvin Häuser <marvin.haeu...@outlook.com> wrote:
> 
> Daryl, Jaben,
> 
> As you are the package maintainers of StdLib, could you please comment on the 
> situation?
> If modifications to have StdLib working for drivers are welcome, I would 
> offer my help in cleaning up the existing stuff and/or write my own patches, 
> though of course there is little point if there is no reaction from the 
> reviewers.
> If it is of any interest, I want to write a shim library for Capstone and 
> would prefer not to have three copies of Std functions in my tree (StdLib, 
> CryptoPkg (SSL) and Capstone), but rather an improved StdLib that works for 
> all.
> 
> Thanks to those who have offered alternative solutions, but I prefer to keep 
> it ‚clean‘ and have the libraries shipping with EDK2 work. :)
> 

Marvin,

It looks like the StdLib has a dependency on the ShellLib and ShellCEntryLib. 
One option would be to implement a new instance of the ShellLib and the 
ShellCEntryLib that don't depend on the Shell. You can get a template of how to 
do file operations from here: 
https://github.com/tianocore/edk2/blob/master/EmbeddedPkg/Library/EfiFileLib/EfiFileLib.c

You would have a couple of options. You good go with the EfiFileLib volume 
synatax, you could just do pure EFI (unclear how to specify a volume (driver 
letter)). You could also just add code to fallback if you can't find the shell 
protocols to due more pure EFI stuff. That way you don't need to modify the 
StdLib just how you build the StdLib from the DSC. 

For example you could implement the functions in the ShellLib that the StdLib 
requires and use something like the EfiFileLib to make it easier. I think 
ShellCEntryLib would just be a copy of 
https://github.com/tianocore/edk2/tree/master/MdePkg/Library/UefiApplicationEntryPoint

Thanks,

Andrew Fish

> Regards,
> Marvin.
> 
> From: Michael Zimmermann [mailto:sigmaepsilo...@gmail.com]
> Sent: Thursday, June 23, 2016 5:32 AM
> To: Marvin Häuser <marvin.haeu...@outlook.com>
> Cc: edk2-devel@lists.01.org; af...@apple.com
> Subject: Re: [edk2] StdLib usage for drivers?
> 
> well for the patch to go upstream it would have to get improved a lot.
> I've tried to implement this in a way that doesn't need a different dsc but 
> the problem is that ShellLib's constructor ASSERT's if it can't find the 
> shell protocol and removing that would probably be against the spec's.
> 
> Also it appears that the StdLib maintainers are kinda busy because most of 
> the StdLib patches I've sent to this mailing list didn't get reviewed.
> 
> Thanks
> Michael
> 
> On Wed, Jun 22, 2016 at 10:37 PM, Marvin Häuser 
> <marvin.haeu...@outlook.com<mailto:marvin.haeu...@outlook.com>> wrote:
> Hey Michael,
> 
> Thank you for your input! This looks interesting.
> Maybe it would be a good idea to provide the libraries that depend on Shell 
> (in master) with functions that call ASSERT (FALSE); for drivers? I do not 
> need file I/O, so I think your modifications might work out well for me. 
> Would be very nice of course if such changes found their way upstream. :)
> 
> And thank you very much for your comment as well, Andrew! It makes sense that 
> StdLib is primarily targeted at porting console applications to UEFI Shell.
> 
> Regards,
> Marvin.
> 
> From: Michael Zimmermann 
> [mailto:sigmaepsilo...@gmail.com<mailto:sigmaepsilo...@gmail.com>]
> Sent: Wednesday, June 22, 2016 10:28 PM
> To: Marvin H?user 
> <marvin.haeu...@outlook.com<mailto:marvin.haeu...@outlook.com>>
> Cc: edk2-devel@lists.01.org<mailto:edk2-devel@lists.01.org>
> Subject: Re: [edk2] StdLib usage for drivers?
> 
> In my fork of edk2 I've added support for using StdLib without the Shell.
> It's quite hacky(setenv/getenv are stubs, no File IO and maybe other things 
> hidden by linker GC and me not using all features).
> 
> But depending on which StdLib features you need this can work pretty good.
> 
> here's the commit that does the magic:
> https://github.com/efidroid/edk2/commit/bf7a296718486bafaf774ea8bcf187c162c3c167
> 
> and this is how you convert a Shell project to a NonShell one:
> https://github.com/efidroid/uefi_apps_EFIDroidUi/commit/23f0fa08108b8f852564fae733c6a7bce62e2070
> 
> as you can see it works by using StdLib with different libraries/cflags so 
> you have to compile your driver separately if there are other modules which 
> need the normal StdLib.
> 
> Thanks
> Michael
> 
> On Wed, Jun 22, 2016 at 9:38 PM, Marvin H?user 
> <marvin.haeu...@outlook.com<mailto:marvin.haeu...@outlook.com><mailto:marvin.haeu...@outlook.com<mailto:marvin.haeu...@outlook.com>>>
>  wrote:
> Dear EDK2 developers,
> 

Re: [edk2] StdLib usage for drivers?

[Daryl McDaniel]

Marvin,

 

Personally, I would be happy for any contributions you could make for StdLib.

The only remaining Shell dependencies should be:

* File system support.

* One or two functions in PosixLib

 

The next big area is breaking up the MainData structure and moving its members 
into the sub-libraries that they are relevant to.

 

Once that is done, I had planned on trying to break as many of the library’s 
interdependencies as possible.

 

As long as StdLib continues to pass the ISO C95 compliance tests, and gets 
smaller instead of larger, just about anything is fair game.

I had pretty much decided to abandon using the FreeBSD / NetBSD code since it 
is not the best suited for the EDK II environment.

 

As Michael surmised, I’ve been traveling a lot and too busy lately to devote as 
much attention to StdLib as I wish I could.

 

Please remember for your contributions; compatibility with both GCC and 
Microsoft compilers and both IA32 and X64 is required.  If you can also retain 
or provide support for ARM then that would be great.

 

I can’t speak for Jaben, so I will have to wait for his response to determine 
his stance on the issue.

 

I’m willing to answer any questions about AppPkg or StdLib that I can.

 

Thank you for your offer.

Daryl McDaniel

 

 

From: Marvin Häuser [mailto:marvin.haeu...@outlook.com] 
Sent: Monday, July 04, 2016 11:18 AM
To: edk2-devel@lists.01.org
Cc: Michael Zimmermann <sigmaepsilo...@gmail.com>; edk2-li...@mc2research.org; 
Carsey, Jaben <jaben.car...@intel.com>
Subject: RE: [edk2] StdLib usage for drivers?

 

Daryl, Jaben,

 

As you are the package maintainers of StdLib, could you please comment on the 
situation?

If modifications to have StdLib working for drivers are welcome, I would offer 
my help in cleaning up the existing stuff and/or write my own patches, though 
of course there is little point if there is no reaction from the reviewers.

If it is of any interest, I want to write a shim library for Capstone and would 
prefer not to have three copies of Std functions in my tree (StdLib, CryptoPkg 
(SSL) and Capstone), but rather an improved StdLib that works for all.

 

Thanks to those who have offered alternative solutions, but I prefer to keep it 
‚clean‘ and have the libraries shipping with EDK2 work. :)

 

Regards,

Marvin.

 

From: Michael Zimmermann [mailto:sigmaepsilo...@gmail.com] 
Sent: Thursday, June 23, 2016 5:32 AM
To: Marvin Häuser <marvin.haeu...@outlook.com 
<mailto:marvin.haeu...@outlook.com> >
Cc: edk2-devel@lists.01.org <mailto:edk2-devel@lists.01.org> ; af...@apple.com 
<mailto:af...@apple.com> 
Subject: Re: [edk2] StdLib usage for drivers?

 

well for the patch to go upstream it would have to get improved a lot.

I've tried to implement this in a way that doesn't need a different dsc but the 
problem is that ShellLib's constructor ASSERT's if it can't find the shell 
protocol and removing that would probably be against the spec's.

 

Also it appears that the StdLib maintainers are kinda busy because most of the 
StdLib patches I've sent to this mailing list didn't get reviewed.

 

Thanks

Michael

 

On Wed, Jun 22, 2016 at 10:37 PM, Marvin Häuser <marvin.haeu...@outlook.com 
<mailto:marvin.haeu...@outlook.com> > wrote:

Hey Michael,

Thank you for your input! This looks interesting.
Maybe it would be a good idea to provide the libraries that depend on Shell (in 
master) with functions that call ASSERT (FALSE); for drivers? I do not need 
file I/O, so I think your modifications might work out well for me. Would be 
very nice of course if such changes found their way upstream. :)

And thank you very much for your comment as well, Andrew! It makes sense that 
StdLib is primarily targeted at porting console applications to UEFI Shell.

Regards,
Marvin.

From: Michael Zimmermann [mailto:sigmaepsilo...@gmail.com 
<mailto:sigmaepsilo...@gmail.com> ]
Sent: Wednesday, June 22, 2016 10:28 PM
To: Marvin H?user <marvin.haeu...@outlook.com 
<mailto:marvin.haeu...@outlook.com> >
Cc: edk2-devel@lists.01.org <mailto:edk2-devel@lists.01.org> 
Subject: Re: [edk2] StdLib usage for drivers?

In my fork of edk2 I've added support for using StdLib without the Shell.
It's quite hacky(setenv/getenv are stubs, no File IO and maybe other things 
hidden by linker GC and me not using all features).

But depending on which StdLib features you need this can work pretty good.

here's the commit that does the magic:
https://github.com/efidroid/edk2/commit/bf7a296718486bafaf774ea8bcf187c162c3c167

and this is how you convert a Shell project to a NonShell one:
https://github.com/efidroid/uefi_apps_EFIDroidUi/commit/23f0fa08108b8f852564fae733c6a7bce62e2070

as you can see it works by using StdLib with different libraries/cflags so you 
have to compile your driver separately if there are other modules which need 
the normal StdLib.

Thanks
Michael

On

Re: [edk2] StdLib usage for drivers?

[Marvin Häuser]

Daryl, Jaben,

As you are the package maintainers of StdLib, could you please comment on the 
situation?
If modifications to have StdLib working for drivers are welcome, I would offer 
my help in cleaning up the existing stuff and/or write my own patches, though 
of course there is little point if there is no reaction from the reviewers.
If it is of any interest, I want to write a shim library for Capstone and would 
prefer not to have three copies of Std functions in my tree (StdLib, CryptoPkg 
(SSL) and Capstone), but rather an improved StdLib that works for all.

Thanks to those who have offered alternative solutions, but I prefer to keep it 
‚clean‘ and have the libraries shipping with EDK2 work. :)

Regards,
Marvin.

From: Michael Zimmermann [mailto:sigmaepsilo...@gmail.com]
Sent: Thursday, June 23, 2016 5:32 AM
To: Marvin Häuser <marvin.haeu...@outlook.com>
Cc: edk2-devel@lists.01.org; af...@apple.com
Subject: Re: [edk2] StdLib usage for drivers?

well for the patch to go upstream it would have to get improved a lot.
I've tried to implement this in a way that doesn't need a different dsc but the 
problem is that ShellLib's constructor ASSERT's if it can't find the shell 
protocol and removing that would probably be against the spec's.

Also it appears that the StdLib maintainers are kinda busy because most of the 
StdLib patches I've sent to this mailing list didn't get reviewed.

Thanks
Michael

On Wed, Jun 22, 2016 at 10:37 PM, Marvin Häuser 
<marvin.haeu...@outlook.com<mailto:marvin.haeu...@outlook.com>> wrote:
Hey Michael,

Thank you for your input! This looks interesting.
Maybe it would be a good idea to provide the libraries that depend on Shell (in 
master) with functions that call ASSERT (FALSE); for drivers? I do not need 
file I/O, so I think your modifications might work out well for me. Would be 
very nice of course if such changes found their way upstream. :)

And thank you very much for your comment as well, Andrew! It makes sense that 
StdLib is primarily targeted at porting console applications to UEFI Shell.

Regards,
Marvin.

From: Michael Zimmermann 
[mailto:sigmaepsilo...@gmail.com<mailto:sigmaepsilo...@gmail.com>]
Sent: Wednesday, June 22, 2016 10:28 PM
To: Marvin H?user 
<marvin.haeu...@outlook.com<mailto:marvin.haeu...@outlook.com>>
Cc: edk2-devel@lists.01.org<mailto:edk2-devel@lists.01.org>
Subject: Re: [edk2] StdLib usage for drivers?

In my fork of edk2 I've added support for using StdLib without the Shell.
It's quite hacky(setenv/getenv are stubs, no File IO and maybe other things 
hidden by linker GC and me not using all features).

But depending on which StdLib features you need this can work pretty good.

here's the commit that does the magic:
https://github.com/efidroid/edk2/commit/bf7a296718486bafaf774ea8bcf187c162c3c167

and this is how you convert a Shell project to a NonShell one:
https://github.com/efidroid/uefi_apps_EFIDroidUi/commit/23f0fa08108b8f852564fae733c6a7bce62e2070

as you can see it works by using StdLib with different libraries/cflags so you 
have to compile your driver separately if there are other modules which need 
the normal StdLib.

Thanks
Michael

On Wed, Jun 22, 2016 at 9:38 PM, Marvin H?user 
<marvin.haeu...@outlook.com<mailto:marvin.haeu...@outlook.com><mailto:marvin.haeu...@outlook.com<mailto:marvin.haeu...@outlook.com>>>
 wrote:
Dear EDK2 developers,

For an experimental project, I'm currently attempting to write a library 
wrapper for the disassembler library 'Capstone' in a similar manner to 
CryptoPkg's OpensslLib. As most C libraries, it also depends on the standard 
headers, which are not provided by 'stock' EDK2. My first guess has been to use 
StdLib, though its description states:
'Due to the execution environment built by the StdLib component, execution as a 
UEFI driver can cause system stability issues.'

Inspecting OpensslLib I discovered that CryptoPkg deploys its own include files 
for StdLib. Though, from your experience, what are the issues with using 
StdLibPkg with DXE/UEFI drivers? It might be nice to reduce duplicate code, 
though I honestly don't know anything about StdLibPkg and its implementation 
and would be thankful for some insight on that manner.

Thank you in advance for your time!

Regards,
Marvin.
___
edk2-devel mailing list
edk2-devel@lists.01.org<mailto:edk2-devel@lists.01.org><mailto:edk2-devel@lists.01.org<mailto:edk2-devel@lists.01.org>>
https://lists.01.org/mailman/listinfo/edk2-devel

___
edk2-devel mailing list
edk2-devel@lists.01.org<mailto:edk2-devel@lists.01.org>
https://lists.01.org/mailman/listinfo/edk2-devel

___
edk2-devel mailing list
edk2-devel@lists.01.org
https://lists.01.org/mailman/listinfo/edk2-devel

Re: [edk2] StdLib usage for drivers?

[Brian J. Johnson]

On 06/22/2016 02:38 PM, Marvin H?user wrote:

Dear EDK2 developers,

For an experimental project, I'm currently attempting to write a
library wrapper for the disassembler library 'Capstone' in a similar
manner to CryptoPkg's OpensslLib. ...


Marvin,

If you only need IA32/X64, the Udis86 library 
(http://udis86.sourceforge.net/) may be a good alternative.  I was able 
to write a similar UEFI library wrapper for it quite easily.  I think 
all it needed were integer types, varargs, memset, snprintf, and 
vsnprintf, along with a few other small fixes (like 64-bit shifts and 
replacing "%s" with "%a" in format strings.)

--

Brian J. Johnson



  My statements are my own, are not authorized by SGI, and do not
  necessarily represent SGI’s positions.
___
edk2-devel mailing list
edk2-devel@lists.01.org
https://lists.01.org/mailman/listinfo/edk2-devel

Re: [edk2] StdLib usage for drivers?

[Michael Zimmermann]

well for the patch to go upstream it would have to get improved a lot.
I've tried to implement this in a way that doesn't need a different dsc but
the problem is that ShellLib's constructor ASSERT's if it can't find the
shell protocol and removing that would probably be against the spec's.

Also it appears that the StdLib maintainers are kinda busy because most of
the StdLib patches I've sent to this mailing list didn't get reviewed.

Thanks
Michael

On Wed, Jun 22, 2016 at 10:37 PM, Marvin Häuser <marvin.haeu...@outlook.com>
wrote:

> Hey Michael,
>
> Thank you for your input! This looks interesting.
> Maybe it would be a good idea to provide the libraries that depend on
> Shell (in master) with functions that call ASSERT (FALSE); for drivers? I
> do not need file I/O, so I think your modifications might work out well for
> me. Would be very nice of course if such changes found their way upstream.
> :)
>
> And thank you very much for your comment as well, Andrew! It makes sense
> that StdLib is primarily targeted at porting console applications to UEFI
> Shell.
>
> Regards,
> Marvin.
>
> From: Michael Zimmermann [mailto:sigmaepsilo...@gmail.com]
> Sent: Wednesday, June 22, 2016 10:28 PM
> To: Marvin H?user <marvin.haeu...@outlook.com>
> Cc: edk2-devel@lists.01.org
> Subject: Re: [edk2] StdLib usage for drivers?
>
> In my fork of edk2 I've added support for using StdLib without the Shell.
> It's quite hacky(setenv/getenv are stubs, no File IO and maybe other
> things hidden by linker GC and me not using all features).
>
> But depending on which StdLib features you need this can work pretty good.
>
> here's the commit that does the magic:
>
> https://github.com/efidroid/edk2/commit/bf7a296718486bafaf774ea8bcf187c162c3c167
>
> and this is how you convert a Shell project to a NonShell one:
>
> https://github.com/efidroid/uefi_apps_EFIDroidUi/commit/23f0fa08108b8f852564fae733c6a7bce62e2070
>
> as you can see it works by using StdLib with different libraries/cflags so
> you have to compile your driver separately if there are other modules which
> need the normal StdLib.
>
> Thanks
> Michael
>
> On Wed, Jun 22, 2016 at 9:38 PM, Marvin H?user <marvin.haeu...@outlook.com
> <mailto:marvin.haeu...@outlook.com>> wrote:
> Dear EDK2 developers,
>
> For an experimental project, I'm currently attempting to write a library
> wrapper for the disassembler library 'Capstone' in a similar manner to
> CryptoPkg's OpensslLib. As most C libraries, it also depends on the
> standard headers, which are not provided by 'stock' EDK2. My first guess
> has been to use StdLib, though its description states:
> 'Due to the execution environment built by the StdLib component, execution
> as a UEFI driver can cause system stability issues.'
>
> Inspecting OpensslLib I discovered that CryptoPkg deploys its own include
> files for StdLib. Though, from your experience, what are the issues with
> using StdLibPkg with DXE/UEFI drivers? It might be nice to reduce duplicate
> code, though I honestly don't know anything about StdLibPkg and its
> implementation and would be thankful for some insight on that manner.
>
> Thank you in advance for your time!
>
> Regards,
> Marvin.
> ___
> edk2-devel mailing list
> edk2-devel@lists.01.org<mailto:edk2-devel@lists.01.org>
> https://lists.01.org/mailman/listinfo/edk2-devel
>
> ___
> edk2-devel mailing list
> edk2-devel@lists.01.org
> https://lists.01.org/mailman/listinfo/edk2-devel
>
___
edk2-devel mailing list
edk2-devel@lists.01.org
https://lists.01.org/mailman/listinfo/edk2-devel

Re: [edk2] StdLib usage for drivers?

[Marvin Häuser]

Hey Michael,

Thank you for your input! This looks interesting.
Maybe it would be a good idea to provide the libraries that depend on Shell (in 
master) with functions that call ASSERT (FALSE); for drivers? I do not need 
file I/O, so I think your modifications might work out well for me. Would be 
very nice of course if such changes found their way upstream. :)

And thank you very much for your comment as well, Andrew! It makes sense that 
StdLib is primarily targeted at porting console applications to UEFI Shell.

Regards,
Marvin.

From: Michael Zimmermann [mailto:sigmaepsilo...@gmail.com]
Sent: Wednesday, June 22, 2016 10:28 PM
To: Marvin H?user <marvin.haeu...@outlook.com>
Cc: edk2-devel@lists.01.org
Subject: Re: [edk2] StdLib usage for drivers?

In my fork of edk2 I've added support for using StdLib without the Shell.
It's quite hacky(setenv/getenv are stubs, no File IO and maybe other things 
hidden by linker GC and me not using all features).

But depending on which StdLib features you need this can work pretty good.

here's the commit that does the magic:
https://github.com/efidroid/edk2/commit/bf7a296718486bafaf774ea8bcf187c162c3c167

and this is how you convert a Shell project to a NonShell one:
https://github.com/efidroid/uefi_apps_EFIDroidUi/commit/23f0fa08108b8f852564fae733c6a7bce62e2070

as you can see it works by using StdLib with different libraries/cflags so you 
have to compile your driver separately if there are other modules which need 
the normal StdLib.

Thanks
Michael

On Wed, Jun 22, 2016 at 9:38 PM, Marvin H?user 
<marvin.haeu...@outlook.com<mailto:marvin.haeu...@outlook.com>> wrote:
Dear EDK2 developers,

For an experimental project, I'm currently attempting to write a library 
wrapper for the disassembler library 'Capstone' in a similar manner to 
CryptoPkg's OpensslLib. As most C libraries, it also depends on the standard 
headers, which are not provided by 'stock' EDK2. My first guess has been to use 
StdLib, though its description states:
'Due to the execution environment built by the StdLib component, execution as a 
UEFI driver can cause system stability issues.'

Inspecting OpensslLib I discovered that CryptoPkg deploys its own include files 
for StdLib. Though, from your experience, what are the issues with using 
StdLibPkg with DXE/UEFI drivers? It might be nice to reduce duplicate code, 
though I honestly don't know anything about StdLibPkg and its implementation 
and would be thankful for some insight on that manner.

Thank you in advance for your time!

Regards,
Marvin.
___
edk2-devel mailing list
edk2-devel@lists.01.org<mailto:edk2-devel@lists.01.org>
https://lists.01.org/mailman/listinfo/edk2-devel

___
edk2-devel mailing list
edk2-devel@lists.01.org
https://lists.01.org/mailman/listinfo/edk2-devel

Re: [edk2] StdLib usage for drivers?

[Michael Zimmermann]

In my fork of edk2 I've added support for using StdLib without the Shell.
It's quite hacky(setenv/getenv are stubs, no File IO and maybe other things
hidden by linker GC and me not using all features).

But depending on which StdLib features you need this can work pretty good.

here's the commit that does the magic:
https://github.com/efidroid/edk2/commit/bf7a296718486bafaf774ea8bcf187c162c3c167

and this is how you convert a Shell project to a NonShell one:
https://github.com/efidroid/uefi_apps_EFIDroidUi/commit/23f0fa08108b8f852564fae733c6a7bce62e2070

as you can see it works by using StdLib with different libraries/cflags so
you have to compile your driver separately if there are other modules which
need the normal StdLib.

Thanks
Michael

On Wed, Jun 22, 2016 at 9:38 PM, Marvin H?user 
wrote:

> Dear EDK2 developers,
>
> For an experimental project, I'm currently attempting to write a library
> wrapper for the disassembler library 'Capstone' in a similar manner to
> CryptoPkg's OpensslLib. As most C libraries, it also depends on the
> standard headers, which are not provided by 'stock' EDK2. My first guess
> has been to use StdLib, though its description states:
> 'Due to the execution environment built by the StdLib component, execution
> as a UEFI driver can cause system stability issues.'
>
> Inspecting OpensslLib I discovered that CryptoPkg deploys its own include
> files for StdLib. Though, from your experience, what are the issues with
> using StdLibPkg with DXE/UEFI drivers? It might be nice to reduce duplicate
> code, though I honestly don't know anything about StdLibPkg and its
> implementation and would be thankful for some insight on that manner.
>
> Thank you in advance for your time!
>
> Regards,
> Marvin.
> ___
> edk2-devel mailing list
> edk2-devel@lists.01.org
> https://lists.01.org/mailman/listinfo/edk2-devel
>
___
edk2-devel mailing list
edk2-devel@lists.01.org
https://lists.01.org/mailman/listinfo/edk2-devel

Re: [edk2] StdLib usage for drivers?

[Andrew Fish]

> On Jun 22, 2016, at 12:38 PM, Marvin H?user  
> wrote:
> 
> Dear EDK2 developers,
> 
> For an experimental project, I'm currently attempting to write a library 
> wrapper for the disassembler library 'Capstone' in a similar manner to 
> CryptoPkg's OpensslLib. As most C libraries, it also depends on the standard 
> headers, which are not provided by 'stock' EDK2. My first guess has been to 
> use StdLib, though its description states:
> 'Due to the execution environment built by the StdLib component, execution as 
> a UEFI driver can cause system stability issues.'
> 
> Inspecting OpensslLib I discovered that CryptoPkg deploys its own include 
> files for StdLib. Though, from your experience, what are the issues with 
> using StdLibPkg with DXE/UEFI drivers? It might be nice to reduce duplicate 
> code, though I honestly don't know anything about StdLibPkg and its 
> implementation and would be thankful for some insight on that manner.
> 

Marvin,

The edk2 StdLib has a dependency on EFI Shell interfaces and this is what makes 
it not work properly as a driver. If you cd int StdLib and `git grep shell` you 
will see the platform abstractions are not pure EFI, but Shell oriented. 

The Shell adds concepts like environment variables and volume names that don't 
exist in EFI. 

Thanks,

Andrew Fish

> Thank you in advance for your time!
> 
> Regards,
> Marvin.
> ___
> edk2-devel mailing list
> edk2-devel@lists.01.org
> https://lists.01.org/mailman/listinfo/edk2-devel

___
edk2-devel mailing list
edk2-devel@lists.01.org
https://lists.01.org/mailman/listinfo/edk2-devel

[edk2] StdLib usage for drivers?

[Marvin H?user]

Dear EDK2 developers,

For an experimental project, I'm currently attempting to write a library 
wrapper for the disassembler library 'Capstone' in a similar manner to 
CryptoPkg's OpensslLib. As most C libraries, it also depends on the standard 
headers, which are not provided by 'stock' EDK2. My first guess has been to use 
StdLib, though its description states:
'Due to the execution environment built by the StdLib component, execution as a 
UEFI driver can cause system stability issues.'

Inspecting OpensslLib I discovered that CryptoPkg deploys its own include files 
for StdLib. Though, from your experience, what are the issues with using 
StdLibPkg with DXE/UEFI drivers? It might be nice to reduce duplicate code, 
though I honestly don't know anything about StdLibPkg and its implementation 
and would be thankful for some insight on that manner.

Thank you in advance for your time!

Regards,
Marvin.
___
edk2-devel mailing list
edk2-devel@lists.01.org
https://lists.01.org/mailman/listinfo/edk2-devel