Re: [edk2] BaseTools features: multiple workspaces

[Gao, Liming]

I am fine with PACKAGES_PATH. If $(PACKAGES_PATH) exists, Build directory will 
be in $(WORKSPACE), Conf, BaseTools and Package directories will be searched 
from $(PACKAGES_PATH). If $(PACKAGES_PATH) doesn't exist, all of them will be 
in $(WORKSPACE) same to current usage model. 

For the below case, based on the above rule. 
set WORKSPACE=C:\src\udk
set PACKAGES_PATH=C:\udk\2014; C:\udk\2014\src; C:\src\udk\platform

Build: C:\src\udk\build
Conf: C:\udk\2014\conf
BaseTools: C:\udk\2014\bin
udk packages: C:\udk\2014\src
platform packages: C:\src\udk\platform

Thanks
Liming
-Original Message-
From: Laszlo Ersek [mailto:ler...@redhat.com] 
Sent: Thursday, August 06, 2015 4:31 PM
To: Justen, Jordan L; Andrew Fish
Cc: Gao, Liming; Tim Lewis; Paolo Bonzini; edk2-devel@lists.01.org
Subject: Re: [edk2] BaseTools features: multiple workspaces

On 08/05/15 23:46, Jordan Justen wrote:
 On 2015-08-05 13:21:37, Andrew Fish wrote:
 On Aug 5, 2015, at 12:16 PM, Jordan Justen jordan.l.jus...@intel.com 
 wrote:
 On 2015-08-05 07:06:46, Gao, Liming wrote:
 Tim gave another idea to define multiple source locations in 
 Platform DSC file. It doesn't conflict with this proposal. We can 
 implement $(WORKSPACE_MULTIPLE) feature first, and evaluate DSC 
 update. On the fixed tree layout, these two solutions can work. 
 Once the end user reorganizes its tree layout, he needs to set
 $(WORKSPACE_MULTIPLE) env or update Platform DSC file, and build 
 platform DSC.

 At one point Mike suggested updating target.txt to be able to add 
 paths.

 I think this might be better than hard coding them into the dsc 
 (source code) files.

 Tim, any thoughts on this?

 I’m not Tim, but PLEASE NOOO!!

 We don’t want “one tree per product”, we want multiple products per 
 tree. target.txt is really helpful if you want to save typing on the 
 command line, but that is about it.
 
 I happen to agree. I haven't used target.txt in years. I love being 
 able to run these two commands easily in the same tree:
 
 $ OvmfPkg/build.sh
 
 $ EmulatorPkg/build.sh

Same here. I used to rely on target.txt, but I've migrated to complete build 
command lines in my scripts. It is more flexible and it gives me more feedback 
(with set -x in effect).

I have scripts for OVMF (Ia32, Ia32X64, X64), ArmVirtQemu (cross compiling for 
Aarch64) and ArmVirtXen (cross compiling for Aarch64, just for build-testing). 
All that wouldn't fit in target.txt anyway, so I decided to specify everything 
on the command line (even options where I don't diverge from the target.txt 
default).

Thanks
Laszlo

 
 If anything, I'd like to see us make the Conf directory optional. (And 
 use BaseTools/Conf by default.)
 
 Now I have seen some vendors that do sill stuff like write a script 
 that patches target.txt and then calls build.exe. I say silly as the 
 script could have just passed the arguments build.exe directly, as 
 target.txt is just the default value for the args of the build 
 command.
 
 Apparently we have a 'TargetTool' that helps with this? Who knew? (Not 
 I ... seriously.)
 
 Anyway, I'm just responding to not wanting to bake hard coded package 
 paths into the .dsc. Not that I like target.txt. :)
 
 -Jordan
 

___
edk2-devel mailing list
edk2-devel@lists.01.org
https://lists.01.org/mailman/listinfo/edk2-devel

Re: [edk2] BaseTools features: multiple workspaces

[Andrew Fish]

 On Aug 5, 2015, at 8:38 AM, Kinney, Michael D michael.d.kin...@intel.com 
 wrote:
 
 Andrew,
 
 I would recommend a DSC define setting have precedence over the 
 WORKSPACE_MULTIPLE environment variable.  This way, a single development 
 environment can support many platform DSC files.  Most of the DSC files can 
 choose to let the WORKSPACE_MULTIPLE environment variable select the dirs, 
 and only platform DSC files that need dirs different than WORKSPACE_MULTIPLE 
 environment variable should set the DSC file define.
 
 I would also recommend that the priority of the paths be the same for both 
 the WORKSPACE_MULTIPLE environment variable and a DSC define.
 

Mike,

Well in reality the priority is not the same as 1st match wins (well we need to 
define how that works too). So an “override” is possible if a module is in 
multiple paths. 

So the question is if the DSC entry exists:
1) It overrides WORKSPACE_MULTIPLE - prevents side effects from env breaking 
build. 
2) It appends to  $WORKSPACE_MULTIPLE, so env overrides DSC
3) it pre-pends to $WORKSPACE_MULTIPLE, so DSC overrides env. 

Thanks,

Andrew Fish


 One interesting issue is the selection of a path separator in the DSC file 
 define syntax.  
 
 Best regards,
 
 Mike
 
 
 -Original Message-
 From: edk2-devel [mailto:edk2-devel-boun...@lists.01.org] On Behalf Of Andrew 
 Fish
 Sent: Wednesday, August 05, 2015 8:28 AM
 To: Gao, Liming
 Cc: Justen, Jordan L; edk2-devel@lists.01.org; Laszlo Ersek; Paolo Bonzini
 Subject: Re: [edk2] BaseTools features: multiple workspaces
 
 
 On Aug 5, 2015, at 7:06 AM, Gao, Liming liming@intel.com wrote:
 
 Thanks for your all comments. 
 
 Most people prefers to keep WOSKAPCE as single directory, and introduce new 
 env $(WORKSPACE_MULTIPLE). I think Mike solution can make them more clear if 
 WORKSPACE_MULTIPLE is set. 
 
 On QA team impact, it should be small. QA team can still use the current 
 working model to place all packages in root directory, and set WORKSPACE 
 only. QA team just needs to know the package list. 
 
 Tim gave another idea to define multiple source locations in Platform DSC 
 file. It doesn't conflict with this proposal. We can implement 
 $(WORKSPACE_MULTIPLE) feature first, and evaluate DSC update. On the fixed 
 tree layout, these two solutions can work. Once the end user reorganizes its 
 tree layout, he needs to set $(WORKSPACE_MULTIPLE) env or update Platform 
 DSC file, and build platform DSC.
 
 
 We should probably decided up front which has precedence the env variable or 
 the DSC entry. 
 
 It might not hurt to think about the DSC case when implementing the env case, 
 as some arbitrary choices in the env path may make the DSC path harder. 
 
 Thanks,
 
 Andrew Fish
 
 Thanks
 Liming
 -Original Message-
 From: Laszlo Ersek [mailto:ler...@redhat.com] 
 Sent: Tuesday, August 04, 2015 7:20 PM
 To: Justen, Jordan L
 Cc: Gao, Liming; Paolo Bonzini; edk2-devel@lists.01.org
 Subject: Re: [edk2] BaseTools features: multiple workspaces
 
 On 08/03/15 19:35, Jordan Justen wrote:
 On 2015-08-03 02:08:14, Gao, Liming wrote:
 Paolo:
 I think that keep the exiting syntax for WORKSPACE to be a single
 path can minimize the impacts to exiting tools that assume a
 single workspace.
 
 I know you originally suggested to use WORKSPACE, but got some 
 feedback that it might be better to create a new variable to not break 
 some unspecified tools.
 
 Was anyone able to name an actual tool that would be impacted? Why not 
 work to update these tools at the same time? An easy 'fix' will be for 
 any such tool to give an error message if it sees ':' or ';' in the 
 WORKSPACE environment variable. Although this is really not a fix, it 
 will be just as much support as if they ignore the new 
 WORKSPACE_MULTIPLE environment variable.
 
 I think that using the single separated WORKSPACE environment variable 
 is more intuitive/expected. From using env vars such as PATH, we all 
 know what it means.
 
 For WORKSPACE+WORKSPACE_MULTIPLE, I don't know which has priority. I 
 assume WORKSPACE is checked first, and then WORKSPACE_MULTIPLE?
 
 This was described in Liming's email, in point 4b.
 
 
 Will this work for the integrated CryptoPkg + open-ssl? It seems like 
 we need a priority above EDK II for this. I guess WORKSPACE will not 
 be set to EDK II in this case?
 
 Anyway, it just seems a little odd.
 
 For me a colon-separated (well, separator-separated :)) single WORKSPACE 
 variable would be more intuitive, but WORKSPACE_MULTIPLE can be defined well 
 enough too. For example:
 - WORKSPACE must not contain separators
 - if WORKSPACE_MULTIPLE is set, then the effect is as if
 ${WORKSPACE}:${WORKSPACE_MULTIPLE} were specified under the
 well-known PATH semantics.
 
 No clue about the CryptoPkg impact. Can you remind me please why CryptoPkg / 
 OpensslLib are special wrt. WORKSPACE?
 
 Thanks
 Laszlo
 
 
 -Jordan
 
 -Original Message-
 From: edk2-devel [mailto:edk2-devel-boun

Re: [edk2] BaseTools features: multiple workspaces

[Jordan Justen]

On 2015-08-05 07:06:46, Gao, Liming wrote:
 Thanks for your all comments. 
 
 Most people prefers to keep WOSKAPCE as single directory, and
 introduce new env $(WORKSPACE_MULTIPLE).

I'm not sure about this conclusion. I think Paolo, Laszlo and I all
expressed that using WORKSPACE for the multiple directories seemed
more natural.

I think you originally suggested just using WORKSPACE, but maybe now
have changed your mind.

I know Tim had some big concerns with expanding WORKSPACE. I agree
with his concerns, but I don't think it will be much better with
WORKSPACE_MULTIPLE. (The same tools will break.)

Mike and Andrew appear to prefer WORKSPACE_MULTIPLE.

What about this suggestion?

WORKSPACE: expand to a list of directories

WORKSPACE_BUILD: if not set, use first path in WORKSPACE + '/Build'

WORKSPACE_CONF: if not set, scan WORKSPACE paths to find 'Conf' dir

WORKSPACE_TOOLS: if not set, scan WORKSPACE paths to find 'BaseTools' dir

With WORKSPACE and WORKSPACE_MULTIPLE we have a strange intersection
of 'part of the source tree' and 'all of the source tree'. I think the
variables I recommend avoid that partial overlap.

-Jordan

 I think Mike solution can
 make them more clear if WORKSPACE_MULTIPLE is set.
 
 On QA team impact, it should be small. QA team can still use the
 current working model to place all packages in root directory, and
 set WORKSPACE only. QA team just needs to know the package list.
 
 Tim gave another idea to define multiple source locations in
 Platform DSC file. It doesn't conflict with this proposal. We can
 implement $(WORKSPACE_MULTIPLE) feature first, and evaluate DSC
 update. On the fixed tree layout, these two solutions can work. Once
 the end user reorganizes its tree layout, he needs to set
 $(WORKSPACE_MULTIPLE) env or update Platform DSC file, and build
 platform DSC.
 
 Thanks
 Liming
 -Original Message-
 From: Laszlo Ersek [mailto:ler...@redhat.com] 
 Sent: Tuesday, August 04, 2015 7:20 PM
 To: Justen, Jordan L
 Cc: Gao, Liming; Paolo Bonzini; edk2-devel@lists.01.org
 Subject: Re: [edk2] BaseTools features: multiple workspaces
 
 On 08/03/15 19:35, Jordan Justen wrote:
  On 2015-08-03 02:08:14, Gao, Liming wrote:
  Paolo:
I think that keep the exiting syntax for WORKSPACE to be a single
path can minimize the impacts to exiting tools that assume a
single workspace.
  
  I know you originally suggested to use WORKSPACE, but got some 
  feedback that it might be better to create a new variable to not break 
  some unspecified tools.
  
  Was anyone able to name an actual tool that would be impacted? Why not 
  work to update these tools at the same time? An easy 'fix' will be for 
  any such tool to give an error message if it sees ':' or ';' in the 
  WORKSPACE environment variable. Although this is really not a fix, it 
  will be just as much support as if they ignore the new 
  WORKSPACE_MULTIPLE environment variable.
  
  I think that using the single separated WORKSPACE environment variable 
  is more intuitive/expected. From using env vars such as PATH, we all 
  know what it means.
  
  For WORKSPACE+WORKSPACE_MULTIPLE, I don't know which has priority. I 
  assume WORKSPACE is checked first, and then WORKSPACE_MULTIPLE?
 
 This was described in Liming's email, in point 4b.
 
  
  Will this work for the integrated CryptoPkg + open-ssl? It seems like 
  we need a priority above EDK II for this. I guess WORKSPACE will not 
  be set to EDK II in this case?
  
  Anyway, it just seems a little odd.
 
 For me a colon-separated (well, separator-separated :)) single WORKSPACE 
 variable would be more intuitive, but WORKSPACE_MULTIPLE can be defined well 
 enough too. For example:
 - WORKSPACE must not contain separators
 - if WORKSPACE_MULTIPLE is set, then the effect is as if
   ${WORKSPACE}:${WORKSPACE_MULTIPLE} were specified under the
   well-known PATH semantics.
 
 No clue about the CryptoPkg impact. Can you remind me please why CryptoPkg / 
 OpensslLib are special wrt. WORKSPACE?
 
 Thanks
 Laszlo
 
  
  -Jordan
  
  -Original Message-
  From: edk2-devel [mailto:edk2-devel-boun...@lists.01.org] On Behalf 
  Of Paolo Bonzini
  Sent: Monday, August 03, 2015 4:31 PM
  To: Gao, Liming; edk2-devel@lists.01.org
  Subject: Re: [edk2] BaseTools features: multiple workspaces
 
 
  On 03/08/2015 05:56, Gao, Liming wrote:
  Hi, all
We will update BaseTools feature to allow more than one workspaces. The 
  detail design in the below. Please help review it. If you have any 
  comments, please let me know.
 
  1.   Keep $(WORKSPACE) environment as is
 
  a.   $(WORKSPACE) determines location of Build and Conf directory.
 
  2.   New optional $(WORKSPACE_MULTIPLE) environment is added to 
  include more directories with the separator ';', like $(PATH)
 
  Why is $(WORKSPACE_MULTIPLE) necessary?  On Linux it is okay to break 
  preexisting paths that contain a :; on Windows semicolons are allowed in 
  theory but in practice break

Re: [edk2] BaseTools features: multiple workspaces

[Andrew Fish]

:@if exist %WORKSPACE%\unitool.log del 
%WORKSPACE%\unitool.log
Vlv2TbltDevicePkg/bldX64.bat:20:@if exist %WORKSPACE%\Conf\build_rule.txt del 
%WORKSPACE%\Conf\build_rule.txt
Vlv2TbltDevicePkg/bldX64.bat:21:@if exist %WORKSPACE%\Conf\FrameworkDatabase.db 
del %WORKSPACE%\Conf\FrameworkDatabase.db
Vlv2TbltDevicePkg/bldX64.bat:22:@if exist %WORKSPACE%\Conf\target.txt del 
%WORKSPACE%\Conf\target.txt
Vlv2TbltDevicePkg/bldX64.bat:23:@if exist %WORKSPACE%\Conf\tools_def.txt del 
%WORKSPACE%\Conf\tools_def.txt
Vlv2TbltDevicePkg/bld_vlv.bat:23:set WORKSPACE=%CD%
Vlv2TbltDevicePkg/bld_vlv.bat:24:if %WORKSPACE:~-1%==\ set 
WORKSPACE=%WORKSPACE:~0,-1%
Vlv2TbltDevicePkg/bld_vlv.bat:32:if exist %WORKSPACE%\edk2.log del 
%WORKSPACE%\edk2.log
Vlv2TbltDevicePkg/bld_vlv.bat:33:if exist %WORKSPACE%\unitool.log del 
%WORKSPACE%\unitool.log
Vlv2TbltDevicePkg/bld_vlv.bat:34:if exist %WORKSPACE%\Conf\target.txt del 
%WORKSPACE%\Conf\target.txt
Vlv2TbltDevicePkg/bld_vlv.bat:35:if exist %WORKSPACE%\Conf\tools_def.txt del 
%WORKSPACE%\Conf\tools_def.txt
Vlv2TbltDevicePkg/bld_vlv.bat:36:if exist %WORKSPACE%\Conf\build_rule.txt del 
%WORKSPACE%\Conf\build_rule.txt
Vlv2TbltDevicePkg/bld_vlv.bat:37:if exist %WORKSPACE%\Conf\FrameworkDatabase.db 
del %WORKSPACE%\Conf\FrameworkDatabase.db
Vlv2TbltDevicePkg/bld_vlv.bat:50:set EDK_SOURCE=%WORKSPACE%\EdkCompatibilityPkg
Vlv2TbltDevicePkg/bld_vlv.bat:236:copy /y/b %BUILD_PATH%\FV\Vlv%Arch%.fd  
%WORKSPACE%\%BIOS_Name% nul
Vlv2TbltDevicePkg/bld_vlv.sh:27:#WORKSPACE=$(pwd)
Vlv2TbltDevicePkg/bld_vlv.sh:66:config_file=$WORKSPACE/$PLATFORM_PACKAGE/PlatformPkgConfig.dsc
Vlv2TbltDevicePkg/bld_vlv.sh:67:auto_config_inc=$WORKSPACE/$PLATFORM_PACKAGE/AutoPlatformCFG.txt
Vlv2TbltDevicePkg/bld_vlv.sh:70:EDK_SOURCE=$WORKSPACE/EdkCompatibilityPkg
Vlv2TbltDevicePkg/bld_vlv.sh:100:  
ECP_SOURCE=$WORKSPACE/EdkCompatibilityPkgEcp
Vlv2TbltDevicePkg/bld_vlv.sh:101:  
EDK_SOURCE=$WORKSPACE/EdkCompatibilityPkgEcp
Vlv2TbltDevicePkg/bld_vlv.sh:224:cp -f $BUILD_PATH/FV/VLV.fd  
$WORKSPACE/$BIOS_Name
...

Thanks,

Andrew Fish

 I think you originally suggested just using WORKSPACE, but maybe now
 have changed your mind.
 
 I know Tim had some big concerns with expanding WORKSPACE. I agree
 with his concerns, but I don't think it will be much better with
 WORKSPACE_MULTIPLE. (The same tools will break.)
 
 Mike and Andrew appear to prefer WORKSPACE_MULTIPLE.
 
 What about this suggestion?
 
 WORKSPACE: expand to a list of directories
 
 WORKSPACE_BUILD: if not set, use first path in WORKSPACE + '/Build’
 

Build is not architectural, this is a value in the DSC. 

 WORKSPACE_CONF: if not set, scan WORKSPACE paths to find 'Conf' dir
 
 WORKSPACE_TOOLS: if not set, scan WORKSPACE paths to find 'BaseTools' dir
 
 With WORKSPACE and WORKSPACE_MULTIPLE we have a strange intersection
 of 'part of the source tree' and 'all of the source tree'. I think the
 variables I recommend avoid that partial overlap.
 

We have the current stuff all works, and you can opt into a new feature if 
needed. You guys want to break the world to make it more pedantically correct? 
What do we gain from breaking a large number of build targets? 

Thanks,

Andrew Fish

 -Jordan
 
 I think Mike solution can
 make them more clear if WORKSPACE_MULTIPLE is set.
 
 On QA team impact, it should be small. QA team can still use the
 current working model to place all packages in root directory, and
 set WORKSPACE only. QA team just needs to know the package list.
 
 Tim gave another idea to define multiple source locations in
 Platform DSC file. It doesn't conflict with this proposal. We can
 implement $(WORKSPACE_MULTIPLE) feature first, and evaluate DSC
 update. On the fixed tree layout, these two solutions can work. Once
 the end user reorganizes its tree layout, he needs to set
 $(WORKSPACE_MULTIPLE) env or update Platform DSC file, and build
 platform DSC.
 
 Thanks
 Liming
 -Original Message-
 From: Laszlo Ersek [mailto:ler...@redhat.com] 
 Sent: Tuesday, August 04, 2015 7:20 PM
 To: Justen, Jordan L
 Cc: Gao, Liming; Paolo Bonzini; edk2-devel@lists.01.org
 Subject: Re: [edk2] BaseTools features: multiple workspaces
 
 On 08/03/15 19:35, Jordan Justen wrote:
 On 2015-08-03 02:08:14, Gao, Liming wrote:
 Paolo:
  I think that keep the exiting syntax for WORKSPACE to be a single
  path can minimize the impacts to exiting tools that assume a
  single workspace.
 
 I know you originally suggested to use WORKSPACE, but got some 
 feedback that it might be better to create a new variable to not break 
 some unspecified tools.
 
 Was anyone able to name an actual tool that would be impacted? Why not 
 work to update these tools at the same time? An easy 'fix' will be for 
 any such tool to give an error message if it sees ':' or ';' in the 
 WORKSPACE environment variable. Although this is really not a fix, it 
 will be just as much support as if they ignore the new 
 WORKSPACE_MULTIPLE environment variable.
 
 I think that using the single

Re: [edk2] BaseTools features: multiple workspaces

[Kinney, Michael D]

Jordan,

We already have EDK_TOOLS_PATH for the path to the tools, so I don't think we 
should add WORKSPACE_TOOLS.

The DSC file specifies the output directory and the syntax supports both 
workspace relative paths and absolute paths.  I am not sure I understand what 
issue is addressed by adding WORKSPACE_BUILD.  If DSC specifies output 
directory using a workspace relative path, then tools use WORKSPACE.  If DSC 
uses absolute path for the output path, then the output can be places anywhere. 
 

The build command supports the --conf flag, so alternate CONF directories can 
be specified on the build command line.  I am not sure I understand what issue 
is addressed by adding WORKSPACE_CONF.

Best regards,

Mike

-Original Message-
From: edk2-devel [mailto:edk2-devel-boun...@lists.01.org] On Behalf Of Jordan 
Justen
Sent: Wednesday, August 05, 2015 12:09 PM
To: Gao, Liming; Laszlo Ersek
Cc: Paolo Bonzini; edk2-devel@lists.01.org
Subject: Re: [edk2] BaseTools features: multiple workspaces

On 2015-08-05 07:06:46, Gao, Liming wrote:
 Thanks for your all comments. 
 
 Most people prefers to keep WOSKAPCE as single directory, and
 introduce new env $(WORKSPACE_MULTIPLE).

I'm not sure about this conclusion. I think Paolo, Laszlo and I all
expressed that using WORKSPACE for the multiple directories seemed
more natural.

I think you originally suggested just using WORKSPACE, but maybe now
have changed your mind.

I know Tim had some big concerns with expanding WORKSPACE. I agree
with his concerns, but I don't think it will be much better with
WORKSPACE_MULTIPLE. (The same tools will break.)

Mike and Andrew appear to prefer WORKSPACE_MULTIPLE.

What about this suggestion?

WORKSPACE: expand to a list of directories

WORKSPACE_BUILD: if not set, use first path in WORKSPACE + '/Build'

WORKSPACE_CONF: if not set, scan WORKSPACE paths to find 'Conf' dir

WORKSPACE_TOOLS: if not set, scan WORKSPACE paths to find 'BaseTools' dir

With WORKSPACE and WORKSPACE_MULTIPLE we have a strange intersection
of 'part of the source tree' and 'all of the source tree'. I think the
variables I recommend avoid that partial overlap.

-Jordan

 I think Mike solution can
 make them more clear if WORKSPACE_MULTIPLE is set.
 
 On QA team impact, it should be small. QA team can still use the
 current working model to place all packages in root directory, and
 set WORKSPACE only. QA team just needs to know the package list.
 
 Tim gave another idea to define multiple source locations in
 Platform DSC file. It doesn't conflict with this proposal. We can
 implement $(WORKSPACE_MULTIPLE) feature first, and evaluate DSC
 update. On the fixed tree layout, these two solutions can work. Once
 the end user reorganizes its tree layout, he needs to set
 $(WORKSPACE_MULTIPLE) env or update Platform DSC file, and build
 platform DSC.
 
 Thanks
 Liming
 -Original Message-
 From: Laszlo Ersek [mailto:ler...@redhat.com] 
 Sent: Tuesday, August 04, 2015 7:20 PM
 To: Justen, Jordan L
 Cc: Gao, Liming; Paolo Bonzini; edk2-devel@lists.01.org
 Subject: Re: [edk2] BaseTools features: multiple workspaces
 
 On 08/03/15 19:35, Jordan Justen wrote:
  On 2015-08-03 02:08:14, Gao, Liming wrote:
  Paolo:
I think that keep the exiting syntax for WORKSPACE to be a single
path can minimize the impacts to exiting tools that assume a
single workspace.
  
  I know you originally suggested to use WORKSPACE, but got some 
  feedback that it might be better to create a new variable to not break 
  some unspecified tools.
  
  Was anyone able to name an actual tool that would be impacted? Why not 
  work to update these tools at the same time? An easy 'fix' will be for 
  any such tool to give an error message if it sees ':' or ';' in the 
  WORKSPACE environment variable. Although this is really not a fix, it 
  will be just as much support as if they ignore the new 
  WORKSPACE_MULTIPLE environment variable.
  
  I think that using the single separated WORKSPACE environment variable 
  is more intuitive/expected. From using env vars such as PATH, we all 
  know what it means.
  
  For WORKSPACE+WORKSPACE_MULTIPLE, I don't know which has priority. I 
  assume WORKSPACE is checked first, and then WORKSPACE_MULTIPLE?
 
 This was described in Liming's email, in point 4b.
 
  
  Will this work for the integrated CryptoPkg + open-ssl? It seems like 
  we need a priority above EDK II for this. I guess WORKSPACE will not 
  be set to EDK II in this case?
  
  Anyway, it just seems a little odd.
 
 For me a colon-separated (well, separator-separated :)) single WORKSPACE 
 variable would be more intuitive, but WORKSPACE_MULTIPLE can be defined well 
 enough too. For example:
 - WORKSPACE must not contain separators
 - if WORKSPACE_MULTIPLE is set, then the effect is as if
   ${WORKSPACE}:${WORKSPACE_MULTIPLE} were specified under the
   well-known PATH semantics.
 
 No clue about the CryptoPkg impact. Can you remind me please why CryptoPkg / 
 OpensslLib

Re: [edk2] BaseTools features: multiple workspaces

[Jordan Justen]

/IntelGopDriver.depex
 Vlv2TbltDevicePkg/PlatformPkgGcc.fdf:127:FILE = 
 $(WORKSPACE)/Vlv2TbltDevicePkg/Stitch/Gcc/NvStorageVariable.bin
 Vlv2TbltDevicePkg/PlatformPkgGcc.fdf:131:FILE = 
 $(WORKSPACE)/Vlv2TbltDevicePkg/Stitch/Gcc/NvStorageFtwWorking.bin
 Vlv2TbltDevicePkg/PlatformPkgGcc.fdf:135:FILE = 
 $(WORKSPACE)/Vlv2TbltDevicePkg/Stitch/Gcc/NvStorageFtwSpare.bin
 Vlv2TbltDevicePkg/PlatformPkgGcc.fdf:921:DXE_DEPEX DXE_DEPEX Optional 
  $(WORKSPACE)/$(PLATFORM_PACKAGE)/IntelGopDepex/IntelGopDriver.depex
 Vlv2TbltDevicePkg/bldX64.bat:18:@if exist %WORKSPACE%\edk2.log del 
 %WORKSPACE%\edk2.log
 Vlv2TbltDevicePkg/bldX64.bat:19:@if exist %WORKSPACE%\unitool.log del 
 %WORKSPACE%\unitool.log
 Vlv2TbltDevicePkg/bldX64.bat:20:@if exist %WORKSPACE%\Conf\build_rule.txt del 
 %WORKSPACE%\Conf\build_rule.txt
 Vlv2TbltDevicePkg/bldX64.bat:21:@if exist 
 %WORKSPACE%\Conf\FrameworkDatabase.db del 
 %WORKSPACE%\Conf\FrameworkDatabase.db
 Vlv2TbltDevicePkg/bldX64.bat:22:@if exist %WORKSPACE%\Confarget.txt del 
 %WORKSPACE%\Confarget.txt
 Vlv2TbltDevicePkg/bldX64.bat:23:@if exist %WORKSPACE%\Confools_def.txt 
 del %WORKSPACE%\Confools_def.txt
 Vlv2TbltDevicePkg/bld_vlv.bat:23:set WORKSPACE=%CD%
 Vlv2TbltDevicePkg/bld_vlv.bat:24:if %WORKSPACE:~-1%==\ set 
 WORKSPACE=%WORKSPACE:~0,-1%
 Vlv2TbltDevicePkg/bld_vlv.bat:32:if exist %WORKSPACE%\edk2.log del 
 %WORKSPACE%\edk2.log
 Vlv2TbltDevicePkg/bld_vlv.bat:33:if exist %WORKSPACE%\unitool.log del 
 %WORKSPACE%\unitool.log
 Vlv2TbltDevicePkg/bld_vlv.bat:34:if exist %WORKSPACE%\Confarget.txt del 
 %WORKSPACE%\Confarget.txt
 Vlv2TbltDevicePkg/bld_vlv.bat:35:if exist %WORKSPACE%\Confools_def.txt 
 del %WORKSPACE%\Confools_def.txt
 Vlv2TbltDevicePkg/bld_vlv.bat:36:if exist %WORKSPACE%\Conf\build_rule.txt del 
 %WORKSPACE%\Conf\build_rule.txt
 Vlv2TbltDevicePkg/bld_vlv.bat:37:if exist 
 %WORKSPACE%\Conf\FrameworkDatabase.db del 
 %WORKSPACE%\Conf\FrameworkDatabase.db
 Vlv2TbltDevicePkg/bld_vlv.bat:50:set 
 EDK_SOURCE=%WORKSPACE%\EdkCompatibilityPkg
 Vlv2TbltDevicePkg/bld_vlv.bat:236:copy /y/b %BUILD_PATH%\FV\Vlv%Arch%.fd  
 %WORKSPACE%\%BIOS_Name% nul
 Vlv2TbltDevicePkg/bld_vlv.sh:27:#WORKSPACE=$(pwd)
 Vlv2TbltDevicePkg/bld_vlv.sh:66:config_file=$WORKSPACE/$PLATFORM_PACKAGE/PlatformPkgConfig.dsc
 Vlv2TbltDevicePkg/bld_vlv.sh:67:auto_config_inc=$WORKSPACE/$PLATFORM_PACKAGE/AutoPlatformCFG.txt
 Vlv2TbltDevicePkg/bld_vlv.sh:70:EDK_SOURCE=$WORKSPACE/EdkCompatibilityPkg
 Vlv2TbltDevicePkg/bld_vlv.sh:100:  
 ECP_SOURCE=$WORKSPACE/EdkCompatibilityPkgEcp
 Vlv2TbltDevicePkg/bld_vlv.sh:101:  
 EDK_SOURCE=$WORKSPACE/EdkCompatibilityPkgEcp
 Vlv2TbltDevicePkg/bld_vlv.sh:224:cp -f $BUILD_PATH/FV/VLV.fd  
 $WORKSPACE/$BIOS_Name
 ...
 
 Thanks,
 
 Andrew Fish
 
  I think you originally suggested just using WORKSPACE, but maybe now
  have changed your mind.
  
  I know Tim had some big concerns with expanding WORKSPACE. I agree
  with his concerns, but I don't think it will be much better with
  WORKSPACE_MULTIPLE. (The same tools will break.)
  
  Mike and Andrew appear to prefer WORKSPACE_MULTIPLE.
  
  What about this suggestion?
  
  WORKSPACE: expand to a list of directories
  
  WORKSPACE_BUILD: if not set, use first path in WORKSPACE + '/Build’
  
 
 Build is not architectural, this is a value in the DSC. 
 
  WORKSPACE_CONF: if not set, scan WORKSPACE paths to find 'Conf' dir
  
  WORKSPACE_TOOLS: if not set, scan WORKSPACE paths to find 'BaseTools' dir
  
  With WORKSPACE and WORKSPACE_MULTIPLE we have a strange intersection
  of 'part of the source tree' and 'all of the source tree'. I think the
  variables I recommend avoid that partial overlap.
  
 
 We have the current stuff all works, and you can opt into a new feature if 
 needed. You guys want to break the world to make it more pedantically 
 correct? What do we gain from breaking a large number of build targets? 
 
 Thanks,
 
 Andrew Fish
 
  -Jordan
  
  I think Mike solution can
  make them more clear if WORKSPACE_MULTIPLE is set.
  
  On QA team impact, it should be small. QA team can still use the
  current working model to place all packages in root directory, and
  set WORKSPACE only. QA team just needs to know the package list.
  
  Tim gave another idea to define multiple source locations in
  Platform DSC file. It doesn't conflict with this proposal. We can
  implement $(WORKSPACE_MULTIPLE) feature first, and evaluate DSC
  update. On the fixed tree layout, these two solutions can work. Once
  the end user reorganizes its tree layout, he needs to set
  $(WORKSPACE_MULTIPLE) env or update Platform DSC file, and build
  platform DSC.
  
  Thanks
  Liming
  -Original Message-
  From: Laszlo Ersek [mailto:ler...@redhat.com] 
  Sent: Tuesday, August 04, 2015 7:20 PM
  To: Justen, Jordan L
  Cc: Gao, Liming; Paolo Bonzini; edk2-devel@lists.01.org
  Subject: Re: [edk2] BaseTools features: multiple workspaces
  
  On 08/03/15 19:35, Jordan Justen wrote:
  On 2015-08-03 02:08:14, Gao, Liming

Re: [edk2] BaseTools features: multiple workspaces

[Andrew Fish]

 to place all packages in root directory, and
 set WORKSPACE only. QA team just needs to know the package list.
 
 Tim gave another idea to define multiple source locations in
 Platform DSC file. It doesn't conflict with this proposal. We can
 implement $(WORKSPACE_MULTIPLE) feature first, and evaluate DSC
 update. On the fixed tree layout, these two solutions can work. Once
 the end user reorganizes its tree layout, he needs to set
 $(WORKSPACE_MULTIPLE) env or update Platform DSC file, and build
 platform DSC.
 
 Thanks
 Liming
 -Original Message-
 From: Laszlo Ersek [mailto:ler...@redhat.com] 
 Sent: Tuesday, August 04, 2015 7:20 PM
 To: Justen, Jordan L
 Cc: Gao, Liming; Paolo Bonzini; edk2-devel@lists.01.org
 Subject: Re: [edk2] BaseTools features: multiple workspaces
 
 On 08/03/15 19:35, Jordan Justen wrote:
 On 2015-08-03 02:08:14, Gao, Liming wrote:
 Paolo:
 I think that keep the exiting syntax for WORKSPACE to be a single
 path can minimize the impacts to exiting tools that assume a
 single workspace.
 
 I know you originally suggested to use WORKSPACE, but got some 
 feedback that it might be better to create a new variable to not break 
 some unspecified tools.
 
 Was anyone able to name an actual tool that would be impacted? Why not 
 work to update these tools at the same time? An easy 'fix' will be for 
 any such tool to give an error message if it sees ':' or ';' in the 
 WORKSPACE environment variable. Although this is really not a fix, it 
 will be just as much support as if they ignore the new 
 WORKSPACE_MULTIPLE environment variable.
 
 I think that using the single separated WORKSPACE environment variable 
 is more intuitive/expected. From using env vars such as PATH, we all 
 know what it means.
 
 For WORKSPACE+WORKSPACE_MULTIPLE, I don't know which has priority. I 
 assume WORKSPACE is checked first, and then WORKSPACE_MULTIPLE?
 
 This was described in Liming's email, in point 4b.
 
 
 Will this work for the integrated CryptoPkg + open-ssl? It seems like 
 we need a priority above EDK II for this. I guess WORKSPACE will not 
 be set to EDK II in this case?
 
 Anyway, it just seems a little odd.
 
 For me a colon-separated (well, separator-separated :)) single WORKSPACE 
 variable would be more intuitive, but WORKSPACE_MULTIPLE can be defined 
 well enough too. For example:
 - WORKSPACE must not contain separators
 - if WORKSPACE_MULTIPLE is set, then the effect is as if
 ${WORKSPACE}:${WORKSPACE_MULTIPLE} were specified under the
 well-known PATH semantics.
 
 No clue about the CryptoPkg impact. Can you remind me please why CryptoPkg 
 / OpensslLib are special wrt. WORKSPACE?
 
 Thanks
 Laszlo
 
 
 -Jordan
 
 -Original Message-
 From: edk2-devel [mailto:edk2-devel-boun...@lists.01.org] On Behalf 
 Of Paolo Bonzini
 Sent: Monday, August 03, 2015 4:31 PM
 To: Gao, Liming; edk2-devel@lists.01.org
 Subject: Re: [edk2] BaseTools features: multiple workspaces
 
 
 On 03/08/2015 05:56, Gao, Liming wrote:
 Hi, all
 We will update BaseTools feature to allow more than one workspaces. The 
 detail design in the below. Please help review it. If you have any 
 comments, please let me know.
 
 1.   Keep $(WORKSPACE) environment as is
 
 a.   $(WORKSPACE) determines location of Build and Conf directory.
 
 2.   New optional $(WORKSPACE_MULTIPLE) environment is added to 
 include more directories with the separator ';', like $(PATH)
 
 Why is $(WORKSPACE_MULTIPLE) necessary?  On Linux it is okay to break 
 preexisting paths that contain a :; on Windows semicolons are allowed 
 in theory but in practice break in several ways.
 
 Paolo
 
 a.   Produce the same behavior if $(WORKSPACE_MULTIPLE) is not set.
 
 3.   Update edksetup.bat/edksetup.sh to find BaseTools directory 
 from $(WORKSPACE) and $(WORKSPACE_MULTIPLE) directories.
 
 4.   Update BaseTools to support multiple workspaces
 
 a.   For the relative file path (INF, DSC and FDF), BaseTools will 
 search them from $(WORKSPACE) and $(WORKSPACE_MULTIPLE) directories.
 
 b.  Search priority from high to low: $(WORKSPACE), 
 $(WORKSPACE_MULTIPLE).
 
   This is a compatible feature. It has no impact on current EDKII 
 project.
 ___
 edk2-devel mailing list
 edk2-devel@lists.01.org
 https://lists.01.org/mailman/listinfo/edk2-devel
 ___
 edk2-devel mailing list
 edk2-devel@lists.01.org
 https://lists.01.org/mailman/listinfo/edk2-devel
 ___
 edk2-devel mailing list
 edk2-devel@lists.01.org
 https://lists.01.org/mailman/listinfo/edk2-devel
 
 
 ___
 edk2-devel mailing list
 edk2-devel@lists.01.org
 https://lists.01.org/mailman/listinfo/edk2-devel
 
 ___
 edk2-devel mailing list
 edk2-devel@lists.01.org mailto:edk2-devel@lists.01.org
 https://lists.01.org/mailman/listinfo/edk2-devel

Re: [edk2] BaseTools features: multiple workspaces

[Andrew Fish]

 On Aug 5, 2015, at 12:16 PM, Jordan Justen jordan.l.jus...@intel.com wrote:
 
 On 2015-08-05 07:06:46, Gao, Liming wrote:
 Tim gave another idea to define multiple source locations in
 Platform DSC file. It doesn't conflict with this proposal. We can
 implement $(WORKSPACE_MULTIPLE) feature first, and evaluate DSC
 update. On the fixed tree layout, these two solutions can work. Once
 the end user reorganizes its tree layout, he needs to set
 $(WORKSPACE_MULTIPLE) env or update Platform DSC file, and build
 platform DSC.
 
 At one point Mike suggested updating target.txt to be able to add
 paths.
 
 I think this might be better than hard coding them into the dsc
 (source code) files.
 
 Tim, any thoughts on this?
 

I’m not Tim, but PLEASE NOOO!!

We don’t want “one tree per product”, we want multiple products per tree. 
target.txt is really helpful if you want to save typing on the command line, 
but that is about it.

Now I have seen some vendors that do sill stuff like write a script that 
patches target.txt and then calls build.exe. I say silly as the script could 
have just passed the arguments build.exe directly, as target.txt is just the 
default value for the args of the build command. 

Thanks,

Andrew Fish

 -Jordan
 ___
 edk2-devel mailing list
 edk2-devel@lists.01.org
 https://lists.01.org/mailman/listinfo/edk2-devel

___
edk2-devel mailing list
edk2-devel@lists.01.org
https://lists.01.org/mailman/listinfo/edk2-devel

Re: [edk2] BaseTools features: multiple workspaces

[Tim Lewis]

Agree with Andrew.

Tim
-Original Message-
From: edk2-devel [mailto:edk2-devel-boun...@lists.01.org] On Behalf Of Andrew 
Fish
Sent: Wednesday, August 05, 2015 1:22 PM
To: Jordan Justen jordan.l.jus...@intel.com
Cc: Paolo Bonzini pbonz...@redhat.com; edk2-devel@lists.01.org; Laszlo Ersek 
ler...@redhat.com; Tim Lewis tim.le...@insyde.com; Gao, Liming 
liming@intel.com
Subject: Re: [edk2] BaseTools features: multiple workspaces


 On Aug 5, 2015, at 12:16 PM, Jordan Justen jordan.l.jus...@intel.com wrote:
 
 On 2015-08-05 07:06:46, Gao, Liming wrote:
 Tim gave another idea to define multiple source locations in Platform 
 DSC file. It doesn't conflict with this proposal. We can implement 
 $(WORKSPACE_MULTIPLE) feature first, and evaluate DSC update. On the 
 fixed tree layout, these two solutions can work. Once the end user 
 reorganizes its tree layout, he needs to set
 $(WORKSPACE_MULTIPLE) env or update Platform DSC file, and build 
 platform DSC.
 
 At one point Mike suggested updating target.txt to be able to add 
 paths.
 
 I think this might be better than hard coding them into the dsc 
 (source code) files.
 
 Tim, any thoughts on this?
 

I’m not Tim, but PLEASE NOOO!!

We don’t want “one tree per product”, we want multiple products per tree. 
target.txt is really helpful if you want to save typing on the command line, 
but that is about it.

Now I have seen some vendors that do sill stuff like write a script that 
patches target.txt and then calls build.exe. I say silly as the script could 
have just passed the arguments build.exe directly, as target.txt is just the 
default value for the args of the build command. 

Thanks,

Andrew Fish

 -Jordan
 ___
 edk2-devel mailing list
 edk2-devel@lists.01.org
 https://lists.01.org/mailman/listinfo/edk2-devel

___
edk2-devel mailing list
edk2-devel@lists.01.org
https://lists.01.org/mailman/listinfo/edk2-devel
___
edk2-devel mailing list
edk2-devel@lists.01.org
https://lists.01.org/mailman/listinfo/edk2-devel

Re: [edk2] BaseTools features: multiple workspaces

[Andrew Fish]

 On Aug 5, 2015, at 2:36 PM, Jordan Justen jordan.l.jus...@intel.com wrote:
 
 On 2015-08-05 12:51:47, Andrew Fish wrote:
 On Aug 5, 2015, at 12:40 PM, Jordan Justen jordan.l.jus...@intel.com
 wrote:
 Unless we do something about most of those references then WORKSPACE
 will have to be set to your 'edk2' tree. Therefore your Conf files
 must exist in your edk2 tree, and your build output must exist in your
 edk2 tree, and BaseTools must exist in your edk2 tree.
 
   Sorry I'm not following. In our local hack we do something like.
 
 What I meant is what if I want to have Build, Conf, BaseTools and
 package source code come from different directories. For example:
 
 Build: ~/udk/build
 Conf: /usr/lib/udk/2014/conf
 BaseTools: /usr/lib/udk/2014/bin
 udk packages: /usr/lib/udk/2014/src
 platform packages: ~/udk/platform
 
 and, the same platform should be buildable on windows:
 
 Build: C:\src\udk\build
 Conf: C:\udk\2014\conf
 BaseTools: C:\udk\2014\bin
 udk packages: C:\udk\2014\src
 platform packages: C:\src\udk\platform
 
   BaseTools/ - edk2 tools
   BuildResults/ - Build output
   edk2/  - Open source edk2 stuff
   Vendor/ - code from vendors, that is usually rooted in $(WORKSPACE).
 
 It doesn't seem like $WORKSPACE/EmulatorPkg exists. Since we've broken
 $WORKSPACE/EmulatorPkg and the like, making $WORKSPACE a path list
 doesn't break things further.
 
   So we set WORKSPACE_MULTIPLE =
   WORKSPACE/edk2;WORKSPACE/Vendor/Alpha;WORKSPACE/Vendor/Beta
   and we are good.
 
   Thus you can :
   Move the edk2/ code to any location you want down stream of WORKSPACE
   BuildResults is defined in DSC so it can be any location you want down
   stream of WORKSPACE. 
 
 I take your feedback here and what I see is that you think of
 WORKSPACE more as the place where BaseTools, Build and Conf live, and
 less about where EDK II packages live. This is opposite of me, as I
 generally think of WORKSPACE as the place where EDK II package code
 lives.
 
 That's fine. I can adjust my thinking and abandon the notion that
 WORKSPACE refers to package source code. What if we instead add a new
 PACKAGES_PATH list, rather than WORKSPACE_MULTIPLE? At least it sounds
 better (terminology-wise) to me.
 

I don’t have an issue shifting the name to PACKAGES_PATH if that helps folks 
figure out what we are trying to accomplish.

Thanks,

Andrew Fish

 -Jordan

___
edk2-devel mailing list
edk2-devel@lists.01.org
https://lists.01.org/mailman/listinfo/edk2-devel

Re: [edk2] BaseTools features: multiple workspaces

[Jordan Justen]

On 2015-08-05 13:21:37, Andrew Fish wrote:
  On Aug 5, 2015, at 12:16 PM, Jordan Justen jordan.l.jus...@intel.com 
  wrote:
  On 2015-08-05 07:06:46, Gao, Liming wrote:
  Tim gave another idea to define multiple source locations in
  Platform DSC file. It doesn't conflict with this proposal. We can
  implement $(WORKSPACE_MULTIPLE) feature first, and evaluate DSC
  update. On the fixed tree layout, these two solutions can work. Once
  the end user reorganizes its tree layout, he needs to set
  $(WORKSPACE_MULTIPLE) env or update Platform DSC file, and build
  platform DSC.
  
  At one point Mike suggested updating target.txt to be able to add
  paths.
  
  I think this might be better than hard coding them into the dsc
  (source code) files.
  
  Tim, any thoughts on this?
 
 I’m not Tim, but PLEASE NOOO!!
 
 We don’t want “one tree per product”, we want multiple products per
 tree. target.txt is really helpful if you want to save typing on the
 command line, but that is about it.

I happen to agree. I haven't used target.txt in years. I love being
able to run these two commands easily in the same tree:

$ OvmfPkg/build.sh

$ EmulatorPkg/build.sh

If anything, I'd like to see us make the Conf directory optional. (And
use BaseTools/Conf by default.)

 Now I have seen some vendors that do sill stuff like write a script
 that patches target.txt and then calls build.exe. I say silly as the
 script could have just passed the arguments build.exe directly, as
 target.txt is just the default value for the args of the build
 command.

Apparently we have a 'TargetTool' that helps with this? Who knew? (Not
I ... seriously.)

Anyway, I'm just responding to not wanting to bake hard coded package
paths into the .dsc. Not that I like target.txt. :)

-Jordan
___
edk2-devel mailing list
edk2-devel@lists.01.org
https://lists.01.org/mailman/listinfo/edk2-devel

Re: [edk2] BaseTools features: multiple workspaces

[Andrew Fish]

 On Aug 5, 2015, at 2:46 PM, Jordan Justen jordan.l.jus...@intel.com wrote:
 
 On 2015-08-05 13:21:37, Andrew Fish wrote:
 On Aug 5, 2015, at 12:16 PM, Jordan Justen jordan.l.jus...@intel.com 
 wrote:
 On 2015-08-05 07:06:46, Gao, Liming wrote:
 Tim gave another idea to define multiple source locations in
 Platform DSC file. It doesn't conflict with this proposal. We can
 implement $(WORKSPACE_MULTIPLE) feature first, and evaluate DSC
 update. On the fixed tree layout, these two solutions can work. Once
 the end user reorganizes its tree layout, he needs to set
 $(WORKSPACE_MULTIPLE) env or update Platform DSC file, and build
 platform DSC.
 
 At one point Mike suggested updating target.txt to be able to add
 paths.
 
 I think this might be better than hard coding them into the dsc
 (source code) files.
 
 Tim, any thoughts on this?
 
 I’m not Tim, but PLEASE NOOO!!
 
 We don’t want “one tree per product”, we want multiple products per
 tree. target.txt is really helpful if you want to save typing on the
 command line, but that is about it.
 
 I happen to agree. I haven't used target.txt in years. I love being
 able to run these two commands easily in the same tree:
 
 $ OvmfPkg/build.sh
 
 $ EmulatorPkg/build.sh
 
 If anything, I'd like to see us make the Conf directory optional. (And
 use BaseTools/Conf by default.)
 
 Now I have seen some vendors that do sill stuff like write a script
 that patches target.txt and then calls build.exe. I say silly as the
 script could have just passed the arguments build.exe directly, as
 target.txt is just the default value for the args of the build
 command.
 
 Apparently we have a 'TargetTool' that helps with this? Who knew? (Not
 I ... seriously.)
 
 Anyway, I'm just responding to not wanting to bake hard coded package
 paths into the .dsc.

It works well if you are moving things about under the existing WORKSPACE, like 
my example. You are basically just adding directory structure to the flat edk2 
convention for packages. 
That is the use case that I, and I think Tim, care about the most. 

Thanks,

Andrew Fish

 Not that I like target.txt. :)
 
 -Jordan

___
edk2-devel mailing list
edk2-devel@lists.01.org
https://lists.01.org/mailman/listinfo/edk2-devel

Re: [edk2] BaseTools features: multiple workspaces

[Laszlo Ersek]

On 08/03/15 19:35, Jordan Justen wrote:
 On 2015-08-03 02:08:14, Gao, Liming wrote:
 Paolo:
   I think that keep the exiting syntax for WORKSPACE to be a single
   path can minimize the impacts to exiting tools that assume a
   single workspace.
 
 I know you originally suggested to use WORKSPACE, but got some
 feedback that it might be better to create a new variable to not break
 some unspecified tools.
 
 Was anyone able to name an actual tool that would be impacted? Why not
 work to update these tools at the same time? An easy 'fix' will be for
 any such tool to give an error message if it sees ':' or ';' in the
 WORKSPACE environment variable. Although this is really not a fix, it
 will be just as much support as if they ignore the new
 WORKSPACE_MULTIPLE environment variable.
 
 I think that using the single separated WORKSPACE environment variable
 is more intuitive/expected. From using env vars such as PATH, we all
 know what it means.
 
 For WORKSPACE+WORKSPACE_MULTIPLE, I don't know which has priority. I
 assume WORKSPACE is checked first, and then WORKSPACE_MULTIPLE?

This was described in Liming's email, in point 4b.

 
 Will this work for the integrated CryptoPkg + open-ssl? It seems like
 we need a priority above EDK II for this. I guess WORKSPACE will not
 be set to EDK II in this case?
 
 Anyway, it just seems a little odd.

For me a colon-separated (well, separator-separated :)) single WORKSPACE
variable would be more intuitive, but WORKSPACE_MULTIPLE can be defined
well enough too. For example:
- WORKSPACE must not contain separators
- if WORKSPACE_MULTIPLE is set, then the effect is as if
  ${WORKSPACE}:${WORKSPACE_MULTIPLE} were specified under the
  well-known PATH semantics.

No clue about the CryptoPkg impact. Can you remind me please why
CryptoPkg / OpensslLib are special wrt. WORKSPACE?

Thanks
Laszlo

 
 -Jordan
 
 -Original Message-
 From: edk2-devel [mailto:edk2-devel-boun...@lists.01.org] On Behalf Of Paolo 
 Bonzini
 Sent: Monday, August 03, 2015 4:31 PM
 To: Gao, Liming; edk2-devel@lists.01.org
 Subject: Re: [edk2] BaseTools features: multiple workspaces


 On 03/08/2015 05:56, Gao, Liming wrote:
 Hi, all
   We will update BaseTools feature to allow more than one workspaces. The 
 detail design in the below. Please help review it. If you have any 
 comments, please let me know.

 1.   Keep $(WORKSPACE) environment as is

 a.   $(WORKSPACE) determines location of Build and Conf directory.

 2.   New optional $(WORKSPACE_MULTIPLE) environment is added to include 
 more directories with the separator ';', like $(PATH)

 Why is $(WORKSPACE_MULTIPLE) necessary?  On Linux it is okay to break 
 preexisting paths that contain a :; on Windows semicolons are allowed in 
 theory but in practice break in several ways.

 Paolo

 a.   Produce the same behavior if $(WORKSPACE_MULTIPLE) is not set.

 3.   Update edksetup.bat/edksetup.sh to find BaseTools directory from 
 $(WORKSPACE) and $(WORKSPACE_MULTIPLE) directories.

 4.   Update BaseTools to support multiple workspaces

 a.   For the relative file path (INF, DSC and FDF), BaseTools will 
 search them from $(WORKSPACE) and $(WORKSPACE_MULTIPLE) directories.

 b.  Search priority from high to low: $(WORKSPACE), 
 $(WORKSPACE_MULTIPLE).

 This is a compatible feature. It has no impact on current EDKII project.
 ___
 edk2-devel mailing list
 edk2-devel@lists.01.org
 https://lists.01.org/mailman/listinfo/edk2-devel
 ___
 edk2-devel mailing list
 edk2-devel@lists.01.org
 https://lists.01.org/mailman/listinfo/edk2-devel
 ___
 edk2-devel mailing list
 edk2-devel@lists.01.org
 https://lists.01.org/mailman/listinfo/edk2-devel
 

___
edk2-devel mailing list
edk2-devel@lists.01.org
https://lists.01.org/mailman/listinfo/edk2-devel

Re: [edk2] BaseTools features: multiple workspaces

[Jordan Justen]

On 2015-08-03 14:32:15, Andrew Fish wrote:
 
  On Aug 3, 2015, at 2:04 PM, Jordan Justen jordan.l.jus...@intel.com wrote:
  
  On 2015-08-03 12:58:28, Kinney, Michael D wrote:
  Jordan,
  
  If we make a slight change to the requirements, we can provide clear
  priority rules.
  
  WORKSPACE is used to find Build and Conf.
  
  If WORKSPACE_MULTIPLE exists, then it is used as a prioritized list
  of paths to resolve all other workspace relative paths. This means
  WORKSPACE_MULTIPLE will need to also contain that path to WORKSPACE
  as one of its path if WORKSPACE has packages in it too. This means
  rule 4b needs to be adjusted.
  
  I like this a little better, since you only need to search one thing
  for the set of workspaces. But, it doesn't let you keep your workspace
  build area separate from your workspace sources.
  
  We could also add WORKSPACE_BUILD and make WORKSPACE be the separated
  list. If WORKSPACE_BUILD is not set, the first WORKSPACE path could be
  used instead. Adding WORKSPACE_BUILD seems more intuitive to me than
  adding WORKSPACE_MULTIPLE.
  
 
 I think having two lists makes the most sense.

Two lists?

 Especially since WORKSPACE is required for the build to function
 (path to the build executable).

We have 4 distinct things of interest:

1. Package source paths

2. Build

3. Conf (probably combinable with Build location)

4. BaseTools

Today WORKSPACE is all of those. We want to make #1 be more than a
single path. Personally when I think of WORKSPACE, I'm usually
thinking of #1, but I understand it is used for all 4 items.

 source ekdsetup.sh BaseTools
 WORKSPACE_MULTIPLE=“”${WORKSPACE}/edk2/;${WORKSPACE}/Vendor/Intel/;${WORKSPACE}/Vendor/OurCode/

I want support cases where one of the paths is not under WORKSPACE.

For example, why can't we install UDK2014 under C:\UDK2014 or
/usr/include/udk/2014?

In other words, I don't think $(WORKSPACE)/MdePkg nor
$(WORKSPACE)/edk2/MdePkg should be something we require to be present.

-Jordan

 The two lists are also the simplest form for the user. Tim’s request
 to set the WORKSPACE_MULTIPLE from the DSC also is a good example
 that the lists really are separate.
 
 Thanks,
 
 Andrew Fish
 
  -Jordan
  
  -Original Message-
  From: edk2-devel [mailto:edk2-devel-boun...@lists.01.org] On Behalf Of 
  Jordan Justen
  Sent: Monday, August 03, 2015 11:55 AM
  To: Tim Lewis; Gao, Liming; Paolo Bonzini
  Cc: edk2-devel@lists.01.org
  Subject: Re: [edk2] BaseTools features: multiple workspaces
  
  Cc: edk2-devel
  
  On 2015-08-03 10:41:43, Tim Lewis wrote:
  There are dozens of tools in commercial environments (including some
  by Insyde) which will fail in this case. In addition, there are
  codebases that I am aware of where $(WORKSPACE) is allowed directly
  in other build files.
  
  Will all these 'dozens of tools in commercial environments' fare any
  better with the WORKSPACE_MULTIPLE option?
  
  As a quick example, say C:\ssl\CryptoPkg has open-ssl integrated, and
  C:\edk2 is the plain EDK II tree.
  
  Thus, we want to set WORKSPACE=C:\ssl and WORKSPACE_MULTIPLE=C:\edk2
  so the open-ssl integrated version of CryptoPkg will take priority.
  
  Are these tools likely to work as expected? I am guessing no. I'm
  thinking that these tools will probably have to insist on a fully
  integrated single workspace until they can be updated to the new
  design.
  
  At that point whether WORKSPACE can have multiple directories, or we
  add WORKSPACE_MULTIPLE, it will not make a big difference to these
  tools.
  
  -Jordan
  
  -Original Message-
  From: edk2-devel [mailto:edk2-devel-boun...@lists.01.org] On Behalf Of 
  Jordan Justen
  Sent: Monday, August 03, 2015 10:36 AM
  To: Gao, Liming liming@intel.com; Paolo Bonzini 
  pbonz...@redhat.com; edk2-devel@lists.01.org
  Subject: Re: [edk2] BaseTools features: multiple workspaces
  
  On 2015-08-03 02:08:14, Gao, Liming wrote:
  Paolo:
   I think that keep the exiting syntax for WORKSPACE to be a single
   path can minimize the impacts to exiting tools that assume a
   single workspace.
  
  I know you originally suggested to use WORKSPACE, but got some feedback 
  that it might be better to create a new variable to not break some 
  unspecified tools.
  
  Was anyone able to name an actual tool that would be impacted? Why not 
  work to update these tools at the same time? An easy 'fix' will be for 
  any such tool to give an error message if it sees ':' or ';' in the 
  WORKSPACE environment variable. Although this is really not a fix, it 
  will be just as much support as if they ignore the new WORKSPACE_MULTIPLE 
  environment variable.
  
  I think that using the single separated WORKSPACE environment variable is 
  more intuitive/expected. From using env vars such as PATH, we all know 
  what it means.
  
  For WORKSPACE+WORKSPACE_MULTIPLE, I don't know which has priority. I 
  assume WORKSPACE is checked first, and then WORKSPACE_MULTIPLE?
  
  Will this work

Re: [edk2] BaseTools features: multiple workspaces

[Andrew Fish]

 On Aug 3, 2015, at 2:04 PM, Jordan Justen jordan.l.jus...@intel.com wrote:
 
 On 2015-08-03 12:58:28, Kinney, Michael D wrote:
 Jordan,
 
 If we make a slight change to the requirements, we can provide clear
 priority rules.
 
 WORKSPACE is used to find Build and Conf.
 
 If WORKSPACE_MULTIPLE exists, then it is used as a prioritized list
 of paths to resolve all other workspace relative paths. This means
 WORKSPACE_MULTIPLE will need to also contain that path to WORKSPACE
 as one of its path if WORKSPACE has packages in it too. This means
 rule 4b needs to be adjusted.
 
 I like this a little better, since you only need to search one thing
 for the set of workspaces. But, it doesn't let you keep your workspace
 build area separate from your workspace sources.
 
 We could also add WORKSPACE_BUILD and make WORKSPACE be the separated
 list. If WORKSPACE_BUILD is not set, the first WORKSPACE path could be
 used instead. Adding WORKSPACE_BUILD seems more intuitive to me than
 adding WORKSPACE_MULTIPLE.
 

I think having two lists makes the most sense. Especially since WORKSPACE is 
required for the build to function (path to the build executable). 

source ekdsetup.sh BaseTools
WORKSPACE_MULTIPLE=“”${WORKSPACE}/edk2/;${WORKSPACE}/Vendor/Intel/;${WORKSPACE}/Vendor/OurCode/

The two lists are also the simplest form for the user. Tim’s request to set the 
WORKSPACE_MULTIPLE from the DSC also is a good example that the lists really 
are separate. 

Thanks,

Andrew Fish

 -Jordan
 
 -Original Message-
 From: edk2-devel [mailto:edk2-devel-boun...@lists.01.org] On Behalf Of 
 Jordan Justen
 Sent: Monday, August 03, 2015 11:55 AM
 To: Tim Lewis; Gao, Liming; Paolo Bonzini
 Cc: edk2-devel@lists.01.org
 Subject: Re: [edk2] BaseTools features: multiple workspaces
 
 Cc: edk2-devel
 
 On 2015-08-03 10:41:43, Tim Lewis wrote:
 There are dozens of tools in commercial environments (including some
 by Insyde) which will fail in this case. In addition, there are
 codebases that I am aware of where $(WORKSPACE) is allowed directly
 in other build files.
 
 Will all these 'dozens of tools in commercial environments' fare any
 better with the WORKSPACE_MULTIPLE option?
 
 As a quick example, say C:\ssl\CryptoPkg has open-ssl integrated, and
 C:\edk2 is the plain EDK II tree.
 
 Thus, we want to set WORKSPACE=C:\ssl and WORKSPACE_MULTIPLE=C:\edk2
 so the open-ssl integrated version of CryptoPkg will take priority.
 
 Are these tools likely to work as expected? I am guessing no. I'm
 thinking that these tools will probably have to insist on a fully
 integrated single workspace until they can be updated to the new
 design.
 
 At that point whether WORKSPACE can have multiple directories, or we
 add WORKSPACE_MULTIPLE, it will not make a big difference to these
 tools.
 
 -Jordan
 
 -Original Message-
 From: edk2-devel [mailto:edk2-devel-boun...@lists.01.org] On Behalf Of 
 Jordan Justen
 Sent: Monday, August 03, 2015 10:36 AM
 To: Gao, Liming liming@intel.com; Paolo Bonzini 
 pbonz...@redhat.com; edk2-devel@lists.01.org
 Subject: Re: [edk2] BaseTools features: multiple workspaces
 
 On 2015-08-03 02:08:14, Gao, Liming wrote:
 Paolo:
  I think that keep the exiting syntax for WORKSPACE to be a single
  path can minimize the impacts to exiting tools that assume a
  single workspace.
 
 I know you originally suggested to use WORKSPACE, but got some feedback 
 that it might be better to create a new variable to not break some 
 unspecified tools.
 
 Was anyone able to name an actual tool that would be impacted? Why not work 
 to update these tools at the same time? An easy 'fix' will be for any such 
 tool to give an error message if it sees ':' or ';' in the WORKSPACE 
 environment variable. Although this is really not a fix, it will be just as 
 much support as if they ignore the new WORKSPACE_MULTIPLE environment 
 variable.
 
 I think that using the single separated WORKSPACE environment variable is 
 more intuitive/expected. From using env vars such as PATH, we all know what 
 it means.
 
 For WORKSPACE+WORKSPACE_MULTIPLE, I don't know which has priority. I assume 
 WORKSPACE is checked first, and then WORKSPACE_MULTIPLE?
 
 Will this work for the integrated CryptoPkg + open-ssl? It seems like we 
 need a priority above EDK II for this. I guess WORKSPACE will not be set to 
 EDK II in this case?
 
 Anyway, it just seems a little odd.
 
 -Jordan
 
 -Original Message-
 From: edk2-devel [mailto:edk2-devel-boun...@lists.01.org] On Behalf Of 
 Paolo Bonzini
 Sent: Monday, August 03, 2015 4:31 PM
 To: Gao, Liming; edk2-devel@lists.01.org
 Subject: Re: [edk2] BaseTools features: multiple workspaces
 
 
 On 03/08/2015 05:56, Gao, Liming wrote:
 Hi, all
  We will update BaseTools feature to allow more than one workspaces. The 
 detail design in the below. Please help review it. If you have any 
 comments, please let me know.
 
 1.   Keep $(WORKSPACE) environment as is
 
 a.   $(WORKSPACE) determines

Re: [edk2] BaseTools features: multiple workspaces

[Paolo Bonzini]

On 03/08/2015 05:56, Gao, Liming wrote:
 Hi, all
   We will update BaseTools feature to allow more than one workspaces. The 
 detail design in the below. Please help review it. If you have any comments, 
 please let me know.
 
 1.   Keep $(WORKSPACE) environment as is
 
 a.   $(WORKSPACE) determines location of Build and Conf directory.
 
 2.   New optional $(WORKSPACE_MULTIPLE) environment is added to include 
 more directories with the separator ';', like $(PATH)

Why is $(WORKSPACE_MULTIPLE) necessary?  On Linux it is okay to break
preexisting paths that contain a :; on Windows semicolons are allowed
in theory but in practice break in several ways.

Paolo

 a.   Produce the same behavior if $(WORKSPACE_MULTIPLE) is not set.
 
 3.   Update edksetup.bat/edksetup.sh to find BaseTools directory from 
 $(WORKSPACE) and $(WORKSPACE_MULTIPLE) directories.
 
 4.   Update BaseTools to support multiple workspaces
 
 a.   For the relative file path (INF, DSC and FDF), BaseTools will search 
 them from $(WORKSPACE) and $(WORKSPACE_MULTIPLE) directories.
 
 b.  Search priority from high to low: $(WORKSPACE), $(WORKSPACE_MULTIPLE).
 
 This is a compatible feature. It has no impact on current EDKII project.
___
edk2-devel mailing list
edk2-devel@lists.01.org
https://lists.01.org/mailman/listinfo/edk2-devel

Re: [edk2] BaseTools features: multiple workspaces

[Gao, Liming]

Paolo:
  I think that keep the exiting syntax for WORKSPACE to be a single path can 
minimize the impacts to exiting tools that assume a single workspace. 

Thanks
Liming
-Original Message-
From: edk2-devel [mailto:edk2-devel-boun...@lists.01.org] On Behalf Of Paolo 
Bonzini
Sent: Monday, August 03, 2015 4:31 PM
To: Gao, Liming; edk2-devel@lists.01.org
Subject: Re: [edk2] BaseTools features: multiple workspaces


On 03/08/2015 05:56, Gao, Liming wrote:
 Hi, all
   We will update BaseTools feature to allow more than one workspaces. The 
 detail design in the below. Please help review it. If you have any comments, 
 please let me know.
 
 1.   Keep $(WORKSPACE) environment as is
 
 a.   $(WORKSPACE) determines location of Build and Conf directory.
 
 2.   New optional $(WORKSPACE_MULTIPLE) environment is added to include 
 more directories with the separator ';', like $(PATH)

Why is $(WORKSPACE_MULTIPLE) necessary?  On Linux it is okay to break 
preexisting paths that contain a :; on Windows semicolons are allowed in 
theory but in practice break in several ways.

Paolo

 a.   Produce the same behavior if $(WORKSPACE_MULTIPLE) is not set.
 
 3.   Update edksetup.bat/edksetup.sh to find BaseTools directory from 
 $(WORKSPACE) and $(WORKSPACE_MULTIPLE) directories.
 
 4.   Update BaseTools to support multiple workspaces
 
 a.   For the relative file path (INF, DSC and FDF), BaseTools will search 
 them from $(WORKSPACE) and $(WORKSPACE_MULTIPLE) directories.
 
 b.  Search priority from high to low: $(WORKSPACE), $(WORKSPACE_MULTIPLE).
 
 This is a compatible feature. It has no impact on current EDKII project.
___
edk2-devel mailing list
edk2-devel@lists.01.org
https://lists.01.org/mailman/listinfo/edk2-devel
___
edk2-devel mailing list
edk2-devel@lists.01.org
https://lists.01.org/mailman/listinfo/edk2-devel

Re: [edk2] BaseTools features: multiple workspaces

[Jordan Justen]

Cc: edk2-devel

On 2015-08-03 10:41:43, Tim Lewis wrote:
 There are dozens of tools in commercial environments (including some
 by Insyde) which will fail in this case. In addition, there are
 codebases that I am aware of where $(WORKSPACE) is allowed directly
 in other build files.

Will all these 'dozens of tools in commercial environments' fare any
better with the WORKSPACE_MULTIPLE option?

As a quick example, say C:\ssl\CryptoPkg has open-ssl integrated, and
C:\edk2 is the plain EDK II tree.

Thus, we want to set WORKSPACE=C:\ssl and WORKSPACE_MULTIPLE=C:\edk2
so the open-ssl integrated version of CryptoPkg will take priority.

Are these tools likely to work as expected? I am guessing no. I'm
thinking that these tools will probably have to insist on a fully
integrated single workspace until they can be updated to the new
design.

At that point whether WORKSPACE can have multiple directories, or we
add WORKSPACE_MULTIPLE, it will not make a big difference to these
tools.

-Jordan

 -Original Message-
 From: edk2-devel [mailto:edk2-devel-boun...@lists.01.org] On Behalf Of Jordan 
 Justen
 Sent: Monday, August 03, 2015 10:36 AM
 To: Gao, Liming liming@intel.com; Paolo Bonzini pbonz...@redhat.com; 
 edk2-devel@lists.01.org
 Subject: Re: [edk2] BaseTools features: multiple workspaces
 
 On 2015-08-03 02:08:14, Gao, Liming wrote:
  Paolo:
I think that keep the exiting syntax for WORKSPACE to be a single
path can minimize the impacts to exiting tools that assume a
single workspace.
 
 I know you originally suggested to use WORKSPACE, but got some feedback that 
 it might be better to create a new variable to not break some unspecified 
 tools.
 
 Was anyone able to name an actual tool that would be impacted? Why not work 
 to update these tools at the same time? An easy 'fix' will be for any such 
 tool to give an error message if it sees ':' or ';' in the WORKSPACE 
 environment variable. Although this is really not a fix, it will be just as 
 much support as if they ignore the new WORKSPACE_MULTIPLE environment 
 variable.
 
 I think that using the single separated WORKSPACE environment variable is 
 more intuitive/expected. From using env vars such as PATH, we all know what 
 it means.
 
 For WORKSPACE+WORKSPACE_MULTIPLE, I don't know which has priority. I assume 
 WORKSPACE is checked first, and then WORKSPACE_MULTIPLE?
 
 Will this work for the integrated CryptoPkg + open-ssl? It seems like we need 
 a priority above EDK II for this. I guess WORKSPACE will not be set to EDK II 
 in this case?
 
 Anyway, it just seems a little odd.
 
 -Jordan
 
  -Original Message-
  From: edk2-devel [mailto:edk2-devel-boun...@lists.01.org] On Behalf Of 
  Paolo Bonzini
  Sent: Monday, August 03, 2015 4:31 PM
  To: Gao, Liming; edk2-devel@lists.01.org
  Subject: Re: [edk2] BaseTools features: multiple workspaces
  
  
  On 03/08/2015 05:56, Gao, Liming wrote:
   Hi, all
 We will update BaseTools feature to allow more than one workspaces. The 
   detail design in the below. Please help review it. If you have any 
   comments, please let me know.
   
   1.   Keep $(WORKSPACE) environment as is
   
   a.   $(WORKSPACE) determines location of Build and Conf directory.
   
   2.   New optional $(WORKSPACE_MULTIPLE) environment is added to 
   include more directories with the separator ';', like $(PATH)
  
  Why is $(WORKSPACE_MULTIPLE) necessary?  On Linux it is okay to break 
  preexisting paths that contain a :; on Windows semicolons are allowed in 
  theory but in practice break in several ways.
  
  Paolo
  
   a.   Produce the same behavior if $(WORKSPACE_MULTIPLE) is not set.
   
   3.   Update edksetup.bat/edksetup.sh to find BaseTools directory from 
   $(WORKSPACE) and $(WORKSPACE_MULTIPLE) directories.
   
   4.   Update BaseTools to support multiple workspaces
   
   a.   For the relative file path (INF, DSC and FDF), BaseTools will 
   search them from $(WORKSPACE) and $(WORKSPACE_MULTIPLE) directories.
   
   b.  Search priority from high to low: $(WORKSPACE), 
   $(WORKSPACE_MULTIPLE).
   
   This is a compatible feature. It has no impact on current EDKII 
   project.
  ___
  edk2-devel mailing list
  edk2-devel@lists.01.org
  https://lists.01.org/mailman/listinfo/edk2-devel
  ___
  edk2-devel mailing list
  edk2-devel@lists.01.org
  https://lists.01.org/mailman/listinfo/edk2-devel
 ___
 edk2-devel mailing list
 edk2-devel@lists.01.org
 https://lists.01.org/mailman/listinfo/edk2-devel
___
edk2-devel mailing list
edk2-devel@lists.01.org
https://lists.01.org/mailman/listinfo/edk2-devel

Re: [edk2] BaseTools features: multiple workspaces

[Jordan Justen]

On 2015-08-03 02:08:14, Gao, Liming wrote:
 Paolo:
   I think that keep the exiting syntax for WORKSPACE to be a single
   path can minimize the impacts to exiting tools that assume a
   single workspace.

I know you originally suggested to use WORKSPACE, but got some
feedback that it might be better to create a new variable to not break
some unspecified tools.

Was anyone able to name an actual tool that would be impacted? Why not
work to update these tools at the same time? An easy 'fix' will be for
any such tool to give an error message if it sees ':' or ';' in the
WORKSPACE environment variable. Although this is really not a fix, it
will be just as much support as if they ignore the new
WORKSPACE_MULTIPLE environment variable.

I think that using the single separated WORKSPACE environment variable
is more intuitive/expected. From using env vars such as PATH, we all
know what it means.

For WORKSPACE+WORKSPACE_MULTIPLE, I don't know which has priority. I
assume WORKSPACE is checked first, and then WORKSPACE_MULTIPLE?

Will this work for the integrated CryptoPkg + open-ssl? It seems like
we need a priority above EDK II for this. I guess WORKSPACE will not
be set to EDK II in this case?

Anyway, it just seems a little odd.

-Jordan

 -Original Message-
 From: edk2-devel [mailto:edk2-devel-boun...@lists.01.org] On Behalf Of Paolo 
 Bonzini
 Sent: Monday, August 03, 2015 4:31 PM
 To: Gao, Liming; edk2-devel@lists.01.org
 Subject: Re: [edk2] BaseTools features: multiple workspaces
 
 
 On 03/08/2015 05:56, Gao, Liming wrote:
  Hi, all
We will update BaseTools feature to allow more than one workspaces. The 
  detail design in the below. Please help review it. If you have any 
  comments, please let me know.
  
  1.   Keep $(WORKSPACE) environment as is
  
  a.   $(WORKSPACE) determines location of Build and Conf directory.
  
  2.   New optional $(WORKSPACE_MULTIPLE) environment is added to include 
  more directories with the separator ';', like $(PATH)
 
 Why is $(WORKSPACE_MULTIPLE) necessary?  On Linux it is okay to break 
 preexisting paths that contain a :; on Windows semicolons are allowed in 
 theory but in practice break in several ways.
 
 Paolo
 
  a.   Produce the same behavior if $(WORKSPACE_MULTIPLE) is not set.
  
  3.   Update edksetup.bat/edksetup.sh to find BaseTools directory from 
  $(WORKSPACE) and $(WORKSPACE_MULTIPLE) directories.
  
  4.   Update BaseTools to support multiple workspaces
  
  a.   For the relative file path (INF, DSC and FDF), BaseTools will 
  search them from $(WORKSPACE) and $(WORKSPACE_MULTIPLE) directories.
  
  b.  Search priority from high to low: $(WORKSPACE), 
  $(WORKSPACE_MULTIPLE).
  
  This is a compatible feature. It has no impact on current EDKII project.
 ___
 edk2-devel mailing list
 edk2-devel@lists.01.org
 https://lists.01.org/mailman/listinfo/edk2-devel
 ___
 edk2-devel mailing list
 edk2-devel@lists.01.org
 https://lists.01.org/mailman/listinfo/edk2-devel
___
edk2-devel mailing list
edk2-devel@lists.01.org
https://lists.01.org/mailman/listinfo/edk2-devel

Re: [edk2] BaseTools features: multiple workspaces

[Tim Lewis]

Actually, I have less problem with the type of change being proposed, and more 
with the way in which it is accomplished.

Today: set up the build environment requires: set up my tools, set up 
WORKSPACE, build. To build any project, that's it.

But with the proposed solution, this is not enough, because the build/QA 
engineer (or auto-build system) must now know more about the source code. For 
example, they need to know the directory of the other workspace.

It would be better IMO to put the source location in the .dsc file. This allows 
the .dsc file to remain (as it is now) the arbiter of all project source code 
locations. Yes, you can have a default to an environment variable or to a 
build.exe command-line option, but if the .dsc has this, then I can deliver a 
complete buildable archive and not require the build engineer to know any 
additional steps ona per-project basis.
---
Separate topic: I think that overloading $(WORKSPACE) is problematic for 
understanding what will occur when I look for my (a) build output files, (b) my 
build configuration files and (c) the files referred to by $(WORKSPACE) .inf, 
.dsc and .dec files. Since WORKSPACE has a meaning for all of these, and since 
the $(WORKSPACE) syntax implies direct substitution (like in make files and 
other EDK2 usage), it is hard to predict the behavior when $(WORKSPACE) is 
used.  There is a difference between the root of the EDK2 build (where build 
rules come from and where output files go) and the root of the source code. I 
don't think the current proposal disambiguates these.

Tim


-Original Message-
From: edk2-devel [mailto:edk2-devel-boun...@lists.01.org] On Behalf Of Jordan 
Justen
Sent: Monday, August 03, 2015 11:55 AM
To: Tim Lewis tim.le...@insyde.com; Gao, Liming liming@intel.com; Paolo 
Bonzini pbonz...@redhat.com
Cc: edk2-devel@lists.01.org
Subject: Re: [edk2] BaseTools features: multiple workspaces

Cc: edk2-devel

On 2015-08-03 10:41:43, Tim Lewis wrote:
 There are dozens of tools in commercial environments (including some 
 by Insyde) which will fail in this case. In addition, there are 
 codebases that I am aware of where $(WORKSPACE) is allowed directly in 
 other build files.

Will all these 'dozens of tools in commercial environments' fare any better 
with the WORKSPACE_MULTIPLE option?

As a quick example, say C:\ssl\CryptoPkg has open-ssl integrated, and
C:\edk2 is the plain EDK II tree.

Thus, we want to set WORKSPACE=C:\ssl and WORKSPACE_MULTIPLE=C:\edk2 so the 
open-ssl integrated version of CryptoPkg will take priority.

Are these tools likely to work as expected? I am guessing no. I'm thinking that 
these tools will probably have to insist on a fully integrated single workspace 
until they can be updated to the new design.

At that point whether WORKSPACE can have multiple directories, or we add 
WORKSPACE_MULTIPLE, it will not make a big difference to these tools.

-Jordan

 -Original Message-
 From: edk2-devel [mailto:edk2-devel-boun...@lists.01.org] On Behalf Of 
 Jordan Justen
 Sent: Monday, August 03, 2015 10:36 AM
 To: Gao, Liming liming@intel.com; Paolo Bonzini 
 pbonz...@redhat.com; edk2-devel@lists.01.org
 Subject: Re: [edk2] BaseTools features: multiple workspaces
 
 On 2015-08-03 02:08:14, Gao, Liming wrote:
  Paolo:
I think that keep the exiting syntax for WORKSPACE to be a single
path can minimize the impacts to exiting tools that assume a
single workspace.
 
 I know you originally suggested to use WORKSPACE, but got some feedback that 
 it might be better to create a new variable to not break some unspecified 
 tools.
 
 Was anyone able to name an actual tool that would be impacted? Why not work 
 to update these tools at the same time? An easy 'fix' will be for any such 
 tool to give an error message if it sees ':' or ';' in the WORKSPACE 
 environment variable. Although this is really not a fix, it will be just as 
 much support as if they ignore the new WORKSPACE_MULTIPLE environment 
 variable.
 
 I think that using the single separated WORKSPACE environment variable is 
 more intuitive/expected. From using env vars such as PATH, we all know what 
 it means.
 
 For WORKSPACE+WORKSPACE_MULTIPLE, I don't know which has priority. I assume 
 WORKSPACE is checked first, and then WORKSPACE_MULTIPLE?
 
 Will this work for the integrated CryptoPkg + open-ssl? It seems like we need 
 a priority above EDK II for this. I guess WORKSPACE will not be set to EDK II 
 in this case?
 
 Anyway, it just seems a little odd.
 
 -Jordan
 
  -Original Message-
  From: edk2-devel [mailto:edk2-devel-boun...@lists.01.org] On Behalf 
  Of Paolo Bonzini
  Sent: Monday, August 03, 2015 4:31 PM
  To: Gao, Liming; edk2-devel@lists.01.org
  Subject: Re: [edk2] BaseTools features: multiple workspaces
  
  
  On 03/08/2015 05:56, Gao, Liming wrote:
   Hi, all
 We will update BaseTools feature to allow more than one workspaces. The 
   detail design in the below

Re: [edk2] BaseTools features: multiple workspaces

[Kinney, Michael D]

Jordan,

If we make a slight change to the requirements, we can provide clear priority 
rules.

WORKSPACE is used to find Build and Conf.

If WORKSPACE_MULTIPLE exists, then it is used as a prioritized list of paths to 
resolve all other workspace relative paths.  This means WORKSPACE_MULTIPLE will 
need to also contain that path to WORKSPACE as one of its path if WORKSPACE has 
packages in it too.  This means rule 4b needs to be adjusted.

Mike

-Original Message-
From: edk2-devel [mailto:edk2-devel-boun...@lists.01.org] On Behalf Of Jordan 
Justen
Sent: Monday, August 03, 2015 11:55 AM
To: Tim Lewis; Gao, Liming; Paolo Bonzini
Cc: edk2-devel@lists.01.org
Subject: Re: [edk2] BaseTools features: multiple workspaces

Cc: edk2-devel

On 2015-08-03 10:41:43, Tim Lewis wrote:
 There are dozens of tools in commercial environments (including some
 by Insyde) which will fail in this case. In addition, there are
 codebases that I am aware of where $(WORKSPACE) is allowed directly
 in other build files.

Will all these 'dozens of tools in commercial environments' fare any
better with the WORKSPACE_MULTIPLE option?

As a quick example, say C:\ssl\CryptoPkg has open-ssl integrated, and
C:\edk2 is the plain EDK II tree.

Thus, we want to set WORKSPACE=C:\ssl and WORKSPACE_MULTIPLE=C:\edk2
so the open-ssl integrated version of CryptoPkg will take priority.

Are these tools likely to work as expected? I am guessing no. I'm
thinking that these tools will probably have to insist on a fully
integrated single workspace until they can be updated to the new
design.

At that point whether WORKSPACE can have multiple directories, or we
add WORKSPACE_MULTIPLE, it will not make a big difference to these
tools.

-Jordan

 -Original Message-
 From: edk2-devel [mailto:edk2-devel-boun...@lists.01.org] On Behalf Of Jordan 
 Justen
 Sent: Monday, August 03, 2015 10:36 AM
 To: Gao, Liming liming@intel.com; Paolo Bonzini pbonz...@redhat.com; 
 edk2-devel@lists.01.org
 Subject: Re: [edk2] BaseTools features: multiple workspaces
 
 On 2015-08-03 02:08:14, Gao, Liming wrote:
  Paolo:
I think that keep the exiting syntax for WORKSPACE to be a single
path can minimize the impacts to exiting tools that assume a
single workspace.
 
 I know you originally suggested to use WORKSPACE, but got some feedback that 
 it might be better to create a new variable to not break some unspecified 
 tools.
 
 Was anyone able to name an actual tool that would be impacted? Why not work 
 to update these tools at the same time? An easy 'fix' will be for any such 
 tool to give an error message if it sees ':' or ';' in the WORKSPACE 
 environment variable. Although this is really not a fix, it will be just as 
 much support as if they ignore the new WORKSPACE_MULTIPLE environment 
 variable.
 
 I think that using the single separated WORKSPACE environment variable is 
 more intuitive/expected. From using env vars such as PATH, we all know what 
 it means.
 
 For WORKSPACE+WORKSPACE_MULTIPLE, I don't know which has priority. I assume 
 WORKSPACE is checked first, and then WORKSPACE_MULTIPLE?
 
 Will this work for the integrated CryptoPkg + open-ssl? It seems like we need 
 a priority above EDK II for this. I guess WORKSPACE will not be set to EDK II 
 in this case?
 
 Anyway, it just seems a little odd.
 
 -Jordan
 
  -Original Message-
  From: edk2-devel [mailto:edk2-devel-boun...@lists.01.org] On Behalf Of 
  Paolo Bonzini
  Sent: Monday, August 03, 2015 4:31 PM
  To: Gao, Liming; edk2-devel@lists.01.org
  Subject: Re: [edk2] BaseTools features: multiple workspaces
  
  
  On 03/08/2015 05:56, Gao, Liming wrote:
   Hi, all
 We will update BaseTools feature to allow more than one workspaces. The 
   detail design in the below. Please help review it. If you have any 
   comments, please let me know.
   
   1.   Keep $(WORKSPACE) environment as is
   
   a.   $(WORKSPACE) determines location of Build and Conf directory.
   
   2.   New optional $(WORKSPACE_MULTIPLE) environment is added to 
   include more directories with the separator ';', like $(PATH)
  
  Why is $(WORKSPACE_MULTIPLE) necessary?  On Linux it is okay to break 
  preexisting paths that contain a :; on Windows semicolons are allowed in 
  theory but in practice break in several ways.
  
  Paolo
  
   a.   Produce the same behavior if $(WORKSPACE_MULTIPLE) is not set.
   
   3.   Update edksetup.bat/edksetup.sh to find BaseTools directory from 
   $(WORKSPACE) and $(WORKSPACE_MULTIPLE) directories.
   
   4.   Update BaseTools to support multiple workspaces
   
   a.   For the relative file path (INF, DSC and FDF), BaseTools will 
   search them from $(WORKSPACE) and $(WORKSPACE_MULTIPLE) directories.
   
   b.  Search priority from high to low: $(WORKSPACE), 
   $(WORKSPACE_MULTIPLE).
   
   This is a compatible feature. It has no impact on current EDKII 
   project

Re: [edk2] BaseTools features: multiple workspaces

[Ni, Ruiyu]

Path separator in Linux seems to be :. Will you always use ;?

Thanks,
Ray

 -Original Message-
 From: edk2-devel [mailto:edk2-devel-boun...@lists.01.org] On Behalf Of
 Gao, Liming
 Sent: Monday, August 3, 2015 11:57 AM
 To: edk2-devel@lists.01.org
 Subject: [edk2] BaseTools features: multiple workspaces
 
 Hi, all
   We will update BaseTools feature to allow more than one workspaces.
 The detail design in the below. Please help review it. If you have any
 comments, please let me know.
 
 1.   Keep $(WORKSPACE) environment as is
 
 a.   $(WORKSPACE) determines location of Build and Conf directory.
 
 2.   New optional $(WORKSPACE_MULTIPLE) environment is added to
 include more directories with the separator ';', like $(PATH)
 
 a.   Produce the same behavior if $(WORKSPACE_MULTIPLE) is not
 set.
 
 3.   Update edksetup.bat/edksetup.sh to find BaseTools directory
 from $(WORKSPACE) and $(WORKSPACE_MULTIPLE) directories.
 
 4.   Update BaseTools to support multiple workspaces
 
 a.   For the relative file path (INF, DSC and FDF), BaseTools will search
 them from $(WORKSPACE) and $(WORKSPACE_MULTIPLE) directories.
 
 b.  Search priority from high to low: $(WORKSPACE),
 $(WORKSPACE_MULTIPLE).
 
 This is a compatible feature. It has no impact on current EDKII project.
 
 Thanks
 Liming
 ___
 edk2-devel mailing list
 edk2-devel@lists.01.org
 https://lists.01.org/mailman/listinfo/edk2-devel
___
edk2-devel mailing list
edk2-devel@lists.01.org
https://lists.01.org/mailman/listinfo/edk2-devel