Re: [edk2] [PATCH] Convert PosixLike scripts to sh, reduce duplication and fix some bugs

2019-02-25 Thread Rebecca Cran via edk2-devel
Thanks for the review. I have a macOS system, so I'll verify future 
changes work there too.


The problem I'm trying to solve is to make it easier to get started 
building EDK2 under FreeBSD. Currently it needs symlinks in 
BaseTools/Bin/FreeBSD-amd64 for gcc, ld, make, objcopy and python, and 
obviously needs bash installed. Since there seems to be relatively 
little shell code in the repo, I was thinking that avoiding the 
dependency on bash might be a relatively easy way to reduce the steps 
needed to get things working.



I'll try and remember about PatchCheck.py too.


--

Rebecca


On 2/25/19 8:11 PM, Feng, Bob C wrote:

Hi Rebecca,

I like the change of moving duplicated code into common files and the bug fix 
looks good.

I tested this patch on our CI system, it break the build on MacOS. There is no 
realpath command on MacOs shell.

For the patch itself, it need to pass the check of 
BaseTools/Scripts/PatchCheck.py.

Is there actual problem that need to be resolved by using sh instead of bash?

Thanks,
Bob

-Original Message-
From: edk2-devel [mailto:edk2-devel-boun...@lists.01.org] On Behalf Of Rebecca 
Cran via edk2-devel
Sent: Friday, February 22, 2019 1:13 PM
To: edk2-devel@lists.01.org
Subject: [edk2] [PATCH] Convert PosixLike scripts to sh, reduce duplication and 
fix some bugs

Since the scripts in the PosixLike directory are very simple, convert them to 
use /bin/sh, and move duplicated code into common files (one for python 
scripts, one for C binaries).

Fix some bugs in edksetup.sh and BuildEnv: `test` uses single equals instead of 
double equals, and should have a space before the ending bracket. Fix some 
indentation, and if the user runs edksetup.sh with --help etc., quit after 
displaying the help message.
---
  BaseTools/BinWrappers/PosixLike/BPDG  | 15 ++
  BaseTools/BinWrappers/PosixLike/Brotli| 30 ++-
  .../BinWrappers/PosixLike/BrotliCompress  | 12 
  BaseTools/BinWrappers/PosixLike/DevicePath| 30 ++-
  BaseTools/BinWrappers/PosixLike/Ecc   | 15 ++
  BaseTools/BinWrappers/PosixLike/EfiRom| 30 ++-
  BaseTools/BinWrappers/PosixLike/GenCrc32  | 30 ++-
  BaseTools/BinWrappers/PosixLike/GenDepex  | 15 ++
  BaseTools/BinWrappers/PosixLike/GenFds| 15 ++
  BaseTools/BinWrappers/PosixLike/GenFfs| 30 ++-
  BaseTools/BinWrappers/PosixLike/GenFv | 30 ++-
  BaseTools/BinWrappers/PosixLike/GenFw | 30 ++-
  .../BinWrappers/PosixLike/GenPatchPcdTable| 15 ++
  BaseTools/BinWrappers/PosixLike/GenSec| 30 ++-
  .../BinWrappers/PosixLike/GenerateCapsule | 15 ++
  BaseTools/BinWrappers/PosixLike/LzmaCompress  | 30 ++-
  .../BinWrappers/PosixLike/LzmaF86Compress |  2 +-
  BaseTools/BinWrappers/PosixLike/PatchPcdValue | 15 ++
  BaseTools/BinWrappers/PosixLike/Pkcs7Sign | 15 ++
  .../PosixLike/Rsa2048Sha256GenerateKeys   | 15 ++
  .../BinWrappers/PosixLike/Rsa2048Sha256Sign   | 15 ++
  BaseTools/BinWrappers/PosixLike/Split | 30 ++-
  BaseTools/BinWrappers/PosixLike/TargetTool| 15 ++
  BaseTools/BinWrappers/PosixLike/TianoCompress | 30 ++-
  BaseTools/BinWrappers/PosixLike/Trim  | 15 ++
  BaseTools/BinWrappers/PosixLike/UPT   | 15 ++
  BaseTools/BinWrappers/PosixLike/VfrCompile| 30 ++-
  BaseTools/BinWrappers/PosixLike/VolInfo   | 30 ++-
  BaseTools/BinWrappers/PosixLike/build | 15 ++
  BaseTools/BinWrappers/PosixLike/common.sh | 12 
  BaseTools/BinWrappers/PosixLike/commonbin.sh  | 28 +
  BaseTools/BuildEnv|  3 +-
  edksetup.sh   | 12 
  33 files changed, 110 insertions(+), 559 deletions(-)  create mode 100644 
BaseTools/BinWrappers/PosixLike/common.sh
  create mode 100644 BaseTools/BinWrappers/PosixLike/commonbin.sh

diff --git a/BaseTools/BinWrappers/PosixLike/BPDG 
b/BaseTools/BinWrappers/PosixLike/BPDG
index c894384908..a9da3afb4b 100755
--- a/BaseTools/BinWrappers/PosixLike/BPDG
+++ b/BaseTools/BinWrappers/PosixLike/BPDG
@@ -1,14 +1,3 @@
-#!/usr/bin/env bash
-#python `dirname $0`/RunToolFromSource.py `basename $0` $*
+#!/bin/sh
  
-# If a ${PYTHON_COMMAND} command is available, use it in preference to python -if command -v ${PYTHON_COMMAND} >/dev/null 2>&1; then

-python_exe=${PYTHON_COMMAND}
-fi
-
-full_cmd=${BASH_SOURCE:-$0} # see http://mywiki.wooledge.org/BashFAQ/028 for a 
discussion of why $0 is not a good choice here -dir=$(dirname "$full_cmd") 
-cmd=${full_cmd##*/}
-
-export PYTHONPATH="$dir/../../Source/Python${PYTHONPATH:+:"$PYTHONPATH"}"
-exec "${python_

Re: [edk2] [PATCH] Convert PosixLike scripts to sh, reduce duplication and fix some bugs

2019-02-25 Thread Feng, Bob C
Hi Rebecca,

I like the change of moving duplicated code into common files and the bug fix 
looks good.

I tested this patch on our CI system, it break the build on MacOS. There is no 
realpath command on MacOs shell.

For the patch itself, it need to pass the check of 
BaseTools/Scripts/PatchCheck.py.

Is there actual problem that need to be resolved by using sh instead of bash?

Thanks,
Bob

-Original Message-
From: edk2-devel [mailto:edk2-devel-boun...@lists.01.org] On Behalf Of Rebecca 
Cran via edk2-devel
Sent: Friday, February 22, 2019 1:13 PM
To: edk2-devel@lists.01.org
Subject: [edk2] [PATCH] Convert PosixLike scripts to sh, reduce duplication and 
fix some bugs

Since the scripts in the PosixLike directory are very simple, convert them to 
use /bin/sh, and move duplicated code into common files (one for python 
scripts, one for C binaries).

Fix some bugs in edksetup.sh and BuildEnv: `test` uses single equals instead of 
double equals, and should have a space before the ending bracket. Fix some 
indentation, and if the user runs edksetup.sh with --help etc., quit after 
displaying the help message.
---
 BaseTools/BinWrappers/PosixLike/BPDG  | 15 ++
 BaseTools/BinWrappers/PosixLike/Brotli| 30 ++-
 .../BinWrappers/PosixLike/BrotliCompress  | 12 
 BaseTools/BinWrappers/PosixLike/DevicePath| 30 ++-
 BaseTools/BinWrappers/PosixLike/Ecc   | 15 ++
 BaseTools/BinWrappers/PosixLike/EfiRom| 30 ++-
 BaseTools/BinWrappers/PosixLike/GenCrc32  | 30 ++-
 BaseTools/BinWrappers/PosixLike/GenDepex  | 15 ++
 BaseTools/BinWrappers/PosixLike/GenFds| 15 ++
 BaseTools/BinWrappers/PosixLike/GenFfs| 30 ++-
 BaseTools/BinWrappers/PosixLike/GenFv | 30 ++-
 BaseTools/BinWrappers/PosixLike/GenFw | 30 ++-
 .../BinWrappers/PosixLike/GenPatchPcdTable| 15 ++
 BaseTools/BinWrappers/PosixLike/GenSec| 30 ++-
 .../BinWrappers/PosixLike/GenerateCapsule | 15 ++
 BaseTools/BinWrappers/PosixLike/LzmaCompress  | 30 ++-
 .../BinWrappers/PosixLike/LzmaF86Compress |  2 +-
 BaseTools/BinWrappers/PosixLike/PatchPcdValue | 15 ++
 BaseTools/BinWrappers/PosixLike/Pkcs7Sign | 15 ++
 .../PosixLike/Rsa2048Sha256GenerateKeys   | 15 ++
 .../BinWrappers/PosixLike/Rsa2048Sha256Sign   | 15 ++
 BaseTools/BinWrappers/PosixLike/Split | 30 ++-
 BaseTools/BinWrappers/PosixLike/TargetTool| 15 ++
 BaseTools/BinWrappers/PosixLike/TianoCompress | 30 ++-
 BaseTools/BinWrappers/PosixLike/Trim  | 15 ++
 BaseTools/BinWrappers/PosixLike/UPT   | 15 ++
 BaseTools/BinWrappers/PosixLike/VfrCompile| 30 ++-
 BaseTools/BinWrappers/PosixLike/VolInfo   | 30 ++-
 BaseTools/BinWrappers/PosixLike/build | 15 ++
 BaseTools/BinWrappers/PosixLike/common.sh | 12 
 BaseTools/BinWrappers/PosixLike/commonbin.sh  | 28 +
 BaseTools/BuildEnv|  3 +-
 edksetup.sh   | 12 
 33 files changed, 110 insertions(+), 559 deletions(-)  create mode 100644 
BaseTools/BinWrappers/PosixLike/common.sh
 create mode 100644 BaseTools/BinWrappers/PosixLike/commonbin.sh

diff --git a/BaseTools/BinWrappers/PosixLike/BPDG 
b/BaseTools/BinWrappers/PosixLike/BPDG
index c894384908..a9da3afb4b 100755
--- a/BaseTools/BinWrappers/PosixLike/BPDG
+++ b/BaseTools/BinWrappers/PosixLike/BPDG
@@ -1,14 +1,3 @@
-#!/usr/bin/env bash
-#python `dirname $0`/RunToolFromSource.py `basename $0` $*
+#!/bin/sh
 
-# If a ${PYTHON_COMMAND} command is available, use it in preference to python 
-if command -v ${PYTHON_COMMAND} >/dev/null 2>&1; then
-python_exe=${PYTHON_COMMAND}
-fi
-
-full_cmd=${BASH_SOURCE:-$0} # see http://mywiki.wooledge.org/BashFAQ/028 for a 
discussion of why $0 is not a good choice here -dir=$(dirname "$full_cmd") 
-cmd=${full_cmd##*/}
-
-export PYTHONPATH="$dir/../../Source/Python${PYTHONPATH:+:"$PYTHONPATH"}"
-exec "${python_exe:-python}" -m $cmd.$cmd "$@"
+. "$(dirname "$(realpath "$0")")/common.sh"
diff --git a/BaseTools/BinWrappers/PosixLike/Brotli 
b/BaseTools/BinWrappers/PosixLike/Brotli
index 0945d86d92..3dfa26e517 100755
--- a/BaseTools/BinWrappers/PosixLike/Brotli
+++ b/BaseTools/BinWrappers/PosixLike/Brotli
@@ -1,29 +1,3 @@
-#!/usr/bin/env bash
-
-full_cmd=${BASH_SOURCE:-$0} # see http://mywiki.wooledge.org/BashFAQ/028 for a 
discussion of why $0 is not a good choice here -dir=$(dirname "$full_cmd") 
-cmd=${full_cmd##*/}
-
-if [ -n "$WORKSPACE" ] && [ -e "$WORKSPACE/Conf/BaseToolsCBinaries" ] -then
-  

[edk2] [PATCH] Convert PosixLike scripts to sh, reduce duplication and fix some bugs

2019-02-21 Thread Rebecca Cran via edk2-devel
Since the scripts in the PosixLike directory are very simple, convert
them to use /bin/sh, and move duplicated code into common files (one for
python scripts, one for C binaries).

Fix some bugs in edksetup.sh and BuildEnv: `test` uses single equals instead
of double equals, and should have a space before the ending bracket. Fix
some indentation, and if the user runs edksetup.sh with --help etc., quit
after displaying the help message.
---
 BaseTools/BinWrappers/PosixLike/BPDG  | 15 ++
 BaseTools/BinWrappers/PosixLike/Brotli| 30 ++-
 .../BinWrappers/PosixLike/BrotliCompress  | 12 
 BaseTools/BinWrappers/PosixLike/DevicePath| 30 ++-
 BaseTools/BinWrappers/PosixLike/Ecc   | 15 ++
 BaseTools/BinWrappers/PosixLike/EfiRom| 30 ++-
 BaseTools/BinWrappers/PosixLike/GenCrc32  | 30 ++-
 BaseTools/BinWrappers/PosixLike/GenDepex  | 15 ++
 BaseTools/BinWrappers/PosixLike/GenFds| 15 ++
 BaseTools/BinWrappers/PosixLike/GenFfs| 30 ++-
 BaseTools/BinWrappers/PosixLike/GenFv | 30 ++-
 BaseTools/BinWrappers/PosixLike/GenFw | 30 ++-
 .../BinWrappers/PosixLike/GenPatchPcdTable| 15 ++
 BaseTools/BinWrappers/PosixLike/GenSec| 30 ++-
 .../BinWrappers/PosixLike/GenerateCapsule | 15 ++
 BaseTools/BinWrappers/PosixLike/LzmaCompress  | 30 ++-
 .../BinWrappers/PosixLike/LzmaF86Compress |  2 +-
 BaseTools/BinWrappers/PosixLike/PatchPcdValue | 15 ++
 BaseTools/BinWrappers/PosixLike/Pkcs7Sign | 15 ++
 .../PosixLike/Rsa2048Sha256GenerateKeys   | 15 ++
 .../BinWrappers/PosixLike/Rsa2048Sha256Sign   | 15 ++
 BaseTools/BinWrappers/PosixLike/Split | 30 ++-
 BaseTools/BinWrappers/PosixLike/TargetTool| 15 ++
 BaseTools/BinWrappers/PosixLike/TianoCompress | 30 ++-
 BaseTools/BinWrappers/PosixLike/Trim  | 15 ++
 BaseTools/BinWrappers/PosixLike/UPT   | 15 ++
 BaseTools/BinWrappers/PosixLike/VfrCompile| 30 ++-
 BaseTools/BinWrappers/PosixLike/VolInfo   | 30 ++-
 BaseTools/BinWrappers/PosixLike/build | 15 ++
 BaseTools/BinWrappers/PosixLike/common.sh | 12 
 BaseTools/BinWrappers/PosixLike/commonbin.sh  | 28 +
 BaseTools/BuildEnv|  3 +-
 edksetup.sh   | 12 
 33 files changed, 110 insertions(+), 559 deletions(-)
 create mode 100644 BaseTools/BinWrappers/PosixLike/common.sh
 create mode 100644 BaseTools/BinWrappers/PosixLike/commonbin.sh

diff --git a/BaseTools/BinWrappers/PosixLike/BPDG 
b/BaseTools/BinWrappers/PosixLike/BPDG
index c894384908..a9da3afb4b 100755
--- a/BaseTools/BinWrappers/PosixLike/BPDG
+++ b/BaseTools/BinWrappers/PosixLike/BPDG
@@ -1,14 +1,3 @@
-#!/usr/bin/env bash
-#python `dirname $0`/RunToolFromSource.py `basename $0` $*
+#!/bin/sh
 
-# If a ${PYTHON_COMMAND} command is available, use it in preference to python
-if command -v ${PYTHON_COMMAND} >/dev/null 2>&1; then
-python_exe=${PYTHON_COMMAND}
-fi
-
-full_cmd=${BASH_SOURCE:-$0} # see http://mywiki.wooledge.org/BashFAQ/028 for a 
discussion of why $0 is not a good choice here
-dir=$(dirname "$full_cmd")
-cmd=${full_cmd##*/}
-
-export PYTHONPATH="$dir/../../Source/Python${PYTHONPATH:+:"$PYTHONPATH"}"
-exec "${python_exe:-python}" -m $cmd.$cmd "$@"
+. "$(dirname "$(realpath "$0")")/common.sh"
diff --git a/BaseTools/BinWrappers/PosixLike/Brotli 
b/BaseTools/BinWrappers/PosixLike/Brotli
index 0945d86d92..3dfa26e517 100755
--- a/BaseTools/BinWrappers/PosixLike/Brotli
+++ b/BaseTools/BinWrappers/PosixLike/Brotli
@@ -1,29 +1,3 @@
-#!/usr/bin/env bash
-
-full_cmd=${BASH_SOURCE:-$0} # see http://mywiki.wooledge.org/BashFAQ/028 for a 
discussion of why $0 is not a good choice here
-dir=$(dirname "$full_cmd")
-cmd=${full_cmd##*/}
-
-if [ -n "$WORKSPACE" ] && [ -e "$WORKSPACE/Conf/BaseToolsCBinaries" ]
-then
-  exec "$WORKSPACE/Conf/BaseToolsCBinaries/$cmd"
-elif [ -n "$WORKSPACE" ] && [ -e "$EDK_TOOLS_PATH/Source/C" ]
-then
-  if [ ! -e "$EDK_TOOLS_PATH/Source/C/bin/$cmd" ]
-  then
-echo "BaseTools C Tool binary was not found ($cmd)"
-echo "You may need to run:"
-echo "  make -C $EDK_TOOLS_PATH/Source/C"
-  else
-exec "$EDK_TOOLS_PATH/Source/C/bin/$cmd" "$@"
-  fi
-elif [ -e "$dir/../../Source/C/bin/$cmd" ]
-then
-  exec "$dir/../../Source/C/bin/$cmd" "$@"
-else
-  echo "Unable to find the real '$cmd' to run"
-  echo "This message was printed by"
-  echo "  $0"
-  exit 127
-fi
+#!/bin/sh
 
+. "$(dirname "$(realpath "$0")")/commonbin.sh"
diff --git a/BaseTools/BinWrappers/PosixLike/BrotliCompress 
b/BaseTools/BinWrappers/PosixLike/BrotliCompress
index ea86705a28..7cb26a3659 100755
---