Re: [Qemu-devel] [PATCH] script: git script to compile every commit in a range of commits

2013-06-10 Thread Peter Crosthwaite 
Hi,

On Sat, Jun 8, 2013 at 6:30 AM, Jeff Cody jc...@redhat.com wrote:
 On Fri, Jun 07, 2013 at 11:51:36AM -0500, Anthony Liguori wrote:
 Laszlo Ersek ler...@redhat.com writes:

  On 06/07/13 16:44, Jeff Cody wrote:
 
  Thanks.  I can either do the above changes for a v2, or as follow on
  patches.
 
  Whichever is easier for you, certainly! I'm fine with the script
  going-in as is.

 A suggestion I'll make is to split the script into two parts.

 Hi Anthony,

 I'm sorry, but I'm a bit confused by your suggestion.  I think I know
 what you are asking (see below), but I'm not positive.

 git-bisect run is a terribly useful command and I use a bisect script
 that looks like this:

 #!/bin/sh

 set -e

 pushd ~/build/qemu
 make -j1 || exit 1
 popd

 # Add right seed here
 if test $1; then
 $@
 fi

 I'm sure we all have bisect scripts like this.

 What you're proposing is very similar to bisect but instead of doing a
 binary search, it does a linear search starting from the oldest commit.
 Basically:

 I agree that git bisect is useful, but solves a slightly different
 problem than what I am looking to solve.

 For instance, in my working branches I have a whole stack of commits
 that I will rebase and squash into a set of sane patches before
 submitting.  To make sure I don't do something silly in that process,
 and create a patch X that does not build without patch X+1, I want to
 explicitly compile each patch, without skipping over any patches.


 #!/bin/sh

 refspec=$1
 shift

 git rev-list $refspec | while read commit; do
 git checkout $commit
 $@ || exit $?
 done

 And indeed, I have a local script called git-foreach to do exactly
 this.  I suspect a nicer version would make a very good addition to the
 git project.

 So to bisect for a make check failure, I do:

   git bisect run bisect.sh make check

 Or to bisect for a qemu-test failure:

   git bisect run bisect.sh qemu-test-regress.sh

 With git-foreach, you can do:

   git-foreach bisect.sh

 To do a simple build test.  Or you can do:

   git-foreach git show checkpatch-head.sh

 etc.

 Ah!  So if I understand correctly, what you are asking is to split
 the script up into two different scripts:

 1.) A 'foreach' framework script to run an arbitrary command over a
 range of commits, against each commit  (i.e. in the place where I run
 'make clean, git checkout, configure, and make [lines 188-191], simply
 do the git checkout and execute a passed script / command).

 2.) A second script to perform the complication check, intended to be
 called by script 1).  We could then add additional scripts to be
 called by the 'foreach' framework patch as desired.

 Heck, if we wanted to, we could then even create a menu-drive
 meta-script to interactively run any number of tests (checkpatch,
 compilation, etc..) using that framework.


Make sense to me. I have a little script that does this stuff for me,
but my for-each mechanism runs using git am rather than commit ranges
and git checkout. Verifies that the series as-about-to-be-sent applies
cleanly to the master without build breakage or checkpatch fail.

If you make this two stage split developers can choose between either
a commit or am based foreach iterator and the second script as your
call it is common to both.

Regards,
Peter


 Regards,

 Anthony Liguori


 Thanks,

 Jeff

Re: [Qemu-devel] [PATCH] script: git script to compile every commit in a range of commits

2013-06-07 Thread Jeff Cody 
On Thu, Jun 06, 2013 at 10:58:25AM +0200, Laszlo Ersek wrote:
 comments below
 
 On 05/31/13 18:39, Jeff Cody wrote:
 
  +usage() {
  +echo 
  +echo $0 [OPTIONS]
  +echo $desc
  +echo 
  +echo OPTIONS:
  +echo   -r git range
  +optional; default is '$range'
  +
  +echo   -c configure options
  +optional; default is '$config_opt'
  +
  +echo   -m make options
  +optional; default is '$make_opt'
  +
  +echo   -d log dir
  +optional; default is '$logdir'
  +
  +echo   -l log filename
  +optional; default is output-PROCID, where PROCID is the bash 
  process id
  +note: you may specify a full path for the log filename here, 
  and exclude the
  +-d option.
  +
  +echo   -f force a git reset and clean
  +this will cause a 'git reset --hard; git clean -fdx' to be run 
  between checkouts.
  +!! WARNING: This may cause data loss in your git tree.
  +READ THE git-clean and git-reset man pages and make
  +sure you understand the implications of
  +'git clean -fdx' and 'git reset --hard' before 
  using !!
  +
  +echo   -h help
  +}
 
 Sorry for not noticing this before: is there any reason for the trailing
 spaces on most lines in the usage text?


No, not particularly, mainly just formatting with an extra newline,
and trying to keep the code somewhat lined up.

 Plus I suggest lower-casing the THE in READ THE.
 

I agree.

  +
  +while getopts :r:c:m:l:d:hf opt
  +do
  +case $opt in
  +r) range=$OPTARG
  +;;
  +c) config_opt=$OPTARG
  +;;
  +m) make_opt=$OPTARG
  +;;
  +d) logdir=$OPTARG
  +;;
  +l) logfile=$OPTARG
  +;;
  +f) force_clean='y'
  +;;
  +h) usage
  +   exit
  +;;
  +\?) echo Unknown option: -$OPTARG 2
  +usage
  +exit 1
  +;;
  +esac
  +done
  +
  +# append a '/' to logdir if $logdir was specified without one
  +[[ -n $logdir ]]  [[ ${logdir:${#logdir}-1} != / ]]  
  logdir=${logdir}/
  +
  +logfile=${logdir}${logfile}
  +
  +head=`git rev-parse HEAD`
  +total=`git rev-list $range |wc -l`
  +
  +echo log output: $logfile
  +
  +rm -f $logfile
 
 rm -f -- $logfile is safer, but I doubt anyone would pass a pathname
 starting with -...

You are right, and no reason not to use '--', so I should add that in.

 
  +date  $logfile
  +echo git compile check for $range.  $logfile
  +echo * configure options='$config_opt'  $logfile
  +echo * make options='$make_opt'  $logfile
  +echo Performing a test compile on $total patches | tee -a $logfile
  +echo -  
  $logfile
  +echo  | tee -a $logfile
  +
  +clean_repo() {
  +if [[ $force_clean == 'y' ]]
  +then
  +git reset --hard  $logfile 21 || true
  +git clean -fdx -e $logfile  $logfile 21 || true
  +fi
  +}
 
 Does -e mean except? It's not supported on RHEL-6.
 

Yes - this way it will preserve the log output (the default log path
is in the current working directory).  Looks like the -e option was
not introduced until 1.7.3, and RHEL-6 is on 1.7.1.

I'll add a check for the git version, and drop the -e if git is too
old to support it (and in that case, the user will need to make sure
if they supply the -f option to this script that they specify a
different log location than the cwd).

  +
  +# we want to cleanup and return the git tree back to the previous head
  +trap cleanup EXIT
  +
  +cleanup() {
  +echo 
  +echo -n Cleaning up...
  +clean_repo
  +git checkout $head  /dev/null 21
  +echo done.
  +}
  +
  +cnt=1
  +# don't pipe the git job into read, to avoid subshells
  +while read hash
  +do
  +txt=`git log --pretty=tformat:%h: %s $hash^!`
  +echo ${cnt}/${total}: compiling: $txt | tee -a $logfile
  +let cnt=$cnt+1;
  +echo   $logfile
  +clean_repo
  +make clean  /dev/null 21 || true
  +git checkout $hash   $logfile 21  \
  +./configure $config_opt  $logfile 21  \
  +make $make_opt   $logfile 21 ||
  +(
  +echo  | tee -a $logfile
  +echo ERROR: commit $hash failed to build! | tee -a $logfile
  +git show --stat $hash | tee -a $logfile
  +exit 1
  +)
  +done  (git log $range --pretty=tformat:%H --reverse)
  +
  +echo 
  +All patches in $range compiled successfully! | tee -a $logfile
  +exit 0
  
 
 I think my remarks are not important, the script should work in any
 practical environment. We should get this merged and small fixes can be
 posted incrementally if needed.

Re: [Qemu-devel] [PATCH] script: git script to compile every commit in a range of commits

2013-06-07 Thread Laszlo Ersek 
On 06/07/13 16:44, Jeff Cody wrote:

 Thanks.  I can either do the above changes for a v2, or as follow on
 patches.

Whichever is easier for you, certainly! I'm fine with the script
going-in as is.

Cheers,
Laszlo

Re: [Qemu-devel] [PATCH] script: git script to compile every commit in a range of commits

2013-06-07 Thread Peter Crosthwaite 
Hi Jeff,

On Sat, Jun 1, 2013 at 2:39 AM, Jeff Cody jc...@redhat.com wrote:
 This is a git script that will iterate through every commit in a
 specified range, and perform a configure and make.  The intention of
 this script is not to act as a check of code correctness, but to see if
 any commit breaks compilation of the tree.


Should we possibly throw a checkpatch in there?

 The idea is that prior to submitting a patch or patch series, the
 submitter should verify that no patch in the series breaks the build,
 thereby breaking git-bisect.  This script may also be useful for
 reviewers/maintainers dealing with multi-patch series.

 The range passed in to the script is in the form of a standard git range;
 the default is HEAD^!  (i.e. just the latest commit).

 Options may be passed in for configure, as well as make.  The log output
 filename and directory may also be optionally specified.  All options
 are stored via git-config.

 If everything compiled without error, then the exit code from the
 program will be 0.  Upon error, the stats for the offending
 commit will be shown.

 Signed-off-by: Jeff Cody jc...@redhat.com
 ---
  scripts/git-compile-check | 202 
 ++
  1 file changed, 202 insertions(+)
  create mode 100755 scripts/git-compile-check

 diff --git a/scripts/git-compile-check b/scripts/git-compile-check
 new file mode 100755
 index 000..4b90f91
 --- /dev/null
 +++ b/scripts/git-compile-check
 @@ -0,0 +1,202 @@
 +#!/bin/bash
 +#
 +# See $desc, below, for program description
 +#
 +# Copyright (c) 2013 Red Hat, Inc.
 +#
 +# Author(s):
 +#   Jeff Cody jc...@redhat.com
 +#
 +# This program is free software; you can redistribute it and/or modify it 
 under
 +# the terms of the GNU General Public License as published by the Free 
 Software
 +# Foundation; under version 2 of the license
 +#
 +# This program is distributed in the hope that it will be useful, but WITHOUT
 +# ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or 
 FITNESS
 +# FOR A PARTICULAR PURPOSE.  See the GNU General Public License for more
 +# details.
 +#
 +# You should have received a copy of the GNU General Public License along 
 with
 +# this program; if not, see http://www.gnu.org/licenses/gpl-2.0.html.
 +#
 +
 +set -C -u -e
 +set -o pipefail
 +
 +desc=
 +$0 iterates through a git commit range, and performs the
 +following on each commit:
 +  - git checkout
 +  - make clean
 +  - configure
 +  - make
 +
 +It will also optionally perform a git-reset and git-clean between
 +checkouts, if requested via the '-f' option.
 +
 +The script will exit and report on first error on any of the above steps,
 +(except no error checking is performed on 'make clean')
 +
 +NOTE: While executing, the script will checkout out each commit
 +  in the range in the current git tree.  On exit, the HEAD
 +  at the time the script was called is checked out
 +
 +
 +# default range is the last commit
 +def_range=HEAD^!
 +def_config_opt=--target-list=x86_64-softmmu
 +# you may want to have make perform multiple jobs, e.g. -j4
 +# this is ommitted as the default in case the project makefile
 +# is not safe for parallel make processes
 +def_make_opt=
 +def_log=output-$$.log
 +def_logdir=
 +force_clean='n'
 +
 +logfile=$def_log
 +range=`git config compile-check.range || true`
 +config_opt=`git config compile-check.configopt || true`
 +make_opt=`git config compile-check.makeopt || true`
 +logdir=`git config compile-check.logdir || true`
 +
 +if [[ -z $range ]]
 +then
 +range=$def_range
 +git config compile-check.range $range || true
 +fi
 +if [[ -z $config_opt ]]
 +then
 +config_opt=$def_config_opt
 +git config compile-check.configopt $config_opt || true
 +fi
 +if [[ -z $make_opt ]]
 +then
 +make_opt=$def_make_opt
 +git config compile-check.makeopt $make_opt || true
 +fi
 +if [[ -z $logdir ]]
 +then
 +logdir=$def_logdir
 +git config compile-check.logdir $logdir || true
 +fi
 +
 +usage() {
 +echo 
 +echo $0 [OPTIONS]
 +echo $desc
 +echo 
 +echo OPTIONS:
 +echo   -r git range
 +optional; default is '$range'
 +
 +echo   -c configure options
 +optional; default is '$config_opt'
 +
 +echo   -m make options
 +optional; default is '$make_opt'
 +
 +echo   -d log dir
 +optional; default is '$logdir'
 +
 +echo   -l log filename
 +optional; default is output-PROCID, where PROCID is the bash 
 process id
 +note: you may specify a full path for the log filename here, and 
 exclude the
 +-d option.
 +
 +echo   -f force a git reset and clean
 +this will cause a 'git reset --hard; git clean -fdx' to be run 
 between checkouts.
 +!! WARNING: This may cause data loss in your git tree.
 +

Re: [Qemu-devel] [PATCH] script: git script to compile every commit in a range of commits

2013-06-07 Thread Anthony Liguori 
Laszlo Ersek ler...@redhat.com writes:

 On 06/07/13 16:44, Jeff Cody wrote:

 Thanks.  I can either do the above changes for a v2, or as follow on
 patches.

 Whichever is easier for you, certainly! I'm fine with the script
 going-in as is.

A suggestion I'll make is to split the script into two parts.
git-bisect run is a terribly useful command and I use a bisect script
that looks like this:

#!/bin/sh

set -e

pushd ~/build/qemu
make -j1 || exit 1
popd

# Add right seed here
if test $1; then
$@
fi

I'm sure we all have bisect scripts like this.

What you're proposing is very similar to bisect but instead of doing a
binary search, it does a linear search starting from the oldest commit.
Basically:

#!/bin/sh

refspec=$1
shift

git rev-list $refspec | while read commit; do
git checkout $commit
$@ || exit $?
done

And indeed, I have a local script called git-foreach to do exactly
this.  I suspect a nicer version would make a very good addition to the
git project.

So to bisect for a make check failure, I do:

  git bisect run bisect.sh make check

Or to bisect for a qemu-test failure:

  git bisect run bisect.sh qemu-test-regress.sh

With git-foreach, you can do:

  git-foreach bisect.sh

To do a simple build test.  Or you can do:

  git-foreach git show checkpatch-head.sh

etc.

Regards,

Anthony Liguori


 Cheers,
 Laszlo

Re: [Qemu-devel] [PATCH] script: git script to compile every commit in a range of commits

2013-06-07 Thread Jeff Cody 
On Sat, Jun 08, 2013 at 02:36:33AM +1000, Peter Crosthwaite wrote:
 Hi Jeff,
 
 On Sat, Jun 1, 2013 at 2:39 AM, Jeff Cody jc...@redhat.com wrote:
  This is a git script that will iterate through every commit in a
  specified range, and perform a configure and make.  The intention of
  this script is not to act as a check of code correctness, but to see if
  any commit breaks compilation of the tree.
 
 
 Should we possibly throw a checkpatch in there?
 

Hi Peter,

That could be interesting, but I'd rather keep the patch more
single-purpose, focused on checking compile errors rather than also
patch formatting / code style checks.  I'm afraid the output would get
too complicated, and the purpose of the script itself unclear.

  The idea is that prior to submitting a patch or patch series, the
  submitter should verify that no patch in the series breaks the build,
  thereby breaking git-bisect.  This script may also be useful for
  reviewers/maintainers dealing with multi-patch series.
 
  The range passed in to the script is in the form of a standard git range;
  the default is HEAD^!  (i.e. just the latest commit).
 
  Options may be passed in for configure, as well as make.  The log output
  filename and directory may also be optionally specified.  All options
  are stored via git-config.
 
  If everything compiled without error, then the exit code from the
  program will be 0.  Upon error, the stats for the offending
  commit will be shown.
 
  Signed-off-by: Jeff Cody jc...@redhat.com
  ---
   scripts/git-compile-check | 202 
  ++
   1 file changed, 202 insertions(+)
   create mode 100755 scripts/git-compile-check
 
  diff --git a/scripts/git-compile-check b/scripts/git-compile-check
  new file mode 100755
  index 000..4b90f91
  --- /dev/null
  +++ b/scripts/git-compile-check
  @@ -0,0 +1,202 @@
  +#!/bin/bash
  +#
  +# See $desc, below, for program description
  +#
  +# Copyright (c) 2013 Red Hat, Inc.
  +#
  +# Author(s):
  +#   Jeff Cody jc...@redhat.com
  +#
  +# This program is free software; you can redistribute it and/or modify it 
  under
  +# the terms of the GNU General Public License as published by the Free 
  Software
  +# Foundation; under version 2 of the license
  +#
  +# This program is distributed in the hope that it will be useful, but 
  WITHOUT
  +# ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or 
  FITNESS
  +# FOR A PARTICULAR PURPOSE.  See the GNU General Public License for more
  +# details.
  +#
  +# You should have received a copy of the GNU General Public License along 
  with
  +# this program; if not, see http://www.gnu.org/licenses/gpl-2.0.html.
  +#
  +
  +set -C -u -e
  +set -o pipefail
  +
  +desc=
  +$0 iterates through a git commit range, and performs the
  +following on each commit:
  +  - git checkout
  +  - make clean
  +  - configure
  +  - make
  +
  +It will also optionally perform a git-reset and git-clean between
  +checkouts, if requested via the '-f' option.
  +
  +The script will exit and report on first error on any of the above steps,
  +(except no error checking is performed on 'make clean')
  +
  +NOTE: While executing, the script will checkout out each commit
  +  in the range in the current git tree.  On exit, the HEAD
  +  at the time the script was called is checked out
  +
  +
  +# default range is the last commit
  +def_range=HEAD^!
  +def_config_opt=--target-list=x86_64-softmmu
  +# you may want to have make perform multiple jobs, e.g. -j4
  +# this is ommitted as the default in case the project makefile
  +# is not safe for parallel make processes
  +def_make_opt=
  +def_log=output-$$.log
  +def_logdir=
  +force_clean='n'
  +
  +logfile=$def_log
  +range=`git config compile-check.range || true`
  +config_opt=`git config compile-check.configopt || true`
  +make_opt=`git config compile-check.makeopt || true`
  +logdir=`git config compile-check.logdir || true`
  +
  +if [[ -z $range ]]
  +then
  +range=$def_range
  +git config compile-check.range $range || true
  +fi
  +if [[ -z $config_opt ]]
  +then
  +config_opt=$def_config_opt
  +git config compile-check.configopt $config_opt || true
  +fi
  +if [[ -z $make_opt ]]
  +then
  +make_opt=$def_make_opt
  +git config compile-check.makeopt $make_opt || true
  +fi
  +if [[ -z $logdir ]]
  +then
  +logdir=$def_logdir
  +git config compile-check.logdir $logdir || true
  +fi
  +
  +usage() {
  +echo 
  +echo $0 [OPTIONS]
  +echo $desc
  +echo 
  +echo OPTIONS:
  +echo   -r git range
  +optional; default is '$range'
  +
  +echo   -c configure options
  +optional; default is '$config_opt'
  +
  +echo   -m make options
  +optional; default is '$make_opt'
  +
  +echo   -d log dir
  +optional; default is '$logdir'
  +
  +

Re: [Qemu-devel] [PATCH] script: git script to compile every commit in a range of commits

2013-06-07 Thread Jeff Cody 
On Fri, Jun 07, 2013 at 11:51:36AM -0500, Anthony Liguori wrote:
 Laszlo Ersek ler...@redhat.com writes:
 
  On 06/07/13 16:44, Jeff Cody wrote:
 
  Thanks.  I can either do the above changes for a v2, or as follow on
  patches.
 
  Whichever is easier for you, certainly! I'm fine with the script
  going-in as is.
 
 A suggestion I'll make is to split the script into two parts.

Hi Anthony,

I'm sorry, but I'm a bit confused by your suggestion.  I think I know
what you are asking (see below), but I'm not positive.

 git-bisect run is a terribly useful command and I use a bisect script
 that looks like this:
 
 #!/bin/sh
 
 set -e
 
 pushd ~/build/qemu
 make -j1 || exit 1
 popd
 
 # Add right seed here
 if test $1; then
 $@
 fi
 
 I'm sure we all have bisect scripts like this.
 
 What you're proposing is very similar to bisect but instead of doing a
 binary search, it does a linear search starting from the oldest commit.
 Basically:

I agree that git bisect is useful, but solves a slightly different
problem than what I am looking to solve.

For instance, in my working branches I have a whole stack of commits
that I will rebase and squash into a set of sane patches before
submitting.  To make sure I don't do something silly in that process,
and create a patch X that does not build without patch X+1, I want to
explicitly compile each patch, without skipping over any patches.


 #!/bin/sh
 
 refspec=$1
 shift
 
 git rev-list $refspec | while read commit; do
 git checkout $commit
 $@ || exit $?
 done
 
 And indeed, I have a local script called git-foreach to do exactly
 this.  I suspect a nicer version would make a very good addition to the
 git project.
 
 So to bisect for a make check failure, I do:
 
   git bisect run bisect.sh make check
 
 Or to bisect for a qemu-test failure:
 
   git bisect run bisect.sh qemu-test-regress.sh
 
 With git-foreach, you can do:
 
   git-foreach bisect.sh
 
 To do a simple build test.  Or you can do:
 
   git-foreach git show checkpatch-head.sh

 etc.

Ah!  So if I understand correctly, what you are asking is to split
the script up into two different scripts:

1.) A 'foreach' framework script to run an arbitrary command over a
range of commits, against each commit  (i.e. in the place where I run
'make clean, git checkout, configure, and make [lines 188-191], simply
do the git checkout and execute a passed script / command).

2.) A second script to perform the complication check, intended to be
called by script 1).  We could then add additional scripts to be
called by the 'foreach' framework patch as desired.

Heck, if we wanted to, we could then even create a menu-drive
meta-script to interactively run any number of tests (checkpatch,
compilation, etc..) using that framework.

 
 Regards,
 
 Anthony Liguori


Thanks,

Jeff

Re: [Qemu-devel] [PATCH] script: git script to compile every commit in a range of commits

2013-06-06 Thread Laszlo Ersek 
comments below

On 05/31/13 18:39, Jeff Cody wrote:

 +usage() {
 +echo 
 +echo $0 [OPTIONS]
 +echo $desc
 +echo 
 +echo OPTIONS:
 +echo   -r git range
 +optional; default is '$range'
 +
 +echo   -c configure options
 +optional; default is '$config_opt'
 +
 +echo   -m make options
 +optional; default is '$make_opt'
 +
 +echo   -d log dir
 +optional; default is '$logdir'
 +
 +echo   -l log filename
 +optional; default is output-PROCID, where PROCID is the bash 
 process id
 +note: you may specify a full path for the log filename here, and 
 exclude the
 +-d option.
 +
 +echo   -f force a git reset and clean
 +this will cause a 'git reset --hard; git clean -fdx' to be run 
 between checkouts.
 +!! WARNING: This may cause data loss in your git tree.
 +READ THE git-clean and git-reset man pages and make
 +sure you understand the implications of
 +'git clean -fdx' and 'git reset --hard' before using 
 !!
 +
 +echo   -h help
 +}

Sorry for not noticing this before: is there any reason for the trailing
spaces on most lines in the usage text?

Plus I suggest lower-casing the THE in READ THE.

 +
 +while getopts :r:c:m:l:d:hf opt
 +do
 +case $opt in
 +r) range=$OPTARG
 +;;
 +c) config_opt=$OPTARG
 +;;
 +m) make_opt=$OPTARG
 +;;
 +d) logdir=$OPTARG
 +;;
 +l) logfile=$OPTARG
 +;;
 +f) force_clean='y'
 +;;
 +h) usage
 +   exit
 +;;
 +\?) echo Unknown option: -$OPTARG 2
 +usage
 +exit 1
 +;;
 +esac
 +done
 +
 +# append a '/' to logdir if $logdir was specified without one
 +[[ -n $logdir ]]  [[ ${logdir:${#logdir}-1} != / ]]  
 logdir=${logdir}/
 +
 +logfile=${logdir}${logfile}
 +
 +head=`git rev-parse HEAD`
 +total=`git rev-list $range |wc -l`
 +
 +echo log output: $logfile
 +
 +rm -f $logfile

rm -f -- $logfile is safer, but I doubt anyone would pass a pathname
starting with -...

 +date  $logfile
 +echo git compile check for $range.  $logfile
 +echo * configure options='$config_opt'  $logfile
 +echo * make options='$make_opt'  $logfile
 +echo Performing a test compile on $total patches | tee -a $logfile
 +echo -  
 $logfile
 +echo  | tee -a $logfile
 +
 +clean_repo() {
 +if [[ $force_clean == 'y' ]]
 +then
 +git reset --hard  $logfile 21 || true
 +git clean -fdx -e $logfile  $logfile 21 || true
 +fi
 +}

Does -e mean except? It's not supported on RHEL-6.

 +
 +# we want to cleanup and return the git tree back to the previous head
 +trap cleanup EXIT
 +
 +cleanup() {
 +echo 
 +echo -n Cleaning up...
 +clean_repo
 +git checkout $head  /dev/null 21
 +echo done.
 +}
 +
 +cnt=1
 +# don't pipe the git job into read, to avoid subshells
 +while read hash
 +do
 +txt=`git log --pretty=tformat:%h: %s $hash^!`
 +echo ${cnt}/${total}: compiling: $txt | tee -a $logfile
 +let cnt=$cnt+1;
 +echo   $logfile
 +clean_repo
 +make clean  /dev/null 21 || true
 +git checkout $hash   $logfile 21  \
 +./configure $config_opt  $logfile 21  \
 +make $make_opt   $logfile 21 ||
 +(
 +echo  | tee -a $logfile
 +echo ERROR: commit $hash failed to build! | tee -a $logfile
 +git show --stat $hash | tee -a $logfile
 +exit 1
 +)
 +done  (git log $range --pretty=tformat:%H --reverse)
 +
 +echo 
 +All patches in $range compiled successfully! | tee -a $logfile
 +exit 0
 

I think my remarks are not important, the script should work in any
practical environment. We should get this merged and small fixes can be
posted incrementally if needed.

Reviewed-by: Laszlo Ersek ler...@redhat.com

[Qemu-devel] [PATCH] script: git script to compile every commit in a range of commits

2013-05-31 Thread Jeff Cody 
This is a git script that will iterate through every commit in a
specified range, and perform a configure and make.  The intention of
this script is not to act as a check of code correctness, but to see if
any commit breaks compilation of the tree.

The idea is that prior to submitting a patch or patch series, the
submitter should verify that no patch in the series breaks the build,
thereby breaking git-bisect.  This script may also be useful for
reviewers/maintainers dealing with multi-patch series.

The range passed in to the script is in the form of a standard git range;
the default is HEAD^!  (i.e. just the latest commit).

Options may be passed in for configure, as well as make.  The log output
filename and directory may also be optionally specified.  All options
are stored via git-config.

If everything compiled without error, then the exit code from the
program will be 0.  Upon error, the stats for the offending
commit will be shown.

Signed-off-by: Jeff Cody jc...@redhat.com
---
 scripts/git-compile-check | 202 ++
 1 file changed, 202 insertions(+)
 create mode 100755 scripts/git-compile-check

diff --git a/scripts/git-compile-check b/scripts/git-compile-check
new file mode 100755
index 000..4b90f91
--- /dev/null
+++ b/scripts/git-compile-check
@@ -0,0 +1,202 @@
+#!/bin/bash
+#
+# See $desc, below, for program description
+#
+# Copyright (c) 2013 Red Hat, Inc.
+#
+# Author(s):
+#   Jeff Cody jc...@redhat.com
+#
+# This program is free software; you can redistribute it and/or modify it under
+# the terms of the GNU General Public License as published by the Free Software
+# Foundation; under version 2 of the license
+#
+# This program is distributed in the hope that it will be useful, but WITHOUT
+# ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS
+# FOR A PARTICULAR PURPOSE.  See the GNU General Public License for more
+# details.
+#
+# You should have received a copy of the GNU General Public License along with
+# this program; if not, see http://www.gnu.org/licenses/gpl-2.0.html.
+#
+
+set -C -u -e
+set -o pipefail
+
+desc=
+$0 iterates through a git commit range, and performs the
+following on each commit:
+  - git checkout
+  - make clean
+  - configure
+  - make
+
+It will also optionally perform a git-reset and git-clean between
+checkouts, if requested via the '-f' option.
+
+The script will exit and report on first error on any of the above steps,
+(except no error checking is performed on 'make clean')
+
+NOTE: While executing, the script will checkout out each commit
+  in the range in the current git tree.  On exit, the HEAD
+  at the time the script was called is checked out
+
+
+# default range is the last commit
+def_range=HEAD^!
+def_config_opt=--target-list=x86_64-softmmu
+# you may want to have make perform multiple jobs, e.g. -j4
+# this is ommitted as the default in case the project makefile
+# is not safe for parallel make processes
+def_make_opt=
+def_log=output-$$.log
+def_logdir=
+force_clean='n'
+
+logfile=$def_log
+range=`git config compile-check.range || true`
+config_opt=`git config compile-check.configopt || true`
+make_opt=`git config compile-check.makeopt || true`
+logdir=`git config compile-check.logdir || true`
+
+if [[ -z $range ]]
+then
+range=$def_range
+git config compile-check.range $range || true
+fi
+if [[ -z $config_opt ]]
+then
+config_opt=$def_config_opt
+git config compile-check.configopt $config_opt || true
+fi
+if [[ -z $make_opt ]]
+then
+make_opt=$def_make_opt
+git config compile-check.makeopt $make_opt || true
+fi
+if [[ -z $logdir ]]
+then
+logdir=$def_logdir
+git config compile-check.logdir $logdir || true
+fi
+
+usage() {
+echo 
+echo $0 [OPTIONS]
+echo $desc
+echo 
+echo OPTIONS:
+echo   -r git range
+optional; default is '$range'
+
+echo   -c configure options
+optional; default is '$config_opt'
+
+echo   -m make options
+optional; default is '$make_opt'
+
+echo   -d log dir
+optional; default is '$logdir'
+
+echo   -l log filename
+optional; default is output-PROCID, where PROCID is the bash 
process id
+note: you may specify a full path for the log filename here, and 
exclude the
+-d option.
+
+echo   -f force a git reset and clean
+this will cause a 'git reset --hard; git clean -fdx' to be run 
between checkouts.
+!! WARNING: This may cause data loss in your git tree.
+READ THE git-clean and git-reset man pages and make
+sure you understand the implications of
+'git clean -fdx' and 'git reset --hard' before using !!
+
+echo   -h help
+}
+
+while getopts