Re: PATCHES: Brain surgery on the build system, first stage
Le samedi 04 août 2012 à 17:40 +0200, David Kastrup a écrit : * is there a need for vc-clean at all? People can just git clone the repo when they want to have a clean tree. git clean -f -X To remove direoctories, this should be git clean -f -d -X -- John ___ lilypond-devel mailing list lilypond-devel@gnu.org https://lists.gnu.org/mailman/listinfo/lilypond-devel
Re: PATCHES: Brain surgery on the build system, first stage
Il giorno sab, 04/08/2012 alle 16.44 +0100, Phil Holmes ha scritto: FWIW it also failed on the lines $LILYPOND_GIT/configure for reasons I don't understand, but must be something to do with bash. Replacing the variable with the actual path worked. Maybe it's because there's a tilde '~' in LILYPOND_GIT that is not expanded, when I submit next iteration of the patch please set LILYPOND_GIT to a value with no tilde. Best, John ___ lilypond-devel mailing list lilypond-devel@gnu.org https://lists.gnu.org/mailman/listinfo/lilypond-devel
Re: PATCHES: Brain surgery on the build system, first stage
On Tue, Aug 07, 2012 at 06:57:17PM +0200, John Mandereau wrote: Il giorno sab, 04/08/2012 alle 16.44 +0100, Phil Holmes ha scritto: FWIW it also failed on the lines $LILYPOND_GIT/configure for reasons I don't understand, but must be something to do with bash. Replacing the variable with the actual path worked. Maybe it's because there's a tilde '~' in LILYPOND_GIT that is not expanded, when I submit next iteration of the patch please set LILYPOND_GIT to a value with no tilde. pydoc os.path.expanduser - Graham ___ lilypond-devel mailing list lilypond-devel@gnu.org https://lists.gnu.org/mailman/listinfo/lilypond-devel
Re: PATCHES: Brain surgery on the build system, first stage [new version]
Il giorno sab, 04/08/2012 alle 00.39 +0200, John Mandereau ha scritto: Hi guys, I tried to submit commits in dev/jmandereau not in master to Rietveld (git cl upload origin/master), but git-cl said just after I edited the issue description No output from ['git', 'show', 'aclocal.m4'] I tried again with added changes, among which changes suggested by Han-Wen. As suggested in the CG, I added mimetypes.add_type(text/x-script.shell, .m4) in git-cl, but it didn't help, I got the same error message. I kept that change in git-cl and created a dummy branch with the 4 commits to be reviewed squashed in one, then git-cl upload worked: http://codereview.appspot.com/6446096 http://code.google.com/p/lilypond/issues/detail?id=2719 That said, I still think it might be good to view the diff on gitweb or locally with -M, as side-by-side views on Rietveld are a bit strange: when files are moved with some changes, Rietveld shows them but without showing the original path, and files moved without modification show the diff in both panes. John PS: finally, I'm online 24 hours more than planned, because of a false alert of water flow in my flat, reported through a phone call while I was exactly about to leave my flat :-P ___ lilypond-devel mailing list lilypond-devel@gnu.org https://lists.gnu.org/mailman/listinfo/lilypond-devel
Re: PATCHES: Brain surgery on the build system, first stage
Le samedi 04 août 2012 à 16:44 +0100, Phil Holmes a écrit :
The step after I checkout your branch failed with this as the last few
lines:
touch ./out/share/lilypond/current/lilypond-force
make --no-builtin-rules out=www clean
make[1]: Entering directory `/media/IntelSSD/lilypond/lilypond-git'
make[1]: *** No rule to make target `clean'. Stop.
make[1]: Leaving directory `/media/IntelSSD/lilypond/lilypond-git'
make: *** [doc-clean] Error 2
I guess this doc-clean is run as part of vc-clean, isn't it? Anyway as
Han-Wen and David have suggested, my next version of changes will have
no vc-clean, but an addition to CG about suggesting git clean -f -X.
FWIW it also failed on the lines
$LILYPOND_GIT/configure
for reasons I don't understand, but must be something to do with bash.
Replacing the variable with the actual path worked.
Duh, this is one of the little pieces of the script I wrote without own
testing (I came up with parts of the script I sent on my own computer
after many testing iterations, but not many enough to test the full
script). I guess braces are necessary:
${LILYPOND_GIT}/configure
Best,
John
___
lilypond-devel mailing list
lilypond-devel@gnu.org
https://lists.gnu.org/mailman/listinfo/lilypond-devel
Re: PATCHES: Brain surgery on the build system, first stage
Hi Han-Wen,
Thanks for these comments, I'll submit another patch as soon as I get
back to my faster laptop to retest changes extensively.
Le samedi 04 août 2012 à 12:00 -0300, Han-Wen Nienhuys a écrit :
* is there a need for vc-clean at all? People can just git clone the
repo when they want to have a clean tree.
Right, so cvs-clean can die together with maintainerclean.
* you may want to reindent this properly and use for sequential execution
case $(OUT_DIST_FILES)x in x) ;; *) \
mkdir -p $(distdir)/$(localdir)/$(outdir); \
$(LN) $(OUT_DIST_FILES) $(distdir)/$(localdir)/$(outdir);;
This is old code, but right '' is better (';' means sequential
execution too, doesn't it? I guess you recommend '' to stop the
sequence of commands at first failure).
+.gitfilelist: $(top-src-dir)/.git/index
+ cd $(top-src-dir) git ls-files $(top-build-dir)/$@
* I would run git-status before dist (to enforce that the tree is
clean) and take the file list exclusively from committed, so we can
link the tarball and the git versions more strongly.
Agreed, then we could even append
GIT_COMMIT = committish
to VERSION (or a dedicated file).
* If you want to go the extra mile, you can try to make the .tar.gz
deterministic for a given git commit; you'd have to eliminate all
timestamps (set them to the ts from the latest git commit for
example).
As for the timestamps, this should be easy: such a ts can be obtained
with something like
git log --format=format:%cD |head -1
However, the contents of some files generated with Bison and makeinfo
might depend on the version of these programs, so I'm not sure we can
have a fully deterministic tarball; as tarballs we generate are made
with GUB, this is practically not a big deal, though.
Best,
John
___
lilypond-devel mailing list
lilypond-devel@gnu.org
https://lists.gnu.org/mailman/listinfo/lilypond-devel
Re: PATCHES: Brain surgery on the build system, first stage
- Original Message - From: John Mandereau john.mander...@gmail.com To: Lily devel lilypond-devel@gnu.org Sent: Friday, August 03, 2012 11:39 PM Subject: PATCHES: Brain surgery on the build system, first stage [big snip] Hi John - is this patch designed to fix problems like: http://lists.gnu.org/archive/html/lilypond-devel/2011-11/msg00360.html I'd be happy to test once I know what it does a bit better. -- Phil Holmes ___ lilypond-devel mailing list lilypond-devel@gnu.org https://lists.gnu.org/mailman/listinfo/lilypond-devel
Re: PATCHES: Brain surgery on the build system, first stage
Hi Phil, Le samedi 04 août 2012 à 14:02 +0100, Phil Holmes a écrit : Hi John - is this patch designed to fix problems like: http://lists.gnu.org/archive/html/lilypond-devel/2011-11/msg00360.html Yes, exactly; the commit that adresses this issue is among the four top commits in dev/jmandereau commit 8a0905f1409f3d01bf902312cb53cb8f1127e760 Make distributed tarball from Git file list You can also test this commit separately (or together with the parent commit that changes mass-link.py) with the test procedure I gave in my earlier message, except that you won't have make vc-clean available, but it can be replaced by make distclean rm -f configure aclocal.m4 I'd be happy to test once I know what it does a bit better. Basically, that commit changes how dist target finds files to put in the tarball, and removes all GNUmakefiles that are made unnecesszary by this change (i.e. GNUmakefiles that were present only for distributing source files in the tarball). Best, John ___ lilypond-devel mailing list lilypond-devel@gnu.org https://lists.gnu.org/mailman/listinfo/lilypond-devel
Re: PATCHES: Brain surgery on the build system, first stage
Hi John, thanks for working on this; it has been long overdue. Small random comments: * is there a need for vc-clean at all? People can just git clone the repo when they want to have a clean tree. * you may want to reindent this properly and use for sequential execution case $(OUT_DIST_FILES)x in x) ;; *) \ mkdir -p $(distdir)/$(localdir)/$(outdir); \ $(LN) $(OUT_DIST_FILES) $(distdir)/$(localdir)/$(outdir);; +.gitfilelist: $(top-src-dir)/.git/index + cd $(top-src-dir) git ls-files $(top-build-dir)/$@ * I would run git-status before dist (to enforce that the tree is clean) and take the file list exclusively from committed, so we can link the tarball and the git versions more strongly. * If you want to go the extra mile, you can try to make the .tar.gz deterministic for a given git commit; you'd have to eliminate all timestamps (set them to the ts from the latest git commit for example). -- Han-Wen Nienhuys - han...@xs4all.nl - http://www.xs4all.nl/~hanwen ___ lilypond-devel mailing list lilypond-devel@gnu.org https://lists.gnu.org/mailman/listinfo/lilypond-devel
Re: PATCHES: Brain surgery on the build system, first stage
- Original Message - From: John Mandereau john.mander...@gmail.com To: Phil Holmes m...@philholmes.net Cc: Lily devel lilypond-devel@gnu.org Sent: Saturday, August 04, 2012 2:55 PM Subject: Re: PATCHES: Brain surgery on the build system, first stage Hi Phil, Le samedi 04 août 2012 à 14:02 +0100, Phil Holmes a écrit : Hi John - is this patch designed to fix problems like: http://lists.gnu.org/archive/html/lilypond-devel/2011-11/msg00360.html Yes, exactly; the commit that adresses this issue is among the four top commits in dev/jmandereau commit 8a0905f1409f3d01bf902312cb53cb8f1127e760 Make distributed tarball from Git file list You can also test this commit separately (or together with the parent commit that changes mass-link.py) with the test procedure I gave in my earlier message, except that you won't have make vc-clean available, but it can be replaced by make distclean rm -f configure aclocal.m4 I'd be happy to test once I know what it does a bit better. Basically, that commit changes how dist target finds files to put in the tarball, and removes all GNUmakefiles that are made unnecesszary by this change (i.e. GNUmakefiles that were present only for distributing source files in the tarball). Best, John Testing is in progress. Olympic cycling may interrupt some of it... -- Phil Holmes ___ lilypond-devel mailing list lilypond-devel@gnu.org https://lists.gnu.org/mailman/listinfo/lilypond-devel
Re: PATCHES: Brain surgery on the build system, first stage
Han-Wen Nienhuys hanw...@gmail.com writes: Hi John, thanks for working on this; it has been long overdue. Small random comments: * is there a need for vc-clean at all? People can just git clone the repo when they want to have a clean tree. git clean -f -X -- David Kastrup ___ lilypond-devel mailing list lilypond-devel@gnu.org https://lists.gnu.org/mailman/listinfo/lilypond-devel
Re: PATCHES: Brain surgery on the build system, first stage
- Original Message - From: John Mandereau john.mander...@gmail.com To: Phil Holmes m...@philholmes.net Cc: Lily devel lilypond-devel@gnu.org Sent: Saturday, August 04, 2012 2:55 PM Subject: Re: PATCHES: Brain surgery on the build system, first stage Hi Phil, Le samedi 04 août 2012 à 14:02 +0100, Phil Holmes a écrit : Hi John - is this patch designed to fix problems like: http://lists.gnu.org/archive/html/lilypond-devel/2011-11/msg00360.html Yes, exactly; the commit that adresses this issue is among the four top commits in dev/jmandereau commit 8a0905f1409f3d01bf902312cb53cb8f1127e760 Make distributed tarball from Git file list You can also test this commit separately (or together with the parent commit that changes mass-link.py) with the test procedure I gave in my earlier message, except that you won't have make vc-clean available, but it can be replaced by make distclean rm -f configure aclocal.m4 I'd be happy to test once I know what it does a bit better. Basically, that commit changes how dist target finds files to put in the tarball, and removes all GNUmakefiles that are made unnecesszary by this change (i.e. GNUmakefiles that were present only for distributing source files in the tarball). Best, John The step after I checkout your branch failed with this as the last few lines: touch ./out/share/lilypond/current/lilypond-force make --no-builtin-rules out=www clean make[1]: Entering directory `/media/IntelSSD/lilypond/lilypond-git' make[1]: *** No rule to make target `clean'. Stop. make[1]: Leaving directory `/media/IntelSSD/lilypond/lilypond-git' make: *** [doc-clean] Error 2 FWIW it also failed on the lines $LILYPOND_GIT/configure for reasons I don't understand, but must be something to do with bash. Replacing the variable with the actual path worked. Let me know if I can give you further information to help you find out what's happening. -- Phil Holmes ___ lilypond-devel mailing list lilypond-devel@gnu.org https://lists.gnu.org/mailman/listinfo/lilypond-devel
PATCHES: Brain surgery on the build system, first stage
Hi guys,
I tried to submit commits in dev/jmandereau not in master to Rietveld
(git cl upload origin/master), but git-cl said just after I edited the
issue description
No output from ['git', 'show', 'aclocal.m4']
and exited. So below goes the issue description, that includes
instructions to test the changes (even if I could upload the issue to
Rietveld, Patchy would not suffice anyway). I kindly ask a review of
the four commits on top of master that are in dev/jmandereau branch on
Savannah git repo.
Best,
John
==
Brain surgery on the build system, first stage
Sorry to do this just before 2.16 branching out, but if you remember
nightmares about non-distributed files and the like because of the
desire to include last-minute changes to translations and whatnot, IMO
the part of this patch regarding make dist is really worth the
trouble. If you like, I can also include a merge of stepmake/stepmake
and make/, but I thought this would really clutter the review of other
changes, and this merge is trivial if you compare with the patch of
this issue, so that merge will be submitted for review after this
review issue settled down.
Commits are viewable separately on dev/jmandereau:
http://git.savannah.gnu.org/gitweb/?p=lilypond.git;a=shortlog;h=refs/heads/dev/jmandereau
I recommend inspecting changes locally with git with git log -p -M
to sort out real changes in moved files.
Patchy test is welcome, but is not enough. People who have reasonable
horsepower (or many hours of CPU time to allocate for this, typically
12 hours on a Pentium 4 with 2GB RAM) and about five gigabytes of free
space might want to run the following test.
I checked which files disappeared or appeared with these changes in
both tarball and installation of binary and docs, with the following
scripts (they are very dirty, but I'm not sure it's worth adding
installation and distribution checks to Patchy), assuming you've set
LILYPOND_GIT and LILYPOND_BUILD_DIR as in
http://lilypond.org/doc/v2.15/Documentation/contributor/environment-variables
and you start from master branch checked out in $LILYPOND_GIT:
export VERSION=2.15.43
export MAKE_FLAGS=-j3 CPU_COUNT=3
./autogen.sh --noconfigure \
rm -rf $LILYPOND_BUILD_DIR \
mkdir -p $LILYPOND_BUILD_DIR \
cd $LILYPOND_BUILD_DIR \
$LILYPOND_GIT/configure \
mkdir -p $HOME/lily-base \
make dist $HOME/lily-base/make-dist.log \
mv out/lilypond-${VERSION}.tar.gz $HOME/lily-base \
pushd $HOME/lily-base tar -xzf lilypond-${VERSION}.tar.gz popd \
make distclean \
$LILYPOND_GIT/configure --prefix=$HOME/lily-base \
make ${MAKE_FLAGS} $HOME/lily-base/make.log \
make ${MAKE_FLAGS} test $HOME/lily-base/make-test.log \
make ${MAKE_FLAGS} doc $HOME/lily-base/make-doc.log \
make install $HOME/lily-base/make-install.log \
make install-doc $HOME/lily-base/make-install-doc.log
Cd to $LILYPOND_GIT, apply patch (or checkout dev/jmandereau), then do
./autogen.sh \
make vc-clean \
./autogen.sh --noconfigure \
rm -rf $LILYPOND_BUILD_DIR \
mkdir -p $LILYPOND_BUILD_DIR \
cd $LILYPOND_BUILD_DIR \
$LILYPOND_GIT/configure \
mkdir -p $HOME/lily-new \
make dist $HOME/lily-new/make-dist.log \
mv out/lilypond-${VERSION}.tar.gz $HOME/lily-new \
pushd $HOME/lily-new tar -xzf lilypond-${VERSION}.tar.gz popd \
make distclean \
$LILYPOND_GIT/configure --prefix=$HOME/lily-new \
make ${MAKE_FLAGS} $HOME/lily-new/make.log \
make ${MAKE_FLAGS} test $HOME/lily-new/make-test.log \
make ${MAKE_FLAGS} doc $HOME/lily-new/make-doc.log \
make install $HOME/lily-new/make-install.log \
make install-doc $HOME/lily-new/make-install-doc.log
Cd to $HOME, check the output of
LANG= diff -qr lily-base lily-new |grep -i only
then test in-tree build (first line is optional, it's merely
here to free some disk space):
rm -rf $LILYPOND_BUILD_DIR \
cd $LILYPOND_GIT \
./configure \
mkdir -p $HOME/lily-new-intree \
make dist $HOME/lily-new-intree/make-dist.log \
mv out/lilypond-${VERSION}.tar.gz $HOME/lily-new-intree \
pushd $HOME/lily-new-intree tar -xzf lilypond-${VERSION}.tar.gz popd \
make distclean \
./configure --prefix=$HOME/lily-new-intree \
make ${MAKE_FLAGS} $HOME/lily-new-intree/make.log \
make ${MAKE_FLAGS} test $HOME/lily-new-intree/make-test.log \
make ${MAKE_FLAGS} doc $HOME/lily-new-intree/make-doc.log \
make install $HOME/lily-new-intree/make-install.log \
make install-doc $HOME/lily-new-intree/make-install-doc.log
Again, cd to $HOME and see the output of
LANG= diff -qr lily-base lily-new-intree |grep -i only
You should in either case (lily-new, lily-new-intree) see differences
that match either deleted or moved files in the tarball, or
lilypond-book hashing.
commit 61d98da98ea74c5fdb648ea69a400feb8959b37e
Improve make *clean targets: rename cvs- to vc-, adjust distclean
* distclean was usable from any directory in theory, make it toplevel-only,
* vc-clean (was cvs-clean): trigger doc-clean then distclean in addition,
* remove
