Re: [PATCH v4 1/4] Makefile: generate Perl header from template file
On Wed, Dec 06 2017, Ævar Arnfjörð Bjarmason jotted: > On Wed, Dec 6, 2017 at 7:56 PM, Daniel Jacqueswrote: >> On Wed, Dec 6, 2017 at 1:47 PM, Junio C Hamano wrote: >>> >>> Johannes Sixt writes: >>> >>> > The updated series works for me now. Nevertheless, I suggest to squash >>> > in the following change to protect against IFS and globbing characters in >>> > $INSTLIBDIR. >>> >>> Yeah, that is very sensible. >>> >>> > diff --git a/Makefile b/Makefile >>> > index 7ac4458f11..08c78a1a63 100644 >>> > --- a/Makefile >>> > +++ b/Makefile >>> > @@ -2072,7 +2072,7 @@ GIT-PERL-HEADER: $(PERL_HEADER_TEMPLATE) >>> > GIT-PERL-DEFINES perl/perl.mak Makefile >>> > INSTLIBDIR_EXTRA='$(PERLLIB_EXTRA_SQ)' && \ >>> > INSTLIBDIR="$$INSTLIBDIR$${INSTLIBDIR_EXTRA:+:$$INSTLIBDIR_EXTRA}" >>> > && \ >>> > sed -e 's=@@PATHSEP@@=$(pathsep)=g' \ >>> > - -e 's=@@INSTLIBDIR@@='$$INSTLIBDIR'=g' \ >>> > + -e 's=@@INSTLIBDIR@@='"$$INSTLIBDIR"'=g' \ >>> > -e 's=@@GITEXECDIR@@=$(gitexecdir_relative_SQ)=g' \ >>> > -e 's=@@PERLLIBDIR@@=$(perllibdir_relative_SQ)=g' \ >>> > $< >$@+ && \ >> >> Sounds good; I'll apply that to my working patch and include it in my >> next ("v5") submission, which is currently blocked pending avarab@'s Perl >> Makefile changes: >> https://public-inbox.org/git/20171129195430.10069-1-ava...@gmail.com/T/#t > > Thanks, FWIW I'll send another version of that at the end of the week > or so, I'm waiting to see if there's any more comments on it to reduce > list churn. Sorry, I got this conflated with my sha1collisiondetection series, I have nothing new to send out. It seems everyone's happy with the version of my v2 (with Junio's on top) from this series, so it's just a matter of waiting.
Re: [PATCH v4 1/4] Makefile: generate Perl header from template file
On Wed, Dec 6, 2017 at 7:56 PM, Daniel Jacqueswrote: > On Wed, Dec 6, 2017 at 1:47 PM, Junio C Hamano wrote: >> >> Johannes Sixt writes: >> >> > The updated series works for me now. Nevertheless, I suggest to squash >> > in the following change to protect against IFS and globbing characters in >> > $INSTLIBDIR. >> >> Yeah, that is very sensible. >> >> > diff --git a/Makefile b/Makefile >> > index 7ac4458f11..08c78a1a63 100644 >> > --- a/Makefile >> > +++ b/Makefile >> > @@ -2072,7 +2072,7 @@ GIT-PERL-HEADER: $(PERL_HEADER_TEMPLATE) >> > GIT-PERL-DEFINES perl/perl.mak Makefile >> > INSTLIBDIR_EXTRA='$(PERLLIB_EXTRA_SQ)' && \ >> > INSTLIBDIR="$$INSTLIBDIR$${INSTLIBDIR_EXTRA:+:$$INSTLIBDIR_EXTRA}" >> > && \ >> > sed -e 's=@@PATHSEP@@=$(pathsep)=g' \ >> > - -e 's=@@INSTLIBDIR@@='$$INSTLIBDIR'=g' \ >> > + -e 's=@@INSTLIBDIR@@='"$$INSTLIBDIR"'=g' \ >> > -e 's=@@GITEXECDIR@@=$(gitexecdir_relative_SQ)=g' \ >> > -e 's=@@PERLLIBDIR@@=$(perllibdir_relative_SQ)=g' \ >> > $< >$@+ && \ > > Sounds good; I'll apply that to my working patch and include it in my > next ("v5") submission, which is currently blocked pending avarab@'s Perl > Makefile changes: > https://public-inbox.org/git/20171129195430.10069-1-ava...@gmail.com/T/#t Thanks, FWIW I'll send another version of that at the end of the week or so, I'm waiting to see if there's any more comments on it to reduce list churn.
Re: [PATCH v4 1/4] Makefile: generate Perl header from template file
On Wed, Dec 6, 2017 at 1:47 PM, Junio C Hamanowrote: > > Johannes Sixt writes: > > > The updated series works for me now. Nevertheless, I suggest to squash > > in the following change to protect against IFS and globbing characters in > > $INSTLIBDIR. > > Yeah, that is very sensible. > > > diff --git a/Makefile b/Makefile > > index 7ac4458f11..08c78a1a63 100644 > > --- a/Makefile > > +++ b/Makefile > > @@ -2072,7 +2072,7 @@ GIT-PERL-HEADER: $(PERL_HEADER_TEMPLATE) > > GIT-PERL-DEFINES perl/perl.mak Makefile > > INSTLIBDIR_EXTRA='$(PERLLIB_EXTRA_SQ)' && \ > > INSTLIBDIR="$$INSTLIBDIR$${INSTLIBDIR_EXTRA:+:$$INSTLIBDIR_EXTRA}" && > > \ > > sed -e 's=@@PATHSEP@@=$(pathsep)=g' \ > > - -e 's=@@INSTLIBDIR@@='$$INSTLIBDIR'=g' \ > > + -e 's=@@INSTLIBDIR@@='"$$INSTLIBDIR"'=g' \ > > -e 's=@@GITEXECDIR@@=$(gitexecdir_relative_SQ)=g' \ > > -e 's=@@PERLLIBDIR@@=$(perllibdir_relative_SQ)=g' \ > > $< >$@+ && \ Sounds good; I'll apply that to my working patch and include it in my next ("v5") submission, which is currently blocked pending avarab@'s Perl Makefile changes: https://public-inbox.org/git/20171129195430.10069-1-ava...@gmail.com/T/#t
Re: [PATCH v4 1/4] Makefile: generate Perl header from template file
Johannes Sixtwrites: > The updated series works for me now. Nevertheless, I suggest to squash > in the following change to protect against IFS and globbing characters in > $INSTLIBDIR. Yeah, that is very sensible. > diff --git a/Makefile b/Makefile > index 7ac4458f11..08c78a1a63 100644 > --- a/Makefile > +++ b/Makefile > @@ -2072,7 +2072,7 @@ GIT-PERL-HEADER: $(PERL_HEADER_TEMPLATE) > GIT-PERL-DEFINES perl/perl.mak Makefile > INSTLIBDIR_EXTRA='$(PERLLIB_EXTRA_SQ)' && \ > INSTLIBDIR="$$INSTLIBDIR$${INSTLIBDIR_EXTRA:+:$$INSTLIBDIR_EXTRA}" && \ > sed -e 's=@@PATHSEP@@=$(pathsep)=g' \ > - -e 's=@@INSTLIBDIR@@='$$INSTLIBDIR'=g' \ > + -e 's=@@INSTLIBDIR@@='"$$INSTLIBDIR"'=g' \ > -e 's=@@GITEXECDIR@@=$(gitexecdir_relative_SQ)=g' \ > -e 's=@@PERLLIBDIR@@=$(perllibdir_relative_SQ)=g' \ > $< >$@+ && \
Re: [PATCH v4 1/4] Makefile: generate Perl header from template file
Am 05.12.2017 um 22:35 schrieb Junio C Hamano: > Dan Jacqueswrites: > >> Thanks for checking! The patch that you quoted above looks like it's from >> this "v4" thread; however, the patch that you are diffing against in your >> latest reply seems like it is from an earlier version. >> >> I believe that the $(pathsep) changes in your proposed patch are already >> present in v4,... > > You're of course right. The patches I had in my tree are outdated. > > Will replace, even though I won't be merging them to 'pu' while we > wait for Ævar's perl build procedure update to stabilize. The updated series works for me now. Nevertheless, I suggest to squash in the following change to protect against IFS and globbing characters in $INSTLIBDIR. diff --git a/Makefile b/Makefile index 7ac4458f11..08c78a1a63 100644 --- a/Makefile +++ b/Makefile @@ -2072,7 +2072,7 @@ GIT-PERL-HEADER: $(PERL_HEADER_TEMPLATE) GIT-PERL-DEFINES perl/perl.mak Makefile INSTLIBDIR_EXTRA='$(PERLLIB_EXTRA_SQ)' && \ INSTLIBDIR="$$INSTLIBDIR$${INSTLIBDIR_EXTRA:+:$$INSTLIBDIR_EXTRA}" && \ sed -e 's=@@PATHSEP@@=$(pathsep)=g' \ - -e 's=@@INSTLIBDIR@@='$$INSTLIBDIR'=g' \ + -e 's=@@INSTLIBDIR@@='"$$INSTLIBDIR"'=g' \ -e 's=@@GITEXECDIR@@=$(gitexecdir_relative_SQ)=g' \ -e 's=@@PERLLIBDIR@@=$(perllibdir_relative_SQ)=g' \ $< >$@+ && \
Re: [PATCH v4 1/4] Makefile: generate Perl header from template file
Dan Jacqueswrites: > Thanks for checking! The patch that you quoted above looks like it's from > this "v4" thread; however, the patch that you are diffing against in your > latest reply seems like it is from an earlier version. > > I believe that the $(pathsep) changes in your proposed patch are already > present in v4,... You're of course right. The patches I had in my tree are outdated. Will replace, even though I won't be merging them to 'pu' while we wait for Ævar's perl build procedure update to stabilize. Thanks.
Re: [PATCH v4 1/4] Makefile: generate Perl header from template file
Johannes Sixt wrote: > I don't know what I tested last week; most likely not the version of the > patch I quoted above. > > Today's version, with the tip at 5d7f59c391ce, is definitely bogus > with its quoting. It needs the patch below, otherwise an unquoted > semicolon may be expanded from $(pathsep). This would terminate the sed > command, of course. Thanks for checking! The patch that you quoted above looks like it's from this "v4" thread; however, the patch that you are diffing against in your latest reply seems like it is from an earlier version. I believe that the $(pathsep) changes in your proposed patch are already present in v4, having been caught by Johannes, so perhaps you were using an older version of the patch as the diffbase? That change only appeared in v4, so any older version would have had the previous incorrect quoting. I think the reason that is necessary to remove the single quotes is not because the ";" is terminating the `sed` expression, but because it's terminating the entire shell stanza, causing the shell to execute two commands: 1) {"sed" "-e" "s=@@PATHSEP@@="} 2) {"=g" "-e" "s=@@INSTLIBDIR@@=..." ...} By including the ";" within the single-tick-protected `sed` argument, the shell receives the entire block, "s=@@PATHSEP@@=;=", as a single string. Also see Johannes' explanation here: https://github.com/danjacques/git/pull/1/commits/57dc265478692ea2a395fc61fa122cb73ce58dc3 Could you apply this v4 patch series and confirm that it is working for you? Or, if I am mistaken and your patch is correctly against this "v4" patch series, let me know and I'll try and figure out what I'm missing. Thanks again! -Dan
Re: [PATCH v4 1/4] Makefile: generate Perl header from template file
Johannes Sixtwrites: > Today's version, with the tip at 5d7f59c391ce, is definitely bogus > with its quoting. It needs the patch below, otherwise an unquoted > semicolon may be expanded from $(pathsep). This would terminate the sed > command, of course. Of course ;-) Somehow I was lucky that the topic as been de-queued from 'pu' for the past few days. > diff --git a/Makefile b/Makefile > index 484dc44ade..a658c8169a 100644 > --- a/Makefile > +++ b/Makefile > @@ -2071,10 +2071,10 @@ GIT-PERL-HEADER: $(PERL_HEADER_TEMPLATE) > GIT-PERL-DEFINES perl/perl.mak > INSTLIBDIR=`MAKEFLAGS= $(MAKE) -C perl -s --no-print-directory > instlibdir` && \ > INSTLIBDIR_EXTRA='$(PERLLIB_EXTRA_SQ)' && \ > INSTLIBDIR="$$INSTLIBDIR$${INSTLIBDIR_EXTRA:+:$$INSTLIBDIR_EXTRA}" && \ > - sed -e 's=@@PATHSEP@@='$(pathsep)'=g' \ > - -e 's=@@INSTLIBDIR@@='$$INSTLIBDIR'=g' \ > - -e 's=@@GITEXECDIR@@='$(gitexecdir_relative_SQ)'=g' \ > - -e 's=@@PERLLIBDIR@@='$(perllibdir_relative_SQ)'=g' \ > + sed -e 's=@@PATHSEP@@=$(pathsep)=g' \ > + -e 's=@@INSTLIBDIR@@='"$$INSTLIBDIR"'=g' \ > + -e 's=@@GITEXECDIR@@=$(gitexecdir_relative_SQ)=g' \ > + -e 's=@@PERLLIBDIR@@=$(perllibdir_relative_SQ)=g' \ > $< >$@ > > .PHONY: gitweb
Re: [PATCH v4 1/4] Makefile: generate Perl header from template file
Am 01.12.2017 um 18:25 schrieb Johannes Sixt: > Am 01.12.2017 um 18:13 schrieb Johannes Schindelin: >> Hi Hannes, >> >> On Fri, 1 Dec 2017, Johannes Sixt wrote: >> >>> Am 29.11.2017 um 16:56 schrieb Dan Jacques: @@ -1989,6 +1986,15 @@ GIT-PERL-DEFINES: FORCE echo "$$FLAGS" >$@; \ fi +GIT-PERL-HEADER: $(PERL_HEADER_TEMPLATE) GIT-PERL-DEFINES perl/perl.mak Makefile + $(QUIET_GEN)$(RM) $@ && \ + INSTLIBDIR=`MAKEFLAGS= $(MAKE) -C perl -s --no-print-directory instlibdir` && \ + INSTLIBDIR_EXTRA='$(PERLLIB_EXTRA_SQ)' && \ + INSTLIBDIR="$$INSTLIBDIR$${INSTLIBDIR_EXTRA:+:$$INSTLIBDIR_EXTRA}" && \ + sed -e 's=@@PATHSEP@@=$(pathsep)=g' \ >>> >>> This doesn't work, unfortunately. When $(pathsep) is ';', we get an >>> incomplete >>> sed expression because ';' is also a command separator in the sed >>> language. >> >> Funny, I tried this also with ';' as pathsep, and it worked in the Git >> for >> Windows SDK... > > My ancient sed vs. your modern sed, perhaps? I can check this on Monday. I don't know what I tested last week; most likely not the version of the patch I quoted above. Today's version, with the tip at 5d7f59c391ce, is definitely bogus with its quoting. It needs the patch below, otherwise an unquoted semicolon may be expanded from $(pathsep). This would terminate the sed command, of course. diff --git a/Makefile b/Makefile index 484dc44ade..a658c8169a 100644 --- a/Makefile +++ b/Makefile @@ -2071,10 +2071,10 @@ GIT-PERL-HEADER: $(PERL_HEADER_TEMPLATE) GIT-PERL-DEFINES perl/perl.mak INSTLIBDIR=`MAKEFLAGS= $(MAKE) -C perl -s --no-print-directory instlibdir` && \ INSTLIBDIR_EXTRA='$(PERLLIB_EXTRA_SQ)' && \ INSTLIBDIR="$$INSTLIBDIR$${INSTLIBDIR_EXTRA:+:$$INSTLIBDIR_EXTRA}" && \ - sed -e 's=@@PATHSEP@@='$(pathsep)'=g' \ - -e 's=@@INSTLIBDIR@@='$$INSTLIBDIR'=g' \ - -e 's=@@GITEXECDIR@@='$(gitexecdir_relative_SQ)'=g' \ - -e 's=@@PERLLIBDIR@@='$(perllibdir_relative_SQ)'=g' \ + sed -e 's=@@PATHSEP@@=$(pathsep)=g' \ + -e 's=@@INSTLIBDIR@@='"$$INSTLIBDIR"'=g' \ + -e 's=@@GITEXECDIR@@=$(gitexecdir_relative_SQ)=g' \ + -e 's=@@PERLLIBDIR@@=$(perllibdir_relative_SQ)=g' \ $< >$@ .PHONY: gitweb -- 2.14.2.808.g3bc32f2729
Re: [PATCH v4 1/4] Makefile: generate Perl header from template file
On Sun, Dec 03 2017, Junio C. Hamano jotted: > Johannes Sixtwrites: > >>> + sed -e 's=@@PATHSEP@@=$(pathsep)=g' \ >> >> This doesn't work, unfortunately. When $(pathsep) is ';', we get an >> incomplete sed expression because ';' is also a command separator in >> the sed language. > > It is correct that ';' can be and does get used in place of LF when > writing a script on a single line, but even then, as part of a > string argument to 's' command (and also others), there is no need > to quote ';' or otherwise treat it any specially, as the commands > know what their syntax is (e.g. 's=string=replacement=' after seeing > the first '=' knows that it needs to find one unquoted '=' to find > the end of the first argument, and another to find the end of the > replacement string, and ';' seen during that scanning would not have > any special meaning). > > If your sed is so broken and does not satisfy the above expectation, > t6023 would not work for you, I would gess. > > t/t6023-merge-file.sh:sed -e "s/deerit.\$/deerit;/" -e "s/me;\$/me./" < > new5.txt > new6.txt > t/t6023-merge-file.sh:sed -e "s/deerit.\$/deerit,/" -e "s/me;\$/me,/" < > new5.txt > new7.txt > t/t6023-merge-file.sh:sed -e 's/deerit./&/' -e "s/locavit,/locavit;/"< > new6.txt | tr '%' '\012' > new8.txt Since this whole thing is guarded by "ifndef NO_PERL" Dan could just be using "perl -pe" here instead of fiddling around with the portability edge cases of sed, e.g.: perl -pe 's[foo][bar[g' out
Re: [PATCH v4 1/4] Makefile: generate Perl header from template file
Johannes Sixtwrites: >> +sed -e 's=@@PATHSEP@@=$(pathsep)=g' \ > > This doesn't work, unfortunately. When $(pathsep) is ';', we get an > incomplete sed expression because ';' is also a command separator in > the sed language. It is correct that ';' can be and does get used in place of LF when writing a script on a single line, but even then, as part of a string argument to 's' command (and also others), there is no need to quote ';' or otherwise treat it any specially, as the commands know what their syntax is (e.g. 's=string=replacement=' after seeing the first '=' knows that it needs to find one unquoted '=' to find the end of the first argument, and another to find the end of the replacement string, and ';' seen during that scanning would not have any special meaning). If your sed is so broken and does not satisfy the above expectation, t6023 would not work for you, I would gess. t/t6023-merge-file.sh:sed -e "s/deerit.\$/deerit;/" -e "s/me;\$/me./" < new5.txt > new6.txt t/t6023-merge-file.sh:sed -e "s/deerit.\$/deerit,/" -e "s/me;\$/me,/" < new5.txt > new7.txt t/t6023-merge-file.sh:sed -e 's/deerit./&/' -e "s/locavit,/locavit;/"< new6.txt | tr '%' '\012' > new8.txt
Re: [PATCH v4 1/4] Makefile: generate Perl header from template file
On Dez 01 2017, Dan Jacqueswrote: > I am not a `sed` wizard, but perhaps the tool is ignoring the semicolon > because > it's in the middle of the "s" expression? The shell may similarly be ignoring > it > because it's nested in between single-quotes? As far as POSIX is concerned, semicolons are not special, but can be used to separate commands instead of newlines. Andreas. -- Andreas Schwab, sch...@linux-m68k.org GPG Key fingerprint = 58CA 54C7 6D53 942B 1756 01D3 44D5 214B 8276 4ED5 "And now for something completely different."
Re: [PATCH v4 1/4] Makefile: generate Perl header from template file
On Fri, 1 Dec 2017, Johannes Sixt wrote: >>> This doesn't work, unfortunately. When $(pathsep) is ';', we get an >>> incomplete sed expression because ';' is also a command separator in the >>> sed language. >> >> Funny, I tried this also with ';' as pathsep, and it worked in the Git for >> Windows SDK... > > My ancient sed vs. your modern sed, perhaps? I can check this on Monday. If you wouldn't mind, that would be much appreciated. Did you actually observe this issue, or is it just coming up in review? I am not a `sed` wizard, but perhaps the tool is ignoring the semicolon because it's in the middle of the "s" expression? The shell may similarly be ignoring it because it's nested in between single-quotes? -Dan
Re: [PATCH v4 1/4] Makefile: generate Perl header from template file
Am 01.12.2017 um 18:13 schrieb Johannes Schindelin: Hi Hannes, On Fri, 1 Dec 2017, Johannes Sixt wrote: Am 29.11.2017 um 16:56 schrieb Dan Jacques: @@ -1989,6 +1986,15 @@ GIT-PERL-DEFINES: FORCE echo "$$FLAGS" >$@; \ fi +GIT-PERL-HEADER: $(PERL_HEADER_TEMPLATE) GIT-PERL-DEFINES perl/perl.mak Makefile + $(QUIET_GEN)$(RM) $@ && \ + INSTLIBDIR=`MAKEFLAGS= $(MAKE) -C perl -s --no-print-directory instlibdir` && \ + INSTLIBDIR_EXTRA='$(PERLLIB_EXTRA_SQ)' && \ + INSTLIBDIR="$$INSTLIBDIR$${INSTLIBDIR_EXTRA:+:$$INSTLIBDIR_EXTRA}" && \ + sed -e 's=@@PATHSEP@@=$(pathsep)=g' \ This doesn't work, unfortunately. When $(pathsep) is ';', we get an incomplete sed expression because ';' is also a command separator in the sed language. Funny, I tried this also with ';' as pathsep, and it worked in the Git for Windows SDK... My ancient sed vs. your modern sed, perhaps? I can check this on Monday. -- Hannes
Re: [PATCH v4 1/4] Makefile: generate Perl header from template file
Hi Hannes, On Fri, 1 Dec 2017, Johannes Sixt wrote: > Am 29.11.2017 um 16:56 schrieb Dan Jacques: > > @@ -1989,6 +1986,15 @@ GIT-PERL-DEFINES: FORCE > > echo "$$FLAGS" >$@; \ > >fi > > +GIT-PERL-HEADER: $(PERL_HEADER_TEMPLATE) GIT-PERL-DEFINES perl/perl.mak > > Makefile > > + $(QUIET_GEN)$(RM) $@ && \ > > + INSTLIBDIR=`MAKEFLAGS= $(MAKE) -C perl -s --no-print-directory > > instlibdir` && \ > > + INSTLIBDIR_EXTRA='$(PERLLIB_EXTRA_SQ)' && \ > > + INSTLIBDIR="$$INSTLIBDIR$${INSTLIBDIR_EXTRA:+:$$INSTLIBDIR_EXTRA}" && > > \ > > + sed -e 's=@@PATHSEP@@=$(pathsep)=g' \ > > This doesn't work, unfortunately. When $(pathsep) is ';', we get an incomplete > sed expression because ';' is also a command separator in the sed language. Funny, I tried this also with ';' as pathsep, and it worked in the Git for Windows SDK... Ciao, Dscho
Re: [PATCH v4 1/4] Makefile: generate Perl header from template file
Am 29.11.2017 um 16:56 schrieb Dan Jacques: @@ -1989,6 +1986,15 @@ GIT-PERL-DEFINES: FORCE echo "$$FLAGS" >$@; \ fi +GIT-PERL-HEADER: $(PERL_HEADER_TEMPLATE) GIT-PERL-DEFINES perl/perl.mak Makefile + $(QUIET_GEN)$(RM) $@ && \ + INSTLIBDIR=`MAKEFLAGS= $(MAKE) -C perl -s --no-print-directory instlibdir` && \ + INSTLIBDIR_EXTRA='$(PERLLIB_EXTRA_SQ)' && \ + INSTLIBDIR="$$INSTLIBDIR$${INSTLIBDIR_EXTRA:+:$$INSTLIBDIR_EXTRA}" && \ + sed -e 's=@@PATHSEP@@=$(pathsep)=g' \ This doesn't work, unfortunately. When $(pathsep) is ';', we get an incomplete sed expression because ';' is also a command separator in the sed language. + -e 's=@@INSTLIBDIR@@='$$INSTLIBDIR'=g' \ + $< >$@+ && \ + mv $@+ $@ .PHONY: gitweb gitweb: -- Hannes