Re: [PATCH v3] Makefile: replace perl/Makefile.PL with simple make rules

[Junio C Hamano]

Ævar Arnfjörð Bjarmason  writes:

>> Is there anything I'm supposed to do differently now to make our test
>> suite run? If yes then a clear and short hint in the patch description
>> would me more than approriate.
>
> This is a bug in my patch, I can reproduce it on CO7. Will figure out
> what's going on there...

Thanks.

Re: [PATCH v3] Makefile: replace perl/Makefile.PL with simple make rules

[Ævar Arnfjörð Bjarmason]

On Fri, Dec 15, 2017 at 11:35 AM, Michael J Gruber <g...@grubix.eu> wrote:
> Ævar Arnfjörð Bjarmason venit, vidit, dixit 12.12.2017 23:26:
>>
>> On Tue, Dec 12 2017, Randall S. Becker jotted:
>>
>>> -Original Message-
>>> On December 10, 2017 4:14 PM, Ævar Arnfjörð Bjarmason wrote:
>>> Subject: [PATCH v3] Makefile: replace perl/Makefile.PL with simple make 
>>> rules
>>>
>>>> Replace the perl/Makefile.PL and the fallback perl/Makefile used under 
>>>> NO_PERL_MAKEMAKER=NoThanks with a much simpler implementation heavily 
>>>> inspired by how the i18n infrastructure's build process works[1].
>>>> The reason for having the Makefile.PL in the first place is that it was 
>>>> initially[2] building a perl C binding to interface with libgit, this 
>>>> functionality, that was removed[3] before Git.pm ever made it to the 
>>>> master branch.
>>> 
>>>
>>> I would like to request that the we be careful that the git builds do not 
>>> introduce arbitrary dependencies to CPAN. Some platforms (I can think of 
>>> one off the top, being NonStop) does not provide for arbitrary additions to 
>>> the supplied perl implementation as of yet. The assumption about being able 
>>> to add CPAN modules may apply on some platforms but is not a general 
>>> capability. I am humbly requesting that caution be used when adding 
>>> dependencies. Being non-$DAYJOB responsible for the git port for NonStop, 
>>> this scares me a bit, but I and my group can help validate the available 
>>> modules used for builds.
>>>
>>> Note: we do not yet have CPAN's SCM so can't and don't use perl for access 
>>> to git anyway - much that I've tried to change that.
>>>
>>> Please keep build dependencies to a minimum.
>>>
>>> Thanks for my and my whole team.
>>
>> I think you should be happy with this patch then, and it doesn't add any
>> more CPAN dependency than before, and sets up a framework (as discussed
>> in [1]) where we can use more CPAN modules while not requiring packagers
>> such as yourself to package CPAN modules.
>>
>> However, it doesn't sound believable to me that even on NonStop you
>> can't install any CPAN modules whatsoever.
>>
>> That would also mean that this patch doesn't work for you, because it
>> means that you either don't have anything resembling a hierarchical
>> filesystem on which git can be installed in the first place (in which
>> case it wouldn't work), or perl doesn't have an @INC to search through
>> perl libs on on NonStop. What does:
>>
>> perl -V
>>
>> Return for you on that system?
>>
>> If this patch works, and if at the bottom of `perl -V` you see some
>> directories which you could write a package to drop some static *.pm
>> files, then you can grab a *.tar.gz from CPAN such as the one for
>> Error.pm[2] and arrange for the *.pm files contained within its lib/
>> directory to be dropped into one of those @INC directories.
>>
>> It may be that some aspect of the CPAN toolchain is broken for you, or
>> even ExtUtils::MakeMaker, but you typically don't need that to package
>> non-XS perl modules, certainly not any of the ones we've discussed
>> possibly bundling up in git.git on-list recently. As a (very occasional)
>> contributor to perl.git I'd be interested to know if that's what you
>> mean is broken, and if so see if it could be fixed for you.
>>
>> 1. 

Re: [PATCH v3] Makefile: replace perl/Makefile.PL with simple make rules

[Todd Zullinger]

Hi Michael,

Michael J Gruber wrote:
> This patch (currently in origin/next) makes a ton of tests from our test
> suite fail for me on pretty standard systems (Fedora 27, CentOS 7.4.1708).
> 
> Is there anything I'm supposed to do differently now to make our test
> suite run? If yes then a clear and short hint in the patch description
> would me more than approriate.

Interesting.  I'll have to try building next.  I was going
to wait until the first 2.16.0 rc for a full test.

I did apply this patch on top of 2.15.1 on 12/10 and built
an rpm locally for fedora (only f26 though).  I didn't see
any test failures, which is why I thought I'd wait for
2.16.0-rc0 to test further.

FWIW, the minor spec file changes I made (against fedora's
git package master branch) are here:

https://src.fedoraproject.org/fork/tmz/rpms/git/branch/perl-makefile

The only change in the patch since I tested it is: 

+-modules += Git/Packet

in perl/Makefile.

I don't think any of these changes to the rpm spec file or
the Git/Packet addition in modules look like they'd affect
the test suite, but it's early here and I could be wrong.

I'll try to test a build of next soon to see if I get
similar failures on Fedora/CentOS.

-- 
Todd
~~
After one look at this planet any visitor from outer space would say
"I want to see the manager."
-- William S. Burroughs

Re: [PATCH v3] Makefile: replace perl/Makefile.PL with simple make rules

[Michael J Gruber]

Ævar Arnfjörð Bjarmason venit, vidit, dixit 12.12.2017 23:26:
> 
> On Tue, Dec 12 2017, Randall S. Becker jotted:
> 
>> -Original Message-
>> On December 10, 2017 4:14 PM, Ævar Arnfjörð Bjarmason wrote:
>> Subject: [PATCH v3] Makefile: replace perl/Makefile.PL with simple make rules
>>
>>> Replace the perl/Makefile.PL and the fallback perl/Makefile used under 
>>> NO_PERL_MAKEMAKER=NoThanks with a much simpler implementation heavily 
>>> inspired by how the i18n infrastructure's build process works[1].
>>> The reason for having the Makefile.PL in the first place is that it was 
>>> initially[2] building a perl C binding to interface with libgit, this 
>>> functionality, that was removed[3] before Git.pm ever made it to the master 
>>> branch.
>> 
>>
>> I would like to request that the we be careful that the git builds do not 
>> introduce arbitrary dependencies to CPAN. Some platforms (I can think of one 
>> off the top, being NonStop) does not provide for arbitrary additions to the 
>> supplied perl implementation as of yet. The assumption about being able to 
>> add CPAN modules may apply on some platforms but is not a general 
>> capability. I am humbly requesting that caution be used when adding 
>> dependencies. Being non-$DAYJOB responsible for the git port for NonStop, 
>> this scares me a bit, but I and my group can help validate the available 
>> modules used for builds.
>>
>> Note: we do not yet have CPAN's SCM so can't and don't use perl for access 
>> to git anyway - much that I've tried to change that.
>>
>> Please keep build dependencies to a minimum.
>>
>> Thanks for my and my whole team.
> 
> I think you should be happy with this patch then, and it doesn't add any
> more CPAN dependency than before, and sets up a framework (as discussed
> in [1]) where we can use more CPAN modules while not requiring packagers
> such as yourself to package CPAN modules.
> 
> However, it doesn't sound believable to me that even on NonStop you
> can't install any CPAN modules whatsoever.
> 
> That would also mean that this patch doesn't work for you, because it
> means that you either don't have anything resembling a hierarchical
> filesystem on which git can be installed in the first place (in which
> case it wouldn't work), or perl doesn't have an @INC to search through
> perl libs on on NonStop. What does:
> 
> perl -V
> 
> Return for you on that system?
> 
> If this patch works, and if at the bottom of `perl -V` you see some
> directories which you could write a package to drop some static *.pm
> files, then you can grab a *.tar.gz from CPAN such as the one for
> Error.pm[2] and arrange for the *.pm files contained within its lib/
> directory to be dropped into one of those @INC directories.
> 
> It may be that some aspect of the CPAN toolchain is broken for you, or
> even ExtUtils::MakeMaker, but you typically don't need that to package
> non-XS perl modules, certainly not any of the ones we've discussed
> possibly bundling up in git.git on-list recently. As a (very occasional)
> contributor to perl.git I'd be interested to know if that's what you
> mean is broken, and if so see if it could be fixed for you.
> 
> 1. 

Re: [PATCH v3] Makefile: replace perl/Makefile.PL with simple make rules

[Ævar Arnfjörð Bjarmason]

On Tue, Dec 12 2017, Randall S. Becker jotted:

> -Original Message-
> On December 10, 2017 4:14 PM, Ævar Arnfjörð Bjarmason wrote:
> Subject: [PATCH v3] Makefile: replace perl/Makefile.PL with simple make rules
>
>>Replace the perl/Makefile.PL and the fallback perl/Makefile used under 
>>NO_PERL_MAKEMAKER=NoThanks with a much simpler implementation heavily 
>>inspired by how the i18n infrastructure's build process works[1].
>>The reason for having the Makefile.PL in the first place is that it was 
>>initially[2] building a perl C binding to interface with libgit, this 
>>functionality, that was removed[3] before Git.pm ever made it to the master 
>>branch.
> 
>
> I would like to request that the we be careful that the git builds do not 
> introduce arbitrary dependencies to CPAN. Some platforms (I can think of one 
> off the top, being NonStop) does not provide for arbitrary additions to the 
> supplied perl implementation as of yet. The assumption about being able to 
> add CPAN modules may apply on some platforms but is not a general capability. 
> I am humbly requesting that caution be used when adding dependencies. Being 
> non-$DAYJOB responsible for the git port for NonStop, this scares me a bit, 
> but I and my group can help validate the available modules used for builds.
>
> Note: we do not yet have CPAN's SCM so can't and don't use perl for access to 
> git anyway - much that I've tried to change that.
>
> Please keep build dependencies to a minimum.
>
> Thanks for my and my whole team.

I think you should be happy with this patch then, and it doesn't add any
more CPAN dependency than before, and sets up a framework (as discussed
in [1]) where we can use more CPAN modules while not requiring packagers
such as yourself to package CPAN modules.

However, it doesn't sound believable to me that even on NonStop you
can't install any CPAN modules whatsoever.

That would also mean that this patch doesn't work for you, because it
means that you either don't have anything resembling a hierarchical
filesystem on which git can be installed in the first place (in which
case it wouldn't work), or perl doesn't have an @INC to search through
perl libs on on NonStop. What does:

perl -V

Return for you on that system?

If this patch works, and if at the bottom of `perl -V` you see some
directories which you could write a package to drop some static *.pm
files, then you can grab a *.tar.gz from CPAN such as the one for
Error.pm[2] and arrange for the *.pm files contained within its lib/
directory to be dropped into one of those @INC directories.

It may be that some aspect of the CPAN toolchain is broken for you, or
even ExtUtils::MakeMaker, but you typically don't need that to package
non-XS perl modules, certainly not any of the ones we've discussed
possibly bundling up in git.git on-list recently. As a (very occasional)
contributor to perl.git I'd be interested to know if that's what you
mean is broken, and if so see if it could be fixed for you.

1. 

RE: [PATCH v3] Makefile: replace perl/Makefile.PL with simple make rules

[Randall S. Becker]

-Original Message-
On December 10, 2017 4:14 PM, Ævar Arnfjörð Bjarmason wrote:
Subject: [PATCH v3] Makefile: replace perl/Makefile.PL with simple make rules

>Replace the perl/Makefile.PL and the fallback perl/Makefile used under 
>NO_PERL_MAKEMAKER=NoThanks with a much simpler implementation heavily inspired 
>by how the i18n infrastructure's build process works[1].
>The reason for having the Makefile.PL in the first place is that it was 
>initially[2] building a perl C binding to interface with libgit, this 
>functionality, that was removed[3] before Git.pm ever made it to the master 
>branch.


I would like to request that the we be careful that the git builds do not 
introduce arbitrary dependencies to CPAN. Some platforms (I can think of one 
off the top, being NonStop) does not provide for arbitrary additions to the 
supplied perl implementation as of yet. The assumption about being able to add 
CPAN modules may apply on some platforms but is not a general capability. I am 
humbly requesting that caution be used when adding dependencies. Being 
non-$DAYJOB responsible for the git port for NonStop, this scares me a bit, but 
I and my group can help validate the available modules used for builds.

Note: we do not yet have CPAN's SCM so can't and don't use perl for access to 
git anyway - much that I've tried to change that.

Please keep build dependencies to a minimum.

Thanks for my and my whole team.

Randall

-- Brief whoami: NonStop developer since approximately 
UNIX(421664400)/NonStop(2112884442) 
-- In my real life, I talk too much.

Re: [PATCH v3] Makefile: replace perl/Makefile.PL with simple make rules

[Junio C Hamano]

Ævar Arnfjörð Bjarmason   writes:

> I don't know what I was thinking when I wrote this, but this logic
> should be fully robust, and I've confirmed that all tests pass
> with/without an Error.pm installed globally.

Thanks.  Will queue and drop the revert from 'pu'.

[PATCH v3] Makefile: replace perl/Makefile.PL with simple make rules

[Ævar Arnfjörð Bjarmason]

Replace the perl/Makefile.PL and the fallback perl/Makefile used under
NO_PERL_MAKEMAKER=NoThanks with a much simpler implementation heavily
inspired by how the i18n infrastructure's build process works[1].

The reason for having the Makefile.PL in the first place is that it
was initially[2] building a perl C binding to interface with libgit,
this functionality, that was removed[3] before Git.pm ever made it to
the master branch.

We've since since started maintaining a fallback perl/Makefile, as
MakeMaker wouldn't work on some platforms[4]. That's just the tip of
the iceberg. We have the PM.stamp hack in the top-level Makefile[5] to
detect whether we need to regenerate the perl/perl.mak, which I fixed
just recently to deal with issues like the perl version changing from
under us[6].

There is absolutely no reason for why this needs to be so complex
anymore. All we're getting out of this elaborate Rube Goldberg machine
was copying perl/* to perl/blib/* as we do a string-replacement on
the *.pm files to hardcode @@LOCALEDIR@@ in the source, as well as
pod2man-ing Git.pm & friends.

So replace the whole thing with something that's pretty much a copy of
how we generate po/build/**.mo from po/*.po, just with a small sed(1)
command instead of msgfmt. As that's being done rename the files
from *.pm to *.pmc just to indicate that they're generated (see
"perldoc -f require").

While I'm at it, change the fallback for Error.pm from being something
where we'll ship our own Error.pm if one doesn't exist at build time
to one where we just use a Git::Error wrapper that'll always prefer
the system-wide Error.pm, only falling back to our own copy if it
really doesn't exist at runtime. It's now shipped as
Git::FromCPAN::Error, making it easy to add other modules to
Git::FromCPAN::* in the future if that's needed.

Functional changes:

 * This will not always install into perl's idea of its global
   "installsitelib". This only potentially matters for packagers that
   need to expose Git.pm for non-git use, and as explained in the
   INSTALL file there's a trivial workaround.

 * The scripts themselves will 'use lib' the target directory, but if
   INSTLIBDIR is set it overrides it. It doesn't have to be this way,
   it could be set in addition to INSTLIBDIR, but my reading of [7] is
   that this is the desired behavior.

 * We don't build man pages for all of the perl modules as we used to,
   only Git(3pm). As discussed on-list[8] that we were building
   installed manpages for purely internal APIs like Git::I18N or
   private-Error.pm was always a bug anyway, and all the Git::SVN::*
   ones say they're internal APIs.

   There are apparently external users of Git.pm, but I don't expect
   there to be any of the others.

   As a side-effect of these general changes the perl documentation
   now only installed by install-{doc,man}, not a mere "install" as
   before.

1. 5e9637c629 ("i18n: add infrastructure for translating Git with
   gettext", 2011-11-18)

2. b1edc53d06 ("Introduce Git.pm (v4)", 2006-06-24)

3. 18b0fc1ce1 ("Git.pm: Kill Git.xs for now", 2006-09-23)

4. f848718a69 ("Make perl/ build procedure ActiveState friendly.",
   2006-12-04)

5. ee9be06770 ("perl: detect new files in MakeMaker builds",
   2012-07-27)

6. c59c4939c2 ("perl: regenerate perl.mak if perl -V changes",
   2017-03-29)

7. 0386dd37b1 ("Makefile: add PERLLIB_EXTRA variable that adds to
   default perl path", 2013-11-15)

8. 87bmjjv1pu@evledraar.booking.com ("Re: [PATCH] Makefile:
   replace perl/Makefile.PL with simple make rules"

Signed-off-by: Ævar Arnfjörð Bjarmason 
---

This fixes the test failure we had on 32 bit Linux due to v2. That
error had nothing to do with 32 bit per-se, it was just a logic error
in the fallback logic if the system didn't have an Error.pm, and
evidently the 32 bit environment doesn't have that installed.

This also incorporates Junio's typo/grammar fixes of my commit
message, tbdiff with v2:

@@ -27,7 +27,7 @@
 So replace the whole thing with something that's pretty much a copy of
 how we generate po/build/**.mo from po/*.po, just with a small sed(1)
 command instead of msgfmt. As that's being done rename the files
-from *.pm to *.pmc just to indicate that they're genreated (see
+from *.pm to *.pmc just to indicate that they're generated (see
 "perldoc -f require").
 
 While I'm at it, change the fallback for Error.pm from being something
@@ -50,9 +50,9 @@
it could be set in addition to INSTLIBDIR, but my reading of [7] is
that this is the desired behavior.
 
- * We don't man pages for all of the perl modules as we used t, only
-   Git(3pm). As discussed on-list[8] that we were building installed
-   manpages for purely internal APIs like Git::I18N or
+ * We don't build man pages for all of the perl modules as we used to,
+   only Git(3pm). As