Re: Lancaster Consensus, deal with PUREPERL_ONLY=0
Am 04.06.2014 um 07:00 schrieb Jens Rehsack rehs...@gmail.com:
Am 04.06.2014 um 04:32 schrieb David Golden x...@xdg.me:
On Tue, Jun 3, 2014 at 1:35 AM, Jens Rehsack rehs...@gmail.com wrote:
1) compiler cannot create executables
2) compiler works, but no perl headers
3) compiler does, but shared library building fails (that is fragile and
minor modifications at the system causes that process to fail)
These three would all fail to make, which would abort installation.
4) As you say: https://rt.cpan.org/Ticket/Display.html?id=75672
Oooh! Excellent example.
make succeeds and make test could fall back to PP when dynaloading fails.
I wonder if there's a way to be clever about this such that under
PUREPERL_ONLY, make never compiles the extension, and then the main
module doesn't just try bootstrap inside eval{} but actually checks to
see if the extension is in @INC. If so, bootstrap it or die; if not,
fall back to pure Perl.
talked yesterday with riba about that in deeper detail.
We decided to do as (probably) suggested by you and
sub legacy_exists { die bogus legacy XS if the extension is in @INC; return
false }
autoconf-check_pp_wanted and return legacy_exists;
autoconf-have_compiler or return legacy_exists;
autoconf-check_perl_api or die;
if( autoconf-extensive_testing ) {
autoconf-check_perl_extension_links or die;
autoconf-check_perl_extension_loads or die;
}
return ok
http://www.cpantesters.org/cpan/report/a25a36da-ea1d-11e3-9cf0-a01f06268b4b
is also an excellent example.
Regarding if the extension is in @INC (have separate installed XS module)
needs a migration plan for LMU, P::U etc.
This is the final solution and we had several ideas yesterday for a migration
plan - but wasn't complete.
Here is the commit for that:
https://github.com/rehsack/Config-AutoConf/commit/e8a14d079995ea519c3aebc698994ea5964bbf70
If there're no objections, I fire a release with that change (and the one which
skips compile tests for bogus environments as Nigel Hone has ...)
Cheers
--
Jens Rehsack
rehs...@gmail.com
Re: Lancaster Consensus, deal with PUREPERL_ONLY=0
On Sun, Jun 01, 2014 at 08:15:20PM +0200, Jens Rehsack wrote: Am 01.06.2014 um 20:09 schrieb Peter Rabbitson rab...@rabbit.us: On Sun, Jun 01, 2014 at 05:59:16PM +0200, Jens Rehsack wrote: Am 01.06.2014 um 15:03 schrieb David Golden x...@xdg.me: The only thing specified in the lancaster consensus is what must happen if that command-line argument is true. I think making a distinction between 0 and undefined will be surprising to people and I would recommend against it. Given this point - how can we give people an instrument to force XS and fail if it's not available? As I mentioned before - you create a separate ::XS distribution, against which the outliers declare dependencies. In general forcing XS when PP is available is *always* *invairably* the wrong approach (which is why they are called outliers above ;) The user must always have a way to enforce or fail. And not every distribution can be split into 2. So please forget the cases where it's possible to split and let's come back to the question: How can we enable the user/packager to make a clear choice? Let me rephrase: making available a XS-only choice, when both PP ans XS are available is a mistake. Not just making the choice is a mistake, *presenting it* is a mistake in its own right. Unless you have a clear use case that you didn't mention before ;)
Re: Lancaster Consensus, deal with PUREPERL_ONLY=0
On Sun, Jun 01, 2014 at 05:59:16PM +0200, Jens Rehsack wrote: Am 01.06.2014 um 15:03 schrieb David Golden x...@xdg.me: The only thing specified in the lancaster consensus is what must happen if that command-line argument is true. I think making a distinction between 0 and undefined will be surprising to people and I would recommend against it. Given this point - how can we give people an instrument to force XS and fail if it's not available? As I mentioned before - you create a separate ::XS distribution, against which the outliers declare dependencies. In general forcing XS when PP is available is *always* *invairably* the wrong approach (which is why they are called outliers above ;)
Re: Lancaster Consensus, deal with PUREPERL_ONLY=0
On Tue, Jun 3, 2014 at 1:35 AM, Jens Rehsack rehs...@gmail.com wrote:
1) compiler cannot create executables
2) compiler works, but no perl headers
3) compiler does, but shared library building fails (that is fragile and
minor modifications at the system causes that process to fail)
These three would all fail to make, which would abort installation.
4) As you say: https://rt.cpan.org/Ticket/Display.html?id=75672
Oooh! Excellent example.
make succeeds and make test could fall back to PP when dynaloading fails.
I wonder if there's a way to be clever about this such that under
PUREPERL_ONLY, make never compiles the extension, and then the main
module doesn't just try bootstrap inside eval{} but actually checks to
see if the extension is in @INC. If so, bootstrap it or die; if not,
fall back to pure Perl.
David
--
David Golden x...@xdg.me Twitter/IRC: @xdg
Re: Lancaster Consensus, deal with PUREPERL_ONLY=0
Am 04.06.2014 um 04:32 schrieb David Golden x...@xdg.me:
On Tue, Jun 3, 2014 at 1:35 AM, Jens Rehsack rehs...@gmail.com wrote:
1) compiler cannot create executables
2) compiler works, but no perl headers
3) compiler does, but shared library building fails (that is fragile and
minor modifications at the system causes that process to fail)
These three would all fail to make, which would abort installation.
4) As you say: https://rt.cpan.org/Ticket/Display.html?id=75672
Oooh! Excellent example.
make succeeds and make test could fall back to PP when dynaloading fails.
I wonder if there's a way to be clever about this such that under
PUREPERL_ONLY, make never compiles the extension, and then the main
module doesn't just try bootstrap inside eval{} but actually checks to
see if the extension is in @INC. If so, bootstrap it or die; if not,
fall back to pure Perl.
talked yesterday with riba about that in deeper detail.
We decided to do as (probably) suggested by you and
sub legacy_exists { die bogus legacy XS if the extension is in @INC; return
false }
autoconf-check_pp_wanted and return legacy_exists;
autoconf-have_compiler or return legacy_exists;
autoconf-check_perl_api or die;
if( autoconf-extensive_testing ) {
autoconf-check_perl_extension_links or die;
autoconf-check_perl_extension_loads or die;
}
return ok
http://www.cpantesters.org/cpan/report/a25a36da-ea1d-11e3-9cf0-a01f06268b4b is
also an excellent example.
Regarding if the extension is in @INC (have separate installed XS module)
needs a migration plan for LMU, P::U etc.
This is the final solution and we had several ideas yesterday for a migration
plan - but wasn't complete.
Cheers
--
Jens Rehsack
rehs...@gmail.com
Re: Lancaster Consensus, deal with PUREPERL_ONLY=0
Am 02.06.2014 um 13:34 schrieb David Golden x...@xdg.me: On Mon, Jun 2, 2014 at 7:28 AM, Jens Rehsack rehs...@gmail.com wrote: Others (eg. packagers) have always a compiler and want benefit from XS when available and what is more important: if the DEVELOPER MODE FLAGS IS ON, they want to have a -Werror behavior even for I enabled foo and it doesn't appear. I don't understand what you mean by this. http://gcc.gnu.org/onlinedocs/gcc/Warning-Options.html It's the widest know of those flags - GNU autotools provide such a one, too. xlc names it -qhalt=severity ... Lot's of tools provide switches which exit when something doesn't do as wanted (if a specific behavior is intended). I'm fine with Jens' distribution have a non-standard switch, but I would prefer a Toolchain community favors that knob to be used. In Lancaster, we considered having both a give me PP option *and* a give me XS option and the group decided it was less confusing to have just one. I didn't had the user control author mind situation. I evolved by implementing (both sides - provide CPAN distributions and use/package them). Here's how I would do it: (a) if the PP option is there, give them PP (b) if the PP option is missing/false, check if there is a compiler (c) if there is a compiler, give them XS or give them death (d) if there isn't a compiler, give them PP If for (c) that means running tests under some XS-only flag of your own, that's fine with me. (b)-(d) shall die (or exit 0 in Makefile.PL) when XS-force flag is set. I currently prove: a) what is wanted (tri-state: PP-only, XS-only, check) pp-b) do just pp b) check for sane xs (perl.h, xsub.h, perl-api, ...) xs-c) (b) or bail-out check-c) (b) or goto (pp-b) d) compile pp Is that somehow clear? But ... that brings me to another point of view. Take Unix::Statgrab, File::LibMagic or BSD::Resource as example - they won't work without XS either. Shall they generally bail-out when PUREPERL_ONLY=1 is set? Cheers -- Jens Rehsack rehs...@gmail.com
Re: Lancaster Consensus, deal with PUREPERL_ONLY=0
On Mon, Jun 2, 2014 at 7:53 AM, Jens Rehsack rehs...@gmail.com wrote: Am 02.06.2014 um 13:34 schrieb David Golden x...@xdg.me: On Mon, Jun 2, 2014 at 7:28 AM, Jens Rehsack rehs...@gmail.com wrote: Others (eg. packagers) have always a compiler and want benefit from XS when available and what is more important: if the DEVELOPER MODE FLAGS IS ON, they want to have a -Werror behavior even for I enabled foo and it doesn't appear. I don't understand what you mean by this. http://gcc.gnu.org/onlinedocs/gcc/Warning-Options.html Dude! I know what the compiler flag does! I don't understand if you're using -Werror as an analogy for how you want cpan clients to behave or if you mean you want -Werror to halt CPAN installation on error even when a PP fallback is available, or something else entirely. In my example, if not PUREPERL_ONLY, then if a compiler is present, compilation works or it fails (possibly due to -Werror), and the install is halted. But ... that brings me to another point of view. Take Unix::Statgrab, File::LibMagic or BSD::Resource as example - they won't work without XS either. Shall they generally bail-out when PUREPERL_ONLY=1 is set? Yes and no. Because they don't offer a PP option, PUREPERL_ONLY is irrelevant to them. They will fail in their make stage as usual. A more user-friendly author could probe for a compiler and bail out in Makefile.PL if one doesn't exist, possibly with one of the CPAN Testers magic strings like OS unsupported: need a compiler. It's an incorrect and possibly misleading error message, but it will result in a CPAN Testers report of NA instead of FAIL. The other option is exit 0 which will just not send a report at all. E.g. Devel::CheckLib David -- David Golden x...@xdg.me Twitter/IRC: @xdg
Re: Lancaster Consensus, deal with PUREPERL_ONLY=0
On Mon, Jun 2, 2014 at 9:22 AM, Jens Rehsack rehs...@gmail.com wrote: Web-Search for CPAN Testers magic strings doesn't provide suitable results - is there a list of those magic results? http://wiki.cpantesters.org/wiki/CPANAuthorNotes If I would know what is expected, I will make Config::AutoConf behave. Devel::CheckLib doesn't work at all - but I don't want to send pull requests using C::AC until I'm sure that C::AC doesn't introduce more pitfalls than it solves ;) Here's what I think is important to understand: * PASS/FAIL reports are only supposed to happen if tests actually run (e.g. make test, etc.). These should not be sent if a cpan reporting client can detect that perl prereqs are missing. * UNKNOWN reports are supposed to happen when the dist never reaches the test stage (e.g. Makefile.PL or make fails) or if there are no test files to run. This means I don't know if your tests pass or fail because I never got that far * NA reports are supposed to happen when there is no possible way for tests to ever succeed. Generally this is for platform/OS issues, like Win32::Job will never succeed on Unix, or something that needs threads will never run on a non-threaded perl. I've seen it used and used it myself for situations where some configuration issue prevents success -- e.g. you're running on OpenBSD and your perl is neither threaded or single-threaded-but-linking-libpthead. * exit 0 from Makefile.PL has a coincidentally useful behavior that CPAN Testers exploits. The CPAN clients happened to all check for Makefile being created even if Makefile.PL returned exit code zero and bailed out if it didn't exist. That allowed a way to let Makefile.PL abort without triggering a CPAN Testers report message. This turned out to be really useful for non-perl dependencies (e.g. compiler, C libraries, external program, etc.) David
Re: Lancaster Consensus, deal with PUREPERL_ONLY=0
Am 02.06.2014 um 16:05 schrieb David Golden x...@xdg.me: On Mon, Jun 2, 2014 at 9:22 AM, Jens Rehsack rehs...@gmail.com wrote: Web-Search for CPAN Testers magic strings doesn't provide suitable results - is there a list of those magic results? http://wiki.cpantesters.org/wiki/CPANAuthorNotes This can help for Sys::Filesystem - good to know. I'll re-check how that behaves against that description. If I would know what is expected, I will make Config::AutoConf behave. Devel::CheckLib doesn't work at all - but I don't want to send pull requests using C::AC until I'm sure that C::AC doesn't introduce more pitfalls than it solves ;) Here's what I think is important to understand: * PASS/FAIL reports are only supposed to happen if tests actually run (e.g. make test, etc.). These should not be sent if a cpan reporting client can detect that perl prereqs are missing. * UNKNOWN reports are supposed to happen when the dist never reaches the test stage (e.g. Makefile.PL or make fails) or if there are no test files to run. This means I don't know if your tests pass or fail because I never got that far * NA reports are supposed to happen when there is no possible way for tests to ever succeed. Generally this is for platform/OS issues, like Win32::Job will never succeed on Unix, or something that needs threads will never run on a non-threaded perl. I've seen it used and used it myself for situations where some configuration issue prevents success -- e.g. you're running on OpenBSD and your perl is neither threaded or single-threaded-but-linking-libpthead. * exit 0 from Makefile.PL has a coincidentally useful behavior that CPAN Testers exploits. The CPAN clients happened to all check for Makefile being created even if Makefile.PL returned exit code zero and bailed out if it didn't exist. That allowed a way to let Makefile.PL abort without triggering a CPAN Testers report message. This turned out to be really useful for non-perl dependencies (e.g. compiler, C libraries, external program, etc.) That's what I knew and how C::AC behaves. So everything is cool :D Additionally - if one needs - there is a config.log containing all made tests and for those who fail the complete scenario with call and failure output. Cheers -- Jens Rehsack rehs...@gmail.com
Re: Lancaster Consensus, deal with PUREPERL_ONLY=0
Hi Jens, * Jens Rehsack rehs...@gmail.com [2014-06-02 13:30]: * Karen Etheridge p...@froods.org [2014-06-02 01:30]: I'm wondering why it isn't always possible to split a dist into two implementations, one PP and the other with XS optimizations. If the dist simply cannot be implemented using pure Perl (Moose, for example), then surely the right thing to is simply refuse to install on PP systems? Eg. LMU - you cannot split out LMU::XS for historical reasons. how come? Is there a technical difficulty with that? What prevents it? Regards, -- Aristotle Pagaltzis // http://plasmasturm.org/
Re: Lancaster Consensus, deal with PUREPERL_ONLY=0
On Mon, Jun 2, 2014 at 7:28 PM, Aristotle Pagaltzis pagalt...@gmx.de wrote: If I have my system set up to support XS then I want it to use XS, and if for some reason it can’t, then there is a bug somewhere that needs to be fixed, so I do *not* want it to silently give me the PP version and carry on, but to fall down and die screaming so I’ll find out. I've been trying to think of how that could happen and so far all I can come up with is this scenario: * XS bits compile successfully, but the shared object croaks during bootstrap, and the calling module traps the bootstrap error and falls back to PP David
Re: Lancaster Consensus, deal with PUREPERL_ONLY=0
Am 03.06.2014 um 01:28 schrieb Aristotle Pagaltzis pagalt...@gmx.de: * Karen Etheridge p...@froods.org [2014-06-02 01:30]: I do not see any gain in specifying give me XS or give me death. This I find easy to see, at least if we are talking of the general case: If I have my system set up to support XS then I want it to use XS, and if for some reason it can’t, then there is a bug somewhere that needs to be fixed, so I do *not* want it to silently give me the PP version and carry on, but to fall down and die screaming so I’ll find out. Precisely. However, I am unsure of the context of the discussion. I simply detected that we (p5) don't cover that situation, we always fallback to PP. The above reasoning is applicable and important mostly when it comes to production deployments: I never want my servers trying to limp on with a CPU- and/or memory-hungry setup that will only reveal itself as such in monitoring data. Production deployment is somehow similar to packaging - packaging is one step in deployment (even tar xjf ... is packaging or rsync a prepared directory). And then it comes - do the farm has the same arch and is XS possible and wanted, does your package process gives you the ability to distinguish between archs or is your process one blob to rule them all - than is FatPacking your hammer and XS is unwanted. But: Do the folks at CERN really FatPacking? AFAIK (I talk here and there in train to guys having experiments there) no. But maybe the discussion is about some other scenario in which the given reasoning is inapplicable? Since I don’t know, the above use case may be neither here nor there. Depending on one's background, usecases sounds ridiculous. As pkgsrc maintainer (16+ platforms 30+ architectures supported) I do not understand the existence of fat packer at all. pbulk+pkgin is easier to deal with ... But I accept the usecase - even when I don't understand it. Cheers -- Jens Rehsack rehs...@gmail.com
Re: Lancaster Consensus, deal with PUREPERL_ONLY=0
Am 03.06.2014 um 01:42 schrieb David Golden x...@xdg.me: On Mon, Jun 2, 2014 at 7:28 PM, Aristotle Pagaltzis pagalt...@gmx.de wrote: If I have my system set up to support XS then I want it to use XS, and if for some reason it can’t, then there is a bug somewhere that needs to be fixed, so I do *not* want it to silently give me the PP version and carry on, but to fall down and die screaming so I’ll find out. I've been trying to think of how that could happen and so far all I can come up with is this scenario: * XS bits compile successfully, but the shared object croaks during bootstrap, and the calling module traps the bootstrap error and falls back to PP There're several scenarios causing XS builds to fail: 1) compiler cannot create executables * invalid cflags configured (see http://www.cpantesters.org/cpan/report/a25a36da-ea1d-11e3-9cf0-a01f06268b4b for an example) * beta compiler test (sunpro 12.4 on linux fails loading libdwarf) * cc installed, but as missing 2) compiler works, but no perl headers 3) compiler does, but shared library building fails (that is fragile and minor modifications at the system causes that process to fail) 4) As you say: https://rt.cpan.org/Ticket/Display.html?id=75672 I think currently we don't cross-compile (handy market would make me want to change that), which allows tests for 4, but until 3 we can always test. 4 only when we don't cross-compile. And what does I prepared my system to support XS mean? I installed a compiler or I installed precisely the same compiler/binutils used to build perl? I think, there should be a flag the user to be able to set to express the opinion the system is prepared for XS - not any magic detection. Users word counts. Cheers -- Jens Rehsack rehs...@gmail.com
Re: Lancaster Consensus, deal with PUREPERL_ONLY=0
The only thing specified in the lancaster consensus is what must happen if that command-line argument is true. I think making a distinction between 0 and undefined will be surprising to people and I would recommend against it. On Sun, Jun 1, 2014 at 7:19 AM, Jens Rehsack rehs...@gmail.com wrote: Hi, I implemented the interpretation of PUREPERL_ONLY in C::AC as: equals 1 - don't XS (even don't prove) equals 0 - don't PP only (force XS, fail if cannot be proved) undefined - XS/PP depends on prove Ribasushi hinted to verify my interpretation to ask on ML. So what do others think - how should PUREPERL_ONLY=0 being interpreted? Cheers -- Jens Rehsack rehs...@gmail.com -- David Golden x...@xdg.me Twitter/IRC: @xdg
Re: Lancaster Consensus, deal with PUREPERL_ONLY=0
Am 01.06.2014 um 15:03 schrieb David Golden x...@xdg.me: The only thing specified in the lancaster consensus is what must happen if that command-line argument is true. I think making a distinction between 0 and undefined will be surprising to people and I would recommend against it. Given this point - how can we give people an instrument to force XS and fail if it's not available? On Sun, Jun 1, 2014 at 7:19 AM, Jens Rehsack rehs...@gmail.com wrote: Hi, I implemented the interpretation of PUREPERL_ONLY in C::AC as: equals 1 - don't XS (even don't prove) equals 0 - don't PP only (force XS, fail if cannot be proved) undefined - XS/PP depends on prove Ribasushi hinted to verify my interpretation to ask on ML. So what do others think - how should PUREPERL_ONLY=0 being interpreted? Cheers -- Jens Rehsack rehs...@gmail.com
Re: Lancaster Consensus, deal with PUREPERL_ONLY=0
Am 01.06.2014 um 20:09 schrieb Peter Rabbitson rab...@rabbit.us: On Sun, Jun 01, 2014 at 05:59:16PM +0200, Jens Rehsack wrote: Am 01.06.2014 um 15:03 schrieb David Golden x...@xdg.me: The only thing specified in the lancaster consensus is what must happen if that command-line argument is true. I think making a distinction between 0 and undefined will be surprising to people and I would recommend against it. Given this point - how can we give people an instrument to force XS and fail if it's not available? As I mentioned before - you create a separate ::XS distribution, against which the outliers declare dependencies. In general forcing XS when PP is available is *always* *invairably* the wrong approach (which is why they are called outliers above ;) The user must always have a way to enforce or fail. And not every distribution can be split into 2. So please forget the cases where it's possible to split and let's come back to the question: How can we enable the user/packager to make a clear choice? Cheers -- Jens Rehsack rehs...@gmail.com
