Re: Providing command-line arguments to tests run via 'prove'
James E Keenan wrote: I would like to be able to provide the tests run via 'prove' with options something like this: some_variant_of_prove t/*.t --option1 --option2 arg1 arg2 ... where those 4 command-line options/arguments would be available to *each* of the individual test files in the suite. A given test file would not necessarily have to respond to those options/arguments, but if it wanted to, it would do so simply by assigning the contents of the test script's @ARGV to some other array. Can this be done? And, thanks to the contributors to this list, it was done. This has been supported since Test-Harness-3.04 and assumed its current version -- with the 'arisdottle' :: character announcing the beginning of arguments to be passed the test files -- in 3.05. I have begun to make use of this in a Perl 5 module in the Parrot distribution which governs the running of tests of configuration step classes: http://svn.perl.org/parrot/branches/tcif/lib/Parrot/Configure/Options/Test.pm at r26081. This is a work-in-progress, but this was exactly the reason why I put out the call for passing of test arguments in my OP to this thread. My thanks to Andy Armstrong and all others who contributed to this development, which will open up new approaches to testing with Perl. (I am recommending one refinement in the App::Prove code underlying this version of 'prove'. See: http://rt.cpan.org/Ticket/Display.html?id=33609.) Thank you very much. Jim Keenan
Re: Providing command-line arguments to tests run via 'prove'
On 27 Feb 2008, at 00:48, James E Keenan wrote: My thanks to Andy Armstrong and all others who contributed to this development, which will open up new approaches to testing with Perl. Splendid news, thanks Jim. (I am recommending one refinement in the App::Prove code underlying this version of 'prove'. See: http://rt.cpan.org/Ticket/Display.html?id=33609.) Thanks for that. I've applied your patch with minor changes. I've just released 3.10 a few hours ago so I'd rather not send 3.11 out hot on its heels - but it'll be released in the next few days. -- Andy Armstrong, Hexten
Re: Providing command-line arguments to tests run via 'prove'
Andy Lester writes: As much as I like the non-quotedness of the -- or --testargs idea, I really think it needs to be --testargs='--foo --bar'. If we have just -- then its straightforward what to do with the arguments. After everything up to and including -- has been shifted off @ARGV, then you can just do: my @test_args = @ARGV; and if $test_file is the test being run you can then do the moral equivalent of: system $test_file, @test_args; But with --testargs you read in a single string so you get something like: $test_args = '--foo --bar'; And suppose we're trying to run $test_file. Obviously this is wrong: system $test_file, $test_args; For the above arguments you'd want to do: system $test_file, split / /, $test_args; But what if one of the arguments needs to have a space in it. With -- you can do: % runtests whatever -- --foo kapow blam How do you write that with --testargs? Perhaps: % runtests whatever --testargs='--foo kapow blam' Hmmm, but now this is wrong: system $test_file, split / /, $test_args; We need to only split on the unquoted whitespace, and we need to remove the quotes from the outside of what will then be the second argument. Obviously it's possible to do that. But it's hard to get it right, and to meet users' expectations. In particular, different OSes have different quoting rules. With -- it's still the local shell doing the quoting, so the user obviously just uses their normal quoting rules. With --testargs, runtests would be doing the parsing. Should it emulate the local OS's rules (but then behaving differently on each platform), or should it pick one set of rules and implement them everywhere (but then imposing, say, Unix-style quoting on Windows users)? The convention of using '--' to mean 'that's the end of my own arguments' neatly avoids all of these issues. Smylers
Re: Providing command-line arguments to tests run via 'prove'
Michael G Schwern writes: There's nothing about '--' which indicates pass the rest through to the tests. It did to me, honestly! As soon as I opened David's e-mail at the start of this thread the -- jumped out of the paragraph of text, and I guessed what his words would say before I read them. In fact, it normally means stop processing the following as switches and instead treat them as files. Or possibly, it means stop processing the following as switches. Which is what it does mean in this case. Smylers
Re: Providing command-line arguments to tests run via 'prove'
On Thu, Nov 29, 2007 at 10:14:29AM +, Smylers wrote: Michael G Schwern writes: There's nothing about '--' which indicates pass the rest through to the tests. It did to me, honestly! As soon as I opened David's e-mail at the start of this thread the -- jumped out of the paragraph of text, and I guessed what his words would say before I read them. That's because you knew the context already - it's in the subject. Without that context it might not have been so obvious. In fact, it normally means stop processing the following as switches and instead treat them as files. Or possibly, it means stop processing the following as switches. Which is what it does mean in this case. Yes, I've never expected the arguments after -- to necessarily be files. I think that's often the case, especially when the files look like switches, but ... On Wed, Nov 28, 2007 at 10:01:01PM -0800, Michael G Schwern wrote: How do other utilities handle this sort of thing? ... I have a bunch of programs here where I use -- for this very purpose, to pass options through to programs that will be called by the program I am calling. The only real problem, as I mentioned before, is that this is a one off. The scheme can't really be extended to pass two sets of options through to two programs, for example. That said, I'm not overly bothered by this. It's not a feature I've felt a need for in many years of writing fairly complicated test suites. In general, I try to keep my command lines simple. If there is something complicated that I will need to run regularly, I will make it a make target, which makes it easy to run and documents it as a bonus. When it is a make target, it doesn't really matter if it is a little more complicated than it might be. -- Paul Johnson - [EMAIL PROTECTED] http://www.pjcj.net
Re: Providing command-line arguments to tests run via 'prove'
Smylers said:
The convention of using '--' to mean 'that's the end of my own
arguments' neatly avoids all of these issues.
FWIW, I'm with Smylers here. '--' has been around for many years as a
command-line convention for signifying the end of options. (from
bash(1)).
/me likes gentle learning curves, and would like to see command-line
interface consistency; not just within the Perl sphere, but in general.
- Salve, who thinks command-line options deserve the same amount of
attention as programming API's and over-the-wire protocols.
--
#!/usr/bin/perl
sub AUTOLOAD{$AUTOLOAD=~/.*::(\d+)/;seek(DATA,$1,0);print# Salve Joshua Nilsen
getc DATA}$='};{';@_=unpack(C*,unpack(u*,':4@,$'.# [EMAIL
PROTECTED]
'2!--5-(50P%$PL,!0X354UC-PP%/0\`'.\n));eval {'@_'}; __END__ is near! :)
Re: Providing command-line arguments to tests run via 'prove'
--- chromatic [EMAIL PROTECTED] wrote: Could we not add a feature to prove and/or runtests such that, any arguments after a bare -- will be passed on to the scripts it runs? I've often wanted this myself, and --exec seems like overkill to me. Seconded. Just to clarify, --exec *is* overkill in this case, but when it's needed, it's really handy. Cheers, Ovid -- Buy the book - http://www.oreilly.com/catalog/perlhks/ Perl and CGI - http://users.easystreet.com/ovid/cgi_course/ Personal blog - http://publius-ovidius.livejournal.com/ Tech blog - http://use.perl.org/~Ovid/journal/
Re: Providing command-line arguments to tests run via 'prove'
On 29 Nov 2007, at 10:34, Tom Heady wrote: If we have just -- then its straightforward what to do with the arguments. After everything up to and including -- has been shifted off @ARGV, then you can just do: my @test_args = @ARGV; and if $test_file is the test being run you can then do the moral equivalent of: system $test_file, @test_args; ... ++ We currently have an implementation that uses '--' in the repo[1]. We're going round in circles, orbiting an ideal solution that probably doesn't exist. Consensus proving elusive. Will the sky fall and babies cry if we go with '--'? I would hope not. Let's have a quick show of hands and move on, eh? [ ] -- [ ] something else (please specify) Press the red button to vote. [1] http://svn.hexten.net/tapx/trunk -- Andy Armstrong, Hexten
Re: Providing command-line arguments to tests run via 'prove'
On 29 Nov 2007, at 12:20, Andy Armstrong wrote: [X] -- [ ] something else (please specify) -- Andy Armstrong, Hexten
Re: Providing command-line arguments to tests run via 'prove'
On 29 Nov 2007, at 12:20, Andy Armstrong wrote: [X] -- [ ] something else (please specify) Adrian
Re: Providing command-line arguments to tests run via 'prove'
[X] -- [ ] something else (please specify) And don't forget the ponies.
Re: Providing command-line arguments to tests run via 'prove'
On Thu, Nov 29, 2007 at 04:10:12AM +0100, A. Pagaltzis wrote: * Andy Armstrong [EMAIL PROTECTED] [2007-11-29 04:02]: I agree re the semantics of '--' - but I'd rather have a sentinel than a quoted string. Having to get nested quoting right is a bit of cognitive load we can spare people from. So: * Andy's --testargs has the disadvantage of being visually indistinct from regular args * '--' has another meaning by convention How about some other short, not arg like, shell safe punctuation sequence? $ prove -v t/sprocket.t -+ --teeth 12 How about a double colon? $ prove -v t/sprocket.t :: --teeth 12 It’s quite unlikely that anyone would name a test file `::`. In fact no one who has to care about Windows at all (which would be everyone who isn’t writing (Linux|Mac|BSD)::* modules) would want to do that because on Windows you *can’t* name a file `::`. (A single colon would do just as well, but might get visually lost in a long `prove` invocation.) I like the :: (double something, just like --, and not interpreted by the shell), and agree that -- is already useful (and already has a meaning for prove, since it uses Getopt::Long). It would work well with: $ prove -v -r -- -file-with-initial-dash.t t/ :: arg1 arg2 I guess ++ would work too: $ prove -v -r -- -file-with-initial-dash.t t/ ++ arg1 arg2 Mmm, maybe I like ++ over ::. And -- should be kept meaning what it already means. -- Philippe Bruhat (BooK) The best of intentions must still have directions. (Moral from Groo The Wanderer #95 (Epic))
Re: Providing command-line arguments to tests run via 'prove'
On 29 Nov 2007, at 14:54, Andy Lester wrote: On Nov 29, 2007, at 6:20 AM, Andy Armstrong wrote: [ ] -- [ ] something else (please specify) --testargs='--foo --bar' You spoilt your ballot by not checking a box :) (but I'll count it anyway) -- Andy Armstrong, Hexten
Re: Providing command-line arguments to tests run via 'prove'
On Thu, Nov 29, 2007 at 02:57:34PM +, Andy Armstrong wrote: On 29 Nov 2007, at 14:54, Andy Lester wrote: On Nov 29, 2007, at 6:20 AM, Andy Armstrong wrote: [ ] -- [ ] something else (please specify) --testargs='--foo --bar' You spoilt your ballot by not checking a box :) (but I'll count it anyway) I think the chad is hanging. -- Jeb Bush - [EMAIL PROTECTED]
Re: Providing command-line arguments to tests run via 'prove'
On 29 Nov 2007, at 15:14, Paul Johnson wrote: I think the chad is hanging. I'm sure it'll be fine. -- Andy Armstrong, Diebold
Re: Providing command-line arguments to tests run via 'prove'
On Thu, Nov 29, 2007 at 12:20:49PM +, Andy Armstrong wrote: Let's have a quick show of hands and move on, eh? [ ] -- [ ] something else (please specify) Press the red button to vote. [X] ++ -- Philippe Bruhat (BooK) The greatest monster of them all is ignorance. (Moral to Pal'n Drumm Story in Groo #89 (Epic))
Re: Providing command-line arguments to tests run via 'prove'
# from Andy Armstrong # on Thursday 29 November 2007 04:20: [ ] -- [X] something else (please specify) The -- means end of options for prove, which makes everything coming after that into a filename. This usage replaces that meaning with something entirely different and unexpected. Why are we voting? --Eric -- If the above message is encrypted and you have lost your pgp key, please send a self-addressed, stamped lead box to the address below. --- http://scratchcomputing.com ---
Re: Providing command-line arguments to tests run via 'prove'
On 29 Nov 2007, at 17:03, Eric Wilhelm wrote: Why are we voting? To solicit the opinions of interested parties. -- Andy Armstrong, Hexten
Re: Providing command-line arguments to tests run via 'prove'
Salve J Nilsen [EMAIL PROTECTED] writes: Smylers said: The convention of using '--' to mean 'that's the end of my own arguments' neatly avoids all of these issues. FWIW, I'm with Smylers here. '--' has been around for many years as a command-line convention for signifying the end of options. (from bash(1)). end of arguments ne end of options. For what it's worth, I expect '--' to mean end of options. What follows are non-option arguments, but still arguments. In this case, files or directories: # USAGE # #prove [options] [files or directories] Do I have a vote? [ ] -- [X] something else (please specify) I would look to find(1)'s -exec, xterm(1)'s -e, or even perl(1)'s -e for (somewhat analogous) prior art. Any one of these would be good, as far as I'm concerned; each has its downsides, sure, but I don't think there is a solution without. Of all solutions I've seen suggested, '--' is the only one that runs contrary to my expectations. Eirik -- Eirik Berg Hanssen, [EMAIL PROTECTED] Just this .sig then nothing more
Re: Providing command-line arguments to tests run via 'prove'
* Philippe Bruhat (BooK) [EMAIL PROTECTED] [2007-11-29 15:30]: Mmm, maybe I like ++ over ::. And -- should be kept meaning what it already means. I think `++` is a legal filename on Windows, as opposed to `::`. (But I haven’t a Windows box here to check, so if someone else can confirm…) Regards, -- Aristotle Pagaltzis // http://plasmasturm.org/
Re: Providing command-line arguments to tests run via 'prove'
Andy Armstrong wrote: Will the sky fall and babies cry if we go with '--'? I would hope not. It closes off the conventional method of passing in files that look like switches. So yes, that hunk of sky will fall. Let's have a quick show of hands and move on, eh? [ ] -- [X] something else (please specify) One of... --test_args='--foo --bar' or --start_test_args --foo --bar --end_test_args I prefer the former, but if technical limitations in the way we call the test programs force us to keep the arguments as a list [1] then the latter will do. [1] system $program, $file, @args; -- I am somewhat preoccupied telling the laws of physics to shut up and sit down. -- Vaarsuvius, Order of the Stick http://www.giantitp.com/comics/oots0107.html
Re: Providing command-line arguments to tests run via 'prove'
On 30 Nov 2007, at 01:23, A. Pagaltzis wrote: * Andy Armstrong [EMAIL PROTECTED] [2007-11-30 01:45]: If you want to be able to flip flop in and out of test args and prove args modes just repeat it and say that tests can't use '::' as an arg. $ prove -rb :: gargoyle :: --state=hot,save,all :: splendour :: t/ spog.t Please no. That’s unreadable. Let’s just do the simplest thing that could possibly work and then leave well enough alone; see above argumentation. Yeah, that was supposed to be a tongue in cheek illustration of the folly of wanting to flip back and forth between contexts except that I forgot to make it funny :) -- Andy Armstrong, Hexten
Re: Providing command-line arguments to tests run via 'prove'
I wonder if all of these different features for *running* tests should go into modules that play together nicely and compose in such a way that writing our own individual runner scripts is as easy as using Test::Builder is. Agreed. We've got this brand new flexible parser doodad, and we're still working on making a monolithic prove. -- Andy Lester = [EMAIL PROTECTED] = www.petdance.com = AIM:petdance
Re: Providing command-line arguments to tests run via 'prove'
# from Michael G Schwern
# on Thursday 29 November 2007 15:34:
One of...
--test_args='--foo --bar'
or
--start_test_args --foo --bar --end_test_args
I prefer the former, but if technical limitations in the way we call
the test programs force us to keep the arguments as a list...
It's not so-much 'technical limitations' as 'loss of clarity' wrt spaces
that kills the quoted scheme.
I suggest that there be a start sentinel and optional end sentinell
--test-args --foo --bar --baz --whatever and -- such
Would call the tests with qw(--foo --bar --baz --whatever and -- such).
That is, args continue thru to $ARGV[$#ARGV] if there is no
grep({$ARGV[$_] eq '--end-test-args'} 0..$#ARGV) or so.
This means buzzword#42 (simplethingssimple-hardthingspossible) where the
complicated (aliases, wrappers, etc) case is like:
--test-args --foo --bar --baz --whatever and -- such --end-test-args \
--more-prove-args and --things of this nature -- \
--and --some --funny-filenames too
Or so.
--Eric
--
Left to themselves, things tend to go from bad to worse.
--Murphy's Corollary
---
http://scratchcomputing.com
---
Re: Providing command-line arguments to tests run via 'prove'
On 30 Nov 2007, at 00:34, Eric Wilhelm wrote:
It's not so-much 'technical limitations' as 'loss of clarity' wrt
spaces
that kills the quoted scheme.
I suggest that there be a start sentinel and optional end sentinell
--test-args --foo --bar --baz --whatever and -- such
Would call the tests with qw(--foo --bar --baz --whatever and --
such).
That is, args continue thru to $ARGV[$#ARGV] if there is no
grep({$ARGV[$_] eq '--end-test-args'} 0..$#ARGV) or so.
This means buzzword#42 (simplethingssimple-hardthingspossible) where
the
complicated (aliases, wrappers, etc) case is like:
--test-args --foo --bar --baz --whatever and -- such --end-test-
args \
--more-prove-args and --things of this nature -- \
--and --some --funny-filenames too
Why so verbose?
Aristotle's '::' suggestion is my favourite. It's syntactically clean.
It's visually distinct from other switches - so you can easily see
that something special is happing. It's short.
If you want to be able to flip flop in and out of test args and prove
args modes just repeat it and say that tests can't use '::' as an arg.
$ prove -rb :: gargoyle :: --state=hot,save,all :: splendour :: t/spog.t
--
Andy Armstrong, Hexten
Re: Providing command-line arguments to tests run via 'prove'
On Thursday 29 November 2007 16:43:18 Andy Armstrong wrote: $ prove -rb :: gargoyle :: --state=hot,save,all :: splendour :: t/spog.t Y'know, at some point I think I'll just write my own wrapper around TAP::Harness, with attractive members of the appropriate gender, and blackjack, and an amusement park. That is, there *may* be a point here where cramming too much into prove is getting confusing. At some point, we all have to write some Perl. -- c
Re: Providing command-line arguments to tests run via 'prove'
On 30 Nov 2007, at 00:51, chromatic wrote: $ prove -rb :: gargoyle :: --state=hot,save,all :: splendour :: t/ spog.t Y'know, at some point I think I'll just write my own wrapper around TAP::Harness, with attractive members of the appropriate gender, and blackjack, and an amusement park. That is, there *may* be a point here where cramming too much into prove is getting confusing. At some point, we all have to write some Perl. The state stuff is useful I think. Passing args to tests is more marginal and I certainly don't believe it's a big enough deal to warrant the amount of debate it's attracted. I'm going to go with '::'. If anyone dislikes it enough, as you say, they can write their own tool. -- Andy Armstrong, Hexten
Re: Providing command-line arguments to tests run via 'prove'
On 30 Nov 2007, at 01:22, chromatic wrote: I wonder if all of these different features for *running* tests should go into modules that play together nicely and compose in such a way that writing our own individual runner scripts is as easy as using Test::Builder is. Well it kinda is. The state management stuff is stand alone and App::Prove is subclassable - and nothing's too monolithic or intertwingled. It could certainly be more explicitly pluggable but I don't think it's the worst pile of bits for making testing things even now. The priority now should be documentation. It's out of sync with the code in places and generally fails to provide a coherent view of how the bits fit together. -- Andy Armstrong, Hexten
Re: Providing command-line arguments to tests run via 'prove'
Andy Armstrong wrote: Why so verbose? Aristotle's '::' suggestion is my favourite. It's syntactically clean. It's visually distinct from other switches - so you can easily see that something special is happing. It's short. And without context it's totally void of meaning. Someone hit it on the head earlier in the conversation, we understand what :: (or --) means because we're all talking about what it means. It's clear because it's in context. Out of context, '::' means nothing. The absurdity of :: becomes more clear because you can apply the same arguments to any switch of prove. Let's use ++ instead of '--color' because its syntactically clean and visually distinct. Or ;; for --merge. Short and clean isn't enough. A control has to suggest it's functionality. :: doesn't even suggest pass through the following to the test even after its been explained. Because of this, :: is among a class of poorly designed controls you have to learn by rote. For an example of this, just look through perlvar and think about how often you still have to refer back to it. At this point I'm making my argument on repeat, so this will be the last time I say it. If you want to be able to flip flop in and out of test args and prove args modes just repeat it and say that tests can't use '::' as an arg. The flip-flop case is an unrealistic bit of complexity. Ignore it. -- ROCKS FALL! EVERYONE DIES! http://www.somethingpositive.net/sp05032002.shtml
Re: Providing command-line arguments to tests run via 'prove'
On 30 Nov 2007, at 02:07, Michael G Schwern wrote: [snip perfectly reasonable argument] Also, shell is a ubiquitous tool that you're going to use over and over again, so some rote learning is justified. prove is not such a thing. You may well be right. We may all be right. I'm losing the will to think about it any more. You know where it lives if you want to commit something more closely resembling your approach. Anyhow, I'm with chromatic. We're shoving too much functionality into one interface. I've noticed that it's hardly a minimal wrapper any more. I'm not / entirely/ oblivious to such aesthetic niceties :) I've been wanting the state stuff for ages. I considered putting it in App::ProvePlus and releasing that separately but, particularly with Adrian's tuning, it seems like something that anyone who is using prove (as opposed to make test / cpan) might find useful. I don't see it as feature creep; rather it seemed to me that prove was lacking an interface to control which tests execute in what order. I'm uncertain of the utility of being able to pass args to tests from prove - but as a result of implementing it the interface for passing args to tests via TAP::Harness, whether or not you're using --exec, is cleaner. The syntax debate isn't worth any more of anyone's headspace. If people need it they'll work it out; if they don't they're unlikely to trip over it. That's good enough. So I'm unrepentant but have no intention of continuing to bloat prove. -- Andy Armstrong, Hexten
Re: Providing command-line arguments to tests run via 'prove'
* Michael G Schwern [EMAIL PROTECTED] [2007-11-30 02:35]: Let's use ++ instead of '--color' because its syntactically clean and visually distinct. Or ;; for --merge. Except there’s no good precedent for sentinel values whereas there are clear precedents for switches. find(1) uses `;` and `+`. Other tools use other stuff (I know that I know at least one more example that I can’t remember right now). :: doesn't even suggest pass through the following to the test even after its been explained. I was originally going to suggest a single colon. That certainly seemed inherently suggestive to me: prove -rb foo/ bar/ : http://localhost:2342 Now remember that a colon (single or double) can’t possibly be a file name on a large portion of the systems that Perl runs on. That would seem to me like an instant clue that something else is going on. So I don’t think it’s quite as devoid of any suggestiveness outside of context as you make it out to be. The reason I proposed a double colon instead is that the single colon is rather visually demure and gets lost in the shindig of a long command line. The “can’t be a filename” clue still holds for that case, although it’s less obvious what the symbol might be expressing. Because of this, :: is among a class of poorly designed controls you have to learn by rote. So how are you going to pass switch-like test file names to `prove`, which is the customary meaning of `--`? Are you going teach people they should learn a *different* sentinel that “acts like what `--` would act like if we weren’t using `--` for this other tool-specific functionality”? Is rote *un*learning better than rote learning? Regards, -- Aristotle Pagaltzis // http://plasmasturm.org/
Re: Providing command-line arguments to tests run via 'prove'
Andy Armstrong wrote: On 30 Nov 2007, at 01:32, Michael G Schwern wrote: The absurdity of :: becomes more clear because you can apply the same arguments to any switch of prove. Let's use ++ instead of '--color' because its syntactically clean and visually distinct. Or ;; for --merge. It's not a switch. It's the end of the switches. Depends on the way you think about it. Either way the argument still applies. Short and clean isn't enough. A control has to suggest it's functionality. :: doesn't even suggest pass through the following to the test even after its been explained. Because of this, :: is among a class of poorly designed controls you have to learn by rote. For an example of this, just look through perlvar and think about how often you still have to refer back to it. Would you like your shell to use --pipe instead of '|'? '::' looks to me like a divider between distinct things - which is what it is. Again, you have context, so of course it looks like what you expect. Also, dividing distinct things is not the important thing here. What happens to those things? That the remaining stuff will be passed onto each test is the important thing. :: does not convey that. | makes sense to us because we're trained to it's metaphor. But that metaphor is learned and even then is flawed. | doesn't suggest take the output of this program and shove it into this other program UNLESS you already know that it is piping. Even so, wouldn't - or = have made more sense? this - that or this = that or even this - that conveys flow better just as and do for files. It could also be argued that | is a divider, just like ::, something it's more traditionally used for. Shell can be forgiven for trading terseness for ease of understanding because piping is a powerful feature you're going to use a lot. It warrants a lot of compression. Pass these switches to the tests does not. Also, shell is a ubiquitous tool that you're going to use over and over again, so some rote learning is justified. prove is not such a thing. The flip-flop case is an unrealistic bit of complexity. Ignore it. Was supposed to be a joke making just that point :) Don't quit your day job. ;) Anyhow, I'm with chromatic. We're shoving too much functionality into one interface. -- THIS I COMMAND!
Re: Providing command-line arguments to tests run via 'prove'
A. Pagaltzis wrote: * Michael G Schwern [EMAIL PROTECTED] [2007-11-30 02:35]: Let's use ++ instead of '--color' because its syntactically clean and visually distinct. Or ;; for --merge. Except there’s no good precedent for sentinel values whereas there are clear precedents for switches. find(1) uses `;` and `+`. Other tools use other stuff (I know that I know at least one more example that I can’t remember right now). :: doesn't even suggest pass through the following to the test even after its been explained. I was originally going to suggest a single colon. That certainly seemed inherently suggestive to me: prove -rb foo/ bar/ : http://localhost:2342 Now remember that a colon (single or double) can’t possibly be a file name on a large portion of the systems that Perl runs on. That would seem to me like an instant clue that something else is going on. There's no argument that :: or : or ++ or !#*?!#* says that something else is going on. Any odd control says that. The question the user will ask is what is that something else and how do you correctly use it? Because of this, :: is among a class of poorly designed controls you have to learn by rote. So how are you going to pass switch-like test file names to `prove`, which is the customary meaning of `--`? Are you going teach people they should learn a *different* sentinel that “acts like what `--` would act like if we weren’t using `--` for this other tool-specific functionality”? Is rote *un*learning better than rote learning? We might have gotten our wires crossed, because we're in violent agreement here. '--' is a poor choice for pass the remaining stuff through to the test files as well. '--' should mean treat the rest as filenames just like it normally does. -- Life is like a sewer - what you get out of it depends on what you put into it. - Tom Lehrer
Re: Providing command-line arguments to tests run via 'prove'
On 30 Nov 2007, at 01:32, Michael G Schwern wrote: The absurdity of :: becomes more clear because you can apply the same arguments to any switch of prove. Let's use ++ instead of '--color' because its syntactically clean and visually distinct. Or ;; for --merge. It's not a switch. It's the end of the switches. Short and clean isn't enough. A control has to suggest it's functionality. :: doesn't even suggest pass through the following to the test even after its been explained. Because of this, :: is among a class of poorly designed controls you have to learn by rote. For an example of this, just look through perlvar and think about how often you still have to refer back to it. Would you like your shell to use --pipe instead of '|'? '::' looks to me like a divider between distinct things - which is what it is. At this point I'm making my argument on repeat, so this will be the last time I say it. OK :) If you want to be able to flip flop in and out of test args and prove args modes just repeat it and say that tests can't use '::' as an arg. The flip-flop case is an unrealistic bit of complexity. Ignore it. Was supposed to be a joke making just that point :) -- Andy Armstrong, Hexten
Re: Providing command-line arguments to tests run via 'prove'
On Nov 29, 2007, at 04:20, Andy Armstrong wrote: [X] -- [ ] something else (please specify) Best, David
Re: Providing command-line arguments to tests run via 'prove'
On Nov 29, 2007, at 03:34, Ovid wrote: Could we not add a feature to prove and/or runtests such that, any arguments after a bare -- will be passed on to the scripts it runs? I've often wanted this myself, and --exec seems like overkill to me. Seconded. Just to clarify, --exec *is* overkill in this case, but when it's needed, it's really handy. Right, sorry, that's what I meant. Best, David
Re: Providing command-line arguments to tests run via 'prove'
--- James E Keenan [EMAIL PROTECTED] wrote:
I know in advance that some of my users call myperlprogram.pl with
options and some don't -- but I don't know what values they provide
for
those options. I would like my tests to be sufficiently flexible as
to
be able to handle the various use cases.
I would like to be able to provide the tests run via 'prove' with
options something like this:
some_variant_of_prove t/*.t --option1 --option2 arg1 arg2
... where those 4 command-line options/arguments would be available
to
*each* of the individual test files in the suite.
Sorry, I just saw this.
What you want is the --exec switch which, I'm embarassed to realize, is
not documented terribly well. Oops.
See examples/README to see how it works. Also, you'll want to
s/runtests/prove/ in that document.
Basically, if you want four args passed to everything, there's a
variety of ways to do that. One is to create an 'exec' program this
way:
#!/usr/bin/perl
use strict;
use warnings;
my $test = shift;
my @args = split ':' = $ENV{TEST_ARGS};
exec( $test, @args ) or die Cannot exec ($test, @args): $!;
If that's named 'ourtests' *and* made executable, you can run it like
this:
prove --exec ourtests -v t
Cheers,
Ovid
A given test file
would not necessarily have to respond to those options/arguments, but
if
it wanted to, it would do so simply by assigning the contents of the
test script's @ARGV to some other array.
Can this be done?
Thank you very much.
Jim Keenan
--
Buy the book - http://www.oreilly.com/catalog/perlhks/
Perl and CGI - http://users.easystreet.com/ovid/cgi_course/
Personal blog - http://publius-ovidius.livejournal.com/
Tech blog - http://use.perl.org/~Ovid/journal/
Re: Providing command-line arguments to tests run via 'prove'
On 28 Nov 2007, at 20:47, Andy Lester wrote: On Nov 28, 2007, at 2:44 PM, chromatic wrote: Could we not add a feature to prove and/or runtests such that, any arguments after a bare -- will be passed on to the scripts it runs? I've often wanted this myself, and --exec seems like overkill to me. Seconded. My only request would be that we not overload the meaning of --. I could easily see, though: prove -v t/ --testargs --db=test which would be less ambiguous than prove -v t/ -- --db=test Agreed. I think the '--' is redundant My only concern about any of this is that it opens the door for the I want to pass different arguments to different tests We could just say you can't :) Or rather here's the tail of @ARGV - make of it what you will. -- Andy Armstrong, Hexten
Re: Providing command-line arguments to tests run via 'prove'
On 28 Nov 2007, at 20:56, Andy Armstrong wrote: I could easily see, though: prove -v t/ --testargs --db=test which would be less ambiguous than prove -v t/ -- --db=test Agreed. I think the '--' is redundant Although, if we're talking about non-switches (sorry, I'm just catching up with this) then e.g. prove -v t/fetch.t http://example.com would attempt to run a test called http://example.com;. So you do need some, ideally shell safe, marker that tells prove where to stop. -- Andy Armstrong, Hexten
Re: Providing command-line arguments to tests run via 'prove'
On Wednesday 28 November 2007 12:47:59 Andy Lester wrote: My only request would be that we not overload the meaning of --. That ship has (GNU/getops) sailed. -- c
Re: Providing command-line arguments to tests run via 'prove'
On Wednesday 28 November 2007 12:23:16 David E. Wheeler wrote: Could we not add a feature to prove and/or runtests such that, any arguments after a bare -- will be passed on to the scripts it runs? I've often wanted this myself, and --exec seems like overkill to me. Seconded. -- c
Re: Providing command-line arguments to tests run via 'prove'
On Nov 28, 2007, at 2:44 PM, chromatic wrote: Could we not add a feature to prove and/or runtests such that, any arguments after a bare -- will be passed on to the scripts it runs? I've often wanted this myself, and --exec seems like overkill to me. Seconded. My only request would be that we not overload the meaning of --. I could easily see, though: prove -v t/ --testargs --db=test which would be less ambiguous than prove -v t/ -- --db=test My only concern about any of this is that it opens the door for the I want to pass different arguments to different tests -- Andy Lester = [EMAIL PROTECTED] = www.petdance.com = AIM:petdance
Re: Providing command-line arguments to tests run via 'prove'
On Nov 28, 2007, at 3:02 PM, Andy Armstrong wrote: prove -v t/fetch.t http://example.com would attempt to run a test called http://example.com;. So you do need some, ideally shell safe, marker that tells prove where to stop. Which is why I said --testargs prove -v t/fetch.t --testargs http://wakeupandya.com/ xoxo, Andy -- Andy Lester = [EMAIL PROTECTED] = www.petdance.com = AIM:petdance
Re: Providing command-line arguments to tests run via 'prove'
On 28 Nov 2007, at 21:06, Andy Lester wrote: would attempt to run a test called http://example.com;. So you do need some, ideally shell safe, marker that tells prove where to stop. Which is why I said --testargs prove -v t/fetch.t --testargs http://wakeupandya.com/ I thought --testargs was a metasyntactic name for a test argument :) Sorry folks. I'm being even more dense than usual tonight. It's a good idea and I'm +1 on implementing it once the dust settles on what form it should take. -- Andy Armstrong, Hexten
Re: Providing command-line arguments to tests run via 'prove'
On 28 Nov 2007, at 21:13, Andy Armstrong wrote: Sorry folks. I'm being even more dense than usual tonight. It's a good idea and I'm +1 on implementing it once the dust settles on what form it should take. And while I'm at it I think $ prove - should read TAP from STDIN, right? -- Andy Armstrong, Hexten
Re: Providing command-line arguments to tests run via 'prove'
Andy Armstrong writes: Could we not add a feature to prove and/or runtests such that, any arguments after a bare -- will be passed on to the scripts it runs? I've often wanted this myself, and --exec seems like overkill to me. Agreed. I think the '--' is redundant But is it doing harm? There are quite a few commands which use -- to mean 'end of options; what follows is data' (where data can (and often does mean) options to pass opaquely to another command being invoked). Smylers
Re: Providing command-line arguments to tests run via 'prove'
Andy Armstrong writes: On 28 Nov 2007, at 21:06, Andy Lester wrote: Which is why I said --testargs prove -v t/fetch.t --testargs http://wakeupandya.com/ I thought --testargs was a metasyntactic name for a test argument :) Sorry folks. I'm being even more dense than usual tonight. Well I did exactly the same until Andy's second post! Borrowing the established meaning of -- means it can be understood (by at least some people) straight away; making anything new up especially for the purpose means everybody has to learn whatever is chosen. Also, because -- looks different from other options, it works well as a divider, making it more obvious that _everything_ that follows is distinct from that which went before, rather than it just being like a 'normal' option, where what follows are also options. Smylers
Re: Providing command-line arguments to tests run via 'prove'
On 29 Nov 2007, at 00:27, Smylers wrote: Agreed. I think the '--' is redundant But is it doing harm? There are quite a few commands which use -- to mean 'end of options; what follows is data' (where data can (and often does mean) options to pass opaquely to another command being invoked). I'll shortly commit changes to Test::Harness that implement test options with '--' as the punctuation. If we can think of a sensible reason not to do that I'll change it :) -- Andy Armstrong, Hexten
Re: Providing command-line arguments to tests run via 'prove'
On Thu, Nov 29, 2007 at 12:47:39AM +, Andy Armstrong wrote: On 29 Nov 2007, at 00:27, Smylers wrote: Agreed. I think the '--' is redundant But is it doing harm? There are quite a few commands which use -- to mean 'end of options; what follows is data' (where data can (and often does mean) options to pass opaquely to another command being invoked). I'll shortly commit changes to Test::Harness that implement test options with '--' as the punctuation. If we can think of a sensible reason not to do that I'll change it :) The only reasons I can think of are that either we have a more important use for passing through options or we think that we might have in the future. Without breaking backwards compatibility, this is a unique resource. Gosh, that sounded dramatic! -- Paul Johnson - [EMAIL PROTECTED] http://www.pjcj.net
Re: Providing command-line arguments to tests run via 'prove'
On 29 Nov 2007, at 01:13, Paul Johnson wrote: The only reasons I can think of are that either we have a more important use for passing through options or we think that we might have in the future. Without breaking backwards compatibility, this is a unique resource. It is done :) http://hexten.net/tapx/r867/Test-Harness-3.04.tar.gz or svn co http://svn.hexten.net/tapx/trunk =head2 Arguments to Tests It is possible to supply arguments to tests. To do so separate them from prove's own arguments with '--'. For example prove -v t/mytest.t -- --url http://example.com would run Ft/mytest.t with the options '--url http://example.com'. When running multiple tests they will each receive the same arguments. -- Andy Armstrong, Hexten
Re: Providing command-line arguments to tests run via 'prove'
Andy Armstrong wrote: It is done :) http://hexten.net/tapx/r867/Test-Harness-3.04.tar.gz or svn co http://svn.hexten.net/tapx/trunk =head2 Arguments to Tests It is possible to supply arguments to tests. To do so separate them from prove's own arguments with '--'. For example prove -v t/mytest.t -- --url http://example.com would run Ft/mytest.t with the options '--url http://example.com'. When running multiple tests they will each receive the same arguments. Sorry, I came in late. There's nothing about '--' which indicates pass the rest through to the tests. As such, I think it's a poor choice. About all it has going for it is you don't have to quote things. In fact, it normally means stop processing the following as switches and instead treat them as files. By that reading I'd guess your command line above would mean run the test files t/mytest.t, --url and http://example.com;. In fact, if we use -- to mean something other than to stop parsing switches, how do we run files that look like switches? Why isn't this just: prove -v t/mytest.t --test_args='--url http://example.com' It's clear, it's unambiguous, it allows -- to mean what it's supposed to mean. -- Insulting our readers is part of our business model. http://somethingpositive.net/sp07122005.shtml
Re: Providing command-line arguments to tests run via 'prove'
On 29 Nov 2007, at 02:17, Michael G Schwern wrote: Why isn't this just: prove -v t/mytest.t --test_args='--url http://example.com' It's clear, it's unambiguous, it allows -- to mean what it's supposed to mean. I agree re the semantics of '--' - but I'd rather have a sentinel than a quoted string. Having to get nested quoting right is a bit of cognitive load we can spare people from. So: * Andy's --testargs has the disadvantage of being visually indistinct from regular args * '--' has another meaning by convention How about some other short, not arg like, shell safe punctuation sequence? $ prove -v t/sprocket.t -+ --teeth 12 -- Andy Armstrong, Hexten
Re: Providing command-line arguments to tests run via 'prove'
On 29 Nov 2007, at 02:38, Andy Armstrong wrote: How about some other short, not arg like, shell safe punctuation sequence? $ prove -v t/sprocket.t -+ --teeth 12 Actually I prefer $ prove -v t/sprocket.t ++ --teeth 12 :) -- Andy Armstrong, Hexten
Re: Providing command-line arguments to tests run via 'prove'
* Andy Armstrong [EMAIL PROTECTED] [2007-11-29 04:02]: I agree re the semantics of '--' - but I'd rather have a sentinel than a quoted string. Having to get nested quoting right is a bit of cognitive load we can spare people from. So: * Andy's --testargs has the disadvantage of being visually indistinct from regular args * '--' has another meaning by convention How about some other short, not arg like, shell safe punctuation sequence? $ prove -v t/sprocket.t -+ --teeth 12 How about a double colon? $ prove -v t/sprocket.t :: --teeth 12 It’s quite unlikely that anyone would name a test file `::`. In fact no one who has to care about Windows at all (which would be everyone who isn’t writing (Linux|Mac|BSD)::* modules) would want to do that because on Windows you *can’t* name a file `::`. (A single colon would do just as well, but might get visually lost in a long `prove` invocation.) Regards, -- Aristotle Pagaltzis // http://plasmasturm.org/
Re: Providing command-line arguments to tests run via 'prove'
On 29 Nov 2007, at 03:10, A. Pagaltzis wrote: How about a double colon? $ prove -v t/sprocket.t :: --teeth 12 It’s quite unlikely that anyone would name a test file `::`. In fact no one who has to care about Windows at all (which would be everyone who isn’t writing (Linux|Mac|BSD)::* modules) would want to do that because on Windows you *can’t* name a file `::`. (A single colon would do just as well, but might get visually lost in a long `prove` invocation.) Yeah, looks good to me. -- Andy Armstrong, Hexten
Re: Providing command-line arguments to tests run via 'prove'
Andy Armstrong wrote: On 29 Nov 2007, at 02:17, Michael G Schwern wrote: Why isn't this just: prove -v t/mytest.t --test_args='--url http://example.com' It's clear, it's unambiguous, it allows -- to mean what it's supposed to mean. I agree re the semantics of '--' - but I'd rather have a sentinel than a quoted string. Having to get nested quoting right is a bit of cognitive load we can spare people from. So: * Andy's --testargs has the disadvantage of being visually indistinct from regular args * '--' has another meaning by convention How about some other short, not arg like, shell safe punctuation sequence? $ prove -v t/sprocket.t -+ --teeth 12 As Ovid would say, you're captaining the SS Make Shit Up. ;) Quoting and escaping arguments is something you eventually have to learn how to do with any shell. I'd rather have to deal with a known bit of common shell lore then some unrelated bit of funny punctuation unique to prove. Put another way, shell quoting is an O(1) learning situation. Funny little punctuation unique to each command is O(n). Pragmatically, do we expect to have lots of quoting in arguments being passed to test files? There's the additional problem that it restricts the test arguments to only be allowed at the end of the prove command line. This means switch ordering is important, which will lead to problems programmaticly constructing prove commands or chaining switches together without knowing what all the previous switches are. -- 'All anyone gets in a mirror is themselves,' she said. 'But what you gets in a good gumbo is everything.' -- Witches Abroad by Terry Prachett
Re: Providing command-line arguments to tests run via 'prove'
As much as I like the non-quotedness of the -- or --testargs idea, I really think it needs to be --testargs='--foo --bar'. I realize 3.04 is out there right now, but I think that it's not too late to change. xoxo, Andy -- Andy Lester = [EMAIL PROTECTED] = www.petdance.com = AIM:petdance
Re: Providing command-line arguments to tests run via 'prove'
Eric Wilhelm wrote: # from Michael G Schwern # on Wednesday 28 November 2007 20:11: There's the additional problem that it restricts the test arguments to only be allowed at the end of the prove command line. This means switch ordering is important, which will lead to problems programmaticly constructing prove commands or chaining switches together without knowing what all the previous switches are. Indeed. Though one could argue that such conundrums are out-of-scope and it should just be a '--testargs' sentinel (possibly up to a --, though the tests may need that in a literal, so it's got another sentinel: --end-testargs, right? (Yeah, requires a trick to get Getopt to play along.) Making it a quoted string won't work because we're using a list context, so we would have to deparse quotes inside that to preserve spaces. Backslash the spaces? Oh, that's important. No, we really, really don't want to get into multiple levels of quoting like that and we don't want to be attempting to manually split the string and try to parse out shell quoting. In that case, then named sentinels would make the most sense. How do other utilities handle this sort of thing? -- You know what the chain of command is? It's the chain I go get and beat you with 'til you understand who's in ruttin' command here. -- Jayne Cobb, Firefly
Re: Providing command-line arguments to tests run via 'prove'
On Sat, Nov 24, 2007 at 10:05:26PM -0500, James E Keenan wrote: I would like to be able to provide the tests run via 'prove' with options something like this: some_variant_of_prove t/*.t --option1 --option2 arg1 arg2 I want to do that too! Can this be done? I guess one possibility would be to use TAP::Parser with the 'exec' parameter passed to new: my $parser = TAP::Parser-new( exec = [ $^X, $script, @options ] ); Not too practical, though. -- Philippe Bruhat (BooK) The shortest distance between two points is not always the safest. (Moral from Groo The Wanderer #69 (Epic))
Providing command-line arguments to tests run via 'prove'
Let's suppose that I have a suite of test files which I customarily run with 'prove': prove t/*.t ... where the tests in t/ are: alpha.t beta.t gamma.t Let's further suppose that each of these three tests simulates the operation of a Perl program which is normally called with command-line arguments or options: perl myperlprogram.pl --option1 --option2 arg1 arg2 I know in advance that some of my users call myperlprogram.pl with options and some don't -- but I don't know what values they provide for those options. I would like my tests to be sufficiently flexible as to be able to handle the various use cases. I would like to be able to provide the tests run via 'prove' with options something like this: some_variant_of_prove t/*.t --option1 --option2 arg1 arg2 ... where those 4 command-line options/arguments would be available to *each* of the individual test files in the suite. A given test file would not necessarily have to respond to those options/arguments, but if it wanted to, it would do so simply by assigning the contents of the test script's @ARGV to some other array. Can this be done? Thank you very much. Jim Keenan
