Re: stdio
Nothing makes you re-think your reply length like having your mailer
lose your message ;-)
A lot of your message revolves around this idea that there's a "normal
file open" semantic. What I've tried (but clearly failed) to articulate
previously is that this notion is becoming archaic in what is shaping up
to be the post-POSIX world. Gnome VFS and the .NET Framework are good
examples of this.
Now, I don't think that the state of the world is yet such that we can
say that Perl should embrace the Gnome VFS or .NET Framework or whatever
OS/X does or any other meta-file-framework on people, but I do think
that we can safely expect Perl 6 to have to deal with these concepts and
would be well served by building in a standard way to add your More Than
One Way later on through CPAN.
On Fri, 2005-05-06 at 15:10, Larry Wall wrote:
> On Fri, May 06, 2005 at 08:19:05AM -0400, Aaron Sherman wrote:
> : "open" as a verb is extremely ambiguous. In dictionary searches I see as
> : many as 19 definitions just for the verb form.
>
> Well, sure, but also need to take Perl history into account, where dwimmy
> open is considered something of a liability.
I've come across many folks (and I'm one of them) who a) like the "-"
magic a lot and b) are in the majority in my experience. I definitely
see the concern around adverbial bits showing up in the text (e.g. ">"
and "<"), but not magical filenames like "-". Three-argument open is a
godsend, but I'd love to preserve the bits that were useful in Perl 6.
> I think the dwimminess of open() probably arises only from MMD, and
> a string or array of string in the first argument implies ordinary
> file open. That means perhaps we have
>
> open uri($x)
Ok, I had a long reply to this, but I'll sum up:
* open uri($x) implies that I know that $x is a URI.
* Being able to "use IO::URI :stropen" makes an otherwise
cumbersome chore into a breeze, e.g.:
use IO::URI :stropen;
use IO::SomethingElse :stropen;
GetOptions('f|file' => \$input_file);
my $input = open($input_file, :r);
* I don't think any of this should be on by default, with the
possible exception of "-", but that's only possible.
> :
> : sub File::Copy::copy(IO $file1 :r, IO $file2 :w) {...}
I think you took this as a request for long, "::"-separated names in
everyday code. I was just being verbose for one-line clarity, not
because I think subdefs should look like that in real code. To be more
specific:
module File::Copy is export {
sub copy(IO $file1 :r, IO $file2 :w) {...}
}
And I was asking if that would pass the adverb down to the constructor
for IO like so:
$file1 = IO.new($param, :r)
and if not, how one can do that.
> Those are all pretty bletcherous. How 'bout
>
> io('-') ==> io('-');
In Perl 5, File::Copy::copy is not a pipelineable (or potentially lazy)
operation because it can be implemented by OS-level special-purpose
functions. I think that overloading infix:==>(IO $a,IO $b) to behave
this way would be potentially very misleading to the programmer, and
thus should probably be left alone.
Should file copying be a core function or a module as it was in Perl 5?
I don't know, but either way I think it needs to continue to make OS
specific copying available for the six or seven platforms that Perl 5
currently knows about.
[...example code...]
> : I would need some error handling here, and possibly would need to defer
> : to a parent as a fallback.
Sure, that all makes sense. I was white-boarding, and haven't even yet
solved the problem of scoping the change.
> : That brings up the idea of delegation... should this be handled by
> : delegation instead of the way I've done it? Not sure. I'm still trying
> : to figure out how to make this scope correctly so that:
Delegation might work very well here. There was a nice use for it that I
had in mind for IO::Pipe as well. I have to re-read the array delegation
rules to see how that behaves, but thanks for reminding me of it.
--
Aaron Sherman <[EMAIL PROTECTED]>
Senior Systems Engineer and Toolsmith
"It's the sound of a satellite saying, 'get me down!'" -Shriekback
Re: stdio
On Fri, May 06, 2005 at 08:19:05AM -0400, Aaron Sherman wrote:
: "open" as a verb is extremely ambiguous. In dictionary searches I see as
: many as 19 definitions just for the verb form.
Well, sure, but also need to take Perl history into account, where dwimmy
open is considered something of a liability.
I think the dwimminess of open() probably arises only from MMD, and
a string or array of string in the first argument implies ordinary
file open. That means perhaps we have
open uri($x)
to type $x to a \w+: handled string, so MMD can see it. When means
you can pass a uri object to a generic open just fine, as long as your
program generates the type. But random strings wouldn't accidentally
trigger uri processing. Alternately,
open URI: $x
will single dispatch to the URI class.
: "open" as a POSIX function is not ambiguous at all. But will the POSIX
: model be the model on which P6 is based (as P5 was), or will the POSIX
: model simply be one of many models supported by open and other
: built-ins?
One of many.
: I think that it's fair to say that Perl has grown beyond POSIX. If it
: also presents the URI model, that's not all bad, is it?
As long as it's not the accidental default.
: Same goes for the Unix command-line conventions (e.g. cat's "-" special
: filename). I should be able to request a "pure POSIX" open, but I'm not
: sure it should be the default for one of English's most powerfully
: flexible verbs.
io() is probably the place for complete dwimmery.
: > and perhaps even
: >
: > File::Copy::copy("-","-");
:
: This brings up something interesting. If we have:
:
: sub File::Copy::copy(IO $file1, IO $file2) {...}
:
: the above doesn't work.
:
: I think I want:
:
: sub File::Copy::copy(IO $file1 :r, IO $file2 :w) {...}
:
: But does that work the way I think it does? Does that end up calling:
:
: IO.new("-", :r);
:
: and
:
: IO.new("-", :w);
:
: ? If not, how do I ensure that this works correctly?
Those are all pretty bletcherous. How 'bout
io('-') ==> io('-');
: > Sure enough, there's an easy way:
: >
: > class IO is ParrotIO does RegisteredStringConstructor {...}
: > role RegisteredStringConstructor {
: > has %:registry;
: > sub register(Any $pattern, Type $type) {
: > %:registry{$pattern} = $type;
: > }
: > multi method new(Str $string,*%rest) {
: > for %:registry.keys -> $pat {
: > if $string ~~ $pat {
: > return
%:registry{$pat}.bless(:string($string),*%rest);
: > }
: > }
:
: I would need some error handling here, and possibly would need to defer
: to a parent as a fallback.
:
: That brings up the idea of delegation... should this be handled by
: delegation instead of the way I've done it? Not sure. I'm still trying
: to figure out how to make this scope correctly so that:
:
: use IO;
: {
: use IO::URI :open;
: open("http://www.perl.org/",:r);
: }
: open("http://www.perl.org",:r);
:
: opens two very different things: a socket to a host and port as directed
: by a URI vs a file named "www.perl.org" in a directory called "http:".
my &open ::= &open:(URI);
or maybe some kind of currying. Multiple handlers would need some way
of ordering them though. Seems like I put something into the delegation
model to deal with that already, like
has @:iolist handles ;
: > Optional export features of IO::* could allow:
: >
: > * pipeline command execution
: > * thread and/or process coupling
: > * handle duping
: > * much more
: >
: > Thus, you would control these features like so:
: >
: > use IO;
: > use IO::Funky :register_string_open_funkiness;
: > open("funk",:w);
:
: Which is probably just the tip of the iceberg. You might, for example,
: want to lay out a user-defined filesystem, or open database tables as
: files, etc.
I think lumping everything under the IO rubric is perhaps a mistake, unless
we also give shortcuts for all the common cases, in which case maybe we
should go for a flatter hierarchy to begin with, and leave the deep names
for CPAN modules.
: Of course, you can do all of this explicitly through OO syntax, but it
: would be nice to uniformly export locator semantics as strings so that
: command-line and other string-based user interaction could be made more
: powerful with a single "use".
Just so easy things stay easy. Foo::Bar::Baz::meth names are not
construed by everyone as easy to use.
Larry
Re: stdio
On Thu, 2005-05-05 at 15:15, Aaron Sherman wrote:
> Dash this all on the rocks if you want, but understand that this is not
> an off-the-cuff reply, but something that I've spent a lot of time
> mulling over
[...]
> First off, IMHO, open should be an alias for a closure-wrapped
> constructor, like so:
>
> sub open := IO.new;
Some reading later, I see mixed references to &, but I thought & had
been re-glommed for other purposes yep, pugs confirms & works.
> This makes "open" the maximally
> dwimish operator. You can always introduce a "sysopen" and "stdiopen"
> and "sockopen", etc. if you want domain-specific behavior exclusively
> (e.g. treat everything as a filename only, or as a host:port only, etc).
This deserves a bit more detail than I gave it before (pardon, I'm
typing as I debug work code).
"open" as a verb is extremely ambiguous. In dictionary searches I see as
many as 19 definitions just for the verb form.
"open" as a POSIX function is not ambiguous at all. But will the POSIX
model be the model on which P6 is based (as P5 was), or will the POSIX
model simply be one of many models supported by open and other
built-ins?
I think that it's fair to say that Perl has grown beyond POSIX. If it
also presents the URI model, that's not all bad, is it?
Same goes for the Unix command-line conventions (e.g. cat's "-" special
filename). I should be able to request a "pure POSIX" open, but I'm not
sure it should be the default for one of English's most powerfully
flexible verbs.
> and perhaps even
>
> File::Copy::copy("-","-");
This brings up something interesting. If we have:
sub File::Copy::copy(IO $file1, IO $file2) {...}
the above doesn't work.
I think I want:
sub File::Copy::copy(IO $file1 :r, IO $file2 :w) {...}
But does that work the way I think it does? Does that end up calling:
IO.new("-", :r);
and
IO.new("-", :w);
? If not, how do I ensure that this works correctly?
> Sure enough, there's an easy way:
>
> class IO is ParrotIO does RegisteredStringConstructor {...}
> role RegisteredStringConstructor {
> has %:registry;
> sub register(Any $pattern, Type $type) {
> %:registry{$pattern} = $type;
> }
> multi method new(Str $string,*%rest) {
> for %:registry.keys -> $pat {
> if $string ~~ $pat {
> return
> %:registry{$pat}.bless(:string($string),*%rest);
> }
> }
I would need some error handling here, and possibly would need to defer
to a parent as a fallback.
That brings up the idea of delegation... should this be handled by
delegation instead of the way I've done it? Not sure. I'm still trying
to figure out how to make this scope correctly so that:
use IO;
{
use IO::URI :open;
open("http://www.perl.org/",:r);
}
open("http://www.perl.org",:r);
opens two very different things: a socket to a host and port as directed
by a URI vs a file named "www.perl.org" in a directory called "http:".
> Optional export features of IO::* could allow:
>
> * pipeline command execution
> * thread and/or process coupling
> * handle duping
> * much more
>
> Thus, you would control these features like so:
>
> use IO;
> use IO::Funky :register_string_open_funkiness;
> open("funk",:w);
Which is probably just the tip of the iceberg. You might, for example,
want to lay out a user-defined filesystem, or open database tables as
files, etc.
Of course, you can do all of this explicitly through OO syntax, but it
would be nice to uniformly export locator semantics as strings so that
command-line and other string-based user interaction could be made more
powerful with a single "use".
--
Aaron Sherman <[EMAIL PROTECTED]>
Senior Systems Engineer and Toolsmith
"It's the sound of a satellite saying, 'get me down!'" -Shriekback
Re: stdio
On Thu, 2005-05-05 at 12:31, Larry Wall wrote:
> On Thu, May 05, 2005 at 10:14:34AM +0300, Gaal Yahas wrote:
> : On Wed, May 04, 2005 at 11:44:58PM -0700, Larry Wall wrote:
> : > : How do I open a file named "-"?
[...]
> : > $fh = io("-");
> : > $fh = open "-";
> : My concern is again with magic control. I've no gripes with the first
> : or last of those, but I think the second should not be allowed by
> : default.
> I think that, as with various other parts of Perl 6, we can try
> to sweep all the dwimmery into one spot so that it can be easily
> recognized and/or avoided. And the default "open" is not the place
> for dwimmery.
I've been thinking about this particular thing ever since I wrote
File::Copy, and wanted it to accept filenames, magical file-like strings
or filehandles. Of course, that problem is easy in P6, but as I've been
thinking about my dream IO type system in Perl for so long, I have a few
opinions on it ;-)
Dash this all on the rocks if you want, but understand that this is not
an off-the-cuff reply, but something that I've spent a lot of time
mulling over, and I really think Perl 6 programmers everywhere would
enjoy such functionality. I'll even go so far as to offer to write it
once Pugs is in a state that allows for such.
First off, IMHO, open should be an alias for a closure-wrapped
constructor, like so:
sub open := IO.new;
Pardon me if sub isn't the right tool there, I'm really not sure. The
point there is to make open have the same signature(s) as the
constructor(s) for IO. This allows you to instantly introduce every
variant of open you'll ever want. This makes "open" the maximally
dwimish operator. You can always introduce a "sysopen" and "stdiopen"
and "sockopen", etc. if you want domain-specific behavior exclusively
(e.g. treat everything as a filename only, or as a host:port only, etc).
Next, we need IO to be capable of understanding not only polymorphism:
sub File::Copy::copy(IO $file1, IO $file2) {...}
File::Copy::copy(IO::File.new($path1,:mode),
IO::File.new($path2,:mode));
but we also wish to be able to say:
File::Copy::copy(IO.new("-"),IO.new("-"));
and perhaps even
File::Copy::copy("-","-");
and have the right thing happen (right thing not always being a
universal for every program). How can we do this? Well, first IO has to
accept a string for the constructor:
class IO is ParrotIO { # ParrotIO? not sure ...
method new(Str $string) {...}
}
I will assume that we upgrade types based on constructor signatures by
constructing a temporary, though I honestly don't recall that being said
explicitly in A12.
That's all well and good, but how does IO know about its children? They
might not even be loaded yet. You could just assume that any string
constructor is either an IO::Handle (if it's "-"), an IO::Pipe (if it's
"|"), or an IO::File otherwise, but that's a bit restrictive. What if we
decide we want URIs to be readable things? It would be nice to be able
to mix that in.
Sure enough, there's an easy way:
class IO is ParrotIO does RegisteredStringConstructor {...}
role RegisteredStringConstructor {
has %:registry;
sub register(Any $pattern, Type $type) {
%:registry{$pattern} = $type;
}
multi method new(Str $string,*%rest) {
for %:registry.keys -> $pat {
if $string ~~ $pat {
return
%:registry{$pat}.bless(:string($string),*%rest);
}
}
}
}
Now, IO::URI can say:
class IO::URI is IO {
use URI;
IO.register(rx{^ :}, ::IO::URI);
...
}
and then you can:
my $fh = open("gnomevfs:spreadsheet:/home/me/sheet.gnumeric",:r);
while =$fh -> $row {
say $row[0];
}
Of course, my example URI brings up a host of questions, and probably is
incorrect gnomevfs syntax, but you get the idea.
Optional export features of IO::* could allow:
* pipeline command execution
* thread and/or process coupling
* handle duping
* much more
Thus, you would control these features like so:
use IO;
use IO::Funky :register_string_open_funkiness;
open("funk",:w);
Now, if someone can just figure out how to re-write the string
registration so that it "behaves itself" (that is, unregisters when the
use goes out of scope), I think it would be perfect.
--
Aaron Sherman <[EMAIL PROTECTED]>
Senior Systems Engineer and Toolsmith
"It's the sound of a satellite saying, 'get me down!'" -Shriekback
Re: stdio
On Thu, May 05, 2005 at 10:14:34AM +0300, Gaal Yahas wrote:
: On Wed, May 04, 2005 at 11:44:58PM -0700, Larry Wall wrote:
: > : How do I open a file named "-"?
: >
: > Um, depending on what you mean, and whether we continue to support
: > the '=' pseudofile, maybe:
: >
: > $fh = io("-");
: > $fh = open "-";
: > $fh = $name eq '-' ?? $*IN :: open $name;
:
: My concern is again with magic control. I've no gripes with the first
: or last of those, but I think the second should not be allowed by
: default. There has to be a safe mode for opening a file and knowing
: that's what you're opening: not a pipe, not stdio (hence there are places
: I can't permit myself to use #1). But as your third example suggests,
: never allowing open "-" will make unixish tools tedious to write, so
: maybe we need something like
:
: getopt(...);
: $fh = open $in, :allowstdio;
:
: (only named more eloquently).
I think that, as with various other parts of Perl 6, we can try
to sweep all the dwimmery into one spot so that it can be easily
recognized and/or avoided. And the default "open" is not the place
for dwimmery. It should just open an ordinary file by default.
The place for all that dwimmery is probably io(). In contrast, open
should be a multisub/method that gives you exact semantics depending
on the invocant. In fact, io() might just be short for IO.open(),
which specifically requests dwimmery, just as URI.open() specifically
requests URI interpretation. And bare open() is probably short for
File.open(). (Though for io(), it's probable that it actually does the
open lazily depending on usage, since that's part of its dwimmery.)
Larry
Re: stdio
On 5/5/05, Gaal Yahas <[EMAIL PROTECTED]> wrote: > On Thu, May 05, 2005 at 01:32:56AM -0600, Luke Palmer wrote: > > And I don't think arguing in the name of "security" for the default > > case is going to buy us anything. Security doesn't come in scripts in > > any language for free; you have to walk through every line that sees > > the outside world and ask "is there any way somebody could exploit > > this?". And a "-" handler would be one of the things you'd have to > > routinely write, just like making sure they're not opening "; rm -rf > > /". > > Why are you scare-quoting something I never said? I'm scare-quoting everything today for some reason. Sorry. The issue has come up before, and one of the main arguments was "it's more secure that way", which made so little sense, I thought that I'd argue against it again. Or something. Luke
Re: stdio
On Thu, May 05, 2005 at 01:32:56AM -0600, Luke Palmer wrote: > On 5/5/05, Gaal Yahas <[EMAIL PROTECTED]> wrote: > > getopt(...); > > $fh = open $in, :allowstdio; > > Maybe the opposite: > > $fh = open $in, :literal; > > One of the nice things about the magical "-" behavior is that people > are writing more versatile, accepting scripts without realizing it. > That was one of the things that made me really like Perl when I first > started learning it. A few of the little utility / filter scripts > that I wrote already accepted "-" on the command line, and I didn't > even know it (and they still worked perfectly when you used "-"). Yeah, you and me both. > And I don't think arguing in the name of "security" for the default > case is going to buy us anything. Security doesn't come in scripts in > any language for free; you have to walk through every line that sees > the outside world and ask "is there any way somebody could exploit > this?". And a "-" handler would be one of the things you'd have to > routinely write, just like making sure they're not opening "; rm -rf > /". Why are you scare-quoting something I never said? I wasn't talking about security, I was talking about basic least-surprise. Opening "; rm -rf /" with my perl5 does not do anything bad. Opening "-" potentially causes a script to hang. > But I don't think a :literal flag or whatever will be a problem. Yes, I like your proposal, though I don't know which way should be the default. I'm not looking for sysopen (I know where to find it), I'm looking for an easy way to control magic. > You can also open a file named "-", in the absence of a literal > option, like this: > > my $fh = open "./-"; I'd say "fine", except that this isn't portable. > I think he misunderstood you (and if not, then I misunderstood you > :-). You're asking about how to get a filehandle to stdout, he's > telling you how to redirect stdout to a file. > > I think "-" will do the trick. Ah, yes, then again the question is how to conveniently choose whether to do e.g. log-to-stdout or write to a file named "-". -- Gaal Yahas <[EMAIL PROTECTED]> http://gaal.livejournal.com/
Re: stdio
On Wed, May 04, 2005 at 11:44:58PM -0700, Larry Wall wrote:
> : How do I open a file named "-"?
>
> Um, depending on what you mean, and whether we continue to support
> the '=' pseudofile, maybe:
>
> $fh = io("-");
> $fh = open "-";
> $fh = $name eq '-' ?? $*IN :: open $name;
My concern is again with magic control. I've no gripes with the first
or last of those, but I think the second should not be allowed by
default. There has to be a safe mode for opening a file and knowing
that's what you're opening: not a pipe, not stdio (hence there are places
I can't permit myself to use #1). But as your third example suggests,
never allowing open "-" will make unixish tools tedious to write, so
maybe we need something like
getopt(...);
$fh = open $in, :allowstdio;
(only named more eloquently).
> : How do I open stdout (and the other standard handles)?
>
> Maybe something like:
>
> $fh = open "file", :w:c:usefd(1);
> $*OUT.reopen("file",:w:c);
> reopen $*OUT: "file", :w:c;
>
> give or take a few options.
What does '"file"' do in these examples? If we take your trailing-args
suggestion from the Open and pipe thread, the filename could be optional.
How to make this "-"-friendly? If my earlier suggestion is okay, then
by symmetry:
$logfile = "-";
open $log, :a:allowstdio;
--
Gaal Yahas <[EMAIL PROTECTED]>
http://gaal.livejournal.com/
Re: stdio
On Thu, May 05, 2005 at 08:39:52AM +0300, Gaal Yahas wrote:
: How do I open a file named "-"?
Um, depending on what you mean, and whether we continue to support
the '=' pseudofile, maybe:
$fh = io("-");
or
$fh = open "-";
or
$fh = $name eq '-' ?? $*IN :: open $name;
: How do I open stdout (and the other standard handles)?
Maybe something like:
$fh = open "file", :w:c:usefd(1);
or
$*OUT.reopen("file",:w:c);
or
reopen $*OUT: "file", :w:c;
give or take a few options.
Larry
stdio
How do I open a file named "-"? How do I open stdout (and the other standard handles)? -- Gaal Yahas <[EMAIL PROTECTED]> http://gaal.livejournal.com/
