Re: [9fans] Making read(1) an rc(1) builtin?
In article 20110406163229.6a159b...@mail.bitblocks.com,
Bakul Shah ba...@bitblocks.com wrote:
On Tue, 05 Apr 2011 15:53:43 MDT andrey mirtchovski mirtchov...@gmail.com
wrote:
so, an optimized /sys/src/cmd/read.c that doesn't read char-by-char
should give us an improvement, right? right:
9grid% newaread
1.52u 22.56s 15.66r newaread
and that's just for the silly test string. the improvement would be
bigger for longer strings.
read(1) is allowed to read one line and no more. Given a read
of 1 char, a console device will return a line at most but
other devices can return more than one line, thereby
preventing the next guy from reading it. read(1) has to read
one char at a time.
With a builtin read you don't pay the cost of a fork/exec per
char. And it would be less clunky -- instead of foo=`{read}
you can say read foo. It is not like rc going to get fat by
adding read; it already has to read!
It depends upon what you're doing whether it is worth it or not though
(not saying you said that, just adding to the conversation).
CCsh, my Bourne shell compiler, does make some of the things mentioned
in this discussion just become non-issues (whatever that means) since
they are builtin (and I don't have any numbers handy), however,
I'll wager many using it for that purpose are delving into what
I'll call false and/or premature opimizations. So, not that it won't
make the script faster, but whether in the long run it's a practical
compromise. Last comment is that read is a built-in in many UNIX
shells, so of course this is not an exact comparison, but still
in the end need to get to the practical compromise issue being
worthy or not. I won't answer that in the case above though :)
--
Greg Comeau / 4.3.10.1 with C++0xisms now in beta!
Comeau C/C++ ONLINE == http://www.comeaucomputing.com/tryitout
World Class Compilers: Breathtaking C++, Amazing C99, Fabulous C90.
Comeau C/C++ with Dinkumware's Libraries... Have you tried it?
Re: [9fans] Making read(1) an rc(1) builtin?
In article BANLkTinX_0V3+ERKLOsM1OsD=0ayf7q...@mail.gmail.com, Yaroslav yari...@gmail.com wrote: finally, there are /bin/ape/sh which is a pdksh with its features. And in our opinion it is nicely done in terms of supporting ksh'y things (at least it's taken some large and wacked out stuff we've thrown at it like a champ), but I'll guess probably one needs to look also at its footprint vs (or in addition to) rsh's, if this read issue and similar is of such concern to OP. -- Greg Comeau / 4.3.10.1 with C++0xisms now in beta! Comeau C/C++ ONLINE == http://www.comeaucomputing.com/tryitout World Class Compilers: Breathtaking C++, Amazing C99, Fabulous C90. Comeau C/C++ with Dinkumware's Libraries... Have you tried it?
Re: [9fans] Making read(1) an rc(1) builtin?
Yaroslav yari...@gmail.com writes: than appropriate for a line-oriented language like rc(1). As it stands, rc(1) can have it's cake, but it can't eat it without a fork(2). As it was stated earlier, a fork(2) is rather cheap here, so what's your concern then? Maybe it's time to forget the lectures about expensive forks and just cary on? That was supposed to be a joke. Apparently, no body got (or read) it. :( I bet you won't even notice much changes in your stats -c graph. Just moving my mouse causes change in my stats -cs graph, driving both measurements up to 75% or so. When the system is at all loaded, the mouse doesn't respond at all. (Which makes it kind of hard to use the system, given that it's mouse-based!) I wish there were a way to renice usb/kb... -- +---+ |E-Mail: smi...@zenzebra.mv.com PGP key ID: BC549F8B| |Fingerprint: 9329 DB4A 30F5 6EDA D2BA 3489 DAB7 555A BC54 9F8B| +---+
Re: [9fans] Making read(1) an rc(1) builtin?
On Tue, 05 Apr 2011 15:53:43 MDT andrey mirtchovski mirtchov...@gmail.com
wrote:
so, an optimized /sys/src/cmd/read.c that doesn't read char-by-char
should give us an improvement, right? right:
9grid% newaread
1.52u 22.56s 15.66rnewaread
and that's just for the silly test string. the improvement would be
bigger for longer strings.
read(1) is allowed to read one line and no more. Given a read
of 1 char, a console device will return a line at most but
other devices can return more than one line, thereby
preventing the next guy from reading it. read(1) has to read
one char at a time.
With a builtin read you don't pay the cost of a fork/exec per
char. And it would be less clunky -- instead of foo=`{read}
you can say read foo. It is not like rc going to get fat by
adding read; it already has to read!
Re: [9fans] Making read(1) an rc(1) builtin?
On 6 April 2011 17:32, Bakul Shah ba...@bitblocks.com wrote:
On Tue, 05 Apr 2011 15:53:43 MDT andrey mirtchovski mirtchov...@gmail.com
wrote:
so, an optimized /sys/src/cmd/read.c that doesn't read char-by-char
should give us an improvement, right? right:
9grid% newaread
1.52u 22.56s 15.66r newaread
and that's just for the silly test string. the improvement would be
bigger for longer strings.
read(1) is allowed to read one line and no more. Given a read
of 1 char, a console device will return a line at most but
other devices can return more than one line, thereby
preventing the next guy from reading it. read(1) has to read
one char at a time.
With a builtin read you don't pay the cost of a fork/exec per
char. And it would be less clunky -- instead of foo=`{read}
you can say read foo. It is not like rc going to get fat by
adding read; it already has to read!
a builtin read would still have to read one character at a time though.
Re: [9fans] Making read(1) an rc(1) builtin?
With a builtin read you don't pay the cost of a fork/exec per
char. And it would be less clunky -- instead of foo=`{read}
you can say read foo. It is not like rc going to get fat by
adding read; it already has to read!
i'd argue that rc shouldn't have any builtins.
and i don't see how that's an improvement to
rc's syntax. all other assignments involve '=',
and can result in command-local variables.
- erik
Re: [9fans] Making read(1) an rc(1) builtin?
a builtin read would still have to read one character at a time though. it's interesting that read breaks with plan 9 message-preserving tradition. - erik
Re: [9fans] Making read(1) an rc(1) builtin?
finally, there are /bin/ape/sh which is a pdksh with its features.
Re: [9fans] Making read(1) an rc(1) builtin?
2011/4/4 smi...@zenzebra.mv.com: Unfortunately, echon.c doesn't solve the problem either, because it doesn't output a trailing newline. The crux of the problem is how to output -n on a line by itself, followed by a newline. I don't think it can be done symmetrically without adding another option to echo. This has been discussed before: http://9fans.net/archive/2008/03/491 Rob Pike gave an excellent answer to your question: echo -n -n' ' Lyndon's echon.c and echo.c are another solution, as many others which have been discussed here and in the 2008 thread. I'm really impressed of how much we can talk about such a simple program as echo. -- - yiyus || JGL .
Re: [9fans] Making read(1) an rc(1) builtin?
On 04/05/2011 03:19 AM, Anthony Sorace wrote: On Apr 4, 2011, at 17:35, smi...@zenzebra.mv.com wrote: All combined (forking read/test/echo, forking awk/sed/dd, parsing /mnt/acme/%d/events, etc.)... this, I think, is why languages like Perl came into existence and became so popular. I could definitely write an Acme event parser in Perl, or even in bash(1). rc(1) is just a few small features shy of making it practical to do in rc(1). I think this does a very good job of summing up the issue. I think the point you might be missing, though, is that most of the Plan 9 community is quite happy about the current state of things. You're likely right that considerations like this led to perl and bash - but rc's state is not an accident. We know where to get perl and bash if we want them. [bsd@werc ~] man bash |tail -24 BUGS *It's too big and too slow*. There are some subtle differences between bash and traditional versions of sh, mostly because of the POSIX specification. Aliases are confusing in some uses. Shell builtin commands and functions are not stoppable/restartable. Compound commands and command sequences of the form `a ; b ; c' are not handled gracefully when process suspension is attempted. When a process is stopped, the shell immediately executes the next command in the sequence. It suffices to place the sequence of commands between parentheses to force it into a subshell, which may be stopped as a unit. Array variables may not (yet) be exported. There may be only one active coprocess at a time. GNU Bash-4.2 2010 December 28 BASH(1) BTW, that highlighting, above, is mine ;) Put another way, your problem seems to be I can't write an acme client in rc with performance I'm happy with (leaving aside, for the moment, questions of measurement). Your solution is expand rc; I suspect the consensus of the community would be either deal with it or use C. Actually, that might be the consensus of the community on *most* issues. :-) -- Balwinder S bdheeman DheemanRegistered Linux User: #229709 Anu'z Linux@HOME (Unix Shoppe)Machines: #168573, 170593, 259192 Chandigarh, UT, 160062, India Plan9, T2, Arch/Debian/FreeBSD/XP Home: http://werc.homelinux.net/ Visit: http://counter.li.org/
Re: [9fans] Making read(1) an rc(1) builtin?
On 4 April 2011 22:35, smi...@zenzebra.mv.com wrote:
roger peppe rogpe...@gmail.com writes:
when i've needed a -n safe version of echo in
the past, i've used something like this:
fn myecho {echo -n $* ^ '
'}
That doesn't work right when (~ $#* 0). It outputs a rogue space prior
to the newline. echo, with no arguments, should ouput just a newline.
it works fine, with no rogue spaces. did you try it?
I'm trying to write an Acme client in rc(1). I'd like to avoid spawning
a new read(1) process every time I make a keystroke or click the mouse.
Using multi-line reads wouldn't help much, because interactivity needs
to be maintained.
I'm using rc(1) because the /mnt/acme/%d/events interface is
well-documented (in /sys/doc/acme/acme.ps), but the C code under
/acme/bin/source/ for reading /mnt/acme/%d/events it is definitively
cryptic. I've managed to peel away the extra layers of code from one of
the simpler Acme clients, in /acme/bin/source/adict/win.c, and am in the
process of creating a general-purpose Acme event parser in C. The
output of the filter will be in a form easily digestible by scripts, and
would provide a good skeleton example of event parsing for other
coders to build upon. (There doesn't currently appear to be any such
starter code under /acme/bin/source or /sys/doc.)
If only Acme put a single extra space immediately prior the first
integer (c0) in it's event messages, this parsing could have been done
almost entirely within rc(1).
the usual solution to problems like this is to write a little
helper program in C. you might want to look at
http://man.cat-v.org/p9p/1/acmeevent
Re: [9fans] Making read(1) an rc(1) builtin?
This discussion is interesting. rc is not as good a shell as bash because it lacks built-ins that make it a good programming language for writing an acme extension Did I summarizer it correctly? Once summarized, am I the only one who finds it absurd? ron
Re: [9fans] Making read(1) an rc(1) builtin?
This whole discussion has been absurd.
to quote fgb, relax. or to quote dek, \relax{}
- erik
Re: [9fans] Making read(1) an rc(1) builtin?
On 5 April 2011 17:52, Jacob Todd jaketodd...@gmail.com wrote: This whole discussion has been absurd. On Apr 5, 2011 11:50 AM, ron minnich rminn...@gmail.com wrote: This discussion is interesting. rc is not as good a shell as bash because it lacks built-ins that make it a good programming language for writing an acme extension Did I summarizer it correctly? Once summarized, am I the only one who finds it absurd? ron No. The discussion was not absurd. And no, Ron's summary was not a summary. Ruda
Re: [9fans] Making read(1) an rc(1) builtin?
On Monday 04 of April 2011 23:35:26 smi...@zenzebra.mv.com wrote: (...) I know that using awk(1) is a possibility, but awk(1) still has to system() every test -e, just like rc(1) does. (...) instead of test -e (and other object-related tests), why not to pipe from awk to plumb (the program) or to /mnt/plumb/send? i guess plumber is the default `dispatch-if-object-exists' mechanism in p9. using the `plumb' program would require exactly one rfork()/exec() for one message you decide in awk to pass through -- less than one per click. even better, piping to /mnt/plumber/send is just one open()/write(). your script is free to add and/or change plumber rules upon start. -- dexen deVries [[[↓][→]]] ``In other news, STFU and hack.'' mahmud, in response to Erann Gat's ``How I lost my faith in Lisp'' http://news.ycombinator.com/item?id=2308816
Re: [9fans] Making read(1) an rc(1) builtin?
I don't want to get into the rc discussions but this caught my eye: I would use scheme, but the scheme in fgb's contrib doesn't seem to provide any way of stat(2)ing path names without resorting to its foreign function interface. :( I am adding a bunch of stuff to Nils Holm's s9fes (Scheme 9 from Empty Space) Scheme interpreter. Mainly access to most of the syscalls (including stat). libc as needed (though I first try to implement its functions in s9). I can do things like (define fd (sys:open foo 0)) (display (sys:read fd 1)) (sys:close fd) I won't touch threads or things like seg*(), *brk() for now at least. Goal is to see how well one can do systems programming in Scheme. More to do but let me know if you are interested.
Re: [9fans] Making read(1) an rc(1) builtin?
On Tue, Apr 5, 2011 at 9:04 AM, Rudolf Sykora rudolf.syk...@gmail.com wrote: No. The discussion was not absurd. And no, Ron's summary was not a summary. Well, I'm not so sure, because the original motivation was this: I'm trying to write an Acme client in rc(1). I'd like to avoid spawning a new read(1) process every time I make a keystroke or click the mouse. This seemed to be the starting point. I'm not that worried, quite \relax'ed in fact, I just find the whole thing kind of ... well ... I'm at a loss for words :-) ron
Re: [9fans] Making read(1) an rc(1) builtin?
Lyndon Nerenberg lyn...@orthanc.ca writes: Unfortunately, echon.c doesn't solve the problem either, because it doesn't output a trailing newline. That's the whole point. 'echon' replaces 'echo -n ...', then echo.c loses all knowledge of any option flags. Oh. So there's another version of echo, too? I didn't know that. /me pulls up lyndon's contrib again... Hm. I don't see an echo.c in there. But anyway, yes, splitting echo and echo -n into two commands would solve the problem in a symmetric way. ron minnich rminn...@gmail.com writes: This discussion is interesting. rc is not as good a shell as bash because it lacks built-ins that make it a good programming language for writing an acme extension Did I summarizer it correctly? Once summarized, am I the only one who finds it absurd? No, I think a statement of the problem would be more like rc(1) is not as useful a language as $fill_in_your_favorite_turing_complete_luggage because it's not Turing complete. You cannot map arbitrary input to arbitrary output with a program of finite size in pure rc(1). Developing an Acme client is just, historically, how I came to that discovery (and to my subsequent frustration with rc's feature set). I think string parsing and numeric comparisons are reasonable features to include in almost any programming language. I think having primitives for line-oriented input and line-oriented output are more than appropriate for a line-oriented language like rc(1). As it stands, rc(1) can have it's cake, but it can't eat it without a fork(2). -- +---+ |E-Mail: smi...@zenzebra.mv.com PGP key ID: BC549F8B| |Fingerprint: 9329 DB4A 30F5 6EDA D2BA 3489 DAB7 555A BC54 9F8B| +---+
Re: [9fans] Making read(1) an rc(1) builtin?
This discussion is interesting. rc is not as good a shell as bash because it lacks built-ins that make it a good programming language for writing an acme extension Did I summarizer it correctly? Once summarized, am I the only one who finds it absurd? No, I think a statement of the problem would be more like rc(1) is not as useful a language as $fill_in_your_favorite_turing_complete_luggage because it's not Turing complete. You cannot map arbitrary input to arbitrary output with a program of finite size in pure rc(1). space shuttles are not as useful as cars, because the space shuttle is not d.o.t. approved. that might be true as you've stated the problem, but you do realize that rc is an acronym for run commands? and you do realize that the unix philosophy is all about this sort of encapsulation and modularity, right? the shell by design doesn't have to do everything. i really don't understand why you can't just skip reading the acme events you don't want, or if they're mixed in with important events. write a little filter in c that pushes the ones you don't want back to acme. is the problem here that you are unfamiliar with c and don't want to write that bit? - erik
Re: [9fans] Making read(1) an rc(1) builtin?
On 5 Apr 2011, at 9:58 am, yy wrote: I'm really impressed of how much we can talk about such a simple program as echo. Me also. Yet another solution is not to use echo in the first place. This won't work on p9p, but on plan 9 if you want to echo $foo where $foo(1) might possibly be '-n', you can cat /env/foo instead. If you have a var which is never going to contain _just_ '-n' alone, then you can echo $foo
Re: [9fans] Making read(1) an rc(1) builtin?
Me also. Yet another solution is not to use echo in the first place. This won't work on p9p, but on plan 9 if you want to echo $foo where $foo(1) might possibly be '-n', you can cat /env/foo instead. please google the famous bikeshed post. - erik
Re: [9fans] Making read(1) an rc(1) builtin?
On 5 Apr 2011, at 8:56 pm, erik quanstrom wrote: Me also. Yet another solution is not to use echo in the first place. This won't work on p9p, but on plan 9 if you want to echo $foo where $foo(1) might possibly be '-n', you can cat /env/foo instead. please google the famous bikeshed post. That was an easy comment off the top of my head. I am sorry I didn't read your post properly before posting. I get rather caught up in replying to people who want to add features when there's already a hundred workarounds because I thought I wasn't very good at solving problems. I'm very much not good enough to deal with my offline (not computer-related) problems which everyone around me wants to make out are trivial.
Re: [9fans] Making read(1) an rc(1) builtin?
i threw in the line reading code from /sys/src/cmd/read.c into rc.
here are the results:
9grid% cat builtinread
#!/tmp/rcread
for (i in `{seq 1 1}) { echo $i | read } /dev/null
9grid% cat origread
#!/bin/rc
for (i in `{seq 1 1}) { echo $i | read } /dev/null
9grid% cat noread
#!/bin/rc
for(i in `{seq 1 1}) { echo $i } /dev/null
9grid% time builtinread; time origread
1.49u 14.63s 17.61r builtinread
1.63u 21.99s 18.58r origread
I do see a decrease of the context switches by half, but that time is
certainly taken by rc itself.
just the effects of the echo preceding the read:
9grid% time noread
0.87u 9.36s 13.20r noread
9grid%
and just by itself (the rc builtin doesn't support arguments, this
should illustrate that it is in fact a builtin):
9grid% read /tmp/test
test
9grid% read /tmp/test
(broken)
(broken)
9grid% /bin/read /tmp/test
test
9grid%
under a debugging ramdisk we can see the difference in reads for the
two programs (you can guess that the context switch and the reading of
the binary are lost in the noise of reading a file char-by-char:
9grid% read test
ramfs 518560:-Twalk tag 12 fid 579 newfid 693 nwname 1 0:test
ramfs 518560:-Rwalk tag 12 nwqid 1 0:(0003 3 )
ramfs 518560:-Topen tag 12 fid 693 mode 0
ramfs 518560:-Ropen tag 12 qid (0003 3 ) iounit 8192
ramfs 518560:-Tread tag 12 fid 693 offset 0 count 1
ramfs 518560:-Rread tag 12 count 1 't'
ramfs 518560:-Tread tag 12 fid 693 offset 1 count 1
ramfs 518560:-Rread tag 12 count 1 'e'
ramfs 518560:-Tread tag 12 fid 693 offset 2 count 1
ramfs 518560:-Rread tag 12 count 1 's'
ramfs 518560:-Tread tag 12 fid 693 offset 3 count 1
ramfs 518560:-Rread tag 12 count 1 't'
ramfs 518560:-Tread tag 12 fid 693 offset 4 count 1
ramfs 518560:-Rread tag 12 count 1 '
'
test
ramfs 518560:-Tclunk tag 12 fid 693
ramfs 518560:-Rclunk tag 12
9grid% ./read test
ramfs 518560:-Twalk tag 12 fid 579 newfid 693 nwname 1 0:test
ramfs 518560:-Rwalk tag 12 nwqid 1 0:(0003 3 )
ramfs 518560:-Topen tag 12 fid 693 mode 0
ramfs 518560:-Ropen tag 12 qid (0003 3 ) iounit 8192
ramfs 518560:-Twalk tag 12 fid 579 newfid 644 nwname 1 0:read
ramfs 518560:-Rwalk tag 12 nwqid 1 0:(0002 8 )
ramfs 518560:-Topen tag 12 fid 644 mode 3
ramfs 518560:-Ropen tag 12 qid (0002 8 ) iounit 8192
ramfs 518560:-Tread tag 12 fid 644 offset 0 count 32
ramfs 518560:-Rread tag 12 count 32 '01eb 7e20 1084
0420 3e21 13b7 16c6'
ramfs 518560:-Tclunk tag 12 fid 644
ramfs 518560:-Rclunk tag 12
ramfs 518560:-Tread tag 12 fid 693 offset 0 count 1
ramfs 518560:-Rread tag 12 count 1 't'
ramfs 518560:-Tread tag 12 fid 693 offset 1 count 1
ramfs 518560:-Rread tag 12 count 1 'e'
ramfs 518560:-Tread tag 12 fid 693 offset 2 count 1
ramfs 518560:-Rread tag 12 count 1 's'
ramfs 518560:-Tread tag 12 fid 693 offset 3 count 1
ramfs 518560:-Rread tag 12 count 1 't'
ramfs 518560:-Tread tag 12 fid 693 offset 4 count 1
ramfs 518560:-Rread tag 12 count 1 '
'
test
ramfs 518560:-Tclunk tag 12 fid 693
ramfs 518560:-Rclunk tag 12
9grid%
so, an optimized /sys/src/cmd/read.c that doesn't read char-by-char
should give us an improvement, right? right:
9grid% newaread
1.52u 22.56s 15.66r newaread
and that's just for the silly test string. the improvement would be
bigger for longer strings.
code for aread is on sources: /contrib/andrey/aread.c
YMMV.
Re: [9fans] Making read(1) an rc(1) builtin?
(right, sorry erik for the double)
i hate to be pedantic,
By all means, please be pedantic, I was flat wrong.
ifs is not a list; it is a set of characters like strpbrk(2).
And it matches [$ifs]+ which is the other piece I always forget.
for(line in `{ifs=$nl cat}){...}
That is exactly what I was saying, thank you. That's what I get for
spending the last few days writing in php, sigh.
as per plan 9 tradition, the first non-option terminates
option processing. lindon's echon is not required. giving
echo -n as its first argument works fine.
What I thought smiley was referring to is when the first argument is
inteded to output -n, but doesn't. What I gave obviously puts an extra
space in (I guess I was a bit fuzzy earlier), echo -n $it^$nl works if
$it is not a list. I've never seen this be a problem in practice myself.
--
All original matter is hereby placed immediately under the public domain.
Re: [9fans] Making read(1) an rc(1) builtin?
On 4 April 2011 03:53, erik quanstrom quans...@quanstro.net wrote:
i think this is what you want
for(line in `{ifs=$nl cat}){...}
no, because that only sets ifs for the cat command, not
for the `{} construct.
ifs=$nl for (line in `{cat}) { ... }
is perhaps what you meant to say.
but i have no idea why one would avoid the read
idiom. for large input, forking off a read for each
line keeps the memory footprint O(1).
FWIW, i think that avoiding a read(1) loop is
perfectly reasonable thing to want to do.
a quick test i measured that calling read in
shell loop was about 100 times slower than
using `{}, and about 500 times slower than
using awk to read the lines.
in general, unless it was truly necessary, i would
try to use awk or sed rather than use read(1) in
a loop when i was going to deal with significantly
sized input.
when i've needed a -n safe version of echo in
the past, i've used something like this:
fn myecho {echo -n $* ^ '
'}
Re: [9fans] Making read(1) an rc(1) builtin?
roger peppe rogpe...@gmail.com writes:
when i've needed a -n safe version of echo in
the past, i've used something like this:
fn myecho {echo -n $* ^ '
'}
That doesn't work right when (~ $#* 0). It outputs a rogue space prior
to the newline. echo, with no arguments, should ouput just a newline.
Lyndon Nerenberg (VE6BBM/VE7TFX) lyn...@orthanc.ca writes:
(As a side note, if anyone goes into rc(1)'s source to implement any of
this, please add a -- option (or similar) to the echo builtin while
you're there.
Echo is not a builtin, and for one possible solution see
/n/sources/contrib/lyndon/echon.c
Ah, you're right; echo isn't a builtin. I guess echo would be another
big candidate for elevation to builtin status, then.
Unfortunately, echon.c doesn't solve the problem either, because it
doesn't output a trailing newline. The crux of the problem is how to
output -n on a line by itself, followed by a newline. I don't think
it can be done symmetrically without adding another option to echo. It
can be done by wrapping echo in if/switch statement, but that violates
symmetry of invocation. Ideally, echo would be invokable like:
$ echo -n $foo # suppress newline
$ echo -y $foo # force newline
$ echo $foo# newline by default
erik quanstrom quans...@quanstro.net writes:
could you be concrete about your performance problem.
if you don't have a performance problem, then there's no
point to optimizing.
I'm trying to write an Acme client in rc(1). I'd like to avoid spawning
a new read(1) process every time I make a keystroke or click the mouse.
Using multi-line reads wouldn't help much, because interactivity needs
to be maintained.
I'm using rc(1) because the /mnt/acme/%d/events interface is
well-documented (in /sys/doc/acme/acme.ps), but the C code under
/acme/bin/source/ for reading /mnt/acme/%d/events it is definitively
cryptic. I've managed to peel away the extra layers of code from one of
the simpler Acme clients, in /acme/bin/source/adict/win.c, and am in the
process of creating a general-purpose Acme event parser in C. The
output of the filter will be in a form easily digestible by scripts, and
would provide a good skeleton example of event parsing for other
coders to build upon. (There doesn't currently appear to be any such
starter code under /acme/bin/source or /sys/doc.)
If only Acme put a single extra space immediately prior the first
integer (c0) in it's event messages, this parsing could have been done
almost entirely within rc(1).
I know that using awk(1) is a possibility, but awk(1) still has to
system() every test -e, just like rc(1) does. I would use scheme, but
the scheme in fgb's contrib doesn't seem to provide any way of
stat(2)ing path names without resorting to its foreign function
interface. :(
All combined (forking read/test/echo, forking awk/sed/dd, parsing
/mnt/acme/%d/events, etc.)... this, I think, is why languages like Perl
came into existence and became so popular. I could definitely write an
Acme event parser in Perl, or even in bash(1). rc(1) is just a few
small features shy of making it practical to do in rc(1).
--
+---+
|E-Mail: smi...@zenzebra.mv.com PGP key ID: BC549F8B|
|Fingerprint: 9329 DB4A 30F5 6EDA D2BA 3489 DAB7 555A BC54 9F8B|
+---+
Re: [9fans] Making read(1) an rc(1) builtin?
On Apr 4, 2011, at 17:35, smi...@zenzebra.mv.com wrote: All combined (forking read/test/echo, forking awk/sed/dd, parsing /mnt/acme/%d/events, etc.)... this, I think, is why languages like Perl came into existence and became so popular. I could definitely write an Acme event parser in Perl, or even in bash(1). rc(1) is just a few small features shy of making it practical to do in rc(1). I think this does a very good job of summing up the issue. I think the point you might be missing, though, is that most of the Plan 9 community is quite happy about the current state of things. You're likely right that considerations like this led to perl and bash - but rc's state is not an accident. We know where to get perl and bash if we want them. Put another way, your problem seems to be I can't write an acme client in rc with performance I'm happy with (leaving aside, for the moment, questions of measurement). Your solution is expand rc; I suspect the consensus of the community would be either deal with it or use C. Actually, that might be the consensus of the community on *most* issues. :-) Anthony PGP.sig Description: This is a digitally signed message part
Re: [9fans] Making read(1) an rc(1) builtin?
... creating a general-purpose Acme event parser in C. you may want to look at plan9port's acmeevent[1]. -- oleg [1] http://swtch.com/plan9port/man/man1/acmeevent.html
Re: [9fans] Making read(1) an rc(1) builtin?
I think this does a very good job of summing up the issue. I think the point you might be missing, though, is that most of the Plan 9 community is quite happy about the current state of things. You're likely right that considerations like this led to perl and bash - but rc's state is not an accident. We know where to get perl and bash if we want them. we know where to get perl and bash, and obviously haven't. that ought to tell you something. there, fixed that for ya. Put another way, your problem seems to be I can't write an acme client in rc with performance I'm happy with (leaving aside, for the moment, questions of measurement). Your solution is expand rc; I suspect the consensus of the community would be either deal with it or use C. Actually, that might be the consensus of the community on *most* issues. :-) i see this from a little different perspective. rc is a elegant, clean, easy-to-understand shell. none of these would hold if we threw in every feature we could think of. on the other hand, i don't think rc is the last word in shells. i think a new shell would be warmly received. i suppose one is needed every 20 years or so. unfortunately, it's not that easy. tom duff and sr bourne are pretty smart guys. you probablly won't outsmart them. but you are working with a more- capable system, you're free to ignore unix, and you do have the advantage of twenty years' experience with rc. good luck. - erik
Re: [9fans] Making read(1) an rc(1) builtin?
Unfortunately, echon.c doesn't solve the problem either, because it
doesn't output a trailing newline. The crux of the problem is how to
output -n on a line by itself, followed by a newline. I don't think
if you give me 2^n pennies for the nth way i can think of doing this,
i'm gonna be loaded
0. awk 'BEGIN{print -n; exit}'
1. echo x -n | sed 's/^ //'
2. echo print -n\n | hoc
3. echo '-n
' | bc
4. unicode 2d 6e | tr -d '\012' ; echo
5. echo '' -n | tr -d ' '
6. '-n' = 1; whatis -n | sed 's/=.*//'
I'm trying to write an Acme client in rc(1). I'd like to avoid spawning
a new read(1) process every time I make a keystroke or click the mouse.
Using multi-line reads wouldn't help much, because interactivity needs
to be maintained.
i don't know what your application is, you don't need to get
every mouseclick. see the Mail client.
- erik
Re: [9fans] Making read(1) an rc(1) builtin?
Unfortunately, echon.c doesn't solve the problem either, because it doesn't output a trailing newline. That's the whole point. 'echon' replaces 'echo -n ...', then echo.c loses all knowledge of any option flags. --lyndon
[9fans] Making read(1) an rc(1) builtin?
I'm in the process of writing some filters in rc(1). One thing that has
come to concern me about rc(1) is that read(1) is not a builtin
command. For example, with a loop like:
while(message=`{read})
switch($message) {
case foo
dofoo
case bar
dobar
case *
dodefault
}
Each line that's read by the script causes it to fork a new process,
/bin/read, whose sole purpose is to read a single line and die. That
means at least one fork for each line read and, if your input has many
lines, that means spawning many processes. I wonder if it wouldn't make
sense to move read(1) into rc(1) and make it a builtin command. A
wrapper script could then be created, at /bin/read, to call rc -c 'eval
read $*' with the appropriate arguments (or sed $n^q, etc.), for any
program that requires an actual /bin/read to exist.
A similar line of thought holds for /bin/test. The string and numeric
tests (-n, -z, =, !=, , , -lt, -eq, -ne, etc.) can be very frequently
used, and can lead to spawning unnecessarily many processes. For the
file test parameters (-e, -f, -d, -r, -x, -A, -L, -T, etc.), however,
this argument isn't as strong. Since the file tests have to stat(2) a
path, they already require a call to the underlying file system, and an
additional fork wouldn't be that much more expensive. I could see the
string and numeric tests being moved into rc(1) as a test builtin,
with the file tests residing at /bin/ftest (note the f). The test
builtin could scan its arguments and call ftest if needed. A wrapper
script at /bin/test could provide compatibility for existing programs
which expect an executable named /bin/test to exist.
I understand the Unix/Plan 9 philosophy of connecting tools that do one
job and do it well. But I don't think /bin/read and /bin/test are
places where that philosophy is practical (i.e., efficient). After all,
reading input lines really is the perogative of any program that
processes line-oriented data (like rc(1) does). In addition, /bin/read
represents a simple and fairly stable interface that's not likely to
change appreciably in the future. Comparison of numeric and string
values is also a fairly stable operation that's not likely to change,
and is not likely to be needed outside of rc(1). Most programming
languages (C, awk, etc.) have their own mechanisms for integer and
string comparison. I suspect moving these operations into rc(1) (with
appropriate replacement scripts to ensure compatibility) could
appreciably increase the performance of shell scripts, with very little
cost in modularity or compatibility.
Any thoughts on this?
I'm also a bit stumped by the fact that rc(1) doesn't have anything
analogous to bash(1)'s string parsing operations: ${foo#bar},
${foo##bar}, ${foo%bar}, ${foo%%bar}, or ${foo/bar/baz}. Is there any
way to extract substrings (or single characters) from a string in rc(1)
without having to fork a dd, awk, or sed? I've tried setting ifs='' and
using foo=($bar), but rc(1) always splits bar on spaces. Perhaps, if
rc(1) used the first character of $ifs to split $bar, $bar could be
split into individual characters when ifs=''. Then, the characters of
$bar could be addressed without resort to dd and friends.
(As a side note, if anyone goes into rc(1)'s source to implement any of
this, please add a -- option (or similar) to the echo builtin while
you're there. Having to wrap echo in:
# make 'myecho $foo' work even when $foo starts with '-n'
fn myecho {
if(~ $1 --) {
shift
if(~ $1 -n) {
shift
echo -n -n $*
echo
}
if not echo $*
}
if not echo $*
}
can be rather inconvenient.)
--
+---+
|E-Mail: smi...@zenzebra.mv.com PGP key ID: BC549F8B|
|Fingerprint: 9329 DB4A 30F5 6EDA D2BA 3489 DAB7 555A BC54 9F8B|
+---+
Re: [9fans] Making read(1) an rc(1) builtin?
Write some real world tests using bash/GNU tools, rc (with static
linked versions of p9p) and tell us what happend.
Maybe you will be surprised.
[1] http://cm.bell-labs.com/cm/cs/tpop/
On Mon, Apr 4, 2011 at 12:30 AM, smi...@zenzebra.mv.com wrote:
I'm in the process of writing some filters in rc(1). One thing that has
come to concern me about rc(1) is that read(1) is not a builtin
command. For example, with a loop like:
while(message=`{read})
switch($message) {
case foo
dofoo
case bar
dobar
case *
dodefault
}
Each line that's read by the script causes it to fork a new process,
/bin/read, whose sole purpose is to read a single line and die. That
means at least one fork for each line read and, if your input has many
lines, that means spawning many processes. I wonder if it wouldn't make
sense to move read(1) into rc(1) and make it a builtin command. A
wrapper script could then be created, at /bin/read, to call rc -c 'eval
read $*' with the appropriate arguments (or sed $n^q, etc.), for any
program that requires an actual /bin/read to exist.
A similar line of thought holds for /bin/test. The string and numeric
tests (-n, -z, =, !=, , , -lt, -eq, -ne, etc.) can be very frequently
used, and can lead to spawning unnecessarily many processes. For the
file test parameters (-e, -f, -d, -r, -x, -A, -L, -T, etc.), however,
this argument isn't as strong. Since the file tests have to stat(2) a
path, they already require a call to the underlying file system, and an
additional fork wouldn't be that much more expensive. I could see the
string and numeric tests being moved into rc(1) as a test builtin,
with the file tests residing at /bin/ftest (note the f). The test
builtin could scan its arguments and call ftest if needed. A wrapper
script at /bin/test could provide compatibility for existing programs
which expect an executable named /bin/test to exist.
I understand the Unix/Plan 9 philosophy of connecting tools that do one
job and do it well. But I don't think /bin/read and /bin/test are
places where that philosophy is practical (i.e., efficient). After all,
reading input lines really is the perogative of any program that
processes line-oriented data (like rc(1) does). In addition, /bin/read
represents a simple and fairly stable interface that's not likely to
change appreciably in the future. Comparison of numeric and string
values is also a fairly stable operation that's not likely to change,
and is not likely to be needed outside of rc(1). Most programming
languages (C, awk, etc.) have their own mechanisms for integer and
string comparison. I suspect moving these operations into rc(1) (with
appropriate replacement scripts to ensure compatibility) could
appreciably increase the performance of shell scripts, with very little
cost in modularity or compatibility.
Any thoughts on this?
I'm also a bit stumped by the fact that rc(1) doesn't have anything
analogous to bash(1)'s string parsing operations: ${foo#bar},
${foo##bar}, ${foo%bar}, ${foo%%bar}, or ${foo/bar/baz}. Is there any
way to extract substrings (or single characters) from a string in rc(1)
without having to fork a dd, awk, or sed? I've tried setting ifs='' and
using foo=($bar), but rc(1) always splits bar on spaces. Perhaps, if
rc(1) used the first character of $ifs to split $bar, $bar could be
split into individual characters when ifs=''. Then, the characters of
$bar could be addressed without resort to dd and friends.
(As a side note, if anyone goes into rc(1)'s source to implement any of
this, please add a -- option (or similar) to the echo builtin while
you're there. Having to wrap echo in:
# make 'myecho $foo' work even when $foo starts with '-n'
fn myecho {
if(~ $1 --) {
shift
if(~ $1 -n) {
shift
echo -n -n $*
echo
}
if not echo $*
}
if not echo $*
}
can be rather inconvenient.)
--
+---+
|E-Mail: smi...@zenzebra.mv.com PGP key ID: BC549F8B|
|Fingerprint: 9329 DB4A 30F5 6EDA D2BA 3489 DAB7 555A BC54 9F8B|
+---+
Re: [9fans] Making read(1) an rc(1) builtin?
One thing that has come to concern me about rc(1) is that read(1) is
not a builtin command.
The general idea here is that forking a new process is not usually
(ever?) the bottleneck, if you have a script that needs to run faster,
there's other overhead to trim first, and if you really need to, you can:
(giving up line at a time response).
ifs=($nl)
lines=`{cat}
for($lines as $line){...}
There isn't any such trick (that I know) for test, but how much is it
slowing you down?
I'm also a bit stumped by the fact that rc(1) doesn't have anything
analogous to bash(1)'s string parsing operations: ${foo#bar},
${foo##bar}, ${foo%bar}, ${foo%%bar}, or ${foo/bar/baz}.
I could never remember what these did, except the last one.
Is there any way to extract substrings (or single characters) from a
string in rc(1) without having to fork a dd, awk, or sed?
Sure, for some things, except it uses cat! Without any forking, I don't
know (see below).
On the other hand, echo -n is a wart. I wonder, does echo '' -n work?
(My plan9 machine is off and far away.)
On a more friendly note. Hi, I think I know you slightly, telephones.
Tristan
--
All original matter is hereby placed immediately under the public domain.
Re: [9fans] Making read(1) an rc(1) builtin?
(As a side note, if anyone goes into rc(1)'s source to implement any of this, please add a -- option (or similar) to the echo builtin while you're there. Echo is not a builtin, and for one possible solution see /n/sources/contrib/lyndon/echon.c
Re: [9fans] Making read(1) an rc(1) builtin?
On 4 Apr 2011, at 12:41 am, Tristan Plumb wrote:
I'm also a bit stumped by the fact that rc(1) doesn't have anything
analogous to bash(1)'s string parsing operations: ${foo#bar},
${foo##bar}, ${foo%bar}, ${foo%%bar}, or ${foo/bar/baz}.
I could never remember what these did, except the last one.
I could never remember what any of these did, except that they are a
major reason I'm thankful I hardly have anything to do with bash any
more. Cluttering up your working memory with 600 different cryptic
ways to do things is stupid when you're trying to solve a hard
problem. Spending time and effort learning 600 cryptic ways to get
tiny improvements in performance is stupid when you want the machine
to reduce your workload.
I'd also like to reiterate what pmarin wrote about trying it out
first, except I'd say you will be surprised. :) Without dynamic
linking, fork() -- or, to put the problem where it actually occurs,
exec() -- is not particularly slow at all.
On the other hand, echo -n is a wart. I wonder, does echo '' -n work?
(My plan9 machine is off and far away.)
It works very well, doing exactly what it's supposed to, although I
vaguely remember having problems with such a feature in Linux many
years ago. It does look slightly warty, being an odd argument out,
but if you think about it options are always odd arguments.
Re: [9fans] Making read(1) an rc(1) builtin?
The general idea here is that forking a new process is not usually
(ever?) the bottleneck, if you have a script that needs to run faster,
there's other overhead to trim first, and if you really need to, you can:
(giving up line at a time response).
ifs=($nl)
lines=`{cat}
for($lines as $line){...}
i hate to be pedantic, but i see 2 syntax errors, a
unintended side effect and an extra set of parens.
ifs is not a list; it is a set of characters like strpbrk(2).
i think this is what you want
for(line in `{ifs=$nl cat}){...}
but i have no idea why one would avoid the read
idiom. for large input, forking off a read for each
line keeps the memory footprint O(1).
if not dealing with large input, then a few forks don't
matter.
On the other hand, echo -n is a wart. I wonder, does echo '' -n work?
(My plan9 machine is off and far away.)
as per plan 9 tradition, the first non-option terminates
option processing. lindon's echon is not required. giving
echo -n as its first argument works fine.
- erik
Re: [9fans] Making read(1) an rc(1) builtin?
Each line that's read by the script causes it to fork a new process,
we're not running out. even with a mere four billion odd to choose from.
I understand the Unix/Plan 9 philosophy of connecting tools that do one
job and do it well. But I don't think /bin/read and /bin/test are
places where that philosophy is practical (i.e., efficient). After all,
reading input lines really is the perogative of any program that
processes line-oriented data (like rc(1) does). In addition, /bin/read
represents a simple and fairly stable interface that's not likely to
change appreciably in the future.
could you be concrete about your performance problem.
if you don't have a performance problem, then there's no
point to optimizing.
I'm also a bit stumped by the fact that rc(1) doesn't have anything
analogous to bash(1)'s string parsing operations: ${foo#bar},
${foo##bar}, ${foo%bar}, ${foo%%bar}, or ${foo/bar/baz}. Is there any
way to extract substrings (or single characters) from a string in rc(1)
without having to fork a dd, awk, or sed? I've tried setting ifs='' and
using foo=($bar), but rc(1) always splits bar on spaces.
false.
ifs=☺ {x=`{echo -n 'a b c ☺ d e f'}}
; whatis x
x=('a b c ' ' d e f')
; echo $#x
2
(you might not want to try splitting on non-ascii with the rc
on sources. i'm unsure about p9p.)
(As a side note, if anyone goes into rc(1)'s source to implement any of
this, please add a -- option (or similar) to the echo builtin while
you're there. Having to wrap echo in:
when exactly does this come up?
- erik
