Re: Piping
On Wed, 1 Dec 1999, Adriano Camargo Rodrigues da Cunha wrote:
If you think that spawning many processes for a pipe is a waste of resources
on low-end systems like MSX you should use the temporary file approach, just
like MSXDOS and MS-DOS do.
Yeah, I could, but it's not the right way (and it's t
dirty...), since the pipe() system call exists in UZIX kernel.
For a unix system like uzix, the MS-DOS method is totally useless, indeed.
Last night I tried my first implementation of pipes in sash (UZIX
default shell). I tried only with one pipe.
My algorithm is:
1. put arguments from prompt into char *argv[]
2. check for the pipe symbol ("|")
3. if not found pipe symbol, run as a single executable file
4. pipe(pipe_fd) (result: pipe_fd[0]=read, pipe_fd[1]=write)
5. first fork:
close(pipe_fd[0])
close(1)
dup2(pipe_fd[1], 1)
argv = all arguments before the pipe symbol
execute(argv[0])
6. second fork:
close(pipe_fd[1])
close(0)
dup2(pipe_fd[0], 0)
argv = all arguments after the pipe symbol
execute(argv[0])
7. [wait for all processes to terminate? only last one?]
7. return to prompt
This sounds ok. I presume that process 1 and process 2 are running
simultaniously and are put to sleep when they ask for data that isn't
there yet? If not, that is a problem.
Well, it works for the cat/more pair. Doing a cat fudeba.txt |
more works. But it doesn't work for the ls/more pair. System hangs after
displaying the last file in directory (NOTE: system doesn't crash, just
all processes goes to sleep mode).
Did you think of sending a wake-up signal to the process waiting for data
when there is put data in the buffer after is was empty?
Another question is: shouldn't shell wait for all processes to
terminate? Or just for the last process?
I should say both, since the first can still receive input when the last
died. But most of the time it doesn't matter, since when the last process
dies, the first will recieve a "broken pipe" signal for any output it
sends.
By the way, the pipe() syscall implementation is (well, seems to
be) right, since 'man' works (and it uses a pipe to 'more' to display
manual pages).
It sounds like the process isn't awakened. If that is so, only the pipes
that do not block the second process work. So if the data flow is not
enough, it doesn't work.
Bye,
shevek
/***Use gcc to compile***Don't mind the warning/
int*a,k ,v[9];int*main (){int i,j,s=1, x,z ,c[
]={1,4,7,4,3,4,5,4 ,1,1,1 ,2,
3,3,3,4};for(i=0;( i++9) !k
;s=-s){k=0;scanf("%d",z);v [--z ]=s;for(j=0;j
8; j++){z=v[ c[j ]];k|=z==v [c[
j]- c[j+8]]( z(v[ c[j]+c [j+
8]]==z));;;}}printf(" %d won\n",- s*k );}
/***Tic-tac-toe.use 1-9 to play/
MSX Mailinglist. To unsubscribe, send an email to [EMAIL PROTECTED] and put
in the body (not subject) "unsubscribe msx [EMAIL PROTECTED]" (without the
quotes :-) Problems? contact [EMAIL PROTECTED] (www.stack.nl/~wiebe/mailinglist/)
Re: Piping
Shevek,
7. [wait for all processes to terminate? only last one?]
I tested yesterday. Waiting for all the process to terminate
doesn't hangs system anymore. :)
BUT: by an unknown reason, the flushing of the pipe is not correct
yet. The error seems to be in my algorithm, since 'man' (that make a
pipe to 'more') works perfectly... Maybe I forgot something in the
algorithm?
This sounds ok. I presume that process 1 and process 2 are running
simultaniously
Right.
and are put to sleep when they ask for data that isn't
there yet? If not, that is a problem.
It seemed that second process (reader) was waiting for data
forever... Process 1 (writer) allways have the data (since it reads data
from disk).
Did you think of sending a wake-up signal to the process waiting for data
when there is put data in the buffer after is was empty?
It is done.
Another question is: shouldn't shell wait for all processes to
terminate? Or just for the last process?
I should say both
That's what I did yesterday. After the forks, I put a
for (i=0; i2; i++) {
if ((wait(status)==pid1) close(pipe_fd[0]);
else close(pipe_fd[1]);
}
since the first can still receive input when the last
died. But most of the time it doesn't matter, since when the last process
dies, the first will recieve a "broken pipe" signal for any output it
sends.
I thought that "broken pipe" was an error code set by kernel when
it tries to write to a pipe that has no readers... Interesting... I'll
take a look...
Thanks for you reply, Shevek!
Adriano Camargo Rodrigues da Cunha ([EMAIL PROTECTED])
Engenharia de Computacao - UNICAMP
http://www.adrpage.cjb.net http://if.you.dont.like.msx.usuck.com
* Runtime Error 6D at 417A:32CF: Incompetent User. *
MSX Mailinglist. To unsubscribe, send an email to [EMAIL PROTECTED] and put
in the body (not subject) "unsubscribe msx [EMAIL PROTECTED]" (without the
quotes :-) Problems? contact [EMAIL PROTECTED] (www.stack.nl/~wiebe/mailinglist/)
Re: Piping
Adriano Camargo Rodrigues da Cunha wrote: I thought that "broken pipe" was an error code set by kernel when it tries to write to a pipe that has no readers... Interesting... I'll take a look... In the case of a named-pipe (a fifo-special-file-thing) the writer just goes to sleep, and will never get a broken pipe error. Don't confuse the code for both different types of pipes. David Heremans -- "One difference between SuSE and Red Hat is that the former operates in a country where people don't sue each other over coffee being too hot." Linus Torvalds MSX Mailinglist. To unsubscribe, send an email to [EMAIL PROTECTED] and put in the body (not subject) "unsubscribe msx [EMAIL PROTECTED]" (without the quotes :-) Problems? contact [EMAIL PROTECTED] (www.stack.nl/~wiebe/mailinglist/)
Re: Piping
]
] That's what I did yesterday. After the forks, I put a
] for (i=0; i2; i++) {
] if ((wait(status)==pid1) close(pipe_fd[0]);
] else close(pipe_fd[1]);
] }
]
After the two forks, the shell should close both ends of the pipe, before
going into the wait loop. So, your code should become
4. pipe(pipe_fd)
5. in child of first fork: close(pipe_fd[0]),...
6. in child of second fork: close(pipe_fd[1]),...
7. in parent: close(pipe_fd[0]);close(pipe_fd[1]);
8. for (i=0; i2; i++) wait(status);
Otherwise, you keep too many copies of the pipe filehandles open:
pipe_fd[0] is open in child 2 and in parent, should only be open in child 2.
pipe_fd[1] is open in child 1 and in parent, should only be open in child 1.
I think this will solve your problem. If not, you must review the kernel
implementation of the pipe and you must doublecheck if all filehandles for a
terminated process are properly closed by the kernel during the clean-up.
1)
] since the first can still receive input when the last
] died. But most of the time it doesn't matter, since when the last process
] dies, the first will recieve a "broken pipe" signal for any output it
] sends.
]
2)
] I thought that "broken pipe" was an error code set by kernel when
] it tries to write to a pipe that has no readers... Interesting... I'll
] take a look...
Funny, two ways to say the same thing, since you can summarise 1) and 2) as:
1) A process that writes to a closed pipe, gets a "broken pipe" signal (from
the kernel)
2) The kernel sets the "broken pipe" signal, when a process tries to write to
a pipe that has no readers (hence, is closed).
Kind regards,
Alex Wulms
--
Alex Wulms/XelaSoft - MSX of anders NIX - Linux 4 ever
See my homepage for info on the *** XSA *** format
http://www.inter.nl.net/users/A.P.Wulms
MSX Mailinglist. To unsubscribe, send an email to [EMAIL PROTECTED] and put
in the body (not subject) "unsubscribe msx [EMAIL PROTECTED]" (without the
quotes :-) Problems? contact [EMAIL PROTECTED] (www.stack.nl/~wiebe/mailinglist/)
Re: Piping
Hi, again, people.
Unix uses the fork()/exec() approach.
This was an important explanation, Alex. Thank you.
If you think that spawning many processes for a pipe is a waste of resources
on low-end systems like MSX you should use the temporary file approach, just
like MSXDOS and MS-DOS do.
Yeah, I could, but it's not the right way (and it's t
dirty...), since the pipe() system call exists in UZIX kernel.
Last night I tried my first implementation of pipes in sash (UZIX
default shell). I tried only with one pipe.
My algorithm is:
1. put arguments from prompt into char *argv[]
2. check for the pipe symbol ("|")
3. if not found pipe symbol, run as a single executable file
4. pipe(pipe_fd) (result: pipe_fd[0]=read, pipe_fd[1]=write)
5. first fork:
close(pipe_fd[0])
close(1)
dup2(pipe_fd[1], 1)
argv = all arguments before the pipe symbol
execute(argv[0])
6. second fork:
close(pipe_fd[1])
close(0)
dup2(pipe_fd[0], 0)
argv = all arguments after the pipe symbol
execute(argv[0])
7. [wait for all processes to terminate? only last one?]
7. return to prompt
Well, it works for the cat/more pair. Doing a cat fudeba.txt |
more works. But it doesn't work for the ls/more pair. System hangs after
displaying the last file in directory (NOTE: system doesn't crash, just
all processes goes to sleep mode).
Another question is: shouldn't shell wait for all processes to
terminate? Or just for the last process?
By the way, the pipe() syscall implementation is (well, seems to
be) right, since 'man' works (and it uses a pipe to 'more' to display
manual pages).
Thanks for any help,
Adriano Camargo Rodrigues da Cunha ([EMAIL PROTECTED])
Engenharia de Computacao - UNICAMP
http://www.adrpage.cjb.net http://if.you.dont.like.msx.usuck.com
* Hard Disk Failure. Message: "Stacker Rides Again!" *
MSX Mailinglist. To unsubscribe, send an email to [EMAIL PROTECTED] and put
in the body (not subject) "unsubscribe msx [EMAIL PROTECTED]" (without the
quotes :-) Problems? contact [EMAIL PROTECTED] (www.stack.nl/~wiebe/mailinglist/)
RE: Piping
[Howto] cat fudeba.txt | head 20 | tail 10 | less I know for sure (i.e. 95% ;-)) that Messy DOS uses temporary files in this case... (so actaully does something like: cat fudeba.txt tmp001 head -20 tmp001 tmp002; del tmp001 tail -10 tmp002 tmp003; del tmp002 less tmp003; del tmp003 (DOS might even postpone the deletion of the tmp... files to the end!)) Ciao, +-- Eric Boon ---+ | this signature was | | intentionally left blank | ++ MSX Mailinglist. To unsubscribe, send an email to [EMAIL PROTECTED] and put in the body (not subject) "unsubscribe msx [EMAIL PROTECTED]" (without the quotes :-) Problems? contact [EMAIL PROTECTED] (www.stack.nl/~wiebe/mailinglist/)
RE: Piping
On Thu, 25 Nov 1999, Boon, Eric wrote:
[Howto]
cat fudeba.txt | head 20 | tail 10 | less
I know for sure (i.e. 95% ;-)) that Messy DOS uses temporary files in this
case...
(so actaully does something like:
cat fudeba.txt tmp001
head -20 tmp001 tmp002; del tmp001
tail -10 tmp002 tmp003; del tmp002
less tmp003; del tmp003
(DOS might even postpone the deletion of the tmp... files to the end!))
This is correct. Unix uses a much more flexible approach. It uses a
buffer, that is filled by the first process and emptied by the second. If
the buffer is full, the first process blocks (stops executing), until
there is some space. If the buffer is empty, the second process is put to
sleep. This way, there is no problem when the piped data is much (like an
endless pipe, for which in DOS the second process would never be
executed). That is why it is not considered a waste of resources.
I hope you understand it.
Bye,
shevek
/***Use gcc to compile***Don't mind the warning/
int*a,k ,v[9];int*main (){int i,j,s=1, x,z ,c[
]={1,4,7,4,3,4,5,4 ,1,1,1 ,2,
3,3,3,4};for(i=0;( i++9) !k
;s=-s){k=0;scanf("%d",z);v [--z ]=s;for(j=0;j
8; j++){z=v[ c[j ]];k|=z==v [c[
j]- c[j+8]]( z(v[ c[j]+c [j+
8]]==z));;;}}printf(" %d won\n",- s*k );}
/***Tic-tac-toe.use 1-9 to play/
MSX Mailinglist. To unsubscribe, send an email to [EMAIL PROTECTED] and put
in the body (not subject) "unsubscribe msx [EMAIL PROTECTED]" (without the
quotes :-) Problems? contact [EMAIL PROTECTED] (www.stack.nl/~wiebe/mailinglist/)
RE: Piping
First of all. Standard input has a file handle of 0 and standard output has a file handle of 1. Input redirection is done by closing the handle 0 and after that simply opening a new file. The new file will have the handle number of 0. Output redirection is done in the same way. First close handle 1 and then open the file to redirect to. Under MSXDOS2 the output of the first process is redirected to a temporary file. The second process closes its standard input handle of 0 and opens the temporary file of the first process. And so on. -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]]On Behalf Of Adriano Camargo Rodrigues da Cunha Sent: Thursday, November 25, 1999 3:01 PM To: MSX International Mailing List Subject: Piping Hi, Does anyone knows how a pipe between processes is implemented by UNIX shells? I think that lauching all processes at the same time and piping one to another is an extreme waste of resources. And I know that MS-DOS and MSXDOS2 have pipes and they are monotask OSs. For example: cat fudeba.txt | head 20 | tail 10 | less What is the (best) approach? I think that making 4 fork()/exec() is an extreme waste of system resources. And I don't know how making the pipe in a serialized way. I tried to understand the code of the clam shell, but I couldn't. :P [btw, it's not off-topic. it's related to UZIX, even if some people doesn't like it] Thanks, Adriano Camargo Rodrigues da Cunha ([EMAIL PROTECTED]) Engenharia de Computacao - UNICAMP http://www.adrpage.cjb.net http://if.you.dont.like.msx.usuck.com * Bug (n) Technology originally developed by Microsoft in the 1980s. * MSX Mailinglist. To unsubscribe, send an email to [EMAIL PROTECTED] and put in the body (not subject) "unsubscribe msx [EMAIL PROTECTED]" (without the quotes :-) Problems? contact [EMAIL PROTECTED] (www.stack.nl/~wiebe/mailinglist/) MSX Mailinglist. To unsubscribe, send an email to [EMAIL PROTECTED] and put in the body (not subject) "unsubscribe msx [EMAIL PROTECTED]" (without the quotes :-) Problems? contact [EMAIL PROTECTED] (www.stack.nl/~wiebe/mailinglist/)
Re: Piping
] cat fudeba.txt | head 20 | tail 10 | less ] ] What is the (best) approach? I think that making 4 fork()/exec() ] is an extreme waste of system resources. And I don't know how making the Unix uses the fork()/exec() approach. You can see it if you do the following in one terminal: cat | head -20 | tail -10 | less and ask for a process tree (for example with pstree in linux) in another terminal: bash-+-cat |-head |-less `-tail You can see that bash has four child processes. If you think that spawning many processes for a pipe is a waste of resources on low-end systems like MSX you should use the temporary file approach, just like MSXDOS and MS-DOS do. Kind regards, Alex Wulms -- Alex Wulms/XelaSoft - MSX of anders NIX - Linux 4 ever See my homepage for info on the *** XSA *** format http://www.inter.nl.net/users/A.P.Wulms MSX Mailinglist. To unsubscribe, send an email to [EMAIL PROTECTED] and put in the body (not subject) "unsubscribe msx [EMAIL PROTECTED]" (without the quotes :-) Problems? contact [EMAIL PROTECTED] (www.stack.nl/~wiebe/mailinglist/)
