On Apr 29, 2010, at 9:32 AM, Hein, Nashua NH wrote:
John Gillings asked me about an interesting behavior with a major performance difference using pipes.
Lots of interesting stuff snipped, but basically Perl reading from its own pipe is 10 times worse than reading from a file and Perl reading from a DCL pipe is 100 times worse. Ouch.
Also tried large settings for DECC$PIPE_BUFFER_QUOTA / SIZE 'just in case'
Likely irrelevant for Perl piping in from a DCL command using qx(). Try fiddling with PERL_MBX_SIZE, which is only 512 by default (finally bumped to 8192 in Perl 5.12.0). Perl has its own pipe implementation which, like the CRTL's, is based on mailboxes. But it had the ability to set the size of the mailbox buffer before the CRTL did. We should probably make it honor the latecomer DECC$PIPE_BUFFER_SIZE as an alias.
Also likely irrelevant for your DCL PIPE command example. At least I thought the pipe driver implemented its own device class and not a mailbox. Even if it's a mailbox, the buffer size settings for the CRTL only apply to pipes created by the CRTL, and Perl's to pipes created by Perl. This one is created by DCL. Probably only Forrest Kenney or the source listings could reveal what goes on in there.
Some reading http://h71000.www7.hp.com/openvms/journal/v9/mailboxes.pdf (Bruce Ellis)
This is a great article. I think this is what finally made me realize that mailboxes can partially emulate stream behavior on read operations only but not on write. Which makes them totally unsuitable for implementing pipes, but that's a different beef only tangentially related.
On the performance question, it would be interesting to know, when Perl is piping in from a command spawned under its control, how much of the slow-down is the set-up time of creating the process and how much is actually reading through the mailbox.
When reading from a DCL pipe in Perl, it might be worth experimenting with sysread(), which according to its docs, "bypasses buffered I/O." It rather sounds like you have a reader lazily waiting for the writer to do something interesting, but when DCL is the writer, it's even lazier and stalls out waiting for the reader to go ahead and do something with the small change already tossed its way.
________________________________________ Craig A. Berry mailto:craigbe...@mac.com "... getting out of a sonnet is much more difficult than getting in." Brad Leithauser
