Re: Spurious linebreak in non-record output
Hello Craig, To quote Queen: It's a kind of magic. ;-) The tip with autoflush was spot on and resolved the issue entirely for my use case. I'll still give 5.14 a good spin when it comes out. Thanks a lot for the help! Greetings, Martin |+ |Craig A. Berry| | |craigbe...@mac.com| | || To| |21.04.2011 17:40|Dr. Martin P.J. Zinser zin...@sysdev.deutsche-boerse.com| || cc| ||VMSPERL@PERL.ORG | || Subject| ||Re: Spurious linebreak in non-record output | || | || | || | || | || | |+ ---| | | ---| On Apr 21, 2011, at 10:47 AM, Dr. Martin P.J. Zinser wrote: Hello Craig, nobody used the dirty Oracle word around here, just because something is a table it does not have do come from a database ;-) Sorry. I promise to wash my mouth out with soap :-). Anyhow, I made some progress and have an easy reproducer plus example output and config information all wrapped up at http://zinser.no-ip.info/www/trans/lineb.zip No other modules but CGI are involved in the example. The version I use is 3.49. For best effect look at the output files in a terminal set to 132. The terminal output was created with perl lineb.pl terminal.txt The web output was captured with View Source from a Web browser. The Webserver in question is an OSU 3.11 server, communication betweeen the server and perl is via a local DECnet task. Yikes. I've never used OSU and I haven't touched a DECnet object in about 20 years, but it's pretty clear that you're getting the extra linebreak every 4096 bytes just like what was happening before with record-oriented files. This will change in 5.14.0 to every 32768 bytes, or you can edit the buffer size yourself by modifying perlio.c. You can see from the fix for that: http://perl5.git.perl.org/perl.git/commitdiff/5e7d60de18218a624f7f6d5c99bdaf0ff03973df that I limited it to regular files [S_ISREG(statbuf.st_mode) is true] with variable or variable with fixed control. It occurred to me at the time that there might be other record-oriented widgets that need similar treatment, but there is danger in casting the net too wide; binary data streams could be corrupted, for example. To me it seems unlikely that DECnet objects would always only transport text-oriented data, so it probably isn't safe to line buffer them because that would introduce a record boundary for every newline in the data stream, which is right for text data but not for binary data. So I'm not sure what the fix is, or if there is one. Using a record-oriented widget to implement a stream-oriented protocol like HTTP seems like asking for trouble, but it is what it is. The first workaround I would try would be to enable autoflushing with C$| = 1; at the top of your script. Or you could use syswrite instead of print to write your data. Craig A. Berry mailto:craigbe...@mac.com ... getting out of a sonnet is much more difficult than getting in. Brad Leithauser - Diese E-Mail enthaelt vertrauliche oder rechtlich geschuetzte Informationen. Wenn Sie nicht der beabsichtigte
Re: Spurious linebreak in non-record output
Hello Craig, nobody used the dirty Oracle word around here, just because something is a table it does not have do come from a database ;-) Anyhow, I made some progress and have an easy reproducer plus example output and config information all wrapped up at http://zinser.no-ip.info/www/trans/lineb.zip No other modules but CGI are involved in the example. The version I use is 3.49. For best effect look at the output files in a terminal set to 132. The terminal output was created with perl lineb.pl terminal.txt The web output was captured with View Source from a Web browser. The Webserver in question is an OSU 3.11 server, communication betweeen the server and perl is via a local DECnet task. Greetings, Martin
Re: Spurious linebreak in non-record output
On Apr 21, 2011, at 10:47 AM, Dr. Martin P.J. Zinser wrote: Hello Craig, nobody used the dirty Oracle word around here, just because something is a table it does not have do come from a database ;-) Sorry. I promise to wash my mouth out with soap :-). Anyhow, I made some progress and have an easy reproducer plus example output and config information all wrapped up at http://zinser.no-ip.info/www/trans/lineb.zip No other modules but CGI are involved in the example. The version I use is 3.49. For best effect look at the output files in a terminal set to 132. The terminal output was created with perl lineb.pl terminal.txt The web output was captured with View Source from a Web browser. The Webserver in question is an OSU 3.11 server, communication betweeen the server and perl is via a local DECnet task. Yikes. I've never used OSU and I haven't touched a DECnet object in about 20 years, but it's pretty clear that you're getting the extra linebreak every 4096 bytes just like what was happening before with record-oriented files. This will change in 5.14.0 to every 32768 bytes, or you can edit the buffer size yourself by modifying perlio.c. You can see from the fix for that: http://perl5.git.perl.org/perl.git/commitdiff/5e7d60de18218a624f7f6d5c99bdaf0ff03973df that I limited it to regular files [S_ISREG(statbuf.st_mode) is true] with variable or variable with fixed control. It occurred to me at the time that there might be other record-oriented widgets that need similar treatment, but there is danger in casting the net too wide; binary data streams could be corrupted, for example. To me it seems unlikely that DECnet objects would always only transport text-oriented data, so it probably isn't safe to line buffer them because that would introduce a record boundary for every newline in the data stream, which is right for text data but not for binary data. So I'm not sure what the fix is, or if there is one. Using a record-oriented widget to implement a stream-oriented protocol like HTTP seems like asking for trouble, but it is what it is. The first workaround I would try would be to enable autoflushing with C$| = 1; at the top of your script. Or you could use syswrite instead of print to write your data. Craig A. Berry mailto:craigbe...@mac.com ... getting out of a sonnet is much more difficult than getting in. Brad Leithauser
Spurious linebreak in non-record output
Hello, sorry to be a pain, but here is another case of the spurious linebreak. Environment is OpenVMS IA64 8.3-1H1, Perl 5.12.3 with useperlio defined. I am using Perl with the CGI module to dynamically create a medium sized table (a few hundered rows and 6 columns). The script works fine on Perl 5.8.8 on OpenVMS Alpha. When accessing the script via a webserver, spurious linebreaks are inserted into the table, causing garbled output (e.g. a td ends up as t\nd, which also a forgiving parser will not process correctly). Running the same script in debug mode and sending the output to a file does not show the spurious linebreaks. Shorter tables generated by the same script work fine also with 5.12.3 . I've already changed the script to write the table one row at the time to the CGI output, but that this not resolve the issue. Any ideas and/or hope that this will be resolved with the soon to be released 5.14? Greetings, Martin
Re: Spurious linebreak in non-record output
On Apr 20, 2011, at 5:03 PM, Dr. Martin P.J. Zinser wrote: Hello, sorry to be a pain, but here is another case of the spurious linebreak. Environment is OpenVMS IA64 8.3-1H1, Perl 5.12.3 with useperlio defined. Since perlio is the default, do you mean that you disabled it? I am using Perl with the CGI module to dynamically create a medium sized table (a few hundered rows and 6 columns). The script works fine on Perl 5.8.8 on OpenVMS Alpha. When accessing the script via a webserver, spurious linebreaks are inserted into the table, causing garbled output (e.g. a td ends up as t\nd, which also a forgiving parser will not process correctly). Running the same script in debug mode and sending the output to a file does not show the spurious linebreaks. Shorter tables generated by the same script work fine also with 5.12.3 . I've already changed the script to write the table one row at the time to the CGI output, but that this not resolve the issue. Any ideas and/or hope that this will be resolved with the soon to be released 5.14? 5.14 is in deep code freeze at the moment, so if it's a unique problem that hasn't been reported before, no, it won't be fixed in that version. If you are writing lines longer than 4K to record-oriented files, you'll still get linebreaks at 4K intervals in 5.12.3, but 32K intervals in 5.14.0 (you can get the same effect in 5.12.3. by s/4096/32768/ in perlio.c). I can't really analyze the current problem without more information. Does the problem happen on read or write? What does SHOW DEVICE/FULL on your webserver give you? In other words, is it a mailbox or a socket or a global section or what that Perl is writing to? And are you sure that the linebreaks aren't already there when reading the data in (using what, DBD::Rdb?). How big are the I/O operations? Etc. Craig A. Berry mailto:craigbe...@mac.com ... getting out of a sonnet is much more difficult than getting in. Brad Leithauser
