A while back I mentioned in passing that there were some problems with
the LWP "local/http.t" test.
Some background: the LWP local/http test creates a tiny HTTP server in
a subprocess, and does some "request/response" stuff to test both the
server and the client s/w.
I was seeing a response from the server with excess NUL bytes in between
the first line of response and the remainder. You get something like:
"Status: 404 HTTP/1.1 some error\r\n\r\n\0\0\0\0\0\0\0\0\0\0\0Date: ..."
My first impression was that the problem was in the client's "sysread"
from the network socket, but (as a packet-sniff proved) this was
incorrect, the problem was coming from the server writing to network
sockets.
When Perl opens the socket, it also does (paraphrasing):
fp_read = fdopen(socket,"r");
fp_write= fdopen(socket,"w");
so that you can use stdio routines to read or write from the socket.
The writes are done with several "print" statements (via Perl's
do_print routine, which calls "my_fwrite" in VMS.C), followed by
flushes ($| = 1, both explicitly and implicitly since it is to a
socket). The routine my_fwrite uses fputs and fputc to write strings
and NUL bytes, although in this case only the fputs is used: there's
no NULs in the string that my_fwrite is requested to output.
Everything going INTO the fputs routine is fine. But the extra NULs
come out the other end.
If the fputs routine is replaced by a write(fileno(fp_write),...) it works
fine, so I think what is happening is a mismatch between the buffering
of "stdio" and "unixio" in the CRTL.
I've been able to duplicate this effect in C, with no Perl
involvement...so while Perl brought this problem to light, the CRTL
seems to be the cause. The test program in C shows the problem on my system
(VMS/AXP 6.2-1H3, DECC 5.5-002, UCX 4.1) and on a more up-to-date one
also (VMS/AXP 7.1, DECC 5.7-004, Multinet 4.0B).
One big glaring hint of this is that the test program (below) reads a
38 byte request from it's "server" socket, then writes a smaller
response to the socket... which is transmitted as a 38 byte response
with the NUL padding.
I sure hope someone at Compaq/DEC is listening and picks this up; that
would be the "right way" to fix the problems mentioned.
Okay, I since it's not just my old system that has this problem, and
since it makes it very difficult to effectively write network services,
some sort of workaround is in order.
I can see how to do it, but it's really painful....in my_fwrite:
if (fstat(fileno(dest),&s) == 0 && S_ISSOCK(s.st_mode)) {
write(fileno(dest),src,nitm*itmsz);
} else {
...the code currently in my_fwrite...
}
This works, but that damn "fstat" has to be done for *every* *single*
print...that's a performance hit we don't really need. (my_fwrite
doesn't know when the FILE * has been closed and reopened, and yes they
do get reused).
Doing a "write" instead of "fputs" for non-socket prints doesn't work.
Lots of test suite failures.
You have to go up several levels in the call sequence to find the SV that
knows you're writing to a socket, so having different low-level routines
to handle socket/non-socket writes is problematic.
Keeping a list of "socket file#'s" is possible, but you have to insert
updating code in the socket connect/accept and close code. And it would
be an awful kludge.
Anyone have some other ideas?
Test program follows...it has both server and client ... if you see a bunch
of \000's after the first line of response, then you've got the bug too.
--------------------------------------------------------------------------
/* demonstrate stdio + socket bug in client/server */
/* server write is padded with NUL bytes if bug is present*/
#include <types.h>
#include <ctype.h>
#include <socket.h>
#include <in.h>
#include <netdb.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <unixio.h>
#define TEST_PORT 1234
int s_sock;
int c_sock;
int port;
void
server_init(void)
{
struct sockaddr_in sin;
unsigned int n = sizeof(sin);
s_sock = socket(AF_INET,SOCK_STREAM,0);
if (s_sock <= 0) exit(1);
sin.sin_family = AF_INET;
sin.sin_addr.s_addr = INADDR_ANY;
sin.sin_port = 0;
if (bind(s_sock, (struct sockaddr *)&sin, sizeof(sin)) < 0) exit (1);
if (listen(s_sock, 1) < 0) exit(1);
getsockname(s_sock, (struct sockaddr *)&sin, &n);
port = ntohs(sin.sin_port);
printf("server listening on port %d\n",port);
}
void
server_respond(void)
{
struct sockaddr sin2;
unsigned int n = sizeof(sin2);
int s, nr, j;
FILE *fpr, *fpw;
char rbuf[128];
s = accept(s_sock, &sin2, &n);
if (s <= 0) exit(1);
fpr = fdopen(s,"r"); /* open stdio form, just like Perl does */
fpw = fdopen(s,"w");
nr = read(s, rbuf, sizeof(rbuf)-1);
if (nr <= 0) exit(1);
for (j = 0; j < nr; j++) {
if (iscntrl(rbuf[j])) rbuf[j] = '.';
}
rbuf[nr] = '\0';
printf("server read %d bytes\ngot: '%s'\n",nr,rbuf);
/* use fputs with frequent flushes, just like Perl does */
fputs("Status: 404 HTTP/1.1\r\n",fpw);
fflush(fpw);
fsync(fileno(fpw));
fputs("Blah blah ",fpw);
fflush(fpw);
fsync(fileno(fpw));
fputs("blah, this is a test & stuff \r\n",fpw);
fflush(fpw);
fsync(fileno(fpw));
fputs("Four score and 7 years ago.\r\n",fpw);
fflush(fpw);
fsync(fileno(fpw));
/* then close the stdio stuff, just like Perl does */
fclose(fpw);
}
void
client_request(void)
{
struct sockaddr_in sin;
struct hostent *hp;
char wbuf[] = "GET /not_found_stuff_here HTTP/1.0\r\n\r\n";
int nw;
hp = gethostbyname("localhost");
if (!hp) exit(1);
c_sock = socket(AF_INET,SOCK_STREAM,0);
if (!c_sock) exit(1);
sin.sin_family = AF_INET;
memcpy(&sin.sin_addr, hp->h_addr, hp->h_length);
sin.sin_port = htons(port);
if (connect(c_sock, (struct sockaddr *)&sin, sizeof(sin)) < 0) exit(1);
nw = write(c_sock,wbuf,strlen(wbuf));
printf("client wrote %d bytes request\n",nw);
}
void
client_read_response(void)
{
int nr, j;
char rbuf[200], sbuf[4000], *p;
nr = read(c_sock, rbuf, sizeof(rbuf));
p = sbuf;
for (j = 0; j < nr; j++) {
if (iscntrl(rbuf[j])) {
sprintf(p,"\\%03.3o",rbuf[j]);
p += 4;
} else {
*p++ = rbuf[j];
}
}
*p = '\0';
printf("client read %d bytes\ngot: '%s'\n",nr,sbuf);
}
main()
{
server_init();
client_request();
server_respond();
client_read_response();
}
--------------------------------------------------------------------------
--
Drexel University \V --Chuck Lane
----------------->--------*------------<[EMAIL PROTECTED]
(215) 895-1545 / \ Particle Physics [EMAIL PROTECTED]
FAX: (215) 895-5934 /~~~~~~~~~~~ [EMAIL PROTECTED]
