Hm... If you are using http, do you have the content size set in the header? This is basically an issue on transport layer and I believe that is why the length of data is not in OAGIS BODs.
Derek > -----Original Message----- > From: Jeff Fisher [mailto:[EMAIL PROTECTED] > Sent: Wednesday, November 26, 2003 1:03 PM > To: jdjlist > Subject: [jdjlist] Re: Reliable TCP/IP stream reading > > > Hal & Derek, > > Actually, we used to do that when we had a custom protocol > for data sent on > the wire. The big endian, little endian thing was a bitch, > but we didn't > have a problem. As our processes expanded to include more > web service type > of stuff and we converted to xml as a the data format. When > we did that, we > removed the length of the data being sent. Part of that was > to accommodate > using the OAGIS BODs which don't have a place for the length > of the data > being sent. > > So the short answer is yes, that would solve the problem. We > can't use it > because we don't have complete control over what's being sent. > > btw, for everyone else that's reading, closing the clients > output stream > does cause the -1 to be returned for bytes read. It also has > the nasty side > affect, in Java anyway, of closing the client socket as well. > Go figure.... > > I set the soTimeout to be 75ms and that seems to work pretty > well. So at > this point we check for the -1 and if we don't get it, the > most we block is > 75ms. This also takes care of reading data faster off of the > stream than > it's being sent. > > At some point we'll get switched over to a real web server/web service > server and hopefully won't have to deal with this. > > -----Original Message----- > From: Hal Humphrey [mailto:[EMAIL PROTECTED] > Sent: Wednesday, November 26, 2003 2:53 PM > To: jdjlist > Subject: [jdjlist] Re: Reliable TCP/IP stream reading > > > I think I have always included in the header the size of the > data being > sent (not TCPIP header, but the "header" (first data) you receive from > the client). Would the client sending the size of the data being > transferred solve the problem? > --- Jeff Fisher <[EMAIL PROTECTED]> wrote: > > Thanks for the suggestions. Since we are using a buffered input > > stream, we > > are not garunteed that we will get a full buffer on each > read. Also, > > there > > is the situation where a client can send over full buffers each time > > but the > > data is only part of the buffer. Lastly there is the > situation where > > the > > data is exactly the size of a buffer causing another read and a > > block. So > > checking for an amount read being less than a buffer does not work. > > This > > has been verified in testing as well. > > > > Since the client applications are not just java and we send data > > back, I > > couldn't use the suggestion from Dennis. But, it did get me looking > > at the > > Socket class and as such I've set the soTimeout on the socket and > > then catch > > the exception and end the read. While that's not perfect, it does > > seem to > > do the trick. > > > > Thanks > > > > Jeff > > > > -----Original Message----- > > From: Dennis DiMaria [mailto:[EMAIL PROTECTED] > > Sent: Tuesday, November 25, 2003 2:21 PM > > To: jdjlist > > Subject: [jdjlist] Re: Reliable TCP/IP stream reading > > > > > > I couldn't find the class BytesMessage in the 1.3 API. Is it part of > > an optional package? > > > > Normally, reading less data than you asked for does not indicate a > > socket stream will not send more data in a future read. It depends > > on how the tcp/ip stack chops up data into packets on the > "send" side > > and how they are then read on the "receive" side." > > > > The sender could run the shutdownOutput() method (SocketImpl class) > > to > > indicate it's finished sending. It can also close() the > socket. Until > > one of these are done there's no guarantee that the sender is > > finished. > > > > -Dennis D > > > > [EMAIL PROTECTED] wrote: > > > > > > > > > > > > Jeff, > > > > > > Instead of waiting for the number of bytes read to be -1, identify > > when > > the > > > byte buffer has not been completely filled. When this condition > > occurs > > you > > > know you have reached the end of your xml document otherwise the > > call to > > > read() would block until more data is received to fill > the buffer. > > I've > > > done this when reading XMLDocuments from a JMS ByteMessage but it > > will > > > behave in the same way for a tcp/ip stream. > > > > > > int MESSAGE_BUFFER_LENGTH = 500; > > > > > > private byte[] buffer = new byte[MESSAGE_BUFFER_LENGTH]; > > > StringBuffer stringBuffer = new StringBuffer(); > > > BytesMessage message = (BytesMessage)msg; > > > int length = 0; > > > > > > do > > > { > > > length = message.readBytes(m_buffer); > > > if (length > 0) > > > { > > > String read = new String(m_buffer, 0, length, > > "US-ASCII"); > > > stringBuffer.append(read); > > > } > > > } > > > while(length == MESSAGE_BUFFER_LENGTH); > > > > > > Hope this helps, > > > HOWARD GAISFORD > > > > > > > > ... > > > > > > Is there a reliable way to read from a tcp/ip stream that will > > identify > > > when there is nothing left on the stream to read? > > > > > > We have been using the following code > > > > > > public int readData(StringBuffer buf) throws IOException > > > { > > > byte[] InBuf = new byte[2048]; > > > int totBytes = 0; > > > int bytesRead = 0; > > > > > > // > > > // Sanity check > > > if ( buf != null ) { > > > buf.delete(0, buf.length()); > > > > > > Arrays.fill(InBuf, (byte)0); > > > bytesRead = m_In.read(InBuf, 0, 2048); > > > > > > while ( bytesRead != -1 ) { > > > totBytes += bytesRead; > > > buf.append(new String(InBuf, 0, bytesRead)); > > > Arrays.fill(InBuf, (byte)0); > > > bytesRead = m_In.read(InBuf, 0, 2048); > > > } > > > } > > > > > > return totBytes; > > > } > > > > > > This has worked great until recently. We have been sending over > > larger > > xml > > > documents than we had before and now this code does not work. > > > Specifically, read does not return -1 even though there is nothing > > left on > > > the stream to read. This causes the code to block. If we use > > > m_In.available(), that returns a bogus 0 and we only get a partial > > > document. Besides, the available() function is not reliable based > > on the > > > java documentation. > > > > > > There seems to be some size threshold here where these fail. > > > > > > I've checked on the web, but almost exclusivly I've seen code > > similar to > > > the above. Does anyone have any suggestions? > > > > > > Thanks > > > > > > Jeff Fisher > > > --- > > > > > > --- > > You are currently subscribed to jdjlist as: > > [EMAIL PROTECTED] > > To unsubscribe send a blank email to > > [EMAIL PROTECTED] > > http://www.sys-con.com/fusetalk > > To unsubscribe from all mailing lists, click: > > > http://sys-con.com/globaldelete.cfm?email=leave-jdjlist-22325S @mailbox.sys-c > on.com > > --- > You are currently subscribed to jdjlist as: [EMAIL PROTECTED] > To unsubscribe send a blank email to > [EMAIL PROTECTED] > http://www.sys-con.com/fusetalk > To unsubscribe from all mailing lists, click: > http://sys-con.com/[EMAIL PROTECTED] on.com __________________________________ Do you Yahoo!? Free Pop-Up Blocker - Get it now http://companion.yahoo.com/ --- You are currently subscribed to jdjlist as: [EMAIL PROTECTED] To unsubscribe send a blank email to [EMAIL PROTECTED] http://www.sys-con.com/fusetalk To unsubscribe from all mailing lists, click: http://sys-con.com/[EMAIL PROTECTED] on.com --- You are currently subscribed to jdjlist as: [EMAIL PROTECTED] To unsubscribe send a blank email to [EMAIL PROTECTED] http://www.sys-con.com/fusetalk To unsubscribe from all mailing lists, click: http://sys-con.com/[EMAIL PROTECTED] --- You are currently subscribed to jdjlist as: [EMAIL PROTECTED] To unsubscribe send a blank email to [EMAIL PROTECTED] http://www.sys-con.com/fusetalk To unsubscribe from all mailing lists, click: http://sys-con.com/[EMAIL PROTECTED]
