Haha, I received the following after sending the reply and is wondering which one is the dirty word. :)
Derek Trend SMEX Content Filter has detected sensitive content. Place = jdjlist; ; ; jdjlist Sender = Derek Ngok Subject = [jdjlist] Re: Reliable TCP/IP stream reading Delivery Time = November 26, 2003 (Wednesday) 17:49:50 Policy = Dirty Words\Sexual Discrimination Action on this mail = Quarantine message Warning message from administrator: Sender, ScanMail filter has detected possible inappropriate material or language in your email. This email has not been delivered. > -----Original Message----- > From: Derek Ngok > Sent: Wednesday, November 26, 2003 2:50 PM > To: jdjlist > Subject: [jdjlist] Re: Reliable TCP/IP stream reading > > > Hm... If you are using http, do you have the content size set > in the header? This is basically an issue on transport layer > and I believe that is why the length of data is not in OAGIS BODs. > > Derek > > > -----Original Message----- > > From: Jeff Fisher [mailto:[EMAIL PROTECTED] > > Sent: Wednesday, November 26, 2003 1:03 PM > > To: jdjlist > > Subject: [jdjlist] Re: Reliable TCP/IP stream reading > > > > > > Hal & Derek, > > > > Actually, we used to do that when we had a custom protocol > > for data sent on > > the wire. The big endian, little endian thing was a bitch, > > but we didn't > > have a problem. As our processes expanded to include more > > web service type > > of stuff and we converted to xml as a the data format. When > > we did that, we > > removed the length of the data being sent. Part of that was > > to accommodate > > using the OAGIS BODs which don't have a place for the length > > of the data > > being sent. > > > > So the short answer is yes, that would solve the problem. We > > can't use it > > because we don't have complete control over what's being sent. > > > > btw, for everyone else that's reading, closing the clients > > output stream > > does cause the -1 to be returned for bytes read. It also has > > the nasty side > > affect, in Java anyway, of closing the client socket as well. > > Go figure.... > > > > I set the soTimeout to be 75ms and that seems to work pretty > > well. So at > > this point we check for the -1 and if we don't get it, the > > most we block is > > 75ms. This also takes care of reading data faster off of the > > stream than > > it's being sent. > > > > At some point we'll get switched over to a real web > server/web service > > server and hopefully won't have to deal with this. > > > > -----Original Message----- > > From: Hal Humphrey [mailto:[EMAIL PROTECTED] > > Sent: Wednesday, November 26, 2003 2:53 PM > > To: jdjlist > > Subject: [jdjlist] Re: Reliable TCP/IP stream reading > > > > > > I think I have always included in the header the size of the > > data being > > sent (not TCPIP header, but the "header" (first data) you > receive from > > the client). Would the client sending the size of the data being > > transferred solve the problem? > > --- Jeff Fisher <[EMAIL PROTECTED]> wrote: > > > Thanks for the suggestions. Since we are using a buffered input > > > stream, we > > > are not garunteed that we will get a full buffer on each > > read. Also, > > > there > > > is the situation where a client can send over full > buffers each time > > > but the > > > data is only part of the buffer. Lastly there is the > > situation where > > > the > > > data is exactly the size of a buffer causing another read and a > > > block. So > > > checking for an amount read being less than a buffer does > not work. > > > This > > > has been verified in testing as well. > > > > > > Since the client applications are not just java and we send data > > > back, I > > > couldn't use the suggestion from Dennis. But, it did get > me looking > > > at the > > > Socket class and as such I've set the soTimeout on the socket and > > > then catch > > > the exception and end the read. While that's not perfect, it does > > > seem to > > > do the trick. > > > > > > Thanks > > > > > > Jeff > > > > > > -----Original Message----- > > > From: Dennis DiMaria [mailto:[EMAIL PROTECTED] > > > Sent: Tuesday, November 25, 2003 2:21 PM > > > To: jdjlist > > > Subject: [jdjlist] Re: Reliable TCP/IP stream reading > > > > > > > > > I couldn't find the class BytesMessage in the 1.3 API. Is > it part of > > > an optional package? > > > > > > Normally, reading less data than you asked for does not indicate a > > > socket stream will not send more data in a future read. It depends > > > on how the tcp/ip stack chops up data into packets on the > > "send" side > > > and how they are then read on the "receive" side." > > > > > > The sender could run the shutdownOutput() method > (SocketImpl class) > > > to > > > indicate it's finished sending. It can also close() the > > socket. Until > > > one of these are done there's no guarantee that the sender is > > > finished. > > > > > > -Dennis D > > > > > > [EMAIL PROTECTED] wrote: > > > > > > > > > > > > > > > > Jeff, > > > > > > > > Instead of waiting for the number of bytes read to be > -1, identify > > > when > > > the > > > > byte buffer has not been completely filled. When this condition > > > occurs > > > you > > > > know you have reached the end of your xml document otherwise the > > > call to > > > > read() would block until more data is received to fill > > the buffer. > > > I've > > > > done this when reading XMLDocuments from a JMS > ByteMessage but it > > > will > > > > behave in the same way for a tcp/ip stream. > > > > > > > > int MESSAGE_BUFFER_LENGTH = 500; > > > > > > > > private byte[] buffer = new byte[MESSAGE_BUFFER_LENGTH]; > > > > StringBuffer stringBuffer = new StringBuffer(); > > > > BytesMessage message = (BytesMessage)msg; > > > > int length = 0; > > > > > > > > do > > > > { > > > > length = message.readBytes(m_buffer); > > > > if (length > 0) > > > > { > > > > String read = new String(m_buffer, 0, length, > > > "US-ASCII"); > > > > stringBuffer.append(read); > > > > } > > > > } > > > > while(length == MESSAGE_BUFFER_LENGTH); > > > > > > > > Hope this helps, > > > > HOWARD GAISFORD > > > > > > > > > > > ... > > > > > > > > Is there a reliable way to read from a tcp/ip stream that will > > > identify > > > > when there is nothing left on the stream to read? > > > > > > > > We have been using the following code > > > > > > > > public int readData(StringBuffer buf) throws IOException > > > > { > > > > byte[] InBuf = new byte[2048]; > > > > int totBytes = 0; > > > > int bytesRead = 0; > > > > > > > > // > > > > // Sanity check > > > > if ( buf != null ) { > > > > buf.delete(0, buf.length()); > > > > > > > > Arrays.fill(InBuf, (byte)0); > > > > bytesRead = m_In.read(InBuf, 0, 2048); > > > > > > > > while ( bytesRead != -1 ) { > > > > totBytes += bytesRead; > > > > buf.append(new String(InBuf, 0, bytesRead)); > > > > Arrays.fill(InBuf, (byte)0); > > > > bytesRead = m_In.read(InBuf, 0, 2048); > > > > } > > > > } > > > > > > > > return totBytes; > > > > } > > > > > > > > This has worked great until recently. We have been sending over > > > larger > > > xml > > > > documents than we had before and now this code does not work. > > > > Specifically, read does not return -1 even though there > is nothing > > > left on > > > > the stream to read. This causes the code to block. If we use > > > > m_In.available(), that returns a bogus 0 and we only > get a partial > > > > document. Besides, the available() function is not > reliable based > > > on the > > > > java documentation. > > > > > > > > There seems to be some size threshold here where these fail. > > > > > > > > I've checked on the web, but almost exclusivly I've seen code > > > similar to > > > > the above. Does anyone have any suggestions? > > > > > > > > Thanks > > > > > > > > Jeff Fisher > > > > --- > > > > > > > > > --- > > > You are currently subscribed to jdjlist as: > > > [EMAIL PROTECTED] > > > To unsubscribe send a blank email to > > > [EMAIL PROTECTED] > > > http://www.sys-con.com/fusetalk > > > To unsubscribe from all mailing lists, click: > > > > > http://sys-con.com/globaldelete.cfm?email=leave-jdjlist-22325S > @mailbox.sys-c > > on.com > > > > --- > > You are currently subscribed to jdjlist as: [EMAIL PROTECTED] > > To unsubscribe send a blank email to > > [EMAIL PROTECTED] > > http://www.sys-con.com/fusetalk > > To unsubscribe from all mailing lists, click: > > > http://sys-con.com/globaldelete.cfm?email=leave-jdjlist-22325S @mailbox.sys-c on.com __________________________________ Do you Yahoo!? Free Pop-Up Blocker - Get it now http://companion.yahoo.com/ --- You are currently subscribed to jdjlist as: [EMAIL PROTECTED] To unsubscribe send a blank email to [EMAIL PROTECTED] http://www.sys-con.com/fusetalk To unsubscribe from all mailing lists, click: http://sys-con.com/[EMAIL PROTECTED] on.com --- You are currently subscribed to jdjlist as: [EMAIL PROTECTED] To unsubscribe send a blank email to [EMAIL PROTECTED] http://www.sys-con.com/fusetalk To unsubscribe from all mailing lists, click: http://sys-con.com/[EMAIL PROTECTED] --- You are currently subscribed to jdjlist as: [EMAIL PROTECTED] To unsubscribe send a blank email to [EMAIL PROTECTED] http://www.sys-con.com/fusetalk To unsubscribe from all mailing lists, click: http://sys-con.com/[EMAIL PROTECTED] --- You are currently subscribed to jdjlist as: [EMAIL PROTECTED] To unsubscribe send a blank email to [EMAIL PROTECTED] http://www.sys-con.com/fusetalk To unsubscribe from all mailing lists, click: http://sys-con.com/[EMAIL PROTECTED]
