----- Original Message ----- From: "Rogerio Saran" <[EMAIL PROTECTED]> To: <[EMAIL PROTECTED]> Sent: Tuesday, September 24, 2002 8:44 AM Subject: Re: Streaming RPC calls
> Martin, forget webservices. Nowadays they are all about "pull" or > "submit", and you want to "push" data. I'm not too sure why you say I'm trying to "push" data. All interaction is initiated by the client. Do I have an outdated notion of the terms push and pull? > Did you consider a messaging system? No I haven't. I thought SOAP would be the most appropriate because of the cross-platform data encoding standard. I haven't had much experience with messaging. Are there messaging systems out there that are cross-platform and language independent? I thought they were all tied to a particular platform/language. A built-in data encoding standard would also be very nice, but not essential. Do you have one in particular in mind? > > First, you probably want to chunk it to make each "delivery unit" more > manageable. Next, you want them to be delivered as chunks, so a broken > connection will not compromise the whole transfer. Finally you want it > fast so you need to open several sockets at once. I thought about this as well. It was a compromise I was willing to make to gain the benefits of soap. I would certainly want the chunking to be abstracted out of the API though. I don't want to have to worry about it on the application level. > > If you need to exchange a lot of data it does not makes sense to force > it through a single TCP connection. Most servers have a limited ability > to reach high speed in a single connection due to TCP handshaking > limitations. > > A quick and dirty recipe to transfer a zillion records of structured > data as a stream: > > a) Again, forget webservices and RPC. > This is a raw data transfer problem. That is exactly what it isn't. It's not raw data, it's structured data. It has to be understood by different systems. If I could borrrow the XML encoding of SOAP and use it in a messaging system I would be all for it, but that sounds suspiciously like SOAP again, only using a different transport protocol. > > b) Chunk your data and pack it in well formed XML documents, making them > suitable to be handled in lots of platforms. > > c) Send them through a messaging system. The simpler, the better. Is > SMTP good enough? Go for it. If you want a sophisticated, "dog wags the > tail" solution, there are also plenty of message-queue servers around. > > d) Need to ensure chunk sequence? Here you will cover your hands with > some dirt to write a transport handshake mechanism, like windows in TCP. Sequence doesn't matter too much, but the client needs to get an answer back when the entire message has been processed successfully. I did not want to have the client acting as a server as well because of firewall issues, and polling for the response is unresponsive and klunky, so SMTP is definitely out of the question. I'm certainly not committed to SOAP, but I would appreciate it if you could give me a pointer to even one example of an appropriate messaging system. Thanks Martin > > > *Saran > > Martin Jericho wrote: > > > > QUESTION 4: All I really want to do is send a large > > array of structured data in a platform independent way. > > I would like to use the standard RPC encoding of SOAP to > > avoid having to define my own XML schemas for the data, > > but I'm not sure whether today's SOAP implementations > > are mature enough to use for this purpose. What do > > other people do in this situation? I can't imagine I'm > > the only one. > > > > > > Thanks > > Martin Jericho > > > > > > > > > > >
