Hi Anthony, There is no chunking ability in Thrift in terms of application level consumption. If you choose to use the TBufferedTransport, the data transfer between the client and server will be chunked, but only at the transport level i.e. the server will flush its internal transport buffer once it has reached a certain size. However, the client will not pass the data to the application layer until it has received all the data from the server. Supporting a streaming protocol natively would require major changes to the server interface so I think it makes more sense to implement this on the application level.
Cheers, - Aditya -----Original Message----- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] Sent: Sunday, November 09, 2008 7:48 PM To: [email protected] Subject: Large Data Transfers Hello, There is one thing I can't determine as it may not be part of the original design of thrift, but it is a common in network communication. I would like to transfer large amounts of data, 5-10 megs of data, in such away that the client can consume it from a buffer. I assume using a thrift function, if I send the entire chunk all at once, the client side will need to wait until the entire transfer is complete. I would rather be able to set a client / server buffer size,and left Thrift fill the buffer for me while my client consumes it in a FIFO manner. I assume this functionality is in the details of the thrift implementation in some form already, but is there a way for client code to hook into it? Or is there another way that thrift deals with large data chunks. I do have an alternative, and that is to make a thrift function that requests the data in blocks. Before i go this way, I am curious as to how other thrift users would go about implementing this functionality. Thanks. -- -Anthony
