I think usually large data sets like yours are returned via ftp or http file download. You would create the file on the server and do something like a redirect to initiate the download. If it's really big and will take a while to create the file, you could email a download link to the client. Doing it either way will allow the client to restart the download if it gets interrupted. And depending on what client they are using, the client may even restart where it left off.

The O'Reilly Safari Bookshelf works this way. If I want to download a chapter from a book I have on my bookshelf, I "order" it and I'll get an email when the pdf file of the chapter is ready. Doing it this way will give you the added advantage of controlling load on your server, like only allowing 5 outstanding report requests at a time.

On May 16, 2005, at 4:48 AM, Thomas Sundberg wrote:

Hi!

Does anybody have any thoughts regarding large result sets that should be
sent to a client using http or similar stateless protocol. I have a
situation where I need to be able to return a large result set and where I
know that either the client or possible the server may fail due to resource
limitations, out of memory etc.


The result set will be fetched from a database, Java objects will be created
in a collection, the collection will be returned through a network to a
client. This collection may be to large so either the server or the client
will fail. I can limit the number of values returned, but how should the
problem be handled when all values must be returned?


My thought is something along the lines of this:
1 If the result set is to large, chunk it into smaller parts. Return a
object to the client with the collection and a info part containing the
total number of values, the chunk size and next row number.
2 The client request next chunk of data from next row number.
3 Repeat step 2 until the client stops requesting data, i.e. has received
all data wanted.


I assume that somebody has solved this issue and in hoping the he or she
even want to share the conceptual idea with the rest of us.


Since this is of topic, please consider replying only to me.

/Thomas


--
MySQL General Mailing List
For list archives: http://lists.mysql.com/mysql
To unsubscribe: http://lists.mysql.com/[EMAIL PROTECTED]



--
Brent Baisley
Systems Architect
Landover Associates, Inc.
Search & Advisory Services for Advanced Technology Environments
p: 212.759.6400/800.759.0577


-- MySQL General Mailing List For list archives: http://lists.mysql.com/mysql To unsubscribe: http://lists.mysql.com/[EMAIL PROTECTED]



Reply via email to