Hi John, Thanks very much for this. I think you're right, we need to first try tuning our code and our container (we've put very little effort into this so far). However, although we "only" need thousands of threads now, I can conceive of a time when we might need more than this. My attraction to Restlet was more the neat programming model and the abstraction from the underlying HTTP connector, which will keep us flexible. Any efficiency/scalability gain would be a bonus. In the absence of a genuine asynchronous i/o solution we'll stay with what we have for now but monitor developments closely.
> Cool. Let me restate to make sure we're on the same page with the details... Yes, all of your statements were correct. Now for your questions: > * Since you ask about resuming downloads... What does that imply about > the back-end data storage of the simulation data? I.e., do the > simulators actually save the data in a long term, persistent storage > or are they treated as primarily transient data (with perhaps some > sliding window of persistence before being purged or is it only after > a client has acknowledged complete receipt of each data set)? A data file is kept on the server until the client (the owner of the particular simulation run) explicitly deletes it. (This happens when the client has verified that the file is completely downloaded and not corrupted.) > * What kind of server hardware are you running? For the sake of the > rest of this post, I'll assume something "average" these days (2-4GB > RAM, (dual-) dual-core at > 2GHz, plenty of local disk for any > potential transient storage, if necessary). Yep, average commodity server hardware. > * Similarly, what OS platform are you running on? Any modern release > of Linux v2.6 is a simple, no-brainer to support what you need for > this. Yes again, Linux 2.6. > * What problems, exactly, did you run with your servlet solution? > You've mentioned the thread exhaustion problem twice at only a few > hundred threads (which is pathetically low for what you're talking > about needing to do). I.e., did you do any tuning of the servlet > container to provide for more threads (including reducing the default > stack size)? As mentioned above, we've done little tuning. It's more the principle of having to start up a new thread for each connection that bugged me, although with our current requirements this might be OK as you say. I'm trying to assess whether moving to a Restlet approach is worthwhile - my original question was motivated by a desire to be able to scale up from our current requirements and grow as we need to as I wasn't convinced that our current solution scales well. > Hope this helps, Yes indeed! Thanks again, Jon -- -------------------------------------------------------------- Dr Jon Blower Tel: +44 118 378 5213 (direct line) Technical Director Tel: +44 118 378 8741 (ESSC) Reading e-Science Centre Fax: +44 118 378 6413 ESSC Email: [EMAIL PROTECTED] University of Reading 3 Earley Gate Reading RG6 6AL, UK --------------------------------------------------------------
