Mark, What if the server is so busy you can't connect to it to get that message back?
We have three cases: 1) Everything is happy. 2) Things are really busy. 3) A million pounds of (fill in whatever you'd like) in a five pound bag. Things can go from #1 to #3 quickly if the project is big, and funding is low (which is the goal, to bring distributed computing to projects with tiny funding). So, while it would be ideal for the servers to be able to say "no, and please slow way, way down" I think we have to assume that all of the project's servers are unreachable under an extreme load. That is when the need is most extreme, and the hardest to deal with, so that's the only one I'm really thinking about. -- Lynn Mark Pottorff wrote: > It would seem that having the server be able to accept a connection, and then > direct the client on any preferred backoff would be ideal. The server is in > the best position to know how long it was down, how long it will take to > recover to normal bw levels, how many active connections it already has, > whether deadlines have been extended etc. etc. > > The client is in the best position to know how frequently it has an internet > connection available, what the available bw to the server tends to be, how > much data it has to send, etc. > > If the upload server could respond to requests with a message, to the effect > of "I'm trying to focus only on uploads that are within 12 hours of deadline" > and negotiate with the client as to whether the uploads are within that > objective, whether it expects to have an internet connection available later, > number of files to upload etc. then it should be possible to arrive at a > single backoff to a point in time that the server will be available to handle > the request at full speed. "OK, we'll upload one file now and hit you in 3hrs > 12 minutes with the other 4." > > As the server gets caught up, and bw utilization begins to drop, or the > server sees holes in the commitments it has made, the window extends on the > uploads out to 24hrs from deadline etc. > > When an upload completes, a confirmation is made that it arrived with all of > the data. It would seem an ideal time for an overloaded server to convey that > it would prefer to delay any further uploads. Or establish a simple 1-5 > system expressing how busy it is. The client then weighs time to deadline > against likelihood of future internet connection to assess whether it has an > upload of sufficient priority or not. And if not, it delays starting next > upload. > > Really the objective is that all result files across the user base get > uploaded in priority order. Where priority takes in to account bandwidth user > and project have available, deadline, file size, etc. > > Same is true for downloads. If server is drowning in requests, you want to be > servicing the starving clients first, not those with full-time networks and > fat work caches. And only to the extent that they aren't starving anymore, > until the contention is resolved. > > Having a machine sitting idle waiting for downloads can occur, and the order > of downloads isn't optimized either. One more file and I'll be able to start > processing a task, but that's not necessarily the file being downloaded right > now. Just depends on the backoffs it was assigned etc. as retries occurred > and connections were satisfied. > > > Running Microsoft's "System Idle Process" will never help cure cancer, > AIDS nor Alzheimer's. But running rose...@home just might! > http://boinc.bakerlab.org/rosetta/ > > > > _______________________________________________ > boinc_dev mailing list > [email protected] > http://lists.ssl.berkeley.edu/mailman/listinfo/boinc_dev > To unsubscribe, visit the above URL and > (near bottom of page) enter your email address. > _______________________________________________ boinc_dev mailing list [email protected] http://lists.ssl.berkeley.edu/mailman/listinfo/boinc_dev To unsubscribe, visit the above URL and (near bottom of page) enter your email address.
