Projects that use next_rpc_delay need to monitor their server load and adjust the value accordingly. -- David
On 10-Aug-2012 12:02 PM, Richard Haselgrove wrote: > Provided, that is, that the project with the low next_rpc_delay value provides > sufficient server resources to manage the workflow expeditiously. > I've been noticing recently that the SETI servers can commonly take up to 90 > seconds to respond to a routine RPC. While waiting for that reply, the client > appears to be inhibited from initiating an RPC to another project. > Nobody would suggest that SETI should ever consider going low-latency, but I'm > using the example to question whether the combination of low-latency and a > sluggish server could lead to the sort of problems for other projects that > John > described. > > *From:* David Anderson <[email protected]> > *To:* [email protected] > *Sent:* Friday, 10 August 2012, 19:43 > *Subject:* Re: [boinc_dev] Changes to client scheduler from 6.12.33 to > 7.0.25? > > There's nothing wrong with a small next_rpc_delay value, e.g. 1 minute. > This will have no effect on other projects. > > The semantics of this are: > - the client issues a scheduler RPC to the project every 1 minute > - If the project has the highest scheduling priority, > this RPC will include work requests for resource types > that are not above the max buffer limit > (regardless of per-resource backoff). > > It a project (like Patricio's) has sporadic job availability, > then it will typically have highest scheduling priority. > However, as it processes jobs the priority will decrease, > so it won't block out other project. > > -- David > > On 10-Aug-2012 7:36 AM, Richard Haselgrove wrote: > > I would hope, and expect, that if anyone set an extreme low-latency > value > > like a next_rpc_delay of 1 minute or less on a public-facing volunteer > > project, peer pressure from other projects would rapidly force them to > change > > their mind. > > > > But as the front page of the website makes clear, BOINC is also > designed to > > work as a virtual campus supercomputer centre, or as a company desktop > grid. > > Under those circumstances, the risk of blocking other projects is > lessened, > > and I think BOINC should work as described in the Wiki. > > > > > >> ________________________________ From: "[email protected] > <mailto:[email protected]>" > >> <[email protected] <mailto:[email protected]>> To: Patricio > Vidal <[email protected] <mailto:[email protected]>> > >> Cc: [email protected] <mailto:[email protected]>; > [email protected] > <mailto:[email protected]> Sent: > >> Friday, 10 August 2012, 15:27 Subject: Re: [boinc_dev] Changes to > client > >> scheduler from 6.12.33 to 7.0.25? > >> > >> There is still no point in the connection if the client is not going > to > >> ask for work from the project anyway. > >> > >> The observation is that it would only take a very small number of > >> low-latency projects to starve all other projects of connections if > the > >> connection period to the server is short enough. > >> > >> For example, if a connection takes 15 seconds and there are 4 projects > >> with a server specified connection period of a minute, then no other > >> projects will ever be contacted for anything as there will always be a > >> pending connection to a low-latency project. If these connections > only > >> happen when the client would ask for work from the project if it were > >> contactable, then the situation improves greatly. It can still be > the case > >> where all 4 rise to the top of the list of projects from which the > client > >> wants work - in that case, the client will get no work from anywhere > else > >> until one of these projects has supplied work. > >> > >> jm7 > >> > >> > >> |------------> | From: | |------------> > >>> > > --------------------------------------------------------------------------------------------------------------------------------------------------| > >> > >>> > |Patricio Vidal <[email protected] > <mailto:[email protected]>> > | > >>> > > --------------------------------------------------------------------------------------------------------------------------------------------------| > >> > >>> > |------------> > >> | To: | |------------> > >>> > > --------------------------------------------------------------------------------------------------------------------------------------------------| > >> > >>> > |<[email protected] <mailto:[email protected]>> > | > >>> > > --------------------------------------------------------------------------------------------------------------------------------------------------| > >> > >>> > |------------> > >> | Date: | |------------> > >>> > > --------------------------------------------------------------------------------------------------------------------------------------------------| > >> > >>> > |08/10/2012 10:04 AM > | > >>> > > --------------------------------------------------------------------------------------------------------------------------------------------------| > >> > >>> > |------------> > >> | Subject: | |------------> > >>> > > --------------------------------------------------------------------------------------------------------------------------------------------------| > >> > >>> > |Re: [boinc_dev] Changes to client scheduler from 6.12.33 to 7.0.25? > | > >>> > > --------------------------------------------------------------------------------------------------------------------------------------------------| > >> > >>> > |------------> > >> | Sent by: | |------------> > >>> > > --------------------------------------------------------------------------------------------------------------------------------------------------| > >> > >>> > |<[email protected] > <mailto:[email protected]>> > | > >>> > > --------------------------------------------------------------------------------------------------------------------------------------------------| > >> > >> > >> > >> > >> > >> > >>> > In my previous email I meant to say ".. we can't have clients waiting > >> hours..." > >> > >> I just found in the wiki that the next_rpc_delay is exactly designed > for > >> my purpose: http://boinc.berkeley.edu/trac/wiki/LowLatency# > >> > >> Based on the wiki it seems to me the clients should contact the server > >> using the rpc_delay regardless they have work from the project. > >> > >> Patricio. > >> > >> > >> > >> > >> yoyo <yoyo@mailueberfall .de> To Patricio Vidal 08/10/2012 01:21 AM > >> <[email protected] <mailto:[email protected]>> cc > [email protected] <mailto:[email protected]> > >> > >> Subject Re: [boinc_dev] Changes to client scheduler from 6.12.33 to > >> 7.0.25? > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> The client doesn't wait for hours if the server has work. Even not if > the > >> workunits are very short. Before the client runs out of work it > fetches > >> new ones. Additional you can configure on the server, that the client > >> should periodic, e.g. every hour, connect to the server. I use it and > let > >> the client connect at least every 5h. Was this next_epc_delay removed > from > >> trunc? I use the client connect to check if a workunit which is in the > >> queue of the client can be deleted. > >> > >> yoyo > >> > >> Patricio Vidal schrieb: Yes, we need that functionality. Our jobs > last from > >> minutes to few hours so we can have the clients waiting hours to > contact > >> the server. > >> > >> Is there a reason why the client blocks the other projects if it has a > >> very short request cycle and it does not have work? I would think > that if > >> it doesn't has work it should jump to the next project in the list. > >> > >> This functionality is critical for any project that has relative short > >> (few hours) jobs and it needs the results back as soon as possible. I > >> guess this usage is more typical of Boinc projects deployments in > private > >> networks. > >> > >> Would it make sense to add another option to the configuration file > for > >> this behavior? Something like <next_work_request_rpc_delay> ? Or just > >> restore part of the behavior of <next_rpc_delay> ? The old behavior > has > >> been that way for years, right? Is it a issue with other projects so > that > >> the behavior was changed? > >> > >> Patricio. > >> > >> > >> > >> > >> John.McLeod@sybase. com To 08/09/2012 04:50 PM yoyo > >> <[email protected] <mailto:[email protected]>> cc > [email protected] <mailto:[email protected]>, > >> > >> [email protected] > <mailto:[email protected]>, Patricio Vidal > >> <[email protected] <mailto:[email protected]>> > >> > >> Subject Re: [boinc_dev] Changes to client scheduler from 6.12.33 to > >> 7.0.25? > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> They want the clients not to "go to sleep" for a long time if there > is no > >> work for a bit. Trickle messages only work if there is work from that > >> project actually on the client. > >> > >> There are some problems with contacting projects constantly under > certain > >> circumstances. If there is no work on the client from the client and > the > >> client is not interested in fetching work from that project, then > there is > >> no point in a contact at all. There would be nothing that the server > >> could legitimately do. In this case, it is just taking up bandwidth > that > >> some users pay for by the byte or have caps on usage. > >> > >> The code was modified so that no contact would be made if there was no > >> work on the client from the project. It could be modified so that the > >> client would talk to the project on the period specified if it either > had > >> work from that project, or was interested in work from that project. > i.e. > >> The queue is not full and that project was currently the top of the > list > >> for work fetch. The problem with this is if this project has a very > short > >> request cycle and it does not have work, it will block the client from > >> asking any other projects for work and the client would go idle as a > >> result. > >> > >> jm7 > >> > >> > >> |------------> | From: | |------------> > >> > >> > >> > >> > > --------------------------------------------------------------------------------------------------------------------------------------------------| > >> > >> > >> > >> > >> > >> > |yoyo <[email protected] <mailto:[email protected]>> > >> | > >> > >> > >> > >> > > --------------------------------------------------------------------------------------------------------------------------------------------------| > >> > >> > >> > >> > >> > >> > |------------> > >> | To: | |------------> > >> > >> > >> > >> > > --------------------------------------------------------------------------------------------------------------------------------------------------| > >> > >> > >> > >> > >> > >> > |Patricio Vidal <[email protected] > <mailto:[email protected]>> > >> | > >> > >> > >> > >> > > --------------------------------------------------------------------------------------------------------------------------------------------------| > >> > >> > >> > >> > >> > >> > |------------> > >> | Cc: | |------------> > >> > >> > >> > >> > > --------------------------------------------------------------------------------------------------------------------------------------------------| > >> > >> > >> > >> > >> > >> > |<[email protected] <mailto:[email protected]>>, > <[email protected] <mailto:[email protected]>>, > >> <[email protected] > <mailto:[email protected]>> | > >> > >> > >> > >> > > --------------------------------------------------------------------------------------------------------------------------------------------------| > >> > >> > >> > >> > >> > >> > |------------> > >> | Date: | |------------> > >> > >> > >> > >> > > --------------------------------------------------------------------------------------------------------------------------------------------------| > >> > >> > >> > >> > >> > >> > |08/09/2012 03:37 PM > >> | > >> > >> > >> > >> > > --------------------------------------------------------------------------------------------------------------------------------------------------| > >> > >> > >> > >> > >> > >> > |------------> > >> | Subject: | |------------> > >> > >> > >> > >> > > --------------------------------------------------------------------------------------------------------------------------------------------------| > >> > >> > >> > >> > >> > >> > |Re: [boinc_dev] Changes to client scheduler from 6.12.33 to > >> 7.0.25? | > >> > >> > >> > >> > > --------------------------------------------------------------------------------------------------------------------------------------------------| > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > For what reason should the client contact the server so often? > >> You just can use trickle messages to be sent by the client. > >> > >> yoyo > >> > >> Patricio Vidal schrieb: Yes, for our project we need the clients to > contact > >> the server every few minutes. We send jobs that last from 20 minutes > to few > >> hours. If the clients don't contact the server for several hour then > we are > >> in trouble. > >> > >> Regards, Patricio. > >> > >> > >> > >> > >> > >> > >> John.McLeod@sybase. > >> > >> com > >> > >> To 08/09/2012 11:21 AM Patricio Vidal > >> > >> > >> <[email protected] <mailto:[email protected]>> > >> > >> cc > >> > >> [email protected] <mailto:[email protected]>, > >> > >> > >> [email protected] > <mailto:[email protected]> Subject Re: [boinc_dev] > Changes to > >> client scheduler from 6.12.33 to 7.0.25? > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> It was removed in order to avoid hammering projects from clients that > were > >> not going to ask for work and had no work they were working on. > >> > >> Perhaps, the code could be modified slightly again. Only ignore the > >> setting for contact server every X if the client has no work from the > >> server and the client would not ask for work from the project. > >> > >> BTW, a really short contact period will play havoc with multi project > >> clients as they will not be able to get work from elsewhere while you > have > >> no work. > >> > >> jm7 > >> > >> > >> |------------> | From: | |------------> > >> > >> > >> > >> > >> > > --------------------------------------------------------------------------------------------------------------------------------------------------| > >> > >> > >> > >> > >> > >> > >> > >> > |Patricio Vidal <[email protected] > <mailto:[email protected]>> > >> | > >> > >> > >> > >> > >> > > --------------------------------------------------------------------------------------------------------------------------------------------------| > >> > >> > >> > >> > >> > >> > >> > >> > |------------> > >> | To: | |------------> > >> > >> > >> > >> > >> > > --------------------------------------------------------------------------------------------------------------------------------------------------| > >> > >> > >> > >> > >> > >> > >> > >> > |<[email protected] <mailto:[email protected]>> > >> | > >> > >> > >> > >> > >> > > --------------------------------------------------------------------------------------------------------------------------------------------------| > >> > >> > >> > >> > >> > >> > >> > >> > |------------> > >> | Date: | |------------> > >> > >> > >> > >> > >> > > --------------------------------------------------------------------------------------------------------------------------------------------------| > >> > >> > >> > >> > >> > >> > >> > >> > |08/09/2012 11:00 AM > >> | > >> > >> > >> > >> > >> > > --------------------------------------------------------------------------------------------------------------------------------------------------| > >> > >> > >> > >> > >> > >> > >> > >> > |------------> > >> | Subject: | |------------> > >> > >> > >> > >> > >> > > --------------------------------------------------------------------------------------------------------------------------------------------------| > >> > >> > >> > >> > >> > >> > >> > >> > |[boinc_dev] Changes to client scheduler from 6.12.33 to > >> 7.0.25? | > >> > >> > >> > >> > >> > > --------------------------------------------------------------------------------------------------------------------------------------------------| > >> > >> > >> > >> > >> > >> > >> > >> > |------------> > >> | Sent by: | |------------> > >> > >> > >> > >> > >> > > --------------------------------------------------------------------------------------------------------------------------------------------------| > >> > >> > >> > >> > >> > >> > >> > >> > |<[email protected] > <mailto:[email protected]>> > >> | > >> > >> > >> > >> > >> > > --------------------------------------------------------------------------------------------------------------------------------------------------| > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > Hello, > >> > >> I noticed a change in the client scheduler when moving from 6.12.33 to > >> 7.0.25: I have the <next_rpc_delay> set 180 in the config.xml. > >> > >> For 6.12.33 the clients contact the server even if there is no work, > which > >> is our desired behavior: > >> > >> 8/8/2012 11:39:09 PM | AlgoGrid | Requesting new tasks for CPU > 8/8/2012 > >> 11:42:12 PM | AlgoGrid | Requesting new tasks for CPU 8/8/2012 > 11:45:15 PM > >> | AlgoGrid | Requesting new tasks for CPU 8/8/2012 11:48:18 PM | > AlgoGrid | > >> Requesting new tasks for CPU > >> > >> > >> For 7.0.25, when there are available task the scheduler contacts the > >> server as expected: > >> > >> 8/8/2012 9:26:33 PM | AlgoGrid | Requesting new tasks for CPU 8/8/2012 > >> 9:27:10 PM | AlgoGrid | Requesting new tasks for CPU > >> > >> ... but when there is no tasks it starts delaying the request > >> exponentially: > >> > >> 8/8/2012 9:49:45 PM | AlgoGrid | Requesting new tasks for CPU 8/8/2012 > >> 10:08:23 PM | AlgoGrid | Requesting new tasks for CPU 8/8/2012 > 10:32:36 PM > >> | AlgoGrid | Requesting new tasks for CPU 8/8/2012 11:37:35 PM | > AlgoGrid | > >> Requesting new tasks for CPU 8/9/2012 2:59:13 AM | AlgoGrid | > Requesting > >> new tasks for CPU 8/9/2012 10:41:06 AM | AlgoGrid | Requesting new > tasks > >> for CPU > >> > >> > >> Is there a new option to force the rpc call even if there is no tasks > >> available? We need this behavior because our boinc project needs a > quick > >> response from the clients when we schedule a job (we have workunits of > >> about 1 min processing time and we schedule thousands at a time). > >> > >> Thank you, Patricio. > >> > >> > >> The server made the following annotations > >> > >> > >> > >> > > --------------------------------------------------------------------------------- > >> > >> > >> > >> > >> > >> > >> > >> > >> > This message contains information that may be privileged or > >> confidential and is the property of Beckman Coulter, Inc. It is > intended > >> only for the person to whom it is addressed. If you are not the > intended > >> recipient, you are not authorized to read, print, retain, copy, > >> disseminate, distribute or use this message or any part thereof. If > you > >> receive this message in error, please notify the sender immediately > and > >> delete all copies of this message. > >> > >> > >> > >> > > --------------------------------------------------------------------------------- > >> > >> > >> > >> > >> > >> > >> > >> > >> > _______________________________________________ > >> boinc_dev mailing list [email protected] > <mailto:[email protected]> > >> http://lists.ssl.berkeley.edu/mailman/listinfo/boinc_dev To > unsubscribe, > >> visit the above URL and (near bottom of page) enter your email > address. > >> > >> > >> > >> > >> > >> > >> The server made the following annotations > >> > >> > >> > >> > > --------------------------------------------------------------------------------- > >> > >> > >> > >> > >> > >> > >> > This message contains information that may be privileged or > >> confidential and is the property of Beckman Coulter, Inc. It is > intended > >> only for the person to whom it is addressed. If you are not the > intended > >> recipient, you are not authorized to read, print, retain, copy, > >> disseminate, distribute or use this message or any part thereof. If > you > >> receive this message in error, please notify the sender immediately > and > >> delete all copies of this message. > >> > >> > >> > >> > > --------------------------------------------------------------------------------- > >> > >> > >> > >> > >> > >> > >> > _______________________________________________ > >> boinc_dev mailing list [email protected] > <mailto:[email protected]> > >> http://lists.ssl.berkeley.edu/mailman/listinfo/boinc_dev To > unsubscribe, > >> visit the above URL and (near bottom of page) enter your email > address. > >> > >> > >> > >> > >> -- Rate Me, MySkype (yoyo_rkn)Skype Me?! , myICQ 139003243 , myIRC > >> Rechenkraft.net e.V. - Verein zur Förderung von Bildung, Forschung und > >> Wissenschaft durch Einsatz vernetzter Computer weitere interessante > >> Projekte und Hilfe auf unserer Webseite www.Rechenkraft.net und im > Chat > >> Rechenkraft.net e.V. - Non-profit association for the promotion of > >> education, research and science through the use of networked computers > >> other interesting projects and help on our website > www.Rechenkraft.net and > >> on IRC > >> > >> > >> > >> > >> > >> The server made the following annotations > >> > >> > >> > > --------------------------------------------------------------------------------- > >> > >> > >> > >> > >> > This message contains information that may be privileged or > >> confidential and is the property of Beckman Coulter, Inc. It is > intended > >> only for the person to whom it is addressed. If you are not the > intended > >> recipient, you are not authorized to read, print, retain, copy, > >> disseminate, distribute or use this message or any part thereof. If > you > >> receive this message in error, please notify the sender immediately > and > >> delete all copies of this message. > >> > >> > >> > > --------------------------------------------------------------------------------- > >> > >> > >> > >> > >> > _______________________________________________ > >> boinc_dev mailing list [email protected] > <mailto:[email protected]> > >> http://lists.ssl.berkeley.edu/mailman/listinfo/boinc_dev To > unsubscribe, > >> visit the above URL and (near bottom of page) enter your email > address. > >> > >> > >> > >> > >> -- Rate Me, MySkype (yoyo_rkn)Skype Me?! , myICQ 139003243 , myIRC > >> Rechenkraft.net e.V. - Verein zur Förderung von Bildung, Forschung und > >> Wissenschaft durch Einsatz vernetzter Computer weitere interessante > >> Projekte und Hilfe auf unserer Webseite www.Rechenkraft.net und im > Chat > >> Rechenkraft.net e.V. - Non-profit association for the promotion of > >> education, research and science through the use of networked computers > >> other interesting projects and help on our website > www.Rechenkraft.net and > >> on IRC > >> > >> > >> > >> The server made the following annotations > >> > >> > > --------------------------------------------------------------------------------- > >> > >> > >> > >> > This message contains information that may be privileged or confidential > >> and is the property of Beckman Coulter, Inc. It is intended only for > the > >> person to whom it is addressed. If you are not the intended > recipient, > >> you are not authorized to read, print, retain, copy, disseminate, > >> distribute or use this message or any part thereof. If you receive > this > >> message in error, please notify the sender immediately and delete all > >> copies of this message. > >> > >> > > --------------------------------------------------------------------------------- > >> > >> > >> > _______________________________________________ > >> boinc_dev mailing list [email protected] > <mailto:[email protected]> > >> http://lists.ssl.berkeley.edu/mailman/listinfo/boinc_dev To > unsubscribe, > >> visit the above URL and (near bottom of page) enter your email > address. > >> > >> > >> > >> _______________________________________________ boinc_dev mailing list > >> [email protected] <mailto:[email protected]> > >> http://lists.ssl.berkeley.edu/mailman/listinfo/boinc_dev To > unsubscribe, > >> visit the above URL and (near bottom of page) enter your email > address. > >> > >> > >> > > _______________________________________________ boinc_dev mailing list > > [email protected] <mailto:[email protected]> > > http://lists.ssl.berkeley.edu/mailman/listinfo/boinc_dev To > unsubscribe, > > visit the above URL and (near bottom of page) enter your email address. > > > _______________________________________________ > boinc_dev mailing list > [email protected] <mailto:[email protected]> > http://lists.ssl.berkeley.edu/mailman/listinfo/boinc_dev > To unsubscribe, visit the above URL and > (near bottom of page) enter your email address. > > _______________________________________________ boinc_dev mailing list [email protected] http://lists.ssl.berkeley.edu/mailman/listinfo/boinc_dev To unsubscribe, visit the above URL and (near bottom of page) enter your email address.
