Re: [boinc_dev] Estimated Time Remaining, frictional reporting ...
Although I am a CS grad student, I urge you to reconsider choosing CS grad students to work on this problem and consider instead using domain experts in statistics and/or Operations Research or Systems, or perhaps even an interdisciplinary team. Old research shows that it is much more cost-effective to hire domain experts and teach them to program computers than it is to hire CS grads and try to teach them the domain. Suppose your income tax preparation was a complex process. Which would you want do it: a CS grad who wrote the fastest program possible, or a tax law expert who could save you months of work on an IRS tax audit and keep you out of jail? Charles Elliott -Original Message- From: boinc_dev [mailto:boinc_dev-boun...@ssl.berkeley.edu] On Behalf Of David Anderson Sent: Monday, February 10, 2014 10:58 PM To: boinc_dev@ssl.berkeley.edu Subject: Re: [boinc_dev] Estimated Time Remaining, frictional reporting ... In general we've put statistics-gathering into server rather than client because - it gives uniform data over the entire host population - it puts the data all in one place Currently these statistics are just the bare essentials: mean and standard deviation of elapsed time, turnaround time, and credit-related quantities. We maintain these per (host, app version) and per app version. We use them to estimate job duration and to compute credit. As you point out, there are many other types of info we could track, and many visualizations that could offered. This is an area were having a few CS grad students working on BOINC would be a big help. -- David On 10-Feb-2014 4:01 PM, Max Power wrote: Many types of distributed computing applications don't due uniform processing (and reporting on percent done) like SETI, Astropulse or Einstein ... and the biological science applications (and image rendering ones) have taken some time to discipline the reporting of percent done. What the BOINC Client does not do is use the hashsums of computing applications (as sometimes they run in pairs as in Climate Prediction) to form a local knowledge base of -- work unit size (average, median, standard deviation) -- work unit computation length (average, median, standard deviation) -- completed work unit average size (average, median, standard deviation) -- disk use (average, median, standard deviation) -- these could be uplinked to the BOINC design groups and the projects themselves ... as you probably have to do an SQL query to find this stuff out -- THE STATS tab is almost totally devoid of usable statistics ... and the ones above relating to runtime are graphable and usable ... I am not saying this will fix the wonky estimated run time problem ... only regular application reporting to the BOINC client will ever do that. However, the averaged knowledge from these parameters could improve it when the daft application is not reporting. MP, DSN @ H -Original Message- From: McLeod, John Sent: 10 February 2014 05:48 To: Jon Sonntag ; BOINC Developers Mailing l...@berkeley.edu Subject: Re: [boinc_dev] Estimated Time Remaining Not all applications report smooth % complete. So the calculation of time remaining involve the initial estimate as well. Given the bad information given for both % complete and initial estimate, there is no method of predicting how much longer the task will take that is completely right. The most reliable appears to be to combine the initial estimate the DCF (if in use for the project) the % complete, and the time spent already (the only really well known item in the list) to come up with an estimate. ___ boinc_dev mailing list boinc_dev@ssl.berkeley.edu http://lists.ssl.berkeley.edu/mailman/listinfo/boinc_dev To unsubscribe, visit the above URL and (near bottom of page) enter your email address. ___ boinc_dev mailing list boinc_dev@ssl.berkeley.edu http://lists.ssl.berkeley.edu/mailman/listinfo/boinc_dev To unsubscribe, visit the above URL and (near bottom of page) enter your email address. ___ boinc_dev mailing list boinc_dev@ssl.berkeley.edu http://lists.ssl.berkeley.edu/mailman/listinfo/boinc_dev To unsubscribe, visit the above URL and (near bottom of page) enter your email address.
Re: [boinc_dev] Fwd: Scheduler troubles in conjunction with rate limiting from server
Here's an example of the sort of event which can cause the problems Rytis was describing: 11/02/2014 14:28:15 | boincsimap | [sched_op] Starting scheduler request 11/02/2014 14:28:15 | boincsimap | Sending scheduler request: To fetch work. 11/02/2014 14:28:15 | boincsimap | Requesting new tasks for CPU 11/02/2014 14:28:15 | boincsimap | [sched_op] CPU work request: 1.46 seconds; 0.00 devices 11/02/2014 14:28:15 | boincsimap | [sched_op] NVIDIA work request: 0.00 seconds; 0.00 devices 11/02/2014 14:28:17 | boincsimap | Scheduler request completed: got 1 new tasks 11/02/2014 14:28:17 | boincsimap | [sched_op] Server version 703 11/02/2014 14:28:17 | boincsimap | Project requested delay of 7 seconds 11/02/2014 14:28:17 | boincsimap | [sched_op] estimated total CPU task duration: 3680 seconds 11/02/2014 14:28:17 | boincsimap | [sched_op] estimated total NVIDIA task duration: 0 seconds 11/02/2014 14:28:17 | boincsimap | [sched_op] Deferring communication for 00:00:07 11/02/2014 14:28:17 | boincsimap | [sched_op] Reason: requested by project 11/02/2014 14:28:19 | boincsimap | Started download of 20140129.556477 11/02/2014 14:28:24 | boincsimap | Finished download of 20140129.556477 11/02/2014 14:28:47 | boincsimap | Computation for task 20140129.537727_1 finished 11/02/2014 14:28:47 | boincsimap | Starting task 20140129.540879_1 11/02/2014 14:28:47 | boincsimap | [cpu_sched] Starting task 20140129.540879_1 using simap version 512 in slot 1 11/02/2014 14:28:49 | boincsimap | Started upload of 20140129.537727_1_0 11/02/2014 14:29:00 | boincsimap | Finished upload of 20140129.537727_1_0 But because work was requested 30 seconds *before* a task completed, neither the old nor the new versions of inhibit RPCs during upload would have prevented it. As it happens, SIMAP is one of the projects which could honestly use the estimates are linear and can be trusted flag, if available. From: Richard Haselgrove r.haselgr...@btopenworld.com To: David Anderson da...@ssl.berkeley.edu; BOINC Developers Mailing List boinc_dev@ssl.berkeley.edu Sent: Saturday, 8 February 2014, 12:08 Subject: Re: [boinc_dev] Fwd: Scheduler troubles in conjunction with rate limiting from server I thought we had this protection in place already? Specifically, since your checkin 60fc3d3 of April 2011: client: defer reporting completed tasks if an upload started recently; we might be able to report more tasks once the upload completes. http://boinc.berkeley.edu/trac/changeset/60fc3d3f22f66d7a7b5bb5632d2de322cf2f180a/boinc-v2 If that works (and in my experience it does), it exactly covers Rytis' problem: by delaying work fetch until the previous task is reportable, an extra slot is made available within the jobs-in-progress limit. It took a few follow-up revisions to get 60fc3d3 working properly: the only remaining loophole that I can see is that occasionally BOINC might slip in a work fetch after a task has exited, but before the upload has even started. The other situation which could lead to Rytis' observation is if BOINC requested new work shortly before his task exited, but we have always resisted the calls to adjust scheduling on the basis of anticipated/estimated completion times. I'm a little worried by the new checkin: if a project completes tasks, and hence starts uploads, more frequently than once every five minutes, will it ever break free of the deferral? From: David Anderson da...@ssl.berkeley.edu To: BOINC Developers Mailing List boinc_dev@ssl.berkeley.edu Sent: Saturday, 8 February 2014, 0:00 Subject: [boinc_dev] Fwd: Scheduler troubles in conjunction with rate limiting from server I checked in the following change to address the problem Rytis describes below. client: work fetch policy tweak If a project has active uploads, defer work fetch from it for 5 minutes even if there are idle devices (that's the change). This addresses a situation (reported by Rytis) where - a project P has a jobs-in-progress limit less than NCPUS - P's jobs finish and are uploading - the client asks P for work and doesn't get any because of the limit - the client does exponential backoff from P Over the long term, P can get much less than its fair share of work -- David Original Message Subject: Scheduler troubles in conjunction with rate limiting from server Date: Fri, 7 Feb 2014 12:41:04 +0200 From: Rytis Slatkevičius ry...@gridrepublic.org To: David Anderson da...@ssl.berkeley.edu CC: Matthew Blumberg m...@gridrepublic.org Hello David, we observed an interesting trouble with task scheduling: Project A (our project) limits number of tasks per proc to 2 and has resource share of 500; Project B (Einstein) does not limit number of tasks and has resource share of 25. B has longer tasks than A, and also longer tasks than the minimum work buffer. When
[boinc_dev] compile error while trying to build boincmgr of boinc 7.4/7.3.1 at a Gentoo Linux
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 Hi, the client was build fine and seems to work well, but the manager cannot be built due to this error at a 32 bit Gentoo Linux: $ make CXX boincmgr-NoticeListCtrl.o In file included from /usr/include/wx-3.0/wx/cursor.h:69:0, from /usr/include/wx-3.0/wx/event.h:21, from /usr/include/wx-3.0/wx/wx.h:24, from stdwx.h:48, from NoticeListCtrl.cpp:22: /usr/include/wx-3.0/wx/utils.h:603:52: warning: redundant redeclaration of ‘void wxQsort(void*, size_t, size_t, wxSortCallback, const void*)’ in same scope [-Wredundant-decls] In file included from /usr/include/wx-3.0/wx/list.h:34:0, from /usr/include/wx-3.0/wx/wx.h:17, from stdwx.h:48, from NoticeListCtrl.cpp:22: /usr/include/wx-3.0/wx/vector.h:44:23: warning: previous declaration of ‘void wxQsort(void*, size_t, size_t, wxSortCallback, const void*)’ [-Wredundant-decls] In file included from NoticeListCtrl.cpp:36:0: NoticeListCtrl.h:48:25: error: ‘wxWebViewEvent’ has not been declared NoticeListCtrl.h:49:26: error: ‘wxWebViewEvent’ has not been declared NoticeListCtrl.h:59:5: error: ‘wxWebView’ does not name a type NoticeListCtrl.cpp:53:72: error: invalid use of non-static member function ‘void CNoticeListCtrl::OnLinkClicked(int)’ NoticeListCtrl.cpp:53:85: error: ‘EVT_WEBVIEW_NAVIGATING’ was not declared in this scope NoticeListCtrl.cpp:54:5: error: expected ‘}’ before ‘EVT_WEBVIEW_ERROR’ NoticeListCtrl.cpp:54:5: error: expected ‘,’ or ‘;’ before ‘EVT_WEBVIEW_ERROR’ NoticeListCtrl.cpp:57:1: error: expected declaration before ‘}’ token make: *** [boincmgr-NoticeListCtrl.o] Error 1 - -- MfG/Sincerely Toralf Förster pgp finger print:1A37 6F99 4A9D 026F 13E2 4DCF C4EA CDDE 0076 E94E -BEGIN PGP SIGNATURE- Version: GnuPG v2.0.22 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iF4EAREIAAYFAlL6aS0ACgkQxOrN3gB26U5VygD+MfiqHz3Osq+3MbY1tqtQXmQK I66CX7dJ6gJdO1q5n0UBAI8vgOIbDzSbWaXJRujF6ATVwqsoSNRxqAIjMRvF9gNI =WCY2 -END PGP SIGNATURE- ___ boinc_dev mailing list boinc_dev@ssl.berkeley.edu http://lists.ssl.berkeley.edu/mailman/listinfo/boinc_dev To unsubscribe, visit the above URL and (near bottom of page) enter your email address.
Re: [boinc_dev] compile error while trying to build boincmgr of boinc 7.4/7.3.1 at a Gentoo Linux
It looks like wxWidgets was built without webview support, or at least the 'make install' step didn't copy over all the web view header files. - Rom -Original Message- From: boinc_dev [mailto:boinc_dev-boun...@ssl.berkeley.edu] On Behalf Of Toralf Förster Sent: Tuesday, February 11, 2014 1:17 PM To: boinc_dev@ssl.berkeley.edu Subject: [boinc_dev] compile error while trying to build boincmgr of boinc 7.4/7.3.1 at a Gentoo Linux -BEGIN PGP SIGNED MESSAGE- Hash: SHA256 Hi, the client was build fine and seems to work well, but the manager cannot be built due to this error at a 32 bit Gentoo Linux: $ make CXX boincmgr-NoticeListCtrl.o In file included from /usr/include/wx-3.0/wx/cursor.h:69:0, from /usr/include/wx-3.0/wx/event.h:21, from /usr/include/wx-3.0/wx/wx.h:24, from stdwx.h:48, from NoticeListCtrl.cpp:22: /usr/include/wx-3.0/wx/utils.h:603:52: warning: redundant redeclaration of ‘void wxQsort(void*, size_t, size_t, wxSortCallback, const void*)’ in same scope [-Wredundant-decls] In file included from /usr/include/wx-3.0/wx/list.h:34:0, from /usr/include/wx-3.0/wx/wx.h:17, from stdwx.h:48, from NoticeListCtrl.cpp:22: /usr/include/wx-3.0/wx/vector.h:44:23: warning: previous declaration of ‘void wxQsort(void*, size_t, size_t, wxSortCallback, const void*)’ [-Wredundant-decls] In file included from NoticeListCtrl.cpp:36:0: NoticeListCtrl.h:48:25: error: ‘wxWebViewEvent’ has not been declared NoticeListCtrl.h:49:26: error: ‘wxWebViewEvent’ has not been declared NoticeListCtrl.h:59:5: error: ‘wxWebView’ does not name a type NoticeListCtrl.cpp:53:72: error: invalid use of non-static member function ‘void CNoticeListCtrl::OnLinkClicked(int)’ NoticeListCtrl.cpp:53:85: error: ‘EVT_WEBVIEW_NAVIGATING’ was not declared in this scope NoticeListCtrl.cpp:54:5: error: expected ‘}’ before ‘EVT_WEBVIEW_ERROR’ NoticeListCtrl.cpp:54:5: error: expected ‘,’ or ‘;’ before ‘EVT_WEBVIEW_ERROR’ NoticeListCtrl.cpp:57:1: error: expected declaration before ‘}’ token make: *** [boincmgr-NoticeListCtrl.o] Error 1 - -- MfG/Sincerely Toralf Förster pgp finger print:1A37 6F99 4A9D 026F 13E2 4DCF C4EA CDDE 0076 E94E -BEGIN PGP SIGNATURE- Version: GnuPG v2.0.22 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iF4EAREIAAYFAlL6aS0ACgkQxOrN3gB26U5VygD+MfiqHz3Osq+3MbY1tqtQXmQK I66CX7dJ6gJdO1q5n0UBAI8vgOIbDzSbWaXJRujF6ATVwqsoSNRxqAIjMRvF9gNI =WCY2 -END PGP SIGNATURE- ___ boinc_dev mailing list boinc_dev@ssl.berkeley.edu http://lists.ssl.berkeley.edu/mailman/listinfo/boinc_dev To unsubscribe, visit the above URL and (near bottom of page) enter your email address. ___ boinc_dev mailing list boinc_dev@ssl.berkeley.edu http://lists.ssl.berkeley.edu/mailman/listinfo/boinc_dev To unsubscribe, visit the above URL and (near bottom of page) enter your email address.
Re: [boinc_dev] compile error while trying to build boincmgr of boinc 7.4/7.3.1 at a Gentoo Linux
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 On 02/11/2014 07:22 PM, Rom Walton wrote: It looks like wxWidgets was built without webview support, or at least the 'make install' step didn't copy over all the web view header files. - Rom ah - will look into the ebuild and/or ask the dev. thx for the quick answer. - -- MfG/Sincerely Toralf Förster pgp finger print:1A37 6F99 4A9D 026F 13E2 4DCF C4EA CDDE 0076 E94E -BEGIN PGP SIGNATURE- Version: GnuPG v2.0.22 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iF4EAREIAAYFAlL6a00ACgkQxOrN3gB26U54WwD/dwc/DGOfaIKqpp0NOe+W7YtA uTs2Q8EbcvW1/IlPQ+cA/3CGhOAsi1UgeGrIY89lT7PFDfWVTmSy3SolDB//TXGc =fS+3 -END PGP SIGNATURE- ___ boinc_dev mailing list boinc_dev@ssl.berkeley.edu http://lists.ssl.berkeley.edu/mailman/listinfo/boinc_dev To unsubscribe, visit the above URL and (near bottom of page) enter your email address.
Re: [boinc_dev] compile error while trying to build boincmgr of boinc 7.4/7.3.1 at a Gentoo Linux
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 On 02/11/2014 07:22 PM, Rom Walton wrote: It looks like wxWidgets was built without webview support, or at least the 'make install' step didn't copy over all the web view header files. - Rom pff - resulted in 13 additional packages here at a Gentoo desktop and 1+x hour of compile time - now the manager is back too. - -- MfG/Sincerely Toralf Förster pgp finger print:1A37 6F99 4A9D 026F 13E2 4DCF C4EA CDDE 0076 E94E -BEGIN PGP SIGNATURE- Version: GnuPG v2.0.22 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iF4EAREIAAYFAlL6fowACgkQxOrN3gB26U4ccAD/YjiNw5XUOD9SEW+DyNii5VJS jQre+5qAt/r8LU848tgA+gL/xRd/ouYfRIrMigrFmfDujz7RekZG45S4AzgkP5yz =n70Y -END PGP SIGNATURE- ___ boinc_dev mailing list boinc_dev@ssl.berkeley.edu http://lists.ssl.berkeley.edu/mailman/listinfo/boinc_dev To unsubscribe, visit the above URL and (near bottom of page) enter your email address.
Re: [boinc_dev] compile error while trying to build boincmgr of boinc 7.4/7.3.1 at a Gentoo Linux
Hi, it seems that your version of wxWidgets-3.0.0 is not configured for webkit/webview: use these options to configure wxWidgets: ./configure --disable-shared --enable-static --enable-unicode --enable-webkit --enable-webview --prefix=/usr (may be your prefix is different) ... make and 'sudo make install' .and then try again to build BOINC. Wolfgang Schwieger (DeleteNull) -Ursprüngliche Nachricht- Von: Toralf Förster [mailto:toralf.foers...@gmx.de] Gesendet: Dienstag, 11. Februar 2014 19:17 An: boinc_dev@ssl.berkeley.edu Betreff: [boinc_dev] compile error while trying to build boincmgr of boinc 7.4/7.3.1 at a Gentoo Linux -BEGIN PGP SIGNED MESSAGE- Hash: SHA256 Hi, the client was build fine and seems to work well, but the manager cannot be built due to this error at a 32 bit Gentoo Linux: $ make CXX boincmgr-NoticeListCtrl.o In file included from /usr/include/wx-3.0/wx/cursor.h:69:0, from /usr/include/wx-3.0/wx/event.h:21, from /usr/include/wx-3.0/wx/wx.h:24, from stdwx.h:48, from NoticeListCtrl.cpp:22: /usr/include/wx-3.0/wx/utils.h:603:52: warning: redundant redeclaration of ‘void wxQsort(void*, size_t, size_t, wxSortCallback, const void*)’ in same scope [-Wredundant-decls] In file included from /usr/include/wx-3.0/wx/list.h:34:0, from /usr/include/wx-3.0/wx/wx.h:17, from stdwx.h:48, from NoticeListCtrl.cpp:22: /usr/include/wx-3.0/wx/vector.h:44:23: warning: previous declaration of ‘void wxQsort(void*, size_t, size_t, wxSortCallback, const void*)’ [-Wredundant-decls] In file included from NoticeListCtrl.cpp:36:0: NoticeListCtrl.h:48:25: error: ‘wxWebViewEvent’ has not been declared NoticeListCtrl.h:49:26: error: ‘wxWebViewEvent’ has not been declared NoticeListCtrl.h:59:5: error: ‘wxWebView’ does not name a type NoticeListCtrl.cpp:53:72: error: invalid use of non-static member function ‘void CNoticeListCtrl::OnLinkClicked(int)’ NoticeListCtrl.cpp:53:85: error: ‘EVT_WEBVIEW_NAVIGATING’ was not declared in this scope NoticeListCtrl.cpp:54:5: error: expected ‘}’ before ‘EVT_WEBVIEW_ERROR’ NoticeListCtrl.cpp:54:5: error: expected ‘,’ or ‘;’ before ‘EVT_WEBVIEW_ERROR’ NoticeListCtrl.cpp:57:1: error: expected declaration before ‘}’ token make: *** [boincmgr-NoticeListCtrl.o] Error 1 - -- MfG/Sincerely Toralf Förster pgp finger print:1A37 6F99 4A9D 026F 13E2 4DCF C4EA CDDE 0076 E94E -BEGIN PGP SIGNATURE- Version: GnuPG v2.0.22 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iF4EAREIAAYFAlL6aS0ACgkQxOrN3gB26U5VygD+MfiqHz3Osq+3MbY1tqtQXmQK I66CX7dJ6gJdO1q5n0UBAI8vgOIbDzSbWaXJRujF6ATVwqsoSNRxqAIjMRvF9gNI =WCY2 -END PGP SIGNATURE- ___ boinc_dev mailing list boinc_dev@ssl.berkeley.edu http://lists.ssl.berkeley.edu/mailman/listinfo/boinc_dev To unsubscribe, visit the above URL and (near bottom of page) enter your email address.
