Dear Josh, That is true ! Thanks again for the clarification ! I will start doing some experiments and see what it gives from performance perspective ! In the other hand, I will keep updating for any proposal of ideas can be generated then we can refine the scheduler code and retest again !
Many thanks for sharing such valuable information! We will keep updated ! Best Regards PhD Team 2014-10-30 19:56 GMT+01:00 Josh Thompson <[email protected]>: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > Khder, > > The scheduler only allocates a computer randomly if > SCHEDULER_ALLOCATE_RANDOM_COMPUTER is set to 1 in conf.php, which it is > not by > default. > > Yes, you could create pools that each have a weight or ranking assigned. > Each > computer would be assigned to a pool. Then, when the initial set of > computers > is generated, they could be ranked by the pool weight/ranks. It would > then be > up to an administer or external application to manage the weight/rank of > each > pool. > > Josh > > On Thursday, October 30, 2014 7:48:50 PM Khder Omar wrote: > > Dear Josh, > > > > I would like to thank you for the precious hints concerning the > scheduling > > nature within VCL. I can see that the set of computers is filtered in > > advance and then the scheduler will pick up RANDOMLY one of them. To > check > > the possibilities to extend the function considering a large scale > > environment where we can introduce the 'region' or 'pool' term in the > code. > > With multiple management nodes, do you think that using the 'weight' for > > the workload per pool and then per computer therefore will rank it will > be > > more precise on how to balance the load across the hole infrastructure ? > > > > Best Regards, > > PhD team > > > > 2014-10-30 19:28 GMT+01:00 Josh Thompson <[email protected]>: > > > -----BEGIN PGP SIGNED MESSAGE----- > > > Hash: SHA1 > > > > > > Khder, > > > > > > Sorry for the late response. > > > > > > You can certainly experiment with extending the code to include more > > > advanced > > > scheduling functions. The existing ranking of computers is something > like > > > this: > > > > > > 1) generate a set of all computers meeting the minimum requirements of > the > > > image, that the user has access to, and that the image is mapped to; > order > > > that list by node specs from low to high > > > 2) remove from that set any computers that are already assigned > > > reservations > > > 3) if the image is virtual, remove any VMs for which the host doesn't > have > > > enough memory to load the VM (without overbooking) > > > 4) assign the new reservation the first computer from that set for > which > > > an > > > active management node can be found > > > > > > There's actually 3 sets of computers being tracked - those already > loaded > > > with > > > the image being requested, those set aside for a block allocation of > which > > > the > > > user is a member, and then a set of both of those plus any available > > > computers > > > that are currently loaded with another image. > > > > > > There is a configuration option (SCHEDULER_ALLOCATE_RANDOM_COMPUTER) in > > > conf.php that allows the sets of computers to be randomized. The > > > motivation > > > behind this option is for sites that have homogeneous virtual machines > and > > > hosts. Randomizing the assigned computer should help spread the load > > > equally > > > among all existing hosts. > > > > > > I hope that helps. > > > > > > Josh > > > > > > On Tuesday, October 21, 2014 9:01:17 PM Khder Omar wrote: > > > > Hi Josh, > > > > > > > > Thanks for your reaction. I have checked the code source and I think > the > > > > only function which might fulfill my question is function > > > > > > allocComputer(...) > > > > > > > Eventually, the function a piece test schedule code > > > > > > > > if(SCHEDULER_ALLOCATE_RANDOM_COMPUTER) { > > > > > > > > shuffle($blockids); > > > > shuffle($currentids); > > > > shuffle($computerids); > > > > > > > > } > > > > > > > > > > > > I might assume if it was correctly understood that VCL scheduler > process > > > > determines in first place a computer to be assigned to a management > node > > > > from a given array or table in order otherwise it can be randomly > > > > chosen! > > > > Then, we can assume that the scheduling decision is still using a > basic > > > > order as FCFS or a chance/Random order! It that assumed be correct ? > if > > > > > > so, > > > > > > > is there a way to extend the code by including more advanced > scheduling > > > > functions ? > > > > > > > > Best regards, > > > > PhD Team > > > > > > > > 2014-10-21 16:11 GMT+01:00 Josh Thompson <[email protected]>: > > > > > -----BEGIN PGP SIGNED MESSAGE----- > > > > > Hash: SHA1 > > > > > > > > > > Khder, > > > > > > > > > > The scheduling of reservations is actually done in the web code. > Have > > > > > > a > > > > > > > > look > > > > > at the isAvailable function in the web/.ht-inc/utils.php file. > > > > > > > > > > https://svn.apache.org/repos/asf/vcl/trunk/web/.ht-inc/utils.php > > > > > > > > > > Josh > > > > > > > > > > On Tuesday, October 21, 2014 4:39:05 PM Khder Omar wrote: > > > > > > Dear all, > > > > > > > > > > > > We were wondering what kind of scheduler algorithms VCL might > use ? > > > > > > Any > > > > > > > > > hints about the scheduler source code will be appreciated. The > idea > > > > > > is > > > > > > > > > actually to check how VCL will perform while changing the > scheduler > > > > > > algorithm [in management node]. > > > > > > > > > > > > > > > > > > Thanks in advance > > > > > > > > > > > > Best regards, > > > > > > Phd team > > > > > > > > > > - -- > > > > > - ------------------------------- > > > > > Josh Thompson > > > > > VCL Developer > > > > > North Carolina State University > > > > > > > > > > my GPG/PGP key can be found at pgp.mit.edu > > > > > > > > > > All electronic mail messages in connection with State business > which > > > > > are sent to or received by this account are subject to the NC > Public > > > > > Records Law and may be disclosed to third parties. > > > > > -----BEGIN PGP SIGNATURE----- > > > > > Version: GnuPG v2.0.22 (GNU/Linux) > > > > > > > > > > iEYEARECAAYFAlRGd60ACgkQV/LQcNdtPQPMjQCdF/28fx+VlhmZV0WEMobcv+7p > > > > > OAEAn1mct5Iz5bWrLnnX/yQl13wMVwRg > > > > > =RMTU > > > > > -----END PGP SIGNATURE----- > > > > > > - -- > > > - ------------------------------- > > > Josh Thompson > > > VCL Developer > > > North Carolina State University > > > > > > my GPG/PGP key can be found at pgp.mit.edu > > > > > > All electronic mail messages in connection with State business which > > > are sent to or received by this account are subject to the NC Public > > > Records Law and may be disclosed to third parties. > > > -----BEGIN PGP SIGNATURE----- > > > Version: GnuPG v2.0.22 (GNU/Linux) > > > > > > iEYEARECAAYFAlRSgzoACgkQV/LQcNdtPQMRBgCfaBOvBvUNnfdr5BLR7cnr+vMx > > > 5uQAniwpvKVjVGdW//rvNBYCKu485Siy > > > =xWum > > > -----END PGP SIGNATURE----- > - -- > - ------------------------------- > Josh Thompson > VCL Developer > North Carolina State University > > my GPG/PGP key can be found at pgp.mit.edu > > All electronic mail messages in connection with State business which > are sent to or received by this account are subject to the NC Public > Records Law and may be disclosed to third parties. > -----BEGIN PGP SIGNATURE----- > Version: GnuPG v2.0.22 (GNU/Linux) > > iEYEARECAAYFAlRSieYACgkQV/LQcNdtPQOjlACfS97i9Hk0Tmspy+6vfgOcENht > UtYAn008A0w5VBq+eg5iGg1GRlff3zDB > =tCAJ > -----END PGP SIGNATURE----- > >
