Dear Josh, I was wondering during the creation of new image, it does not require to be stored in the computer node itself, it can be stored in an image library like a SAN or NAS storage then we mapped a given image to a certain computer or computer group ! the fact to 'map' an image or image group to a computer or computer group, does it mean 'streaming' the image 'locally' to the computer node ? or only during the reservation ? I was bit confused between the terms used in VCL and in a typical IaaS environment: the image is claimed as a template of a virtual machine, in VCL, I have noticed that the image is being created from a base OS and resources information are assigned in the database such as RAM, CPU, arch, id etc... Then I might assume [correct me if I understood it wrong] the Virtual Machine will be initiated [considering an Hypervisor KVM or ESXi ] while processing the reservation. Therefore, we can describe such image in a typical IaaS as a virtual machine flavor ready to be deployed and run on Host or Computer [within an hypervisor supported].
Thanks! PhD team 2014-10-30 20:07 GMT+01:00 Khder Omar <[email protected]>: > Dear Josh, > > That is true ! Thanks again for the clarification ! I will start doing > some experiments and see what it gives from performance perspective ! In > the other hand, I will keep updating for any proposal of ideas can be > generated then we can refine the scheduler code and retest again ! > > Many thanks for sharing such valuable information! > > We will keep updated ! > > Best Regards > PhD Team > > 2014-10-30 19:56 GMT+01:00 Josh Thompson <[email protected]>: > >> -----BEGIN PGP SIGNED MESSAGE----- >> Hash: SHA1 >> >> Khder, >> >> The scheduler only allocates a computer randomly if >> SCHEDULER_ALLOCATE_RANDOM_COMPUTER is set to 1 in conf.php, which it is >> not by >> default. >> >> Yes, you could create pools that each have a weight or ranking assigned. >> Each >> computer would be assigned to a pool. Then, when the initial set of >> computers >> is generated, they could be ranked by the pool weight/ranks. It would >> then be >> up to an administer or external application to manage the weight/rank of >> each >> pool. >> >> Josh >> >> On Thursday, October 30, 2014 7:48:50 PM Khder Omar wrote: >> > Dear Josh, >> > >> > I would like to thank you for the precious hints concerning the >> scheduling >> > nature within VCL. I can see that the set of computers is filtered in >> > advance and then the scheduler will pick up RANDOMLY one of them. To >> check >> > the possibilities to extend the function considering a large scale >> > environment where we can introduce the 'region' or 'pool' term in the >> code. >> > With multiple management nodes, do you think that using the 'weight' for >> > the workload per pool and then per computer therefore will rank it will >> be >> > more precise on how to balance the load across the hole infrastructure ? >> > >> > Best Regards, >> > PhD team >> > >> > 2014-10-30 19:28 GMT+01:00 Josh Thompson <[email protected]>: >> > > -----BEGIN PGP SIGNED MESSAGE----- >> > > Hash: SHA1 >> > > >> > > Khder, >> > > >> > > Sorry for the late response. >> > > >> > > You can certainly experiment with extending the code to include more >> > > advanced >> > > scheduling functions. The existing ranking of computers is something >> like >> > > this: >> > > >> > > 1) generate a set of all computers meeting the minimum requirements >> of the >> > > image, that the user has access to, and that the image is mapped to; >> order >> > > that list by node specs from low to high >> > > 2) remove from that set any computers that are already assigned >> > > reservations >> > > 3) if the image is virtual, remove any VMs for which the host doesn't >> have >> > > enough memory to load the VM (without overbooking) >> > > 4) assign the new reservation the first computer from that set for >> which >> > > an >> > > active management node can be found >> > > >> > > There's actually 3 sets of computers being tracked - those already >> loaded >> > > with >> > > the image being requested, those set aside for a block allocation of >> which >> > > the >> > > user is a member, and then a set of both of those plus any available >> > > computers >> > > that are currently loaded with another image. >> > > >> > > There is a configuration option (SCHEDULER_ALLOCATE_RANDOM_COMPUTER) >> in >> > > conf.php that allows the sets of computers to be randomized. The >> > > motivation >> > > behind this option is for sites that have homogeneous virtual >> machines and >> > > hosts. Randomizing the assigned computer should help spread the load >> > > equally >> > > among all existing hosts. >> > > >> > > I hope that helps. >> > > >> > > Josh >> > > >> > > On Tuesday, October 21, 2014 9:01:17 PM Khder Omar wrote: >> > > > Hi Josh, >> > > > >> > > > Thanks for your reaction. I have checked the code source and I >> think the >> > > > only function which might fulfill my question is function >> > > >> > > allocComputer(...) >> > > >> > > > Eventually, the function a piece test schedule code >> > > > >> > > > if(SCHEDULER_ALLOCATE_RANDOM_COMPUTER) { >> > > > >> > > > shuffle($blockids); >> > > > shuffle($currentids); >> > > > shuffle($computerids); >> > > > >> > > > } >> > > > >> > > > >> > > > I might assume if it was correctly understood that VCL scheduler >> process >> > > > determines in first place a computer to be assigned to a management >> node >> > > > from a given array or table in order otherwise it can be randomly >> > > > chosen! >> > > > Then, we can assume that the scheduling decision is still using a >> basic >> > > > order as FCFS or a chance/Random order! It that assumed be correct >> ? if >> > > >> > > so, >> > > >> > > > is there a way to extend the code by including more advanced >> scheduling >> > > > functions ? >> > > > >> > > > Best regards, >> > > > PhD Team >> > > > >> > > > 2014-10-21 16:11 GMT+01:00 Josh Thompson <[email protected]>: >> > > > > -----BEGIN PGP SIGNED MESSAGE----- >> > > > > Hash: SHA1 >> > > > > >> > > > > Khder, >> > > > > >> > > > > The scheduling of reservations is actually done in the web code. >> Have >> > > >> > > a >> > > >> > > > > look >> > > > > at the isAvailable function in the web/.ht-inc/utils.php file. >> > > > > >> > > > > https://svn.apache.org/repos/asf/vcl/trunk/web/.ht-inc/utils.php >> > > > > >> > > > > Josh >> > > > > >> > > > > On Tuesday, October 21, 2014 4:39:05 PM Khder Omar wrote: >> > > > > > Dear all, >> > > > > > >> > > > > > We were wondering what kind of scheduler algorithms VCL might >> use ? >> > > >> > > Any >> > > >> > > > > > hints about the scheduler source code will be appreciated. The >> idea >> > > >> > > is >> > > >> > > > > > actually to check how VCL will perform while changing the >> scheduler >> > > > > > algorithm [in management node]. >> > > > > > >> > > > > > >> > > > > > Thanks in advance >> > > > > > >> > > > > > Best regards, >> > > > > > Phd team >> > > > > >> > > > > - -- >> > > > > - ------------------------------- >> > > > > Josh Thompson >> > > > > VCL Developer >> > > > > North Carolina State University >> > > > > >> > > > > my GPG/PGP key can be found at pgp.mit.edu >> > > > > >> > > > > All electronic mail messages in connection with State business >> which >> > > > > are sent to or received by this account are subject to the NC >> Public >> > > > > Records Law and may be disclosed to third parties. >> > > > > -----BEGIN PGP SIGNATURE----- >> > > > > Version: GnuPG v2.0.22 (GNU/Linux) >> > > > > >> > > > > iEYEARECAAYFAlRGd60ACgkQV/LQcNdtPQPMjQCdF/28fx+VlhmZV0WEMobcv+7p >> > > > > OAEAn1mct5Iz5bWrLnnX/yQl13wMVwRg >> > > > > =RMTU >> > > > > -----END PGP SIGNATURE----- >> > > >> > > - -- >> > > - ------------------------------- >> > > Josh Thompson >> > > VCL Developer >> > > North Carolina State University >> > > >> > > my GPG/PGP key can be found at pgp.mit.edu >> > > >> > > All electronic mail messages in connection with State business which >> > > are sent to or received by this account are subject to the NC Public >> > > Records Law and may be disclosed to third parties. >> > > -----BEGIN PGP SIGNATURE----- >> > > Version: GnuPG v2.0.22 (GNU/Linux) >> > > >> > > iEYEARECAAYFAlRSgzoACgkQV/LQcNdtPQMRBgCfaBOvBvUNnfdr5BLR7cnr+vMx >> > > 5uQAniwpvKVjVGdW//rvNBYCKu485Siy >> > > =xWum >> > > -----END PGP SIGNATURE----- >> - -- >> - ------------------------------- >> Josh Thompson >> VCL Developer >> North Carolina State University >> >> my GPG/PGP key can be found at pgp.mit.edu >> >> All electronic mail messages in connection with State business which >> are sent to or received by this account are subject to the NC Public >> Records Law and may be disclosed to third parties. >> -----BEGIN PGP SIGNATURE----- >> Version: GnuPG v2.0.22 (GNU/Linux) >> >> iEYEARECAAYFAlRSieYACgkQV/LQcNdtPQOjlACfS97i9Hk0Tmspy+6vfgOcENht >> UtYAn008A0w5VBq+eg5iGg1GRlff3zDB >> =tCAJ >> -----END PGP SIGNATURE----- >> >> >
