-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Khder,
Mapping is just a way for the scheduler part of the frontend to know what computers a given image can be loaded on.This could be because of an architectural restriction or it could simply be because a researcher bought her own hardware and only her images should be run on them. Mapping doesn't actually have to do with where the image files reside. I don't think the backend code ever even looks at the mapping information. Remember that VCL predates the terms cloud computing, IaaS, PaaS, etc. So, the terms used for VCL might be somewhat different from more recent systems. In VCL, each "image" or "environment" consists of a set of disk image files and the associated information about the image that is in the database. Initial "base images" must be created somewhat manually. Then, further images are derived from those base images. I don't know that VCL really has flavors because each image will have its own CPU and RAM specifications. Does that answer your questions? Josh On Tuesday, November 04, 2014 11:00:01 PM Khder Omar wrote: > Dear Josh, > > I was wondering during the creation of new image, it does not require to be > stored in the computer node itself, it can be stored in an image library > like a SAN or NAS storage then we mapped a given image to a certain > computer or computer group ! the fact to 'map' an image or image group to a > computer or computer group, does it mean 'streaming' the image 'locally' to > the computer node ? or only during the reservation ? > I was bit confused between the terms used in VCL and in a typical IaaS > environment: the image is claimed as a template of a virtual machine, in > VCL, I have noticed that the image is being created from a base OS and > resources information are assigned in the database such as RAM, CPU, arch, > id etc... > Then I might assume [correct me if I understood it wrong] the Virtual > Machine will be initiated [considering an Hypervisor KVM or ESXi ] while > processing the reservation. Therefore, we can describe such image in a > typical IaaS as a virtual machine flavor ready to be deployed and run on > Host or Computer [within an hypervisor supported]. > > Thanks! > > PhD team > > 2014-10-30 20:07 GMT+01:00 Khder Omar <[email protected]>: > > Dear Josh, > > > > That is true ! Thanks again for the clarification ! I will start doing > > some experiments and see what it gives from performance perspective ! In > > the other hand, I will keep updating for any proposal of ideas can be > > generated then we can refine the scheduler code and retest again ! > > > > Many thanks for sharing such valuable information! > > > > We will keep updated ! > > > > Best Regards > > PhD Team > > > > 2014-10-30 19:56 GMT+01:00 Josh Thompson <[email protected]>: > >> -----BEGIN PGP SIGNED MESSAGE----- > >> Hash: SHA1 > >> > >> Khder, > >> > >> The scheduler only allocates a computer randomly if > >> SCHEDULER_ALLOCATE_RANDOM_COMPUTER is set to 1 in conf.php, which it is > >> not by > >> default. > >> > >> Yes, you could create pools that each have a weight or ranking assigned. > >> Each > >> computer would be assigned to a pool. Then, when the initial set of > >> computers > >> is generated, they could be ranked by the pool weight/ranks. It would > >> then be > >> up to an administer or external application to manage the weight/rank of > >> each > >> pool. > >> > >> Josh > >> > >> On Thursday, October 30, 2014 7:48:50 PM Khder Omar wrote: > >> > Dear Josh, > >> > > >> > I would like to thank you for the precious hints concerning the > >> > >> scheduling > >> > >> > nature within VCL. I can see that the set of computers is filtered in > >> > advance and then the scheduler will pick up RANDOMLY one of them. To > >> > >> check > >> > >> > the possibilities to extend the function considering a large scale > >> > environment where we can introduce the 'region' or 'pool' term in the > >> > >> code. > >> > >> > With multiple management nodes, do you think that using the 'weight' > >> > for > >> > the workload per pool and then per computer therefore will rank it will > >> > >> be > >> > >> > more precise on how to balance the load across the hole infrastructure > >> > ? > >> > > >> > Best Regards, > >> > PhD team > >> > > >> > 2014-10-30 19:28 GMT+01:00 Josh Thompson <[email protected]>: > >> > > -----BEGIN PGP SIGNED MESSAGE----- > >> > > Hash: SHA1 > >> > > > >> > > Khder, > >> > > > >> > > Sorry for the late response. > >> > > > >> > > You can certainly experiment with extending the code to include more > >> > > advanced > >> > > scheduling functions. The existing ranking of computers is something > >> > >> like > >> > >> > > this: > >> > > > >> > > 1) generate a set of all computers meeting the minimum requirements > >> > >> of the > >> > >> > > image, that the user has access to, and that the image is mapped to; > >> > >> order > >> > >> > > that list by node specs from low to high > >> > > 2) remove from that set any computers that are already assigned > >> > > reservations > >> > > 3) if the image is virtual, remove any VMs for which the host doesn't > >> > >> have > >> > >> > > enough memory to load the VM (without overbooking) > >> > > 4) assign the new reservation the first computer from that set for > >> > >> which > >> > >> > > an > >> > > active management node can be found > >> > > > >> > > There's actually 3 sets of computers being tracked - those already > >> > >> loaded > >> > >> > > with > >> > > the image being requested, those set aside for a block allocation of > >> > >> which > >> > >> > > the > >> > > user is a member, and then a set of both of those plus any available > >> > > computers > >> > > that are currently loaded with another image. > >> > > > >> > > There is a configuration option (SCHEDULER_ALLOCATE_RANDOM_COMPUTER) > >> > >> in > >> > >> > > conf.php that allows the sets of computers to be randomized. The > >> > > motivation > >> > > behind this option is for sites that have homogeneous virtual > >> > >> machines and > >> > >> > > hosts. Randomizing the assigned computer should help spread the load > >> > > equally > >> > > among all existing hosts. > >> > > > >> > > I hope that helps. > >> > > > >> > > Josh > >> > > > >> > > On Tuesday, October 21, 2014 9:01:17 PM Khder Omar wrote: > >> > > > Hi Josh, > >> > > > > >> > > > Thanks for your reaction. I have checked the code source and I > >> > >> think the > >> > >> > > > only function which might fulfill my question is function > >> > > > >> > > allocComputer(...) > >> > > > >> > > > Eventually, the function a piece test schedule code > >> > > > > >> > > > if(SCHEDULER_ALLOCATE_RANDOM_COMPUTER) { > >> > > > > >> > > > shuffle($blockids); > >> > > > shuffle($currentids); > >> > > > shuffle($computerids); > >> > > > > >> > > > } > >> > > > > >> > > > > >> > > > I might assume if it was correctly understood that VCL scheduler > >> > >> process > >> > >> > > > determines in first place a computer to be assigned to a management > >> > >> node > >> > >> > > > from a given array or table in order otherwise it can be randomly > >> > > > chosen! > >> > > > Then, we can assume that the scheduling decision is still using a > >> > >> basic > >> > >> > > > order as FCFS or a chance/Random order! It that assumed be correct > >> > >> ? if > >> > >> > > so, > >> > > > >> > > > is there a way to extend the code by including more advanced > >> > >> scheduling > >> > >> > > > functions ? > >> > > > > >> > > > Best regards, > >> > > > PhD Team > >> > > > > >> > > > 2014-10-21 16:11 GMT+01:00 Josh Thompson <[email protected]>: > >> > > > > -----BEGIN PGP SIGNED MESSAGE----- > >> > > > > Hash: SHA1 > >> > > > > > >> > > > > Khder, > >> > > > > > >> > > > > The scheduling of reservations is actually done in the web code. > >> > >> Have > >> > >> > > a > >> > > > >> > > > > look > >> > > > > at the isAvailable function in the web/.ht-inc/utils.php file. > >> > > > > > >> > > > > https://svn.apache.org/repos/asf/vcl/trunk/web/.ht-inc/utils.php > >> > > > > > >> > > > > Josh > >> > > > > > >> > > > > On Tuesday, October 21, 2014 4:39:05 PM Khder Omar wrote: > >> > > > > > Dear all, > >> > > > > > > >> > > > > > We were wondering what kind of scheduler algorithms VCL might > >> > >> use ? > >> > >> > > Any > >> > > > >> > > > > > hints about the scheduler source code will be appreciated. The > >> > >> idea > >> > >> > > is > >> > > > >> > > > > > actually to check how VCL will perform while changing the > >> > >> scheduler > >> > >> > > > > > algorithm [in management node]. > >> > > > > > > >> > > > > > > >> > > > > > Thanks in advance > >> > > > > > > >> > > > > > Best regards, > >> > > > > > Phd team > >> > > > > > >> > > > > - -- > >> > > > > - ------------------------------- > >> > > > > Josh Thompson > >> > > > > VCL Developer > >> > > > > North Carolina State University > >> > > > > > >> > > > > my GPG/PGP key can be found at pgp.mit.edu > >> > > > > > >> > > > > All electronic mail messages in connection with State business > >> > >> which > >> > >> > > > > are sent to or received by this account are subject to the NC > >> > >> Public > >> > >> > > > > Records Law and may be disclosed to third parties. > >> > > > > -----BEGIN PGP SIGNATURE----- > >> > > > > Version: GnuPG v2.0.22 (GNU/Linux) > >> > > > > > >> > > > > iEYEARECAAYFAlRGd60ACgkQV/LQcNdtPQPMjQCdF/28fx+VlhmZV0WEMobcv+7p > >> > > > > OAEAn1mct5Iz5bWrLnnX/yQl13wMVwRg > >> > > > > =RMTU > >> > > > > -----END PGP SIGNATURE----- > >> > > > >> > > - -- > >> > > - ------------------------------- > >> > > Josh Thompson > >> > > VCL Developer > >> > > North Carolina State University > >> > > > >> > > my GPG/PGP key can be found at pgp.mit.edu > >> > > > >> > > All electronic mail messages in connection with State business which > >> > > are sent to or received by this account are subject to the NC Public > >> > > Records Law and may be disclosed to third parties. > >> > > -----BEGIN PGP SIGNATURE----- > >> > > Version: GnuPG v2.0.22 (GNU/Linux) > >> > > > >> > > iEYEARECAAYFAlRSgzoACgkQV/LQcNdtPQMRBgCfaBOvBvUNnfdr5BLR7cnr+vMx > >> > > 5uQAniwpvKVjVGdW//rvNBYCKu485Siy > >> > > =xWum > >> > > -----END PGP SIGNATURE----- > >> > >> - -- > >> - ------------------------------- > >> Josh Thompson > >> VCL Developer > >> North Carolina State University > >> > >> my GPG/PGP key can be found at pgp.mit.edu > >> > >> All electronic mail messages in connection with State business which > >> are sent to or received by this account are subject to the NC Public > >> Records Law and may be disclosed to third parties. > >> -----BEGIN PGP SIGNATURE----- > >> Version: GnuPG v2.0.22 (GNU/Linux) > >> > >> iEYEARECAAYFAlRSieYACgkQV/LQcNdtPQOjlACfS97i9Hk0Tmspy+6vfgOcENht > >> UtYAn008A0w5VBq+eg5iGg1GRlff3zDB > >> =tCAJ > >> -----END PGP SIGNATURE----- - -- - ------------------------------- Josh Thompson VCL Developer North Carolina State University my GPG/PGP key can be found at pgp.mit.edu All electronic mail messages in connection with State business which are sent to or received by this account are subject to the NC Public Records Law and may be disclosed to third parties. -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.22 (GNU/Linux) iEYEARECAAYFAlRal+cACgkQV/LQcNdtPQOAGACeL2Sanhs7ShZaCyKcSo5N2BYk wLEAn08YbcyiCmR4rY9u4kZJocfxY1Ju =pxdL -----END PGP SIGNATURE-----
