Re: VCL Experimental Architecture
I agree. There are much larger issues that need to be resolved before discussing implementation details in this forum. I think it's great that the folks discussing the experimental architecture have identified areas where VCL can be improved and I would like to find a way for the community to efficiently and prudently incorporate them into VCL. I would like to understand why these ideas can't be incrementally incorporated and tested into the current VCL codebase. I think this approach would have a much higher chance of success, no? Was this approach considered? If so, what were the barriers to this approach which couldn't be resolved? It will be counterproductive if a developer has to choose one version or the other as the basis of his or her contributions. This will result in a split community. In addition, having 2 vastly different architectures under the VCL project will cause significant confusion. Agree? Regards, Andy Josh Thompson wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On Sunday May 17, 2009, Michael Brown wrote: The wiki document for the experimental branch now suggests the use of a database abstraction layer. I propose the use of an object relational mapper to accomplish this abstraction. This has the advantage of being extremely easy to use, less code needs to be written, therefore fewer errors will result. The con to an object relational mapper is performence during bulk database operations. This should not be an issue because vcl performs no large transactions. Does the community agree that any future major proposals should include a database abstraction layer using an object relational mapper because of the above advantages? This part of the community (me) doesn't agree with any implementation details of the so called "experimental branch" until we've had good discussion about problems with VCL in its current form and seriously looked at how those problems could be fixed without doing a complete rewrite. (I put "experimental branch" in quotes because a branch implies being a derivative work of trunk, but in this case it is a completely separate architecture designed without the input of those that developed the existing codebase.) I suggest work on this "experimental branch" halt until the community can come together on this issue instead of being fractured any further. Josh - -- - --- Josh Thompson Systems Programmer Virtual Computing Lab (VCL) North Carolina State University josh_thomp...@ncsu.edu 919-515-5323 my GPG/PGP key can be found at pgp.mit.edu -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.6 (GNU/Linux) iD8DBQFKEW2TV/LQcNdtPQMRAsREAJ9bcdCZRlgqaUR410njudbKln5YRQCbBEtt PWAY2YWVpxc1qBmhKctwR/0= =dapR -END PGP SIGNATURE- -- Andy Kurth Virtual Computing Lab Office of Information Technology North Carolina State University andy_ku...@ncsu.edu 919.513.4090
Re: VCL Experimental Architecture
I agree with Josh. To date there has only been a list of "problems" and a diagram. There as not been much(if any) discussion on the details of the problems mentioned. The current code base is proven and is doing what it is tasked to do. Currently it is managing close to a 1000 nodes and supporting over 10,000 users, and has served over 72,000 desktops(requests) since Jan 1, 2009 at NCSU and for NCSU's partners. So if we can, lets start a discussion which is focused on the set of tasks that others in the community need (or would like) VCL to handle. Aaron > -BEGIN PGP SIGNED MESSAGE- > Hash: SHA1 > > On Sunday May 17, 2009, Michael Brown wrote: >> The wiki document for the experimental branch now suggests the use of a >> database abstraction layer. I propose the use of an object relational >> mapper to accomplish this abstraction. This has the advantage of being >> extremely easy to use, less code needs to be written, therefore fewer >> errors will result. The con to an object relational mapper is >> performence >> during bulk database operations. This should not be an issue because >> vcl >> performs no large transactions. >> >> Does the community agree that any future major proposals should include >> a >> database abstraction layer using an object relational mapper because of >> the >> above advantages? > > This part of the community (me) doesn't agree with any implementation > details > of the so called "experimental branch" until we've had good discussion > about > problems with VCL in its current form and seriously looked at how those > problems could be fixed without doing a complete rewrite. (I > put "experimental branch" in quotes because a branch implies being a > derivative work of trunk, but in this case it is a completely separate > architecture designed without the input of those that developed the > existing > codebase.) > > I suggest work on this "experimental branch" halt until the community can > come together on this issue instead of being fractured any further. > > Josh > - -- > - --- > Josh Thompson > Systems Programmer > Virtual Computing Lab (VCL) > North Carolina State University > > josh_thomp...@ncsu.edu > 919-515-5323 > > my GPG/PGP key can be found at pgp.mit.edu > -BEGIN PGP SIGNATURE- > Version: GnuPG v1.4.6 (GNU/Linux) > > iD8DBQFKEW2TV/LQcNdtPQMRAsREAJ9bcdCZRlgqaUR410njudbKln5YRQCbBEtt > PWAY2YWVpxc1qBmhKctwR/0= > =dapR > -END PGP SIGNATURE- >
Re: VCL Experimental Architecture
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On Sunday May 17, 2009, Michael Brown wrote: > The wiki document for the experimental branch now suggests the use of a > database abstraction layer. I propose the use of an object relational > mapper to accomplish this abstraction. This has the advantage of being > extremely easy to use, less code needs to be written, therefore fewer > errors will result. The con to an object relational mapper is performence > during bulk database operations. This should not be an issue because vcl > performs no large transactions. > > Does the community agree that any future major proposals should include a > database abstraction layer using an object relational mapper because of the > above advantages? This part of the community (me) doesn't agree with any implementation details of the so called "experimental branch" until we've had good discussion about problems with VCL in its current form and seriously looked at how those problems could be fixed without doing a complete rewrite. (I put "experimental branch" in quotes because a branch implies being a derivative work of trunk, but in this case it is a completely separate architecture designed without the input of those that developed the existing codebase.) I suggest work on this "experimental branch" halt until the community can come together on this issue instead of being fractured any further. Josh - -- - --- Josh Thompson Systems Programmer Virtual Computing Lab (VCL) North Carolina State University josh_thomp...@ncsu.edu 919-515-5323 my GPG/PGP key can be found at pgp.mit.edu -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.6 (GNU/Linux) iD8DBQFKEW2TV/LQcNdtPQMRAsREAJ9bcdCZRlgqaUR410njudbKln5YRQCbBEtt PWAY2YWVpxc1qBmhKctwR/0= =dapR -END PGP SIGNATURE-
VCL Experimental Architecture
The wiki document for the experimental branch now suggests the use of a database abstraction layer. I propose the use of an object relational mapper to accomplish this abstraction. This has the advantage of being extremely easy to use, less code needs to be written, therefore fewer errors will result. The con to an object relational mapper is performence during bulk database operations. This should not be an issue because vcl performs no large transactions. Does the community agree that any future major proposals should include a database abstraction layer using an object relational mapper because of the above advantages?
RE: VCL Experimental Architecture
Before we go down the path of creating twice as much work for half the gain let me challenge a core assumption here: contrary to what is being assumed, the PaaS model will not work for education unless structured within an IaaS framework! This is not because of elegant technical considerations but rather the most basic and practical of considerations--within the educational domain the "achievable" economics of the two models are significantly different with the decisive advantage going to the IaaS model! The proof is somewhat tedious but I will work it up. In the meantime, could we do this in a way that minimizes the reverse engineering necessary to port the key functionality of what you are purposing to do back into a more viable economic framework? I will concede up front and without debate that what you are proposing is a better solution for certain service provider scenarios. But with that said, it is NOT a better solution for the grossly underserved educational community where cost is a tradeoff between free lunch and technology facilitated learning opportunities. The harsh reality is that cost is the single most, and sometimes the only, relevant criteria in making educational decisions. Absent affordability it doesn't much matter what the solution looks like because the underserved will not be using it. If we want a different outcome, this is where it has to begin. Sam Averitt -Original Message- From: John Bass [mailto:jcbassoma...@gmail.com] Sent: Thursday, May 14, 2009 4:16 PM To: vcl-dev@incubator.apache.org Subject: Re: VCL Experimental Architecture I agree. Here is a starter list of high level issues: - architecture modularity . core . resource management . gui/api . operational metrics and logging . component templating - security model - database independence/abstraction - language independence - storage implementation (user and image) - virtualization management - future extensibility - experimental development branch I'm working on a summary of security model issues. The idea behind the vcl security model is to list and encapsulate each of the components, objects, roles, etc in the architecture so that most any security model can be implemented instead of locking the project into a particular security model (although we'll have to start with one model). Currently, the security model is undefined and 'hard-wired' into the architecture. This greatly limits the ability for vcl to be deployed into new environments with different security policies or to adapt to future security policies in currently deployed environments. More on this later. Please feel free to jump in on a topic. There is lots to discuss. John Bass john_b...@ncsu.edu www.sosi.ncsu.edu www.cnl.ncsu.edu (919) 515-0154 On Thu, May 14, 2009 at 3:40 PM, Josh Thompson wrote: > -BEGIN PGP SIGNED MESSAGE- > Hash: SHA1 > > I really think that before any more implementation design work is done > toward > long term restructuring, a lot of high level discussion and design work > needs > to be done. It causes a fractured community if a few people in the > community > decide to redesign large parts of the codebase without having discussions > about it on this list. > > Josh > > On Thursday May 14, 2009, Andrew Brown wrote: > > I've just made a clarification on the vcl experimental architecture > > document: > > > > In regards to the core's frontend API interface about values including > > image_id, ticket_id, and so fourth, the frontend needs to map which > > resource manager corresponds to those values, but they may or may not be > > unique. To solve this, it needs to be paired with its resource manager > id. > > Here's how that will work: > > > > "Above" the core, i.e. in communication with a frontend, values such as > > image_id and ticket_id will be a tuple: (resource_manager_id, actual_id) > > > > "Below" the core, i.e. in communication with a resource manager, these > > values will be just a single integer. > > > > This way the core won't need to keep some kind of mapping (which wouldn't > > work anyways since the ids are only unique to a single resource manager) > > nor will it need to poll all resource managers for the right one. The > > frontend will be responsible for passing both values back to the core. > > -Andrew > - -- > - --- > Josh Thompson > Systems Programmer > Virtual Computing Lab (VCL) > North Carolina State University > > josh_thomp...@ncsu.edu > 919-515-5323 > > my GPG/PGP key can be found at pgp.mit.edu > -BEGIN PGP SIGNATURE- > Version: GnuPG v1.4.6 (GNU/Linux) > > iD8DBQFKDHO3V/LQcNdtPQMRAnZyAJ9zI+/qLJmqFiqKETNdIU2vgJ2/PwCeLbSp > Q/0EoJ3rssbEgdUAEjc6yIQ= > =5H1g > -END PGP SIGNATURE- >
Re: VCL Experimental Architecture
I agree. Here is a starter list of high level issues: - architecture modularity . core . resource management . gui/api . operational metrics and logging . component templating - security model - database independence/abstraction - language independence - storage implementation (user and image) - virtualization management - future extensibility - experimental development branch I'm working on a summary of security model issues. The idea behind the vcl security model is to list and encapsulate each of the components, objects, roles, etc in the architecture so that most any security model can be implemented instead of locking the project into a particular security model (although we'll have to start with one model). Currently, the security model is undefined and 'hard-wired' into the architecture. This greatly limits the ability for vcl to be deployed into new environments with different security policies or to adapt to future security policies in currently deployed environments. More on this later. Please feel free to jump in on a topic. There is lots to discuss. John Bass john_b...@ncsu.edu www.sosi.ncsu.edu www.cnl.ncsu.edu (919) 515-0154 On Thu, May 14, 2009 at 3:40 PM, Josh Thompson wrote: > -BEGIN PGP SIGNED MESSAGE- > Hash: SHA1 > > I really think that before any more implementation design work is done > toward > long term restructuring, a lot of high level discussion and design work > needs > to be done. It causes a fractured community if a few people in the > community > decide to redesign large parts of the codebase without having discussions > about it on this list. > > Josh > > On Thursday May 14, 2009, Andrew Brown wrote: > > I've just made a clarification on the vcl experimental architecture > > document: > > > > In regards to the core's frontend API interface about values including > > image_id, ticket_id, and so fourth, the frontend needs to map which > > resource manager corresponds to those values, but they may or may not be > > unique. To solve this, it needs to be paired with its resource manager > id. > > Here's how that will work: > > > > "Above" the core, i.e. in communication with a frontend, values such as > > image_id and ticket_id will be a tuple: (resource_manager_id, actual_id) > > > > "Below" the core, i.e. in communication with a resource manager, these > > values will be just a single integer. > > > > This way the core won't need to keep some kind of mapping (which wouldn't > > work anyways since the ids are only unique to a single resource manager) > > nor will it need to poll all resource managers for the right one. The > > frontend will be responsible for passing both values back to the core. > > -Andrew > - -- > - --- > Josh Thompson > Systems Programmer > Virtual Computing Lab (VCL) > North Carolina State University > > josh_thomp...@ncsu.edu > 919-515-5323 > > my GPG/PGP key can be found at pgp.mit.edu > -BEGIN PGP SIGNATURE- > Version: GnuPG v1.4.6 (GNU/Linux) > > iD8DBQFKDHO3V/LQcNdtPQMRAnZyAJ9zI+/qLJmqFiqKETNdIU2vgJ2/PwCeLbSp > Q/0EoJ3rssbEgdUAEjc6yIQ= > =5H1g > -END PGP SIGNATURE- >
Re: VCL Experimental Architecture
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 I really think that before any more implementation design work is done toward long term restructuring, a lot of high level discussion and design work needs to be done. It causes a fractured community if a few people in the community decide to redesign large parts of the codebase without having discussions about it on this list. Josh On Thursday May 14, 2009, Andrew Brown wrote: > I've just made a clarification on the vcl experimental architecture > document: > > In regards to the core's frontend API interface about values including > image_id, ticket_id, and so fourth, the frontend needs to map which > resource manager corresponds to those values, but they may or may not be > unique. To solve this, it needs to be paired with its resource manager id. > Here's how that will work: > > "Above" the core, i.e. in communication with a frontend, values such as > image_id and ticket_id will be a tuple: (resource_manager_id, actual_id) > > "Below" the core, i.e. in communication with a resource manager, these > values will be just a single integer. > > This way the core won't need to keep some kind of mapping (which wouldn't > work anyways since the ids are only unique to a single resource manager) > nor will it need to poll all resource managers for the right one. The > frontend will be responsible for passing both values back to the core. > -Andrew - -- - --- Josh Thompson Systems Programmer Virtual Computing Lab (VCL) North Carolina State University josh_thomp...@ncsu.edu 919-515-5323 my GPG/PGP key can be found at pgp.mit.edu -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.6 (GNU/Linux) iD8DBQFKDHO3V/LQcNdtPQMRAnZyAJ9zI+/qLJmqFiqKETNdIU2vgJ2/PwCeLbSp Q/0EoJ3rssbEgdUAEjc6yIQ= =5H1g -END PGP SIGNATURE-
VCL Experimental Architecture
I've just made a clarification on the vcl experimental architecture document: In regards to the core's frontend API interface about values including image_id, ticket_id, and so fourth, the frontend needs to map which resource manager corresponds to those values, but they may or may not be unique. To solve this, it needs to be paired with its resource manager id. Here's how that will work: "Above" the core, i.e. in communication with a frontend, values such as image_id and ticket_id will be a tuple: (resource_manager_id, actual_id) "Below" the core, i.e. in communication with a resource manager, these values will be just a single integer. This way the core won't need to keep some kind of mapping (which wouldn't work anyways since the ids are only unique to a single resource manager) nor will it need to poll all resource managers for the right one. The frontend will be responsible for passing both values back to the core. -Andrew
