Re: VCL Architecture Change Proposal
I'm giving my own viewpoint - from outside the group working on the architecture - but I do have some experience with software projects. Brian writes: > This cloud proposal is new and different, and it will redesign and > likely recode the existing VCL core. Although it seems different, Well, the first sentence says it is different. :-) > this experimental architecture is designed to accommodate the current > roadmap through a different architecture implementation. The > fundamental assumption is that the most reasonable way to implement > the roadmap outlined here > (http://cwiki.apache.org/confluence/display/VCL/Future+Development+topics ) > is have the VCL core become resource agnostic. I believe that a resource > agnostic VCL core can schedule fluid resources more flexibly by revising > the daemon portion of the VCL architecture to be event driven. This > modification allows the core to be resource agnostic and schedule a > variety of fundamentally different resources through the use of resource > specific resource managers. This is IMHO a very desireable goal, and I think that we have widespread agreement on that. We've already been moving towards this goal in the 2.X code base - but via incremental changes rather than through a radical redesign/restructure/recode effort. So, to me, the question is of the route(s) we take rather than disagreement over the long term goal(s). > Moreover, with the 2.X code base, [... discussion of limitations of 2.X and > desired aspects of the new architecture] Given that we're trying to release a 2.X version with increased functionality and with increased ease of bringing into production (which is *very* important as the number of adopters increases) it seems sensible to me to focus on 2.X. Aaron has discussed the importance of this, especially considering the limited development resources. Does this mean we should abandon work on this new version (I'll call it 2.D with D for different)? No, IMHO. But neither do we need to take it on as an urgent project. We (the entire VCL community) have the time to scope it out carefully and to have a good discussion of the requirements, tradeoffs, implementation approaches, etc., as Josh has suggested. We'll also have discussed the roadmap for progress - e.g. whether there should be a fork in the code. Then, when development resources (people) become available we'll be ready to proceed in a very productive manner. > I'd like to suggest the reuse of the existing API enabled VCL 2.X code > base as a bare-metal resource manager, which can be driven by this > experimental branch of VCL code. ... This sound to me like the type of discussion that should be going on. --henry
Re: VCL Architecture Change Proposal
This cloud proposal is new and different, and it will redesign and likely recode the existing VCL core. Although it seems different, this experimental architecture is designed to accommodate the current roadmap through a different architecture implementation. The fundamental assumption is that the most reasonable way to implement the roadmap outlined here (http://cwiki.apache.org/confluence/display/VCL/Future+Development+topics ) is have the VCL core become resource agnostic. I believe that a resource agnostic VCL core can schedule fluid resources more flexibly by revising the daemon portion of the VCL architecture to be event driven. This modification allows the core to be resource agnostic and schedule a variety of fundamentally different resources through the use of resource specific resource managers. Moreover, with the 2.X code base, the frontend and backend components are inseparable through a single database. No component in the architecture can be rethought, modified, or redesigned because the architecture requires a very high level of integration between all components in the frontend, backend, and database. For example, as the developer of the ESX netboot provisioning module (esx.pm), most of the time was spent compensating for the static data model of the VCL core. Additionally, this new architecture will include a secure database abstraction layer instead of the use of insecure SQL executed without the use of prepared statements and welded to MySQL as the current architecture is. I'd like to suggest the reuse of the existing API enabled VCL 2.X code base as a bare-metal resource manager, which can be driven by this experimental branch of VCL code. The existing VCL 2.x is well fitted to managing bare-metal resources. Through the proposed VCL architecture, the existing VCL 2.X code would manage bare-metal resources while allowing the new core to aggregate a variety of new and unforeseen resource environments. For example, VCL could schedule fluid resources more efficiently such as ESX, KVM, or XEN, or specialized resources such as Storage, J2EE environments, python execution environments, mainframe Z environments, etc ... Best, Brian Brian Bouterse Secure Open Systems Initiative 919.698.8796 On May 11, 2009, at 3:23 PM, Josh Thompson wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Brian, I haven't finished reading through all of the details in the document you created in the wiki, but it seems to be a design document for a new/ different cloud system. I didn't see anything explaining how it relates to VCL in its current form or a roadmap providing a migration path from the current form to what you've designed. That aside, can you explain further why you are proposing this? I think I'm seeing a bit of a cart before the horse issue here where you've designed out something new before there's been any discussion on this list about whether or not large parts of VCL should be redesigned. After explaining your reasoning further, I think it would be a good idea for us (the community) to have a discussion on this list about any areas of VCL that are needing to be redesigned. Josh On Monday May 11, 2009, Brian Bouterse wrote: I propose exploring an alternative VCL architecture through an experimental code branch at Apache.org. This code would be housed in the svn at apache.org as a branch, and would be explored/developed in parallel with the current 2.X branch. The purpose of this branch is to explore a VCL architecture that allows the following: To allow VCL manage fluid resources (virtualized) more effectively Increase modularity by decoupling VCL components from a single, monolithic database via APIs Manage storage resources Create a database abstraction layer Increase image library confidentiality and integrity I've put a first-pass design document on the apache VCL wiki which outlines the architecture proposed with some level of detail. That document is located here: http://cwiki.apache.org/confluence/display/VCL/Experimental+VCL+Architectur e+Document I'd like feedback of all kinds from the Apache VCL community to explore the architecture's merits and challenges at both the architectural and implementation levels. Best, Brian Brian Bouterse Secure Open Systems Initiative 919.698.8796 - -- - --- Josh Thompson Systems Programmer Virtual Computing Lab (VCL) North Carolina State University josh_thomp...@ncsu.edu 919-515-5323 my GPG/PGP key can be found at pgp.mit.edu -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.6 (GNU/Linux) iD8DBQFKCHssV/LQcNdtPQMRAnbEAJ9bCBXcamP/UsEPduQIPU8Wi/B0EgCdFxwz O1sBuRimEG3+sIGp9kr1pug= =hNc7 -END PGP SIGNATURE-
Re: VCL Architecture Change Proposal
Hi Brian, I have some additional concerns. Don't take this the wrong way, it's great to think ahead, but I not sure this is the right time for such a major proposal. As you know the apache VCL project is still in incubation stage and we're struggling to get off the ground, grow the community, cut the first release, name change issues, etc. Proposing a major Architecture change at this time is probably not the best approach in working through the other tasks, it's going to muddy the waters even more. As Josh mentioned maybe a better approach is to start discussions on the list around areas of improvement for the existing VCL code base before jumping directly to a redesign/architecture change. If the improvements require an architecture change then great, else if the changes can be added in to existing code base also great. Best Regards, Aaron --On May 11, 2009 3:23:23 PM -0400 Josh Thompson wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Brian, I haven't finished reading through all of the details in the document you created in the wiki, but it seems to be a design document for a new/different cloud system. I didn't see anything explaining how it relates to VCL in its current form or a roadmap providing a migration path from the current form to what you've designed. That aside, can you explain further why you are proposing this? I think I'm seeing a bit of a cart before the horse issue here where you've designed out something new before there's been any discussion on this list about whether or not large parts of VCL should be redesigned. After explaining your reasoning further, I think it would be a good idea for us (the community) to have a discussion on this list about any areas of VCL that are needing to be redesigned. Josh On Monday May 11, 2009, Brian Bouterse wrote: I propose exploring an alternative VCL architecture through an experimental code branch at Apache.org. This code would be housed in the svn at apache.org as a branch, and would be explored/developed in parallel with the current 2.X branch. The purpose of this branch is to explore a VCL architecture that allows the following: To allow VCL manage fluid resources (virtualized) more effectively Increase modularity by decoupling VCL components from a single, monolithic database via APIs Manage storage resources Create a database abstraction layer Increase image library confidentiality and integrity I've put a first-pass design document on the apache VCL wiki which outlines the architecture proposed with some level of detail. That document is located here: http://cwiki.apache.org/confluence/display/VCL/Experimental+VCL+Architec tur e+Document I'd like feedback of all kinds from the Apache VCL community to explore the architecture's merits and challenges at both the architectural and implementation levels. Best, Brian Brian Bouterse Secure Open Systems Initiative 919.698.8796 - -- - --- Josh Thompson Systems Programmer Virtual Computing Lab (VCL) North Carolina State University josh_thomp...@ncsu.edu 919-515-5323 my GPG/PGP key can be found at pgp.mit.edu -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.6 (GNU/Linux) iD8DBQFKCHssV/LQcNdtPQMRAnbEAJ9bCBXcamP/UsEPduQIPU8Wi/B0EgCdFxwz O1sBuRimEG3+sIGp9kr1pug= =hNc7 -END PGP SIGNATURE- Aaron Peeler OIT Advanced Computing College of Engineering-NCSU 919.513.4571 http://vcl.ncsu.edu
Re: VCL Architecture Change Proposal
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Brian, I haven't finished reading through all of the details in the document you created in the wiki, but it seems to be a design document for a new/different cloud system. I didn't see anything explaining how it relates to VCL in its current form or a roadmap providing a migration path from the current form to what you've designed. That aside, can you explain further why you are proposing this? I think I'm seeing a bit of a cart before the horse issue here where you've designed out something new before there's been any discussion on this list about whether or not large parts of VCL should be redesigned. After explaining your reasoning further, I think it would be a good idea for us (the community) to have a discussion on this list about any areas of VCL that are needing to be redesigned. Josh On Monday May 11, 2009, Brian Bouterse wrote: > I propose exploring an alternative VCL architecture through an > experimental code branch at Apache.org. This code would be housed in > the svn at apache.org as a branch, and would be explored/developed in > parallel with the current 2.X branch. The purpose of this branch is > to explore a VCL architecture that allows the following: > > To allow VCL manage fluid resources (virtualized) more effectively > Increase modularity by decoupling VCL components from a single, > monolithic database via APIs > Manage storage resources > Create a database abstraction layer > Increase image library confidentiality and integrity > > I've put a first-pass design document on the apache VCL wiki which > outlines the architecture proposed with some level of detail. That > document is located here: > http://cwiki.apache.org/confluence/display/VCL/Experimental+VCL+Architectur >e+Document > > I'd like feedback of all kinds from the Apache VCL community to > explore the architecture's merits and challenges at both the > architectural and implementation levels. > > Best, > Brian > > > Brian Bouterse > Secure Open Systems Initiative > 919.698.8796 - -- - --- Josh Thompson Systems Programmer Virtual Computing Lab (VCL) North Carolina State University josh_thomp...@ncsu.edu 919-515-5323 my GPG/PGP key can be found at pgp.mit.edu -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.6 (GNU/Linux) iD8DBQFKCHssV/LQcNdtPQMRAnbEAJ9bCBXcamP/UsEPduQIPU8Wi/B0EgCdFxwz O1sBuRimEG3+sIGp9kr1pug= =hNc7 -END PGP SIGNATURE-
VCL Architecture Change Proposal
I propose exploring an alternative VCL architecture through an experimental code branch at Apache.org. This code would be housed in the svn at apache.org as a branch, and would be explored/developed in parallel with the current 2.X branch. The purpose of this branch is to explore a VCL architecture that allows the following: To allow VCL manage fluid resources (virtualized) more effectively Increase modularity by decoupling VCL components from a single, monolithic database via APIs Manage storage resources Create a database abstraction layer Increase image library confidentiality and integrity I've put a first-pass design document on the apache VCL wiki which outlines the architecture proposed with some level of detail. That document is located here: http://cwiki.apache.org/confluence/display/VCL/Experimental+VCL+Architecture+Document I'd like feedback of all kinds from the Apache VCL community to explore the architecture's merits and challenges at both the architectural and implementation levels. Best, Brian Brian Bouterse Secure Open Systems Initiative 919.698.8796
