I'm not really familiar with SmartOS (looks like its built on OpenSolaris or similar). +1 for dtrace. Great stuff in there.
But I'd like to just put it out there to keep an eye on for the differences between SmartOS and Linux. Most deployers would probably want to run on Linux (RH/CentOS or Ubuntu) and not Solaris. And some of the scripts I've seen in the puppet-hilary project look like they're assuming solaris-like commands and directory structures. Puppet can mask most of these differences if the scripts are converted into puppet manifests. Erik On Fri, Oct 19, 2012 at 2:22 PM, Nicolaas Matthijs <nicolaas.matth...@caret.cam.ac.uk> wrote: > Hi Mark, > > I think it means that the project starts to regard a cloud deployment as a > prime deployment strategy and makes decisions with that in mind. > > The stack choice is geared towards a scenario where we can scale outwards > linearly and add new nodes on the fly to deal with peak load, and there's a > desire to support this across multiple data centres as well. A lot of this > came out of discussions with OmniTI. Obviously, it's too soon to claim that > all of this will work, but we're trying to make it an inherent part of our > testing and performance measuring strategy. > > As part of the development process, a reference environment has been set up, > which will be used to run nightly performance tests. We started off using > Amazon EC2, but have now switched to Joyent because of the need for SmartOS > and dtrace for performance debugging. Because of that, one of the things that > the project will produce is a set of puppet scrips to help set up the various > VMs involved in a deployment on one of these cloud providers. > > Whilst doing feature development, we'll also be looking at cloud-specific > services to build features. For example, we might provide a hook that allows > for using Amazon S3 for storing user-generated content instead of local file > storage. > > A lot of this goes hand in hand with the goal to be a proper multi-tenant > application, where deployment infrastructure can be shared between multiple > institutions and an anticipation of a future where this will become more and > more common. > > Hope that helps, > Nicolaas > > > > On 19 Oct 2012, at 15:52, markjnor...@earthlink.net wrote: > >> What exactly does "cloud ready" mean in this context? Good idea, I'd like >> to understand what you mean by it. >> >> - Mark Norton >> >> >> -----Original Message----- >>> From: Nicolaas Matthijs <nicolaas.matth...@caret.cam.ac.uk> >>> Sent: Oct 18, 2012 7:25 PM >>> To: openfo...@collab.sakaiproject.org, "oae-dev >>> oae-dev@collab.sakaiproject.org" <oae-dev@collab.sakaiproject.org> >>> Subject: [DG: Open Forum] Sakai OAE Update >>> >>> Hi everyone, >>> >>> As most of you know, a number of Sakai OAE stakeholder organizations have >>> withdrawn from the managed project over the last 6 months. Key reasons >>> cited for these departures include: >>> >>> - Concerns about the scalability and viability of the Nakamura architecture >>> - Slower than desired progress on feature development >>> - Misalignments between stakeholders >>> >>> These withdrawals have had a significant negative impact on the level of >>> financial and in-kind contributions underwriting the development team. In >>> addition, reports from piloting institutions together with performance test >>> data produced by the team have made it clear that OAE’s performance and >>> scalability characteristics leave much to be desired. While sufficient for >>> small-scale pilots involving a few thousand users, OAE’s performance under >>> load is wholly insufficient for adopters considering an enterprise >>> production deployment supporting tens of thousands of users and above, >>> including the web-scale deployments we are likely to see in the future. >>> Upgrading between versions also remains a challenging exercise for >>> operations teams. While the remaining stakeholders believe that the >>> concepts embodied in OAE’s design work remain relevant and valuable, >>> current circumstances dictate that we adopt a new technical approach. >>> >>> In late August we recommended to the project stakeholders that the managed >>> project cease development on OAE Nakamura, setting aside its technical debt >>> in favor of a re-engineered back-end that re-positions the project as a >>> multi-tenant, highly scalable, cloud-ready application. The strategy that >>> we set out involves an initial and temporary reduction in application >>> scope; existing features will be disabled until their corresponding >>> back-end services have been recreated. Our proposal was accepted, and in >>> early September a reconstructed and considerably leaner project team >>> commenced work on providing a new back-end architecture written in node.js >>> and using Apache Cassandra for persistence.[1] The code is licensed ECL >>> 2.0 and is publicly available in a Github repo named “Hilary”.[2] >>> Additional repos have been created for the test data model loader, Tsung >>> performance tests, nightly performance test scripts and puppet deployment >>> scripts.[3] >>> >>> We have retained OmniTI, a web performance and scalability consultancy, to >>> help guide our work. A series of one-week sprints have been organized in >>> order to keep the pace of development high as well as allowing for quick >>> resets under the “fail fast” rule. We start from a set of purposefully >>> designed and documented REST APIs (see attachment). Each REST API and its >>> accompanying internal service will include unit and integration tests. The >>> team will set up and maintain a reference deployment in the cloud, on which >>> extensive nightly performance tests will be run. The existing UI, >>> benefiting from a number of client-side performance improvements included >>> in the OAE 1.4.0 release, will be retained along with the Widget Library >>> and SDK. Administrative UIs will be added and there's also a desire to >>> start introducing some of the new unimplemented UI designs along the way, >>> as these have been through several rounds of usability testing and were >>> presented to the community at the summer Sakai/Jasig conference. >>> >>> The team has set itself a mid-December deadline--with intermediate >>> deadlines and sanity checks along the way--to produce working code that >>> demonstrates that the chosen path is viable. Attached is the September >>> progress report outlining the first 30 days of work along with our API doc >>> and “alternative scenarios” proposal. While the proposed architecture >>> remains under evaluation and could change, we believe that our >>> recommendation to retire OAE Nakamura in favor of a re-architected >>> multi-tenant, cloud-ready platform featuring purposefully designed REST >>> APIs and services that are properly instrumented and continually measured >>> is the right approach to take. >>> >>> Kind regards, >>> Nicolaas and Anthony >>> >>> [1] http://nodejs.org/, http://cassandra.apache.org/ >>> [2] https://github.com/sakaiproject/Hilary >>> [3] https://github.com/sakaiproject/OAE-model-loader, >>> https://github.com/sakaiproject/node-oae-tsung, >>> https://github.com/sakaiproject/oae-nightly-stats, >>> https://github.com/sakaiproject/puppet-hilary >>> >> > > _______________________________________________ > oae-dev mailing list > oae-dev@collab.sakaiproject.org > http://collab.sakaiproject.org/mailman/listinfo/oae-dev _______________________________________________ oae-dev mailing list oae-dev@collab.sakaiproject.org http://collab.sakaiproject.org/mailman/listinfo/oae-dev
