Ok. Those of you who lurk in #chicken on irc.freenode.net will probably be familiar with how we all like to main about Ruby on Rails.
As it happens, before I returned to Scheme from my long hiatus out in the horrible world of commercial software development (eg, Java, PHP, Perl and C++) I was finding the world of web application development rather impoverished of tools; in that whenever I wrote a web app, I was always spending far more time on stuff I felt should be automated - or, conversely, having to fight some toolkit that promises to automate everything, but then makes it hard to do things outside of its limited view of the world. Being the kind of person who's fond of thinking of designs for things, I naturally began tinkering with ideas in my head about how I'd fix this, when I happened to come across Paul Graham's writings about Lisp, take a look at the current state of Scheme implementations again, found things muchly advanced from when I last learnt languages for run rather than because somebody was offering me money to code in the thing, came across Chicken, joined #chicken, and met Peter Bex who (a) grumbled about Ruby on Rails even more than me and (b) maintains Spiffy. So we got chatting, and I quickly began to adapt my sketched ideas for a web app framework to make use of Scheme's strengths. And the name Chicken Wings was suggested, rather cleverly implying freedom and flight rather than being stuck on a one-dimensional set of rails! The conclusion I came to is this: 1) A web app could consist of a .scm file that loads spiffy and third- party extensions, then proceeds to load or define application- specific business logic procedures, thus leading to a nice global environment full of tools, and registering resources of a more dynamic nature in a hash table. The framework defines a parameter, wings-environment, which is bound to a closure that accepts a key and returns the value associated with that key in the global hash table, or nil otherwise. The reason for this will become clear in a moment. 2) The web app finally, having set up its environment, starts spiffy serving files out of a document root directory. 3) These files contain the frontend of the application. Static files (CSS, HTML, images, etc), as well as files that spiffy knows to handle specially. All of the existing spiffy handlers will fit into the Wings model OK, but there's a few more I have in mind. More on those later. 4) There are web app frameworks that are "continuation based" or "widget based" or various other paradigms. Each of these paradigms is nice for some things, but sucks for others. Continuation systems are great for things with a very sequential page flow, like multi-page signup processes and complex operations. Widget systems are great because you can bind the widgets to your data sources easily. Frameworks like Rails are based around an object-relational mapping to SQL databases, and if you use that supplied data store, it automates a load of stuff to do with generating pages from data in SQL or forms that edit same; but if you want to edit data stored in other kinds of database, you have to do much more work yourself. Etc. In general, I find existing frameworks too monolithic. Therefore, with Wings, you just use Spiffy's map-URLs-to-files-and- decide-what-to-do-with-them logic. You can have as many different file type handlers as you like. One could write a handler that implements continuations; when a request for the page comes in, the handler looks for a continuation ID in the request. If there isn't one, it starts a new 'session' from the defined entry point, otherwise resumes a saved continuation. Then one could use continuations for parts of the app that benefit from it, while not using them elsewhere. One could write a handler that provides a quick and dirty admin interface to your SQL data. The actual file on disk just contains metadata about how the tables link together and how fields should be interpreted and so on. The handler turns this into an admin interface, like Django provides. One could write a handler that takes a page description that uses smart widgets, and handle HTML form state in a snazzy way, with automatic loading of widget state from a data source at the start, and saving of it back when the form is submitted. This model also means that you can allow front-end web developers free reign over the document root, so they can create new pages at will, while the back-end guys get free reign over the application .scm file and the units it loads. And the two groups share a nice Wiki that documents the data sources and Scheme procedures the back-end guys provide for the front-end guys. Whereas with Ruby on Rails, to create a page at all, you need somebody to write an action method in the controller class. Which either means a front-end guy bothering a back-end guy to do it, or the front-end guys messing around in a file full of back-end code, or not having any specialisation between front-end guys and back-end guys. Which sucks for me, since I'm definitely a back-end guy. Gimme business logic algorithms over Javascript and HTML form handling any day. As a case in point, when I worked at UpMyStreet.com (which was written in PHP at the time), I (as the resident lead back-end guy) was always being bothered when the front-end guy had to make relatively minor changes to what went where on pages, and changing the page structure, and so on. So we documented all the functions that extracted data from the database, and taught the HTML guys enough basic PHP to call a function and output the result with nice formatting, and let them create their own PHP pages (and rearrange them, and split them into two separate pages, and whatever, as they saw fit). They'd just ask us when they needed some new formatting function or some new angle on the data, that would require us to write and document a new function for them. This worked really well. Also, I want to decouple the data sources. I'm designing a basic interface for a data source to be registered in the environment so that business logic functions and page templates can use it without needing to know if it's an SQL database, an XML-RPC web service, or just a function that maps inputs to outputs. As a taster for the feel of the thing, a read-only data source called "foo" would be made available by a mapping in the environment from (get foo) to a function that accepts whatever the primary key of "foo" is, and either raises a "not found" condition or returns three values: a result, a last-modified date for use in intelligent handling of HTTP caches (or nil if there's no global state being referenced), and a cache validity period in seconds. The cache information can be used to cache the data source, or all data sources referenced in a file (plus the filesystem modification timestamp on the file itself) can be aggregrated together to generate HTTP cache control headers. I'm also defining interfaces for create, update, and delete operations. However, what the skeleton of Wings provides is a way for these different handlers to interoprate pleasantly, bringing the benefits of a monolithic application framework, built from loosely-coupled independent components. The first component I intend to write is a way to declare the GET arguments to a page and have them automatically sanity-checked and decoded. For use in page template languages that let you directly embed Scheme, it will work as a macro that reads the page metadata file (located in a file with the filename of the page, plus ".wings") and extracts the arguments declaration, then expands into a let that binds Scheme variables to the processed values of the arguments. An arguments declaration might look like this: (positional ((user-id integer)) named ((comment-id "c" optional integer) (search-terms "s" optional string)) data-sources ((user userdb (user-id)) (comment comments optional (comment-id)))) What this means is: * If there is a path component AFTER the script name - eg, "1" in http://www.example.com/test.ws/1 - then it's passed through string- >integer and bound to "user-id". If it's not present, we generate a 404 Not Found. If it's present but not a valid integer, then the page just returns with a 404 Not Found. * If there is a GET variable called "c" then it's likewise processed as an integer and bound to comment-id. Otherwise, comment-id is bound to nil. * If there is a GET variable called "s" then it's not processed (anything is a valid string), but is bound to search-terms. Or search- terms is bound to nil if there's no such variable. * The "userdb" data source is accessed by looking for a key in the wings environment called (get userdb), which should be a closure, which is applied to the value of user-id. If it throws a not found exception, then we return a 404 response, otherwise we bind the result to 'user'. * The 'comments' data source is accessed by looking for (get comments) in the wings environment, and applying the result to comment-id. If it throws a not found exception, it's caught and comment bound to nil (because of the 'optional'). Otherwise, the result is bound to comment. Also, the last-modified timestamps and expiry times of all the data source lookups are examined, and the most recent last-modifed and shortest expiry time kept. The last-modified time of the file itself is also examined, and extra last-modified times and expiry intervals can be supplied to the macro by the script, to produce a final most recent last-modified and smallest expiry interval. The macro can handle conditional HTTP GETs with "If-Modified-Since" headers, if the IMS sent by the client is less recent than the computed last modification timestamp, by just returning a "use what you've got" HTTP response and never processing the body of the macro. And it can output cache control headers based upon what it's computed. What does this get you? 1) Automatic handling of HTTP caching 2) In your code, you have nicely bound variables like "user" that are already type-checked and converted, including referencing IDs to actual data objects from data sources where required, with proper handling of invalid IDs. 3) You get to use short names in your URL query string: c=123&s=search +terms. 4) The names used to refer to arguments in your code are decoupled from how those arguments are represented. You can make an argument positional (part of the path) or named (part of the query string), and change your mind later, without needing to change your code. Also, we can provide a utility function that, given the site-root- relative path to a file and an alist of argument names and values, returns a URL: (make-url "/test.ws" '((user-id . 123) (comment-id . 456))) This would read /test.ws.wings to find the declaration, and thus return: "/test.ws/123?c=456" Again, if you change the way you encode your page arguments by altering the declaration, then the behaviour of this function will change to match the behaviour of the argument parser, so nowhere else in your code do you need to change anything :-) [needless to say, we'll cache the .wings files rather then rereading them ALL THE TIME!] But it gets better. The page argument handler knows of argument types like "integer", "boolean", "float", "rational", "symbol" (with an optional list of allowed values), and "string" - but also compound types such as "list", "alist", "set", and "hash". These would only be valid for query string named parameters. If you declare a named argument like so: (foo "x" (list integer)) ...the system will look for a query string like "x[0]=123&x[1]=456", and then bind '(123 456) to foo. If you declare: (foo "x" (alist integer)) ...it will look for "x[wibble]=123&x[wobble]=456" and then bind '((wibble . 123) (wobble . 456)) to foo. If you declare: (foo "x" (set integer)) ...it will look for "x=123&x=456" and then bind '(123 456) or '(456 123) to foo. If you declare: (foo "x" (hash symbol integer)) ...it will look for "x[wibble]=123&x[wobble]=456" and then bind a hash table mapping 'wibble to 123 and 'wobble to 456 to foo. And, of course, the make-url function will accept list, alist, or hash table values for foo, and generate appropriate query string contents. Oh, and for file type handlers that don't work by running Scheme code in some guise, I also plan to provide a function (which, I hope, the page argument parser macro will use as a helper function) that returns the argument bindings for this page as an alist, which can then be incorporated into an interpreted environment of some kind for access. I propose to create an egg called 'wings' to contain general utility functions for access .wings metadata files (there'll be other uses for the files in future, so I plan to create a generic framework), the definition of the wings-environment parameter and utility procedures for accessing same, utility and support functions for the data source API using the wings environment, then the aforementioned page argument handling tools. Followed closely by an egg called 'wings-ds-sql' that provides a basic SQL data source generator. Data sources can be generated using it by providing SQL query templates for SELECT, INSERT, UPDATE, and DELETE queries, or for the simple case where the data source is a single table, just a declaration of the columns and how they are to be handled, for all the queries to be generated automatically. After which, I plan an egg called 'war-rocket-ajax', in which I will present a Spiffy file type handler that implements a full version of a widget-based page templating system I prototyped in PHP some time ago: http://www.snell-pym.org.uk/archives/2007/06/17/another-thing-i-hate- about-web-application-frameworks/ http://www.snell-pym.org.uk/archives/2006/12/17/the-implementation-of- web-applications/ The neat thing about War Rocket Ajax, and the thing that gives it its name, is that it's designed to provide snazzy Ajaxy functionality while providing seamless graceful degradation for clients that don't or won't support it, with minimal extra programmer effort. Hopefully, around then, other people will start contributing better stuff, too ;-) My hope is that the framework is loosely coupled enough to allow pleasant interoperation while not constraining. The make-url procedure defined above will gladly generate URLs that start continuation-based sessions, that just spit static content out, that go into complex widget-based editing forms, or that just fetch some data from a database and slip it into an SXML template and render it, or whatever. And the data source API should allow pages to use data sources ranging from pure functions to global variables or any kind of external state from the filesystem to database clusters. It allows for purely create-only data sources, such as an SMTP client, read- only data sources such as the current date and time or a temperature sensor, and so on. The basic functional interface to data sources does not constrain them to deal with 'record'-like objects as SQL database do, and so on. So, to conclude, for now, I request that a nice new egg directory be created in SVN, called 'wings', with write access from my SVN account! ABS -- Alaric Snell-Pym Work: http://www.snell-systems.co.uk/ Play: http://www.snell-pym.org.uk/alaric/ Blog: http://www.snell-pym.org.uk/?author=4 _______________________________________________ Chicken-users mailing list [email protected] http://lists.nongnu.org/mailman/listinfo/chicken-users
