Hello, I have few comments:
- That file layout basically forces you to have your development environment as close to the production environment. This is especially visible if you're relying on python c extensions. Since you don't want to have the same environment constraints as appengine it should be more flexible in this regard and offer a way to generate the project dependencies somewhere else than the depeloper's machine. - There's no builtin support for logging configuration. - The update_fetch feels like a hack as it's not extensible to do lifecycle (hooks for shutdown, start, etc). Also, it's shouldn't be a application url because you'd want to run a hook before starting it or after stopping it. I guess you could accomplish that with a wsgi wrapper but there should be a clear separation between the app and hooks that manage the app. - I'm not entirely clear on why you avoid a build process (war-like) prior to deployment. It works fine for appengine - but you don't have it's constraints. -- Ionel On Fri, Apr 1, 2011 at 23:55, Ian Bicking <i...@colorstudy.com> wrote: > Hi all. I wrote a blog post. I would be interested in reactions from this > crowd. > > http://blog.ianbicking.org/2011/03/31/python-webapp-package/ > > Copied to allow responses: > > At PyCon there was an open space about deployment, and the idea of drop-in > applications > (Java-WAR-style<http://en.wikipedia.org/wiki/WAR_file_format_%28Sun%29> > ). > > I generally get pessimistic about 80% solutions, and dropping in a WAR file > feels like an 80% solution to me. I’ve used the Hudson/Jenkins installer > (which I think is *specifically* a project that got WARs on people’s > minds), and in a lot of ways that installer is nice, but it’s also kind of > wonky, it makes configuration unclear, it’s not always clear when it > installs or configures itself through the web, and when you have to do this > at the system level, nor is it clear where it puts files and data, etc. So a > great initial experience doesn’t feel like a great ongoing experience to me > — and *it doesn’t have to be that way*. If those were *necessary*compromises, > sure, but they aren’t. And because we > *don’t have* WAR files, if we’re proposing to make something new, then we > have every opportunity to make things better. > > So the question then is what we’re trying to make. To me: we want > applications that are easy to install, that are self-describing, > self-configuring (or at least guide you through configuration), reliable > with respect to their environment (not dependent on system tweaking), > upgradable, and respectful of persistence (the data that outlives the > application install). A lot of this can be done by the "container" (to use > Java parlance; or "environment") — if you just have the app packaged in a > nice way, the container (server environment, hosting service, etc) can > handle all the system-specific things to make the application actually work. > > At which point I am of course reminded of my Silver > Lining<http://cloudsilverlining.org/>project, which defines something very > much like this. Silver Lining isn’t > *just* an application format, and things aren’t fully extracted along > these lines, but it’s pretty close and it addresses a lot of important > issues in the lifecycle of an application. To be clear: Silver Lining is an > application packaging format, a server configuration library, a cloud server > management tool, a persistence management tool, and a tool to manage the > application with respect to all these services over time. It is a bunch of > things, maybe too many things, so it is not unreasonable to pick out a > smaller subset to focus on. Maybe an easy place to start (and good for > Silver Lining itself) would be to separate at least the application format > (and tools to manage applications in that state, e.g., installing new > libraries) from the tools that make use of such applications (deploy, etc). > > Some opinions I have on this format, exemplified in Silver Lining: > > - It’s not zipped or a single file, unlike WARs. Uploading zip files is > not a great API. Geez. I know there’s this desire to "just drop in a file"; > but there’s no getting around the fact that "dropping a file" becomes a > *deployment protocol* *and* *it’s an incredibly impoverished protocol*. > The format is also not subtly git-based (ala Heroku) because git pushis > not a good deployment protocol. > - But of course there isn’t really any deployment protocol inferred by > a format anyway, so maybe I’m getting ahead of myself ;) I’m saying a tool > that deploys should take as an argument a directory, not a single file. (If > the tool then zips it up and uploads it, fine!) > - Configuration "comes from the outside". That is, an application > requests services, and the *container* tells the application where > those services are. For Silver Lining I’ve used environmental variables. I > think this one point is really important — the container *tells* the > application. As a counter-example, an application that comes with a Puppet > deployment recipe is essentially *telling* the server how to arrange > itself to suit the application. This will never be reliable or simple! > - The application indicates what "services" it wants; for instance, it > may want to have access to a MySQL database. The container then provides > this to the application. In practice this means installing the actual > packages, but also creating a database and setting up permissions > appropriately. The alternative is never having *any* dependencies, > meaning you have to use SQLite databases or ad hoc structures, etc. But in > fact installing databases really isn’t that hard these days. > - *All* persistence has to use a service of some kind. If you want to > be able to write to files, you need to use a file service. This means the > container is fully aware of everything the application is leaving behind. > All the various paths an application should use are given in different > environmental variables (many of which don’t need to be invented anew, > e.g., > $TMPDIR). > - It uses vendor libraries exclusively for Python libraries. That means > the application bundles all the libraries it requires. Nothing ever gets > installed at deploy-time. This is in contrast to using a > requirements.txt list of packages at deployment time. If you want to > use those tools for development that’s fine, just not for deployment. > - There *is* also a way to indicate other libraries you might require; > e.g., you might lxml, or even something that isn’t quite a library, > like git (if you are making a github clone). You can’t do those as > vendor libraries (they include non-portable binaries). Currently in Silver > Lining the application description can contain a list of Ubuntu package > names to install. Of course that would have to be abstracted some. > - You can ask for scripts or a request to be invoked for an application > after an installation or deployment. It’s lame to try to test if > is-this-app-installed on *every* request, which is the frequent > alternative. Also, it gives the application the chance to signal that the > installation failed. > - It has a very simple (possibly/probably too simple) sense of > configuration. You don’t have to use this if you make your app > self-configuring (i.e., build in a web-accessible settings screen), but in > practice it felt like some simple sense of configuration would be helpful. > > Things that could be improved: > > - There are some places where you might be encouraged to use routines > from the silversupport package. There are very few! But maybe an > alternative could be provided for these cases. > - A little convention-over-configuration is probably suitable for the > bundled libraries; silver includes tools to manage things, but it gets a > little twisty. When creating a new project I find myself creating several > .pth files, special customizing modules, etc. Managing vendor libraries > is also not obvious. > - Services <http://cloudsilverlining.org/services.html> are IMHO quite > important and useful, but also need to be carefully specified. > - There’s a bunch of runtime expectations that aren’t part of the > format, but in practice would be part of how the application is written. > For > instance, I make sure each app has its own temporary directory, and that it > is cleared on update. If you keep session files in that location, and you > expect the environment to clean up old sessions — well, either all > environments should do that, or none should. > - The process model is not entirely clear. I tried to simply define one > process model (unthreaded, multiple processes), but I’m not sure that’s > suitable — most notably, multiple processes have a significant memory > impact > compared to threads. An application should at least be able to indicate > what > process models it accepts and prefers. > - Static files are *all* convention over configuration — you put static > files under static/ and then they are available. So static/style.csswould > be at > /style.css. I think this is generally *good*, but putting all static > files under one URL path (e.g., /media/) can be good for other reasons > as well. Maybe there should be conventions for both. > - Cron jobs are important. Though maybe they could just be yet another > kind of service? Many extra features could be new services. > - Logging is also important; Silver Lining attempts to handle that > somewhat, but it could be specified much better. > - Silver Lining also supports PHP, which seemed to cause a bit of > stress. But just ignore that. It’s really easy to ignore. > > There is a description of the configuration file for > apps<http://cloudsilverlining.org/appconfig.html>. > The environmental > variables<http://cloudsilverlining.org/envvariables.html>are also notably > part of the application’s expectations. The file layout is > explained (together with a bunch of Silver Lining-specific concepts) in > Development > Patterns <http://cloudsilverlining.org/devpatterns.html>. Besides all that > there is admittedly some other stuff that is only really specified in code; > but in Silver Lining’s defense, specified in code is better than unspecified > ;) App Engine provides another example of an application format, and would > be worth using as a point of discussion or contrast (I did that myself when > writing Silver Lining). > > Discussing WSGI stuff with Ben Bangert at PyCon he noted that he didn’t > really feel like the WSGI pieces needed that much more work, or at least > that’s not where the interesting work was — the interesting work is in the > tooling. An application format could provide a great basis for building this > tooling. And I honestly think that the tooling has been held back more by > divergent patterns of development than by the difficulty of writing the > tools themselves; and a good, general application format could fix that. > > > _______________________________________________ > Web-SIG mailing list > Web-SIG@python.org > Web SIG: http://www.python.org/sigs/web-sig > Unsubscribe: > http://mail.python.org/mailman/options/web-sig/ionel.mc%40gmail.com > >
_______________________________________________ Web-SIG mailing list Web-SIG@python.org Web SIG: http://www.python.org/sigs/web-sig Unsubscribe: http://mail.python.org/mailman/options/web-sig/archive%40mail-archive.com
