Re: What The Enterprise wants from Django
On 19.01.2010 23:26, Jacob Kaplan-Moss wrote: > Finally, we ruminated over the difficulties in building rich internet > applications. Sure, writing HTML/CSS/JS/Python/SQL by hand works fine, > but we doesn't really have a good answer for the people who want > something IDE or GUI-ish. Meanwhile, Adobe and Microsoft are putting > all sorts of marketing dollars into Flex/Silverlight, and although > HTML5 can do some amazing things, the lack of tooling is a big danger. > (I've written at more length about this in the past: > http://jacobian.org/writing/snakes-on-the-web/#s-rich-web-applications). It's a bit early to speak about Lino, but Jacob's post incited me to write a kind of preview for Django developers: http://code.google.com/p/lino/wiki/LinoAndDjango Encouraging words are welcome now, but wait a few months before you seriously investigate this project. Luc -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-develop...@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: What The Enterprise wants from Django
On 21 Jan., 17:45, Andy McKaywrote: > On 2010-01-21, at 9:15 AM, chris.moff...@gmail.com wrote: > > > I agree that managing settings gets to be a bit difficult in many > > environments - even non Fortune 1000 environments. Its also an issue for us that write and use portable apps. Luckily a way for apps to provide defaults could easily be implemented in a admin.autodisover sort of way. Somebody already implemented something similar to what i wanted to write today: http://code.djangoproject.com/ticket/1282 For the rest of the "probelms", as someone who is constantly integrating heterogenous systems with django, I think the solution might be "outside" of django. Some problems are sometimes dealt easier in python space. Just as a deliverance similar/friendly solution might be a better solution than making a django admin skin "engine", a more generic configuration app might be better for advanced uses. -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-develop...@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: What The Enterprise wants from Django
On Jan 20, 3:15 pm, sagowrote: > I've had one very long and complex issue with a major client over > legacy databases with Composite Primary Keys (and other composite keys > more generally), an issue which has also come up in other contexts. > One of my smaller clients switched to a strange bastardization of > Django and SQLAlchemy because this was an issue. I understand the > history of #373 and why it is hard - just for disclosure. Composite primary keys is a major issue over here and we also ended up with a bastard of SQLAlchemy and Django. -- Paulo -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-develop...@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: What The Enterprise wants from Django
On 2010-01-21, at 9:15 AM, chris.moff...@gmail.com wrote: > I agree that managing settings gets to be a bit difficult in many > environments - even non Fortune 1000 environments. One of the things > we've played with in Satchmo is a database backed solution that allows > you to define reasonable defaults - even override-able in the settings > - but allow people to edit through the admin interface. I've seen that route before and I have to say that I would be strongly against that. The only saving grace could be multi-db which could allow you to have a specific settings database that's separate from the core. Having data and settings intermingled in the same area just causes too many problems. Further, it detracts from the ability to run the project without the database around. In the scenario of a larger environment with more delineation of roles, it may not be possible. -- Andy McKay, @clearwind http://clearwind.ca/djangoski -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-develop...@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: What The Enterprise wants from Django
Hi! My previous experience was from Java J2EE world and as Jacob said we adopted Django because it really shines solving some complexity problems associated to J2EE world as MVC patter, templates, etc., but I agree that a common way to configure external applications would be nice. I'll also have been searching a good RIA to connect with Django. One of the frameworks I like best is Qooxdoo http://qooxdoo.org/, perhaps the best javascript opensource RIA frameworks I have seen. In order to connect with Qooxdoo just Json-RCP or xml-rcp is needed. Perhaps we'll need an special support for this in Django to mark a view as rcp as some external projects do as RCP4Django or other. I'd like to see Qooxdoo as the default Django RIA (not the default javascript framework) as Qooxdoo people uses also Python and the framework fits quite well in what I think is an enterprise RIA framework. I know that this could be against the Django policy of not to bless a javascript framework, but in my opinion if we want to shine at RIA you have to choose. We have created also some HTTP-XML based web services in Django in a fraction of time we needed to create them in Java. For that we are using Django without ORM, but we have found a great help to allow documenting the web service and create the test pages. Anyway, I'm glad to see Fortune companies are interested in Django :) -- Antoni Aloy López Blog: http://trespams.com Site: http://apsl.net -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-develop...@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: What The Enterprise wants from Django
I agree that managing settings gets to be a bit difficult in many environments - even non Fortune 1000 environments. One of the things we've played with in Satchmo is a database backed solution that allows you to define reasonable defaults - even override-able in the settings - but allow people to edit through the admin interface. Bruce has been working to split it out from Satchmo here - http://bitbucket.org/bkroeze/django-livesettings/overview/ I'm not saying this is the end all be all solution but it's something we've been using for a bit so it should be useful in informing the direction the community wants to take. I also have some ideas for how it should be improved or tweaked but in my opinion it's a nice start and it would be good to see what people think about using it in other apps. It's also consistent with the Django philosophy of building it outside of core and seeing if it reaches critical mass for the community. -Chris -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-develop...@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: What The Enterprise wants from Django
On Thu, Jan 21, 2010 at 10:24 AM, sagowrote: > Case studies are helpful if folks can share them. Just a quick note that the DSF is working on gathering case studies; getting folks who have permission to share is indeed a tricky issue. Not much is set up yet, but I, at least, would be *thrilled* to help companies using Django put together case studies that could be shared publicly. Until we've got something more formal in place, consider it a standing offer -- drop me a line if you'd like to share a success story. Jacob -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-develop...@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: What The Enterprise wants from Django
On Thu, Jan 21, 2010 at 8:58 AM, Luke Plantwrote: > I would have guessed that a big issue with Django from an enterprise > perspective is its use of 'singletons'. [snip] > How much is this an issue in practice? Other than databases, it hasn't been much of an issue with most of the BigCos I've been involved with. That's because... > One solution to this problem would be to have multiple Django > instances hosting different parts of your URL space, allowing them to > have multiple settings files. Bingo. All of the larger Django installations I've worked with haven't been just one project -- they've been dozens (hundreds in one case) of little bits, often cobbled together by different teams and sometimes different companies (in-house vs. contractors vs. off-the-shelf products). Django's URL/view decoupling looks *really* nice in these cases -- Django apps, properly written, don't care *where* in the URL space they live -- so this means that Django fits in well in these heterogenous environments. And I've yet to encounter a larger company that's anything but wildly heterogeneous. Peter's question below about enterprisy complexity and the dangers of becoming J2EE are good ones; I'm glad they're part of the discussion. It's difficult to know where to draw the line between simplicity and flexibility. Right now we have exactly 90 documented settings, some of which point at classes or interfaces that themselves have more complexity. To me this already sounds way too high, but it's nothing compared to the configuration options available in other environments -- PHP has about 600 php.ini settings! -- so there's obviously a matter of taste here. It's also important to keep in mind that part of the reason Django's had (some) success in corporate environments is *because* of the simplicity, not despite it. I've heard, over and over again, that teams that have switched to Django have done so at least in part because of the simplicity and ease-of-use. So we should be careful about setting up a false dichotomy where us hippie open source freaks only value simplicity and build toys and those square suit types who love FactoryManagerFactories because CIO magazine tells them to. The real world, as always, is a lot more complex. Jacob -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-develop...@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: What The Enterprise wants from Django
> I would have guessed that a big issue with Django from an enterprise > perspective is its use of 'singletons'. > > How much is this an issue in practice? Yes, that's been my experience. For complex deployments with tens or hundreds of apps, it is often the case that it is absolutely essential to have settings on a per-app basis. Unfortunately those settings very often clash. > It strikes me that this is just one example of a whole class of "best > practice" stuff that we should document, but don't at present - like > project layout, automated deployment, continuous integration, > configuration management, etc. It would help smaller companies I think, folks who need to deploy a couple of SOA nodes or a site with ten or twenty apps across a handful of servers. When it comes to huge deployments, it is so diverse. Best practice for Google isn't best practice for wisconsinhairstylists.com Case studies are helpful if folks can share them. But its a political minefield (or unhelpful pontification, depending on the care you take) to get into the business of canonizing deployment structures! -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-develop...@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: What The Enterprise wants from Django
On Jan 21, 2010, at 9:58 AM, Luke Plant wrote: > On Tuesday 19 January 2010 21:26:17 Jacob Kaplan-Moss wrote: > >> So there we are. This is very much a brain dump, and I don't really >> expect any concrete action to result from it. However, I found some >> really interesting stuff there, and I thought I'd share. > > Thanks for your clarification elsewhere in this thread. > > One question I have (for Jacob and anyone else with experience): > > I would have guessed that a big issue with Django from an enterprise > perspective is its use of 'singletons'. Though we don't use that > terminology, we have dozens of them in the code - every other setting > we have implies the existence of a singleton, since the setting can > have only one value. For example, EMAIL_BACKEND means that we have > one email component singleton which is used everywhere. If you wanted > one set of views to use a different email backend, you are out of > luck. > > The 'admin' app was the first major component to allow multiple > instances, but there are various other apps which might need this > change in theory. > > How much is this an issue in practice? Speaking as a web developer using Django in a *very* enterprisey corporation, I will say this singleton issue has been a problem for me. As a self-inflicted example, I wrote an LDAP backend that looks up the necessary settings from settings.py. I need to rewrite it to store the settings in the db, because I need to be able to query multiple LDAP sources. I run into similar issues with lots of different apps. So yes, singletons are an issue for me. The difference between ops settings vs. app settings has also been a problem. The lack of established, vetted best practices (for deployment, layout, everything Russ mentioned) also has bitten me regularly. In part because I wind up spending a lot of time doing the research to figure out what I need to do, in part because the state of the art moves forward at a rapid pace. But, to put it in perspective, I'm the only person who even *can* deploy a Django app at my place of work. Which is not great, but I'm also a Linux sys-admin, so it is at least in my job description. But none of my fellow SAs have any clue what to do to get stuff working on one of our corporate RHEL 5 boxen, and no inclination to learn. Anyway, I'm glad to see this discussion taking place. Django indeed has a sweet spot, and it isn't really for enterprise installations -- though it is *very* good in that sweet spot. There are places where the early design decisions to favor convention over configuration impose a lack of flexibility in favor of simplicity. A definite trade-off. It makes it harder to use in places where flexibility and integration are required. Here's a big question, though: given that satisfying the enterprise crowd, of which I am an occasional part, will require additional "stuff" that may bring along with it additional complexity, making it potentially harder to get started for newbies, is that a direction that the project should head? Is it worth satisfying the enterprise crowd to increase adoption, at the cost of turning into the kind of configuration hell that J2EE servers exemplify? Or, more positively, is it possible to add the required enterprisey bits while still maintaining a certain level of simplicity? Note that Django is already headed towards greater complexity (multi-db, email backends), due to requirements from the current community. My two cents, appropriately adjusted for the current economic situation. ---Peter Herndon -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-develop...@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: What The Enterprise wants from Django
On Tuesday 19 January 2010 21:26:17 Jacob Kaplan-Moss wrote: > So there we are. This is very much a brain dump, and I don't really > expect any concrete action to result from it. However, I found some > really interesting stuff there, and I thought I'd share. Thanks for your clarification elsewhere in this thread. One question I have (for Jacob and anyone else with experience): I would have guessed that a big issue with Django from an enterprise perspective is its use of 'singletons'. Though we don't use that terminology, we have dozens of them in the code - every other setting we have implies the existence of a singleton, since the setting can have only one value. For example, EMAIL_BACKEND means that we have one email component singleton which is used everywhere. If you wanted one set of views to use a different email backend, you are out of luck. The 'admin' app was the first major component to allow multiple instances, but there are various other apps which might need this change in theory. How much is this an issue in practice? I imagine that for most sub-systems, most people actually want singletons, but for the cases you don't, it is a big pain. Personally I think that, while there has been an unhealthy proliferation of settings which we need to trim, Django has hit the sweet spot between easy set up and configuration on the one hand, and flexibility on the other. The alternative - turning everything into components with dependency injection everywhere - could be horrific for our sweet spot. One solution to this problem would be to have multiple Django instances hosting different parts of your URL space, allowing them to have multiple settings files. This has obvious limitations, but from the perspective of a large project, perhaps it's not too bad - I imagine that large sites are already used to carving up the URL space to be served by completely different technologies in some cases, at least for hosts running LAMP or similar. Regards, Luke -- "Procrastination: Hard work often pays off after time, but laziness always pays off now." (despair.com) Luke Plant || http://lukeplant.me.uk/ -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-develop...@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: What The Enterprise wants from Django
2010/1/21 Russell Keith-Magee: > > It strikes me that this is just one example of a whole class of "best > practice" stuff that we should document, but don't at present - like > project layout, automated deployment, continuous integration, > configuration management, etc. > > We've historically avoided having documentation on how to use Django > with external tools (except when we have to, as in the case of > mod_wsgi). A big part of the solution to the problem you've described > is to establish a vetted best practice, and documenting that practice > on our website as a community resource. I'd certainly be interested in contributing to this effort. I have lots of this information scattered about on my blog, internal company notes and presentations from DJUGL and the likes. I've avoided trying to get it on the wiki for the reasons you describe. What is the best way of going about documenting this so it can be vetted by the community and enshrined as somewhat official best practice - without the work in progress being used before it's been stamped in someway? > > The rest of the solution falls to the toolchain outside of Django > itself. It still needs to be solved, but it isn't a Django problem > specifically. One option here might be to take the Apache model, and > use the DSF to incubate tools that have benefits to Django. However, > that's part of a much bigger discussion. > > Yours, > Russ Magee %-) > > -- > You received this message because you are subscribed to the Google Groups > "Django developers" group. > To post to this group, send email to django-develop...@googlegroups.com. > To unsubscribe from this group, send email to > django-developers+unsubscr...@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/django-developers?hl=en. > > > > -- Gareth Rushgrove Web Geek morethanseven.net garethrushgrove.com -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-develop...@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: What The Enterprise wants from Django
2010/1/20 Jacob Kaplan-Moss: > On Wed, Jan 20, 2010 at 12:31 PM, Luke Plant wrote: >> I don't understand how avoiding the settings.py mechanism will produce >> *more* flexibility. > > The problem -- at least as I see it -- is that of a intertwingulment > of "application" settings with "ops" settings. > > As organizations get larger, roles stratify more. Thus you end up with > a pretty big gulf between the ops team and the application developers. > In these environments, application developers probably shouldn't be > messing with things like the `DATABASE_*` settings -- in some > organizations its even a breach of policy for the developers to *know* > the database password! In the past I've used code like the following in settings files: # we're going to allow overriding of settings via a yml file. # This makes live nicer for anyone managing the box # and means settings can be overloaded without redeploying the site SETTINGS_OVERIDE = "/etc/$(project).yml" try: file = open(SETTINGS_OVERIDE) for key, value in yaml.load(file).items(): globals()[key]=value except: # we don't always have the file around or need the setting # defined so best to be quite if things go wrong pass You can also make this more limited, by allowing only the overriding certain settings. You can then use tools like puppet on the ops side to push config files in to /etc (or wherever suites you system). Example here http://morethanseven.net/2009/02/15/let-you-sys-admin-override-your-django-settings/ This pattern is also used in my project templates paster stuff: http://github.com/garethr/django-project-templates and http://pypi.python.org/pypi/django-project-templates/ > > OTOH, ops types don't know what the heck to do with `INSTALLED_APPS`... > > And then there's middleware: some middleware (caching, etags) might > need to be controlled by ops; some by apps. > > Obviously you can *do* this with Django, and I'd argue it's not > particularly hard. Still, it's something that bigger orgs have to > figure out themselves, and that can be a barrier to adoption. > > The enterprisey world solves this with standards like WAR and servers > like Glassfish: if I'm an application developer, all I need to do to > deploy is somehow produce a WAR. And if I'm a ops person, I just take > that WAR and slap it into Glassfish. It's rarely that easy in > practice, of course, but the execs making the tech decisions don't > know that. > > Jacob > > -- > You received this message because you are subscribed to the Google Groups > "Django developers" group. > To post to this group, send email to django-develop...@googlegroups.com. > To unsubscribe from this group, send email to > django-developers+unsubscr...@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/django-developers?hl=en. > > > > -- Gareth Rushgrove Web Geek morethanseven.net garethrushgrove.com -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-develop...@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: What The Enterprise wants from Django
On Thu, Jan 21, 2010 at 2:54 AM, Jacob Kaplan-Mosswrote: > On Wed, Jan 20, 2010 at 12:31 PM, Luke Plant wrote: >> I don't understand how avoiding the settings.py mechanism will produce >> *more* flexibility. > > The problem -- at least as I see it -- is that of a intertwingulment > of "application" settings with "ops" settings. I'm in full agreement that this is a problem. However, isn't the solution just a matter of adequately documenting the options that are already available? We could certainly improve the documentation for complex deployments - e.g., pointing out that: * settings.py can import base_settings.py * DJANGO_SETTINGS_MODULE lets you call your settings.py whatever you want * documenting which settings are operations, and which are application I'm not convinced that we have anything particularly technical to address here. > And then there's middleware: some middleware (caching, etags) might > need to be controlled by ops; some by apps. > > Obviously you can *do* this with Django, and I'd argue it's not > particularly hard. Still, it's something that bigger orgs have to > figure out themselves, and that can be a barrier to adoption. It strikes me that this is just one example of a whole class of "best practice" stuff that we should document, but don't at present - like project layout, automated deployment, continuous integration, configuration management, etc. We've historically avoided having documentation on how to use Django with external tools (except when we have to, as in the case of mod_wsgi). A big part of the solution to the problem you've described is to establish a vetted best practice, and documenting that practice on our website as a community resource. The rest of the solution falls to the toolchain outside of Django itself. It still needs to be solved, but it isn't a Django problem specifically. One option here might be to take the Apache model, and use the DSF to incubate tools that have benefits to Django. However, that's part of a much bigger discussion. Yours, Russ Magee %-) -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-develop...@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: What The Enterprise wants from Django
Using djangorecipe and zc.buildout offer your exactly that kind of mechanism. You write the name of settings of choice in your buildout: [django] recipe = djangorecipe version = 1.1 settings = development eggs = ${eggs:eggs} wsgi = true project = project And after that you can have by example another production's configuration file like "prod.cfg" overloading your django part and using another settings name. In your project dir, you'll have a file named development.py and another one named prod.py. In these files you can override any configuration you want. For deployement, zc.buildout is working pretty fine with django: http://pypi.python.org/pypi/djangorecipe On Wed, Jan 20, 2010 at 1:46 AM, mrtswrote: > On Jan 20, 5:59 am, David Cramer wrote: >> The first three have been huges ones with us. We're just now running >> into the settings issue, but would love to see what people can come up >> with for solutions (we dont have a good one). > > The override_settings() idea that Eric Florenzano describes at > http://djangodose.com/articles/2009/09/handling-development-staging-and-production-enviro/ > with some improvements and fixes as described in the following > http://djangodose.com/articles/2009/09/handling-development-staging-and-production-enviro/#comment-16302200 > have made my life considerably easier. > > Hope the following helps others as well. > > # 1. > # settings is a directory, containing host-specific overrides > # and the generic override logic > > $ ls settings/ > __init__.py override.py host1.py host2.py > > # 2. > # __init__.py has the same contents that settings.py > # usually has, except it automatically imports host-specific > # settings in the end > > $ tail -2 settings/__init__.py > override.override_settings('projectname', platform.node().split('.') > [0], > sys.modules[__name__]) > > # 3. > # The generic override logic is as follows > > $ cat settings/override.py > from django.utils.importlib import import_module > import warnings > > def override_settings(package, name, settings_module): > rel_path = 'settings.' + name > try: > override_module = import_module(rel_path, package=package) > except ImportError: > warnings.warn("Failed to import settings from '%s.%s'" % > (package, rel_path)) > else: > for key in dir(override_module): > if key.isupper() and not key.startswith('__'): > setattr(settings_module, key, getattr(override_module, > key)) > > -- > You received this message because you are subscribed to the Google Groups > "Django developers" group. > To post to this group, send email to django-develop...@googlegroups.com. > To unsubscribe from this group, send email to > django-developers+unsubscr...@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/django-developers?hl=en. > > > > -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-develop...@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: What The Enterprise wants from Django
> I don't understand how avoiding the settings.py mechanism will produce > *more* flexibility. Bad choice of words on my part, sorry. Of course code is always more flexible than static data. The two key bits of required 'flexibility' I didn't have out of the box (and were difficult to hack and get working) are: 1. App level settings that are mutually exclusive (one app needs settings different from another - this happens a lot). 2. Setting definitions that aren't tied in timing and scope terms to Python's import process (I've seen this interact badly with start-up code, for example). And it would be cool to have a level of indirection so that settings could be guaranteed to be dynamic (some are, some aren't, those that are often need monkey-patch style hacks to change). And tools for debugging where settings for a failed request are defined. And a security layer around some settings. And fields and fields of other ponies. Seriously though, I don't think the settings.py approach is broken. It makes sense, makes getting from nothing to runnable code a breeze, is logical for small deployments, and has bags of flexibility for most use-cases. I'm happy adjusting it in my projects. I've never deployed or seen deployed a system that used settings.py in anything like its default version, but the replacements aren't standard enough to be an alternative default; every client has different technical, management and internal politics constraints. My comments were meant in the spirit of Jacob's. Just FYIs for a particular niche of users. I should also say that most of the developers I work with do love working with Django. So their criticisms have to be taken against a background of strong positivity. -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-develop...@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: What The Enterprise wants from Django
On 2010-01-20, at 10:54 AM, Jacob Kaplan-Moss wrote: On Wed, Jan 20, 2010 at 12:31 PM, Luke Plantwrote: I don't understand how avoiding the settings.py mechanism will produce *more* flexibility. The problem -- at least as I see it -- is that of a intertwingulment of "application" settings with "ops" settings. +1 -- Andy McKay, @clearwind http://clearwind.ca/djangoski -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-develop...@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: What The Enterprise wants from Django
> A lack of any mechanism for developers to understand what third party > apps are reliable, what to avoid, known problems, and so on. This has > been a particular complaint since the Django ethos is (rightly) that > the core is the core, and other functionality should be packaged into > third party apps. App authors do a great job of being really generous > with their work, but it is very hard for stuff outside Django's core > to get any adoption. The processes involved in adopting a technology > can be complex and involve lots of stakeholders, legal and technical > due diligence. It is hard enough ticking those boxes to get Django > used, it simply isn't worth doing the same for all the other bits. > There's a lot of wheel reinventing going on, in my experience. A Django > + project might be useful to give the rock-star third-party apps a > fighting chance at being used. Pinax is somewhat close, but has a very > different purpose and small, agile, company ethos. While not perfect I find how the http://plugins.jquery.com/ site works. It gives me more confidence in picking a plugin to use for a particular project to see how other users have rated an app. The demos and examples of the plugins is also very nice. Something similar for Django apps would be great. -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-develop...@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: What The Enterprise wants from Django
On Wed, Jan 20, 2010 at 12:31 PM, Luke Plantwrote: > I don't understand how avoiding the settings.py mechanism will produce > *more* flexibility. The problem -- at least as I see it -- is that of a intertwingulment of "application" settings with "ops" settings. As organizations get larger, roles stratify more. Thus you end up with a pretty big gulf between the ops team and the application developers. In these environments, application developers probably shouldn't be messing with things like the `DATABASE_*` settings -- in some organizations its even a breach of policy for the developers to *know* the database password! OTOH, ops types don't know what the heck to do with `INSTALLED_APPS`... And then there's middleware: some middleware (caching, etags) might need to be controlled by ops; some by apps. Obviously you can *do* this with Django, and I'd argue it's not particularly hard. Still, it's something that bigger orgs have to figure out themselves, and that can be a barrier to adoption. The enterprisey world solves this with standards like WAR and servers like Glassfish: if I'm an application developer, all I need to do to deploy is somehow produce a WAR. And if I'm a ops person, I just take that WAR and slap it into Glassfish. It's rarely that easy in practice, of course, but the execs making the tech decisions don't know that. Jacob -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-develop...@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: What The Enterprise wants from Django
On Wednesday 20 January 2010 17:15:39 sago wrote: > Startup and Settings are the big killers though, head and shoulders > above the previous issues. I've nothing much to add to your > comments other than to say that some of the Django deployments I > know of are highly heterogeneous, with various servers running > different SOA elements, some Django some not. The configuration > issues are formidable and involve all kinds of dependencies beyond > Django (or even Python). I struggle to see a way that Django could > solve those issues, but allowing more flexibility and avoiding > tying configuration to an executable module (i.e. settings.py) > would be useful. I don't understand how avoiding the settings.py mechanism will produce *more* flexibility. With an executable module storing settings, you can implement whatever mechanism you want for settings: ## settings.py ## import sys from acme import network_config network_config.load_django_settings_into_module(sys.modules[__name__]) ## end. Granted, code that does the above might need to be written, but the point of settings.py is that you *can* write it, without forcing anyone to use that system. Luke -- "Procrastination: Hard work often pays off after time, but laziness always pays off now." (despair.com) Luke Plant || http://lukeplant.me.uk/ -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-develop...@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: What The Enterprise wants from Django
I also work with several fortune 500 clients (though not specifically as a Django consultant, Django is a preferred tool) and the issues your client mentions, Jacob, are very similar to those I deal with. Some other notes from the concerns of multinational clients: I've had one very long and complex issue with a major client over legacy databases with Composite Primary Keys (and other composite keys more generally), an issue which has also come up in other contexts. One of my smaller clients switched to a strange bastardization of Django and SQLAlchemy because this was an issue. I understand the history of #373 and why it is hard - just for disclosure. A lack of any mechanism for developers to understand what third party apps are reliable, what to avoid, known problems, and so on. This has been a particular complaint since the Django ethos is (rightly) that the core is the core, and other functionality should be packaged into third party apps. App authors do a great job of being really generous with their work, but it is very hard for stuff outside Django's core to get any adoption. The processes involved in adopting a technology can be complex and involve lots of stakeholders, legal and technical due diligence. It is hard enough ticking those boxes to get Django used, it simply isn't worth doing the same for all the other bits. There's a lot of wheel reinventing going on, in my experience. A Django + project might be useful to give the rock-star third-party apps a fighting chance at being used. Pinax is somewhat close, but has a very different purpose and small, agile, company ethos. Startup and Settings are the big killers though, head and shoulders above the previous issues. I've nothing much to add to your comments other than to say that some of the Django deployments I know of are highly heterogeneous, with various servers running different SOA elements, some Django some not. The configuration issues are formidable and involve all kinds of dependencies beyond Django (or even Python). I struggle to see a way that Django could solve those issues, but allowing more flexibility and avoiding tying configuration to an executable module (i.e. settings.py) would be useful. Like the previous posters, I've also had to roll various custom solutions to this problem. Is suspect at least some of the solution will always be custom, though it might be worth making such solutions easier to create. It would also be useful for everyone to scream more loudly that monkey- patching ANYTHING is a very bad idea. Its getting better now (as some of the major things it was solving are solved properly), but it was a woefully overused technique for a while that has meant a lot of deployment pain! That's less about Django and more about evangelizing bad development practice under the guise of Python-Fu. Ian. -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-develop...@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: What The Enterprise wants from Django
On Jan 19, 4:26 pm, Jacob Kaplan-Mosswrote: > The next big one was the "startup signal" issue -- they've got lots of > code that needs to run at startup time, and there's no great mechanism > to do that currently. The core devs have talked about this one a *lot* > over the years, and it's obviously a hard one -- for one, there's no > clear definition of what "startup" means -- but a common theme seems > to be that bigger, more complex applications need some way to cleanly > run one-time, expensive startup tasks. I've got a mechanism which I described in these posts: http://groups.google.com/group/django-developers/msg/2b94fa2017b07e67 and http://groups.google.com/group/django-developers/msg/7f14fc0d9ee92b43 The thread refers to logging but the approach works for any start up code; I've provided three points where code can be hooked, which I've called "bootstrap" (e.g. to initialize logging), "pre-model" (before app loading, say to install class_prepared listeners) and "post-model" (after app loading, to do any work which needs apps to be initialized and models to be available). I would welcome some feedback ... Launchpad branch is here: https://code.launchpad.net/~vinay-sajip/django/logging An example settings file which uses the functionality in the branch is in the second of the two posts mentioned above. Regards, Vinay Sajip -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-develop...@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: What The Enterprise wants from Django
Startup signal - when using mod_wsgi the wsgi init app can be a good place to call the startup work, starting threads, setting logging level etc. It worked for me, but it's probably much simpler than a complex corporate deployment. On Jan 19, 11:26 pm, Jacob Kaplan-Mosswrote: > Hi folks -- > > I had a conversation this morning with one of my clients. In the > interests of being a good corporate citizen I'll refrain from > mentioning who other than (a) they're big (Fortune 1000), (b) you've > heard of them, and (c) they're using Django. > > Before our chat, they'd invited any engineers internally to bitch > about Django, and they shared some of the pain points they'd come > across. I took some notes; the really good news is that it's a short > list, and we already know about most of it. > > Before I share the list, I should mention that I'm not arguing we have > to immediately devote resources to these problems, nor even that we > have to solve them. I'm sharing this just as food for thought, and as > a perspective from a class of developers who mostly don't participate > in our process at all. > > So: > > The main issue they have is multiple database support. Music to my > ears (and especially Alex's :) since it looks like 1.2 will solve most > (all?) of their issues. My main takeaway from this point was that the > best way to please companies like this one will be to ship 1.2 on > time. Which will happen. So we rock. > > The next big one was the "startup signal" issue -- they've got lots of > code that needs to run at startup time, and there's no great mechanism > to do that currently. The core devs have talked about this one a *lot* > over the years, and it's obviously a hard one -- for one, there's no > clear definition of what "startup" means -- but a common theme seems > to be that bigger, more complex applications need some way to cleanly > run one-time, expensive startup tasks. > > Next, we discussed the difficulty of maintaining large installations > with multiple sites, a plethora of apps, and unspeakable possible > combinations thereof. Django's settings mechanism is simple and easy > to use, but once you have hundreds of sites it starts getting really > tricky to keep things in sync across hundreds of settings files. We > talked at some length about different possibilities for dynamic > settings infrastructure; this would especially come in handy for folks > wanting to build application servers/services like App Engine or the > nascent Toppcloud. > > If you've not yet checked out Toppcloud, do so now. I'll > wait. > > Finally, we ruminated over the difficulties in building rich internet > applications. Sure, writing HTML/CSS/JS/Python/SQL by hand works fine, > but we doesn't really have a good answer for the people who want > something IDE or GUI-ish. Meanwhile, Adobe and Microsoft are putting > all sorts of marketing dollars into Flex/Silverlight, and although > HTML5 can do some amazing things, the lack of tooling is a big danger. > (I've written at more length about this in the > past:http://jacobian.org/writing/snakes-on-the-web/#s-rich-web-applications). > > Of course, there may not be much us backend folks can do about this > problem -- I'm certainly not enough of a GUI application guy to be > able to even start to think about this problem -- but the lack of an > end-to-end solution will nonetheless put some off from writing web > applications with open source tools. > > So there we are. This is very much a brain dump, and I don't really > expect any concrete action to result from it. However, I found some > really interesting stuff there, and I thought I'd share. > > Jacob -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-develop...@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: What The Enterprise wants from Django
On Jan 20, 5:59 am, David Cramerwrote: > The first three have been huges ones with us. We're just now running > into the settings issue, but would love to see what people can come up > with for solutions (we dont have a good one). The override_settings() idea that Eric Florenzano describes at http://djangodose.com/articles/2009/09/handling-development-staging-and-production-enviro/ with some improvements and fixes as described in the following http://djangodose.com/articles/2009/09/handling-development-staging-and-production-enviro/#comment-16302200 have made my life considerably easier. Hope the following helps others as well. # 1. # settings is a directory, containing host-specific overrides # and the generic override logic $ ls settings/ __init__.py override.py host1.py host2.py # 2. # __init__.py has the same contents that settings.py # usually has, except it automatically imports host-specific # settings in the end $ tail -2 settings/__init__.py override.override_settings('projectname', platform.node().split('.') [0], sys.modules[__name__]) # 3. # The generic override logic is as follows $ cat settings/override.py from django.utils.importlib import import_module import warnings def override_settings(package, name, settings_module): rel_path = 'settings.' + name try: override_module = import_module(rel_path, package=package) except ImportError: warnings.warn("Failed to import settings from '%s.%s'" % (package, rel_path)) else: for key in dir(override_module): if key.isupper() and not key.startswith('__'): setattr(settings_module, key, getattr(override_module, key)) -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-develop...@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: What The Enterprise wants from Django
The first three have been huges ones with us. We're just now running into the settings issue, but would love to see what people can come up with for solutions (we dont have a good one). Glad to see multi db is finally shipping, and excited to see what can be done for startup procs. On Jan 19, 3:26 pm, Jacob Kaplan-Mosswrote: > Hi folks -- > > I had a conversation this morning with one of my clients. In the > interests of being a good corporate citizen I'll refrain from > mentioning who other than (a) they're big (Fortune 1000), (b) you've > heard of them, and (c) they're using Django. > > Before our chat, they'd invited any engineers internally to bitch > about Django, and they shared some of the pain points they'd come > across. I took some notes; the really good news is that it's a short > list, and we already know about most of it. > > Before I share the list, I should mention that I'm not arguing we have > to immediately devote resources to these problems, nor even that we > have to solve them. I'm sharing this just as food for thought, and as > a perspective from a class of developers who mostly don't participate > in our process at all. > > So: > > The main issue they have is multiple database support. Music to my > ears (and especially Alex's :) since it looks like 1.2 will solve most > (all?) of their issues. My main takeaway from this point was that the > best way to please companies like this one will be to ship 1.2 on > time. Which will happen. So we rock. > > The next big one was the "startup signal" issue -- they've got lots of > code that needs to run at startup time, and there's no great mechanism > to do that currently. The core devs have talked about this one a *lot* > over the years, and it's obviously a hard one -- for one, there's no > clear definition of what "startup" means -- but a common theme seems > to be that bigger, more complex applications need some way to cleanly > run one-time, expensive startup tasks. > > Next, we discussed the difficulty of maintaining large installations > with multiple sites, a plethora of apps, and unspeakable possible > combinations thereof. Django's settings mechanism is simple and easy > to use, but once you have hundreds of sites it starts getting really > tricky to keep things in sync across hundreds of settings files. We > talked at some length about different possibilities for dynamic > settings infrastructure; this would especially come in handy for folks > wanting to build application servers/services like App Engine or the > nascent Toppcloud. > > If you've not yet checked out Toppcloud, do so now. I'll > wait. > > Finally, we ruminated over the difficulties in building rich internet > applications. Sure, writing HTML/CSS/JS/Python/SQL by hand works fine, > but we doesn't really have a good answer for the people who want > something IDE or GUI-ish. Meanwhile, Adobe and Microsoft are putting > all sorts of marketing dollars into Flex/Silverlight, and although > HTML5 can do some amazing things, the lack of tooling is a big danger. > (I've written at more length about this in the > past:http://jacobian.org/writing/snakes-on-the-web/#s-rich-web-applications). > > Of course, there may not be much us backend folks can do about this > problem -- I'm certainly not enough of a GUI application guy to be > able to even start to think about this problem -- but the lack of an > end-to-end solution will nonetheless put some off from writing web > applications with open source tools. > > So there we are. This is very much a brain dump, and I don't really > expect any concrete action to result from it. However, I found some > really interesting stuff there, and I thought I'd share. > > Jacob -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-develop...@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
What The Enterprise wants from Django
Hi folks -- I had a conversation this morning with one of my clients. In the interests of being a good corporate citizen I'll refrain from mentioning who other than (a) they're big (Fortune 1000), (b) you've heard of them, and (c) they're using Django. Before our chat, they'd invited any engineers internally to bitch about Django, and they shared some of the pain points they'd come across. I took some notes; the really good news is that it's a short list, and we already know about most of it. Before I share the list, I should mention that I'm not arguing we have to immediately devote resources to these problems, nor even that we have to solve them. I'm sharing this just as food for thought, and as a perspective from a class of developers who mostly don't participate in our process at all. So: The main issue they have is multiple database support. Music to my ears (and especially Alex's :) since it looks like 1.2 will solve most (all?) of their issues. My main takeaway from this point was that the best way to please companies like this one will be to ship 1.2 on time. Which will happen. So we rock. The next big one was the "startup signal" issue -- they've got lots of code that needs to run at startup time, and there's no great mechanism to do that currently. The core devs have talked about this one a *lot* over the years, and it's obviously a hard one -- for one, there's no clear definition of what "startup" means -- but a common theme seems to be that bigger, more complex applications need some way to cleanly run one-time, expensive startup tasks. Next, we discussed the difficulty of maintaining large installations with multiple sites, a plethora of apps, and unspeakable possible combinations thereof. Django's settings mechanism is simple and easy to use, but once you have hundreds of sites it starts getting really tricky to keep things in sync across hundreds of settings files. We talked at some length about different possibilities for dynamic settings infrastructure; this would especially come in handy for folks wanting to build application servers/services like App Engine or the nascent Toppcloud. If you've not yet checked out Toppcloud, do so now. I'll wait. Finally, we ruminated over the difficulties in building rich internet applications. Sure, writing HTML/CSS/JS/Python/SQL by hand works fine, but we doesn't really have a good answer for the people who want something IDE or GUI-ish. Meanwhile, Adobe and Microsoft are putting all sorts of marketing dollars into Flex/Silverlight, and although HTML5 can do some amazing things, the lack of tooling is a big danger. (I've written at more length about this in the past: http://jacobian.org/writing/snakes-on-the-web/#s-rich-web-applications). Of course, there may not be much us backend folks can do about this problem -- I'm certainly not enough of a GUI application guy to be able to even start to think about this problem -- but the lack of an end-to-end solution will nonetheless put some off from writing web applications with open source tools. So there we are. This is very much a brain dump, and I don't really expect any concrete action to result from it. However, I found some really interesting stuff there, and I thought I'd share. Jacob -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-develop...@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.