Re: Request for comments on a new Open Source Paas platform for Django
On Tuesday, March 5, 2013 7:56:37 PM UTC, Dave Murphy wrote: > > On Tuesday, March 5, 2013 7:03:36 PM UTC, Bruno Girin wrote: >> >> So I'd much rather have the charm auto-generate part of the config in a >> sensible way and then tell people: if you use Juju, don't provide those >> settings in your config, the charm will do it. >> > > ...and if they do? How is this any different to the code not handling > environment variables correctly? > > People are going to need to adapt their project to the charm no matter > what approach you take. > True. My point here is that if you ask people to adapt their project to work in Juju, it's easier for them and less error prone to say "don't provide this bit" than say "provide this bit in this particular way". I quite like Christopher's suggestion of calling a python function from settings.py too. -- You received this message because you are subscribed to the Google Groups "Django users" group. To unsubscribe from this group and stop receiving emails from it, send an email to django-users+unsubscr...@googlegroups.com. To post to this group, send email to django-users@googlegroups.com. Visit this group at http://groups.google.com/group/django-users?hl=en. For more options, visit https://groups.google.com/groups/opt_out.
Re: Request for comments on a new Open Source Paas platform for Django
On Tuesday, March 5, 2013 7:03:36 PM UTC, Bruno Girin wrote: > > So I'd much rather have the charm auto-generate part of the config in a > sensible way and then tell people: if you use Juju, don't provide those > settings in your config, the charm will do it. > ...and if they do? How is this any different to the code not handling environment variables correctly? People are going to need to adapt their project to the charm no matter what approach you take. -- You received this message because you are subscribed to the Google Groups "Django users" group. To unsubscribe from this group and stop receiving emails from it, send an email to django-users+unsubscr...@googlegroups.com. To post to this group, send email to django-users@googlegroups.com. Visit this group at http://groups.google.com/group/django-users?hl=en. For more options, visit https://groups.google.com/groups/opt_out.
Re: Request for comments on a new Open Source Paas platform for Django
For what it's worth, "other PaaS solutions" solve this by letting people call a python function from settings.py I think it's a good solution. On Mar 5, 2013 8:04 PM, "Bruno Girin" wrote: > > > On Tuesday, 5 March 2013 13:35:41 UTC, Dave Murphy wrote: >> >> On Tuesday, March 5, 2013 1:17:13 PM UTC, Michael wrote: >>> >>> Are there other better options that wouldn't force people to change >>> their code to use the charm? >>> >> >> For the charm to be of sufficient value, it needs to be opinionated, >> otherwise it's going to suffer from trying to work out-of-the-box for >> everyone. >> > > True but using environment variables only moves the problem somewhere > else: it means that you still need to auto-generate the part of the startup > script that sets the environment variables. Besides, as said above, it > would force django app developers to explicitly use those environment > variables and if they forget one of misspell it, the danger is that it > would fail silently. Then the risk is that it's hard to debug because > you're hunting setting values in two places: the django settings and the > startup scripts that sets the environment variables. > > So I'd much rather have the charm auto-generate part of the config in a > sensible way and then tell people: if you use Juju, don't provide those > settings in your config, the charm will do it. > > Bruno > > -- > You received this message because you are subscribed to a topic in the > Google Groups "Django users" group. > To unsubscribe from this topic, visit > https://groups.google.com/d/topic/django-users/979lJojyxs0/unsubscribe?hl=en > . > To unsubscribe from this group and all its topics, send an email to > django-users+unsubscr...@googlegroups.com. > To post to this group, send email to django-users@googlegroups.com. > Visit this group at http://groups.google.com/group/django-users?hl=en. > For more options, visit https://groups.google.com/groups/opt_out. > > > -- You received this message because you are subscribed to the Google Groups "Django users" group. To unsubscribe from this group and stop receiving emails from it, send an email to django-users+unsubscr...@googlegroups.com. To post to this group, send email to django-users@googlegroups.com. Visit this group at http://groups.google.com/group/django-users?hl=en. For more options, visit https://groups.google.com/groups/opt_out.
Re: Request for comments on a new Open Source Paas platform for Django
On Tuesday, 5 March 2013 13:35:41 UTC, Dave Murphy wrote: > > On Tuesday, March 5, 2013 1:17:13 PM UTC, Michael wrote: >> >> Are there other better options that wouldn't force people to change their >> code to use the charm? >> > > For the charm to be of sufficient value, it needs to be opinionated, > otherwise it's going to suffer from trying to work out-of-the-box for > everyone. > True but using environment variables only moves the problem somewhere else: it means that you still need to auto-generate the part of the startup script that sets the environment variables. Besides, as said above, it would force django app developers to explicitly use those environment variables and if they forget one of misspell it, the danger is that it would fail silently. Then the risk is that it's hard to debug because you're hunting setting values in two places: the django settings and the startup scripts that sets the environment variables. So I'd much rather have the charm auto-generate part of the config in a sensible way and then tell people: if you use Juju, don't provide those settings in your config, the charm will do it. Bruno -- You received this message because you are subscribed to the Google Groups "Django users" group. To unsubscribe from this group and stop receiving emails from it, send an email to django-users+unsubscr...@googlegroups.com. To post to this group, send email to django-users@googlegroups.com. Visit this group at http://groups.google.com/group/django-users?hl=en. For more options, visit https://groups.google.com/groups/opt_out.
Re: Request for comments on a new Open Source Paas platform for Django
On Tuesday, March 5, 2013 1:17:13 PM UTC, Michael wrote: > > Are there other better options that wouldn't force people to change their > code to use the charm? > For the charm to be of sufficient value, it needs to be opinionated, otherwise it's going to suffer from trying to work out-of-the-box for everyone. -- You received this message because you are subscribed to the Google Groups "Django users" group. To unsubscribe from this group and stop receiving emails from it, send an email to django-users+unsubscr...@googlegroups.com. To post to this group, send email to django-users@googlegroups.com. Visit this group at http://groups.google.com/group/django-users?hl=en. For more options, visit https://groups.google.com/groups/opt_out.
Re: Request for comments on a new Open Source Paas platform for Django
On Tue, Mar 5, 2013 at 1:55 PM, Dave Murphy wrote:
> On Sunday, March 3, 2013 4:45:56 PM UTC, Bruno Girin wrote:
>
>> Patrick solved that problem by separating different config elements in
>> different files but this implies that juju'ised applications would need to
>> follow the same structure. Is that a good idea?
>
>
> If you're aiming for a PaaS-style charm for Django, then trying to emulate
> The Twelve-Factor App [1] and it's opinions on configuration [2] would be a
> good starting point. Basically, if there are things specific to the
> environment then they should be in environment variables, not in generated
> or modified config files.
>
>
+1 - but Django doesn't support env vars overriding settings out of the box
(although configglue does enable this) which means forcing people using the
charm to update their settings (rather than just encouraging good
practise). That said, it might be a small (and worthwhile) thing to ask of
people wanting to use the charm. (ie. to use this charm, your settings need
to read the following env vars...) If not, writing a local_settings.py that
overrides the required settings from the env may work:
local_settings.py:
{{{
import os
from userproject.settings import *
LANGUAGE_CODE = os.environ['DJANGO_LANGUAGE_CODE']
...
}}}
Are there other better options that wouldn't force people to change their
code to use the charm?
--
You received this message because you are subscribed to the Google Groups
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.
Re: Request for comments on a new Open Source Paas platform for Django
On Sunday, March 3, 2013 4:45:56 PM UTC, Bruno Girin wrote: > Patrick solved that problem by separating different config elements in > different files but this implies that juju'ised applications would need to > follow the same structure. Is that a good idea? If you're aiming for a PaaS-style charm for Django, then trying to emulate The Twelve-Factor App [1] and it's opinions on configuration [2] would be a good starting point. Basically, if there are things specific to the environment then they should be in environment variables, not in generated or modified config files. -- You received this message because you are subscribed to the Google Groups "Django users" group. To unsubscribe from this group and stop receiving emails from it, send an email to django-users+unsubscr...@googlegroups.com. To post to this group, send email to django-users@googlegroups.com. Visit this group at http://groups.google.com/group/django-users?hl=en. For more options, visit https://groups.google.com/groups/opt_out.
Re: Request for comments on a new Open Source Paas platform for Django
On Sunday, 3 March 2013 17:45:56 UTC+1, Bruno Girin wrote:
> The main stumbling block at the moment and for which we could do with
> Django expertise is about the structure of the settings files. Some
> settings are application specific and should be left alone by Juju, others
> are environment specific and should be generated by Juju (database config
> for instance). Patrick solved that problem by separating different config
> elements in different files but this implies that juju'ised applications
> would need to follow the same structure. Is that a good idea?
>
>
I had a go at something similar a while back [1] (well, a very cut-down
version of what you guys are attempting), and for that I used configglue's
in-built support for local settings that override the project settings [2],
but I'm assuming in this case we'd not want to force juju'ised projects to
use configglue either.
I've not tried this on any production app, or thought about it more than
the example below - perhaps others can say the obvious issues they see -
but one idea that comes to mind is just reversing how people normally split
up their settings files, something like:
{{{
$ django-admin startproject test_settings
$ cd test_settings/
$ echo 'from django.conf import settings;print("Debug is: %s. LangCode is:
%s" % (settings.DEBUG, settings.LANGUAGE_CODE))' | ./manage.py shell --plain
(prints "Debug is: True. LangCode is: en-us")
$ echo -e "from test_settings.settings import *\nDEBUG = False" >
local_settings.py
$ echo 'from django.conf import settings;print("Debug is: %s. LangCode is:
%s" % (settings.DEBUG, settings.LANGUAGE_CODE))' | ./manage.py shell
--plain --settings=local_settings
(prints "Debug is: False. LangCode is: en-us")
}}}
-Michael
[1]
http://micknelson.wordpress.com/2011/11/22/a-generic-juju-charm-for-django-apps/
[2]
http://bazaar.launchpad.net/~michael.nelson/charms/oneiric/apache-django-wsgi/trunk/view/head:/hooks/manifests/database_settings.pp
--
You received this message because you are subscribed to the Google Groups
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.
Re: Request for comments on a new Open Source Paas platform for Django
Hi Chris, I've been working with Patrick on this charm and I implemented a simple version of support for private repositories. It basically creates a .netrc file with the user name and password for the correct machine. It's not ideal but it did enable me to get code from a private github repo. We need something more robust in the long term but it's a start. Regarding the concept of running gunicorn on a different machine, this will not be necessary going forward. v2.0 of Juju is meant to support co-location where you could have different services on the same machine (in practice this is already supported in the jitsu package). However, Patrick's idea is to ensure that you can use any WSGI server, whether Apache or gunicorn without having to force one or the other. Everything else you suggest is definitely in the pipe. The main stumbling block at the moment and for which we could do with Django expertise is about the structure of the settings files. Some settings are application specific and should be left alone by Juju, others are environment specific and should be generated by Juju (database config for instance). Patrick solved that problem by separating different config elements in different files but this implies that juju'ised applications would need to follow the same structure. Is that a good idea? Cheers, Bruno On Saturday, 2 March 2013 20:14:19 UTC, Christopher Glass wrote: > > Hi Patrick, > > Great to hear you're interested in writing a Django charm for juju! I have > toyed around with the idea, but never got around to implementing something > good. > I started looking at the current Django charm a little while ago, and > while it works to some extend I think we could make really great things > happen with a little work. > > As far as feedback for your point goes, here are a few points and > suggestion I'd like to add to the discussion: > > - Most of the Django websites will likely live in private git/bzr/whatever > repositories, and so in the workflow you outlined, you need to somehow push > the *private identifier* to the running juju instance. In the "standard" > scenario that means pushing your private ssh key to the instance, so it can > git clone from a private repository on github... I think it's safe to say > that most people will at least frown at the idea :) > Maybe we should instead make this a "push" process? > > - It seems a little strange to me to run gunicorn on another machine. Most > of the Django project I have encountered run Django with gunicorn on the > webservers themselves (add gunicorn to INSTALLED_APPS and then "manage.py > run_gunicorn"). Perhaps we should be a little more opinionated about things > and for the sake of scaling simplicity deploy nginx or apache locally too > (wither with a charm subordinate or at install), so that we can > load-balance to all of the servers easily with any frontend (that means all > webservers would serve static files, which might not be optimal, but we can > refine that later). > > - We should absolutely define a cache relation (redis or memcached). > > Theses points would make the whole workflow look like the following (the > juju syntax might be a little wrong, but please bear with me :) ) > > juju bootstrap > > juju deploy --config my_django_conf.yaml cs:django_server my_django_site > juju deploy cs:postgresql # or mysql,mongodb, etc > juju deploy cs:memcached # or redis if that's still popular > juju deploy cs:haproxy > > juju add-relation my_django_site postgresql > juju add-relation my_django_site memcached > juju add-relation my_django_site haproxy # strictly speaking that's > optional if you have only one django machine > > juju expose haproxy > > # when needed (I hope we all need it someday!) > juju add-unit my_django_site > juju add-unit memcached > juju add-unit postgresql > > So now we would have a running django server with no code. > But if it's a push process, we can implement many of the config changes as > git hooks, which makes the workflow continue with: > > cd my-django-site > git init . # If that's not done already of course > git add . > git commit -m "produciton push yay!" > git remote add production > git+ssh://my_django_site/some_configurable_url.git > git push production master # or of course whatever branch you put in the > config.yaml > > Of course, that requires a non-trivial amount of git triggers to be > written, and we should put some requirements.pip.txt file and > requirements.apt.txt or whatever in the project tree, but I think that's > acceptable. > The whole thing basically follows what many PaaS providers already do, so > I guess most Django developers with some sites in production probably are > familiar with the workflow. > > This would just add the juju coolness to it :) > > Hope this fuels the discussion, > > - Chris > > On Friday, March 1, 2013 8:13:36 PM UTC+1, Patrick wrote: >> >> Hi, >> >> I'm building a J
Re: Request for comments on a new Open Source Paas platform for Django
Hi Patrick, Great to hear you're interested in writing a Django charm for juju! I have toyed around with the idea, but never got around to implementing something good. I started looking at the current Django charm a little while ago, and while it works to some extend I think we could make really great things happen with a little work. As far as feedback for your point goes, here are a few points and suggestion I'd like to add to the discussion: - Most of the Django websites will likely live in private git/bzr/whatever repositories, and so in the workflow you outlined, you need to somehow push the *private identifier* to the running juju instance. In the "standard" scenario that means pushing your private ssh key to the instance, so it can git clone from a private repository on github... I think it's safe to say that most people will at least frown at the idea :) Maybe we should instead make this a "push" process? - It seems a little strange to me to run gunicorn on another machine. Most of the Django project I have encountered run Django with gunicorn on the webservers themselves (add gunicorn to INSTALLED_APPS and then "manage.py run_gunicorn"). Perhaps we should be a little more opinionated about things and for the sake of scaling simplicity deploy nginx or apache locally too (wither with a charm subordinate or at install), so that we can load-balance to all of the servers easily with any frontend (that means all webservers would serve static files, which might not be optimal, but we can refine that later). - We should absolutely define a cache relation (redis or memcached). Theses points would make the whole workflow look like the following (the juju syntax might be a little wrong, but please bear with me :) ) juju bootstrap juju deploy --config my_django_conf.yaml cs:django_server my_django_site juju deploy cs:postgresql # or mysql,mongodb, etc juju deploy cs:memcached # or redis if that's still popular juju deploy cs:haproxy juju add-relation my_django_site postgresql juju add-relation my_django_site memcached juju add-relation my_django_site haproxy # strictly speaking that's optional if you have only one django machine juju expose haproxy # when needed (I hope we all need it someday!) juju add-unit my_django_site juju add-unit memcached juju add-unit postgresql So now we would have a running django server with no code. But if it's a push process, we can implement many of the config changes as git hooks, which makes the workflow continue with: cd my-django-site git init . # If that's not done already of course git add . git commit -m "produciton push yay!" git remote add production git+ssh://my_django_site/some_configurable_url.git git push production master # or of course whatever branch you put in the config.yaml Of course, that requires a non-trivial amount of git triggers to be written, and we should put some requirements.pip.txt file and requirements.apt.txt or whatever in the project tree, but I think that's acceptable. The whole thing basically follows what many PaaS providers already do, so I guess most Django developers with some sites in production probably are familiar with the workflow. This would just add the juju coolness to it :) Hope this fuels the discussion, - Chris On Friday, March 1, 2013 8:13:36 PM UTC+1, Patrick wrote: > > Hi, > > I'm building a Juju based Open Source Paas platform for Django and > I need your help because it is a hard task to make a PAAS system > that is flexible enough to deploy any projects and at the same time > simple to use. > > For the ones that don't know Juju, it's a service orchestration software > compatible with LXC (local), EC2, HPCloud, OpenStack and Baremetal/Maas > developed by Canonical (the company that makes Ubuntu). > > Check out the web site for more details: https://juju.ubuntu.com/ > > So quickly, here's how it would works: > > After installing Juju and configuring it with for your favourite cloud > provider you > will need to create a configuration file in the YAML format named > my_django_conf.yaml > in this example:: > > my_django_site: > vcs: git > repos_url: https://github.com/my_username/my_site.git > site_secret_key: abcdefgh123456789 > use_virtualenv: True > > Then you will need these commands to bootstrap and launch all the servers:: > > juju bootstrap > > juju deploy --config my_django_conf.yaml my_django_site > juju deploy postgresql # or mysql,mongodb, etc > juju deploy gunicorn # Or mod_wsgi, etc > > juju add-relation my_django_site postgresql > juju add-relation my_django_site gunicorn > > juju expose gunicorn # Open the tcp port in the firewall > > You will end up with 3 servers running. One will be the controller > and one for each service (django and the database). > Gunicorn will be a special charm that will be installed on your Django > server. > After that, adding a new Django node would be a
Request for comments on a new Open Source Paas platform for Django
Hi, I'm building a Juju based Open Source Paas platform for Django and I need your help because it is a hard task to make a PAAS system that is flexible enough to deploy any projects and at the same time simple to use. For the ones that don't know Juju, it's a service orchestration software compatible with LXC (local), EC2, HPCloud, OpenStack and Baremetal/Maas developed by Canonical (the company that makes Ubuntu). Check out the web site for more details: https://juju.ubuntu.com/ So quickly, here's how it would works: After installing Juju and configuring it with for your favourite cloud provider you will need to create a configuration file in the YAML format named my_django_conf.yaml in this example:: my_django_site: vcs: git repos_url: https://github.com/my_username/my_site.git site_secret_key: abcdefgh123456789 use_virtualenv: True Then you will need these commands to bootstrap and launch all the servers:: juju bootstrap juju deploy --config my_django_conf.yaml my_django_site juju deploy postgresql # or mysql,mongodb, etc juju deploy gunicorn # Or mod_wsgi, etc juju add-relation my_django_site postgresql juju add-relation my_django_site gunicorn juju expose gunicorn # Open the tcp port in the firewall You will end up with 3 servers running. One will be the controller and one for each service (django and the database). Gunicorn will be a special charm that will be installed on your Django server. After that, adding a new Django node would be as simple as:: juju add-unit my_django_site As I said, where it gets tricky is how do I make the configuration flexible enough and at the same time simple. After looking at what was existing in Django's Paas world, I came with this: 1 - We need a configurable requirements files for both pip and apt-get. By default it would be looking for package in there files at install time:: requirements_pip_files: requirements.txt,requirements.pip requirements_apt_files: requirements.apt and we could also configure extra packages by adding variables like this in the YAML file:: additional_distro_packages: vim,emacs,etc additional_pip_packages: virtualenvwrapper,celery,South,etc 2 - I'm suggesting to use separate configurations files in a settings/ directory so by default it will be injecting configuration in those files:: settings_database_path: settings/20-engine.py settings_static_path: settings/20-static.py settings_uploads_path: settings/20-media.py settings_cache_path: settings/30-cache.py settings_secret_key_path: settings/20-secret.py I'm suggesting splitting settings because when the configuration is modified, for some reason, it would be difficult and risky to parse settings.py and change only the right thing. So instead, I would be using topic files rendered with templates. So if you would need to do more advanced stuff you could just fork the charm and modify the templates for your needs. 3 - Finally, I was thinking adding some options to execute custom scripts that would run a various time during the deployment. Like after packages installation ,database configuration and static file configuration:: post_database_script: type: string default: | #!/bin/sh python manage.py syncdb --noinput python manage.py migrate --noinput post_static_script: type: string default: | #!/bin/sh python manage.py collectstatic -v 0 --noinput Note that this is not making unanimity so far. There is several reasons that makes the scripts approach tricky: * You don't want to execute these scripts every time a little detail change. * You might need the database configuration to be ready for some script. * You could be not using south * You might want to import some initial data and maybe only once at install time. * You could want to compress static files after running collectstatic * etc An other idea could be to use a Fabric plug-in that use Juju's database to connect to the machines and run commands like this for example:: fab -R my_django_site python manage.py pull would pull the latest version of the site and reload the application on every deployed Django machines. The bottom line here is that it's not simple to find out what a standard Django deployment (and is maintenance) looks like. That being said, I'm really looking forwards for you comments and suggestions. Patrick -- You received this message because you are subscribed to the Google Groups "Django users" group. To unsubscribe from this group and stop receiving emails from it, send an email to django-users+unsubscr...@googlegroups.com. To post to this group, send email to django-users@googlegroups.com. Visit this group at http://groups.google.com/group/django-users?hl=en. For more options, visit https://groups.google.com/groups/opt_out.
