Re: declarative settings

[Alexei Znamensky]

Howdy,

You guys ever seen or used this:
https://pypi.org/project/django-configurations/

Reusable settings. That would be something I would like to see in Django.


On Fri, 28 Feb 2020 at 5:49 AM, Robert Marsanyi  wrote:

> I have to say, having settings as Python code is one of the features of
> Django for me.  Being able to calculate settings values, take advantage of
> Python’s module/class parsing, and so on all make settings more than just a
> set of dictionary values.  I think we would lose a bit of functionality if
> we chose to reduce them to something that YAML or its ilk knows how to
> express.
>
> --
> You received this message because you are subscribed to a topic in the
> Google Groups "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/django-developers/eDdUYM3YN1U/unsubscribe
> .
> To unsubscribe from this group and all its topics, send an email to
> django-developers+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/django-developers/3de22278-7bad-4409-a2d1-185792a4375c%40googlegroups.com
> 
> .
>
-- 
Alexei Znamensky | russoz gmail com
http://github.com/russoz

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CAFK_j12yBGoi5NMSAGyA5FZUv%2BNwzDYgDg8jS%2BCEVcDqP1s4xg%40mail.gmail.com.

Re: declarative settings

[Robert Marsanyi]

I have to say, having settings as Python code is one of the features of 
Django for me.  Being able to calculate settings values, take advantage of 
Python’s module/class parsing, and so on all make settings more than just a 
set of dictionary values.  I think we would lose a bit of functionality if 
we chose to reduce them to something that YAML or its ilk knows how to 
express.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/3de22278-7bad-4409-a2d1-185792a4375c%40googlegroups.com.

Re: declarative settings

[Adrian Turjak]

Declarative settings, and a lack of a good settings file parsing system,
led me into some rather interesting directions not that long ago.

I maintain an OpenStack project called Adjutant, and I built it on
Django, but ended up using yaml as my config file and having settings.py
read it and pull in certain values. But I ended up doing some rather
weird nested stuff, and kept introducing small sections of code that
needed some config value. But because I didn't have a way to declare
those configs somewhere, it essentially turned into a nightmare to know
exactly what values were configurable, and where in the code they were used.

OpenStack uses a config library system called oslo.config which allows
declaring config, and then consuming it. Sadly for me that didn't handle
nesting. dynamic config registration... or too much weird complexity.
Plus it was based off INI files, and didn't support yaml or toml. So I
ended up writing my own variant which did.

The library I wrote is:
https://gitlab.com/catalyst-cloud/confspirator

And the way I use it:
https://github.com/openstack/adjutant/tree/master/adjutant/config
https://github.com/openstack/adjutant/blob/master/adjutant/settings.py

Which this being how I handle my plugins, and their config:
https://github.com/openstack/adjutant/blob/master/adjutant/feature_set.py

The whole thing is maybe a little over the top, but it allows dynamic
config registration via plugins, declarative values, so you can't ever
use a config without defining it, and it can read those config values
from envvars or you can pass it a dict structure to parse (which you can
load in from json, yaml, or toml).

And a way to then use that config tree to generate example config:
https://github.com/openstack/adjutant/blob/master/adjutant/commands/management/commands/exampleconfig.py

This may not be what you need, but given my random pain in what I think
is a similar area, I thought I may as well share. I do intent to
maintain CONFspirator, and will be adding native support for
parsing/loading yaml/toml, as well as utils for example config
generation soon.

On 31/12/19 11:45 am, Christian González wrote:
> Hello,
>
> I recently "finished" my first really working version of GDAPS, my
> Generic Django Application Plugin System. It's noway perfect, but does
> what it should: providing pluggable apps for an app framework, including
> a more or less flexible frontend with each django app.
>
> I had much struggle with it, and one of the lessons I learned was
> Django's setup system, and how it deals with loading apps. Unfortunately
> Django can't load/unload apps on the fly, so it is necessary to restart
> Django whenever a new GDAPS app is installed via pip.
>
> But: I want to resurrect an old theme again which would, in a way,
> improve some of the loading problems I encountered. Django's settings
> are code. Which is, in fact, a very good thing, as it makes it extremely
> flexible and adaptable to different setups. But, as discussed with the
> SECRET_KEY here, some of the settings _have_ to be coded very
> complicated, and it makes some things like per-app-settings extremely
> uncomfortable.
>
> What if - and please don't kill me instantly - yes, I am a newcomer, and
> not a good programmer maybe - but some things are viewed better from
> "outside" - what if Django settings could be "declarative"?
>
> So instead of Python code like
>
> INSTALLED_APPS = [
>     'django.contrib.admin',
>     'django.contrib.auth',
>     'django.contrib.contenttypes'
> ]
>
> This would be in an e.g. JSON file
>
> {
>
>     "INSTALLED_APPS": [
>         "django.contrib.admin",
>         "django.contrib.auth",
>         "django.contrib.contenttypes"
>     ] ,
>     ROOT_URLCONF: "fooproject.urls"
> }
>
> Django's settings.py would look different: It would load that
> settings.json file and set the appropriate values into local code - so
> this wouldn't make much difference.
>
> Except 2 things:
>
> 1. Apps could have (default) settings, and they could be merged MUCH
> easier. Things like namespaced classes that are overwriting values like
> DRF/graphene does, would be completely unnecessary. The main
> settings.json file could be the "last word" in the process of settings,
> so anything an app would suggest could be overrided in the main file.
>
> 2. Installed apps could be managed much more comfortable. Adding an app
> could be done by a script (JSON editing is easy. Editing code
> (=settings.py) is error prone and uncomfortable). I have a Django
> command script ATM for that, but just because I add a line into
> settings.py to add some additional apps to the list.
>
> This even could be done with backwards compatibility, becau

Re: declarative settings

[Christian González]

Am 01.01.20 um 15:04 schrieb Adam Johnson:
>
> This seems a weird suggest, but when apps depend on another app it
> should be able to override settings for that dependency too
> (because it "uses" that app anyway).
>
>
> It does seem weird to suggest. I haven't seen such a use case myself.
> Can you describe in more detail?
>
> For me, I can only imagine it would violate the "principle of least
> astonishment" that if I had app A installed for some time, then
> installed app B on top, app B would change the default or manually
> configured settings of app A. I would prefer if app B complained (via
> system check) that I had an incompatible app A configuration.
>
Maybe you're right. I was about to answer how the system should behave
when not thinking in "apps" - which are in a flat hierarchy. In GDAPS I
try to think in modules/plugins of ONE application - that's what I try
to achieve. Let's explain shortly: My main aim is to create an Open
Source Electronic Medical Record - meaning I want to create a base
system where maybe 3rd party companies could add their own plugins to
the system. Everything based upon django as backend (with a
REST/Graphene web API), and variable frontends, let's say a web frontend
with Vue.js.

But even when a base module has some "default settings", and I install a
module B which overrides that defaults because system behaviour is
different when using this module, it doesn't make sense. Such a case
should be done in code (module B circumvents the default settings etc.)
- but settings merging / hierarchy maybe isn't the best way to go. I
also strictly believe in "namespaces are a good thing."

So forget overriding settings, there only remains the easy settings
access via one object, which would be great.

Christian

-- 
Dr. Christian González
https://nerdocs.at

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/88ae1ed1-425d-1f8c-aef6-6062e9b7caf3%40nerdocs.at.


pEpkey.asc
Description: application/pgp-keys

Re: declarative settings

[Christian González]

Am 02.01.20 um 08:06 schrieb Alexei Znamensky:
> Having said that, I recently stumbled on this package
> django-configurationsby chance, while working on something else. It
> looks quite promising, as one can use (multiple) class inheritance to
> merge and combine different settings. As far as I can tell, it lacks
> the dynamic aspect of your GDAPS idea, so I suppose it would not solve
> the problem of loading plugins in runtime. But, for the part in which
> you want to load settings from json/yaml/toml/database/wherever, it
> might be simpler to piggyback on that project and have Configuration
> classes that load markup, and a config class that loads from DB
> (although this would probably be trickier than it looks).
yes, I already had stumbled upon django-configurations too. But apart
from not solving the dynamic loading "problem" of Django (which is not
my main concern: reloading the django server is ok for me ATM), it drags
in a bunch of own config necessities and changes that have to be done to
make it work: The "quick start" replaces the whole settings.py with a
class, and you have to change the manage.py file too. No plug'n'play.
Sure, you can make this class load a config file then too.

But for me it reminds me of the joke:

    Waiter: What would you like to eat?
    Guest: I'd like to have a burger with fries, but instead of the
fries please bring me rice,
        and instead of the burger I'd like a soup.

If I replace everything the config does, it can do something else. But
it still does not solve my problem at all... which is easily *changing*
the installed apps programmatically - so that, after a restart - new
apps are available. In GDAPS, I only add one line, if you install GDAPS
apps per pip, they are found automatically and work after a restart. But
I think this is Django's "fault" - it could do better.

What would be really a game changer: in django.conf there are
LazySettings and Settings classes, and they currently load from a module
(https://github.com/django/django/blob/c574bec0929cd2527268c96a492d25223a9fd576/django/conf/__init__.py#L171),
after checking the ENVIRONMENT_VARIABLE.

A quick proposal would be to

    1. load the environment variable to get the settings name
    2. NEW: look for a declarative settings file and load that
    3. if that doesn't exist (or additionally), load the settings from
project's settings.py as usual

This way everything would be completely unchanged and backwards
compatible, while adding declarative settings functionality.

This doesn't solve per-app settings - but while loading the declarative
config, installed apps would be defined already - and their paths should
be possible to determine by then. So *their* declarative settings could
be found and merged as well.

Christian

-- 
Dr. Christian González
https://nerdocs.at

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/5c3f0bab-4d82-973c-140f-bc6254a52937%40nerdocs.at.


pEpkey.asc
Description: application/pgp-keys

Re: declarative settings

[Alexei Znamensky]

Hi Christian,

Definitely not a boring theme, but as pointed by others, there are many 
aspects to be taken into consideration.

Having said that, I recently stumbled on this package django-configurations 
by chance, while working on something else. It looks quite promising, as 
one can use (multiple) class inheritance to merge and combine different 
settings. As far as I can tell, it lacks the dynamic aspect of your GDAPS 
idea, so I suppose it would not solve the problem of loading plugins in 
runtime. But, for the part in which you want to load settings from 
json/yaml/toml/database/wherever, it might be simpler to piggyback on that 
project and have Configuration classes that load markup, and a config class 
that loads from DB (although this would probably be trickier than it looks).

Happy New Year to you as well, greetings from NZ
Alex

On Wednesday, January 1, 2020 at 7:11:04 AM UTC+13, Christian González 
wrote:
>
> Hi all,
>
> thanks for your fast response - it seems that this is not a "ah, forget 
> it, boooring." theme.
> I'll try to answer all (for me) relevant things at once here - sorry to 
> write such a huge mail.
> *@Adam Johnson:*
>
> * A class to read django.conf.settings and add defaults/other logic 
> through properties - for example 
> https://github.com/adamchainz/django-cors-headers/blob/31b9c2ef8a333a40f18081ffc1f1cba9fb34574d/src/corsheaders/conf.py
>  
> . This has the benefit of easily providing defaults, complex logic such as 
> deriving one value from multiple settings, and being read from settings at 
> runtime rather than import time.
>
> This is a good approach too, but it's not namespaced automatically. And 
> app settings can't provide and override settings from other apps. This 
> seems a weird suggest, but when apps depend on another app it should be 
> able to override settings for that dependency too (because it "uses" that 
> app anyway).
>
>
> I am in favour of most libraries adding a single setting that controls 
> them, like DRF has REST_FRAMEWORK. I know django-cors-headers doesn't 
> implement the pattern but this is for legacy reasons since I took it over.
>
> Yes, this basically means namespacing. Depending on the size of the 
> library, this could be one big bunch of a setting.
>
> I think this pattern solves your first concern Christian. 
>
> No it doesn't completely. I used this pattern myself in GDAPS, copied from 
> graphene (which copied from DRF). But this does _not_ solve the problem: 
> This pattern  creates a per-app settings object which checks at the time of 
> using (which is anywhere after main settings.py) if there is a global 
> override of the local app's variable - and takes the right one. In this way 
> it works. but this is not very convenient - you always have to ask the 
> right settings object - it maybe would be more convenient, simpler and 
> easier to code to just ask for a namespaced setting, e.g. in a dotted path 
> style.
>
> like settings("django.ROOT_URLCONF") or settings("foo_app.SETTINGA")
>
> This is not a really good approach as it implements strings.
>
> As for your second concern, I think checks make manual installation of 
> apps easier. If a user adds your app to INSTALLED_APPS, the system checks 
> will run and can highlight any missing/misconfigured settings from 
> following the installation docs. Most libraries take 3 minutes to do the 
> initial install so I'm not sure of the value of automating that step 
> compared to adding system checks.
>
> Perfect.
>
> *@Matthew Pava*
>
> I have worked with Drupal for a long time before Django, and am 
> maintaining a few sites with Drupal, one of it rather big, with many 
> domains & shared content, 40+ modules, API to ther web services, some 
> modules custom coded. I learned a lot of it, but PHP is really ugly. Django 
> does much better, but, compared with Drupal, it has a few drawbacks. I 
> know, Drupal is a CMS, Django is a framework (which you can build a CMS 
> with) - BUT: Drupal is able (like Wordpress) to add modules/plugins per web 
> download and on-the-fly - it installs modules into place, has a very good 
> hooks system and a plugin manager. It also was a help when I designed GDAPS 
> and was one of my blueprints. Drupal however keeps ALL of it's settings in 
> the database which creates many problems on it's own 
> (development/deployment problems how to sync settings from disk to database 
> and vice versa). I wouldn't do that. Settings in code/conf files is a good 
> thing IMHO.
>
> The package manager in Drupal is composer, yes, and together with GDAPS, 
> you could use GDAPS Django packages exactly the sameway. Just install it 
> using pip/pipenv, and Dajngo/GDAPS finds them automatic

Re: declarative settings

[Adam Johnson]

>
> This seems a weird suggest, but when apps depend on another app it should
> be able to override settings for that dependency too (because it "uses"
> that app anyway).


It does seem weird to suggest. I haven't seen such a use case myself. Can
you describe in more detail?

For me, I can only imagine it would violate the "principle of least
astonishment" that if I had app A installed for some time, then installed
app B on top, app B would change the default or manually configured
settings of app A. I would prefer if app B complained (via system check)
that I had an incompatible app A configuration.

On Tue, 31 Dec 2019 at 18:10, Christian González <
christian.gonza...@nerdocs.at> wrote:

> Hi all,
>
> thanks for your fast response - it seems that this is not a "ah, forget
> it, boooring." theme.
> I'll try to answer all (for me) relevant things at once here - sorry to
> write such a huge mail.
> *@Adam Johnson:*
>
> * A class to read django.conf.settings and add defaults/other logic
> through properties - for example
> https://github.com/adamchainz/django-cors-headers/blob/31b9c2ef8a333a40f18081ffc1f1cba9fb34574d/src/corsheaders/conf.py
> . This has the benefit of easily providing defaults, complex logic such as
> deriving one value from multiple settings, and being read from settings at
> runtime rather than import time.
>
> This is a good approach too, but it's not namespaced automatically. And
> app settings can't provide and override settings from other apps. This
> seems a weird suggest, but when apps depend on another app it should be
> able to override settings for that dependency too (because it "uses" that
> app anyway).
>
>
> I am in favour of most libraries adding a single setting that controls
> them, like DRF has REST_FRAMEWORK. I know django-cors-headers doesn't
> implement the pattern but this is for legacy reasons since I took it over.
>
> Yes, this basically means namespacing. Depending on the size of the
> library, this could be one big bunch of a setting.
>
> I think this pattern solves your first concern Christian.
>
> No it doesn't completely. I used this pattern myself in GDAPS, copied from
> graphene (which copied from DRF). But this does _not_ solve the problem:
> This pattern  creates a per-app settings object which checks at the time of
> using (which is anywhere after main settings.py) if there is a global
> override of the local app's variable - and takes the right one. In this way
> it works. but this is not very convenient - you always have to ask the
> right settings object - it maybe would be more convenient, simpler and
> easier to code to just ask for a namespaced setting, e.g. in a dotted path
> style.
>
> like settings("django.ROOT_URLCONF") or settings("foo_app.SETTINGA")
>
> This is not a really good approach as it implements strings.
>
> As for your second concern, I think checks make manual installation of
> apps easier. If a user adds your app to INSTALLED_APPS, the system checks
> will run and can highlight any missing/misconfigured settings from
> following the installation docs. Most libraries take 3 minutes to do the
> initial install so I'm not sure of the value of automating that step
> compared to adding system checks.
>
> Perfect.
>
> *@Matthew Pava*
>
> I have worked with Drupal for a long time before Django, and am
> maintaining a few sites with Drupal, one of it rather big, with many
> domains & shared content, 40+ modules, API to ther web services, some
> modules custom coded. I learned a lot of it, but PHP is really ugly. Django
> does much better, but, compared with Drupal, it has a few drawbacks. I
> know, Drupal is a CMS, Django is a framework (which you can build a CMS
> with) - BUT: Drupal is able (like Wordpress) to add modules/plugins per web
> download and on-the-fly - it installs modules into place, has a very good
> hooks system and a plugin manager. It also was a help when I designed GDAPS
> and was one of my blueprints. Drupal however keeps ALL of it's settings in
> the database which creates many problems on it's own
> (development/deployment problems how to sync settings from disk to database
> and vice versa). I wouldn't do that. Settings in code/conf files is a good
> thing IMHO.
>
> The package manager in Drupal is composer, yes, and together with GDAPS,
> you could use GDAPS Django packages exactly the sameway. Just install it
> using pip/pipenv, and Dajngo/GDAPS finds them automatically.
>
> But this can only be achieved by adding a line to settings.py (call the
> plugin manager), like I did:
> https://gdaps.readthedocs.io/en/latest/Installation.html#installation
>
> These are just hacks because Django doe

Re: declarative settings

[Christian González]

Hi all,

thanks for your fast response - it seems that this is not a "ah, forget
it, boooring." theme.
I'll try to answer all (for me) relevant things at once here - sorry to
write such a huge mail.

*@Adam Johnson:*
> * A class to read django.conf.settings and add defaults/other logic
> through properties - for example
> https://github.com/adamchainz/django-cors-headers/blob/31b9c2ef8a333a40f18081ffc1f1cba9fb34574d/src/corsheaders/conf.py
> . This has the benefit of easily providing defaults, complex logic
> such as deriving one value from multiple settings, and being read from
> settings at runtime rather than import time.

This is a good approach too, but it's not namespaced automatically. And
app settings can't provide and override settings from other apps. This
seems a weird suggest, but when apps depend on another app it should be
able to override settings for that dependency too (because it "uses"
that app anyway).


> I am in favour of most libraries adding a single setting that controls
> them, like DRF has REST_FRAMEWORK. I know django-cors-headers doesn't
> implement the pattern but this is for legacy reasons since I took it over.
Yes, this basically means namespacing. Depending on the size of the
library, this could be one big bunch of a setting.
> I think this pattern solves your first concern Christian.

No it doesn't completely. I used this pattern myself in GDAPS, copied
from graphene (which copied from DRF). But this does _not_ solve the
problem: This pattern  creates a per-app settings object which checks at
the time of using (which is anywhere after main settings.py) if there is
a global override of the local app's variable - and takes the right one.
In this way it works. but this is not very convenient - you always have
to ask the right settings object - it maybe would be more convenient,
simpler and easier to code to just ask for a namespaced setting, e.g. in
a dotted path style.

like settings("django.ROOT_URLCONF") or settings("foo_app.SETTINGA")

This is not a really good approach as it implements strings.

> As for your second concern, I think checks make manual installation of
> apps easier. If a user adds your app to INSTALLED_APPS, the system
> checks will run and can highlight any missing/misconfigured settings
> from following the installation docs. Most libraries take 3 minutes to
> do the initial install so I'm not sure of the value of automating that
> step compared to adding system checks.
Perfect.

*@Matthew Pava*

I have worked with Drupal for a long time before Django, and am
maintaining a few sites with Drupal, one of it rather big, with many
domains & shared content, 40+ modules, API to ther web services, some
modules custom coded. I learned a lot of it, but PHP is really ugly.
Django does much better, but, compared with Drupal, it has a few
drawbacks. I know, Drupal is a CMS, Django is a framework (which you can
build a CMS with) - BUT: Drupal is able (like Wordpress) to add
modules/plugins per web download and on-the-fly - it installs modules
into place, has a very good hooks system and a plugin manager. It also
was a help when I designed GDAPS and was one of my blueprints. Drupal
however keeps ALL of it's settings in the database which creates many
problems on it's own (development/deployment problems how to sync
settings from disk to database and vice versa). I wouldn't do that.
Settings in code/conf files is a good thing IMHO.

The package manager in Drupal is composer, yes, and together with GDAPS,
you could use GDAPS Django packages exactly the sameway. Just install it
using pip/pipenv, and Dajngo/GDAPS finds them automatically.

But this can only be achieved by adding a line to settings.py (call the
plugin manager), like I did:
https://gdaps.readthedocs.io/en/latest/Installation.html#installation

These are just hacks because Django doesn't have declarative settings.
I suggest that if changing to declarative, it MUST be backwards
compatible, yes. So there could be an _internal_ settings.py which loads
environment variables, the bespoken config file, and like I said, **if**
there is a settings.py in the project, Django could use it - to override
settings in a way just code could do it.

*@Jacob Rief and @Tom Forbes*

Sure, JSON was just one idea. It's "xml 2.0", and not suited to be
written by humans easily. yaml or toml would be better, in fact, in a
first approach of GDAPS I used toml as meta config data for my Django
plugins. I decided (because there IS no common framework in Django, and
I didn't want to add another dependency) to let it go and am using an
attribute of AppConfig ATM - code again.

> /Some settings must be taken from the environment./*
*

Sure. DJANGO_SECRET_KEY etc. But this could be handled by the "internal"
settings.py file.

> /So rather than just “allowing people to use JSON files for
settings”// //it would be very inte

Re: declarative settings

[Adam Johnson]

When using settings for my third party packages I use:

* A class to read django.conf.settings and add defaults/other logic through
properties - for example
https://github.com/adamchainz/django-cors-headers/blob/31b9c2ef8a333a40f18081ffc1f1cba9fb34574d/src/corsheaders/conf.py
. This has the benefit of easily providing defaults, complex logic such as
deriving one value from multiple settings, and being read from settings at
runtime rather than import time.
* A bunch of system checks to validate them - for example
https://github.com/adamchainz/django-cors-headers/blob/31b9c2ef8a333a40f18081ffc1f1cba9fb34574d/src/corsheaders/checks.py
. For example, if a setting is *required*, you can add a check error; or if
a setting defaults to something unsuitable for production, you can add a
deploy check. System checks make it easy for users to upgrade your package
too, since many users don't read changelogs even for breaking version
changes.

I am in favour of most libraries adding a single setting that controls
them, like DRF has REST_FRAMEWORK. I know django-cors-headers doesn't
implement the pattern but this is for legacy reasons since I took it over.

I think this pattern solves your first concern Christian. As for your
second concern, I think checks make manual installation of apps easier. If
a user adds your app to INSTALLED_APPS, the system checks will run and can
highlight any missing/misconfigured settings from following the
installation docs. Most libraries take 3 minutes to do the initial install
so I'm not sure of the value of automating that step compared to adding
system checks.

Jacob you're right it's a bit of an inconvenience to write a class to wrap
django.conf.settings but I'm not sure there's any general purpose wrapper
that could be provided. There are too many cases for evolution - a setting
changes name, one value combines multiple settings, etc.


On Tue, 31 Dec 2019 at 00:09, Tom Forbes  wrote:

> My two cents: JSON isn’t great as a configuration language - It’s
> annoyingly picky about some things. YAML or TOML are “better” (for some
> definition of better) choices for this domain, in my option. However,
> Django is historically quite hesitant about including third party packages
> and I think including a TOML or YAML parser even as an optional dependency
> might not be the way forward.
>
> So rather than just “allowing people to use JSON files for settings” it
> would be very interesting to explore what a pluggable settings backend
> would look like. It seems that work in this area would be best spent on a
> general abstraction rather than a specific one. There was some discussion
> around this idea in the GSOC thread and I’m sure it’s come up before.
> Personally I think configuration management is an area of Django that is
> quite underdeveloped compared to other frameworks.
>
> On 30 Dec 2019, at 22:46, Christian González <
> christian.gonza...@nerdocs.at> wrote:
>
> Hello,
>
> I recently "finished" my first really working version of GDAPS, my
> Generic Django Application Plugin System. It's noway perfect, but does
> what it should: providing pluggable apps for an app framework, including
> a more or less flexible frontend with each django app.
>
> I had much struggle with it, and one of the lessons I learned was
> Django's setup system, and how it deals with loading apps. Unfortunately
> Django can't load/unload apps on the fly, so it is necessary to restart
> Django whenever a new GDAPS app is installed via pip.
>
> But: I want to resurrect an old theme again which would, in a way,
> improve some of the loading problems I encountered. Django's settings
> are code. Which is, in fact, a very good thing, as it makes it extremely
> flexible and adaptable to different setups. But, as discussed with the
> SECRET_KEY here, some of the settings _have_ to be coded very
> complicated, and it makes some things like per-app-settings extremely
> uncomfortable.
>
> What if - and please don't kill me instantly - yes, I am a newcomer, and
> not a good programmer maybe - but some things are viewed better from
> "outside" - what if Django settings could be "declarative"?
>
> So instead of Python code like
>
> INSTALLED_APPS = [
> 'django.contrib.admin',
> 'django.contrib.auth',
> 'django.contrib.contenttypes'
> ]
>
> This would be in an e.g. JSON file
>
> {
>
> "INSTALLED_APPS": [
> "django.contrib.admin",
> "django.contrib.auth",
> "django.contrib.contenttypes"
> ] ,
> ROOT_URLCONF: "fooproject.urls"
> }
>
> Django's settings.py would look different: It would load that
> settings.json file and set the appropriate values into local code - so
> this wouldn't make much difference.
>
> Except 2 things:
>
> 1. Apps could have (default) settings, and they could be merged MUCH
> easier. Things like namespaced classes that are overwriting values like
> DRF/graphene does, would be completely unnecessary. The main
> settings.json file could be the "last word" in 

Re: declarative settings

[Tom Forbes]

My two cents: JSON isn’t great as a configuration language - It’s annoyingly 
picky about some things. YAML or TOML are “better” (for some definition of 
better) choices for this domain, in my option. However, Django is historically 
quite hesitant about including third party packages and I think including a 
TOML or YAML parser even as an optional dependency might not be the way forward.

So rather than just “allowing people to use JSON files for settings” it would 
be very interesting to explore what a pluggable settings backend would look 
like. It seems that work in this area would be best spent on a general 
abstraction rather than a specific one. There was some discussion around this 
idea in the GSOC thread and I’m sure it’s come up before. Personally I think 
configuration management is an area of Django that is quite underdeveloped 
compared to other frameworks.

> On 30 Dec 2019, at 22:46, Christian González  
> wrote:
> 
> Hello,
> 
> I recently "finished" my first really working version of GDAPS, my
> Generic Django Application Plugin System. It's noway perfect, but does
> what it should: providing pluggable apps for an app framework, including
> a more or less flexible frontend with each django app.
> 
> I had much struggle with it, and one of the lessons I learned was
> Django's setup system, and how it deals with loading apps. Unfortunately
> Django can't load/unload apps on the fly, so it is necessary to restart
> Django whenever a new GDAPS app is installed via pip.
> 
> But: I want to resurrect an old theme again which would, in a way,
> improve some of the loading problems I encountered. Django's settings
> are code. Which is, in fact, a very good thing, as it makes it extremely
> flexible and adaptable to different setups. But, as discussed with the
> SECRET_KEY here, some of the settings _have_ to be coded very
> complicated, and it makes some things like per-app-settings extremely
> uncomfortable.
> 
> What if - and please don't kill me instantly - yes, I am a newcomer, and
> not a good programmer maybe - but some things are viewed better from
> "outside" - what if Django settings could be "declarative"?
> 
> So instead of Python code like
> 
> INSTALLED_APPS = [
> 'django.contrib.admin',
> 'django.contrib.auth',
> 'django.contrib.contenttypes'
> ]
> 
> This would be in an e.g. JSON file
> 
> {
> 
> "INSTALLED_APPS": [
> "django.contrib.admin",
> "django.contrib.auth",
> "django.contrib.contenttypes"
> ] ,
> ROOT_URLCONF: "fooproject.urls"
> }
> 
> Django's settings.py would look different: It would load that
> settings.json file and set the appropriate values into local code - so
> this wouldn't make much difference.
> 
> Except 2 things:
> 
> 1. Apps could have (default) settings, and they could be merged MUCH
> easier. Things like namespaced classes that are overwriting values like
> DRF/graphene does, would be completely unnecessary. The main
> settings.json file could be the "last word" in the process of settings,
> so anything an app would suggest could be overrided in the main file.
> 
> 2. Installed apps could be managed much more comfortable. Adding an app
> could be done by a script (JSON editing is easy. Editing code
> (=settings.py) is error prone and uncomfortable). I have a Django
> command script ATM for that, but just because I add a line into
> settings.py to add some additional apps to the list.
> 
> This even could be done with backwards compatibility, because Django
> would keep it's settings.py file optionally:
> 
> * read json settings (if they exist), use them
> * load settings.py which allows to override them again (using some
> special code tricks like dynamic loading, environments etc.)
> 
> Please tell me what you think about that.
> 
> Christian
> 
> 
> --
> Dr. Christian González
> https://nerdocs.at
> 
> --
> You received this message because you are subscribed to the Google Groups 
> "Django developers  (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to django-developers+unsubscr...@googlegroups.com.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/django-developers/3047a7b6-3fa0-d574-4bb6-7842b7aed44a%40nerdocs.at.
> 

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/D7000C61-69C0-4E59-9DB8-4BB1413B0C98%40tomforb.es.


signature.asc
Description: Message signed with OpenPGP

Re: declarative settings

[Jacob Rief]

You have hit a salient point in Django. It indeed is a mess how every third 
party
application must add its own configuration classes in order to make their 
own
default settings configurable through a settings.py. This results in 
settings directives
which can have any attribute name and do not follow any namespacing.

Some external apps, such as easy-thumbnails 
 add their own configuration 
framework 

with many different setting attributes, while others such as DRF use one 
Python dictionary
to keep all configurations in one closed namespace. The latter in my 
opinion is the
better approach but has its own issues. Having a consistent naming 
convention of setting
attributes, and a reconfiguration framework would certainly be beneficial 
in Django.

In my opinion, just switching to JSON does not resolve these naming 
convention issues,
but adds a bunch of other problems:

*Some configuration settings must be lazy.*
For instance, ugettext_lazy is used very often in the setting.py. This 
can't be handled by JSON.

*Writing JSON by hand is harder than writing Python.*
An alternative would be the usage of yaml or toml.

*Some settings must be taken from the environment.*
Especially database passwords and the SECRET_KEY shall be injected through 
the environment.
Typically here one would use os.getenv('DJANGO_SECRET_KEY') or similar. 
This can't be
handled by JSON either.

On Monday, December 30, 2019 at 11:46:03 PM UTC+1, Christian González wrote:
>
>
> * read json settings (if they exist), use them 
> * load settings.py which allows to override them again (using some 
> special code tricks like dynamic loading, environments etc.) 
>
>
If implemented, wouldn't it make more sense to use a JSON file to override 
a settings.py
rather than doing it vice versa? 

– just my two cents,
Jacob

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/f5d9a23a-2927-454e-a3d3-6d2361c3aa06%40googlegroups.com.

RE: declarative settings

[Matthew Pava]

It's an interesting idea, and I'm not opposed to it myself; however, keeping 
the settings as Python code is not an abnormal practice compared to other 
software.

I've been working with some Drupal stuff lately, and it is written in PHP. 
Drupal is a content management system that can be extended with various modules 
written in PHP. It's settings file is also just a code file of PHP. Drupal does 
take advantage of a package manger called composer, which would be similar to 
our pipenv. Both of those managers use JSON files for their appropriate 
settings. But package managers are not frameworks.

Perhaps we could set up a hybrid in which we have a declarative settings file 
that can be utilized by a coded settings file, but I feel that might make the 
whole system a bit too complex to maintain.

Just my thoughts.

-Original Message-
From: django-developers@googlegroups.com 
[mailto:django-developers@googlegroups.com] On Behalf Of Christian González
Sent: Monday, December 30, 2019 4:46 PM
To: django-developers
Subject: declarative settings

Hello,

I recently "finished" my first really working version of GDAPS, my Generic 
Django Application Plugin System. It's noway perfect, but does what it should: 
providing pluggable apps for an app framework, including a more or less 
flexible frontend with each django app.

I had much struggle with it, and one of the lessons I learned was Django's 
setup system, and how it deals with loading apps. Unfortunately Django can't 
load/unload apps on the fly, so it is necessary to restart Django whenever a 
new GDAPS app is installed via pip.

But: I want to resurrect an old theme again which would, in a way, improve some 
of the loading problems I encountered. Django's settings are code. Which is, in 
fact, a very good thing, as it makes it extremely flexible and adaptable to 
different setups. But, as discussed with the SECRET_KEY here, some of the 
settings _have_ to be coded very complicated, and it makes some things like 
per-app-settings extremely uncomfortable.

What if - and please don't kill me instantly - yes, I am a newcomer, and not a 
good programmer maybe - but some things are viewed better from "outside" - what 
if Django settings could be "declarative"?

So instead of Python code like

INSTALLED_APPS = [
    'django.contrib.admin',
    'django.contrib.auth',
    'django.contrib.contenttypes'
]

This would be in an e.g. JSON file

{

    "INSTALLED_APPS": [
        "django.contrib.admin",
        "django.contrib.auth",
        "django.contrib.contenttypes"
    ] ,
    ROOT_URLCONF: "fooproject.urls"
}

Django's settings.py would look different: It would load that settings.json 
file and set the appropriate values into local code - so this wouldn't make 
much difference.

Except 2 things:

1. Apps could have (default) settings, and they could be merged MUCH easier. 
Things like namespaced classes that are overwriting values like DRF/graphene 
does, would be completely unnecessary. The main settings.json file could be the 
"last word" in the process of settings, so anything an app would suggest could 
be overrided in the main file.

2. Installed apps could be managed much more comfortable. Adding an app could 
be done by a script (JSON editing is easy. Editing code
(=settings.py) is error prone and uncomfortable). I have a Django command 
script ATM for that, but just because I add a line into settings.py to add some 
additional apps to the list.

This even could be done with backwards compatibility, because Django would keep 
it's settings.py file optionally:

* read json settings (if they exist), use them
* load settings.py which allows to override them again (using some special code 
tricks like dynamic loading, environments etc.)

Please tell me what you think about that.

Christian


--
Dr. Christian González
https://nerdocs.at

--
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/3047a7b6-3fa0-d574-4bb6-7842b7aed44a%40nerdocs.at.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/8ea6a5cda5f64ff6babaaf3e42fa9629%40iss2.ISS.LOCAL.

declarative settings

[Christian González]

Hello,

I recently "finished" my first really working version of GDAPS, my
Generic Django Application Plugin System. It's noway perfect, but does
what it should: providing pluggable apps for an app framework, including
a more or less flexible frontend with each django app.

I had much struggle with it, and one of the lessons I learned was
Django's setup system, and how it deals with loading apps. Unfortunately
Django can't load/unload apps on the fly, so it is necessary to restart
Django whenever a new GDAPS app is installed via pip.

But: I want to resurrect an old theme again which would, in a way,
improve some of the loading problems I encountered. Django's settings
are code. Which is, in fact, a very good thing, as it makes it extremely
flexible and adaptable to different setups. But, as discussed with the
SECRET_KEY here, some of the settings _have_ to be coded very
complicated, and it makes some things like per-app-settings extremely
uncomfortable.

What if - and please don't kill me instantly - yes, I am a newcomer, and
not a good programmer maybe - but some things are viewed better from
"outside" - what if Django settings could be "declarative"?

So instead of Python code like

INSTALLED_APPS = [
    'django.contrib.admin',
    'django.contrib.auth',
    'django.contrib.contenttypes'
]

This would be in an e.g. JSON file

{

    "INSTALLED_APPS": [
        "django.contrib.admin",
        "django.contrib.auth",
        "django.contrib.contenttypes"
    ] ,
    ROOT_URLCONF: "fooproject.urls"
}

Django's settings.py would look different: It would load that
settings.json file and set the appropriate values into local code - so
this wouldn't make much difference.

Except 2 things:

1. Apps could have (default) settings, and they could be merged MUCH
easier. Things like namespaced classes that are overwriting values like
DRF/graphene does, would be completely unnecessary. The main
settings.json file could be the "last word" in the process of settings,
so anything an app would suggest could be overrided in the main file.

2. Installed apps could be managed much more comfortable. Adding an app
could be done by a script (JSON editing is easy. Editing code
(=settings.py) is error prone and uncomfortable). I have a Django
command script ATM for that, but just because I add a line into
settings.py to add some additional apps to the list.

This even could be done with backwards compatibility, because Django
would keep it's settings.py file optionally:

* read json settings (if they exist), use them
* load settings.py which allows to override them again (using some
special code tricks like dynamic loading, environments etc.)

Please tell me what you think about that.

Christian


-- 
Dr. Christian González
https://nerdocs.at

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/3047a7b6-3fa0-d574-4bb6-7842b7aed44a%40nerdocs.at.


pEpkey.asc
Description: application/pgp-keys