Re: AWS RDS Proxy and session pinning

2022-02-02 Thread dans...@gmail.com 
We already do that - our backend has some other work to do since we use AWS 
Secrets Manager to store database passwords, and also the hostname, so that 
our settings.DATABASES does not contain the PASSWORD but must contain a 
SECRET_ID.  The question is whether this is worth promoting since AWS RDS 
Proxy (and other similar proxy managers) are quite common.

On Monday, January 31, 2022 at 6:35:00 PM UTC-5 Adam Johnson wrote:

> Hi!
>
> Good to hear about the use of RDS Proxy. I have considered looking at it 
> to help scaling.
>
> You should be able to bypass the timezone check yourself with a little 
> subclassing. You can implement this yourself with a subclassed database 
> backend like so: 
> https://docs.djangoproject.com/en/4.0/ref/databases/#subclassing-the-built-in-database-backends
>  
> . Use it to replace the ensure_timezone() method with appropriate logic.
>
> That may be all that's required. It would be good to know if that works 
> before considering changing Django.
>
> Thanks,
>
> Adam
>
> On Mon, Jan 31, 2022 at 10:30 PM dans...@gmail.com  
> wrote:
>
>> Sorry - lest I miscommunicate - the DBAs make the default equivalent to 
>> EST5EDT, rather than UTC.   Django team leads (me and David), want USE_TZ = 
>> True to be on for all the applications, and because of this the postgresql 
>> backend will issue SET TIMEZONE UTC in 
>> django/db/backends/postgreql/base.py:209 (ensure_timezone).  At the same 
>> time, different applications and frameworks have different opinions and the 
>> DBAs try to satisfy us all, but can forget.  This issue of session pinning 
>> with connection pooling servers (such as RDS Proxy)  may be more general, 
>> and it may be good to not always set the timezone, but I want to discuss 
>> before filing an issue.
>>
>> On Monday, January 31, 2022 at 5:24:46 PM UTC-5 dans...@gmail.com wrote:
>>
>>> Hi Django developers,
>>>
>>> At the National Library of Medicine we are doing a lot of Django and 
>>> AWS.  For some of the applications with heavier traffic, we are using RDS 
>>> Proxy within AWS.  RDS Proxy is sort of like a managed version of pgbouncer 
>>> or pg-pool-II - it must be rather AWS customized because of the way it 
>>> manages authentication.
>>>
>>> Anyway, we found some problems for two reasons:
>>>
>>>- Our DBAs make the default timezone of the database UTF-8 is AWS 
>>>CloudFormation to create the databases.
>>>- When "ensure_timezone" runs and sets the timezone, that database 
>>>session is "pinned", so that it will not be shared.
>>>
>>> Since we already have our own PostgreSQL backend for a couple of other 
>>> reasons, I just wrote a version of ensure_timezone which fails loudly if 
>>> the timezone is not UTF-8 rather than set the timezone.  I probably should 
>>> have discussed the issue on this list, and I am remedying this.
>>>
>>> The failure to manage database connections with RDS Proxy is a pretty 
>>> severe error, and I am wondering what the community thinks about a 
>>> connection specific setting about how the timezone should be handled?   For 
>>> lower traffic applications, SET TIMEZONE is fine.  For higher traffic 
>>> applications, raising ImproperlyConfigured is better.   
>>>
>>> Does the group think that a Postgreql specific setting/option should be 
>>> implemented to prevent session pinning?
>>>
>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "Django developers (Contributions to Django itself)" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to django-develop...@googlegroups.com.
>> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/django-developers/ecc0c527-465a-45c7-b6ff-3c35c56824bfn%40googlegroups.com
>>  
>> 
>> .
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/654dc44a-98d3-422a-aa45-fc719eae0da6n%40googlegroups.com.

Re: AWS RDS Proxy and session pinning

2022-01-31 Thread 'Adam Johnson' via Django developers (Contributions to Django itself) 
Hi!

Good to hear about the use of RDS Proxy. I have considered looking at it to
help scaling.

You should be able to bypass the timezone check yourself with a little
subclassing. You can implement this yourself with a subclassed database
backend like so:
https://docs.djangoproject.com/en/4.0/ref/databases/#subclassing-the-built-in-database-backends
. Use it to replace the ensure_timezone() method with appropriate logic.

That may be all that's required. It would be good to know if that works
before considering changing Django.

Thanks,

Adam

On Mon, Jan 31, 2022 at 10:30 PM dans...@gmail.com 
wrote:

> Sorry - lest I miscommunicate - the DBAs make the default equivalent to
> EST5EDT, rather than UTC.   Django team leads (me and David), want USE_TZ =
> True to be on for all the applications, and because of this the postgresql
> backend will issue SET TIMEZONE UTC in
> django/db/backends/postgreql/base.py:209 (ensure_timezone).  At the same
> time, different applications and frameworks have different opinions and the
> DBAs try to satisfy us all, but can forget.  This issue of session pinning
> with connection pooling servers (such as RDS Proxy)  may be more general,
> and it may be good to not always set the timezone, but I want to discuss
> before filing an issue.
>
> On Monday, January 31, 2022 at 5:24:46 PM UTC-5 dans...@gmail.com wrote:
>
>> Hi Django developers,
>>
>> At the National Library of Medicine we are doing a lot of Django and
>> AWS.  For some of the applications with heavier traffic, we are using RDS
>> Proxy within AWS.  RDS Proxy is sort of like a managed version of pgbouncer
>> or pg-pool-II - it must be rather AWS customized because of the way it
>> manages authentication.
>>
>> Anyway, we found some problems for two reasons:
>>
>>- Our DBAs make the default timezone of the database UTF-8 is AWS
>>CloudFormation to create the databases.
>>- When "ensure_timezone" runs and sets the timezone, that database
>>session is "pinned", so that it will not be shared.
>>
>> Since we already have our own PostgreSQL backend for a couple of other
>> reasons, I just wrote a version of ensure_timezone which fails loudly if
>> the timezone is not UTF-8 rather than set the timezone.  I probably should
>> have discussed the issue on this list, and I am remedying this.
>>
>> The failure to manage database connections with RDS Proxy is a pretty
>> severe error, and I am wondering what the community thinks about a
>> connection specific setting about how the timezone should be handled?   For
>> lower traffic applications, SET TIMEZONE is fine.  For higher traffic
>> applications, raising ImproperlyConfigured is better.
>>
>> Does the group think that a Postgreql specific setting/option should be
>> implemented to prevent session pinning?
>>
> --
> You received this message because you are subscribed to the Google Groups
> "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-developers+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/django-developers/ecc0c527-465a-45c7-b6ff-3c35c56824bfn%40googlegroups.com
> 
> .
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CAMyDDM1Xj_JYKDLBHwPc%3D7Lz8NMqNVQz2vOLQ-cz9cjEuvDd7Q%40mail.gmail.com.

Re: AWS RDS Proxy and session pinning

2022-01-31 Thread dans...@gmail.com 
Sorry - lest I miscommunicate - the DBAs make the default equivalent to 
EST5EDT, rather than UTC.   Django team leads (me and David), want USE_TZ = 
True to be on for all the applications, and because of this the postgresql 
backend will issue SET TIMEZONE UTC in 
django/db/backends/postgreql/base.py:209 (ensure_timezone).  At the same 
time, different applications and frameworks have different opinions and the 
DBAs try to satisfy us all, but can forget.  This issue of session pinning 
with connection pooling servers (such as RDS Proxy)  may be more general, 
and it may be good to not always set the timezone, but I want to discuss 
before filing an issue.

On Monday, January 31, 2022 at 5:24:46 PM UTC-5 dans...@gmail.com wrote:

> Hi Django developers,
>
> At the National Library of Medicine we are doing a lot of Django and AWS.  
> For some of the applications with heavier traffic, we are using RDS Proxy 
> within AWS.  RDS Proxy is sort of like a managed version of pgbouncer or 
> pg-pool-II - it must be rather AWS customized because of the way it manages 
> authentication.
>
> Anyway, we found some problems for two reasons:
>
>- Our DBAs make the default timezone of the database UTF-8 is AWS 
>CloudFormation to create the databases.
>- When "ensure_timezone" runs and sets the timezone, that database 
>session is "pinned", so that it will not be shared.
>
> Since we already have our own PostgreSQL backend for a couple of other 
> reasons, I just wrote a version of ensure_timezone which fails loudly if 
> the timezone is not UTF-8 rather than set the timezone.  I probably should 
> have discussed the issue on this list, and I am remedying this.
>
> The failure to manage database connections with RDS Proxy is a pretty 
> severe error, and I am wondering what the community thinks about a 
> connection specific setting about how the timezone should be handled?   For 
> lower traffic applications, SET TIMEZONE is fine.  For higher traffic 
> applications, raising ImproperlyConfigured is better.   
>
> Does the group think that a Postgreql specific setting/option should be 
> implemented to prevent session pinning?
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/ecc0c527-465a-45c7-b6ff-3c35c56824bfn%40googlegroups.com.

AWS RDS Proxy and session pinning

2022-01-31 Thread dans...@gmail.com 
Hi Django developers,

At the National Library of Medicine we are doing a lot of Django and AWS.  
For some of the applications with heavier traffic, we are using RDS Proxy 
within AWS.  RDS Proxy is sort of like a managed version of pgbouncer or 
pg-pool-II - it must be rather AWS customized because of the way it manages 
authentication.

Anyway, we found some problems for two reasons:

   - Our DBAs make the default timezone of the database UTF-8 is AWS 
   CloudFormation to create the databases.
   - When "ensure_timezone" runs and sets the timezone, that database 
   session is "pinned", so that it will not be shared.

Since we already have our own PostgreSQL backend for a couple of other 
reasons, I just wrote a version of ensure_timezone which fails loudly if 
the timezone is not UTF-8 rather than set the timezone.  I probably should 
have discussed the issue on this list, and I am remedying this.

The failure to manage database connections with RDS Proxy is a pretty 
severe error, and I am wondering what the community thinks about a 
connection specific setting about how the timezone should be handled?   For 
lower traffic applications, SET TIMEZONE is fine.  For higher traffic 
applications, raising ImproperlyConfigured is better.   

Does the group think that a Postgreql specific setting/option should be 
implemented to prevent session pinning?

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/bf7ac5e1-05e1-4d5b-a2a5-8d1dda04b66bn%40googlegroups.com.