Re: Google Summer of Code 2019

2019-02-16 Thread Aleksi Häkli
Just my 2 cents or ramblings on the topic of GSoC and deployment:

Most Django deployments we do are executed by running containers from 
custom built Docker images from private repositories.

They are mostly run with either

- Docker Compose on small scale, or
- Heroku, Kubernetes or other container orchestrator on larger scale.

The Docker images and their packaging are starting to look quite similar 
these days with their 12-factor setups and service oriented approaches.

The baseline package is usually just a Python template that runs Django 
installation with all the checks, static file compilation, and whatnot at 
the end of the image build. 

Then there is, usually, a shell startup script, that is run as an 
entrypoint, which runs migrate and other state related commands on 
container startup and execs the Django application as configured.

The Docker images we build ideally support running multimode with the 

- web server container running Gunicorn, uwsgi, or a similar HTTP 
application server, and 
- the background worker container running a task executor like Celery or 
Django Q

and the container orchestrator just supplying the command (one can think of 
this as the "mode") to run in.

This is all and good, and Django supports a runner setup like this 
beautifully without any gimmicks with Docker images and containers. 

Well, you have to figure out the necessary build and startup commands and 
so forth, but there are good online resources for figuring out those things 
nowadays.

Progressing to the point I am trying to bring up, I think that 
containerization instructions with some good examples including stuff like 

- Docker image packaging for a basic Django web server using FOSS 
components,
- configuration and settings examples in a Docker build and deployment 
setup for Django,
- multimode container examples for running in web server or web worker role,
- sane defaults for security hardening and checkups, and
- defining startup scripts for running tasks like migrations before 
application execution

could be quite invaluable in the Django deployment documentation at e.g. 
https://docs.djangoproject.com/en/dev/howto/deployment/

The deployments are starting to look so similar these days that supplying 
good stock examples for packaging a basic web server and web worker image 
and running it as a container could be a good idea that could save 
tremendous amounts of time for people building their first or second 
deployments. Best practices never hurt anybody either.

Of course e.g. many of the Cookiecutter templates available have examples 
for Docker packaging, but none of the ones I have seen discuss the process 
in an easily accessible format that ties in well with Django development 
and documentation. They are usually more expert oriented and just supply 
some opinionated configuration examples that might or might not work or be 
secure or up-to-date. 

I imagine at least including some Docker packaging and workload 
containerization components in the GSoC proposition could be helpful. 
Packaging could align with the GSoC motives as e.g. Kubernetes enablement 
is well in line with modern IaaS hosting interests.

Regards,
A DevOps guy

On Friday, 15 February 2019 13:22:22 UTC+2, Josh Smeaton wrote:
>
> If you really think you want to work on a deployment project, you should 
> get the requirements together very quickly, so someone on this list can 
> sanity check that it's something both feasible and useful.
>
> I have done a **lot** of different deployments, and other than deploying 
> to heroku, they have never been the same. I'm rather skeptical that some 
> tool can be built for deploying a Django application that didn't already 
> mimic an existing tool like Ansible or Salt (or Heroku CLI).
>
> Examples: what application server (gunicorn, uwsgi, mod_wsgi)? What web 
> server (nginx, apache, caddy)? What database server, and is it remote or 
> local? How are environment variables securely provisioned and deployed? 
> Where is static content served from, and do you need S3 keys? How are you 
> managing TLS certificates?
>
> Start a new thread on this list when you have an outline for what you want 
> to do. At that stage we should be able to tell you if you should proceed 
> with a detailed project scope or not. I don't want you to work on a 
> detailed project scope and then get knocked back for the idea being 
> infeasible. Remember, the project would need to be very useful for Django 
> users.
>
> I'd also just like to call out that projects like Django Channels would 
> likely be in scope too, provided the current maintainers felt comfortable 
> mentoring a student. A quick note about Channels specifically - the current 
> maintainers have **just** taken up that mantle, so you should have a 
> specific idea in mind if that's what you're interested in, not ask the team 
> for ideas.
>
>
>
>
> On Friday, 15 February 2019 20:53:34 UTC+11, Shashank shet wrote:
>>
>> That's a good 

Re: Post mortem on today's packaging error.

2019-02-12 Thread Aleksi Häkli
Jazzband (https://jazzband.co/about/releases) uses an approach that builds 
and pushes the PyPI packages to an intermediate repository that is owned by 
the Jazzband organization.

The Jazzband intermediate repository then allows publishing them from the 
Jazzband organization to PyPI via a push-button deployment from the Travis 
build. 

This eliminates the need for having the public PyPI warehouse credentials 
published to Travis or other target, but requires setting up a private 
repository.

An approach like this (2) of course introduces two additional components to 
the trust chain compared to the current model (1):

1. With a simple PyPI upload, only the PyPI warehouse and the uploader has 
to be ultimately trusted, and package signatures are easy to check against 
known public PGP keys with 2 parties of trust, but
2. with an intermediate private PyPI upload from e.g. Travis. both Travis 
and the private intermediate server have to be trusted in addition to PyPI 
warehouse and the original author with 4 parties of trust.

On Tuesday, 12 February 2019 09:36:09 UTC+2, Florian Apolloner wrote:
>
>
>
> On Monday, February 11, 2019 at 11:01:55 PM UTC+1, Adam Johnson wrote:
>>
>> Jamesie’s suggestion to use CI is also valid but a bunch more work. I 
>> guess the main advantage is you get a blank slate container to work in, 
>> which a fresh checkout to a temp dir provides most of the gain for less 
>> work.
>>
>
> If I remember we have been hesitant in the past because that would require 
> us to give credentials to PyPi etc to the CI service. That said I think 
> that is a risk we could take.
>
> Cheers,
> Florian
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/a1f957d4-87c4-4f09-87cc-d8fefa6c5c98%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


Help needed on Django-Guardian.

2019-02-04 Thread Aleksi Häkli
We have had good success in project maintenance with the Jazzband organization 
which collaboratively tends to a number of Django projects. I think that the 
crowd of maintainers could help in the Guardian project infrastructure and take 
care of things like versioning, allowing better focus for issues related to 
functionality.

https://github.com/jazzband

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/1f716662-96d8-479a-99b9-17b35bb21591%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.