Re: [Mailman-Developers] Rolling releases for Container Images and Funding Campaign for Mailman

[Barry Warsaw]

On Nov 12, 2017, at 11:06, Simon Hanna  wrote:

> Scenario: Core changes something about rest that is backwards incompatible. 
> The change is commited to master.

The REST API is versioned, so it would be a bug to introduce a backward 
incompatibility in the same API version.  That’s why for example a new API 
version was introduced to handle the UUID data type change.  There was no way 
to make that backward compatible, so we had to bump the API version, but we 
didn’t remove the old API version so clients written against that still work.  
Adding a new API generally doesn’t require bumping the API version.

Cheers,
-Barry



signature.asc
Description: Message signed with OpenPGP
___
Mailman-Developers mailing list
Mailman-Developers@python.org
https://mail.python.org/mailman/listinfo/mailman-developers
Mailman FAQ: http://wiki.list.org/x/AgA3
Searchable Archives: 
http://www.mail-archive.com/mailman-developers%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-developers/archive%40jab.org

Security Policy: http://wiki.list.org/x/QIA9

[Mailman-Developers] Rolling releases for Container Images and Funding Campaign for Mailman

[Simon Hanna]

These images are built using git-heads *only* if they are passing our
test suite and are re-generated weekly. You should be aware that while
all these components are tested with their individual test suites, their
combination might sometimes not be stable. This will get you
updates/bug-fixes much faster :)

This contradicts what you said in my merge request for Postorius
https://gitlab.com/mailman/postorius/merge_requests/232#note_43388756

You either use released versions, or make sure that the master branches
are always compatible. You can't have both with the current development
model.

Not exactly, I still believe all our git-heads should be compatible with
released (i.e. stable) versions of dependencies, regardless of the fact
that we maintain those dependencies.

Also, there is at least one test per-project which tests with git-heads
of dependencies we maintain, so that we know if we make any incompatible
changes.

I am not sure what in the current development model prevents us
to do both?

The story for container images is different than testing with git-heads.
We need better integration testing for the entire suite to be stable in
conjunction, rather than independently. In fact, containers might be the
perfect way to actually do the integration testing.
Scenario: Core changes something about rest that is backwards 
incompatible. The change is commited to master. Since your containers 
use the master branches, all future builds will fail until mailmanclient 
and postorius/hyperkitty.
If you want to always have Postorius (and others) compatible with the 
latest release of master, development for Postorius will be a Pain, 
because you can't use the master branch of mailman for testing and quick 
changes. Also your containers and all integration tests that can be done 
with them will always fail.


Until you fix Postorius and the other clients... So why not just have 
the master branches always compatible and make life easier for developers.

I don't see a good reason why you want to do it the other way round...

Any bug in core that affects Postorius will also just remain there until 
you release a new version of core.


If you really really want to have it your way, there have to be more 
frequent releases.

- Improve the container images to work with new micro-services
architecture,
to achieve scaling and redundancy in services.

The current bottleneck is the rest api from core. IMO bulk actions,
filtering and sorting should be added there before trying to have
multiple postorius/hyperkitty instances serving the same core...

I didn't say they will be served from the same core ;-)

Yes, the Core's API can be further improved to be more efficient. But
that
doesn't prevent us from scaling. I understand there are issues around
multiple
instances of containers, but I have ideas to get around most of them.
I just don't want that people that have issues with big installations 
get the answer:
We know of that issue, please deploy Mailman on X containers/machines as 
a workaround

___
Mailman-Developers mailing list
Mailman-Developers@python.org
https://mail.python.org/mailman/listinfo/mailman-developers
Mailman FAQ: http://wiki.list.org/x/AgA3
Searchable Archives: 
http://www.mail-archive.com/mailman-developers%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-developers/archive%40jab.org

Security Policy: http://wiki.list.org/x/QIA9

Re: [Mailman-Developers] Rolling releases for Container Images and Funding Campaign for Mailman

[Barry Warsaw]

On Nov 8, 2017, at 04:38, Simon Hanna  wrote:
> 
> On 11/08/2017 12:03 AM, Abhilash Raj wrote:
>> New images are available on quay.io and, moving forward, the rest of
>> the image builds will also be moved to Quay[4][5].
> Since docker deployment is currently the recommended install, with manual 
> installs using the master branches for "experts" I wonder why you are using 
> Quay.
> It looks like it's a non-free system. Why use Quay but refuse to use a 
> non-free translation system like transifex?

Is there a free (software) equivalent to Quay?  Caveat: I don’t know anything 
about Quay, so I don’t know what services and advantages it provides.

> If this campaign succeeds, here is a road map of what I intend to get
>> done:
> Sounds great, what is the limit where the campaign succeeds? What will happen 
> to the funds if it doesn’t?

The funds all go to the GNU Mailman project’s FSF donation fund.  A portion of 
every donation goes to the FSF, and the Mailman Steering Committee ultimately 
decides what to do with what’s left in our account.  In the past we’ve used it 
sponsor GSoC students coming to Pycon sprints.

> To be honest, I don't think we should all in on containers.
> Yes they are nice, but I guess most people would prefer distro packages.
> Even if people want containers, I for instance would prefer plain 
> systemd-nspawn

There are folks working on Debian packages.  I’m not aware of similar efforts 
for other distro ecosystems.  But I don’t think it’s all or nothing, and it 
does make sense for us (the GNU Mailman project) to support containers to help 
with adoption, experimentation, and deployment, while working with distro 
volunteers to package the code up for those platforms.

Cheers,
-Barry



signature.asc
Description: Message signed with OpenPGP
___
Mailman-Developers mailing list
Mailman-Developers@python.org
https://mail.python.org/mailman/listinfo/mailman-developers
Mailman FAQ: http://wiki.list.org/x/AgA3
Searchable Archives: 
http://www.mail-archive.com/mailman-developers%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-developers/archive%40jab.org

Security Policy: http://wiki.list.org/x/QIA9

Re: [Mailman-Developers] Rolling releases for Container Images and Funding Campaign for Mailman

[Simon Hanna]

On 11/08/2017 12:03 AM, Abhilash Raj wrote:

New images are available on quay.io and, moving forward, the rest of
the image builds will also be moved to Quay[4][5].
Since docker deployment is currently the recommended install, with 
manual installs using the master branches for "experts" I wonder why you 
are using Quay.
It looks like it's a non-free system. Why use Quay but refuse to use a 
non-free translation system like transifex?

These images are built using git-heads *only* if they are passing our
test suite and are re-generated weekly. You should be aware that while
all these components are tested with their individual test suites, their
combination might sometimes not be stable. This will get you
updates/bug-fixes much faster :)

This contradicts what you said in my merge request for Postorius
https://gitlab.com/mailman/postorius/merge_requests/232#note_43388756

You either use released versions, or make sure that the master branches 
are always compatible. You can't have both with the current development 
model.

If this campaign succeeds, here is a road map of what I intend to get
done:
Sounds great, what is the limit where the campaign succeeds? What will 
happen to the funds if it doesn't?

   - Add Admin Dashboard project from GSoC 2014 (maybe?)

Didn't find anything using google. What is that project about?

- Add better testing of container images and provide deployment
   instructions for Kubernetes & Docker Swarm

To be honest, I don't think we should all in on containers.
Yes they are nice, but I guess most people would prefer distro packages.
Even if people want containers, I for instance would prefer plain 
systemd-nspawn

- Improve the container images to work with new micro-services
architecture,
   to achieve scaling and redundancy in services.
The current bottleneck is the rest api from core. IMO bulk actions, 
filtering and sorting should be added there before trying to have 
multiple postorius/hyperkitty instances serving the same core...

___
Mailman-Developers mailing list
Mailman-Developers@python.org
https://mail.python.org/mailman/listinfo/mailman-developers
Mailman FAQ: http://wiki.list.org/x/AgA3
Searchable Archives: 
http://www.mail-archive.com/mailman-developers%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-developers/archive%40jab.org

Security Policy: http://wiki.list.org/x/QIA9

[Mailman-Developers] Rolling releases for Container Images and Funding Campaign for Mailman

[Abhilash Raj]

Hi Everyone,

As you all know I have been working on container images for Mailman 3.
We now have a new "rolling" tag for both mailman-core and
mailman-web images. These images have latest source installed
for every Mailman component. You can find more information about them 
on the website [3].

New images are available on quay.io and, moving forward, the rest of 
the image builds will also be moved to Quay[4][5].

These images are built using git-heads *only* if they are passing our
test suite and are re-generated weekly. You should be aware that while
all these components are tested with their individual test suites, their
combination might sometimes not be stable. This will get you
updates/bug-fixes much faster :)



As most of you already know, Mailman 3 is the new and improved version
with extra features, better security and much better architecture. We
released Mailman Suite 3.0 in April 2015 and have come a long ways since
then. Mailman Suite 3.1, release May 2016, was aimed to provide
feature-parity with Mailman 2.x series and we think we _almost_ hit that
goal.

Apart from no monthly password reminders, Mailman 3 has a much better
Administrator/User interface, REST API for scripting, a really awesome
archiver, support for multiple domains, support for external plugins,
support for SSO/social login and so much more!

I love working on Mailman and would enjoy being able to do so full time
for next 6-8 weeks. Mailman 3 is not very far from becoming the default
version everyone would use, but it still needs some work to get there. I
need help from you, the users of Mailman, to get us there. If you or
your organization would like to move to (or, already moved to) Mailman
3, I urge you to donate[1] to us.

There are options to donate using Credit Card, Paypal, Bitcoin, Wire
Transfer
(of any currency), Check and money order.

If this campaign succeeds, here is a road map of what I intend to get
done:

- Move Django apps(UI/Archiver) to Python 3 (or bilingual)
- Fork `mailman import` command to provide an upgrade path to Mailman
3.x from Mailman 2.x
- Fix MySQL compatibility in Core
- Changes in Postorius:
  - Add support for missing options that are already exposed in Core’s
API
 - e.g. Support for setting templates
  - Find the commonly used options that are not exposed in Core, add
them to Core's API and add to Postorius
  - Add Admin Dashboard project from GSoC 2014 (maybe?)
- Add better testing of container images and provide deployment
  instructions for Kubernetes & Docker Swarm
- Improve the container images to work with new micro-services
architecture,
  to achieve scaling and redundancy in services.
- Administrator/User documentation for Postorius & Mailman
- (optional) Fork [mmcli](https://github.com/rajeevs1992/mailmancli)
  project from Rajeev, fix if there is anything missing  and add it as
  an
  additional command line tool to work with Mailman Core. Maybe pull it
  under Mailman umbrella.

Except for these, if there is something more important that is
preventing the adoption of Mailman 3 from your end, we can discuss them.
I'd like to mention that I have been working on Mailman 3 for quite some
time now and I intend to implement every single item on the list. You
donations would help it get done much sooner, hopefully in time for 3.2
release schedule (at PyCon US 2018).


You can follow the progress of this campaign here[2].

[1]: https://my.fsf.org/civicrm/contribute/transact?reset=1=22
[2]: https://wiki.list.org/x/17892025
[3]: https://asynchronous.in/docker-mailman/#rolling-releases
[4]: https://quay.io/repository/maxking/mailman-web
[5]: https://quay.io/repository/maxking/mailman-core

-- 
  Abhilash Raj
  maxk...@asynchronous.in
___
Mailman-Developers mailing list
Mailman-Developers@python.org
https://mail.python.org/mailman/listinfo/mailman-developers
Mailman FAQ: http://wiki.list.org/x/AgA3
Searchable Archives: 
http://www.mail-archive.com/mailman-developers%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-developers/archive%40jab.org

Security Policy: http://wiki.list.org/x/QIA9