On Mon, 10 Aug 2020 at 21:49, Daniel Gruno <[email protected]> wrote: > > On 10/08/2020 22.46, sebb wrote: > > On Mon, 10 Aug 2020 at 18:51, Daniel Gruno <[email protected]> wrote: > >> > >> Hi folks, > >> with the incoming new UI and with ElasticSearch moving far away from > >> what Pony Mail supports currently, I think it's also time to get started > >> on a "next generation" of Pony Mail, that supports the new structures > >> laid out in ES 7.x and above, and while we're at it....I think we should > >> ditch Lua and use a pure python implementation instead, to increase the > >> number of potential contributors (and make use of Python's excellent > >> libraries). > >> > >> SO, with all that said, I'll be setting up a new repository for this > >> 'next generation' of Pony Mail, so as to not start mixing the old and > >> the new too much. I think this is justified, as it's a very major shift > >> from the old code-base, and the two would be internally incompatible > >> (except for the JSON API, I believe that should remain as is.) > >> > >> My plan is to: > >> > >> - create a new repository > >> - import the UIX code-base into this > >> - import the tools we have from pony v1, tweak those later on > >> - get started on a pure python back-end for the UI (I have some > >> semblance of a prototype that I'm hacking on currently) > >> - finally, write a migration script for migrating from the old DBs to > >> the new format. > > > > Before one can even think about migrating a database, there needs to > > be a full test suite. > > In particular, there need to be exhaustive tests to show that the same > > Permalinks will be generated. > > I think perhaps you misunderstand me here :) > The database *contents* would be migrated verbatim, just with the new ES > structure, where each doctype in the old setup is its own index with > doctype _doc. The IDs stay the same as they were before.
I realise that. However, I don't think this can be done without changing the backend code which is responsible for generating the IDs going forward. There have already been unplanned changes to the generators which affect the output. Such changes need to be caught before they cause compatibility issues. > thus, the ponymail database (or whatever you have called it) would get > split into several databases instead: > - ponymail-mbox > - ponymail-attachments > - ponymail-sources > - ponymail-sessions > etc etc > > With regards, > Daniel. > > > > >> Comments/feedback is always welcome as usual :) > >> > >> With regards, > >> Daniel. >
