Hello James community!
I would like to start a discussion regarding the upgrade of
ElasticSearch, hoping we can reach a consensus, as I spent quite a lot
of time on this already.
As you know, the version 7.10 has reached EOL already, so we need to
migrate from it.
Thus a while ago I started a very painful migration to ES 8.2 here:
https://github.com/apache/james-project/pull/1018
Before being rightfully reminded by Matthieu Baechler that ES 7.10 is
the last OSI-compliant version of ElasticSearch, before you know they
switched to a new license that's not really open source anymore...
OpenSearch is indeed a fork of ES 7.10 using the Apache License, which
is definitely more in favor for adoption for the migration than ES 8. On
that, I totally agree.
From then, with Benoit Tellier, we did our little extra research then.
If we want to migrate from ES to OpenSearch, there is a few options on
the table actually:
- solution 1: not modifying the ES7 code. Well it's possible, but you
can only use versions 1.x of OpenSearch (1.3.3 atm). However, from 2.x
version of OpenSearch, the support for es7 client has been dropped in
favor of their own clients.
- solution 2: using the java high level rest client from OpenSearch
(https://opensearch.org/docs/latest/clients/java-rest-high-level/): That
client is a basic fork of the java high level rest client from ES. As
this client has been dropped in upper version of ES for a new client
(that you can see in the PR I did before:
https://github.com/apache/james-project/pull/1018), the fork is thus
identical.
Benoit did a little POC on it and it seems you only need to change the
imports and it works with OpenSearch 2.0 without issues (also said here
in their doc:
https://opensearch.org/docs/latest/clients/java-rest-high-level/#migrating-to-the-opensearch-java-high-level-rest-client)
- solution 3: using the new java client
(https://opensearch.org/docs/latest/clients/java/). That client has been
forked from the new java client from ES probably at its beginnings,
before the change of license. In the POC I did here:
https://github.com/apache/james-project/pull/1051, you can see the
structure is very similar to the java client from ES, but with obviously
some changes or bugs as the fork went its way since when from the
original one. That migration is complicated honestly, but because I did
the one to ES then the remaining work is minimal as proven in the POC.
Just a few issues though encountered... (in the POC you can see them)
I think solution 1 is IMO, not an option, as we probably want to migrate
to the latest version of OpenSearch as we are at it now.
Solution 2 is very easy (replace dependencies and imports... nothing
more) and allows to use OpenSearch 2.0.
I would say let's go with it if I didn't invest so much time migrating
to the new java client. Because this is the issue actually. Amazon
states that the java client is supposed to replace the high level one at
some point (like on forums, or the page of the github project
(https://github.com/opensearch-project/opensearch-java). It's a bit
blurry on really when the high level client would be dropped but I
wouldn't be surprised to see it on next major upgrade for example.
So at some point we will have to migrate the client eventually... do we
try to do it now (solution 3) or do we do things simple for now
(solution 2) and keep the work done under the hood for the day the
migration is necessary? (cause a big chunk of it has been done I think).
I'm honestly fine either way, but would love to hear what the community
has to say on the topic.
Sorry it was long! But I hope I gave all the keys necessary to
understand our options here regarding this migration.
Cheers,
Rene.
---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org
For additional commands, e-mail: server-dev-h...@james.apache.org
