This is an automated email from the ASF dual-hosted git repository.
shuber pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/unomi.git
The following commit(s) were added to refs/heads/master by this push:
new f6f4711 UNOMI-225 ElasticSearch 7 Migration documentation and
clarification of SOURCE and TARGET settings
f6f4711 is described below
commit f6f4711270a45a1145f9ead44644fb4faf1a550e
Author: sergehuber <[email protected]>
AuthorDate: Tue Dec 10 07:22:49 2019 +0100
UNOMI-225 ElasticSearch 7 Migration documentation and clarification of
SOURCE and TARGET settings
---
manual/src/main/asciidoc/new-data-model.adoc | 27 ++++++++++++++++------
.../unomi/shell/migration/actions/Migrate.java | 2 +-
.../unomi/shell/migration/impl/MigrationTo150.java | 10 ++++----
3 files changed, 26 insertions(+), 13 deletions(-)
diff --git a/manual/src/main/asciidoc/new-data-model.adoc
b/manual/src/main/asciidoc/new-data-model.adoc
index 31707f3..8d39086 100644
--- a/manual/src/main/asciidoc/new-data-model.adoc
+++ b/manual/src/main/asciidoc/new-data-model.adoc
@@ -55,10 +55,16 @@ Previously named: `geonames` is now using the index name
`context-geonameentry`
In order to migrate the data from ElasticSearch 5 to 7, Unomi provides a
migration tool that is directly integrated.
+It is HIGHLY RECOMMENDED to perform a full cluster backup/snapshot of the
`source` cluster, and ideally to perform the
+migration on a restore of the `source` cluster. For more information on
ElasticSearch 5 snapshots and restore you can
+find it here:
https://www.elastic.co/guide/en/elasticsearch/reference/5.6/modules-snapshots.html
+
Note that it is possible to do the migration procedure on a single machine,
but you will need to change the ports on one
-of the ElasticSearch cluster. In the following example we have changed the
ports on the `source` cluster.
+of the ElasticSearch cluster. In the following example we have changed the
ports on the `source` cluster, which requires
+a cluster restart to take effect. It is also possible to change the ports on
the `target` cluster but those would have
+to be then changed again to your final setting.
-So in the `config/elasticsearch.yml` file we have modified the default ports
to:
+So in the `source` 's ElasticSearch 5 `config/elasticsearch.yml` file we have
modified the default ports to:
transport.tcp.port: 9310
http.port: 9210
@@ -66,14 +72,16 @@ So in the `config/elasticsearch.yml` file we have modified
the default ports to:
Make SURE you change the ports out of the default 9200-9205 and 9300-9305
range (or whatever your cluster uses) otherwise
both clusters will attempt to merge!
-On the destination ElasticSearch cluster configuration you will need to add
the following setting in the `config/elasticsearch.yml`:
+On the `target` ElasticSearch 7 cluster configuration you will need to add the
following setting in the `config/elasticsearch.yml`:
reindex.remote.whitelist: "localhost:9210"
-Replace "localhost:9210" which whatever location your `source` cluster is
available at.
+Replace "localhost:9210" which whatever location your `source` cluster is
available at. Restart or start your
+`target` ElasticSearch 7 cluster.
-Important: Make sure you haven't started Apache Unomi before (using the
`unomi:start` command) otherwise you will need to
-restart your Apache Unomi installation from scratch.
+Important: Make sure you haven't started Apache Unomi before (using the
`unomi:start` command or the autostart command
+line parameter) otherwise you will need to restart your Apache Unomi
installation from scratch. The best way to be sure
+of that is to start a new Unomi install by uncompressing the archive and not
launching it.
You can then start both instances of ElasticSearch 5 and ElasticSearch 7 and
finally start Apache Unomi using:
@@ -84,5 +92,10 @@ Once in the console launch the migration using the following
command:
migrate 1.4.0
Follow the instructions and answer the prompts. If you used the above
configuration as an example you can simply use the
-default values. Note that it is also possible to change the index prefix to be
different from the default `context` value
+default values.
+
+Be careful because the first address that the tool will ask for is the
`target` (ElasticSearch 7) cluster, not the
+ES 5 one.
+
+Note that it is also possible to change the index prefix to be different from
the default `context` value
so that you could host multiple Apache Unomi instances on the same
ElasticSearch cluster.
diff --git
a/tools/shell-commands/src/main/java/org/apache/unomi/shell/migration/actions/Migrate.java
b/tools/shell-commands/src/main/java/org/apache/unomi/shell/migration/actions/Migrate.java
index 03adb99..6e73cbd 100644
---
a/tools/shell-commands/src/main/java/org/apache/unomi/shell/migration/actions/Migrate.java
+++
b/tools/shell-commands/src/main/java/org/apache/unomi/shell/migration/actions/Migrate.java
@@ -72,7 +72,7 @@ public class Migrate implements Action {
try {
httpClient = HttpUtils.initHttpClient(session);
- String esAddress = ConsoleUtils.askUserWithDefaultAnswer(session,
"Elasticsearch address (default = http://localhost:9200): ",
"http://localhost:9200";);
+ String esAddress = ConsoleUtils.askUserWithDefaultAnswer(session,
"Enter ElasticSearch 7 TARGET address (default = http://localhost:9200): ",
"http://localhost:9200";);
for (Migration migration : getMigrations()) {
if (fromVersion.compareTo(migration.getToVersion()) < 0) {
diff --git
a/tools/shell-commands/src/main/java/org/apache/unomi/shell/migration/impl/MigrationTo150.java
b/tools/shell-commands/src/main/java/org/apache/unomi/shell/migration/impl/MigrationTo150.java
index 44e144c..e9f4164 100644
---
a/tools/shell-commands/src/main/java/org/apache/unomi/shell/migration/impl/MigrationTo150.java
+++
b/tools/shell-commands/src/main/java/org/apache/unomi/shell/migration/impl/MigrationTo150.java
@@ -57,11 +57,11 @@ public class MigrationTo150 implements Migration {
@Override
public void execute(Session session, CloseableHttpClient httpClient,
String esAddress, BundleContext bundleContext) throws IOException {
- String es5Address = ConsoleUtils.askUserWithDefaultAnswer(session,
"Source Elasticsearch 5.6 cluster address (default: http://localhost:9210) : ",
"http://localhost:9210";);
- String sourceIndexPrefix =
ConsoleUtils.askUserWithDefaultAnswer(session, "Source index name (default:
context) : ", "context");
- String destIndexPrefix =
ConsoleUtils.askUserWithDefaultAnswer(session, "Destination index prefix
(default: context) : ", "context");
- int numberOfShards =
Integer.parseInt(ConsoleUtils.askUserWithDefaultAnswer(session, "Number of
shards (default: 5) : ", "5"));
- int numberOfReplicas =
Integer.parseInt(ConsoleUtils.askUserWithDefaultAnswer(session, "Number of
replicas (default: 1) : ", "1"));
+ String es5Address = ConsoleUtils.askUserWithDefaultAnswer(session,
"SOURCE Elasticsearch 5.6 cluster address (default: http://localhost:9210) : ",
"http://localhost:9210";);
+ String sourceIndexPrefix =
ConsoleUtils.askUserWithDefaultAnswer(session, "SOURCE index name (default:
context) : ", "context");
+ String destIndexPrefix =
ConsoleUtils.askUserWithDefaultAnswer(session, "TARGET index prefix (default:
context) : ", "context");
+ int numberOfShards =
Integer.parseInt(ConsoleUtils.askUserWithDefaultAnswer(session, "Number of
shards for TARGET (default: 5) : ", "5"));
+ int numberOfReplicas =
Integer.parseInt(ConsoleUtils.askUserWithDefaultAnswer(session, "Number of
replicas for TARGET (default: 1) : ", "1"));
Set<String> monthlyIndexTypes = new HashSet<>();
monthlyIndexTypes.add("event");
monthlyIndexTypes.add("session");
